Legal Affairs Poll

The magazine Legal Affairs is conducting a poll on the "Top 20 Legal Thinkers in America." It's an unscientific poll. Nonetheless, how were the candidates chosen?

This page supplies the answer:

To come up with the list of top legal thinkers, we relied primarily on a few sources. The academics are drawn from rankings by Fred Shapiro, a librarian at Yale Law School, and by Brian Leiter, a professor of law and philosophy at the University of Texas. Shapiro and Leiter rank the influence of legal academics based on the number of books they've published and the number of social science and law journal articles that cite their work.

For federal judges, we considered the number of opinions they've written and the number of citations those opinions received. We also relied on the methodology of William Landes, a law professor at the University of Chicago, Lawrence Lessig, a professor at Stanford Law School, and Michael Solimine, a law professor at the University of Cincinnati. We relied on statistics calculated by Stephen J. Choi, a professor at Boalt Hall, and G. Mitu Gulati, a professor at Georgetown Law Center. They give particular consideration to the number of citations by courts outside a judge's circuit, reasoning that those rulings have had the widest influence. We included state court judges based on their appearances in casebooks and in the press.

Well, count me skeptical that the list was compiled according to these standards. The list of professors includes several young academics (such as Noah Feldman, and Nathaniel Persily, both of whom became professors in 2001). I find it impossible to believe that either of these scholars, smart as they might be, have been more cited than one of the nation's top constitutional scholars -- Michael McConnell -- who is conspicuously not on the list. Or Mary Ann Glendon, who has taught at Harvard since 1974. Or Charles Fried, who has taught at Harvard since 1961 (with occasional detours into government). Indeed, McConnell, Glendon, and Fried rank 31st, 32nd, and 35th, respectively, on Brian Leiter's own list of the most-cited law faculty! How could assistant professors who started in 2001 have already gotten enough citations to place them in the top 30 of all time?

Answer: They haven't. A LEXIS search as of today reveals that Noah Feldman's name is used 96 times in the database of all law review articles, and Nathaniel Persily appears 101 times. (Just to compare: My own name appears 36 times in the same database, and I'm not even a professor.) Michael McConnell, by contrast, appears 2,265 times, Glendon appears 2,115 times, while Charles Fried appears 1,962 times.

What's more, if the factor of "youth" was given some sort of bonus (although the stated criteria don't mention this), I can't tell why Feldman and Persily were chosen over other relatively young scholars who have far more citations. Caleb Nelson, for example, authored a paper that won a national prize for the best paper by a new law professor, and has appeared in the law review database 196 times. Saikrishna Prakash, to take another example, appears in the database 468 times. Other examples: Christine Jolls (598 mentions in the law review database), or Adrian Vermeule (416), or Ernest Young (369), or Yochai Benkler (419), or John Manning (436). (UPDATE: Todd Zywicki at the Volokh site linked to this post; which reminds me to point out that his name appears 280 times in the law review database.)

Why would scholars like these be excluded in favor of Feldman and Persily if the supposed criteria were publications and citations? It's a mystery.

The list of judges is similarly idiosyncratic: Unexpected inclusions and bizarre omissions. It includes such figures as Nancy Gertner (a district judge appointed by Clinton), Paul Cassell (a district judge appointed in 2002), and Gerard Lynch (a district judge appointed by Clinton in 2000).

Given that district court decisions aren't strictly binding on other courts, I find it enormously unlikely that any of these district judges are more cited than certain appellate judges who are conspicuously not on the list. D.C. Circuit judge Stephen Williams is widely respected as one of the nation's most brilliant judges, and he has served since 1986. Indeed, he is one of the few appellate judges (Posner is another) who is so respected by the Supreme Court that lawyers making Supreme Court arguments find it useful to highlight the fact that he (rather than another judge) was the author of an opinion. How could a list supposedly based on citations leave him off while including relatively obscure district judges appointed in the past few years? Or take 5th Circuit judge Patrick Higginbotham, also widely known as one of the nation's finest judges, and who has served since 1982. Again, how could the named district judges beat him out? There are other long-serving and highly distinguished judges I could mention as well, but you get the point.

Conclusion: The description of the list's criteria is misleading. Some people may have been chosen by publications and citation rankings, but those can't possibly have been the only criteria. Otherwise, there is no explanation for some of the inclusions or omissions.

UPDATE: If you doubt that the list was created by an inexplicable process, consider that among the writers/commentators listed as candidates for "Top 20 Legal Thinkers in America" are John Grisham and Scott Turow. Mildly entertaining novelists? Yes. Top legal thinkers? Come on. Quod erat demonstrandum.