Diane Ravitch -- Wrong on Charter Schools

Diane Ravitch has a new post full of criticisms of charter schools. Unfortunately, the arguments are unsupportable and contradictory.

1. Ravitch has gone to the NAEP website and downloaded snapshots of how students in public charter schools scored compared to students in other public schools. She then concludes, "Overall, public schools continue to outperform charter schools."

No, no, no. You cannot tell anything about how a given type of school is "performing" just by looking at a cross-section of its students' test scores -- without even attempting to take into account the students' backgrounds and previous test scores. And as I point out below, charter school students tend to arrive with somewhat lower test scores than average.

2. Immediately after having praised other public schools for supposedly outperforming charters in student test scores, Ravitch pivots 180 degrees and criticizes charter schools for drawing off the "most successful" students and "disabl[ing]" the public schools!

My beef with charter schools is that most skim the most motivated students out of the poorest communities, and many have disproportionately small numbers of children who need special education or who are English-language learners. The typical charter, operating in this way, increases the burden on the regular public schools, while privileging the lucky few. Continuing on this path will further disable public education in the cities and hand over the most successful students to private entrepreneurs.

It's just not convincing to say with one breath that public schools are "outperform[ing]" and in the next breath that they're being "disable[d]," or to criticize charter schools for serving the "most successful" students just after having claimed that charter students don't score well on national exams.

3. Besides being contradictory, Ravitch's argument is wrong. Charter school students may be "motivated" in some sense, but that certainly doesn't mean that they are all academically successful. Quite the contrary: parents whose children are doing well in the public schools often tend to stay put, while it is precisely the parents whose children are struggling who may tend to seek alternative schools (whether through vouchers or charters). Painting with a broad brush, many charter school and voucher parents have said, "Gee, little Johnny isn't doing so well, maybe I should check into a different school." Such "motivation" doesn't give rise to some sort of huge charter school advantage.

Some evidence for this point: Zimmer et al.'s October 2009 paper analyzing data from locations representing 45% of the charter schools in the nation. They find NO evidence that charter schools are cream-skimming. To the contrary, "in all but one case (Chicago reading scores, which are virtually identical to the district-wide average), students switching to charter schools have prior test scores that are BELOW district-wide or statewide averages."

For another example, take Texas, which is home to over 450 charter campuses, about 10% of all the charter campuses nationwide. In Texas, charter schools that serve predominantly students identified as “at risk” can be rated under an alternative accountability system. In 2007-08, 43.3% of charter schools in Texas qualified to be rated under that system, compared to a mere 3.3% of public school district campuses in Texas (see page 147 here). No doubt, most of the parents of these "at-risk" youth could be described as "motivated" -- motivated to find something, anything, that would help their children learn and stay in school. But this is not obviously an advantage for the charter schools' academic performance.

Incidentally, people often make the same accusation about private schools generally, i.e., that they just skim off all the best students. To the contrary, Derek Neal and Jeffrey Grogger found that “there is evidence of negative selection into Catholic schools. Relative to their public-school counterparts, urban whites who attend these schools appear to possess unmeasured traits that inhibit attainment.” They add this footnote: “Evidence of negative selection is common in this literature. Coleman and Hoffer (1987), Evans and Schwab (1995), and Neal (1997) all report evidence of negative selection into Catholic schools. A common hypothesis concerning this result is that some parents send their children to Catholic schools seeking a remedy for existing problems with discipline and motivation.”

The Possibility of Social Science Fakery

There are many scandalous cases of physical scientists or medical researchers faking their data. E.g., Schon's plastics research, Ninov's "discovery" of element 118, Scott Reuben's faked medical trials, and many more.

For anyone contemplating inventing their own data, it's a risk-reward calculation: The risk of being discovered and drummed out of a profession versus the reward of whatever fame accrues to new scientific discoveries or publications. But it's amazing that physical scientists would ever fake data, given the high risk of being discovered when other scientists are unable to replicate their findings.

But some do it anyway. Which makes me wonder about social science, where the risk would be far lower. There's no general expectation that social science "discoveries" are going to be replicable 100% of the time in any new dataset or research that comes along. For example, if one psychologist does a psychological experiment of some sort, and someone else does a similar experiment with different results, that can always be explained by the fact that the experiment happened at a different time and place involving different people as subjects, or maybe there was some seemingly minor difference in the experimental conditions, etc.

So if people are willing to fake results even when they surely know of the high risk of being caught, how much more willing would they be to fake results when the risk is far lower? How much social science fakery is there?

Peer Review

Out of all the recent emails that were apparently hacked or leaked from a prominent laboratory in England ("Climategate," some are calling it), this one bothers me. It shows how "peer review" worked from the inside:

From: Keith ***
To: Edward ***
Subject: Re: Review- confidential REALLY URGENT
Date: Wed Jun 4 16:02:09 2003

[OMITTED]

At 09:50 AM 6/4/03 -0400, you wrote:

Hi Keith,
Okay, today. Promise! Now something to ask from you. Actually somewhat important too. I got a paper to review (submitted to the Journal of Agricultural, Biological, and Environmental Sciences), written by a Korean guy and someone from Berkeley, that claims that the method of reconstruction that we use in dendroclimatology (reverse regression) is wrong, biased, lousy, horrible, etc. They use your Tornetrask recon as the main whipping boy. I have a file that you gave me in 1993 that comes from your 1992 paper. Below is part of that file. Is this the right one? Also, is it possible to resurrect the column headings? I would like to play with it in an effort to refute their claims.

If published as is, this paper could really do some damage. It is also an ugly paper to review because it is rather mathematical, with a lot of Box-Jenkins stuff in it. It won't be easy to dismiss out of hand as the math appears to be correct theoretically, but it suffers from the classic problem of pointing out theoretical deficiencies, without showing that their improved inverse regression method is actually better in a practical sense. So they do lots of monte carlo stuff that shows the superiority of their method and the deficiencies of our way of doing things, but NEVER actually show how their method would change the Tornetrask reconstruction from what you produced. Your assistance here is greatly appreciated.

Assuming that this email is valid -- and no one seems to be claiming that any emails were altered -- this guy is being asked to peer review a manuscript. Does he think the manuscript is actually wrong? No -- he admits that it's full of math that "appears to be correct" (to be sure, he implies that he finds the math hard to understand). The only thing he can find wrong with the paper is that it's too theoretical, but he obviously isn't content recommending rejection on that basis. So he's worried that the paper "could really do some damage" to another paper by Keith Briffa. Unable to determine the paper's merits quite yet, but also without any doubts about trying his best to get the paper rejected anyway, he seeks help from the very person being criticized as to how to ding the article.

Is this any way for the peer review mechanism to work? I could hardly imagine anything more directly opposite to the ideal of giving manuscripts on a blinded basis to independent and unbiased reviewers who have sufficient expertise to judge an article's merit for themselves.

Cert petition in Massachusetts free speech case

My good friend and classmate Mark Rienzi, now of Catholic Univ. Law School, has just filed a cert petition with the U.S. Supreme Court in a First Amendment case -- McCullen v. Coakley, No. 09-592. The Supreme Court should grant cert, given the important issues at stake.

The case involves a Massachusetts law penalizing people who engage in speech near abortion clinics -- including peaceful speech, or even holding a sign silently. (Keep in mind: Your position on abortion shouldn't affect how you think of the free speech issue here, any more than the Klan's or Greenpeace's right to free speech depends on whether their message is agreeable to everyone.)

Now many people may think immediately that such a challenge is hopeless. After all, didn't the Supreme Court uphold speech restrictions outside abortion clinics in the case of Hill v. Colorado?

There are three responses to this.

First, the Massachusetts law is worse -- far worse -- in First Amendment terms than the Colorado law at issue in Hill. Here are three stark differences:

(A) The Colorado law applied to all medical facilities. But the Massachusetts law applies only to abortion clinics (not even to hospitals where some abortions may be performed). This is more worrisome in terms of the legislature's intent to focus on abortion-related speech.

(B) The Colorado law applied to all speakers -- which is "the level of neutrality that the Constitution demands," in the Supreme Court's express words. But the Massachusetts law applies only to abortion protestors, with exemptions for all “persons entering or leaving” an abortion clinic or all “employees or agents of [a clinic] acting within the scope of their employment.” In other words, an anti-abortion protestor could go to jail for silently and peacefully offering someone a leaflet on a public sidewalk, while the clinic employee would be exempt for saying, "Ignore that leaflet."

(C) The Colorado law applied only to unwanted speech -- speech directed at an unwilling listener. But the Massachusetts law prohibits offering leaflets, displaying signs, engaging in conversation with even willing listeners, speaking with others at a “normal conversational distance,” or even merely remaining stationary and silently holding signs on public streets and sidewalks, whether or not any listeners (let alone unwilling listeners) are present.

Indeed, the Supreme Court's Hill decision expressly relied on these aspects of the Colorado law in upholding it. Yet the Massachusetts law thumbs its nose at all three.

Second -- and this is a bit of legal inside baseball -- the First Circuit's decision uses a ridiculous version of the Salerno doctrine, which has been read to suggest that in a facial challenge, a plaintiff must show that all applications of the statute are unconstitutional. As I and other legal scholars have shown, that doctrine does not and should not mean that, say, in a First Amendment or equal protection case, the plaintiff has the responsibility of affirmatively proving something about all specific applications, both real and hypothetical. Instead, the only thing the doctrine can mean is that where the plaintiff shows content or viewpoint discrimination (for example), all applications of the statute are in fact unconstitutional.

But the First Circuit, like some other courts, has been misusing Salerno such that a First Amendment content- and viewpoint-discrimination case is now being judged using a rational basis test. As long as the court can imagine just one hypothetical reason for the statute, the state wins against a facial challenge.

This is a stunning and dangerous development. Rational basis has no place in these types of cases.

Third, the Supreme Court's decision in Hill was wrong, and should be overruled. Commentators and scholars from Michael McConnell to Laurence Tribe have said that Hill was inconsistent with the First Amendment's protection of free speech ("slam-dunk simple and slam-dunk wrong," is how Tribe put it).

* * *

As of now, Massachusetts has waived its right to respond to the petition. The Supreme Court likely wouldn't grant cert without having a response. Thus, the next step is for the Court to consider whether to require Massachusetts to respond, or to reject the petition outright. Given that the law is much more egregious than Hill -- such that even the three remaining members of the Hill majority (Justices Breyer, Stevens, and Ginsburg) might oppose it because it is so much more targeted and restrictive -- a call for response by someone seems likely. Time will tell.

UPDATE: See this update on the case.

Krugman on Education

A recent Paul Krugman column made a puzzling assertion about education:

The rise of American education was, overwhelmingly, the rise of public education — and for the past 30 years our political scene has been dominated by the view that any and all government spending is a waste of taxpayer dollars. Education, as one of the largest components of public spending, has inevitably suffered.

It's hard to know what precisely he's saying here, but he seems to be trying to imply that education has "inevitably suffered" because of a lack of government spending.

Any such implication would be difficult to defend. As you can see in the above graph (from an Education Next article by Arthur Peng and James Guthrie), education spending has skyrocketed in real terms precisely during the period during which, according to Krugman, our politics were "dominated" by an anti-government-spending view. So what's the counterfactual here -- how much more does Krugman think we should have been spending?

Can Anybody Be A Genius? A Combined Book Review

This is a combined book review that I shopped around a while back:


Anders Ericcson is now one of the most famous cognitive psychologists in the nation. Currently teaching at Florida State, Ericcson has spent his career pioneering the study of how experts become experts. He is particularly known for his research supporting the finding (originally due to polymath Herbert Simon) that expertise in any subject -- whether it be music, science, golf, or darts, to quote his webpage -- comes only after 10,000 or so hours of deliberate practice.

Ericcson's scholarly work is considerable (in addition to publishing numerous articles in scholarly journals, Ericcson has edited and contributed to four books on expertise, including the magisterial "Cambridge Handbook of Expertise and Expert Performance"). But his recent fame is due to his prominence in several popularizations: Malcolm Gladwell's "Outliers: The Story of Success," Daniel Coyle's "The Talent Code: Greatness Isn't Born, It's Grown: Here's How," Geoff Colvin's "Talent is Overrated: What Really Separates World-Class Performers from Everybody Else," and I suspect David Shenk's optimistically-titled "The Genius in All of Us" (due to be released in 2010).

Following Ericcson's research, a running theme of all these books is that innate talent is heavily overrated, and that what we think of as a natural genius is actually the result of many hours of deliberate practice. Deliberate practice means not just mindlessly running through musical scales or hitting golf balls on the range while chatting with friends about the stock market, but deliberately and intensely focusing on every aspect of what you are doing, carefully monitoring for errors and analyzing how to improve, and doing so ad nauseam.

The importance of practice is not new, of course. That is why we have jokes about how to get to Carnegie Hall ("practice, practice, practice"), and why we so often quote Thomas Edison's famous saying that "genius is 1 percent inspiration and 99 percent perspiration." Still, it is worth being reminded that great talents almost never spring out of the blue; instead, they turn out to have had many years of deliberate practice before they emerged into greatness. Tiger Woods, for example, didn't just spontaneously become the world's best golfer; his father was an obsessive golf addict, and started having Tiger practice by age 2. Neither was Mozart (who is discussed in all three books) as effortless a genius as is often believed; his father was an accomplished musician and teacher, and drilled Mozart in piano and music theory from an early age.

Indeed, at times, these authors seem to dismiss the very possibility of innate talents and abilities. Says Colvin, "when it comes to innate, unalterable limits on what healthy adults can achieve, anything beyond [] physical constraints is in dispute." Or in the words of Gladwell, "the closer psychologists look at the careers of the gifted, the smaller the role innate talent seems to play and the bigger the role preparation seems to play." In this sentiment, they are indebted to Ericcson, who writes that expert performance is not "due to innate talent."

The actual evidence against innate talent turns out to be rather unclear. Both Gladwell and Colvin highlight ("Exhibit A," says Gladwell) one of Ericcson's studies, in which he and two colleagues studied 30 violinists at the Music Academy of West Berlin. After the students were divided into three ability-based groups (potential concert soloists, merely good violinists, and a group of students studying to be music teachers), it turned out that the groups had differing estimates of the total amount of time spent practicing before age 18: 7,410 hours for the best violinists, 5,301 hours for the good violinists, and 3,420 hours for the would-be music teachers.

But this sort of study doesn't prove that innate talent is irrelevant or non-existent. These students all already had enough ability to get into a music school in the first place, and the study therefore ignores all of the people who might have given up the violin because they had so little aptitude for it that practice never paid off. Then there are selection effects that may have affected who was in which program: perhaps the students with the least innate ability all applied to the music education program, which demanded less practice, while those with the most innate talent applied to study for a solo career.

On top of that, innate talent could itself cause students to spend more time practicing, because it's pleasurable to spend time mastering something at which you have the ability to excel. As Ericcson and his colleagues briefly concede, “heritable individual differences might influence processes related to motivation and the original enjoyment of the activities in the domain and, even more important, affect the inevitable differences in the capacity to engage in hard work.”

Now, if researchers were able to do a random assignment study -- say, assigning a random group of 7-year-olds to spend 20 hours a week practicing the violin for the next 10 years, and another random group of 7-year-olds to serve as a control group by not playing the violin -- then, and only then, would we be able to see whether anyone who practices 10,000 hours is virtually assured of turning into the next Itzhak Perlman. Needless to say, no one has ever carried out any such study, and no one ever will.

Moreover, it seems intuitively obvious that even if genius arises out of years of intense training, there was often some innate ability to start with. Take, for example, Bart Conner, the gold-medal Olympic gymnast, who startled his parents when he suddenly started walking on his hands at age 6 (as Ken Peterson notes in a recent book). Although Conner didn't rise to Olympic status until spending many hours in training, his sudden ability to walk on his hands in childhood was neither typical (as most parents could tell you), nor was it preceded by 10,000 hours of deliberate practice at handstands.

Indeed, while the fact that Michael Jordan excelled at basketball and Bill Gates excelled at computer programming is inexorably tied to the many hours each of them practiced their respective fields, it's still rather dubious to imply that it could just as easily have been Jordan who founded Microsoft and Gates who dunked from the free throw line, had they only switched what they chose to practice.

* * *

As for the merits of each book standing alone, Coyle's book is perhaps the most intellectually satisfying. Although it's very much a book for non-scientist readers, it delves into the neuroscience behind expertise. The key to expertise, from a physical perspective, is a substance known as myelin, a sort of fatty "insulation" that wraps around neuronal fibers in the brain in proportion to how often that brain circuit is fired. When you engage in deliberate practice -- or any repeated activity at all -- your brain responds by focusing more myelin on the brain circuits involved in that activity. The key role of myelin sheds light on why deliberate practice develops good habits, and conversely why bad habits are so hard to break: once myelinated, a brain circuit doesn't easily lose that myelination. Thus, "the only way to change [habits] is to build new habits by repeating new behaviors -- by myelinating new circuits."

Given that Colvin is a senior editor at Fortune Magazine, it is unsurprising that his book is aimed at delivering lessons for the time-pressed businessman looking to get an edge on the competition. Some of the lessons are no doubt useful (practice giving a presentation beforehand), but some don't seem to have much to do with the expertise research. One chapter ("Applying the Principles in Our Organizations") consists mostly of advice such as "deliberately putting managers" into new jobs that will force them to learn new areas, or avoiding "picking the wrong team members," or trying to build "trust" among fellow business associates, or trying to block "the inevitable personal agendas." All of this may be good business advice, but the citations to expertise research are rather thin.

Moreover, in many areas that businesses worry most about -- such as forecasting market developments, or making good strategic decisions -- there may not be any such thing as "expertise" in the first place. Philip Tetlock's superb book "Expert Political Judgment" collected some 80,000 predictions from people reputed to be political or economic "experts," and then waited to see if the predictions came true; the so-called "experts" turned out to be little more accurate than random chance.

Business is likely no different. Because the world is so complex, and because so many intervening events can arise, it is extraordinarily easy for managers, economists, and politicians to fool themselves into believing that their preferred course of action really was a good idea at the time. Moreover, it may be years or decades between a manager's or politician's decision, and the ultimate results of that decision. All of this, in turn, prevents them from receiving instantaneous and unambiguous feedback on their performance -- something that is absolutely necessary to deliberate practice. When you miss a golf shot, you instantly know that you've missed, by how much, and in what direction. None of that is true as to many business or political decisions.

Gladwell broadens his focus far beyond the expertise research, discussing the way in which extreme outcomes are due to communities (he opens by discussing a community of Italian immigrants that was abnormally healthy, apparently due to its small town atmosphere), culture (one chapter analyzes how Korean pilots are more likely to crash since co-pilots feel too timid to point out when the pilot is in error), and sheer luck in timing (one chapter notes that many current giants of the computer industry were born around 1955, just in time to be college-age when computers became more widely usable in the mid-1970s, but not so old that they were already safely esconced in corporate jobs). From these motley collection of these fascinating stories -- Gladwell has clearly spent 10,000 hours practicing the art of storytelling -- Gladwell draws the conclusion that we get too caught up "in the myths of the best and the brightest and the self-made," whereas success really depends heavily on circumstances or cultural backgrounds.

That conclusion is true enough, but Gladwell immediately veers in an odd direction with his policy prescription: "To build a better world we need to replace the patchwork of lucky breaks and arbitrary advantages that today determine success -- the fortunate birth dates and the happy accidents of history -- with a society that provides opportunities for all."

What? Gladwell has just spent pages and pages telling us that Asian children do well at math because they come from a cultural heritage "where, for hundreds of years, penniless peasants [were] slaving away in the rice paddies three thousand hours a year," and that to be a "great New York lawyer," the perfect combination is someone born in the 1930s to Jewish immigrant parents who worked in the garment district. All of that may be true, but these advantages, such as they are, are impossible to confer on anybody, let alone all of society.

Even the advantage that Bill Gates had -- Gladwell points out that "our world only allowed one thirteen-year-old unlimited access to a time-sharing terminal in 1968" -- doesn't provide a useful guide. In 1968, no one had any clue that personal computers would one day become as common as telephones (IBM's Thomas Watson once said, "I think there is a world market for maybe five computers," an infamous example of the impossibility of predicting future business developments). No one could possibly have known that Gates's teenage hobby of programming was actually a huge headstart on an industry that would one day be worth billions. Similarly, it makes no sense to suggest that we need a national policy of giving "all" children a headstart on whatever will be a brand-new multi-billion dollar industry in the year 2040. No one knows how to do such a thing.

More on Deas Vail

I've been a huge fan of Deas Vail since discovering them nearly 2 years ago. The best way to describe them would be this: indie pop-infused rock with soaring and haunting melodies sung by one of the best vocalists in rock. Having seen them three times in concert, I can say that they're far better live than most bands are in the studio with the benefit of editing, autotune, etc.



As of October 27, their new album "Birds and Cages" is available on Amazon, or Itunes, or through their new record label Mono vs. Stereo (for only $7.00!).

I have to admit, even as much as I loved their first album "All The Houses Look the Same," I was a little bit nervous last year when they announced that they were recording both a 5-song EP ("White Lights") and a full-length album, all at the same time. I wondered if maybe they would be stretched too thin.

Boy was I ever wrong. To my ears, it's one of the best albums I've ever bought, let alone in 2009. If anything, they've taken their music to a new level. Wes Blaylock (the lead singer) is just as jaw-dropping with his soaring vocal pyrotechnics, and his wife Laura (keyboardist) is featured more often in some quite lovely vocal duos. Andy Moore (guitar) is constantly pushing the envelope with different sounds and effects. Kelsey Harelson's drumming rarely just keeps a steady beat; instead, he manages, in song after song, to use the offbeats in a way that propels the music forward and gives it a sense of mission and urgency. Finally, Justin Froning's energetic bass playing rounds out the band nicely.

.

The tunes are just as full of earworms (i.e., melodic hooks that you can't get out of your head) as ever, but the music is often adventurous and arresting. "Sunlight" has a progressive rock sound that is hard to place in a single key, and the verses are in 7/8 time. The intro to "The Great Physician" has a really nice chord progression that is far different from your traditional I-IV-V. The verses in "Atlantis" are in 5/4 time. "Puzzles and Pieces" is just beautiful -- I could easily hear this song featured in one of the many television shows that run out the clock by playing a poignant song while the various characters look pensive.



As another reviewer put it, "Deas Vail is one of the few bands who can take pop/rock music and force the listener to apply 'beautiful' to it as an adjective." If that sounds appealing to you, "Birds and Cages" should be the first thing you purchase.