Charter Schools' Effect on Public Schools

Do charter schools force the nearby public schools to improve (i.e., via putting competitive pressure on them)? Or do they drain resources or teachers or perhaps positive peer groups from those nearby public schools?

One recent paper on this topic is Scott A. Imberman, The Effect of Charter Schools on Non-Charter Students: An Instrumental Variables Approach.

The author explains that the usual school fixed effects models could be unduly affected by selection effects in the places that charters choose to open. For example, what if a local public school were on a downward trend, and then a charter opened? Depending on how that trend independently moved (i.e., whether it continued downwards or reverted back upwards), it might look like the charter school had either more of a positive effect or more of a negative effect than it really did.

So the author adds in an instrumental variable model, the instrument being "the availability of buildings between 30,000 and 90,000 square feet near regular public schools." He says that the reason for choosing this instrument is that "charter schools need a space of substantial size in order to operate," and therefore an increase in large nearby buildings "increases the probability of a charter opening nearby in a manner that is orthogonal to student outcomes." In other words, as I understand it, the notion is to measure the effect of charters on public schools not by looking directly at whether a charter is nearby (too much selection effect driven by the quality or trendline of the public school itself), but by looking at the number of large buildings nearby (which ultimately is just a proxy for the number of charters in the second stage of the equation, but isn't itself affected by trends in the public school).

Oddly, when he looked at his dataset (from an anonymous urban school district) with the usual fixed effects models, and then using the instrumental variables model, the results were the opposite. Specifically:

While my school fixed-effects results show moderate gains in test scores and worsening discipline, my IV estimates show that charter schools improve discipline and worsen test scores in non-charter schools. IV estimates suggest that a ten percentage point increase in charter share within 1.5 miles of a non-charter school reduces disciplinary infractions by a statistically significant 0.55 per year while test scores drop by up to 4 national percentile rankings in levels models and 1.5 NPR in value added models depending on the test.

I was a bit surprised by this flip in results. I wonder if the instrument is really uncontaminated here -- public schools that have lots of large buildings nearby are probably more likely to be in urban areas, are they not? And in those areas, public school performance, as well as the number of people interested in charters, might be different from other areas in his dataset.

In any event, it bothers me that in both models, school disciplinary incidents seem to be inversely related to test scores. That doesn't make intuitive sense to me at all.

UPDATE: Pat Wolf points me to Michael P. Murray, "Avoiding Invalid Instruments and Coping with Weak Instruments," Journal of Economic Perspectives 20 n.4 (2006): 111-132, which points out that "all instruments arrive on the scene with a dark cloud of invalidity hanging overhead. This cloud never goes entirely away, but researchers should chase away as much of the cloud as they can." [Unrelated note: The author of that paper thanks, among many other notable folks, Daron Acemoglu for his helpful comments; I think there must be some sort of rule that Acemoglu has to be thanked in all published economics papers.]