The Unreliability of Confessions

From BPS comes this summary of a study that was just published:

Not surprisingly, confessions are extremely persuasive in court, but according to Jessica Klaver and colleagues, all too often these confessions are false, leading to the wrong person being found guilty.

Now Klaver's team have used an elegant laboratory task to compare two types of interrogation technique and found that it is so-called 'minimising' questions and remarks - those that downplay the seriousness of the offence, and which blame other people or circumstances - that are the most likely to lead to a false confession.

Over two hundred Asian and Caucasian students were invited to take part in what they were told was a test of their personality and typing skills. During the typing part of the task, they were warned in advance that pressing the 'Alt' key would cause the computer to crash and a loss of all data. Subsequently, when the participants were required by the task to type 'Z' (near the 'Alt' key), the researchers contrived it so that the computer duly crashed, and the participants were accused of pressing the 'Alt' key.

Next the students were subjected to either 'minimising' remarks (e.g. "Don't worry. It was just an accident" and "This programme seems not to be working lately") or 'maximising' remarks that played up both the evidence for the student being guilty and the seriousness of the alleged error (e.g. "You must have pressed it" and "We have run over 50 people on this test in the past three weeks and the computer hasn't crashed once").

Overall, 43 per cent of the students subsequently signed a confession statement, stating falsely that they had indeed pressed the 'Alt' key. Crucially, the confession rate was four times higher among the students subjected to minimising remarks as opposed to maximising remarks.
* * *

Add to that the fact that psychologists have long known that eyewitness testimony is extremely unreliable and manipulable, as is human memory more generally. One fairly recent study:

Victims who get a good long look at violent criminals are unlikely to identify them accurately later, Yale and U.S. Navy researchers have found.

This caveat follows from a unique study of 509 Navy and Marine officers undergoing elite survival training at Fort Bragg, N.C.

Results suggest that police and juries may give eyewitness testimony too much credibility, said Dr. Charles A. Morgan III, a Yale psychiatrist and lead author of the study.

"Memory in healthy people is not inherently terribly accurate. There's a substantial amount of error," Morgan said. "Maybe we should demand more evidence."

* * *

Officers who were absolutely positive that they had selected the right person were no more likely to be correct than officers who expressed some doubt.

"Unfortunately, that's what people on juries listen to," Morgan said.

These sorts of studies are reason to be cynical not just about the death penalty, but about the entire criminal justice system. What does it say that the evidence deemed most reliable by juries -- confessions and eyewitness accounts -- might often be false? At the same time, lots of crimes are indeed committed each year, and it seems implausible that the criminal justice system would manage to catch only the innocent people. It's a quandary. Maybe people who are actually tried and convicted by a jury are more likely to be innocent (because of the possibility of faulty testimony or confessions) than are people who plea bargain knowing that they're guilty.

A couple of new bands

Here are a couple of Arkansas bands that I like.

The first is Hannah Blaylock and Eden's Edge. This is a bluegrass band that reminds me a lot of Alison Krauss, except that Blaylock's voice is a bit less breathy. Their main website is here and their MySpace page is here. They won a 2008 Independent Music Award in the Americana category.



The second band is Deas Vail, whose lead singer is Wes Blaylock (Hannah's brother). Their MySpace page is here. They are a progressive rock band with lots of hook-filled melodies. Wes Blaylock has a high, sweet tenor voice, and uses falsetto a good bit (the sound reminds me of Copeland). You can listen to the whole album here.

One review:

The guitars are perfectly understated for the overall tone and atmosphere of the album, allowing the piano and vocals to radiate throughout, making this album elegantly proportioned musically. Deas Vail has created one of the most captivating album that I've listened to in a long time. It allows you to feel, to contemplate, to realize.Wes Blaylock's voice is astounding. It's almost ethereal at times but never so much that its delicacy is overtaken by the instruments. His classical vocal training allows his falsetto voice to fit perfectly among the collective sound of the instruments.

Another:

There are only a few artists who I feel truly privileged to have stumbled upon, and Deas Vail (read: day-ahs-vale) is one of them. Although they are a relatively young band and have just released their first full-length album, All the Houses Look the Same, earlier this year, they seem to effortlessly make music that is original and accessible and interesting. Their brand of indie rock breaks away from the norm because of the ambient instrumentation and "nursery rhyme-ish" melody lines (as leader singer Wes says); they manage to avoid excessive use of heavy guitars in favor of melodic pianos and synths. The song structures on this album defy traditional rock formulas, which sometimes makes it difficult to grasp onto the hooks or choruses of their music at first listen. However, I consider this to be a good thing, because after a few listens, one begins to appreciate the complexity of their instrumentation and lyrics; there are consistently new things to discover, which keeps Deas Vail's music far from being boring.

Here is a lengthy interview with the band.


Here's a video:

Poem for the Day

Oh, how I love Humanity,
With love so pure and pringlish,
And how I hate the horrid French,
Who never will be English!

* * *

The villas and the chapels where
I learned with little labour
The way to love my fellow-man,
And hate my next-door neighbour.

G.K. Chesterton, "The World State," in Collected Poems (London: Cecil Palmer, 1927), at p. 13.

Charter Schools' Effect on Public Schools

Do charter schools force the nearby public schools to improve (i.e., via putting competitive pressure on them)? Or do they drain resources or teachers or perhaps positive peer groups from those nearby public schools?

One recent paper on this topic is Scott A. Imberman, The Effect of Charter Schools on Non-Charter Students: An Instrumental Variables Approach.

The author explains that the usual school fixed effects models could be unduly affected by selection effects in the places that charters choose to open. For example, what if a local public school were on a downward trend, and then a charter opened? Depending on how that trend independently moved (i.e., whether it continued downwards or reverted back upwards), it might look like the charter school had either more of a positive effect or more of a negative effect than it really did.

So the author adds in an instrumental variable model, the instrument being "the availability of buildings between 30,000 and 90,000 square feet near regular public schools." He says that the reason for choosing this instrument is that "charter schools need a space of substantial size in order to operate," and therefore an increase in large nearby buildings "increases the probability of a charter opening nearby in a manner that is orthogonal to student outcomes." In other words, as I understand it, the notion is to measure the effect of charters on public schools not by looking directly at whether a charter is nearby (too much selection effect driven by the quality or trendline of the public school itself), but by looking at the number of large buildings nearby (which ultimately is just a proxy for the number of charters in the second stage of the equation, but isn't itself affected by trends in the public school).

Oddly, when he looked at his dataset (from an anonymous urban school district) with the usual fixed effects models, and then using the instrumental variables model, the results were the opposite. Specifically:

While my school fixed-effects results show moderate gains in test scores and worsening discipline, my IV estimates show that charter schools improve discipline and worsen test scores in non-charter schools. IV estimates suggest that a ten percentage point increase in charter share within 1.5 miles of a non-charter school reduces disciplinary infractions by a statistically significant 0.55 per year while test scores drop by up to 4 national percentile rankings in levels models and 1.5 NPR in value added models depending on the test.

I was a bit surprised by this flip in results. I wonder if the instrument is really uncontaminated here -- public schools that have lots of large buildings nearby are probably more likely to be in urban areas, are they not? And in those areas, public school performance, as well as the number of people interested in charters, might be different from other areas in his dataset.

In any event, it bothers me that in both models, school disciplinary incidents seem to be inversely related to test scores. That doesn't make intuitive sense to me at all.

UPDATE: Pat Wolf points me to Michael P. Murray, "Avoiding Invalid Instruments and Coping with Weak Instruments," Journal of Economic Perspectives 20 n.4 (2006): 111-132, which points out that "all instruments arrive on the scene with a dark cloud of invalidity hanging overhead. This cloud never goes entirely away, but researchers should chase away as much of the cloud as they can." [Unrelated note: The author of that paper thanks, among many other notable folks, Daron Acemoglu for his helpful comments; I think there must be some sort of rule that Acemoglu has to be thanked in all published economics papers.]

Hoxby on Charter Schools

Worth noting: Caroline Hoxby's speech on the promise of charter schools.

Map of Slavery

Via Matthew Yglesias and Edge of the West, here is a map showing the prevalence of slavery in the Southern states in 1861. (Click on the link for a larger version.)




What I found fascinating was that -- notwithstanding the Great Migrations of the 20th century -- the map very closely mirrors where blacks tend to live today. Here is a map based on the 1990 census (click for a much larger image):



A map based on the 2000 census is here.

The similarities are obvious just by considering the fact that both in 1861 and today, most blacks lived in the Black Belt and especially in the Delta region of Arkansas, Louisiana, and Mississippi. Also notable are the absences of blacks in the northern mountainous regions of Arkansas, and of Georgia up through Appalachia. Fewer blacks lived in the coastal regions of Mississippi and Alabama.

Then if you compare the maps more closely, even more similarities jump out. For example, both in 1861 and today, hardly any blacks lived in Livingston Parish, Louisiana, especially compared to the surrounding counties. For another example, there is a pocket of three mostly white counties north of Birmingham, and then higher concentrations of black people in a stripe of countries stretching from northern Alabama through Nashville. In Florida, the highest concentrations of blacks are consistently found in Leon County and a few surrounding counties in the panhandle.

Much about America has changed in the past 150 years, but there are still underlying similarities that show up in surprising ways.

P.S. Although this 1950 map isn't very legible, I think it shows most of the same similarities:

UPDATE: I wonder how much of this is related to the existence of sundown towns in largely white areas. As author James Loewen points out, "Most Americans have no idea such towns or counties exist, or they think such things happened mainly in the Deep South. Ironically, the traditional South has almost no sundown towns." It actually doesn't seem so ironic to me: Areas that traditionally had slavery and where upwards of 50% of the population was black would have had a very hard time keeping blacks completely out. Indeed, given that so many blacks today live in areas where slavery was once practiced, by definition they could not have been expelled in most of those places.

Instead, most sundown towns were in historically white areas both in the North and in white areas of slave states. For example, all of the sundown towns in Arkansas, so far as I can tell, were in areas that hardly had any slavery. Their racism arose from the fear of the unknown, I'd guess.

Anyway, because of the prevalence of sundown towns, blacks would have had a hard time moving to many locations outside of the traditional areas that had practiced slavery.

Links

1. Get deep sleep.

2. John Brockman's site The Edge has always been a source of endlessly fascinating interviews, articles, etc. His "The Edge Annual Question -- 2008" is "What Have You Changed Your Mind About? Why?" Easily over 100 prominent thinkers and scientists have written answers to that question, including Richard Dawkins, Nassim Taleb, Daniel Kahneman, Clay Shirky, Simon Baron-Cohen, Daniel C. Dennett, Helena Cronin, Aubrey de Grey, Nicholas Carr, John Allen Paulos, George Dyson, Paul Davies, Steven Pinker, Jon Haidt, Judith Rich Harris, Daniel Gilbert, Lee Smolin, Lee Silver, Robert Provine, Nick Bostrom, Freeman Dyson, Michael Shermer, Howard Gardner, and a lot of names that I didn't immediately recognize but who have interesting thoughts nonetheless (if such a thing can be imagined). The easiest way to read through all the entries is to start at Page 1.

Unsurprisingly, I liked this thought by John McCarthy:

Attitudes Trump Facts

I have a collection of web pages on the sustainability of material progress that treats many problems that have been proposed as possible stoppers. I get email about the pages, both unfavorable and favorable, mostly the latter.

I had believed that the email would concern specific problems or would raise new ones, e.g. "What about erosion of agricultural land?"

There's some of that, but overwhelmingly the email, both pro and con, concerns my attitude, not my (alleged) facts. "How can you be so blithely cornucopian when everybody knows ..." or "I'm glad someone has the courage to take on all those doomsters."

It seems, to my surprise, that people's attitude that the future stems at least as much from personality as from opinions about facts. People look for facts to support their attitudes — which have earlier antecedents.

I found it intriguing that two scientists (Robert Provine and Irene Pepperberg) attack the notion that science should consist of hypothesis testing.

I also was intrigued by this observation from Leo Chalupa, a brain scientist:

Here is a real puzzle to ponder: Every cell in your body, including all 100 billion neurons in your brain is in a constant process of breakdown and renewal. Your brain is different than the one you had a year or even a month ago, even without special brain exercises. So how is the constancy of one’s persona maintained? The answer to that question offers a far greater challenge to our understanding of the brain than the currently in vogue field of brain plasticity.

This observation by Paul Ewald is useful:

I still think that it is wise to trust the experts when their profession has a good understanding of the processes under consideration. * * *

I am becoming convinced, however, that the opposite view is often true when the expert opinion pertains to the unknown: the longer and more widespread the accepted wisdom has been accepted, the more hesitant we should be to trust it, especially if the experts have been studying the question intensively during this period of acceptance and contradictory findings or logic have been presented. The reason is simple. If an explanation has been widely and broadly accepted and convincing evidence still cannot be mustered, then it is quite reasonable to expect that the experts are barking up the wrong, albeit cherished, trees. That is, its acceptance has more to do with the limitations of intellectual ingenuity than with evidence.

This argument provides a clear guideline for allocating trust to experts: distrust expert opinion in accordance with what is not known about the subject. This guideline is, of course, difficult to apply because one has to first ascertain whether a discipline actually has valid answers for a given area of inquiry. Consider something as simple as a sprained ankle. Evolutionary considerations suggest that the inflammation and pain associated with sprained ankles are adaptive responses to promote healing, and that suppressing them would be detrimental to long-term functionality of the joint. I have searched the literature to find out whether any evidence indicates that treatment of sprained ankles with ice, compression, anti-inflammatories, and analgesics promotes or hinders healing and long-term functionality of the joint. In particular, I have been looking for comparisons of treated individuals with untreated controls. I have not found any and am coming to the conclusion that this widely advocated expert opinion is a detrimental holdover from ancient Greek medicine, which often confused the return of the body to a more healthy appearance with the return of the body to a state of health.

More generally, I am coming to the disquieting realization that much of scientific opinion and even more of medical opinion falls into the murky area circumscribed by a lack of adequate knowledge about the processes at hand.