Empiricism

A while back, I suggested that liberals are on average no more empirically minded -- or shall we say, "reality-based" -- than are conservatives. It's just that liberals and conservatives tend to be empirically minded about different things.

In reading the book Stumbling On Happiness, by Daniel Gilbert, I came across his description of a study on this point:

In one study, volunteers were asked to evaluate two pieces of scientific research on the effectiveness of capital punishment as a deterrent. They were shown one research study that used the "between-states technique" (which involved comparing the crime rates of states that had capital punishment with the crime rates of states that did not) and one research study that used the "within-states technique" (which involved comparing the crime rates of a single state before and after it instituted or outlawed capital punishment).

For half the volunteers, the between-states study concluded that capital punishment was effective and the within-states study concluded it was not. For the other half of the volunteers, these conclusions were reversed. The results showed that volunteers favored whichever technique produced the conclusion that verified their own personal political ideologies. When the within-states technique produced an unfavorable conclusion, volunteers immediately recognized that within-states comparisons are worthless because factors such as employment and income vary over time, and thus crime rates in one decade (the 1980s) can't be compared with crime rates in another decade (the 1990s). But when the between-states technique produced an unfavorable conclusion, volunteers immediately recognized that between-states comparisons are worthless because factors such as employment and income vary with geography, and thus crime rates in one place (Alabama) can't be compared with crime rates in another place (Massachusetts).

Clearly, volunteers set the methodological bar higher for studies that disconfirmed their favored conclusions.

The study is this: C. G. Lord, L. Ross, and M. R. Leper, "Biased Assimilation and Attitude Polarization: The Effects of Prior Theories on Subsequently Considered Evidence," Journal of Personality and Social Psychology 37:2098-109 (1979).

Gilbert adds this endnote: "It is no consolation that in subsequent studies, both established scientists and scientists in training showed the same tendency to favor techniques that produced favored conclusions. See J. J. Koehler, 'The Influence of Prior Beliefs on Scientific Judgments of Evidence Quality,' Organizational Behavior and Human Decision Processes 56: 28-55 (1993)."