Selection Effects

Homeschoolers regularly achieve average test scores that are substantially higher than the average scores from public school students.

On occasion, you'll see this result chalked up to selection effects: Parents who choose to homeschool in the first place tend to be just the sort of parents who are intensely devoted to their children's education. For example, this article:

To complicate all these issues, the data on home schooling are very limited. By definition, parents who home school take a more active role in their children’s education. It may very well be the case that these kids would succeed in most any academic environment, precisely because their parents have made education a family priority. It is thus difficult to come to definite conclusions about home schooling as an educational method.

My analysis of college admissions scores and acceptance rates suffers from the same selection effect, but perhaps to a greater degree. Here I am focusing not just on home schoolers, but on only those home schoolers bound for college. When I look at college admission tests, I am limiting the sample to those home schoolers who value higher education. The stories of superior academic achievement, higher test scores, and campus success may be less a function of home schooling than an indication of students who take their academic careers very seriously.

Something about this argument bothers me, and I've been mulling it over, trying to put it into words. Let me try coming at it in a roundabout way.

Imagine that you had two identical public schools in a given town -- identical in every way. For some reason, parents have gotten the notion that School A is better than School B (not true, because both are identical). Nonetheless, the parents who are most intensely interested in academics all do their best to move into the district for School A. As a result, School A's test scores rise above School B's -- not because it does anything differently, but solely because academically-oriented people selected to enroll in that particular school.

What does this tell us? If we know that selection effects are really the cause, then there really isn't anything about School A that is superior. Because of that, if you were randomly assigning students to schools, there would be no reason to expect the students in School A to do any better. And for the same reason, if you were deciding where to send your own kid to school, it would be worth knowing that School A looks better only because of the pre-existing qualities of the kids who choose to go there, rather than anything about School A itself.¹

But all of this presupposes that it is possible to tease apart the selection effect from the thing that you're trying to measure. Put another way, the mere act of selecting School A does not increase academic performance. If people who are already academically superior -- for independent reasons -- select School A, that act of selection gives rise to the artificial statistical superiority of School A, but that's all. The selection has no more meaning than that. Selection can be separated from school performance.

Can the same be said of homeschooling? I'm not so sure. For one thing, homeschooling is not, and never could be, random. It's not as if anyone could ever assign students randomly to be homeschooled, to see what the effects of homeschooling would be in those families that have no interest in education.

Conversely, it's always going to be the case that -- except for a few negligent people who use homeschooling as an excuse for truancy -- homeschoolers take an intense interest in their children's education.

In other words, you can't really separate the act of selecting homeschooling from the intensity of interest in education. To the contrary, the very fact that someone selects that option is a sign of intense interest in education. It's not as easy to say, "These are all people who were already independently superior at education, and the fact that they selected homeschooling does nothing more than create a statistical artifact." To the contrary, the very act of selecting homeschooling could show a child that her parents take a genuine interest in her education, and could thus be one of the things that makes her more academically successful.

More generally: Watch out for so-called "selection effects" where the very act of selecting a particular activity could be beneficial in itself.

¹With an individual decision, of course, the picture is a bit more complicated. If the student body at School A is superior, it might be worth trying to send your kid there simply for the superior peer effects.

Scalia's Mistake

My friend and classmate Brian Fitzpatrick has an op-ed by that title here, in which he argues that Justice Scalia should not have recused himself from the Pledge of Allegiance case.

More Religious Maps

I wonder why there's such a hotbed of Episcopalians in South Dakota?

Omaha School Segregation?

The New York Times describes a new law in Nebraska as creating "racially identifiable" school districts:

Ernie Chambers is Nebraska's only African-American state senator, a man who has fought for causes including the abolition of capital punishment and the end of apartheid in South Africa. A magazine writer once described him as the "angriest black man in Nebraska."

He was also a driving force behind a measure passed by the Legislature on Thursday and signed into law by the governor that calls for dividing the Omaha public schools into three racially identifiable districts, one largely black, one white and one mostly Hispanic.

The law, which opponents are calling state-sponsored segregation, has thrown Nebraska into an uproar, prompting fierce debate about the value of integration versus what Mr. Chambers calls a desire by blacks to control a school district in which their children are a majority.

Civil rights scholars call the legislation the most blatant recent effort in the nation to create segregated school systems or, as in Omaha, to resegregate districts that had been integrated by court order. Omaha ran a mandatory busing program from 1976 to 1999.

* * *
The bill contained provisions creating a "learning community" to include 11 school districts in the Omaha area operating with a common tax levy while maintaining current borders. It required districts to work together to promote voluntary integration.

But the legislation changed radically with a two-page amendment by Mr. Chambers that carved the Omaha schools into racially identifiable districts, a move he told his colleagues would allow black educators to control schools in black areas.

One thing that I've learned is never to trust a journalist's description of any law whatsoever; always go the original for yourself. It took a bit of work, but I found the actual bill here (52 page PDF file).

Here's the interesting thing: While the New York Times reporter keeps using the term "racially identifiable," the law itself does no such thing. In fact, it took me a while to figure out just how the law affected Omaha at all. Turns out that section 41 (on page 15) is the relevant provision:

Sec. 41. (1) On or before July 1, 2007, each learning community coordinating council shall submit a plan to the state committee to divide any Class V school districts in the learning community into new Class V school districts organized around the attendance areas of existing high school buildings which are not currently being used exclusively for specialized programs, with two or three such high school buildings in each new Class V school district. Such new Class V districts shall consist of school buildings having attendance areas which are contiguous. The effective date for reorganizations pursuant to this section shall be July 1, 2008. Such reorganizations shall not be subject to the approval or disapproval of any school board pursuant to section 37 of this act.

I know that this is the relevant section only because the state legislature's website has an article that contains this passage:

Saying the people in various communities within Omaha need more control over the educational opportunities in their areas of the city, Chambers offered an amendment that will, in effect, separate the OPS [Omaha Public School] system into three districts.

Under the Chambers amendment, the learning community's coordinating council is required to submit recommendations for splitting OPS into separate districts around attendance areas that contain two or three of the seven existing high schools.

It seems misleading to imply that the Nebraska state legislature is explicitly classifying students by race. Instead, it is creating school district lines around existing high schools that happen (so we are told) to correspond roughly to white neighborhoods, black neighborhoods, and Hispanic neighborhoods. I'm not sure why this is really any different from what goes on in most major cities. Why else are there so many inner-city schools that are 98-100% black? What is the principle here: It's ok to send blacks to a neighborhood school that is mostly black, but it's "segregation" to redraw the school district lines so that blacks have more political power?

Note that Chambers thinks there is already segregation in the public schools there:

Chambers said OPS is already segregated because of its system of neighborhood schools. The proposal is not designed to segregate students and redraw district boundaries based on racial lines, he said.

"OPS has put a lot of people on a false trail," he said.

And note that a couple of commenters at La Shawn Barber's site (comments 21 and 27) provide much more information on Omaha events than the Times does.

UPDATE: Ernie Chambers sounds like quite a character:

Chambers is famous for an unsurpassed knowledge of legislative rules, which he uses to derail bills that threaten those he calls the “downtrodden.” This attracts the criticism that he is “the great obstructionist,” better at halting legislation than creating laws. As one colleague observed, “In Washington they call it a filibuster. In Lincoln, they call it Ernie.” Once, Chambers filibustered on the state budget until his colleagues agreed to set aside half a million dollars for a minority scholarship fund. In the 2005 session, he blocked the legalization of concealed weapons, as well as a constitutional amendment protecting the right to hunt, which he said would “trivialize and pollute” the state constitution. In classic Ernie Chambers style, he introduced a raft of riders to the amendment that would protect such other rights as “creating, recreating, conversating and procreating,” “hunting for the link between Noah’s Ark, Joan of Arc and Archimedes,” and “sitting on the front porch on a warm summer evening, drinking a glass of cold lemonade, dreamily watching the silvery moon rise to begin its journey across a darkening velvet sky powdered with stardust.”

Reminds me of Rep. Sedgwick's observation that the First Amendment was not really necessary, because "if the committee were governed by that general principle [that rights should be enumerated], they might have gone into a very lengthy enumeration of rights; they might have declared that a man should have a right to wear his hat if he pleased; that he might get up when he pleased, and go to bed when he thought proper."

More on Laptops for Students

Just what I suspected:

Give a kid a laptop and it might not make any difference.
That's the message from research presented here Monday, which suggests that spending millions of dollars to bring technology into kids' homes and schools has decidedly mixed results.

Taxpayer-supported school computer and Internet giveaways are political gold, but studies have questioned whether they actually help student achievement. This research, presented at the American Educational Research Association's annual meeting, confirms skeptics' doubts.

* * *

But using computers, for instance, to teach reading in primary grades actually showed negative results.

Technology giveaways aren't limited to U.S. schools. Researchers in England studied 80 schools that had received electronic "whiteboards," computerized chalkboards that allow teachers to use special markers for lessons. The $2,000 whiteboards also allow them to save their work to a computer and even surf the Internet with a class. Researchers found that teachers and students like them, but that they have a "very small and short-lived" effect on skills.

Sheep Sour

I found a patch of sheep sour in the yard today, and was showing my children that you can eat it. My late grandfather, who was a farmer, showed me the same thing probably 25 years ago. Then it occurred to me to wonder whether anyone else called it "sheep sour." Turns out that there are only a few webpages that use the term. A much more common name for the plant is creeping wood sorrel. That said, my grandfather's folksy term is closer to the official botanical term: Oxalis Corniculata. ("Oxalis" means "sour," and "corniculata" means "horned," as a sheep might be.)

The Kids

1. From my six-year old son: "How come we can't taste our own taste buds?" I don't know, although I think it would be annoying if we could.

2. My four-year old daughter hates chocolate. How much? This much: The other day, we were eating sour gummies – VERY sour. The kind that make you wince for several seconds. We gave one to our daughter. She took one lick, and as expected, she screwed up her face at the taste. Then she solemnly asked, "Does this have chocolate inside it?" On being assured that it didn’t, she kept eating and wincing.

Credentialism

This post from the Carnival of Homeschooling led to this article about two Pennsylvania sociologists who decry our society's increasing emphasis on useless credentials:

Collins, for instance, marvels at the common notion that producing more degree-holders will help more people achieve the American Dream, claiming the concept “has a kind of dog-chasing-its-tail quality to it.” Increasing the number of credentialed people competing for a finite number of jobs tends to ratchet up the educational requirements for those jobs without increasing anyone’s income. “Imagine if we said we want every school in the country to have a championship football team, that every team should win 90 percent of its games,” he says. “People would recognize the flaw in that thinking. But we say that about education all the time.”

Says Berg, “More and more lower-income people are attending college at higher rates than they ever did before. And they are taking jobs way below what college degrees would have gotten them years ago.” Graduates burdened by student loans discover that the job market is so glutted that they can’t find work that pays well enough to discharge their debts. “It’s a real menace,” he adds. “These kids are mortgaged to the hilt.”
* * *

The dirty secret in the article title is Collins’ observation that credential inflation keeps universities flush with tuition dollars, which help to finance the livelihoods of senior faculty such as himself. “Most intellectuals in liberal society, we take it pretty much as an article of faith that we need to expand education,” he says. “It’s also for us a rather self-serving argument. It provides our positions.”

That's no surprise. Professors and intellectuals are prone to the same cognitive error as every other business enterprise: What's good for them simply must be good for America.

I was especially intrigued by this paragraph:

The introduction of jet travel in the late 1950s gave Berg’s team a unique opportunity to test whether cognitive skills are best honed in school or on the job. To run the nation’s new system of air-traffic control, the Federal Aviation Administration hired and trained 507 men, half of whom had no formal education beyond high school. Years later, when Berg and his assistants evaluated the group’s job performances by counting the numbers of awards the men had earned from supervisors, half the college graduates had earned no awards, while a little over one-third had earned two or more. Among high-school graduates, though, only 30 percent had earned no awards and 43 percent had earned at least two. Results were mixed among controllers who had some education past high school. “Education,” Berg concluded, “proves not to be a factor in the daily performances of one of the most demanding decision-making jobs in America.”

As Berg saw it, the “great training robbery” of the postwar years was the massive waste of government spending on an education system that does little or nothing to enhance worker productivity. At the time, annual expenditures on U.S. higher education stood at about $17 billion. Today, the figure is $270 billion, of which $55 billion is supplied by state appropriations, with another $81 billion coming from federally funded student financial aid.

So why do so many people pursue higher education? One word: signaling.

Education and Jobs sold surprisingly well when it was published, and it influenced a young Stanford economist named Michael Spence who would go on to win the 2001 Nobel Prize for his work on “market signaling.” Spence was among the first to apply rigorous economic theory to Berg’s notion that employers were using education as a simple screening device and not as an indicator that job applicants with more education possess superior skills. Spence’s theory is that, even if education is devoid of any practical content, a high level of education can “signal” to an employer that an individual is productive enough to endure the costs in time, effort, and money to advance.

I wouldn't be surprised if blame-avoidance played some role as well. If a particular person doesn't work out on the job, it's easier to say, "How was I to know that an MBA wouldn't be up to snuff?," while if the person was a mere high school graduate . . .

The sociologists also point out that when society expects people to go to college in order to have a good job, this inhibits social mobility. This is because only the wealthier segments of society tend to be able to sacrifice 7 or 8 working years to pursue bachelors' and graduate degrees.

Bottom line: What our society needs is a cheaper way to signal intelligence and the capability for hard work.

The last thing that we need to do is increase access to higher education, as if a society that was 100% college-educated could survive by everyone doing each other's taxes or filing lawsuits against each other. (Compare G.K. Chesterton's great line: "We cannot all live by taking in each other's washing.")

Someone would have to do the manufacturing, the farming, the manual labor, the service jobs, etc. And even in the world of business, I'd bet that the vast majority of people end up using knowledge that they learned on the job. Most college graduates are going to end up under-employed or employed in a different field. Thus, increasing access to college means that, on average, you're putting more people in debt and increasing their under-employment.

Just throwing out an admittedly tongue-in-cheek idea here, but maybe Congress should ban all but the top 20% of students from going to college. (Or maybe 25%? I'm open to negotiation on the exact percentage.) Counterintuitive, no? But that might be the only thing that would work. There's a collective action problem here. If everyone else in a society is racing to get the best credentials, then you as an individual may well be worse off if you drop out of the race -- even though everyone in society would be better off if more people dropped out. Same for businesses: If you're the human resources director for a corporation, and you know that most hard-working and intelligent people in our society do in fact have a college education or a graduate degree, then it is only rational for you to demand those credentials too -- even if their formal education is going to be absolutely useless on the job.

But with a ban in place, you'd have a lot of people in America who would be hard-working, intelligent, and deserving of good jobs. And the people doing the hiring would recognize that fact. They would no longer be able to assume that if you lacked a college degree, you must be in the bottom 30%.

Thus, businesses would necessarily have to come up with some other way to find out whether someone is worth hiring. And I bet that whatever method that they come up with will be cheaper than requiring everyone to spend 4-to-8 years and many thousands of dollars (both in real costs and in opportunity costs), not to learn anything that they will actually use on the job, but just to signal that they are a decent person.

Mobile Bioweapons Lab

If this Iraqi trailer wasn't a mobile bioweapons lab, as this Washington Post article claims, then how come the Iraqis had hidden it in a cave? (By the way, the location of the trailer's discovery hasn't been reported anywhere but here, as far as I can tell.) There could be an innocent explanation for that, I suppose, but it would be nice to know.

Scalia's Gesture

There was a bit of manufactured controversy recently over the fact that Justice Scalia possibly said a Sicilian vulgarity ("vaffanculo") after mass when a reporter "asked the justice how he responds to critics who might question his impartiality as a judge given his public worship."

If a reporter ever collared Justice Ginsburg or Breyer outside of a synagogue and asked, "What do you say to people who question whether a Jew can be an impartial Supreme Court Justice," I hope they'd say the same.

Greenhouse and the Cert. Pool

If you don't read to the bottom of this Orin Kerr post about the cert. pool at the Supreme Court, you'd miss this comment from NY Times journalist Linda Greenhouse:

I’m agnostic on the merits or demerits of the cert pool, but I do have experience that might be relevant to this discussion. For the 25+ years that I have been covering the court, I have looked at every paid cert petition and made my own judgment about whether the court would deem it certworthy, and whether it would be newsworthy if granted or if denied. It is in fact quite easy to cull out the handful of petitions on each conference list that require further exploration than a quick glance at the question presented. (Certainly that would also be true of the IFP petitions, which I look at only if they are relisted or if it’s a case I know about or have been following.) It’s a weekly burden but not an overwhelmingly onerous one, and what’s interesting is that the experienced members of the press corps usually agree on which petitions will or ought to be on the justices’ screen. My point is that the raw material itself, rather than variations in who reads it, tends to drive the collective judgment, and I’m not sure the court’s docket-setting function would change much if the cert pool were smaller, or non-existant.

Wow. I wonder if anyone else in the United States -- except for Justice Stevens, perhaps -- could say the same.

Yogurt

I've been making my own yogurt lately. Thanks to this Jane Galt post, I got this yogurt maker from Amazon. It's unbelievably simple: You heat a quart of milk, not to a boil, but slightly short of that, to a point where a little steam is rising from the milk. This is to make sure that any bad bacteria are dead. Then you let the milk cool to around 100-110 degrees (the yogurt maker comes with a thermometer for this purpose). Then you mix in a few tablespoons of existing yogurt (the source of good yogurt bacteria). They say you should use plain yogurt here, not flavored; I think this is because of the risk that fruit-flavored yogurts might have extraneous bacteria in the fruit.

Now that it's all mixed up, you pour into the little cups that fit inside the yogurt maker, which simply keeps them at a constant warm temperature. That's it. Several hours later, it has fermented into yogurt.

Total prep time: About 5 minutes of actual work.

One thing I notice is that it is runnier than store-bought yogurt, which is usually thickened with corn starch.

As for the starter, I like to use Stonyfield yogurt, because it is made with L. Reuteri, a bacteria that seems to have several health benefits. See here and here.

One thing that occurs to me, though, is this: How did humans ever figure out how to use only good bacteria in making fermented milk products (yogurt, cheese, etc.)? Yogurt was invented so long ago that no one seems to know. But think about it: A thousand or more years ago, how did people tell yogurt apart from spoiled milk? Ugh.

Maps of Religion

Via Eve Tushnet, a cool collection of maps showing religious adherence throughout the U.S.

Trip 3

Last week, I had lunch with one of my best friends from college, who is going into his tenth year of teaching English literature at a Georgia high school. He's black, as are many of his students. He made a couple of points that I found interesting:

1. No Child Left Behind: He agreed with the ideals behind this law, but he was very stressed out over the fact that his school has to show a certain level of "Average Yearly Progress," or else the federal government can take over the school and demand that the teachers reapply for their jobs.

The problem: Too many students who are arriving in 10th grade English not even knowing how to read, and having been “socially promoted” for their entire lives. My friend said (and I paraphrase from memory):

“I’m doing the best that I know how to do, but you know how reading tests are written: They’re full of little logical tricks to make sure that you read and understood the question. How am I going to get someone who doesn’t even know how to read to pass that kind of test? It’s just impossible for me to spend one or two semesters and get someone caught up on 9 or 10 years of schooling. And then there are always some kids that just don’t care, and no matter what I try, they just won’t do the work. So the government is going to tell me that because of a handful of students that are unreachable, therefore I’m a bad teacher? No way.”


2. The conversation turned to the problems caused by fatherlessness in the black community. My friend told a story that I found very touching. Again, I paraphrase from memory:

“One of my kids just cried like a baby the other day. This was one of the bad kids, always disruptive, bad attitude. He and I were arguing about something the other day, and I said, ‘Let’s take this out in the hall.’

“So we get out there, and he says, ‘You can’t tell me what to do. You’re not my dad.’

“Then I said, ‘No, I am your dad. I’m the only grown male who is willing to stand out here, before God and before anyone else who is listening, and to tell you that I love you and that I’m here for you. Now you tell me, if that’s not a dad, then what is?’

“That’s when the kid just started sobbing.

“Anyway, after that, he was much better behaved in class. But the sad part is, that I’m still probably going to have to fail him. He hasn’t turned in about half of the assignments, and I don’t see how he is going to be able to pass my test.”

Trip 2

At one point, I visited an old college friend who is now a pharmacist. He made a couple of observations that I found interesting:

1. His job is absolutely miserable. Overworked; on his feet for 14 hours a day; constantly dealing with sick and impatient customers who are angry that their prescriptions cost so much or that their insurance didn't cover something.

2. His gut instinct was that Americans use too many prescription drugs, and that their usage would go down if insurance didn't cover so many prescription drugs. He said, for example, that he'll see people come in who are on 15 or 20 meds, half of which are unnecessary for their conditions. But then as soon as he tells them that a particular med isn't covered by their insurance, they always say, "Well, put that one back; I may come back for it later." Then they never come back for it.

Another example of over-expenditures: He filled a prescription for a new and expensive antibiotic that costs upwards of $75, but then the customer's insurance didn't cover it. When he told the customer the cost, the customer demanded that he call the doctor, at which point the doctor said that amoxycillin (a cheap generic) would do as well. This made him wonder why the doctor didn't prescribe the cheaper drug in the first place.

A third point he made was that if insurance didn't cover so many drugs, the drug companies would find a way to lower their prices, simply because they would realize that a particular drug would never sell if people had to pay $100+ out of pocket.

I wonder if there's any way to quantify the amount of money that Americans waste each year precisely because of insurance, which encourages customers to use more expensive drugs more often than necessary, and which encourages drug companies to jack up prices.

Trip

I just got back from a week-long trip to Georgia, where I participated in the University of Georgia Mens' Glee Club Reunion on Saturday (I had helped organize the event over the past year or so). It was enormous fun to see old friends, and to sing together in a concert on Saturday.

A few highlights from the trip:

• We were trying to drive back into Arkansas on Sunday. In the early evening, we started seeing a lot of weird lightning to the north and the south, flashing not to the ground but across the clouds. Then, I started to notice that while the sun had not quite gone down yet (there was a patch of blue in the sky), there was an enormous cloud dead ahead that was pitch black.
  
  We stopped for gas. The wind was the strongest that I have ever felt. My 6-year-old son thought it was hilarious: He had to run in place just to stay in the same place; if he stopped moving, the wind blew him practically off his feet.
  
  Anyway, something didn't feel right. I quickly turned on the radio, and the first station I found had an announcer saying things like, "There has been a tornado that has touched down in west Memphis," and "someone has seen softball-sized hail," and "whatever you do, get out of your vehicle."
  
  Needless to say, I didn't much feel like continuing to drive straight into the pitch-black cloud. So we turned around and backtracked some 25 miles to Jackson, Tennessee, where we spent the night. There were weather sirens going off.
  
  Don't get me wrong: I don't think we were ever in any danger. I never saw an actual tornado, and where we were, all we got was some fierce rain for a while. But here's an article from the local newspaper on the damage done in Tennessee that night.
  
  
  
• Interesting corporate mottos that I saw on the road:
  
  -- On an 18-wheeler heading west: [Acme] Caskets: Dedicated to the Dignity of Life. Life?
  
  -- On a billboard for a regional bank: "[Acme] Bank: Where Banking Comes Together. Together with what?

I have a couple more observations that I'll put in separate posts.