Selection Effects
Homeschoolers regularly achieve average test scores that are substantially higher than the average scores from public school students.
On occasion, you'll see this result chalked up to selection effects: Parents who choose to homeschool in the first place tend to be just the sort of parents who are intensely devoted to their children's education. For example, this article:
Imagine that you had two identical public schools in a given town -- identical in every way. For some reason, parents have gotten the notion that School A is better than School B (not true, because both are identical). Nonetheless, the parents who are most intensely interested in academics all do their best to move into the district for School A. As a result, School A's test scores rise above School B's -- not because it does anything differently, but solely because academically-oriented people selected to enroll in that particular school.
What does this tell us? If we know that selection effects are really the cause, then there really isn't anything about School A that is superior. Because of that, if you were randomly assigning students to schools, there would be no reason to expect the students in School A to do any better. And for the same reason, if you were deciding where to send your own kid to school, it would be worth knowing that School A looks better only because of the pre-existing qualities of the kids who choose to go there, rather than anything about School A itself.1
But all of this presupposes that it is possible to tease apart the selection effect from the thing that you're trying to measure. Put another way, the mere act of selecting School A does not increase academic performance. If people who are already academically superior -- for independent reasons -- select School A, that act of selection gives rise to the artificial statistical superiority of School A, but that's all. The selection has no more meaning than that. Selection can be separated from school performance.
Can the same be said of homeschooling? I'm not so sure. For one thing, homeschooling is not, and never could be, random. It's not as if anyone could ever assign students randomly to be homeschooled, to see what the effects of homeschooling would be in those families that have no interest in education.
Conversely, it's always going to be the case that -- except for a few negligent people who use homeschooling as an excuse for truancy -- homeschoolers take an intense interest in their children's education.
In other words, you can't really separate the act of selecting homeschooling from the intensity of interest in education. To the contrary, the very fact that someone selects that option is a sign of intense interest in education. It's not as easy to say, "These are all people who were already independently superior at education, and the fact that they selected homeschooling does nothing more than create a statistical artifact." To the contrary, the very act of selecting homeschooling could show a child that her parents take a genuine interest in her education, and could thus be one of the things that makes her more academically successful.
More generally: Watch out for so-called "selection effects" where the very act of selecting a particular activity could be beneficial in itself.
1With an individual decision, of course, the picture is a bit more complicated. If the student body at School A is superior, it might be worth trying to send your kid there simply for the superior peer effects.
On occasion, you'll see this result chalked up to selection effects: Parents who choose to homeschool in the first place tend to be just the sort of parents who are intensely devoted to their children's education. For example, this article:
To complicate all these issues, the data on home schooling are very limited. By definition, parents who home school take a more active role in their children’s education. It may very well be the case that these kids would succeed in most any academic environment, precisely because their parents have made education a family priority. It is thus difficult to come to definite conclusions about home schooling as an educational method.Something about this argument bothers me, and I've been mulling it over, trying to put it into words. Let me try coming at it in a roundabout way.
My analysis of college admissions scores and acceptance rates suffers from the same selection effect, but perhaps to a greater degree. Here I am focusing not just on home schoolers, but on only those home schoolers bound for college. When I look at college admission tests, I am limiting the sample to those home schoolers who value higher education. The stories of superior academic achievement, higher test scores, and campus success may be less a function of home schooling than an indication of students who take their academic careers very seriously.
Imagine that you had two identical public schools in a given town -- identical in every way. For some reason, parents have gotten the notion that School A is better than School B (not true, because both are identical). Nonetheless, the parents who are most intensely interested in academics all do their best to move into the district for School A. As a result, School A's test scores rise above School B's -- not because it does anything differently, but solely because academically-oriented people selected to enroll in that particular school.
What does this tell us? If we know that selection effects are really the cause, then there really isn't anything about School A that is superior. Because of that, if you were randomly assigning students to schools, there would be no reason to expect the students in School A to do any better. And for the same reason, if you were deciding where to send your own kid to school, it would be worth knowing that School A looks better only because of the pre-existing qualities of the kids who choose to go there, rather than anything about School A itself.1
But all of this presupposes that it is possible to tease apart the selection effect from the thing that you're trying to measure. Put another way, the mere act of selecting School A does not increase academic performance. If people who are already academically superior -- for independent reasons -- select School A, that act of selection gives rise to the artificial statistical superiority of School A, but that's all. The selection has no more meaning than that. Selection can be separated from school performance.
Can the same be said of homeschooling? I'm not so sure. For one thing, homeschooling is not, and never could be, random. It's not as if anyone could ever assign students randomly to be homeschooled, to see what the effects of homeschooling would be in those families that have no interest in education.
Conversely, it's always going to be the case that -- except for a few negligent people who use homeschooling as an excuse for truancy -- homeschoolers take an intense interest in their children's education.
In other words, you can't really separate the act of selecting homeschooling from the intensity of interest in education. To the contrary, the very fact that someone selects that option is a sign of intense interest in education. It's not as easy to say, "These are all people who were already independently superior at education, and the fact that they selected homeschooling does nothing more than create a statistical artifact." To the contrary, the very act of selecting homeschooling could show a child that her parents take a genuine interest in her education, and could thus be one of the things that makes her more academically successful.
More generally: Watch out for so-called "selection effects" where the very act of selecting a particular activity could be beneficial in itself.
1With an individual decision, of course, the picture is a bit more complicated. If the student body at School A is superior, it might be worth trying to send your kid there simply for the superior peer effects.
Labels: education
posted by Stuart Buck at
11:12 AM
1 Comments:
Of course, even if schools A and B started out absolutely identical, and retain the same facilities and staff, the very fact that school A is now filled with students being pressured to succeed by their parents, and school B by students whose parents are indifferent to academics MAKES school A the better learning enviroment.
Post a Comment
Subscribe to Post Comments [Atom]
<< Home