Value-Added Has Errors . . . Which Means What?

A recent New York Times article profiled a dedicated and well-reviewed middle school teacher who may not get tenure thanks to the New York City value-added model that places her at the 7th percentile (judging by how much growth her students showed on tests).

Assuming that the article is correct, and that the value-added model here is being unfairly applied for any number of reasons, that means that value-added models can make Type II errors (false negatives).

OK, so we should ditch value-added models and keep on with the "last in, first out" system in which the newest teachers get laid off first (in the case of layoffs). No messing around with incomprehensible math that could be wrong.

But wait, the "last in, first out" model can make Type II errors too. Look at what happened last year in Wisconsin, where the zero-deductible health insurance plan was so expensive (note: this is one reason why Governor Walker wanted to strip collective bargaining over such benefits) that it required Milwaukee to lay off some new teachers acknowledged as good:

Megan Sampson was named outstanding first-year teacher by the Wisconsin Council of Teachers of English last week.

Second-year social studies teacher Kevin Condon, also at Bradley Tech High School, has four licenses and can command the attention of 40 students in an open-concept classroom.

Both are among 482 educators - more than 12% of the full-time teachers in the district - who have received layoff notices from Milwaukee Public Schools.

On Monday - the last day of the year for schools in MPS and the first day teachers reunited after hearing the news of the layoffs - some teachers expressed frustration at losing their jobs because of experience, not performance. . . .

Meanwhile, the Milwaukee School Board president and the Milwaukee Teachers' Education Association leadership continued to disagree on how to handle the tightened $1.3 billion district budget, and whether teachers should accept a lower-cost health-care plan to avoid layoffs. . . .

Bonds said if all teachers switched to the lower-cost plan, about $48 million could be saved, enough to pay for 480 educators.

"I'm not aware of any place in the nation that pays 100% of teachers' health-care benefits and doesn't require a contribution from those who choose to take a more expensive plan," Bonds said.
. . .

Sampson and her laid-off colleagues, all who have less than three years of teaching experience, also expressed frustration that their jobs would be filled by more veteran, but not necessarily better, educators.

Sampson and Emily Kaphaem, a world geography and citizenship teacher at Tech, said they have received exemplary performance reviews.

"I feel kind of let down by my city today," said Kaphaem, 25, as she lost the fight to hold back tears in Principal Ed Kupka's office.

Kupka is equally frustrated. He hand-selected the new teachers because of their talent and enthusiasm for turning around Tech, recently designated as one of the worst-performing high schools in the state.

"Based on the pressures we're under as a low-performing school, I absolutely would have chosen a different nine (for layoffs)," Kupka said.

So now what? Well, for one thing, it's silly for ideological advocates on either side to act as if they have a perfect system that will preserve the best teachers, either by 1) improperly equating quality with years of experience or 2) improperly assuming that each teacher can be given an exact number that represents her quality. It's equally silly for ideological advocates to act as if they've proven the opposite side's system to be some horrible inequity just because it results in an occasional good teacher getting fired. Unless God is running the system, there are going to be errors.

What we need to know is which system makes more errors. But what constitutes an "error"? One way to think about this would be to assume, for the sake of argument, that teachers really are good teachers when they have received outstanding (not merely adequate) performance reviews from their principals. So then, if layoffs are going to occur, would more teachers deemed outstanding by their principals be laid off under 1) a value-added model, or 2) the "last in, first out" standard?