The Signal and the Noise in Ed Reform

Saw this in Leonard Mlodinow's review of Nate Silver's new book The Signal and the Noise (New York Times on-line this morning):

Healthily peppered throughout the book are answers to its subtitle, “Why So Many Predictions Fail — but Some Don’t”: we are fooled into thinking that random patterns are meaningful; we build models that are far more sensitive to our initial assumptions than we realize; we make approximations that are cruder than we realize; we focus on what is easiest to measure rather than on what is important; we are overconfident; we build models that rely too heavily on statistics, without enough theoretical understanding; and we unconsciously let biases based on expectation or self-interest affect our analysis.

It struck me that this is a pretty good description of the "science" of education (and teacher) evaluation espoused by contemporary "reformers" (those who Debbie Meier calls "deformers"): models sensitive to assumptions, crude approximations, measuring what can be measured rather than what is important, basing models on self-interest. Silver's point is that it is very, very difficult to distinguish the signal from the noise. A little humility is in order ...