Healthily peppered throughout the book are answers to its subtitle, “Why So Many Predictions Fail — but Some Don’t”: we are fooled into thinking that random patterns are meaningful; we build models that are far more sensitive to our initial assumptions than we realize; we make approximations that are cruder than we realize; we focus on what is easiest to measure rather than on what is important; we are overconfident; we build models that rely too heavily on statistics, without enough theoretical understanding; and we unconsciously let biases based on expectation or self-interest affect our analysis.It struck me that this is a pretty good description of the "science" of education (and teacher) evaluation espoused by contemporary "reformers" (those who Debbie Meier calls "deformers"): models sensitive to assumptions, crude approximations, measuring what can be measured rather than what is important, basing models on self-interest. Silver's point is that it is very, very difficult to distinguish the signal from the noise. A little humility is in order ...
Wednesday, October 24, 2012
The Signal and the Noise in Ed Reform
Saw this in Leonard Mlodinow's review of Nate Silver's new book The Signal and the Noise (New York Times on-line this morning):