The Signal and the Noise in Ed Reform

Saw this in Leonard Mlodinow's review of Nate Silver's new book The Signal and the Noise (New York Times on-line this morning):

Healthily peppered throughout the book are answers to its subtitle, “Why So Many Predictions Fail — but Some Don’t”: we are fooled into thinking that random patterns are meaningful; we build models that are far more sensitive to our initial assumptions than we realize; we make approximations that are cruder than we realize; we focus on what is easiest to measure rather than on what is important; we are overconfident; we build models that rely too heavily on statistics, without enough theoretical understanding; and we unconsciously let biases based on expectation or self-interest affect our analysis.

It struck me that this is a pretty good description of the "science" of education (and teacher) evaluation espoused by contemporary "reformers" (those who Debbie Meier calls "deformers"): models sensitive to assumptions, crude approximations, measuring what can be measured rather than what is important, basing models on self-interest. Silver's point is that it is very, very difficult to distinguish the signal from the noise. A little humility is in order ...

Race to the Top: Is the AFT Losing the Race?

Steven Brill, in a comprehensive article in the New York Times, details the impact that Race to the Top is having on education legislation at the state level.

As states compete for portions of the $4.3 billion dollar Race fund, legislatures are passing new laws eliminating or reducing teacher tenure and mandating inclusion of test score results on teacher compensation, in order to conform with the demands of the Race.

Gradually, the national AFT and the local unions are loosening their long-held demands regarding both issues. If the states have not achieved the Race goals, they must submit memoranda of understanding (MOUs) in their Race proposals regarding agreements to achieve them. At this point some of the states claim to achieve these goals, but the MOUs include clauses such as "as consistent with state laws" that in effect nullify them. Nonetheless, the pressure on the unions is severe.

Brill appears to me to be a cheerleader for the Reformers, a tightly organized group of political and business elites. So the article must be read with some caution.

Must we obsess about student test scores?

This is a cross-post from Chip's Journey. It connects with Amy Shuffelton's very interesting post below about testing children. I share her daughter's evaluation of the whole process.

---

For too long, US education policies have defined progress in terms of student test scores, while ignoring the things that really matter. We've operated on the misguided belief that "learning the basics" is best accomplished by a narrow skills focus and micro-management of test scores.

This occurs despite the fact that few of us would be satisfied if our children could successfully answer multiple-choice questions, but failed to develop intellectual curiosity, cultural awareness, practical skills, a philosophy of life, a strong moral character, emotional balance, social fitness, sensitivity to social problems, or physical fitness. What a tragedy then, if the focus on skills per se (as with the failed No Child Left Behind Act) were not even necessary. What if one could help the whole child develop, including teaching basic skills? What if our current irrational obsession with testing actually stood in the way of the things we truly value?

The start of a new administration in Washington is a good time to ask whether we have the schools we need. Above all, it's not a time to seek ever-more efficient means to produce incremental gains in test scores.

We have an alternative to that in our own history. One of the best program evaluation studies ever conducted was the Eight-Year Study, research conducted between 1932 to 1940 by the Progressive Education Association (PEA). Thirty high schools participated. Instead of narrowly-defined subjects, there were broad themes of significance to the students. "The starting point of the curriculum would be life as the student saw it" (Benedict, 1947, p. 14). Moreover, the schools were community-based. "The schools believed they belonged to the citizens of the community" (ibid, p. 17).

The students from the experimental schools did only slightly better on standardized test scores, but they showed major improvement in other areas, including intellectual competence, cultural development, practical competence, philosophy of life, character traits, emotional balance, social fitness, sensitivity to social problems, and physical fitness. Students from the most progressive schools showed the most improvement, more than those in the somewhat-progressive schools, and much more than those in traditional schools.

Outcomes of the study included better forms of student assessment, innovative research techniques, new ideas for curriculum, instruction, and teacher education. But above all, it provided an answer to the questions above: It is possible to help the whole child develop, without losing basic skills. In fact, schooling can be conceived in such a way that teachers and community members are learners as well. Doing that appears to be the best way to help the individual learner, not drilling on perceived deficits, as we do now. And yes, the irrational obsession with testing actually stands in the way of teaching the things almost every parent, teacher, or citizen truly value.

References

Aikin, Wilford (1942). The story of the eight-year study. New York: Harper.

Benedict, Agnes E. (1947). Dare our secondary schools face the atomic age? . New York: Hinds, Hayden & Eldredge.