(Guest Post by Matthew Ladner)
In the American film classic Animal House there is a scene where students smoke marijuana with their English professor, played by Donald Sutherland, and speculate that it could be the case that the molecules in your fingernails each contain a microscopic universe.
You can’t prove that there aren’t microscopic universes in your fingernails, after all, so they might be in there!
A nice post from Mike Petrilli on the Florida NAEP score gains prompted a response from Kevin Welner from NEPC that shows that the spirit of Sutherland’s Professor Dave Jennings is alive and well at the University of Colorado.
Again there is no attempt to address any of the gaping holes in retention theory. These holes include the fact that Florida’s 4th grade reading scores had improved substantially before the retention policy went into effect, and that they have continued to rise even as retention has fallen off substantially, and that they have fallen off substantially because of a very large improvement in 3rd grade scores.
Welner attempts to tiptoe around this by noting that our EdNext article addressing these points were addressed to a previous Walter Haney paper on the subject rather than the NEPC stuff, which is a distinction without much of a difference. The Chatterji paper contains a carbon copy of the Haney analysis. Amazingly, Chatterji dinged Burke and I for not doing a literature review (not the norm in our tribe) and then cites neither the Education Next paper nor Haney’s analysis. At best, she employed a double standard and at worst, she owes Professor Haney an apology.
Welner’s broader project is to attempt to use the causation problem as a shield. We don’t know, after all, exactly what caused Florida’s remarkable learning gains. Florida’s reformers had to implement their reforms in the real world rather than in a petri dish or in an Intention to Treat Random Assignment study. Welner believes that this allows him the opportunity for strategic nihilism:
The truth might be: (a) there are not actual improvements (the current study is too weak to say whether or not there are), (b) there are improvements, and they’re caused by a combination of all these things, (c) there are improvements, and they’re caused by something none of us pointed to (perhaps the green shirts??), or (d) there are improvements, and they’re caused by some of the things we’re pointing to BUT some of the other things we’re pointing to are actually harming students (just not enough harm to overcome the benefits of the other things).
In other words, when it comes to understanding the FL package of reforms, we are flying blind.
Welner is flying blind all right, but it is by choice. Let’s take each of these little gems on one at a time:
A. The NAEP results show very substantial improvements, as do other indicators.
B. I have always held that the exact cause for the improvement is impossible to know, because Florida’s reformers enacted multiple reforms simultaneously. The logical response to this is not to do none of the Florida reforms, but to do all of them.
C. Florida lurked near the bottom on NAEP for many years, enacted reforms in 1999, and then enjoyed sustained gains over time. While it could be the case that some mysterious X-factor caused the improvement, I’ve yet to hear a plausible theory regarding this. Dan Lips and I addressed multiple possibilities in the Education Next article, including demographic change, spending, etc, and found no evidence to support them.
D. This could be the case, but I haven’t seen a single scrap of evidence to suggest that it is actually the case- return to B above.
Welner is of course correct that there is a correlation and causation problem to consider. As a practical matter, there is nothing else to do but to carefully examine the evidence and history and draw the best conclusions that we can. Dan Lips and I did this in the Education Next article. Florida’s reforms coincided with the student population becoming poorer and less Anglo. State lawmakers increased funding per pupil, but it wasn’t by much and is still below the national average. NEPC complains about a lack of mention of the preschool voucher program when those kids have yet to age into the 4th grade NAEP sample. The class size amendment was implemented very slowly, long after Florida’s scores had begun to rise.
If Dr. Welner would like to provide a plausible explanation for why Florida’s NAEP scores increased so much after 1998, I’d be very interested to read it.
If he prefers to attempt to continue to play games, NEPC’s credibility will go on double secret probation.
Jay P. Greene's Blog 
Hey, Matt, what’s so funny ’bout peace, love and strategic nihilism?
My reply to Dr. Ladner is posted here:
http://nepc.colorado.edu/blog/reply-matthew-ladner
“NEPC’s credibility will go on double-secret probation.” Where have you been, Matt? NEPC’s credibility left the building a long, long time ago.
The arguments that Professor Welner offers against the straightforward implication of Florida’s success deserve consideration, and rejection. Legislators who weigh arguments with aggregate social welfare in mind will not hold arguments for alternative providers of education services to a higher standard of evidence than they hold arguments for the NEA/AFT/AFSCME cartel’s schools.
Every law on the books is a threat by a government to kidnap (arrest), assault (subdue), and forcibly infect with HIV (imprison) someone, under specified circumstances. In a compassionate society the burden of proof would fall on advocates for organized violence in the form of compulsory attendance (truancy) statutes, applied to students, compulsory education (educational neglect), statutes, applied to parents, compulsory support of school (taxation), and the policy that gives to schools operated by the BEA/AFT/AFSCME cartel an exclusive position in receipt of the taxpayers’ K-12 education subsidy. Instead, Professor Welner argues as though school choice supporters must make an air-tight case while cartel defenders barely have to make a case at all. I read one of Professor Welner’s publications, an argument against tuition tax credits. Professor Welner deploys “equity” as though this undefined term settles the argument. He does not hold his arguments to the same stardard .of proof that he requires of arguments for options outside the cartel’s schools.
Neither do I. The default poaition is freedom.