Russ Whitehurst Takes on the Finland Du Jour

I’ve been railing against education policy arguments based on identifying places with successful outcomes and then claiming we should imitate certain practices or policy features of those places to achieve success elsewhere.  I argued forcefully against it in my review of Marc Tucker’s book.  Anytime someone says Finland (or Massachusetts or whoever) is doing well because they have high standards or little choice or no school athletics, or whatever, they are just engaging in quackery — pure BS.

It is a matter of basic logic that one cannot know what causes success only by looking at a successful place (or set of successful places).  You cannot know whether any factor contributes to success without also considering unsuccessful places and examining whether that same factor tends to be more present or absent in successful relative to unsuccessful places.  This is an error known as “selection on dependent variable” and it is taught in any decent introduction to research methods course.

People who regularly draw policy recommendations based on the Finland du jour should be made to hold up a giant sign that says “I do not understand the basic research methods of the field in which I claim to be an expert.”  And that’s just it.  These people aren’t really experts.  There used to be a time when a clever writer could be considered an expert on education policy simply by virtue of articulating a clear-sounding argument.  That time is gone.

Even journalists should be expected to have some basic understanding of the methods in education policy research.  No one would accept that science reporters could be ignorant of fundamental principles of the scientific method.  No one would accept that diplomatic correspondents would have no knowledge of diplomatic history.  Even journalists — especially when they are writing book-length arguments — need to have some understanding of research methods that would include the obvious point that no causal claims can be made from selection on dependent variable analyses.

So, I was delighted to see the Brookings Institution’s Russ Whitehurst take on David Kirp’s book, Improbable Scholars: The Rebirth of a Great American School System and a Strategy for America’s Schools, which draws lessons from an examination of one “successful” district — Union City, NJ.  Whitehurst begins by illustrating why selection on dependent variable does not allow for causal claims:

A while back, I read a journalistic account of a small island off Italy in which people live to be quite old.  Could it be because they sleep late, or drink lots of wine, or live a communal existence, or don’t eat refined sugar?  Unfortunately there is no way to know based on the information provided in the article if it is one or more of the lifestyle characteristics the author identified as distinctive, or whether something else is going on.  Even the claim of unusual longevity is questionable since there is no birth registry. And taking everything at face value, maybe there are other places in Italy in which people live as long or longer with different lifestyles.

Whitehurst goes on to question whether Union City is really such a successful district.  And then he applies the problems of his Italy example to Kirp’s use of Union City to make causal conclusions about how school improvement can be achieved.  Lastly, Whitehurst concludes with:

Once we have valid descriptions of the distinctive operational differences between good and not-so-good schools, controlling for differences in student background and out-of-school factors that are beyond district control, the social science of district reform can move to planned and carefully evaluated interventions.  That is our playbook. That is where we need to be.  I don’t think the path goes through Union City.

Amen, Brother Russ.

Advertisements

6 Responses to Russ Whitehurst Takes on the Finland Du Jour

  1. Greg Forster says:

    Small correction: they should be made to wear a sign the reads “I do no know or deliberately choose to ignore the most basic principles of the field in which I claim to be an expert.”

    The deeper problem here is that the credibility of science has been so debauched by technocrats that people no longer think it’s worth keeping track of what’s scientific and what’s not.

  2. James Shuls says:

    You know, my college roommate was from Finland. He was one of the top javelin throwers in the country and a true scholar athlete. This is clearly evidence that Finland is doing things right. We should learn from them. Maybe it has something to do with the fact that they drink lots of wodka(vodka) and coffee.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s