Testing Requirements Hurt Choice

In this week’s debate over the wisdom of requiring choice students to take state tests, three points deserve greater emphasis.  First, testing requirements hurt choice because test results fail to capture most of the benefits produced by choice schools.  As Collin Hitt’s piece persuasively argued, a series of rigorous studies have found large long-term benefits for students able to attend schools of choice even when short-term test results show little or no benefit.  Those studies show that charter and private choice schools cause students to graduate high school and go to college at much higher rates.  Those students go to more competitive universities at much higher rates.  And choice causes those students to enjoy much higher salaries later in life.  But if you only looked at short-term test results for these students you would not have expected the magnitude of these benefits.

One (of the many) problems with imposing testing requirements on schools of choice is that it highlights a measure of performance that grossly under-states the benefits of choice.  Given the precarious political position of choice programs, highlighting a measure that severely under-states performance puts those programs in jeopardy.  I can understand why choice opponents favor testing requirements — since they want ammunition to shut choice down or regulate it into oblivion.  But why would choice supporters favor this?  It’s a huge mistake.

Second, the only piece of evidence that Fordham presents to support the claim that state testing requirements improve performance at choice schools is the finding that scores rose when Milwaukee private choice schools were required to take the high stakes state test.  But as Pat Wolf, one of the authors of that study, noted — the score increase may well be just an artifact of private choice schools deciding to start prepping students for that high-stakes test now that they were required to take it.  In other words, Fordham is confusing real learning increases with test manipulation.  Pat was gently warning Fordham not to misinterpret the results in this way.  Despite that warning, Fordham continues to mis-use this research to make their point.

If Fordham continues to incorrectly cite this bit of evidence to support their point, they are in danger of becoming the Diane Ravitch of think tanks.  Wolf similarly warned Ravitch that she misunderstood the graduation rate component of the Milwaukee voucher study, wrongly claiming that attrition was biasing results to show higher graduation rates for voucher students.  Ravitch did not grasp that the result was based on an intention-to-treat analysis and, if anything, that type of analysis under-states the positive effect of choice on graduation rates.  Ravitch either couldn’t understand this or didn’t care about getting it right, so she continues to repeat this incorrect interpretation of the research to advance her agenda.  In the argument about choice and state testing requirements, Fordham is similarly repeating a faulty interpretation that they’ve been warned is mistaken.

Third, despite the lack of evidence that state testing requirements improve outcomes or ensure quality (as they largely acknowledge in an earlier report, “The Proficiency Illusion”), Mike Petrilli continues to push for them because… well, because we’ve got to do something:

Bad schools happen. They happen in the public sector, the charter sector, and, yes, the private sector. And since education is a “public good” as well as a “private good”—because kids’ lives literally hang in the balance and so does the future of the society whose taxpayers are underwriting these costs—we can’t just look the other way….

But the answer cannot be “let the market figure it out.” Because it hasn’t, and it won’t—and somebody must.

Of course, doing something that is ineffective or counter-productive may be worse than doing nothing.  If state testing requirements don’t necessarily make schools better and fail to capture the bulk of the benefits choice schools are producing, then imposing state testing requirements on choice schools just to do something is a really bad idea.  In an effort to prevent all bad things from happening, Fordham may ensure that more bad things will happen.

Fordham’s argument that we need to do something reminds me of the brilliant song Jason Segel wrote for the fictional band, Aldous Snow and the Infant Sorrow, in the movie Forgetting Sarah Marshall — appropriately titled “We’ve Got to Do Something!”  As he is grabbing a cane from a blind man in the music video Aldous (Russell Brand) sings:

You gotta do something,
We gotta do something,
Sometimes I sit in my room and I don’t know what to do,
but we’ve gotta do something!

…and if I was in Government,
Then I’d Government things much more differentlier,
because it ain’t the best way to government things,



UPDATE — On reflection, the Ravitch comparison was too harsh.  Ravitch repeated a factual error even after the error was pointed out to her.  Fordham is repeating an ambiguous finding, not necessarily a factually incorrect one.  One could interpret the test score gain produced by choice schools in Milwaukee after high-stakes testing was required as a test prep artifact or as a real learning gain.  I’m strongly inclined toward the test prep explanation , but the other interpretation is not factually mistaken.  Both the original study and Pat’s post warned Fordham about the ambiguity of this result, yet Fordham continues to cite it as proof without clarification or qualification.  It’s not pulling a Ravitch, but it’s also not good.

10 Responses to Testing Requirements Hurt Choice

  1. […] Jason Bedrick of the Cato Institute’s school reform team, University of Arkansas’ Jay P. Greene, the otherwise-sensible Robert Enlow and Greg Forster at the Milton & Rose D. Friedman […]

  2. Russ Walsh says:

    So, Jay, if I understand your thinking correctly, choice schools should be exempt from standardized testing because these tests give a false picture of all the good things that choice schools offer. I couldn’t agree more. Standardized tests can not capture all that goes on in any school, choice or no choice. If we follow this thinking to the logical conclusion, I would have to assume that choice schools would also be exempt from Value Added Measures (VAMs) of teacher performance. No test scores, no VAM. So administrators would have to rely only on observation to determine teacher effectiveness. But wait, isn’t that one of the key complaints of choice advocates aboout public schools? That the public schools fail to weed out the “bad” teachers. Are there no “bad” teachers in the choice schools? I guess since they are showing such phenomenal gains that they can’t have any bad teachers.

    I would also have to believe that (again following your logic) there could be no closure of “choice” schools, because there would be no hard, standardized test measure of their success or failure.

    So as I see it, Jay, you are advocating for a dual educational system. One for choice schools and one for no chice schools. It seems that you would like the choice schools to also have choice in how they document their success. I suppose those no choice schools don’t deserve the same consideration.

  3. […] P. Greene said this: “test results fail to capture most of the benefits produced by choice […]

  4. […] P. Greene said this: “test results fail to capture most of the benefits produced by choice […]

  5. […] thus, as chair and head of the department we have Greene, lamenting the negative consequences of high-stakes testing on the prospects of expanding the school choice […]

  6. JIM says:

    If a few school reformers want public schools to live and die by the sword of testing, and schools of choice want public funds, then reformers ought to operate under equal accountability measures. What’s going to happen, is once these school reformers finish off public schools in a couple of decades, schools of choice will not have to worry about teachers’ unions and due process – just like the Walton’s want it. What a sham that an unelected crew of the Waltons, Bill Gates, and Eli Broad are allowed to legally dictate how we educate our children by trumping the locally driven democratic process – our votes are meaningless as the billionaires carry more weight.

    And BTW, Paul Thomas routed over on his blog.

  7. […] instance, choice enthusiast Jay P. Greene, who heads a Department of Education Reform at the University of Arkansas, recently wrote on his […]

  8. On voit direϲtement quе vous maîtrisez superbement bien
    ce sujet

  9. email marketing companies Directory

    Testing Requirements Hurt Choice | Jay P. Greene's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: