(This is an update of a post I originally wrote on August 21. I’ve included the new DC voucher findings.)
Here is what I believe is a complete (no cherry-picking) list of analyses taking advantage of random-assignment experiments of the effect of vouchers on participants. As I’ve previously written, 9 of the 10 analyses show significant, positive effects for at least some subgroups of students.
All of them have been published in peer reviewed journals or were subject to outside peer review by the federal government.
Four of the 10 studies are independent replications of earlier analyses. Cowen replicates Greene, 2001. Rouse replicates Greene, Peterson, and Du. Barnard, et al replicate Peterson and Howell. And Krueger and Zhu also replicate Peterson and Howell. All of these independent replications (except for Krueger and Zhu) confirm the basic findings of the original analyses by also finding positive effects.
Anyone interested in a more complete discussion of these 10 analyses and why it is important to focus on the random-assignment studies, should read Patrick Wolf’s article in the BYU Law Review that has been reproduced here.
I’m eager to hear how Leo Casey and Eduwonkette, who’ve accused me of cherry-picking the evidence, respond.
-
These 6 studies conclude that all groups of student participants experienced reading or math achievement gains and/or increased likelihood of graduating from high school as a result of vouchers:
Greene, Jay P. 2001. “Vouchers in Charlotte,” Education Matters 1 (2):55-60.
Greene, Jay P., Paul E. Peterson, and Jiangtao Du. 1999. “Effectiveness of School Choice: The Milwaukee Experiment.” Education and Urban Society, 31, January, pp. 190-213.
Wolf, Patrick, Babette Gutmann, Michael Puma, Brian Kisida, Lou Rizzo, Nada Eissa, and Marsha Silverberg. March 2009. Evaluation of the DC Opportunity Scholarship Program: Impacts After Three Years. U.S. Department of Education, Institute of Education Sciences. Washington, DC: U.S. Government Printing Office. (In the fourth year report the sample size shrunk so that the positive achievement effect barely missed meeting a strict threshold for statistical significance — p < .06 just missing the bar of p < .05. But this new report was able for the first time to measure the effect of vouchers on the likelihood that students would graduate high school. As it turns out, vouchers significantly boosted high school graduation rates. As Paul Peterson points out, this suggests that vouchers boosted both achievement and graduation rates in the 4th year. Read the 4th year evaluation here.)
-
These 3 studies conclude that at least one important sub-group of student participants experienced achievement gains from the voucher and no subgroup of students was harmed:
This 1 study concludes that no sub-group of student participants experienced achievement gains from the voucher:
(Update: For a review of systemic effect research — how expanded competition affects achievement in traditional public schools — see here.)
Jay P. Greene's Blog 
It seems to me that random assignment studies are only valid in so far as they measure school choice through the “educational treatment” framework. While it is indeed good news that at least some students tend to do better when their parents get to choose their schools, and none do worse, it is dangerous to rely on this as the foundational justification for school choice.
In fact, when the “higher achievement” argument is made in support of school choice, we find ourselves on the verge of sounding a lot like the education establishment, who believe they should be able to dictate what is best for a child in terms of his or her education. In other words, when we rely solely on the achievement argument, we in essence make the parent’s ability to choose subservient to the ends we desire (higher achievement). That sounds a lot like: “we’ll give the family unit freedom when it meets the state’s end.” Very dangerous.
You make an excellent point, Geoff. I don’t mean to suggest that the case for choice hinges entirely on these results. I’m just trying to clarify what the highest quality research says about this one aspect of school choice — the effect of receiving a voucher on student achievement as measure by standardized tests.
On the other hand, using test scores as evidence helps bolster the philosophical case in favor of freedom. Yes, we support vouchers not just because they raise test scores, but also because we believe in freedom. But why do we believe in freedom? Partly because we believe that freedom works. That’s what this evidence shows. I’ve written on this at greater length here.
Greg, that’s a fair point. But I’m not sure everyone would agree on what it means for freedom to “work” in the realm of education. Does it have to mean higher math and reading scores?
What if it means restoring sphere sovereignty to families of all income levels–i.e. letting the family unit be where decisions regarding education are made rather than the district, state or federal government. Of course when it comes to using public dollars to help restore that role to families, one could argue that there is a need for minimum educational standards (just like people are not allowed use food stamps to live off of twinkies). And that is a debate that involves ideology and evidence. But those standards should be minimal so as to allow maximal deference to the family.
Far too often the burden place on school choice programs by certain policymakers and the media is that they have to result in learning gains that are much higher than any other educational internvention in order to merit support from politicians and the general public. When we rely too heavily on the academic improvement argument, we risk helping to further merit this unfair and unecessary burden. Academic improvement is important to look at, but we should avoid treating it as is the lynchpin of why we should empower lower-income families.
I totally agree with your underlying philosophical concerns, although I think it’s less of a problem on this particular question because the schools themselves (public and private) generally make it a core goal to improve their English and math test scores. So the point here is that freedom helps schools accomplish what they define as their core mission.
That having been said, I have said before and will gladly say again that there are other reasons to support school choice besides just the well-established empirical fact that they improve test scores, so even if there were some doubt about that I would still support them. I would just have one fewer reasons to do so.
“For four decades the public has been encouraged to understand school choice almost exclusively as part of the dogmatics of market ideology; choice, goes the mantra, will make education efficient and raise test scores. The unwashed middle of America has found this message credible but both incomplete and abstract. Quite properly they love the market, but they see what Milton Friedman could not – that education is different from airlines and banks…. Any practical campaign to engage the mind of the American Center must begin by limiting free market arguments to their proper instrumental role. The market cannot be mistaken for the good in itself which is centered in the family and more specifically in the relationship of the child to the parent. The dominant problem in our conversation is not the unsustainibility of a school; it is the unsustainibility of the family where school has forcibly taken the parent’s place.” – John E. Coons, The Role of the Philanthropic Community in Strengthening and Sustaining Faith Based Schools and Developing a Robust Field of Study (Carnegie Conversation on Catholic Education, 2007)
JR,
I’m not making a market argument, or really any argument, about the desirability of school choice. I am simply trying to summarize the high quality research on a particular question. Armed with that information people can make whatever argument for or against school choice they like.
Just in case this isn’t clear: the original motivation for these posts was the calumny promoted by Sock Puppeteer Leo Casey and others that Jay had misrepresented the empirical research. He’s showing that he didn’t.
To quote from Steve Vryhof…
“No wonder the voucher debate is so heated and inconclusive. In some ways, the arguments on both sides miss the point. Proponents of both sides share old, unexamined assumptions. For example, the argument that vouchers can be used to “keep heat on failing public schools” assumes that saving and improving all public schools is the main priority. Emphasis on “quality” and “higher standards,” while important, assumes that test scores measure the only purpose of schooling. Talk of “cost savings” and “market efficiency” assumes the bottom line is the major concern.
[…]
The point is that all these unexamined assumptions need to be brought out in to the open, clarified, and scrutinized. Objections to vouchers and other school choice options must not be allowed to distract us from the deeper, more fundamental, issues surrounding the public school monopoly: (1) the injustice that, in this free society, the state and not the parents determines the objectives of education, and (2) the absurdity of the claim that education can be “neutral” with respect to religion. We should begin by acknowledging that education depends on a community of meaning, that key factors are relationships and a value system, that non-public schools often do it better, and that education goes better when kids are in the schools where they and their parents want them to be.
[…]
Until these deeper, more fundamental issues are faced, unless the debates are recast at a more foundational level, the unending calls for reform simply waste time. Stricter certification requirements for teachers, decentralized decision-making, “feel-good” business-school partnerships, massive infusions of cash from foundations, higher standards and more frequent tests, and so on, and so on – all miss the most important point. As Candace Allen, the 1989 Colorado Enterprising Teacher of the Year, put it, “It’s time to lay down the weapons of reform… The more we reform, the more we bind ourselves to the system. What we need, simply, is liberty.”
– Steven C. Vryhof, Between Memory and Vision: The Case for Faith-Based Schooling, pg. 149-150
Just wanted to make sure that folks know that I am not challenging the research findings that Jay has presented. I think the evidence is clear. And it may even be sufficient to convince some people to support vouchers.
But if that is their only reason for supporting vouchers, then what would be their reason for not shifting their alliegance to another “prescription” once it has been shown to produce greater learning gains?
We have to make sure we are publicly making the case that parental choice is fundamental to who we are as a Democracy and not simply a means to helping the USA compete globally.
So again, I am not challenging the evidence that vouchers have positive impacts on student achievement. I am simply trying put the evidence in perspective.
To play devil’s advocate to Geoff, perhaps middle America sees what Freidman could not because middle America is more adept at seeing what is not there.
And to answer the question in the last post, about the reason for not shifting, the best answer is that there must be a method to give schools cause to use the best practices, and market incentives are a great method.
[…] The Left, Pro-Educational Choice? March 22, 2010 Candace de Russy Leave a comment Go to comments Now here’s a novel insight: Greg Forster, a senior fellow at the Friedman Foundation for Educational Choice, in making the case that school vouchers deliver substantially more educational improvement than charters, argues: […]
[…] the evidence is consistent and clear that private school choice, markets in education, work. And private school choice even […]
[…] the evidence is consistent and clear that private school choice, markets in education, work. And private school choice even […]
A number of our products are exclusively manufactured for
Essential Beauty. When you are planning to franchise your business always ensure that you tie-up with a
professional consultant who can help you in developing the program and the whole structure.
It can be if you are looking at the bottom line financial
figures, but you can’t look at just the numbers, you have
to see and know what that money is buying you.
Oh my goodness! Awesome article dude! Thank you, However I am experiencing troubles with your RSS.
I don’t understand why I am unable to join it. Is there anyone else
getting identical RSS issues? Anyone that knows the answer will you kindly respond?
Thanks!!
gaming laptops and okulista dzieciecy szczecin slowackiego.
gaming laptops