(Guest Post by Jason Richwine)
The new experimental evaluation of the Louisiana voucher program poses a challenge to school-choice advocates such as myself. How do we explain the voucher students’ negative test outcomes – including a massive 0.4 SD drop in math scores – when evaluations in other cities showed neutral to mildly positive effects? Supporters have quickly coalesced around the explanation that Louisiana imposed regulations so suffocating that only the worst private schools participated in the voucher program.
I find that explanation unsatisfactory for a few reasons. First, it feels like a post-hoc excuse. Yes, choice advocates have been warning about burdensome regulation for years, including in Louisiana, but how many predicted that the state’s voucher system would go down in flames because of it? The magnitude of the score declines must surprise even the most vociferous critics of regulation.
More importantly, if the participating private schools are so bad – and other people apparently knew they were bad, given the declining enrollments – then why did the voucher recipients choose them? Did the parents fail to research their options? Do they not value academics much at all? Blaming the results on an unusually bad set of private schools is tempting, but it creates the new problem of having to explain why parents made such dubious choices.
Personally, I do not find it plausible that school quality alone could have so much impact, especially in one year. The traits that students bring with them to school – natural abilities, resilience, family support networks – generally explain much more of the variance in student achievement than school quality. Only the absolute worst schools could have such deleterious effects, but there is no indication that the Louisiana voucher schools were the bottom of the barrel. Even if the one third of private schools that participated really were the worst third in the state, we are still talking about schools that are below average – not uniformly awful.
In trying to reconcile Louisiana with the successful experiments in DC, Milwaukee, Charlotte, etc., I suggest exploring other explanations. In particular, how well did the private schools align their curricula with the demands of the state tests? Maybe the private schools were simply teaching different material rather than teaching the state’s curriculum badly. Also worth examining is whether the randomization process, which was done within a complicated set of priority levels for admission, was conducted appropriately. Another issue is how schools adapt after the first year of statewide implementation. And, remember, it is not uncommon for studies to change significantly from the working paper phase to publication. So let’s be patient. Explaining this anomalous study will require more research.
Jay P. Greene's Blog 
Jason, you write: “First, it feels like a post-hoc excuse. Yes, choice advocates have been warning about burdensome regulation for years, including in Louisiana, but how many predicted that the state’s voucher system would go down in flames because of it?”
See here: http://newsok.com/article/5465998
“The only schools who are willing to do whatever the state tells them they must do are the schools that are most desperate for money,” Greene said. “If you don’t have enough kids in your private school and your finances are in bad shape, you’re in danger of closing — probably because you’re not very good — then you’re willing to do whatever the state says.”
And here: https://www.redefinedonline.org/2015/11/testing-undermine-educational-choice/
“Not only does overregulation limit the educational options available to low-income students, it may even reduce the quality of the options available. Better quality private schools that have little trouble filling their seats are less likely to accept vouchers if they decide — as two-thirds of Louisiana’s private schools did — that the regulations are too burdensome. By impeding the proper functioning of the market, regulations intended to raise quality may have the unintended consequence of lowering it.”
And here: http://www.cato.org/blog/year-educational-choice-update-vi
“Educational choice advocates can issue three cheers for the court’s decision, but the voucher program itself deserves only one. Unfortunately, the program is hobbled by excessive regulations that are intended to guarantee quality but might actually be undermining it. For example, the state requires private schools that accept vouchers to administer state test, which drives what schools teach and even how and when they teach it.”
And here: http://www.nationalaffairs.com/publications/detail/the-next-step-in-school-choice
“Beyond basic health and safety regulations, top-down accountability measures are generally unnecessary at best and harmful at worst. Centralized standards, especially in the form of state testing mandates, induce conformity that can undermine the innovation and diversity that give educational choice its value. “
Jason also writes: “I suggest exploring other explanations. In particular, how well did the private schools align their curricula with the demands of the state tests? Maybe the private schools were simply teaching different material rather than teaching the state’s curriculum badly.”
I do find this plausible. Schools that were not teaching the state standards may have accepted the vouchers before being able to fully switch over to the state standards. That said, most of those schools were Catholic schools that (as I understand, though I could be wrong) were already teaching the state standards. Moreover, the fact that the average private school that accepted voucher students was losing enrollment whereas the average school that chose to participate was gaining enrollment lends weight to the “desperate, low-performing schools” theory.
And Jason further writes: “Also worth examining is whether the randomization process, which was done within a complicated set of priority levels for admission, was conducted appropriately. Another issue is how schools adapt after the first year of statewide implementation.”
I find the poor-randomization theory less plausible, though I agree that we should see if these results are replicated going forward before we draw any hard conclusions.
“Personally, I do not find it plausible that school quality alone could have so much impact, especially in one year.”
Do you apply this same logic to public school test scores? Should public schools and/or public school teachers be held accountable for their test score changes year to year?
I agree that we should search for other explanations and we certainly should gather more evidence- starting with year two results.
At this point however we have the following data points- an unusually low private school participation rate, a negative enrollment trend for the schools that did participate, and the first random assignment study of voucher effects to ever find a significant and negative academic impact.
I’m all for looking further into this, but for now I’m ready to call this duck a duck until there is some reason not to do so.
My point was the utter hypocrisy of a privatization supporter *now* claiming that out-of-school factors could have such an impact on school performance. Ya think? That’s what public school teachers have been trying to tell you for, I dunno, a couple decades now. But now that your wonderful vouchers and charters are turning out to be crap, *now* you have the nerve to trot out the notion that maybe out-of-school factors are the problem? So if this is the excuse du jour of voucher schools, can you at the very least admit it would apply equally so to public schools and maybe, just maybe, public schools aren’t, in fact “failing”?
(“You” being you plural, as in, all privatization supporters, not simply the commenter to whom I’m replying.)
Dienne, the plural “you” that you’re addressing is more diverse in their views than you may imagine. Some of us here have been cautioning for a while that top-down regulations can have adverse consequences. As I’ve written before, if there was a robust system of choice, we could dispense with all of the top-down regulations on district schools.
But why do we need “choice” if schools aren’t failing? Isn’t the whole point that “zip code shouldn’t be destiny”? Seems like charters and vouchers are suddenly – surprise! – finding out that “zip codes” had little to do with it, other than poor kids tend to be crowded in certain ones. And that maybe the factors that those kids bring to school have more to do with the “quality” of the school than the teachers or the kind of school? So the whole rationale for the house of cards is falling apart and your house of cards is tumbling down.
Look, I don’t necessarily have anything against private schools (my daughters attend one, incidentally, but we pay for it ourselves), as long as they are funded separately from public schools. If the country wants to have a “choice” system, then we need to raise our taxes to fully fund both the public and the privatized systems. But as it stands, charters and vouchers are feeding off of public schools, which such public schools are where the vast majority of the nation’s children go to school. You can’t keep taking money and resources from public schools, handing it over to privatized schools and then turn around and point at the public schools as “failures”, while meanwhile the privatized schools are doing no better while having tons and tons of scandals with massive amounts of money disappearing into people’s pockets.
We have about a dozen random-assignment studies showing a positive impact from school choice, so poverty alone is no explanation.
Choice isn’t about “failing” schools. Yes, too many choice advocates make that case, and many of us have taken them to task for it. As I’ve written many times before, even a great school can’t meet the needs of all the students who just happen to be nearby. THAT’S what choice is about: fostering a diversity of options and empowering parents to choose whatever works best for their kids.
As for resources, we spend more money per pupil than nearly every other advanced nation yet we have worse results than almost all of them. Money clearly isn’t the problem here.
“As for resources, we spend more money per pupil than nearly every other advanced nation yet we have worse results than almost all of them. Money clearly isn’t the problem here.”
Surely you can’t be ignorant of the fact that money is indeed the problem! We have worse results than almost all of them? Surely you can’t be ignorant of the fact that if you compare our low-poverty schools to the low-poverty schools in other countries, we come out on top. As you start adding in more and more of our higher-poverty schools to the mix, our scores go down. How can you even compare our school districts with 20-30% poverty rates with those in other countries with less than 5% poverty rates? Money clearly isn’t the problem? Really?
Jack, that simply isn’t true. Their wealthy students beat our wealthy students. Their low-income students beat our low-income students. It’s only when we inappropriately compare our wealthy students to their overall student performance that we come out on top.
See here: http://educationnext.org/does-poverty-explain-the-mediocre-performance-of-american-schools/
“The United States, however, does equally poorly on a relative basis at teaching its most prosperous students as those coming from lower-income homes. Overall, the U.S. proficiency rate in math places the country at the 27th rank among the 34 OECD countries that participate in PISA. And ironically, the ranking is lower for students from advantaged backgrounds (28th) than from disadvantaged ones (20th).”
So no, poverty is not the problem. And spending isn’t the problem either: http://educationnext.org/is-the-us-catching-up/
We spend more than nearly every other country per pupil: http://www.politifact.com/florida/statements/2015/apr/21/jeb-bush/does-united-states-spend-more-student-most-countri/
http://nces.ed.gov/programs/coe/indicator_cmd.asp