(Guest post by Matthew Ladner)
The Department of Education released the final report of the evaluation of the DC Opportunity Scholarship Program today. The major finding of this report, and it is MAJOR, is that students who were randomly selected to receive vouchers had an 82% graduation rate. That’s 12 percentage points higher than the students who didn’t receive vouchers. Students who actually used their vouchers had graduation rates that were 21% higher. Even better, the subgroup of students who received vouchers and came from designated Schools in Need of Improvement (SINI schools) had graduation rates that were 13 percentage points higher than the same subgroup of students who weren’t offered vouchers–and the effect was 20 percentage points higher for the SINI students who used their vouchers!
This is a huge finding. The sorry state of graduation rates, especially for disadvantaged students, has been the single largest indicator that America’s schools are failing to give every student an equal chance at success in life. Graduating high school is associated with a number of critical life outcomes, ranging from lifetime earnings to incarceration rates. And, despite countless efforts and attempts at reform, changing the dismal state of graduation rates has been an uphill battle.
Of course, the uphill battle will continue. As most are aware, Congress voted to kill the DC voucher program last year, despite evidence that the program had significantly improved reading achievement for students who received scholarships. That evidence didn’t count for much when faced with opposition from teachers’ unions.
In the final report, the reading achievement findings just miss the Department of Education’s threshold for statistical significance. As a result, the spin put out by the administration claims that there is “No conclusive evidence that the OSP affected student achievement.” This is wrong of course. Last year’s (third year) report DID find conclusive evidence that the Program raised student achievement in reading. A close read of this year’s final report reveals that the sample size of students in the final year was smaller because a number of the students participating in the study had graded-out of the Program. It’s not surprising then that the statistical significance of the reading effects fell just short of the required level. Still, with a p-value of .06, we can say that we are 94% certain that the treatment group did outperform the control group in reading in the final year. Moreover, the final report found statistically significant achievement gains for 3 of the 6 subgroups they examined.
In sum, the five-year evaluation of the DC voucher program has shown that low-income students who recieved scholarships have higher graduation rates, higher student achievement, increased parental views of safety, and increased parent satisfaction. There was not one single negative finding over the entire course of the evaluation. I’d say that’s quite a success for a program that spent a fraction of the per-pupil amount spent in DC public schools.
So when does the re-authorization begin?
Jay P. Greene's Blog 
Vouchers don’t improve achievement, but obviously there is something about the schools the kids used the vouchers at which DID improve their achievement rates. Is anyone looking in to precisely what that is and why it worked so much better?
Is this another report that doesn’t seperate out the kids who win the voucher and then attend a private school from the kids who win the voucher and stay in the public school?
Yes it is, you missed that sentence in Matt’s article… 😛
Rebecca-
It is important to remember that the just barely missed signficance result is under the intention to treat model that measures an offer of a voucher rather than the use of one. Couple that with student attrition over time and barely missing the standard statistical threshold anyway, and much higher graduation rates, and I’d bet my left big toe that vouchers do improve achievement.
On the question of why the graduation rates are so much better, school culture plays a huge role in student success. Schools with a focus on academic achievement with a culture lead and controlled by the staff will succeed regardless of whether they are public or private schools. I think these results clearly show that with about a third of the money provided to DCPS the kids get to attend more effective schools.
What’s the rationale for emphasizing the offer of a voucher, the 12% number, versus the actually going to a private school because you get a voucher, the 21% number?
Shouldn’t it be the other way? Ie, attending a private school via voucher has X effect, and the public policy decision to offer vouchers has the smaller Y effect…but of course that’s not the “fault” of the voucher-receiving schools.
I feel like the way it’s presented dilutes the effect of what the Average Joe would care about.
GGW-
It has something to do with metaphysics. I’m more interested in the impact of exercise on my health than the impact of being told that exercise is good for me, so I am with you. On the other hand, it could be a possible source of selection bias if the lottery winners who actually enrolled their kids with a voucher were different as a group than those that won the lottery and did not enroll- more motivated, etc.
Luckily the graduation results are as clear as the summer sun.
[…] D.C.’s voucher program raised graduation rates significantly, writes Matthew Ladner on Jay P. Greene’s Blog. The final evaluation report found […]
[…] Congress each year – has resulted in the program being phased-out.This is incredibly unfortunate. Matt Ladner notes over at Jay Greene’s blog:In sum, the five-year evaluation of the DC voucher program has shown that low-income students who […]
[…] schools was insignificant, Matt Ladner points out from the data there is 94 percent certainty “that the treatment group did outperform the control group in reading in the final year” (the Department’s cut-off for statistical significance is 95 […]
[…] Dr. Matt Ladner, vice president of research at the Goldwater Institute reports: […]
[…] […]
[…] aside the uninformed claim that vouchers never work (in fact, they improve graduation rates, force public schools to improve, and improve test scores at least some of the […]
[…] from the Department of Education shows that students who used their vouchers had graduation rates 21% higher than those who did not receive them. It doesn’t take a high school diploma to see how amazing […]
[…] from the Department of Education shows that students who used their vouchers had graduation rates 21% higher than those who did not receive them. It doesn’t take a high school diploma to see how amazing […]
[…] from the Department of Education shows that students who used their vouchers had graduation rates 21% higher than those who did not receive them. It doesn’t take a high school diploma to see how amazing […]
[…] from the Department of Education shows that students who used their vouchers had graduation rates 21% higher than those who did not receive them. It doesn’t take a high school diploma to see how amazing […]
[…] here’s Matthew Ladner from about a year ago on what we know about the DC voucher program: In sum, the five-year evaluation of the DC voucher program has shown that low-income students who […]
[…] MAJOR, is that students who were randomly selected to receive vouchers had an 82% graduation rate.That’s 12 percentage points higher than the students who didn’t receive vouchers. Students who actually used their vouchers had […]