Over at the EdNext blog, Paul Peterson has a very thoughtful piece about the good news found in the recently released evaluation of the DC voucher program. You can read Peterson’s article here. Here is an excerpt, where Peterson explains how it is possible that the Program had such a strong effect on students’ ability to graduate, yet still showed no “conclusive” achievement effects.
“But how were such high graduation rates achieved, when voucher students learned no more than the other students? The answer to that riddle is that the study shows exactly the opposite: Those who went to private school scored 4.75 points higher on the reading test, an effect size of 0.13 standard deviations.
Admittedly, that is not as big an effect as is the voucher impact on graduation rates, and it is only fair to point out that statistician purists insist that any finding, before it can be declared undeniably true, must have only 5 chances in 100 of being wrong. The chances that the reading impact is in fact phony are greater than 5—in fact they are 6 in 100–and so it must be declared—by the statistician purists who supervise reports by government agencies—that “there is no conclusive evidence that [the vouchers] affected student achievement (p. xv).”
But notice the wording—there is “no conclusive evidence.” That is quite different language from saying there is “no evidence” that vouchers raised achievement. Indeed, if you invested $1,000 every time you had 94 chances in 100 of picking the right stock—and only 6 chances of getting it wrong–as is the case here, then, with modern technology, you could become richer than Bill Gates by sundown.”
This is an especially interesting perspective when you consider that Congress is insisting that DC vouchers be killed, and that students should be returned to the DC public schools that have only 6 in 100 chances of doing better in terms of reading achievement.
And, in case you missed them, you can read the editorial that appeared in the Washington Post about the study here, Mike DeBonis of the Washington Post had an article about the study which can be found here, and you can see Ed Week’s coverage here. None of these observers, however, provided the insight that Peterson’s analysis did.
(Guest post by Brian Kisida)
Jay P. Greene's Blog 
A p-value of 0.05 is arbitrary, and flexible, depending on context. For instance, in drug trials, it is often required to achieve a p-value of 0.01 or less. P-values of 0.10 are fairly common for sociological phenomena, like this.
I also noticed that just 27 percent of the treatment group used the scholarships consistently while 47 percent of the control group exercised school choice even without the scholarship.
But, Congressmen, won’t that raise class sizes in the public system??!!
Your friends in the union are always telling us that we should be lowering class size. Often lost in the voucher debate is the fact that if class size reduction is the goal, then “buying out” students with vouchers is a much cheaper way of doing it than building more classrooms.