Beneath the Surface – DC Vouchers and Charters

June 23, 2010

(Guest post by Greg Forster)

I don’t want to jump to conclusions about yesterday’s DC school voucher study, since the study is only just out and we haven’t had time to digest it. But something really caught my attention when I first read the grad rate result that Matt highlighted yesterday:

The major finding of this report, and it is MAJOR, is that students who were randomly selected to receive vouchers had an 82% graduation rate.  That’s 12 percentage points higher than the students who didn’t receive vouchers.

Hold on! I thought to myself. That implies the control group’s graduation rate was seventy percent!

Sure enough, there it is, front and center in the study:

The offer of an OSP scholarship raised students’ probability of completing high school by 12 percentage points overall (figure ES-3). The graduation rate based on parent-provided information was 82 percent for the treatment group compared to 70 percent for the control group.
 
 
 

 

Seventy percent? I thought to myself. That doesn’t sound like the DC school system I know.

Sure enough, Education Week pegs the DC grad rate at forty-nine percent. So what’s the deal with this crazy control group?

Yes, one factor is that the control group is made up of “choosers” – families that sought out school choice. They’re likely to be systematically different from non-choosers, which is the whole reason we do these random assignment shindigs. But come on – they’re not that different.

Then it occurred to me – the 49% DC “public school” grad rate is for district schools; it doesn’t account for charter schools.

A whopping 38% of DC public school students are in charter schools. Now, given that the control group for this voucher study is made up entirely of “choosers,” what percent of that control group do you think are in charter schools? A lot higher than 38% is my guess. (I can’t seem to find data for this anywhere in the report – they spend so much time talking about how some of the private schools converted into charters, you would think they’d have found a few lines to talk about how some of the control group were in charters!)

[Update: HT to Brian for finding the figure (see comments) – 35% are in charters and 12% are in private schools. When I placed a bet that more than 38% would be in charter schools, I forgot that choosers would also choose private schools even without the voucher – which greatly strengthens my argument since private schools likely have even bigger effects on grad rates than charter schools.]

I mean, if your argument to explain the 70% grad rate in the control group is that choosers are very different from non-choosers, then doesn’t that very difference imply we should expect huge numbers of choosers who lose the voucher lottery to fall back on charter schools?

I hope you see where I’m going with this.

It seems obvious that if school choice improve graduation rates – which it clearly does, not only in this study but in previous ones in Milwaukee – then a lot of that benefit is being masked in this study because the control group is also excercising a lot of school choice!

What’s the real grad-rate benefit from school choice? Not “12 percentage points,” but somewhere between 12 percentage points and 33 percentage points.

14 Comments | vouchers | Tagged: , | Permalink
Posted by Greg Forster

Harsanyi: Duncan’s Fundamental Dishonesty

April 8, 2009

(Guest Post by Matthew Ladner)

Denver Post columnist David Harsanyi met with Secretary of Education Arne Duncan yesterday. The encounter did not go well for Secretary Duncan. He claimed that the Wall Street Journal editorial was “fundamentally dishonest” and maintained that no one had even tried to contact him, despite the newspaper’s contention that it did, repeatedly.

The Wall Street Journal, however, provided Harsanyi with evidence of extensive contacts with high level high ranking Duncan subordinates. Harsanyi wrote:

When I called the Wall Street Journal, I discovered a different — that is, meticulously sourced and exceedingly convincing — story, including documented e-mail conversations between the author and higher-ups in Duncan’s office. The voucher study — which showed progress compounding yearly — had been around since November and its existence is mandated by law. So at best, Duncan was willfully ignorant.

So let’s review. Harasanyi essentially asks Joanne Jacob’s question “What did Arne Duncan know and when did he know it?” directly to Secretary Duncan. His response: I KNEW NOTHING!

 

 

 

 

 

 

 

 

 

 

 

It looks as though the very next thing out of Secretary Duncan’s mouth was a denunciation of the Wall Street Journal and then a claim that they had made no effort to contact him. Given that this is empirically falsifiable, it certainly doesn’t add much to the Sergeant Schulz routine on his knowledge regarding the study.

Harsanyi goes on to discuss the incoherence of what Duncan had to say about the program:

But the most “fundamentally dishonest” aspect of the affair was Duncan’s feeble argument against the program. First, he strongly intimated that since only 1 percent of children were able to “escape” (and, boy, that’s some admission) from D.C. public schools through this program, it was not worth saving.

So, you may ask, why not allow the 1 percent to turn into 2 percent or 10 percent, instead of scrapping the program? After all, only moments earlier, Duncan claimed that there was no magic reform bullet and it would take a multitude of innovations to fix education.

Then, Duncan, after thrashing the scholarship program and study, emphasized that he was opposed to “pulling kids out of a program” in which they were “learning.” Geez. If they’re learning in this program, why kill it? And if the program was insignificant, as Duncan claimed, why keep these kids in it? Are these students worse off? Or are they just inconveniencing the rich kids?

Duncan can’t be honest, of course. Not when it’s about politics and paybacks to unions who are about as interested in reforming education as teenagers are in calculus.

Again with the magic bullet! The question isn’t whether vouchers are a magic bullet or not, but whether they help disadvantaged children learn better. The evidence is clear- THEY DO.

UPDATE: Mark Hemingway weighs in on the Denver Post column at NRO’s the Corner.

3 Comments | vouchers | Tagged: , , | Permalink
Posted by matthewladner

DoE agency attempts to Bury the Third Year DC Evaluation

April 3, 2009

DoE to DC kids

(Guest Post byMatthew Ladner)

The third year evaluation of the DC Opportunity Scholarship Program is out, and the results are POSITIVE.

I’ll leave a discussion of the results to others. I’m feeling more than a little perturbed by the blatant Machiavellian politics surrounding the use of the report. 

The Department of Education released this report today, on a Friday afternoon. This constitutes a completely obvious attempt to draw as little media attention as possible.

Worse still: WHERE WAS THIS REPORT DURING THE DEBATE ON THE FUTURE OF THE PROGRAM?

Let’s review: the Congress essentially voted to kill the program a few weeks ago, this report must have been sitting on some bureaucrat’s desk in the department of education. A number of Democrats, including President Obama by the way, have stated that they are open to the school vouchers depending upon the results of research.

A few weeks later, the Department releases the study in a way obviously calculated to draw the least amount of attention. I’m starting to read the report, but the press release looks to contain some negative spin as well.

The future of 1,700 students at stake, after all, and such attempts at manipulation are simply sickening.

3 Comments | vouchers | Tagged: , | Permalink
Posted by matthewladner