Verdict in the WSJ: “School Vouchers Work”

May 3, 2011

Wall Street Journal columnist, Jason Riley has a must-read piece in the WSJ today.  The piece features the work of my University of Arkansas colleague, Patrick Wolf, JPGB’s very own Greg Forster, as well as a reference to the competitive effects study that Ryan Marsh and I conducted in Milwaukee.  There are too many highlights, but here is a (big) taste:

‘Private school vouchers are not an effective way to improve student achievement,” said the White House in a statement on March 29. “The Administration strongly opposes expanding the D.C. Opportunity Scholarship Program and opening it to new students.” But less than three weeks later, President Obama signed a budget deal with Republicans that includes a renewal and expansion of the popular D.C. program, which finances tuition vouchers for low-income kids to attend private schools.

School reformers cheered the administration’s about-face though fully aware that it was motivated by political expediency rather than any acknowledgment that vouchers work.

When Mr. Obama first moved to phase out the D.C. voucher program in 2009, his Education Department was in possession of a federal study showing that voucher recipients, who number more than 3,300, made gains in reading scores and didn’t decline in math. The administration claims that the reading gains were not large enough to be significant. Yet even smaller positive effects were championed by the administration as justification for expanding Head Start….

The positive effects of the D.C. voucher program are not unique. A recent study of Milwaukee’s older and larger voucher program found that 94% of students who stayed in the program throughout high school graduated, versus just 75% of students in Milwaukee’s traditional public schools. And contrary to the claim that vouchers hurt public schools, the report found that students at Milwaukee public schools “are performing at somewhat higher levels as a result of competitive pressure from the school voucher program.” Thus can vouchers benefit even the children that don’t receive them.

Research gathered by Greg Forster of the Foundation for Educational Choice also calls into question the White House assertion that vouchers are ineffective. In a paper released in March, he says that “every empirical study ever conducted in Milwaukee, Florida, Ohio, Texas, Maine and Vermont finds that voucher programs in those places improved public schools.” Mr. Forster surveyed 10 empirical studies that use “random assignment, the gold standard of social science,” to assure that the groups being compared are as similar as possible. “Nine [of the 10] studies find that vouchers improve student outcomes, six that all students benefit and three that some benefit and some are not affected,” he writes. “One study finds no visible impact. None of these studies finds a negative impact.”

Such results might influence the thinking of an objective observer primarily interested in doing right by the nation’s poor children. But they are unlikely to sway a politician focused on getting re-elected with the help of teachers unions.

“I think Obama and Duncan really care about school reform,” says Terry Moe, who teaches at Stanford and is the author of a timely new book, “Special Interest: Teachers Unions and America’s Public Schools.” “On the other hand they have to be sensitive to their Democratic coalition, which includes teachers unions. And one way they do that is by opposing school vouchers.”

The reality is that Mr. Obama’s opposition to school vouchers has to do with Democratic politics, not the available evidence on whether they improve outcomes for disadvantaged kids. They do—and he knows it.

5 Comments | vouchers | Tagged: , , , , , , , , , | Permalink
Posted by Jay P. Greene

School Choice Yearbook

February 10, 2011

The Alliance for School Choice has released their annual school choice yearbook.  It is filled with a ton of useful facts, figures, and other resources.  Be sure to check it out.  Here are some of the highlights from the press release:

• More than 190,000 students are enrolled in school choice programs in the United
States, a growth of nearly 100 percent since 2004-05.
• Seven of the 20 school choice programs in America are specifically tailored to
serve children with special needs, benefiting more than 26,000 students.
• Nearly all of America’s school choice programs provide assistance primarily to
children in low- to middle-income families or to children with special needs.
• Florida is home to the greatest number of students who benefit from school
choice, with 54,000 student participants in the state’s two existing programs.
Two states—Arizona and Ohio—have three school choice programs each.
• All 20 school choice programs are non-discriminatory and feature levels of
administrative, financial, and/or academic accountability.
• Despite a turbulent economy, no existing programs saw funding cuts in 2010.
Two new programs—one for students with disabilities in Oklahoma and another
for students with special needs in Louisiana—were enacted last year with
bipartisan support.

Leave a Comment » | charter schools, tax-credit scholarships, vouchers | Tagged: , | Permalink
Posted by Jay P. Greene

New Study Links Tax Credit to Florida Public School Gains

June 3, 2010

(Guest Post by Matthew Ladner)

A new study by David Figlio links higher gains among Florida public schools with higher levels of competition from the Step Up for Students tax credit program. You can read the St. Pete Times story by Ron Matus here.  Matus wrote:

Figlio emphasized the boost was significant, but modest.

“Anybody looking for a silver bullet has to keep looking,” he said. “What we find is certainly positive and statistically strong, but it’s not like public schools are revolutionizing overnight because of this, either.”

So it turns out that the public school gains associated with a state program with an initial statewide cap of $50m in a state with a multi-billion dollar public school budget were statistically significant but modest. Would it be reasonable to expect anything more from such a modest program? I suggest we scale this public school improvement program up to say a cool billion per year and then measure the impact.

My favorite line in the story comes from a hostile academic:

Another researcher remained skeptical. Stanford labor economist Martin Carnoy, who has studied the impact of vouchers and reviewed the latest study, said Figlio and Hart did “an honest job with the data.”

“But here is the real story: even after several years the effect size is TINY,” he wrote in an e-mail. “They are so small that even small downside effects would nullify them, leaving vouchers as mainly an ideological exercise.”

This is one of the more unintentionally hilarious statements I have read in some time. The field of education reform battle is covered with the dead bodies of reforms that show nothing in the way of a statistically significant impact. Increasing per pupil funding, Head Start, teacher certification, almost everything studied by the “What Works” clearinghouse so far, etc. All of these failures cost a great deal of money and deliver nothing in the way of sustained academic gains.

So the state of Florida passes a small law that actually saves the state money and shows a statistically significant and small result of improving public schools, and we are supposed to wring our hands and despair because something bad could come along and nullify the gains? Ummmmm, no.

First of all, nothing bad did come along and nullify the gains- quite the opposite. This program was only a part of the strategy to increase parental choice in Florida. That strategy also includes charter schools, McKay vouchers and virtual schooling- all of which either already are or soon will be much larger programs than Step Up for Students.

Second, the parental choice strategy was itself a part of a larger effort to improve Florida public schools. Parental choice reinforced the central K-12 reform of grading schools A-F. Transparency, rewards for success, consequences for failure formed the core of the Florida strategy.

Did it work?

The Step Up for Students program played a contributing role in Florida’s symphony of success rather than “destroying public education.”  This is what Milton Friedman argued all along. Bravo- the obvious conclusion to draw is to push both parental choice and public school reform still further in Florida and elsewhere.

3 Comments | competitive effects, tax-credit scholarships | Tagged: , , , | Permalink
Posted by matthewladner

School Choice Reduces Crime, Increases College-Attendance, and Makes Your Breath Smell Better

November 29, 2009

Well, at least the first two claims are supported by rigorous new research based on school choice lotteries in Charlotte, North Carolina. Harvard researcher, David Deming, looked at a public school choice program that allows families to rank order their preferred schools and then admits students by a weighted lottery formula.  The program is designed primarily to facilitate school integration but it also allows random-assignment designed research of the effects of choice.  In the paper, “Better Schools, Less Crime?” , Deming found: 

Seven years after random assignment, lottery winners have been arrested for fewer and less serious crimes, and have spent fewer days incarcerated…  The reduction in crime persists through the end of the sample period, several years after enrollment in the preferred school is complete. The effects are concentrated among African-American males whose ex ante characteristics define them as “high risk.”

In another paper Deming wrote with Justine Hastings, Tom Kane and Doug Staiger, they examined the same Charlotte program but this time focused on the effects of choice on high school completion and college attendance.  They found:

We find strong evidence that high school lottery winners from neighborhoods assigned to the lowest-performing schools benefited greatly from choice. Girls are 12 percentage points more likely to attend a four-year college. Boys are 13 percentage points more likely to graduate from high school but are less likely to attend a four-year college. We present suggestive evidence that changes in relative rank within schools may explain these puzzling gender differences. In contrast with the results for students from low-performing home school zones, we find little evidence of gains for students whose home schools are of average quality.

So, expanding school choice reduces the likelihood that students will become criminals (particularly among African-American males) and increases the chances that boys will graduate high school and girls will attend college.  Given previous research showing that choice increases achievement for participating students, students who remain in traditional public schools, and improves civic goals (like school integration) in addition to these new findings, maybe choice really does make your breath smell better.  It seems to do so many other useful things.

2 Comments | charter schools, vouchers | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

The Heathers Think-Tanks

July 15, 2009

DC-based think tanks run the risk of being obsessed with the latest policy fashion rather than searching for the best long-term solutions.  In the DC bubble, sticking to one’s principles and the evidence is difficult when there are no near-term prospects for advancing policies that are supported by those principles and evidence.  It’s tempting instead to switch one’s policy focus so that it is line with the the current administration and congressional majority.

I was reminded of these hazards of DC think tanks when I received an invitation to the latest Thomas B. Fordham Institute event: “With charter schools ascendant, is there still a future for vouchers?” 

There’s nothing wrong with organizing a panel to consider the relative policy merits of charters and vouchers.  What’s weird is the suggestion that if one policy is currently popular, another might not have a future.  It’s like having a panel that addresses the question:  With Democrats ascendant, is there still a future for the Republican Party? 

Things change.  The current dominance of the Democratic Party won’t last forever.  It may not even last more than a few years.  Similarly, the current popularity of charters relative to vouchers may not last very long.  Rather than assessing the future of policies based on their current popularity, shouldn’t we assess their substantive merits so that we can advocate for the policies that are the most effective?

And if we must obsess on the political prospects of policies rather than their substantive merits, it’s weird to pit the two policies against each other.  Wouldn’t it seem more reasonable to think that as school choice becomes more common, whether with charters or with vouchers, all forms of choice will become more politically palatable?  As I’ve argued before, vouchers have helped make the world safe for charters, so the two policies may work well together. 

Just because the current congressional majority is hostile to vouchers doesn’t mean that the idea has no future or that we have to pit it against other, similar policies that are currently more in fashion.  Dismissing policies because they aren’t on the agenda of the current majority is like the type of argument heard in the 1988 film, Heathers:  “Grow up Heather, bulimia’s so ’87.”

3 Comments | charter schools, vouchers | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

Why Should We Let People Vote?

July 14, 2009

Normally I’m a big fan of Dan Willingham’s ideas but he does have some blind spots.  In particular, Dan seems to miss the point on school choice.  His argument is that for school choice to work, parents have to be rational in making choices:

The logic of school choice seems obvious. If parents selected their children’s schools, they would not choose bad ones, so bad schools would not be able to survive. Schools would have to improve or close, just as a store that offers poor service will lose business to a store that offers better service.

Here’s my problem with that logic: I think it’s highly likely that many parents will choose bad schools.

People often make irrational decisions.

Dan is mistaken in that choice does not require perfect rationality on the part of parents.  All that it required is that parents, on average, will do better at picking schools for their children than the bureaucrats who design schools and compell children to attend those schools.

We all understand that human beings are imperfect and often make mistakes.  Even all of the research on systematic irrationality produces results that are familiar to most people.  The point is that the distant bureaucrat who assigns students to schools controlled by the bureaucrat also suffers from all of these same human foibles. 

Nor can we simply assume that the distant bureaucrat will be focused on academic quality more than parents are.  The distant bureaucrat, even more than parents, has interests that distract from the focus on academic quality.  For example, the bureaucrat might be more concerned about protecting the jobs and incomes of the adults working in schools because those people influence the bureaucrat’s own job status and income.  Just consider whether superintendents are free to do whatever works for kids regardless of the effect on adults working in the schools.

Some might counter that at least the bureaucrats are highly-trained and have access to a lot of information, while parents lack the expertise and information necessary to assess academic quality.  If we really believed this made the bureaucrats superior at making educational choices, we should ask ourselves:  why do we let people vote?

Rather than have individuals make choices about their leaders and policies, shouldn’t we let highly trained experts with superior access to information select our leaders for us?  Regular people may be prone to systematic irrationality when they vote.  In fact, there is a lot of research to support such a conclusion.  For example, people are more likely to vote for more attractive candidates.  Why should people be allowed to vote when: “it’s highly likely that many [voters] will choose bad [candidates]? ”

Of course, the reason why we have democracy despite our awareness of human irrationality is the same reason why we should have schools choice:  on average, people are better at making decisions that affect their own interests than are others.  Even poorly-educated people lacking information are likely to have more knowledge of their interests and how to pursue them than are others making decisions on their behalf.

14 Comments | vouchers | Tagged: , , , | Permalink
Posted by Jay P. Greene

The Rush to Judgment

June 30, 2009

David Figlio’s latest report on Florida’s Corporate Tax Credit (CTC) Scholarship program was released yesterday.  I can’t find the report online but Ron Matus of the St. Pete Times sent it to me and you can read about it in his article. 

(UPDATE:  Here’s a link to the study.)

I agree with almost everything said in the article.  I even agree with Mark Pudlow, the spokesperson for the teacher union when he said: “There is no quick fix for struggling students.” 

The problem is that the standard for success when it comes to school choice is that it has to produce a quick fix or critics deem it a failure and declare: “we really ought to reconsider why we’re doing it.”  No one demands that every other education policy produce huge gains in a single year or they should be “reconsidered.”  Yes, promoters of policies may make unrealistic promises to get them adopted, but the standard for success should be long-term progress, not promises made by politicians.

So let’s slow the rush to judgment and review what we really know about the CTC program.  First, Figlio finds that 92.7% of all CTC students in private schools provided a usable standardized test to the evaluation.  This shows widespread compliance with the legal requirements for those students to be tested to satisfy political concerns for accountability.

Second, Figlio finds that the CTC program has largely targeted students who are significantly more disadvantaged than students remaining in Florida’s public schools — even significantly more disadvantaged than public school students receiving subsidized or free lunch.  So, concerns that the program would cream off the best students appear unfounded.

Third, and most importantly, Figlio’s report does not make any claims about whether students benefited academically from participating in the CTC program.  He simply provides descriptive information on the academic achievement of CTC students as well as subsidized lunch students in Florida public schools.  But we know that CTC students are even more disadvantaged than those public school students and Figlio makes no attempt in this report to control fully for those disadvantages.

Figlio makes these points explicitly and repeatedly in the report: “it is important to recognize that they are not causal estimates of the effect of program participation on student outcomes. Causal comparisons require more complete modeling of the selection decisions into the scholarship program and fuller data from a baseline than is afforded using the 2006-07 school year test score collection. More compelling causal estimates of program participation will be possible following the collection of the 2008-09 school year’s test score data. The comparisons in this subsection should be interpreted as purely descriptive in nature.”

Unfortunately, most people never pay attention to these warnings and rush ahead as if descriptive information is causal.  Folks wrongly conclude that if CTC students make year to year test score gains that are about the same as subsidized lunch public school students, then they must not be benefiting from the program.  Nothing in Figlio’s report supports that conclusion. 

To know whether CTC students are benefiting we would have to know how they would be doing had they remained in public schools.  The best way to judge that is with a random-assignment study where students admitted to the CTC program by lottery are compared with students who lose the lottery and remain in public schools.  Unfortunately, that research design is not possible because there was no lottery.  The next best thing would be to use a research design that approximated random-assignment (like a regression discontinuity) or a rigorous quasi-experimental design that controlled for all observed differences between the two groups.  But Figlio didn’t do that in this report.  He just provided descriptive statistics while promising a more rigorous research design next year.

Of course, we might wonder why Figlio bothered reporting this descriptive information without a more rigorous analysis.  I suspect that he was required to produce a report each year by the legislature, so he complied even though he didn’t have the information he needed for a causal analysis. 

And the descriptive information is useful.  It suggests that choice was no miracle cure since the raw differences between CTC and public students in academic progress were not huge.  Again, miracle cure is the wrong standard for judging a program’s success. 

The CTC program may well have attracted students who had been on a downward trajectory before they switched to a private school.  And the CTC program may well helped those students level-off and may, over time, enable them to make significantly greater progress than they would have made had they remained in public schools.  This is what we’ve seen from rigorous evaluations of other choice programs, including the most recent evaluation of the DC voucher program.  But these things require careful research designs and time to show themselves.  Let’s give David Figlio more time to use a better research design so that we can actually say something about the academic effects of the CTC program.

(edited for typos)

8 Comments | vouchers | Tagged: , , , | Permalink
Posted by Jay P. Greene

New DC Voucher Study

January 13, 2009

My colleague Pat Wolf released today a new study of the DC voucher program based on focus group interviews of families.  It found high levels of parental satisfaction with the program, even among families that returned to the public system.  People appreciated having the choices and felt more involved in their children’s education.

Of course, these satisfaction outcomes don’t usually move the debate very much.  Opponents of voucher programs tend not to be persuaded by parental reports of satisfaction because they doubt the judgment of parents.  That’s why they are skeptical about choice.  And supporters of vouchers view satisfaction outcomes as important, but they are already inclined to trust parental assessments.

But the report provides plenty of contextual information that is useful and interesting even if it is not decisive.  A new test score analysis of the DC voucher program is expected sometime this Spring.

Leave a Comment » | vouchers | Tagged: , , , , | Permalink
Posted by Jay P. Greene

A Modest Proposal for B.B.

August 18, 2008

The advocates of B.B. (Broader, Bolder; or is it Bigger Budgets? or is it Bloated Behemoth?) have yet to muster the evidence to support widespread implementation of their vision to expand the mission of schools to include health care, legal assistance, and other social services. They do present background papers showing that children who suffer from social problems fare worse academically, but they have not shown that public schools are capable of addressing those social problems and increasing student learning.

And if you dare to question whether there is evidence about the effectiveness of public schools providing social services in order to raise achievement, you are accused of being opposed to “better social and economic environments for children.” Right. And if you question the effectiveness of central economic planning are you also then opposed to a better economy? And if you question the effectiveness of an untested drug therapy are you then opposed to quality health-care?

To help the B.B. crowd generate the evidence one would need before pursuing a reform agenda on a large-scale, I have a modest proposal. How about if we have a dozen large-scale, well-funded pilot programs of the “community school” concept advocated by B.B.? And, at the same time let’s have a dozen large-scale, well-funded pilot voucher programs. We’ll carefully evaluate the effects of both to learn about whether one, the other, or both are things that we should try on an even larger scale.

I’m all for trying out new ideas and carefully evaluating the results. I can’t imagine why the backers of B.B. wouldn’t want to do the same. So as soon as Larry Mishel at the union-funded Economic Policy Institute, Randi Weingarten of the AFT, and Leo Casey of the AFT’s blog, Edwize, endorse my modest proposal, we’ll all get behind the idea of trying new approaches and studying their effects — “community schools” and vouchers.

Wait, my psychic powers are picking something up. I expect that some might say we’ve already tried vouchers and they haven’t worked. In fact, Randi Weingarten just wrote something very much like that when she declared in the NY Daily News that vouchers “have not been shown by any credible research to improve student achievement.” Let’s leave aside that there have been 10 random assignment evaluations (the gold-standard in research) of voucher programs and 9 show significant positive effects, at least for certain sub-groups of students. And let’s leave aside that 3 of those analyses are independent replications of earlier studies that confirm the basic positive findings of the original analyses (and 1 replication does not). And let’s leave aside that 6 of those 10 studies have been published in peer-reviewed journals (including the QJE, the Journal of the American Statistical Association, and the Journal of Policy Studies), three in a Brookings book, and one in a federal government report (even if Chris Lubienski somehow denies that any of this constitutes real peer-review). And let’s leave aside that there have been more than 200 analyses of the effects of expanding choice and competition, which Clive Belfield and Henry Levin reviewed and concluded: “A sizable majority of these studies report beneficial effects of competition across all outcomes… The above evidence shows reasonably consistent evidence of a link between competition (choice) and education quality. Increased competition and higher educational quality are positively correlated.”

Let’s leave all of that aside and ask Randi Weingarten how many random-assignment studies of the community school concept she has. Uhm, none. How many evaluations of community schools, period? Uhm, still none. But that doesn’t stop her from drawing the definitive conclusion: “Through partnerships with universities, nonprofit groups and other organizations, community schools provide the learning conditions and resources that support effective instruction and bring crucial services to an entire community.” How does she know?

But I’m eager to help her and all of us learn about community schools if she is willing to do the same to learn about vouchers. Better designed and better funded voucher programs could give us a much better look at vouchers’ full effects. Existing programs have vouchers that are worth significantly less than per pupil spending in public schools, have caps on enrollments, and at least partially immunize public schools from the financial effects of competition. If we see positive results from such limited voucher programs, what might happen if we could try broader, bolder ones and carefully studied the results?

And if community schools really deliver all that is being promised, great, let’s do that too. But if our goal is to do what works, why not give both ideas a real try?

(Link added)

10 Comments | vouchers | Tagged: , , , , , , , , | Permalink
Posted by Jay P. Greene

Best. Choice. Argument. Ever.

August 6, 2008

 

 Brilliant.

 

(HT, Stuart Buck and Lydia McGrew at http://www.whatswrongwiththeworld.net/2008/08/great_video_clip_on_government.html#comments )

4 Comments | vouchers | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

« Previous Entries
Next Entries »