Jay P. Greene's Blog

(Guest Post by Collin Hitt)

So you say charter schools don’t work. That’s an empirical claim. It needs to be backed up by evidence. Here’s a helpful guide to the most rigorous research available. Once you’ve tackled this material, you’ll be in position to prove your point.

As you probably know, the gold standard method of research in social science is called random assignment. Charter schools are particularly well-suited for random assignment evaluations, since they’re usually required by law to admit students by lottery. The lotteries are fair to families – that’s why they’re put in place. But they also allow researchers to make fair comparisons between students who win or lose lotteries to attend charter schools.

To date, nine studies lottery-based evaluations of charter schools have been released. Let’s go through them, starting with the earliest work.

The first random assignment study of charter schools was released in 2004 by Caroline Hoxby and Jonah Rockoff. It focused on Chicago International Charter School. After three years, charter students had significantly higher reading scores, equal to 3.3 to 4.2 points on 100-point rankings.  Gains were even stronger for younger students.

That same year, the University of California San Diego released a study of the Preuss charter school located on the university’s campus. Test scores for charter students appeared unchanged, but the school improved college-going rates by 23 percent: 90 percent of Preuss juniors were headed to four year colleges.

So the first two random-assignment studies of charter schools won’t help your point. They find gains for charter schools. But those studies are becoming dated; most of the national charter boom has occurred since they were published. Also, the San Diego study employs few statistical controls. So these studies don’t disprove your point either. Let’s review the newer stuff.

In 2010, Harvard’s Will Dobbie and Roland Fryer released a study of the Harlem Promise Academy. Entering kindergartners experienced large gains by the third grade, sufficient to eliminate the black-white achievement gap, equal to 0.58 standard deviations (sd) in reading and 0.49 sd in math. Students who entered Harlem Promise Academy in early middle school saw smaller gains that nevertheless by the eighth grade closed the achievement gap in math and reduced it by half in reading.

Later in 2010, researchers from MIT, Harvard and Michigan released a study of KIPP Academy in Lynn, Massachusetts. The charter school is part of the national charter network, the Knowledge is Power Program (KIPP). After a single year in the school, students saw achievement gains of 0.12 sd in English and 0.35 sd in math.

And earlier this year, researchers from Yale and Brown released a study of an unnamed charter network in an anonymous school district. There were no visible math gains for charter students, but they did see awfully big reading gains of 0.35 sd and writing gains 0.79 sd.

Charter advocates will point to these studies to try to prove you wrong, since these charter schools are definitely working. In turn, you could attempt to discredit the statistical math of the authors above. (Good luck.) Or you could make a more obvious point: these studies together look only at five charter operators. There are hundreds of charter operators across the country. The researchers could be cherry-picking – studying schools that they suspected beforehand were high-performing.

Larger random-assignment studies could address these issues, if they looked at a wider number of charter schools. Luckily, we’ve got four studies that do just that, all of them fairly recent.

The first is a 2009 study led by Caroline Hoxby. It examines practically every charter school in New York City. For every year students were enrolled in a charter school, they saw 0.04 sd gains in reading and 0.09 sd gains in math. The findings here are similar to the middle school gains that Fryer found at Harlem Promise Academy, though the citywide charter gains are clearly smaller than the Promise Academy’s extraordinary gains for kindergarteners.

Later that year, a citywide study of Boston charter middle and high schools found that charters produced “extraordinarily large” gains, according to the authors, who were based at Duke, Harvard, MIT and Michigan. After only one year, Boston’s charter high schools produced gains of 0.16 sd in reading and 0.19 sd in math. Charter middle schools in the city produced similar reading gains of 0.17 sd and a remarkable 0.54 sd in math.

In 2010, the US Department of Education released the first nationwide random-assignment study of charter middle schools. It contained two useful findings. Charter schools in affluent areas produced lower results than neighboring schools, which makes some sense. Charter schools in the suburbs are competing with higher quality schools than found in the inner cities. Charter schools in urban areas, enrolling a large percentage of poor students, posted significant gains in math, over two years equal to 0.18 sd.

In 2011, the team behind the 2009 study of Boston charter schools presented findings from a statewide evaluation of Massachusetts charter middle and high schools. Overall, results were positive. As with the USDOE study of middle schools, they found that charter schools in non-urban areas produced no positive gains. On the other hand, schools located in urban areas produced middle school gains of 0.12 sd in English and 0.33 sd in math and high school gains of 0.33 sd in English and 0.39 sd in math. These gains almost perfectly mirror the findings at KIPP Lynn, which is one of many schools included in the statewide sample.

So Harlem Promise Academy and KIPP produced results that are fairly similar to other charter schools nearby. So any allegation of cherry-picking in the studies of those two schools will need to be dropped.

Altogether, these studies have remarkably similar findings that urban charter schools are producing significant gains in reading or math, or both. Suburban charter schools perform less well – you could cite this fact, but frankly this a minor concern in the battle to close the racial achievement gap in American education.

You could make a methodological point: lottery studies don’t tell us about students who never participated in lotteries. In other words, what about students who never signed up for charter schools, who don’t have charter schools in the area, or who signed up for a charter school that didn’t need to run a lottery? Some researchers use less-rigorous “observational” methods to answer these questions.

Indeed, many of the studies above include secondary observational studies to test the validity of this very argument. They look at similar but artificial comparison groups of non-charter students who for unknown reasons didn’t enroll in lotteries. Those secondary analyses broadly confirm the main random-assignment findings.

Altogether, the best research tells a consistent story: charter schools are working. In order to find much evidence to the contrary, you’ll need to dig into third or fourth tier research. And you’ll need to invent a justification to ignore the random assignment literature, though you probably shouldn’t bother. Relying solely on third-rate research simply says that you were never interested in evidence in the first place.

(Guest Post by Collin Hitt

A Romney red state and an Obama blue state both went for charter schools yesterday. Georgia and Washington faced statewide ballot initiatives to create and expand charter schools. Both measures passed.

The past two years have been years of school choice. As Greg has noted, vouchers and tax credit scholarship programs have seen a renaissance and massive growth. Charter schools have grown in turn, often receiving little notice while voucher programs and tenure reforms draw the attention of a dwindling number of anti-reformers.

Here’s a bit of policy background for understanding yesterday’s elections. Two laws must be in place in order for charter schools to open in meaningful numbers. There needs to be a law that allows charter schools to exist, obviously. And there needs to be a law giving charter schools a realistic chance of being approved by those overseeing the application process.

Forty-one states had created charter school laws, before yesterday. But several of those laws, like Virginia’s, are practically meaningless since they entrust the entire approval process for charter schools to school boards and union-dominated interests. This issue – that of charter school “authorizing” – is most important now facing the charter school movement.

Some states have created independent commissions to vet charter school proposals. Others have entrusted colleges to approve charter schools. These entities don’t need local school district say-so in order to approve a charter school in a given area. Independent authorizers bring impartiality to a process otherwise dominated by special interests. Charter commissions and universities aren’t crusading change agents either, just less partial authorities that in many states have allowed the charter school sector to bloom.

Georgia has had a charter school law on the books for years. It had an independent charter school commission for a time, as well, until the state supreme court ruled that a constitutional amendment was needed in order to bypass school district authority when approving charter schools. So the legislature placed an amendment on the ballot to do just that. Amendment One was up for a vote last night.

Washington was, before yesterday, the largest state to not have a charter school law. Through the state’s ballot initiative process, a new charter school law was proposed. This “Initiative 1240” would create a charter law, legally allowing charter schools to open. It would also create a statewide commission like that debated in Georgia, which would realistically approve applications to open charter schools. According to the Washington Policy Center’s Liv Finne, “Initiative 1240 would give Washington the best charter school law in the country.”

Both Georgia and Washington approved the measures. Georgia and Washington are not similar states. Georgia is reliably red and Romney carried it by 8 points. Washington is blue; Obama won there by 12. Georgia approved the charter school measure by 16 points, while Washington adopted the charter law by 2.4 points.

There were, of course, some common factors in both states, mainly the fierce union and bureaucratic opposition to charter schools. Both initiatives were on the one hand attacked as part of an elitist agenda of corporations and billionaires; this was supposed to incite the Occupy and labor union crowds. The teacher unions tried to hide behind the usual smokescreen of local control, as well, hoping to turn the Tea Party and rural voters against a market driven reform. The tactics – devoid of policy substance – didn’t work in either place.

Charter schools prevailed. And not just that, they prevailed on a day that was a mixed bag for other education reform voters. Between candidates and ballot initiatives, there were a number of notable elections, and the results were all over the place. Idaho rejected merit pay and tenure reforms. Michigan rebuked the teachers unions’ attempt to constitutionally guarantee collective bargaining. Republicans retook the Senate in Wisconsin.Tony Bennett lost in Indiana; Mike Pence won in Indiana.

Jay has noted that the status quo has no more intellectual defenders. Elites have bailed on the old way of doing things. There’s disagreement about which reforms are best. And the unions will still block reforms whenever they can, while remaining a potent force in elections.

But when it comes to whether American needs more school choice, the debate is over. Yesterday was evidence of that. Washington and Georgia agree, charter schools are good. Mitt Romney and Barack Obama agree, we need more school choice. There is still disagreement over how much school choice is appropriate, whether charter schools are sufficient, or whether private school options should be expanded as well. That debate will be important, but it was the thoughtful conversation that anti-reformers never wanted America to have.