Jay P. Greene's Blog

(Guest Post by Collin Hitt)

A new report finds generally positive gains for charter schools across the country. This adds to a growing literature that finds positive results for charter schools. But more importantly, the report is from Stanford CREDO, whose previous research has been the most cited research by charter opponents over the past four years.

CREDO does not use random assignment methods, which are the gold standard in social science research. They use a matching method that has generated significant controversy, particularly because some have claimed that their results are biased against charter schools. There’s no need to rehash that debate here. For now let’s take CREDO’s results at face value. Their finding:

The National Charter School Study 2013 looks at performance of students in charter schools in 26 states and New York City, which is treated separately as the city differs dramatically from the rest of the state. In those states (and New York City), charter school students now have greater learning gains in reading than their peers in traditional public schools. Traditional public schools and charter schools have equivalent learning gains in mathematics.

I actually see CREDO’s newest report as a more significant political development than as an advance of scientific understanding. The group’s previous work had been used as a potent weapon against charter schools, despite the fact that there are mounds of gold standard studies that finds gains for charter schools.

In 2009, CREDO shot to prominence with a report that covered 15 states and the District of Columbia. Five states saw gains for charter students (AR, CO, IL, LA and MO). Six saw declines for charter students (AZ, FL, MN, NM, OH and TX). Three states (CA, GA and NC) and DC saw mixed results.

CREDO’s report was repeatedly used and misused by opponents of charter schools. I saw this firsthand in Illinois, where CREDO actually found positive results. The report was used to argue against charter schools generally. I even saw it used, repeatedly, as evidence against the creation of independent authorizers for charter schools – this despite the fact that the original report did not include any information from states like IN, MI, NJ, NY or WI that had some of the most active and well regarded independent authorizers of charter schools.

In the intervening four years, CREDO has released additional reports for six states. Five found gains for charter schools (IN, MA, MI, NJ and NY), while only one found declines for charters (PA). They’ve also updated previous state results, most recently in Illinois, with charters posting stronger gains than previously reported. These intermittent reports have done little to force charter opponents to update their talking points. I think this new report is different.

Anyone following CREDO’s work since 2009 will be unsurprised by today’s findings. The new national report includes several states that, by CREDO’s estimates, are home to high-performing charters that were omitted from their 2009 report. CREDO is arguing that charter quality in general has improved, as well. I’ll buy that, too, though I also suspect that districts have begun to more strongly respond to charter school competition in ways that have improved performance in district schools. Improvements systemwide from increased competition would actually obscure the benefits of attending a charter school, in studies like CREDO’s.

The next couple of weeks will be an interesting test for journalists who cover charter schools. For years, CREDO’s report has repeatedly been quoted as unambiguous evidence that charter schools don’t work. No one can now do that in good faith. CREDO now finds that the evidence on charter school performance is generally positive and improving significantly.

(Guest Post by Collin Hitt)

WBEZ in Chicago has published one of the more incomplete stories on charter schools I’ve read in a long time.  It asserts:

Charter advocates and even the Chicago Tribune editorial board say 19,000 kids are on charter school waiting lists in the city.

There’s just one problem with that number: it’s not accurate. It significantly overstates demand.

Charter schools in Chicago each maintain their own lotteries and waitlists. Many eager families apply to more than one school, and then suffer the double disappointment of failing to win a lottery to get into either. Thus – WBEZ’s main point – waitlist statistics overstate the demand of the charter school because many children are on more than one waitlist.

The effect of the story is to discredit the school choice movement in Chicago. A full story would have reached the opposite conclusion. WBEZ presents this line as its coup de grace.

Though Andrew Broy from the Illinois Network of Charter Schools has insisted that 19,000 children are on waiting lists for Chicago charter schools, he now says he believes the real number is around 65 percent of that.

Assuming Broy’s guess is correct, that means as many as 6,650 students are on more than one waitlist. How does that undermine the case for charter school demand? Thousands of parents have sought numerous options to flee their current schools. These are determined and desperate people. But limits on charter school supply have denied them any option. To boot, WBEZ reports that there are waitlists at district-run magnet schools as well, suggesting that some parents on charter waitlists have tried and failed to win lotteries into other schools of choice as well. That’s a tragic story that, to me, calls for more charter schools.

WBEZ took particular pleasure in needling the Chicago Tribune, who frequently cites the 19,000 number. There’s a reason, however, that the Trib reports that stat. It’s officially reported by the Illinois State Board of Education. Neither the state board nor WBEZ calculated a different figure that adjusts for double counting. If they did, I’m sure the Trib would report it.

All of this boils down to an obvious point. Waitlists are a crude way to measure charter school demand. Some children land on more than one waitlist. Some families don’t ever apply, discouraged by the long waiting lists they read about. Some families attempt to transfer after enrollment periods and waitlists have closed. Other families haven’t yet found a charter school they like – an inevitable side effect of policymakers limiting the number of charter schools permitted to open.

A complete journalistic account would have found other ways to estimate demand. One such way: commission a survey of families. Guess who did that – the Chicago Tribune, in partnership with the Joyce Foundation.

On the Trib’s behalf, the University of Chicago conducted a scientific phone poll of Chicago adults. Half of the 1,010 respondents were Chicago Public School parents.  It reports:

Many existing charter schools in Chicago have waiting lists because more parents want to enroll their children in public charters schools than these schools can currently accommodate. Would you agree or disagree that Chicago Public Schools (CPS) should make it easier for public charter schools to expand in neighborhoods where public charter schools have waiting lists ? (AFTER RESPONSE, PROBE) Do you…

Strongly agree 46.4

Somewhat agree 17.3

Somewhat disagree 8.0

Strongly disagree 24.0

Neither agree nor disagree 2.4

DON’T KNOW 1.2

REFUSED 0.7

So 63.7 percent of Chicagoans support increasing charter schools to meet demand. Chicago Public Schools enrolls 400,000 students. There are 19,000 students on waitlists and another 40,000 or so enrolled in charter schools. Together that’s 15 percent of total enrollment, yet 63 percent of Chicago residents support increasing supply. So do we really think – as WBEZ suggests – that the 19,000 number overestimates demand?

Relying on polling, of course, has its limits. Here’s something else that WBEZ could have done: looked at charter demand elsewhere. Gary, Indiana is a beleaguered suburb of Chicago. It’s a rough place with its share of academic struggles. Indiana policy is more permissive of charter schools than is Illinois’. In Gary, 31 percent of students are enrolled in charter schools, as compared to 11 percent in Chicago. In Detroit, 41 percent of students are enrolled in charter schools. In Milwaukee, 21 percent are in charters and even more students are enrolled in that district’s school voucher program.

Chicago and Milwaukee and Detroit are different. But not so different that charter school demand would be two to four times lower in the Windy City. There is unmet and unmeasured demand for charter schools in Chicago, which brings me to my final point.

Charter schools create their own demand. There were zero students on charter waiting lists when there were zero charter schools in Chicago. As charter schools expand in the city, more families will sign up to attend them. Waitlists tell a compelling story – people want into charter schools. But they don’t tell the whole story, and neither did WBEZ.

(Guest Post by Collin Hitt)

Mathematica released a major study of KIPP charter schools today. KIPP is primarily a middle school network, with schools across the country. The Mathematica study uses a random-assignment research design, making it at least the eleventh such study we now have on charter schools. It found significant gains in math, positive but insignificant gains in reading. So, again, every random assignment study yet conducted on urban charter schools finds positive effects.

The random assignment study was limited to 13 KIPP charter schools in six states. KIPP’s network is much larger than that. So the authors employed a matching technique in order to evaluate the impact of a larger number of KIPP schools: they compared KIPP students to other kids who on paper were nearly identical. Matching techniques are far less rigorous than lottery-based estimates. But, since Mathematica had lottery-based estimates against which they could compare their matching technique estimates, they were able to validate their matched sample of students as a credible comparison group. They found that their random assignment estimates closely tracked their matching estimates, at relevant schools.

So they employed their matching techniques at a larger sample of 41 schools.  Mathematica then concluded that after just three years in KIPP, students made gains in math, reading, science and social studies that ranged from 8 to 14 months of additional learning. In the parlance of Matt Ladner, “boom.”

(Guest Post by Collin Hitt)

Earlier this winter on JPGB I summarized the nine studies of charter schools (that I knew of) that use a rigorous random-assignment research design. I missed one. Harvard’s Roland Fryer and Stanford’s Vilsa Curto have released a study of the SEED charter school in inner-city Washington D.C.

SEED enrolls most poor, minority students. But it isn’t your typical charter school. While open to anyone who wins a seat through its enrollment lottery, SEED is a boarding school that’s quite expensive. It’s a ‘No Excuses’ school with strict discipline and high expectations. With control over students’ schooling, diets and leisure time, the school’s founders promised big results. From Fryer and Curto’s November 2012 report:

Our lottery estimates reveal that SEED is effective at increasing achievement among poor minority students. Students who enroll in SEED increase their achievement by 0.211 (standard deviations) in reading and 0.229 (standard deviations) in math, per year. Thus, SEED schools have the power to eliminate the racial achievement gap in four years.

Those are remarkable gains. But SEED comes at a steep cost.

At the SEED School in Washington, D.C., about $39,275 is spent per pupil per year, compared to $20,523 per student in District of Columbia Public Schools (DCPS).

Leave aside for a moment that DC Public Schools spend $20,523 and get abysmal results. Sending a kid to SEED nearly doubles that cost. That marginal increase is somewhat understandable, since the school is housing and feeding adolescent kids. Nevertheless, those costs don’t typically fall squarely on the DC school system. So is it worth it?

Our lottery estimates suggest that attending the SEED school for one year is associated with a 3.8 percent increase in earnings (Chetty et al., 2012), a 1.0 to 1.3 percent decrease in the probability of committing a property or violent crime (Levitt and Lochner, 2001), and a 4.4 to 6.6 percent decrease in the probability of having a health disability (Auld and Sidhu, 2005; Elias, 2005; Kaestner, 2009). If SEED affects educational attainment as dramatically as achievement, the implied returns are dramatic (e.g. Card, 1999; Philip Oreopoulos, 2007). The public benefits alone from converting a high school dropout to graduate are more than $250,000.

For obvious reasons, SEED’s results are not indicative of all charter schools. It’s a unique type of charter school. But SEED’s results, while incredibly strong, shouldn’t have caught anyone by complete surprise. Fryer and Curto’s study is the tenth random assignment study of charter schools. All of them find gains for urban, low-income students.

(For the pointer, I thank Jon Mills)