It’s National Charter Schools Week, so here is the post I’ve written for the George W. Bush Institute Blog on the issue:
According to the Global Report Card, more than a third of the 30 school districts with the highest math achievement in the United States are actually charter schools. This is particularly impressive considering that charters constitute about 5% of all schools and about 3% of all public school students. And it is even more amazing considering that some of the highest performing charter schools, like Roxbury Prep in Boston or KIPP Infinity in New York City, serve very disadvantaged students.
As impressive and amazing as these results by charter schools may be, it would be wrong to conclude from this that charter schools improve student achievement. The only way to know with confidence whether charters cause better outcomes is to look at randomized control trials (RCTs) in which students are assigned by lottery to attending a charter school or a traditional public school. RCTs are like medical experiments where some subjects by chance get the treatment and others by chance do not. Since the two groups are on average identical, any difference observed in later outcomes can be attributed to the “treatment,” and not to some pre-existing and uncontrolled difference. We demand this type of evidence before we approve any drug, but the evidence used to justify how our children are educated is usually nowhere near as rigorous.
Happily, we have four RCTs on the effects of charter schools that allow us to know something about the effects of charter schools with high confidence. Here is what we know: students in urban areas do significantly better in school if they attend a charter schools than if they attend a traditional public school. These academic benefits of urban charter schools are quite large. In Boston, a team of researchers from MIT, Harvard, Duke, and the University of Michigan, conducted a RCT and found: “The charter school effects reported here are therefore large enough to reduce the black-white reading gap in middle school by two-thirds.”
A RCT of charter schools in New York City by a Stanford researcher found an even larger effect: “On average, a student who attended a charter school for all of grades kindergarten through eight would close about 86 percent of the ‘Scarsdale-Harlem achievement gap’ in math and 66 percent of the achievement gap in English.”
The same Stanford researcher conducted an RCT of charter schools in Chicago and found: “students in charter schools outperformed a comparable group of lotteried-out students who remained in regular Chicago public schools by 5 to 6 percentile points in math and about 5 percentile points in reading…. To put the gains in perspective, it may help to know that 5 to 6 percentile points is just under half of the gap between the average disadvantaged, minority student in Chicago public schools and the average middle-income, nonminority student in a suburban district.”
And the last RCT was a national study conducted by researchers at Mathematica for the US Department of Education. It found significant gains for disadvantaged students in charter schools but the opposite for wealthy suburban students in charter schools. They could not determine why the benefits of charters were found only in urban, disadvantaged settings, but their findings are consistent with the three other RCTs that found significant achievement gains for charter students in Boston, Chicago, and New York City.
When you have four RCTs – studies meeting the gold standard of research design – and all four of them agree that charters are of enormous benefit to urban students, you would think everyone would agree that charters should be expanded and supported, at least in urban areas. If we found the equivalent of halving the black-white test score gap from RCTs from a new cancer drug, everyone would be jumping for joy – even if the benefits were found only for certain types of cancer.
Unfortunately, many people’s views on charter schools are heavily influenced by their political and financial interests rather than the most rigorous evidence. They don’t want to believe the findings of the four RCTs, so they simply ignore them or cite studies with inferior research designs in which we should have much less confidence.
Progress will be made in our application of research to charter school policies by encouraging everyone to focus on the most rigorous studies, of which we have several. To do that, supporters of charter schools also have to refrain from citing weaker evidence, which only serves to legitimize the use of inferior studies by charter opponents. As exciting as the outstanding performance of charter schools is in my own Global Report Card research, that evidence shouldn’t be used to endorse charter schools. Supporters don’t need to rely on the Global Report Card to make the case for charter schools because they have four gold-standard RCTs on their side. Opponents of charter schools have no equally rigorous evidence on their side. And that’s the point we should all be making.
Like this:
Be the first to like this post.
DC’s NAEP numbers allowed for some additional controls to be introduced when comparing charter and district schools than I was able to do with the Milwaukee comparison. The following chart shows the percentage of general education program students who qualify for a free or reduced lunch scoring “Basic or Better” on the 2011 NAEP exams. Special education students, ELL students and middle/high income students are not included in order to get a quick closer to apples to apples comparison.
Posted by matthewladner 
