Julian Betts and Emily Tang at the University of California at San Diego have a new systematic review of the research on charter schools. They look at more than 30 studies that meet minimal criteria for research quality. They find that charters have statistically significant positive effects on math and reading achievement in elementary grades and on math in middle school. There are no significant effects for reading in middle school or for high school student achievement. The size of the effects are modest, ranging between 2% and 6% of a standard deviation. (See Table 2)
It’s important to step back and review an entire literature, rather than focus on a single study. It is sensible to focus on higher quality research, since results are highly sensitive to research design. But it is completely inappropriate and misleading to pick a single study while ignoring all others of equal or higher quality simply because that one study produces the result you like.
Of course, highlighting the one study she likes is Diane Ravtich’s stock in trade. All we hear from Ravitch and her Army of Angry Teachers is about the CREDO study. That’s one study — and not a high quality one. And even then Ravitch distorts what CREDO finds.
But Betts and Tang’s review includes CREDO and dozens of other studies. When we look at the full set of research, we see some significant and positive results. And in Table 4, Betts and Tang show us that if you exclude the CREDO study, the positive effects for charters get stronger so that charters significantly improve math achievement across all grades.
Of course, you shouldn’t exclude that one study, but it is informative that the one study that Ravitch and her Army of Angry Teachers hold up as proof of their view that charters don’t work is clearly an outlier from the full set of research. And if we focus on the highest quality random-assignment studies of charter schools, the positive results are even stronger.
I wonder if Diane does this in her historical research. Does she pick the one quotation or document that supports her argument while misleading readers about the entire set of information? It’s harder to catch Ravitch in this sort of deceptive scholarship in historical work, but in quantitative empirical research, it is the essence of what she does.
(edited for typos)