It’s National Charter Schools Week, so here is the post I’ve written for the George W. Bush Institute Blog on the issue:
According to the Global Report Card, more than a third of the 30 school districts with the highest math achievement in the United States are actually charter schools. This is particularly impressive considering that charters constitute about 5% of all schools and about 3% of all public school students. And it is even more amazing considering that some of the highest performing charter schools, like Roxbury Prep in Boston or KIPP Infinity in New York City, serve very disadvantaged students.
As impressive and amazing as these results by charter schools may be, it would be wrong to conclude from this that charter schools improve student achievement. The only way to know with confidence whether charters cause better outcomes is to look at randomized control trials (RCTs) in which students are assigned by lottery to attending a charter school or a traditional public school. RCTs are like medical experiments where some subjects by chance get the treatment and others by chance do not. Since the two groups are on average identical, any difference observed in later outcomes can be attributed to the “treatment,” and not to some pre-existing and uncontrolled difference. We demand this type of evidence before we approve any drug, but the evidence used to justify how our children are educated is usually nowhere near as rigorous.
Happily, we have four RCTs on the effects of charter schools that allow us to know something about the effects of charter schools with high confidence. Here is what we know: students in urban areas do significantly better in school if they attend a charter schools than if they attend a traditional public school. These academic benefits of urban charter schools are quite large. In Boston, a team of researchers from MIT, Harvard, Duke, and the University of Michigan, conducted a RCT and found: “The charter school effects reported here are therefore large enough to reduce the black-white reading gap in middle school by two-thirds.”
A RCT of charter schools in New York City by a Stanford researcher found an even larger effect: “On average, a student who attended a charter school for all of grades kindergarten through eight would close about 86 percent of the ‘Scarsdale-Harlem achievement gap’ in math and 66 percent of the achievement gap in English.”
The same Stanford researcher conducted an RCT of charter schools in Chicago and found: “students in charter schools outperformed a comparable group of lotteried-out students who remained in regular Chicago public schools by 5 to 6 percentile points in math and about 5 percentile points in reading…. To put the gains in perspective, it may help to know that 5 to 6 percentile points is just under half of the gap between the average disadvantaged, minority student in Chicago public schools and the average middle-income, nonminority student in a suburban district.”
And the last RCT was a national study conducted by researchers at Mathematica for the US Department of Education. It found significant gains for disadvantaged students in charter schools but the opposite for wealthy suburban students in charter schools. They could not determine why the benefits of charters were found only in urban, disadvantaged settings, but their findings are consistent with the three other RCTs that found significant achievement gains for charter students in Boston, Chicago, and New York City.
When you have four RCTs – studies meeting the gold standard of research design – and all four of them agree that charters are of enormous benefit to urban students, you would think everyone would agree that charters should be expanded and supported, at least in urban areas. If we found the equivalent of halving the black-white test score gap from RCTs from a new cancer drug, everyone would be jumping for joy – even if the benefits were found only for certain types of cancer.
Unfortunately, many people’s views on charter schools are heavily influenced by their political and financial interests rather than the most rigorous evidence. They don’t want to believe the findings of the four RCTs, so they simply ignore them or cite studies with inferior research designs in which we should have much less confidence.
Progress will be made in our application of research to charter school policies by encouraging everyone to focus on the most rigorous studies, of which we have several. To do that, supporters of charter schools also have to refrain from citing weaker evidence, which only serves to legitimize the use of inferior studies by charter opponents. As exciting as the outstanding performance of charter schools is in my own Global Report Card research, that evidence shouldn’t be used to endorse charter schools. Supporters don’t need to rely on the Global Report Card to make the case for charter schools because they have four gold-standard RCTs on their side. Opponents of charter schools have no equally rigorous evidence on their side. And that’s the point we should all be making.
Jay P. Greene's Blog 
What an excellent summary of information, supplemented by cogent analysis! The TV show/series Jay appeared on should pounce on this topic.
The ongoing willingness to disregard the highest quality research, a willingness that includes the President, remains an obstacle. Jay and his colleagues are correct to persist in trying to change that picture.
“According to the Global Report Card, more than a third of the 30 school districts with the highest math achievement in the United States are actually charter schools”
School districts or schools?
School districts or schools?
Yes! Next question?
(The GRC looks at entities that are coded as “districts,” most of which are traditional districts, but it is also common for an individual charter school to be coded as a “district” – a district that contains only one school. This disconnect in the unit of comparison is one reason Jay urges us in this post to base our conclusions on the RCT studies rather than the GRC results.)
Schools for charters vs Districts for public schools.
This different treatment makes the comparison completely bogus. A much more accurate comparison would be to take the best performing single school in the Public School District and compare it to the Charter school.
It’s not as good as a RCT but it is probably available — if you could burrow into the public system’s data back to a comparable time period.
DCPS has a few excellent H.S. with scores that exceed the charter competition. Any comparison of NAEP scores DCPS vs. charters would need to control for socio-economic class which is driving the increase in high school (or middle & high school) charters.
When word gets out, NEA/AFT/AFSCME cartel will lean on legistatures to rig the chartering process so as to:
a) apply the term “charter school” to schools which do not differ in any relevant aspect from the standard cartel school, or…
b) schools so crippled by budgetary and other restrictions as to make them unattractive to parents.
As has happened in Hawaii.
Many states do this. All you have to do is give some arm of he blob an effective veto over the formation of new charter schools.
When I looked at the issue of whether charter schools positively contributed to student achievement a couple of years ago the evidence was mixed. In some cities, including Chicago, performance at the charter schools exceeded comparable public schools. In other cities, the opposite was true. One can find news articles about charter schools that have failed at their education mission.
While charter schools have the potential to improve education results, more focus than Mr. Greene suggests should be placed on WHY certain charter schools succeed and others fail to outperform the comparable public schools. What are the successful charter schools doing right? What are the others doing wrong? If we are going to expand the number of charter schools, let’s do it following the successful models.
But the only reason charters and other forms of choice work when so many previous efforts have failed so comprehensively is because they’re the only reforms that don’t follow your suggested model – anointing a powerful elite class to determine what’s best and then impose their views on everyone else. In a word, technocracy. Eliminating technocracy is the secret sauce that has made choice more consistently effective than any other reform.
Greg,
Excellent. For decades we have heard the refrain advanced by stichmo. The one “best practice” we have been timid to follow is parent choice.
The problem is that charter schools are not more consistently effective: http://www.educationjustice.org/newsletters/nlej_iss21_art5_detail_CharterSchoolAchievement.htm
“The Center for Research on Education Outcomes (CREDO) at Stanford University found in a 2009 report that 17% of charter schools outperformed their public school equivalents, while 37% of charter schools performed worse than regular local schools, and the rest were about the same. A 2010 study by Mathematica Policy Research found that, on average, charter middle schools that held lotteries were neither more nor less successful than regular middle schools in improving student achievement, behavior, or school progress. Among the charter schools considered in the study, more had statistically significant negative effects on student achievement than statistically significant positive effects. These findings are echoed in a number of other studies. ”
This is not a track record that says let’s throw money at anyone who wants to start a charter school in the name of “choice.”
It appears stichmo did not read the original post.
1) The studies Jay discusses in this post are methodologically superior to the CREDO study.
2) If you want to bring in the inferior quality studies you should look at all of them instead of cherry picking the one that confirms your prejudices. When you look at all of them, guess what you find?
3) All that said, I do have my own concerns about charters. Namely, they aren’t nearly as effective as vouchers!
There was only one national study.cited by the author. It stated:
“Key findings from the evaluation include:
• On average, charter middle schools that hold lotteries are neither more nor less successful than traditional public schools in improving student achievement, behavior, and school progress.”
“• The impact of charter middle schools on student achievement varies significantly across schools.”
“• In our exploratory analysis, for example, we found that study charter schools serving more low income or low achieving students had statistically significant positive effects on math test scores, while charter schools serving more advantaged students—those with higher income and prior achievement—had significant negative effects on math test scores”
“• Some operational features of charter middle schools are associated with more positive (or less negative) impacts on achievement.”
It also noted “…because the study could only include charter middle schools that held lotteries, the results do not necessarily apply to the full set of charter middle schools in the U.S.”
These findings don’t support the kind of one-sided cheerleading I am reading.
This reaffirms that you did not read the post…..or choose to misstate its thesis.
@Stitchmo
The CREDO study is seriously flawed. It only looked at 16 states. 41 states and DC has charter schools. It also has a serious statistical mistake in its methodology:
Click to access memo_on_the_credo_study.pdf
@ Michael – Three of the allegedly superior studies cited by the author were limited to one state; the fourth was limited to 15 states. By your standards, all of the cited studies are “seriously flawed.”
The original post identified random assignment studies that showed positive effects in urban areas. It noted that other random assignment studies did not show similar positive effects. It cautioned against drawing conclusions, either way, from weaker studies. What’s the problem?
@stichmo
Those studies were RCTs. Standford’s CREDO was not. That’s the distinction.
Charter schools are also formed for many different reasons. While many of them are an attempt to create a strong academic school, some are founded on themes like: raising students’ global consciousness, environmentalism, or ethnic pride. Some of these do not put strong academics ahead of other agendas.
[…] Greene, a professor of education reform at the University of Arkansas, notes that all studies of charter schools using this rigorous methodology have found that charters […]
[…] Greene, a highbrow of preparation remodel during a University of Arkansas, notes that all studies of licence schools regulating this severe methodology have found that charters […]
[…] (Oh, and by the way, the highest-quality research suggests some real benefits to charters.) […]
[…] Charter Benefits Are Proven by the Best Evidence, Jay P. Greene, May 7, 2012 […]
[…] more information, see “Charter Benefits Are Proven by the Best Evidence,” Dr. Jay P. Greene, 21st Century Chair in Education Reform at the University of […]
[…] Arkansas University’s Jay P. Greene summarizes the research, which is worth quoting at length: […]
[…] Arkansas University’s Jay P. Greene summarizes the research, which is worth quoting at length: […]