Jay P. Greene's Blog

Yes, answers Roland Fryer in an amazing study released this month.  Based on earlier work, he identified 5 features of charter schools that helped them produce strong results: “increased time, better human capital, more student-level differentiation, frequent use of data to inform instruction, and a culture of high expectations.”  Fryer then somehow convinced the superintendent and school board in Houston to pursue these five reforms in a serious way in 9 struggling traditional public schools.  (CORRECTION — the Houston folks report that they were eager to pursue some promising reforms and required no convincing.  They should be commended for that.)  Here, in brief, is what they did:

To increase time on task, the school day was lengthened one hour and the school year was lengthened ten days. This amounts to 21 percent more school than students in these schools obtained in the year pre-treatment and roughly the same as successful charter schools in New York City. In addition, students were strongly encouraged and even incentivized to attend classes on Saturday. In an effort to significantly alter the human capital in the nine schools, 100 percent of principals, 30 percent of other administrators, and 52 percent of teachers were removed and replaced with individuals who possessed the values and beliefs consistent with an achievement-driven mantra and, wherever possible, a demonstrated record of achievement. To enhance student-level differentiation, we supplied all sixth and ninth graders with a math tutor in a two-on-one setting and provided an extra dose of reading or math instruction to students in other grades who had previously performed below grade level. This model was adapted from the MATCH school in Boston – a charter school that largely adheres to the methods described in Dobbie and Fryer (2011b). In order to help teachers use interim data on student performance to guide and inform instructional practice, we required schools to administer interim assessments every three to four weeks and provided schools with three cumulative benchmarks assessments, as well as assistance in analyzing and presenting student performance on these assessments. Finally, to instill a culture of high expectations and college access for all students, we started by setting clear expectations for school leadership. Schools were provided with a rubric for the school and classroom environment and were expected to implement school-parent-student contracts. Specific student performance goals were set for each school and the principal was held accountable for these goals.

And the result:

In the grade/subject areas in which we implemented all five policies described in Dobbie and Fryer (2011b) – sixth and ninth grade math – the increase in student achievement is dramatic. Relative to students who attended comparison schools, sixth grade math scores increased 0.484σ (.097) in one year. In seventh and eighth grades, the treatment effect in math is 0.125σ (.065) and is statistically significant. A very similar pattern emerges in high school math: large effects in ninth grade and a more modest but statistically significant effect in tenth and eleventh grade, which suggest that two-on-one tutoring is particularly effective. The results in reading exhibit a different pattern. If anything, the reading scores demonstrate a slight decrease in middle school, though not statistically significant, and a modest increase in high school. Impacts on attendance – which are positive and statistically insignificant – are difficult to interpret given the longer school day and longer school year.

Strikingly, both the magnitude of the increase in math and the muted effect for reading are consistent with the results of successful charter schools. Taking the treatment effects at face value, treatment schools in Houston would rank third out of twelve in math and fifth out of twelve in reading among charter schools in NYC with statistically significant positive results in the sample analyzed in Dobbie and Fryer (2011b).

Using data from the National Student Clearinghouse, we investigate treatment effects on two college outcomes: whether a student enrolled in any college (extensive margin) and whether they chose a four-year college, conditional on enrolling in any college (intensive margin). Calculated at the mean, students are 6.2 percentage points less likely to attend college, though the effect is not statistically significant. Conditional on attending college, however, treatment students are 17.7 percentage points more likely to enroll in a four-year institution, relative to a mean of 46% in comparison schools – a 40% increase.

Traditional public schools can get results like a KIPP school without having to actually become KIPP schools.  They just have to imitate a few of the key features employed by KIPP and other successful charter schools.  This is incredibly encouraging news.  It means that traditional public schools are really capable of making significant progress if only they become more open to learning from successful charter schools.  They can make that progress without having to cure poverty and all other social ills (although I’m sure that would be nice too).

Of course, there are serious concerns about bringing these reforms to scale, which Fryer considers in his conclusion.  He dismisses union opposition as a serious obstacle based on the fact that the unionized school system in Denver is pursuing a similar reform strategy.  I’m not so easily convinced that unions nationwide will jump aboard a plan that involves huge turnover in staffing and significantly more hours and days per year.  Cost is another barrier to bringing this reform strategy to scale, but he notes that the marginal cost is only $1,837 per student and the rate of return on that investment would be roughly 20%.

But the most serious concerns seem to be fidelity to implementation and shortages of quality labor.  We could all be heart surgeons if we just did what heart surgeons do.  But there are only so many people capable of doing that work and not every office building can be re-organized as a hospital.  Then again, successful teaching isn’t exactly heart surgery (although it can be just about as important), so perhaps there is real hope of bringing this to scale.  We won’t know until we try it in more places with more schools.