Education reform seems to be consumed by a string of fads. When things don’t work out, we tend to move on to the next fad without reflecting very much on what went wrong so that we might avoid that error in the future. Mike McShane and I recently edited a book on Failure, which explicitly attempted to correct this problem by acknowledging failures and trying to draw lessons from them.
One of the recent fads that enchanted reformers was Portfolio Management, which was supposed to ensure that only high-quality school options were available to families. It’s beginning to be painfully clear that Portfolio Management is failing. It appears to be failing politically, as Denver retreated from Portfolio Management before it even really got going and New Orleans shifted control of the portfolio back to the long-reviled traditional school district board. But now there is some evidence to suggest that Portfolio Management is suffering educationally as well.
To the extent that NAEP results are informative about school quality (and I’ve previously expressed my doubts about this), test scores for Louisiana charter schools have been falling off a cliff. In 8th grade math, for example, scores rose to as high as 280 in 2013, but have dropped to 264 in 2017. A change of 10 scale points is supposed to correspond roughly to a grade level, so this is a pretty precipitous drop over the last four years. In 8th grade reading scores rose to as high as 261 in 2013 before falling to 254 in 2017. 4th grade reading and math scores have similarly declined.
I’d like to hear what champions of the Louisiana portfolio model think is going on. I thought Portfolio Management was supposed to give us only high quality options — and it largely relies on test scores as an indicator of quality — so why are the scores dropping? Are Portfolio Managers actually not very good at predicting quality? Have there been other regulatory changes that came along with Portfolio Management that have harmed the educational environment? For example, the leaders of the Recovery School District were at the forefront of eliminating exclusionary discipline from schools. Could the change in school discipline have eroded behavioral control and harmed achievement? Of course, it is always possible that there have been changes in the composition of students in charter schools which have caused these declines, although virtually all schools in New Orleans are charters and the composition of the city has not changed that much in 4 years.
But it is important to remember that just eyeballing NAEP scores is a horrible way to assess causal effects of programs, so we should be very wary of attributing any change in scores to any policy or practice. Nonetheless, NAEP is useful for raising questions and generating hypotheses. I’d like to hear the hypotheses that supporters of Porfolio Management have to offer that might account for the precipitous drop in NAEP scores in Louisiana’s charter sector over the last several years.