Education reform seems to be consumed by a string of fads. When things don’t work out, we tend to move on to the next fad without reflecting very much on what went wrong so that we might avoid that error in the future. Mike McShane and I recently edited a book on Failure, which explicitly attempted to correct this problem by acknowledging failures and trying to draw lessons from them.
One of the recent fads that enchanted reformers was Portfolio Management, which was supposed to ensure that only high-quality school options were available to families. It’s beginning to be painfully clear that Portfolio Management is failing. It appears to be failing politically, as Denver retreated from Portfolio Management before it even really got going and New Orleans shifted control of the portfolio back to the long-reviled traditional school district board. But now there is some evidence to suggest that Portfolio Management is suffering educationally as well.
To the extent that NAEP results are informative about school quality (and I’ve previously expressed my doubts about this), test scores for Louisiana charter schools have been falling off a cliff. In 8th grade math, for example, scores rose to as high as 280 in 2013, but have dropped to 264 in 2017. A change of 10 scale points is supposed to correspond roughly to a grade level, so this is a pretty precipitous drop over the last four years. In 8th grade reading scores rose to as high as 261 in 2013 before falling to 254 in 2017. 4th grade reading and math scores have similarly declined.
I’d like to hear what champions of the Louisiana portfolio model think is going on. I thought Portfolio Management was supposed to give us only high quality options — and it largely relies on test scores as an indicator of quality — so why are the scores dropping? Are Portfolio Managers actually not very good at predicting quality? Have there been other regulatory changes that came along with Portfolio Management that have harmed the educational environment? For example, the leaders of the Recovery School District were at the forefront of eliminating exclusionary discipline from schools. Could the change in school discipline have eroded behavioral control and harmed achievement? Of course, it is always possible that there have been changes in the composition of students in charter schools which have caused these declines, although virtually all schools in New Orleans are charters and the composition of the city has not changed that much in 4 years.
But it is important to remember that just eyeballing NAEP scores is a horrible way to assess causal effects of programs, so we should be very wary of attributing any change in scores to any policy or practice. Nonetheless, NAEP is useful for raising questions and generating hypotheses. I’d like to hear the hypotheses that supporters of Porfolio Management have to offer that might account for the precipitous drop in NAEP scores in Louisiana’s charter sector over the last several years.
Jay P. Greene's Blog 
Reminds me of this great exchange on twitter yesterday, commenting on the NYT editorial board having asserted in 2014 that all chemical weapons were out of Syria:
Noah Pollak: One of the media’s many biases: When military action goes wrong, journalists will spend years reporting out & writing what happened. When diplomacy goes wrong and people die, there are no journalistic investigations. Nobody is held accountable. It’s like it never happened.
Clifford May: Also, when battles go badly, the military conducts post-action reviews to learn and do better next time. When diplomacy goes goes badly, the diplomats become university deans.
And the failing military generals are relieved of their posts or decide they need to spend more time with their families.
The 2015 NAEP, the 2017 NAEP and recent state test scores are all pointing in the same direction: https://jaypgreene.com/2017/11/15/riding-on-the-trainwreck-of-new-orleans/
And NAEP scores for LA charters are dropping while the results for charters almost everywhere else are rising: https://jaypgreene.com/2018/04/11/charters-celenaep-good-times-during-lost-decade/
[…] at Jay’s blog, Jay and Matt wrote two critical posts of portfolio management and harbormasters (our team calls them […]
[…] a failure and that it is no different from a school district (although it is privately controlled). Read here. and here. The latter post is advice written to the Arnold Foundation about why it should not invest […]
[…] a failure and that it is no different from a school district (although it is privately controlled). Read here. and here. The latter post is advice written to the Arnold Foundation about why it should not invest […]