(Guest Post by Matthew Ladner)
When I was a kid in the 1970s someone wrote a book about UFOs having visited earth in the past called Chariots of the Gods. The only thing I remember about this book was a photo of an ancient stone staircase and a caption that read “Staircase going down-to a spaceport?” Someone else wrote a book called “Crash Go the Chariots.” I never examined this book, but I imagine that it was a bit like shooting fish in a barrel.
I am in the midst of reading Jonathan Haidt’s The Righteous Mind: Why Good People are Divided by Politics and Religion and found the following passage quite striking. The book summarizes a great deal of psychological research to reach this point which I cannot do justice to, but roll with this in any case:
In the same way, each individual reasoner is really good at one thing: finding evidence to support the position that he or she already holds, usually for intuitive reasons. We should not expect individuals to produce good, open-minded, truth-seeking reasoning, particularly when self-interest or reputational concerns are in play.
Sounds dismal so far, but fortunately he goes on…
But if you put individuals together in the right way, such that some individuals can use their reasoning powers to disconfirm the claims of others, and all individuals feel some common bond or shared fate that allows them to interact civilly, you can create a group that ends up producing good reasoning as an emergent property of the social system. This is why it is so important to have intellectual and ideological diversity within any group or institution whose goal is to find truth (such as an intelligence agency or community of scientists) or to produce good public policy (such as a legislature or an advisory board).
Again, I can’t summarize the research that lead to this conclusion (read the book and see what you think) but basically Haidt lays out a case that says if you believe that you are some sort of scientist guided solely by sweet reason and evidence, you are the victim of self-delusion. Good reasoning happens through a social challenge process over time.
In that spirit of civil challenge, I’ll offer the following two charts. The first a scatterplot of the 8th grade reading 2017 scores by gains (2017 minus 2009 scores) for all 50 states and all 16 state charter sectors with scores in both 2017 and 2009.
There is a huge amount of success to celebrate in this chart for multiple state charter sectors, most of which have either unusually large gains, or unusually high scores, or else unusually high scores and gains.
Do however take note of Louisiana’s charter sector position in the above chart. Now let’s look at the same chart for mathematics:
Again, there is a great deal to like here- multiple charter sectors with large gains (MI, WI, GA, MD, TX) or high scores (CO, ID) or both high scores and gains. Arizona, this one is for you:
Again however take a look at Louisiana’s charter sector-again low scores and low gains. A quick trip to the NAEP data explorer will demonstrate that this same trend is evident in 4th grade NAEP scores, and state scores have been trending in a negative fashion in recent years as well.
In recent years a school of thought arose in our space that a centralized authority or “harbor-master” could produce better outcomes by carefully controlling both the entrance and the exit of schools from charter sectors, primarily on the basis of standardized test scores. In the case of the Recovery School District in Louisiana, there was a period of strong gains in state test scores to buttress this belief. State test scores however have repeatedly shown themselves vulnerable to gaming and manipulation, especially when large consequences ride on them. Soviet steel makers reported non-stop prodigious growth in steel production for decades when facing such incentives as well. How much steel was actually made-who knows? Educators however have neither the incentive or ability to “teach to” or otherwise game the NAEP exams, which is why they have long stood as an external audit on state testing.
In this case, NAEP and state tests are both pointing in the same direction.
My preexisting belief, laid out previously on this blog, is that RSD was a very clever policy innovation for a district that at one point had little more than empty buildings to leverage. I’ve also however long suspected that the notion was over-hyped and ignored some very basic political realities. When for instance Denver announced that yeah, well, about that whole making facilities available thing, we’ve kind of discovered that we are a school district and that our dominant special interests don’t really like this idea after all. The reaction in my tribe was something along the lines of “duh- what did you think was going to happen?”
I’ll go further and say that RSD has a false allure for many of our technocratic friends: things can get better, but only if someone is in charge. Someone like me. True relinquishment means letting go: let operators develop new models, let parents respond to them, let the primordial soup bubble and bake. Don’t assume that you know what constitutes a “high quality education,” don’t aim a narrow vision of quality at a single type of community, allow the interplay of different choice programs to flourish.
Now, having fully confessed my preexisting beliefs, I have offered evidence that I believe confirms them. If someone has countervailing evidence to present, the comment section awaits your thoughtful challenge. If you buy into the harbor-master system, come and defend it.
Jay P. Greene's Blog 
Here’s the money line from this piece: “I’ll go further and say that RSD has a false allure for many of our technocratic friends: things can get better, but only if someone is in charge. Someone like me.”
Few books are as important as Haidt’s for understanding our present situation.
Only about 43% of charters in Louisiana are RSD charters in New Orleans, where the RSD has been most effective. (It’s hard to argue that it has been effective so far outside New Orleans.) We can’t really say too much about NAEP results in New Orleans, since it’s not a TUDA city. But we can look at other citywide results: not just test scores, but graduation rates (50% in 2005 to 76%–5-year rate–in 2015, above the state average) and college-going rates (less than 20% in 2005; 64% in 2016, above the state average). Finally, ACT scores up from 17 in 2005 to 18.8 in 2015, the fastest increase in the state.
Can you name any other large, high-poverty city that outperforms its state on graduation rates and college-going rates? I don’t think there is one.
It’s hard for me to conclude anything but that RSD-NOLA was a dramatic success. RSD in the rest of the state, not so much.
David-
Thank you for your response. I agree that statewide charters are an imperfect proxy for RSD but the % of RSD charters in most states is zero, and the state scores in NOLA have been declining as well. I’m not ready to declare RSD a failure but it seems awkward to tout it as a national model. If it has trouble when going from NOLA to Baton Rouge, what are we to make of it?
It’s a good question, Matthew. Clearly, the presence of a recovery district is not sufficient. In New Orleans, New Schools for New Orleans helped a great deal, TFA helped, TNTP helped, and national funders helped. We also see some of that kind of ecosystem in other cities with strong authorizers and strong charter sectors, such as Boston, Newark, DC, and Indianapolis. New Schools for Baton Rouge is trying to build it in that city, but when the RSD first took over schools in Baton Rouge, it did not exist. And the ASD has struggled in Memphis, without such an ecosystem. There are probably important lessons to be learned there.
But in both Baton Rouge and Memphis, I think we can credit the RSD and ASD with spurring the local district to take significant action to deal with failing schools. I would advise governors and legislators that creating such a lever for change is a wise investment. Carrots are nice, but without a stick they are less effective.
Haidt makes the point that when we confronted with evidence we dislike we ask ourselves “must I believe this?” When however encountering desired information we tend to ask ourselves “can I believe this?” This of course applies to me as much as anyone, but what you are relating seems to confirm a belief I have expressed that if Katrina had destroyed Houston instead of NOLA we would have been out of luck because there are a limited number of TFA kids etc. to throw at a district that is that much larger. Given the overall scarcity of philanthropic dollars it raised the concern in my mind that NOLA RSD may lack “external validity” so to speak.
Now we find ourself confronted with two rough NAEP outings in a row for statewide LA charter schools and declining state scores in NOLA. My inner bias says “can you believe that RSD was a clever policy innovation for post Katrina NOLA but not a general/sustainable/scalable solution?” I get back an answer “Yes very easily.” So see the same data and yet ask yourself “do I need to believe this?” You seem to be leaning “No” but tell me why. I grant that NOLA had nice results initially and I would not want to turn back the reforms but I’m having a hard time seeing RSD as the way of the future. If you lived in one of those other red dots, you might be viewing LA charter policy with a skeptical eye. A Californian for example would have some justification in looking at this data and saying “RSD? No thanks I’m good.”
At least that’s how it seems to me. How does it seem to you?
[…] at Jay’s blog, Jay and Matt wrote two critical posts of portfolio management and harbormasters (our team calls them […]