Crash Go the Harbormasters?

April 23, 2018

(Guest Post by Matthew Ladner)

When I was a kid in the 1970s someone wrote a book about UFOs having visited earth in the past called Chariots of the Gods. The only thing I remember about this book was a photo of an ancient stone staircase and a caption that read “Staircase going down-to a spaceport?” Someone else wrote a book called “Crash Go the Chariots.” I never examined this book, but I imagine that it was a bit like shooting fish in a barrel.

I am in the midst of reading Jonathan Haidt’s The Righteous Mind: Why Good People are Divided by Politics and Religion and found the following passage quite striking. The book summarizes a great deal of psychological research to reach this point which I cannot do justice to, but roll with this in any case:

In the same way, each individual reasoner is really good at one thing: finding evidence to support the position that he or she already holds, usually for intuitive reasons. We should not expect individuals to produce good, open-minded, truth-seeking reasoning, particularly when self-interest or reputational concerns are in play. 

Sounds dismal so far, but fortunately he goes on…

But if you put individuals together in the right way, such that some individuals can use their reasoning powers to disconfirm the claims of others, and all individuals feel some common bond or shared fate that allows them to interact civilly, you can create a group that ends up producing good reasoning as an emergent property of the social system. This is why it is so important to have intellectual and ideological diversity within any group or institution whose goal is to find truth (such as an intelligence agency or community of scientists) or to produce good public policy (such as a legislature or an advisory board).

Again, I can’t summarize the research that lead to this conclusion (read the book and see what you think) but basically Haidt lays out a case that says if you believe that you are some sort of scientist guided solely by sweet reason and evidence, you are the victim of self-delusion. Good reasoning happens through a social challenge process over time.

In that spirit of civil challenge, I’ll offer the following two charts. The first a scatterplot of the 8th grade reading 2017 scores by gains (2017 minus 2009 scores) for all 50 states and all 16 state charter sectors with scores in both 2017 and 2009.

There is a huge amount of success to celebrate in this chart for multiple state charter sectors, most of which have either unusually large gains, or unusually high scores, or else unusually high scores and gains.

Do however take note of Louisiana’s charter sector position in the above chart. Now let’s look at the same chart for mathematics:

Again, there is a great deal to like here- multiple charter sectors with large gains (MI, WI, GA, MD, TX) or high scores (CO, ID) or both high scores and gains. Arizona, this one is for you:

Again however take a look at Louisiana’s charter sector-again low scores and low gains. A quick trip to the NAEP data explorer will demonstrate that this same trend is evident in 4th grade NAEP scores, and state scores have been trending in a negative fashion in recent years as well.

In recent years a school of thought arose in our space that a centralized authority or “harbor-master” could produce better outcomes by carefully controlling both the entrance and the exit of schools from charter sectors, primarily on the basis of standardized test scores. In the case of the Recovery School District in Louisiana, there was a period of strong gains in state test scores to buttress this belief. State test scores however have repeatedly shown themselves vulnerable to gaming and manipulation, especially when large consequences ride on them. Soviet steel makers reported non-stop prodigious growth in steel production for decades when facing such incentives as well. How much steel was actually made-who knows? Educators however have neither the incentive or ability to “teach to” or otherwise game the NAEP exams, which is why they have long stood as an external audit on state testing.

In this case, NAEP and state tests are both pointing in the same direction.

My preexisting belief, laid out previously on this blog, is that RSD was a very clever policy innovation for a district that at one point had little more than empty buildings to leverage. I’ve also however long suspected that the notion was over-hyped and ignored some very basic political realities. When for instance Denver announced that yeah, well, about that whole making facilities available thing, we’ve kind of discovered that we are a school district and that our dominant special interests don’t really like this idea after all. The reaction in my tribe was something along the lines of “duh- what did you think was going to happen?”

I’ll go further and say that RSD has a false allure for many of our technocratic friends: things can get better, but only if someone is in charge. Someone like me. True relinquishment means letting go: let operators develop new models, let parents respond to them, let the primordial soup bubble and bake. Don’t assume that you know what constitutes a “high quality education,” don’t aim a narrow vision of quality at a single type of community, allow the interplay of different choice programs to flourish.

Now, having fully confessed my preexisting beliefs, I have offered evidence that I believe confirms them. If someone has countervailing evidence to present, the comment section awaits your thoughtful challenge. If you buy into the harbor-master system, come and defend it.



District/Charter Combos that Outperform RSD

February 19, 2018

(Guest Post by Matthew Ladner)

Just out of curiosity, I decided to look into the Reardon data to see how many Arizona district/charter combos outperformed or tied the Recovery School District in academic growth. The above list is by no means exhaustive, more the product of throwing in some district names into the data base before my morning caffeine. The list is not short, and includes several very poor and isolated school districts.

Mind you that no one bombed any of these districts with millions in philanthropy. You can’t go to a bar in Snowflake Arizona and meet a group of six Teach for America students the way you can in the French Quarter. They also managed this somehow without a “harbor-master.” Finally, the Reardon data ends in 2015, and since then the trend in statewide data in Arizona has been positive, but in the RSD not so much. Hopefully Reardon will update his study so we can track this over time.

I want to be careful to note that I regard the Recovery School District to have been a very clever innovation for a district that had almost nothing to leverage but empty school buildings after a hurricane. If that hurricane however had leveled Houston or Dade County I’m afraid that the limited supply of philanthropic dollars and TFA teachers would have been unequal to the (much larger) task. In order to reach scale, we are going to need solutions that do without substantial infusions of outside money, as that is likely to be in increasing short supply.

Having said that, RSD landing in the 92nd percentile in growth in the Reardon data was truly a magnificent accomplishment. The leap however from “well done” to “everyone needs to do this now!!!!” looks very dangerous imo.