Jay P. Greene's Blog

(Guest Post by Matthew Ladner)

Neerav Kingsland has written a response to posts by Jay and yours truly on Louisiana charters generally and the Recovery School District specifically. I will begin by confessing a sin- Neerav is correct that I go a bit over the top at times. The “Prime Directive” of JPGB has always been first and foremost for the authors to entertain ourselves. We do occasionally take ideas seriously, but we try to keep things light around here, which often involves reasoning by pop-culture analogy. Maybe some gratuitous use of an animated gif here or there, or an occasional musical interlude. All in moderation of course…

Like Leo Moracchioli we try not to take ourselves too seriously, but one idea that I do take very seriously is the one quoted in the original post from Jonathan Haidt, which merits repetition:

In the same way, each individual reasoner is really good at one thing: finding evidence to support the position that he or she already holds, usually for intuitive reasons. We should not expect individuals to produce good, open-minded, truth-seeking reasoning, particularly when self-interest or reputational concerns are in play. 

But if you put individuals together in the right way, such that some individuals can use their reasoning powers to disconfirm the claims of others, and all individuals feel some common bond or shared fate that allows them to interact civilly, you can create a group that ends up producing good reasoning as an emergent property of the social system. This is why it is so important to have intellectual and ideological diversity within any group or institution whose goal is to find truth (such as an intelligence agency or community of scientists) or to produce good public policy (such as a legislature or an advisory board).

I do not wish to allow even the JPGB Prime Directive to interfere in this. I appreciate that David Osborne offered some comments, and that Neerav also took the time to respond in a civil fashion. I offer an apology if my bombast lacked civility as it must not interfere in the free exchange of evidence and ideas.

On the substance, I’ll offer the following comments in the spirit of the Haidt quote:

My preexisting bias before the release of the 2017 NAEP was that the Louisiana RSD was a clever policy innovation given the circumstances of post-Katrina New Orleans, but that the concept employed enormous amounts of financial and human capital. Perhaps too much of both to be of general interest.

David Osborne noted in a comment that only 43% of Louisiana charters are New Orleans RSD charters, which is a fair point to make. This however is about 43% more than the typical state, and a portion of the rest Louisiana’s charters are RSD charters operating outside New Orleans. Osborne noted in a comment that those charters aren’t going so well. In my preexisting frame, I interpret this as RSD not being able to make the trip down the Atchafalaya Bridge from New Orleans to Baton Rouge, much less to other states. This seems like confirmatory evidence to me, but maybe not. If not, why not?

NAEP data is still indeed inexact on this point, but state testing data for New Orleans specifically has also been in decline. NAEP shows large statewide declines in charter scores since 2013, and the state’s own testing data pointing in the same direction specifically for New Orleans. 

What sort of evidence would lead a reasonable person to conclude that the RSD model is very difficult to replicate and to sustain?  We all have theories of change, but is this one falsifiable? I believe that the combination of statewide NAEP scores and the decline of state test scores in New Orleans is an issue. I’m in favor of RSD continuing in New Orleans, but nothing about the evidence produced last few years is giving me the itch to replicate it in my home town.

Speaking of Phoenix, Neerav (correctly) noted a Credo multivariate study showing meh charter results in Arizona. Harvard’s Marty West found similar results, but both Credo and West used data that ended in 2012. I won’t go into the details here but I believe Credo, West, recent NAEP and recent AZMerit can all be correct. The 2012 Philadelphia Eagles went 4-12 but they won the Superbowl last season. Between lots of openings, closings, and schools maturing past their training wheels stage, AZ charters turned over their roster like a pro-sports team.

RSD advocates have a theory of change largely based upon increased state test scores in New Orleans. The more recent state data and NAEP however both seem to be signalling a warning sign. If this is just my preconceived notions getting the better of me, please help me out. Nevermind Arizona, based on anything and everything we can gather from Credo, NAEP, recent state scores and even random assignment studies, why should Louisiana charter policies be touted in preference to Michigan’s based upon the available results?