Advice to the Arnold Foundation

The Laura and John Arnold Foundation is impressive for its intellectual honesty and curiosity.  They have an education reform strategy with which I have some important differences, but they are nevertheless interested in hearing criticism of their approach, so they invited me to present my critique to their board.  Below is the essence of what I prepared for that meeting.  I don’t expect that this will cause them to alter course, nor should it.  It’s their money and they should do whatever they think best.  But the amazing thing about the Arnolds and the head of their education effort, Neerav Kingsland,  is that they are at least open to the possibility of being wrong and want to hear criticism in case they would like to reconsider any aspects of their strategy.

The heart of the Arnold reform strategy is Portfolio Management, which is a term with which they are not enamored, but is essentially a rapid expansion of choices across different sectors with a centralized and muscular system for engaging in quality control.  The Portfolio Manager would govern schools of all types in a location — traditional, charter, and perhaps private — and select which schools should be allowed to operate, which should be closed, and police certain aspects of their operations, including admissions, transportation, and perhaps special education, discipline, and other issues.  I’m a fan of the rapid expansion of choices, but I believe that the centralized and muscular quality control system produces significant educational and political damage.  I am not suggesting that the Arnold Foundation (or the charter movement in general ) abandon all quality control efforts, but I think quality is best promoted by relying heavily on parent judgement and otherwise relying on a decentralized system of authorizers with the most contextual information to make decisions about opening and closing schools if parents seem to have difficulty assessing quality on their own.  The problem with Portfolio Management is the centralized and overly-active nature of a single quality-control entity.  Here is my case in 7 points:

  1. Conceptually, Portfolio Management is no different from School Districts, so there is no reason to expect it to be any better at quality control than School Districts are.  People often claim that PM is different because its mission is only to maintain the quality of the portfolio of schools, not interfere in their operations.  But social scientists think about organizations based on their powers and incentives, not their mission.  Regardless of what PM is supposed to do, we should focus on what it can do and what is in its interest to do.  There is nothing that a PM can do or should want to do, given its organizational interests, that is not also the case for a School District.  PMs can open and close schools, just like  School Districts do.  PMs can set policies that affect school operations, just like School Districts do.  Remember that PMs have already crept into setting policies about admissions, transportation, special education, and discipline — all of which affect school operations.  And the types of schools they decide to let open or force to close shape the curriculum and pedagogy of those schools.  Also remember that School District boards do not actually operate schools, just like PMs don’t.   School District boards just set policies and decide which schools should open and close, just like PMs.  Given that they have the same powers and organizational interests, the only difference I can see between PM and School District boards is that the PM is imagined to be a good guy, who will properly be motivated by quality and avoid interfering unproductively in school operations, while School District board members (even if appointed) are imagined to be bad guys who are more concerned with satisfying special interests and following procedures than with school quality.
  2. Even if you can manage to get a PM system in place (and there are very few), and even if you manage to get “good guys” in charge of it, the good guys won’t stay in charge for very long.  The poster boy for PM, New Orleans, with its exceptional hurricane origin and large charter sector to advocate on its behalf, reverted to control by the previously reviled and inept locally elected school board in about a decade.  Yes, there is a law that swears that the school board now serving as PM will not interfere in charter operations, but these oaths of non-interference hardly provide any protection.  As discussed in 1., PMs already take actions that affect school operations by regulating their admissions, transportation, special education, and discipline.  And their ability to open and close schools can effectively control any other aspect of school operations that they wish.
  3. Even if you can get PM and keep the “good guys” in charge for longer than a decade, the PM is unable to effectively control quality because it has no tools that reliable identify quality.  Basically, the only tool available is the level of test scores.  We all fantasize about a world in which student learning growth on math and reading tests is calculated and used by central authorities to judge quality, but the reality is that very few school systems actually rely heavily on value-added measures (VAM).  In New Orleans, for example, only 5% of the school quality grade is based on VAM.  The rest is based on the level of student performance.  No reasonable person believes that the level of student performance is a reliable proxy for school quality.  Instead, the level of performance is largely a function of the severity of disadvantage among the students.  And yet the PM in New Orleans is making judgments about school closure based on a flawed measure that effectively punishes schools for trying to serve a high concentration of kids who are too disadvantaged.  And even in the imaginary world in which VAM is used, learning growth on math and reading tests only captures a narrow portion of school quality, which is why those measures are not consistent predictors of later life outcomes, like graduation, college attendance, and earnings.  As I’ve written before, you can’t manage quality if you can’t predict it, and PM does not possess any tools to reliably predict school quality.
  4. Even if we thought test score levels or the imaginary future of VAM were good enough for PMs to manage the quality of their portfolio, the heavy reliance on those measures distorts schools in ways that are educationally harmful.  To avoid the risk of being judged low quality, schools will tend to narrow their curriculum to tested subjects and even within those subjects focus more narrowly on tested items.  Other subjects and non-tested material can produce important benefits for students, but PM provides incentives for schools to neglect those benefits.  The result is that we get a homogeneous set of schools that are narrowly focused on improving test outcomes.  That kind of school might be good for some kids, but is certainly not good for all.
  5. The homogeneous set of school options that results from PM is very unlikely to offer anything that appeals to more advantaged families.  It basically results in a charter movement that is designed to serve certain urban students with no-excuses-type schools.  Suburban and more advantaged families have no interest in this kind of schooling for their own children.  By failing to offer more advantaged families any benefits, the charter movement then loses their political support, and advantaged families have much more political power than disadvantaged families.  PM advocates seem to have forgotten that politics is driven largely by self-interest manifested in organized groups.  The crushing defeat of the charter referendum in Massachusetts is at least partially explained by the political foolishness of narrowly focusing the charter movement on a certain type of school to serve disadvantaged students.  No matter what science you present to prove that those schools are good and no matter what appeals to justice you make, advantaged families will not support a movement that poses any risks to their own children and offers them no benefits.
  6. In addition to alienating advantaged, mostly white, families, PM has also alienated minority community leaders.  First, a centralized and muscular system of quality control, like PM, that is only established in urban districts clearly communicates to minority communities a lack of trust in their ability to judge quality as parents or even to judge it as decentralized charter authorizers.  It effectively says that suburbanites can choose whatever they like, but folks in big cities can’t be trusted.  Even worse, raising the barrier to entry for operating a charter school (without actually improving quality, as we already discussed) disproportionately excludes minority community leaders from operating charter schools.  It is the same principle as occupational licensure.  Things that make it hard to enter and have nothing to do with quality typically have the effect of keeping minorities and more disadvantaged people out.  Not surprisingly, we get a charter sector that is largely developed and run by white folks from elite college.  If Rev. Johnson would like to open a charter school in the classrooms he uses for Sunday school, he will have a particularly hard time completing the 700 page application in a way that satisfies the PM’s rubrics and his educational background may not appear as impressive.  But he may know his community well and his political support could be helpful.  Centralized and muscular quality control, like PM, tends to turn these folks away.
  7. The Arnold Foundation invests heavily in another initiative that promotes rigorous science for medical and policy decision-making, yet they do not seem to apply that same standard of proof to their own education strategy.  When pressed, the main evidence they point to in support of PM is a study by Doug Harris that shows that New Orleans made significant gains post-Katrina that cannot fully be explained by changes in the composition of students in the district.  Even if true, however, that study cannot tell us what New Orleans did to produce this improvement.  Perhaps the huge expansion in school choice deserves the credit and the muscular quality control added no benefit or even hurt.  Perhaps New Orleans produced gains because it imported a small army of elite college kids, greatly increasing human capital in the school system.  Maybe the large increase in spending in New Orleans deserves some of the credit.  The point is that attributing the gain to PM is unscientific, since Harris’ research was not designed to address this question.  Saying that we should pursue PM nationwide because New Orleans has it and has improved is roughly the equivalent of saying that you should wear copper bracelets because I wore them and my arthritis feel much better.  If someone made the later claim, the Arnold Foundation would (rightly) scoff at them as quacks.  I understand that foundations cannot have rigorous evidence to support all steps in their reform theory, but before pursuing a reform strategy that promises to close a bunch of schools that parents want and alienating both advantaged suburbanites and minority community leaders, they might want more evidence for the strategy than they have.

10 Responses to Advice to the Arnold Foundation

  1. Mike G says:

    1. Props to Arnolds/Neerav for inviting a critic in.

    2. Props to Jay for clearly laying out his case.

    My questions:

    3. What if, in certain places, there are just 2 politically viable options:

    a. status quo

    b. expand choice thru a NOLA-like PM system; hope to ride “the good guys” for as long as they can hold out. With the notion that if it turns out parents prefer “lots of options” to “neighborhood schools”, they’ll generally protect this version of choice.

    In that scenario, would you take “b”?

    4. As a political scientist or perhaps just as an ed reform observer, do you think that sometimes my description is fair, that in some places “c” (expanded choice, much lighter regulation) is not viable?

    • Thanks, Mike!

      I am a big believer that choice is a self-sustaining reform in that the recipients have strong incentives to advocate for the continuation and expansion of choice programs. This is especially true if choice programs also benefit some advantaged families who are in a better position to organize and advocate on behalf of choice.

      The educational and political damage done by centralized, muscular quality control is largely self-inflicted by reformers. not the result of compromises with opponents or undecideds. Just look at Milwaukee or Massachusetts to see how little political support was actually gained by self-injurious policies, like focusing programs on the most disadvantaged, producing a homogeneous set of schools focused on test scores, turning away minority community leaders, etc… I might be able to tolerate the defects of the PM approach if it actually was necessary for gaining enough support to get choice at all. Instead in places like New Orleans, in the Louisiana Scholarship Program, and elsewhere, centralized and muscular quality control was the opening offer from reformers, not the result of a compromise to gain more support. And this top-down quality control tends to be the opening offer because reformers have worked themselves into believing that it is actually the best policy, not a barely tolerable compromise as you suggest in your question.

      I agree that getting choice programs adopted is tough. But we have to be careful and not give away too much during that founding or we jeopardize the long-term educational and political success of those programs and the entire choice movement.

      • Mike G says:

        Thanks for the thoughtful response, Jay.

        I would add one small complicating “market” aspect, at least in Massachusetts: the supply of founders who wanted to create schools that served disadvantaged kids “organically” exceeded the supply of those who wanted to start suburban charters.

        I’d concede that there wasn’t a robust effort to “find” suburban school founders, to the best of my knowledge.

        But I’d suggest that perhaps part of this story is “founder motivation” which is “bottom-up,” along with “reformer strategy” that is top-down…

      • The supply of charters seeking to serve disadvantaged kids is almost entirely driven by foundation subsidies, talent pipelines, and centralized authorizer preferences. Start-up funds for schools geared toward the disadvantaged are much more readily available from foundations than for other types of schools. The TFA talent pipeline that has been built and heavily subsidized by foundations churns out a steady flow of elite college kids who want to found the next KIPP. And everyone knows that it is much easier to get a charter approved by the state authorizer if it targets the disadvantaged.

        If you want to see what the world looks like without these efforts to boost the supply of charters focused on the disadvantaged, look at AZ. Foundations are not active there to subsidize start-ups, so you have more innovative charters, including those that might appeal to the more advantaged. The TFA talent pipeline does not flow much in uncool places like AZ, so most of the charter proposals come from non-TFA types. And a looser authorizing process makes it less risky to propose charters not narrowly focused on serving the disadvantaged. The result is a nicely diverse set of charter schools with offerings that appeal to both advantaged and disadvantaged families.

      • Mike G says:

        Hi Jay –

        Thanks for your most recent comment. Agree on AZ charters.

        Before I deleted it, I’d written “Circa 1999” in my comment about Massachusetts charters. Wish I’d included it for clarity.

        I.e., the founders around that time…none of us came from TFA, to the best of my knowledge. The more common story was someone who was intrigued by education policy to help poor kids, saw the challenges of district reform, and got interested in charters as an alternative to reach the same kids.

      • I see. But even in 1999 donor subsidies and authorizer preferences were steering charters toward a narrow focus on disadvantaged students in places like MA. AZ was headed toward a different type of supply even back in 1999.

        We (the reform movement) made choices that gave us these problems and we can make different choices to start getting out of them. The damage has largely been self-inflicted, but at least we can stop hurting ourselves (politically and educationally).

  2. Greg Forster says:

    Other than that, Mrs. Lincoln, how was the portfolio management?

    One quibble:

    “But social scientists think about organizations based on their powers and incentives, not their mission.”

    That is excessively functionalist. Better to say that 1) social scientists think about organizations based on powers and incentives in addition to their missions, and 2) the mission of school districts is, de jure, the same as that of portfolio management, so if districts haven’t been sufficient what’s better about portfolio management?

  3. Robert Sommers says:

    Excellent treatise. We are facing the reality that we have made little to no progress in education with the reforms we’ve put in place. Time for a new set of efforts. Learning to Improve is an interesting starting point for me.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s