Jay P. Greene's Blog

The Laura and John Arnold Foundation is impressive for its intellectual honesty and curiosity.  They have an education reform strategy with which I have some important differences, but they are nevertheless interested in hearing criticism of their approach, so they invited me to present my critique to their board.  Below is the essence of what I prepared for that meeting.  I don’t expect that this will cause them to alter course, nor should it.  It’s their money and they should do whatever they think best.  But the amazing thing about the Arnolds and the head of their education effort, Neerav Kingsland,  is that they are at least open to the possibility of being wrong and want to hear criticism in case they would like to reconsider any aspects of their strategy.

The heart of the Arnold reform strategy is Portfolio Management, which is a term with which they are not enamored, but is essentially a rapid expansion of choices across different sectors with a centralized and muscular system for engaging in quality control.  The Portfolio Manager would govern schools of all types in a location — traditional, charter, and perhaps private — and select which schools should be allowed to operate, which should be closed, and police certain aspects of their operations, including admissions, transportation, and perhaps special education, discipline, and other issues.  I’m a fan of the rapid expansion of choices, but I believe that the centralized and muscular quality control system produces significant educational and political damage.  I am not suggesting that the Arnold Foundation (or the charter movement in general ) abandon all quality control efforts, but I think quality is best promoted by relying heavily on parent judgement and otherwise relying on a decentralized system of authorizers with the most contextual information to make decisions about opening and closing schools if parents seem to have difficulty assessing quality on their own.  The problem with Portfolio Management is the centralized and overly-active nature of a single quality-control entity.  Here is my case in 7 points:

1. Conceptually, Portfolio Management is no different from School Districts, so there is no reason to expect it to be any better at quality control than School Districts are.  People often claim that PM is different because its mission is only to maintain the quality of the portfolio of schools, not interfere in their operations.  But social scientists think about organizations based on their powers and incentives, not their mission.  Regardless of what PM is supposed to do, we should focus on what it can do and what is in its interest to do.  There is nothing that a PM can do or should want to do, given its organizational interests, that is not also the case for a School District.  PMs can open and close schools, just like  School Districts do.  PMs can set policies that affect school operations, just like School Districts do.  Remember that PMs have already crept into setting policies about admissions, transportation, special education, and discipline — all of which affect school operations.  And the types of schools they decide to let open or force to close shape the curriculum and pedagogy of those schools.  Also remember that School District boards do not actually operate schools, just like PMs don’t.   School District boards just set policies and decide which schools should open and close, just like PMs.  Given that they have the same powers and organizational interests, the only difference I can see between PM and School District boards is that the PM is imagined to be a good guy, who will properly be motivated by quality and avoid interfering unproductively in school operations, while School District board members (even if appointed) are imagined to be bad guys who are more concerned with satisfying special interests and following procedures than with school quality.
2. Even if you can manage to get a PM system in place (and there are very few), and even if you manage to get “good guys” in charge of it, the good guys won’t stay in charge for very long.  The poster boy for PM, New Orleans, with its exceptional hurricane origin and large charter sector to advocate on its behalf, reverted to control by the previously reviled and inept locally elected school board in about a decade.  Yes, there is a law that swears that the school board now serving as PM will not interfere in charter operations, but these oaths of non-interference hardly provide any protection.  As discussed in 1., PMs already take actions that affect school operations by regulating their admissions, transportation, special education, and discipline.  And their ability to open and close schools can effectively control any other aspect of school operations that they wish.
3. Even if you can get PM and keep the “good guys” in charge for longer than a decade, the PM is unable to effectively control quality because it has no tools that reliable identify quality.  Basically, the only tool available is the level of test scores.  We all fantasize about a world in which student learning growth on math and reading tests is calculated and used by central authorities to judge quality, but the reality is that very few school systems actually rely heavily on value-added measures (VAM).  In New Orleans, for example, only 5% of the school quality grade is based on VAM.  The rest is based on the level of student performance.  No reasonable person believes that the level of student performance is a reliable proxy for school quality.  Instead, the level of performance is largely a function of the severity of disadvantage among the students.  And yet the PM in New Orleans is making judgments about school closure based on a flawed measure that effectively punishes schools for trying to serve a high concentration of kids who are too disadvantaged.  And even in the imaginary world in which VAM is used, learning growth on math and reading tests only captures a narrow portion of school quality, which is why those measures are not consistent predictors of later life outcomes, like graduation, college attendance, and earnings.  As I’ve written before, you can’t manage quality if you can’t predict it, and PM does not possess any tools to reliably predict school quality.
4. Even if we thought test score levels or the imaginary future of VAM were good enough for PMs to manage the quality of their portfolio, the heavy reliance on those measures distorts schools in ways that are educationally harmful.  To avoid the risk of being judged low quality, schools will tend to narrow their curriculum to tested subjects and even within those subjects focus more narrowly on tested items.  Other subjects and non-tested material can produce important benefits for students, but PM provides incentives for schools to neglect those benefits.  The result is that we get a homogeneous set of schools that are narrowly focused on improving test outcomes.  That kind of school might be good for some kids, but is certainly not good for all.
5. The homogeneous set of school options that results from PM is very unlikely to offer anything that appeals to more advantaged families.  It basically results in a charter movement that is designed to serve certain urban students with no-excuses-type schools.  Suburban and more advantaged families have no interest in this kind of schooling for their own children.  By failing to offer more advantaged families any benefits, the charter movement then loses their political support, and advantaged families have much more political power than disadvantaged families.  PM advocates seem to have forgotten that politics is driven largely by self-interest manifested in organized groups.  The crushing defeat of the charter referendum in Massachusetts is at least partially explained by the political foolishness of narrowly focusing the charter movement on a certain type of school to serve disadvantaged students.  No matter what science you present to prove that those schools are good and no matter what appeals to justice you make, advantaged families will not support a movement that poses any risks to their own children and offers them no benefits.
6. In addition to alienating advantaged, mostly white, families, PM has also alienated minority community leaders.  First, a centralized and muscular system of quality control, like PM, that is only established in urban districts clearly communicates to minority communities a lack of trust in their ability to judge quality as parents or even to judge it as decentralized charter authorizers.  It effectively says that suburbanites can choose whatever they like, but folks in big cities can’t be trusted.  Even worse, raising the barrier to entry for operating a charter school (without actually improving quality, as we already discussed) disproportionately excludes minority community leaders from operating charter schools.  It is the same principle as occupational licensure.  Things that make it hard to enter and have nothing to do with quality typically have the effect of keeping minorities and more disadvantaged people out.  Not surprisingly, we get a charter sector that is largely developed and run by white folks from elite college.  If Rev. Johnson would like to open a charter school in the classrooms he uses for Sunday school, he will have a particularly hard time completing the 700 page application in a way that satisfies the PM’s rubrics and his educational background may not appear as impressive.  But he may know his community well and his political support could be helpful.  Centralized and muscular quality control, like PM, tends to turn these folks away.
7. The Arnold Foundation invests heavily in another initiative that promotes rigorous science for medical and policy decision-making, yet they do not seem to apply that same standard of proof to their own education strategy.  When pressed, the main evidence they point to in support of PM is a study by Doug Harris that shows that New Orleans made significant gains post-Katrina that cannot fully be explained by changes in the composition of students in the district.  Even if true, however, that study cannot tell us what New Orleans did to produce this improvement.  Perhaps the huge expansion in school choice deserves the credit and the muscular quality control added no benefit or even hurt.  Perhaps New Orleans produced gains because it imported a small army of elite college kids, greatly increasing human capital in the school system.  Maybe the large increase in spending in New Orleans deserves some of the credit.  The point is that attributing the gain to PM is unscientific, since Harris’ research was not designed to address this question.  Saying that we should pursue PM nationwide because New Orleans has it and has improved is roughly the equivalent of saying that you should wear copper bracelets because I wore them and my arthritis feel much better.  If someone made the later claim, the Arnold Foundation would (rightly) scoff at them as quacks.  I understand that foundations cannot have rigorous evidence to support all steps in their reform theory, but before pursuing a reform strategy that promises to close a bunch of schools that parents want and alienating both advantaged suburbanites and minority community leaders, they might want more evidence for the strategy than they have.

This entry was posted on Tuesday, October 3rd, 2017 at 3:39 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.