Nudge interventions, in which students receive texts encouraging them to do things that are thought to be good for them, have yielded another disappointing result. In a newly released study, Kelli A. Bird, Benjamin L. Castleman, Jeffrey T. Denning, Joshua Goodman, Cait Lamberton, and Kelly Ochs Rosinger report:
We investigate, through two randomized controlled trials, the impact of a national and state-level campaign to encourage students to apply for financial aid for college. The campaigns collectively reached over 800,000 students, with multiple treatment arms to investigate different potential mechanisms. We find no impacts on financial aid receipt or college enrollment overall or for any student subgroups. We find no evidence that different approaches to message framing, delivery, or timing, or access to one-on-one advising affected campaign efficacy.
For those keeping tabs, I expressed my strong doubts about texting in this review of Ben Castleman’s book even as Bill Gates was praising texting interventions and NPR’s Hidden Brain was featuring it as a success. And then my skepticism was strengthened when texting resulted in null to negative college completion outcomes in a study that received little attention. Now even its early proponents are finding disappointing results.
But this isn’t the first time I’ve predicted that a reform initiative would fail when it would take foundations years and millions of dollars before reaching the same effective conclusion. I was raising alarms about the Measuring Effective Teacher (MET) initiative while almost everyone was jumping on the teacher evaluation and quality bandwagon. Several years later and without much fanfare we finally hear that the initiative yielded virtually no benefits. Similarly, when a large chunk of reformers were getting behind no-excuse charter schools as the correct school model, I was warning that the model appeared unable to yield longer-term benefits, so we might not want to put all of our eggs (and all students) into the no-excuse basket. And recently, we’ve seen that Boston no-excuse charter schools have no effect on students completing college. Lastly, I’ve been predicting the political and educational failure of Portfolio Management for some time now. While the Arnold Foundation has doubled down by joining Reed Hastings in giving $200 million to the City Fund to push the idea, I expect it will be shortly after they burn through that money that we’ll hear about a reconsideration of their reform strategy.
Why am I able to anticipate these failures in education reform initiatives, while the people devoting fortunes to these efforts and their staff have such a hard time avoiding strategies that result in failure? I’m not that smart and they aren’t that dumb. I suspect the answer is that foundations have organizational interests that tend to draw them to a mistaken theory about education policy. In its essence, that theory holds that there are policy interventions that could improve outcomes for large numbers of students if only we could discover them and get policymakers and practitioners to adopt them at scale.
I begin with a very different theory. I suspect that there are relatively few educational practices that would produce uniformly positive results. Instead, I’m inclined to think of education as similar to parenting, in which the correct approaches are highly context-specific. Even within the same family, we may choose to parent different children facing similar issues in very different ways. There may be some uniformly desirable parenting practices, but most of them are already known and widely disseminated. So, if we wanted to improve parenting, the best we could do would be to empower parents to be in a better position to judge their context and make their own decisions about how to raise their children. Similarly, the best we could do to improve education is to empower families and communities to make decisions within their own context. There is relatively little we could tell all schools or educators to do to improve outcomes.
But foundations and the research community that follows their lead have a very hard time with this kind of theory. They think their job is to use science to identify the correct educational practices and then get everyone on board with doing it. To decentralize solutions to communities and families is to relinquish control and the status of expertise. So, despite repeated failures in finding top-down policies and practices to improve outcomes at scale, they continue to search for what can’t be found.
As Mike McShane and I pointed out in our edited volume, there is nothing inherently wrong with failure in education policy. The problem is if we refuse to acknowledge that failure and learn from it. Given the string of reform failures that large foundations have experienced over the past decade, it might be productive if they devoted some serious energy to reconsidering their basic assumptions about top-down policy solutions and became more open to the possibility that the most they could do is to empower communities and families to find their own solutions.
Jay P. Greene's Blog 
Bravo, Jay.
The data fit your hypothesis, but your causal model is
undertheorized. How would you distinguish the foundations being “dumb” from their having “organizational interests that tend to draw them to a mistaken theory about education policy”?
Your description of the foundations (they “have a very hard time with this kind of theory,” they “think their job is” something else) doesn’t suggest the problem is “organizational interests.” It suggests the problem is, if not that they’re “dumb” in the sense of having low IQ, they’re foolish and cling stubbornly to a blinkered ideology.
My working theory is that a large part of the problem is that grantmakers actually do not have “organizational interests“ as that term is commonly understood. Businesses have to keep customers happy, schools have to keep students and faculty happy, politicians have to keep voters happy. But grantmakers don’t have to keep anyone happy. No doubt the individual staff have their own motives. But in most circumstances (there are exceptions) the organization itself loses nothing no matter how poorly it performs, and gains nothing no matter how well it performs. This helps explain why they’re all so dysfunctional.
https://www.jamesgmartin.center/2019/08/the-truth-teller/
A lot of researchers/educators/others need to explain to the public, especially parents, why the public schools are so dysfunctional and what can be done to compensate their children for the damage done to them. And what today’s foundations and others would propose in response to the two best studies on low achievement ever done: in 1965 and 1966.
https://www.jamesgmartin.center/2019/08/the-truth-teller/
Researchers and others need to explain to the public, especially parents of schoolchildren in the last decade, what could be done to compensate them for the damage done to their educations. Sandra
This is an interesting argument, and I appreciate your sharing it. However, I’m left with questions: What does it look like to “empower communities and families to find their own solutions”? What is the role of each level of government? What does the funding structure look like?
Thanks for your time.
Just as foundation officials as well as state and federal policymakers are poorly situated to know what every family, teacher, and school should be doing, I’m in no position to describe in detail the optimal arrangements for the roles of every actor in education policymaking. I think it is sufficient for me to say that more responsibility for making education decisions should be delegated to the families, teachers, and schools that directly bear the consequences of those decisions.
Thanks for replying. I want to push back not to be antagonistic but out of genuine curiosity. In your blog entry, you use the specific word “empower.” Does that mean simply “delegating responsibility”? If so, does the federal government have any role in education? And does the federal government provide any funding?
Thanks for your time.