Childhood Trauma and School Choice

September 9, 2019


(Guest post by Greg Forster)

Opinion leaders in Oklahoma are orchestrating a full-court press to use remediation of “childhood trauma” as the next narrative under which they should get big, indiscriminate spending increases without accountability for results. As someone who had a traumatic childhood myself, I’m all in favor of expanding access to effective services in this area. That’s why I’m against big, indiscriminate spending increases without accountability, and in favor of school choice, which actually helps kids with traumatic experiences.

OCPA carries my article:

The usual reply to concerns such as this—other than “you’re cruel and heartless for asking whether the money we spend on helping people actually helps people”—is to lament how hard it is to have a positive impact on such intractable problems. When children are abused or mistreated, or have mental health disorders or other traumatic problems, they lose their proper chance to grow in their human potential. Getting wounded people on the road to recovery and growth is very, very hard.

That is all true. It is not, however, a reason to spend large amounts of money without accountability. It is a reason to look for more promising policy approaches.

There follows a recitation of findings on how school choice decreases rates of childhood trauma and provides a more supportive and effectively nurturing school environment.

I’m also unscrupulous enough to remind everyone of how poorly the Oklahoma government school system handled some notorious cases of childhood trauma just last year:

Last year, a number of cases came to light in Oklahoma in which minority students were being targeted by racist bullies. One family was finally given permission—permission!—to transfer their child out of the school where he was being constantly persecuted. Untold thousands more children continue to suffer in their assigned government schools, whether because of racist bullying or whatever other adversity they experience, because they didn’t happen to catch the attention of the media and make the system look bad.

Why on earth should one family get permission to choose, and not everyone else? Why should even that one family have had to go begging to the powerful and get permission to protect their own child? Did we lose a war?

Yes, by all means let’s throw more money at the unaccountable government monopoly system as the only permissible defense for children experiencing trauma!

I promise not to be traumatized if you let me know what you think.

Op-Ed in Houston Chronicle Against State Takeover of Houston ISD

September 5, 2019

Josh McGee and I have an oped in the Houston Chronicle today arguing against state takeovers.  Here’s the money quote:

State policymakers may imagine that they are smarter and better than the elected officials they would displace, but, even if they were right, the intelligence and goodness of the school board is hardly the issue. Distant and unaccountable bureaucrats, no matter how well-trained and well-intentioned, are unlikely to understand and address the specific needs of communities as well as locally elected officials are, no matter how fractious and chaotic they may appear. Conservatives have long understood this principle, which is why they have traditionally supported decentralization of responsibility over schools to local governments, communities, and families, so it is puzzling that self-styled conservatives in Texas would support state takeovers.

There is no simple solution to chronic low academic performance, but the problem is almost certainly better addressed by empowering communities and families rather than disenfranchising them.

This is a mistake reformers have made over and over.  It’s time we learn from our mistakes and avoid repeating them.

Reinventing Education Schools

August 21, 2019


(Guest post by Greg Forster)

OCPA has published my in-depth policy brief on why efforts to “reform” education schools don’t go deep enough – we need to reinvent them. It will take a generation and the initial policy changes required are politically difficult, but lesser reforms aren’t enough:

These problems do not arise merely from post-1960s radicalism or special-interest politics. Real as those issues are, the deeper roots of the trouble with education schools go back a century. Modern education schools were created as part of a radical movement that rejected the traditional understanding of education as an extension of the home, helping parents in their job of nurturing children and preparing citizens. Education schools were created with a new, technocratic view of the teacher as child development expert, and an ambition to use schools as a political tool to transform the social order in a new image. We shouldn’t abandon education schools, and we probably couldn’t abandon them if we tried. However, neither leaving the schools to reform themselves nor trying to reform them directly by political force is likely to work. Instead, a few simple (though politically difficult) policy changes could create an incentive structure that would make reinvention plausible, attractive, and sustainable for the schools over the long term.

Let me know what you think!

What Do Education Reform Failures Have in Common?

August 20, 2019

Nudge interventions, in which students receive texts encouraging them to do things that are thought to be good for them, have yielded another disappointing result. In a newly released study, Kelli A. Bird, Benjamin L. Castleman, Jeffrey T. Denning, Joshua Goodman, Cait Lamberton, and Kelly Ochs Rosinger report:

We investigate, through two randomized controlled trials, the impact of a national and state-level campaign to encourage students to apply for financial aid for college. The campaigns collectively reached over 800,000 students, with multiple treatment arms to investigate different potential mechanisms. We find no impacts on financial aid receipt or college enrollment overall or for any student subgroups. We find no evidence that different approaches to message framing, delivery, or timing, or access to one-on-one advising affected campaign efficacy.

For those keeping tabs, I expressed my strong doubts about texting in this review of Ben Castleman’s book even as Bill Gates was praising texting interventions and NPR’s Hidden Brain was featuring it as a success. And then my skepticism was strengthened when texting resulted in null to negative college completion outcomes in a study that received little attention. Now even its early proponents are finding disappointing results.

But this isn’t the first time I’ve predicted that a reform initiative would fail when it would take foundations years and millions of dollars before reaching the same effective conclusion.  I was raising alarms about the Measuring Effective Teacher (MET) initiative while almost everyone was jumping on the teacher evaluation and quality bandwagon.  Several years later and without much fanfare we finally hear that the initiative yielded virtually no benefits. Similarly, when a large chunk of reformers were getting behind no-excuse charter schools as the correct school model, I was warning that the model appeared unable to yield longer-term benefits, so we might not want to put all of our eggs (and all students) into the no-excuse basket. And recently, we’ve seen that Boston no-excuse charter schools have no effect on students completing college. Lastly, I’ve been predicting the political and educational failure of Portfolio Management for some time now. While the Arnold Foundation has doubled down by joining Reed Hastings in giving $200 million to the City Fund to push the idea, I expect it will be shortly after they burn through that money that we’ll hear about a reconsideration of their reform strategy.

Why am I able to anticipate these failures in education reform initiatives, while the people devoting fortunes to these efforts and their staff have such a hard time avoiding strategies that result in failure? I’m not that smart and they aren’t that dumb. I suspect the answer is that foundations have organizational interests that tend to draw them to a mistaken theory about education policy.  In its essence, that theory holds that there are policy interventions that could improve outcomes for large numbers of students if only we could discover them and get policymakers and practitioners to adopt them at scale.

I begin with a very different theory.  I suspect that there are relatively few educational practices that would produce uniformly positive results.  Instead, I’m inclined to think of education as similar to parenting, in which the correct approaches are highly context-specific.  Even within the same family, we may choose to parent different children facing similar issues in very different ways.  There may be some uniformly desirable parenting practices, but most of them are already known and widely disseminated.  So, if we wanted to improve parenting, the best we could do would be to empower parents to be in a better position to judge their context and make their own decisions about how to raise their children.  Similarly, the best we could do to improve education is to empower families and communities to make decisions within their own context.  There is relatively little we could tell all schools or educators to do to improve outcomes.

But foundations and the research community that follows their lead have a very hard time with this kind of theory. They think their job is to use science to identify the correct educational practices and then get everyone on board with doing it.  To decentralize solutions to communities and families is to relinquish control and the status of expertise. So, despite repeated failures in finding top-down policies and practices to improve outcomes at scale, they continue to search for what can’t be found.

As Mike McShane and I pointed out in our edited volume, there is nothing inherently wrong with failure in education policy.  The problem is if we refuse to acknowledge that failure and learn from it. Given the string of reform failures that large foundations have experienced over the past decade, it might be productive if they devoted some serious energy to reconsidering their basic assumptions about top-down policy solutions and became more open to the possibility that the most they could do is to empower communities and families to find their own solutions.


The Origin Story of Education Savings Accounts

August 19, 2019

(Guest Post by Matthew Ladner)

Over at Redefined I decided it would be a good idea to get the history of Education Savings Accounts written before any of us involved get hit by a bus. In the first post Dan Lips returns from walking the earth like Kung Fu (aka working on stuff other than K-12) to recount the school choice debates which helped inspire him to develop an account based choice proposal. In a sequel post I explain the circumstances by which we on the ground in Arizona put the ESA theory to practice.

School Monopoly as Reverse Patronage

August 8, 2019

(Guest post by Greg Forster)

OCPA carries my article on the government school monopoly as “reverse patronage system”:

In the 19th century, under the patronage system, hiring and firing in most government jobs was directly controlled by political officeholders. Politicians in each party would hire their party’s people to staff the government from top to bottom. (On one famous occasion, Abraham Lincoln kept his Civil War generals waiting while he attended to more important business: deciding which party faction to give control of a Post Office appointment.) Each change of party would bring massive turnover. This was also called the “spoils system” because government jobs were like the spoils of war for whoever won the election. 

It’s not hard to see why we got rid of patronage. But at least it was transparent and evenhanded. Nobody was under any illusions about what was going on. Both sides had equal rights to use the system for their own advantage. And when abuses got too far out of hand, there was always a measure of accountability—however attenuated—at the ballot box.

In the government school monopoly, we have a reverse form of patronage. Instead of politicians picking their government employees, government employees pick their politicians. This is far worse, both because it greatly increases the power of special interests to leech money out of the system and because it undermines the only power that imposed even an attenuated form of accountability upon the old patronage—the power to vote the rascals out.

Let me know what you think!

Beware of Economists Bearing Evidence

July 25, 2019

Image result for beware greeks bearing gifts

Josh Angrist, the MIT economist and a leading voice on research methods and education policy, has a recent piece in Forbes in which he praises the dawning of a new era in which policymakers are guided by economists conducting experimental analyses of promising education reforms.  He writes:

Alas, school reform has rarely been grounded in the sort of empirical analysis required of a new drug or medical treatment. Many educational innovations are propelled primarily by a politician or philanthropist’s good feelings.  It shouldn’t surprise us that weakly researched innovations often lead to disappointing results.  But this unscientific approach is now changing. America’s large urban districts are piloting new models for education delivery, such as small schools, charter schools, various sorts of magnet programs, and vouchers. Importantly, these innovations are often deployed through experiments…  Economists nowadays use these experiments to provide credible, non-partisan evidence on the consequences of school reform.

To be sure, experimental methods are the best way to identify causal effects, and most of my own research uses this approach.  Unfortunately, this improvement in methods does not always yield credible and non-partisan evidence because it is all too common for researchers to misinterpret the policy implications of these experiments, even when they are properly conducted.  Several examples of this type of misinterpretation can be found in Angrist’s brief Forbes article.  I’ll pick one to illustrate the point.

One of Angrist’s claims is that a certain type of charter school has been demonstrated as an effective policy with this rigorous new approach to research: “I’ve seen compelling evidence that urban charter schools emphasizing high expectations and data-driven instruction are winners, capable of closing the black-white achievement gap in just a few years.”  The evidence to which Angrist is primarily referring is the experimental evaluation of Boston charter schools in which he has been involved with several co-authors.  That research has shown large test score gains among students admitted to those Boston charters by lottery relative to those not admitted.

The problem is that increasing test scores does not necessarily mean that a policy is a “winner.” Test scores are an imperfect proxy for a set of knowledge and skills that we hope translate into greater educational and life success for students.  Unfortunately, a growing body of research is showing a disconnect between changing test scores and changing later life outcomes for students. But we don’t have to look across the entire research literature to find numerous examples of this disconnect between changing test scores and changing later life outcomes.  We can find evidence of it in the very Boston charter schools on which Angrist relies for his claim.

A new study by one of Angrist’s former students, Elizabeth Setren, examined test scores for students admitted by lottery to Boston charters but also tracked those students all the way through college completion.  The main purpose of her study was to disaggregate effects for special needs and English language learner (ELL) students, so she never actually reports the combined results for all students.  But we can see from the results for general education students, who comprise the vast majority of students in the study, what the overall results must be.

Like Angrist’s previous research, Setren finds large test score gains for students admitted to Boston charter schools by lottery. As shown in Table 4, general education students admitted to Boston charters benefit by .268 standard deviations (sd) on math tests and .163 sd on English Language Arts tests.  ELL and special ed charter students show similar test score benefits.  But as shown in Table 5, Boston charter school students are no more likely to graduate from high school than the lotteried control group, even five or six years after starting high school.

In Table 6, we can see that despite this lack of improvement in high school graduation rates, Boston charters are more likely to have their general education students enroll in post-secondary education, driven largely by an increase in enrollment in 4-year institutions with a possible decline in enrollment in 2-year schools.  Boston charters’ special needs students show no statistically significant increase in post-secondary enrollment.  Toward the bottom of Table 6 we can see college completion rates. Neither special needs nor general education students are more likely to complete a post-secondary degree in 4 years than the control group of students denied admission to Boston charters by lottery.  In fact, the estimated effect for general education students is negative, but not statistically significant.

So, the overall picture does not show a policy that is a “winner.” One of Angrist’s former students, using the type of experimental method he endorses to examine the policy he claims is proven to work actually shows that in the long run the policy may produce no benefits or may even produce a harm.  General education students admitted by lottery to Boston charters do experience large test score benefits, but they are no more likely to graduate high school.  Those students are also more likely to enroll in post-secondary education but no more likely to obtain a post-secondary credential than the control group. Students who take out loans to enroll in college but do not finish it may be worse off, so this pattern of results may suggest that Boston charters actually harm their students’ long-term educational outcomes.

And once again large gains in test scores are not a reliable proxy for improvement in later life outcomes.  In the Forbes piece, Angrist suggests otherwise: “Though imperfect, test-based measures of value-added predict gains in important economic outcomes like college enrollment and earnings.”  Notice the rhetorical sleight of hand in Angrist’s claim.  The issue is not whether test scores are correlated with later life outcomes but whether rigorously identified changes in test scores produced by policy interventions translate into later changes in life outcomes.   In the case of Boston charters, changes in test scores are not consistent with changes in later life outcomes, at least for general education students who constitute the bulk of the program.

Angrist is right that experiments are good and useful. But he is wrong about the dawning of a new age of science-driven education policymaking.  Science is only as good as the proxies we use for outcomes we may really care about and only as reliable as the accuracy with which the scientists describe the research literature. So, when economists come to policymakers to say that science has spoken and we now know what works, policymakers have every reason to retain some skepticism.

Updated post note — The original version of this post noted that special needs students failed to improve on test scores but did show a higher likelihood of completing a 2 year college.  That was a correct reading of the results as displayed in Table 4, but was inconsistent with the text of the paper.  The author has acknowledged that the table was in error, so I have modified the post to reflect her corrected results.