Jay P. Greene's Blog

Archaeologists in Giza have just discovered an ancient document summarizing the social science evidence on the effects of freeing people from slavery.  Given that many people will be celebrating Passover next month, perhaps they should stop just repeating ideological talking points and consider what the evidence has to say.  Here is a translation of that ancient document:

The confirmation of Moses as leader of the Hebrews was a signal moment for the Exodus movement. For the first time, the Hebrews are being led by someone fully committed to making the end of slavery and departure from Egypt the centerpiece of the Hebrew agenda.

But even as ending slavery is poised to go global, a wave of new research has emerged suggesting that freeing people from slavery may harm the people who are freed. The results are startling — the worst in the history of the field, researchers say.

While many policy ideas have murky origins, ending slavery emerged fully formed from a single, brilliant promise Abram received from G-d, the literal godfather later to be awarded a Nobel Prize in Economics (check this?). G-d declared: “You shall surely know that your seed will be strangers in a land that is not theirs, and they will enslave them and oppress them, for four hundred years. And also the nation that they will serve will I judge, and afterwards they will go forth with great possessions.”

The freedom idea sat dormant for nearly 400 years before taking root in a few places, most notably Goshen. As people began to experiment with ending slavery researchers were able to collect data to compare freed people with similar people who remained as slaves. Many of the results were released over the last 18 months.

The first results came in late 1446 BCE. Researchers examined the initial results of departing Egypt. “In mobility” they found, “freed slaves experienced a significant increase in wandering the desert.” They also saw a marked decline in food production, with Hebrews having to rely entirely on food assistance programs.

The next results came a few months later, in February, when researchers published a major study of civic order. They found large negative results in both idol worship and lawlessness. Former slaves who started as devout followers of G-d and then were freed dropped to forming a golden calf in a single year. Results were somewhat better in the second year, but were still well below the starting point.

This is very unusual. When people try to improve human behavior, sometimes they succeed and sometimes they fail. The successes usually register as modest improvements, while the failures generally have no effect at all. It’s rare to see efforts to improve human behavior having the opposite result. Thethi Neferti, a professor at the Luxor Graduate School of Social Science, calls the negative effects “as large as any I’ve seen in the literature”

In June, a third freeing-slaves study was released by the Thomas B. Fordham Institute, a conservative think tank and proponent of ending slavery (but only if it produces good outcomes and there are sufficient regulations in place). The study, which was financed by the pro-freedom Hatshepsut Family Foundation, focused on a large ending slavery program in Memphis. “Subjects freed from slavery fared worse economically compared to their closely matched peers who continued as slaves,” the researchers found. Freed slaves often became share-croppers and experienced public discrimination, leading social scientists to conclude that they would have been better off on the plantation, where food was more reliably available.

Three consecutive reports, each studying one of the largest new freeing-slaves programs, found that ending slavery hurt people’s outcomes. Researchers and advocates began a spirited debate about what, exactly, was going on.

Meriptah Djedptahaufankh of the Brookings Institution noted that the performance gap between freed people and slaves had narrowed significantly over time. He argued that stronger incentives for masters to provide slaves with food, clothing, and shelter were proving effective. The assumed superiority of freedom may no longer hold.

Some freedom supporters observed that many farms in Memphis chose not to employ freed slaves, and those that did had recently experienced declining crop production. Perhaps the participating farms were unusually bad and eager for labor. But this is another way of saying that exposing freed slaves to the vagaries of private-sector competition is inherently risky. The free market often does a terrible job of providing basic services to the freed slaves — see, for instance, the lack of grocery stores and banks in many neighborhoods with former slaves.

Others have argued that the reliable supply of food is the wrong measure of whether ending slavery is desirable. It’s true that ending-slavery programs in Cairo and some others elsewhere, which produced no improvements in access to food, increased the likelihood of freed slaves experiencing dignity and autonomy. One study of freeing slaves in Giza found positive results for feelings of self-worth among freed slaves.

But research has also linked the availability of food to a host of positive outcomes later in life. And freedom advocates often cite poor food supply for slaves to justify freeing slaves in the first place.

The new studies about freeing slaves stand in marked contrast to research findings that well-regulated indentured servitude in Heliopolis and elsewhere have a strong, positive impact on the availability of food. But while freeing slaves and changing slaves to indentured servants are often grouped under the umbrella of “freedom,” the best indentured servant programs tend to be run by well-meaning aristocrats, open to all servants and accountable to public authorities. The less “free” that ending slavery programs are, the better they seem to work.

The new evidence on ending slavery does not seem to have deterred the Moses administration, which has proposed the departure of all Hebrews from conditions of slavery in Egypt. Moses’ enthusiasm for freedom, which have been the primary focus of his plague-bringing efforts and advocacy, appears to be undiminished.

(Guest Post by Jason Bedrick)

In the last couple weeks, the New York Times has cranked their effort to discredit educational choice policies up to 11. The formula is simple: downplay the positive findings from all the previous gold-standard research and focus instead on more recent studies from Louisiana, Ohio, and Indiana — two of which are not random-assignment studies and one of which hasn’t even been released (not to mention the likelihood that overregulation is hampering Louisiana’s voucher program). Sadly, this distorted narrative is spreading, but some are pushing back. Yesterday, Paul DiPerna of EdChoice and Neal McCluskey of the Cato Institute each provided essential context for understanding the research on school choice.

In Education Next, DiPerna wrote:

Contrary to recent editorials in some major U.S newspapers, the empirical research on school choice programs is far more positive than not. Summaries of the effects of multiple programs generally show positive effects, as does a meta-analysis of gold-standard experimental research on school choice by Shakeel, Anderson, and Wolf (2016). Participating students usually show modest improvements in reading or math test scores, or both. Annual gains are relatively small but cumulative over time. Graduation and college attendance rates are substantially higher for choice students compared to peers. Programs are almost always associated with improved test scores in affected public schools. They also save money. Those savings can be used to increase per-pupil spending in local school districts. Studies also consistently show that programs increase parent satisfaction, racial integration and civic outcomes.

It’s true that recent studies have reported some initial negative effects on choice students’ test scores. The most sobering come from the rigorous, experimental evaluation of the Louisiana Scholarship Program (LSP). The LSP has a different, much more restrictive regulatory framework for private schools than other choice programs. The negative results in math should be monitored, but it’s important to note that the evaluation is only in its second of seven planned years.

Broad perspective and context are essential. Negative initial findings in one or two locations, based solely on one performance metric, should not halt the creation or expansion of school choice programs in other parts of the country. Generalizing those findings across states is problematic because education is sensitive to state and local cultural, political, governmental and economic conditions. The many places where we have observed significant positive results from choice programs swamp the few where we have seen negative findings. We need to consider the complete research base and not disproportionately emphasize the most recent studies.

McCluskey also turned a gimlet eye on the studies that found negative impacts on test scores:

First, the vast majority of random-assignment studies of private school voucher programs—the “gold-standard” research method that even controls for unobserved factors like parental motivation—have found choice producing equivalent or superior academic results, usually for a fraction of what is spent on public schools. Pointing at three, as we shall see, very limited studies, does not substantially change that track record.

Let’s look at the studies Carey highlighted: one on Louisiana’s voucher program, one on Ohio, and one on Indiana. Make that two studies: Carey cited Indiana findings without providing a link to, or title of, the research, and he did not identify the researchers. The Times did the same in their editorial. Why? Because the Indiana research has not been published. What Carey perhaps drew on was a piece by Mark Dynarski at the Brookings Institution. And what was that based on? Apparently, a 2015 academic conference presentation by R. Joseph Waddington and Mark Berends, who at the time were in the midst of analyzing Indiana’s program and who have not yet published their findings.

Next there is Ohio’s voucher program. The good news is that the research has been published, indeed by the choice-favoring Thomas B. Fordham Institute. And it does indicate that what the researchers were able to study revealed a negative effect on standardized tests. But Carey omitted two important aspects of the study. One, it found that choice had a modestly positive effect on public schools, spurring them to improve. Perhaps more important, because the research design was something called “regression discontinuity” it was limited in what it was able to reliably determine. Basically, that design looks at performance clustered around some eligibility cut-off—in this case, public schools that just made or missed the performance level below which students became eligible for vouchers—so the analysis could not tell us about a whole lot of kids. Wrote the researchers: “We can only identify with relative confidence the estimated effects…for those students who had been attending the highest-performing EdChoice-eligible public schools and not those who would have been attending lower-performing public schools.”

That is a big limit.

Finally, we come to the Louisiana study, which was random-assignment. Frankly, its negative findings are not new information. The report came out over a year ago, and we at Cato have written and talked about it extensively. And there are huge caveats to the findings, including that the program’s heavy regulations—e.g., participating schools must give state tests to voucher recipients and become part of a state accountability system—likely encouraged many of the better private schools to stay out. There are also competing private choice programs in the Pelican State. In addition, the rules requiring participating private schools to administer state tests are new, and there is a good chance that participating institutions were still transitioning. Indeed, as Carey noted, the study showed private school outcomes improving from the first year to the second. That could well indicate that the schools are adjusting to the change. And as in Ohio, there was evidence that the program spurred some improvements in public schools.

Both blog posts are worth reading in full, but the main point is this: the research literature is generally positive. The few negative findings are disconcerting and should cause education reformers to think critically about policy design, but the literature still generally finds that students exercising school choice tend to perform as well or better than their district school peers, they’re more likely to graduate high school and enroll in college, they’re less likely to be involved in crime, and all these positive effects come at a much lower cost per pupil to the taxpayer. Additionally, the overwhelming majority of studies find that choice programs have a modest but statistically significant positive effect of the performance of district schools.

Educational choice remains a win-win solution.

If D.C. wants to push school choice, it should make the D.C. voucher universal. Let D.C. clean its own long-neglected house and leave the states in charge of their own education systems.

As I told the New York Times in a story that ran today:

Greg Forster, a fellow at EdChoice, a research and advocacy organization that promotes school choice options, said that while he welcomes more support for the idea of school choice, he wants the issue to remain a state responsibility. “We have achieved a lot of victories at the state level by building bridges,” Mr. Forster said. Having Mr. Trump as an advocate “is a bigger problem for the school choice movement than it is a blessing, in my book,” he said.

He added that there is “no need for a federal push for school choice” because the options are increasingly gaining ground, leading to 61 private school choice programs in 30 states and the District of Columbia.

See also.

In my last post I described a method for understanding what ed reform foundations are really pursuing by examining the content of Tweets issued by their grantees.  When some assistants and I conducted this analysis we found that ed reform foundation grantees devote significantly more energy to promoting diversity than promoting school choice. In the prior post I wondered whether this strategy of emphasizing diversity relative to choice is wise given Republican dominance of state governments, where most education policy is formulated and implemented.

The grantees of major ed reform foundations not only give a lower priority to advocating for school choice — both charters and private school choice — but they also seem to prefer top-down accountability approaches over parental empowerment.  It was too difficult for non-expert research assistants to judge whether Tweets championed accountability to regulators as opposed to accountability to parents, but they could reliably count the number of Tweets that mentioned the words accountability, quality, and equity.  When Tweets are advocating top-down accountability they tend to use these words since they typically do not envision having to answer to parents as accountability and because they often argue that quality and equity are the goals of their top-down regulatory efforts.  Of course, some Tweets that use these words are not advocating top-down accountability, but it is also the case that one does not need to specifically mention the words accountability, quality, or equity to be arguing for top-down accountability.  While obviously imprecise, I think the number of Tweets talking about accountability, quality, or equity is a reasonable proxy for support of top-down accountability approaches.

If we compare the number of Tweets using any of these three words to the number of Tweets advocating school choice, we find far greater emphasis on top-down accountability than choice.  Among grantees of the Gates Foundation, Tweets mentioned accountability, quality, or equity 6.7 times as often as they advocated school choice.  Among Broad Foundation grantees the ratio was 3.1.  Arnold Foundation grantees mentioned accountability, quality, or equity 1.9 times as often as they advocated choice.  And at the Walton Foundation the figure was 1.0, representing a relatively even emphasis on top-down accountability and choice.

Another indication of how much foundation grantees favored top-down accountability relative to parental empowerment could be found in how they reacted to Betsy DeVos’ nomination for Secretary of Education.  Keep in mind that the time-period we examined was October 1 to December 15 of 2016, so DeVos had just been nominated toward the end of that period.  In addition, she had not yet testified, so support or opposition of her nomination was a reaction to her perceived position on issues rather than her command (or lack thereof) of the details of education policy.  Much more opposition to DeVos was mobilized and expressed after her confirmation hearings, which was after the time period we examined.  Lastly, it is important to consider that DeVos is a relatively centrist Republican reformer.  Her supporters included moderate advocates of top-down accountability, while opposition to her was marked by hostility to parental empowerment or support for choice only if it was accompanied by fairly strong top-down accountability measures.

When my assistants coded Tweets as supporting or opposing DeVos they found that grantees of the Broad Foundation opposed her 2 to 1, although this was based on a small number of Tweets.  Given that Eli Broad ultimately wrote a public letter opposing DeVos, this result is not surprising but does provide some validation for the method of analyzing Tweets as a window into foundation strategy.  Gates Foundation grantees had slightly more Tweets against DeVos than favoring her.  But among the Arnold and Walton foundation grantees, support for DeVos was much stronger, with Tweets 2.9 and 5.9 times more likely to support than oppose her, respectively.

Foundations can, of course, support whatever causes they prefer.  The major education reform foundations do not have to be enamored with school choice, can devote the bulk of their energy to promoting diversity, and can take whatever positions they like with respect to top-down accountability and Betsy DeVos.  My point in reporting these results is simply that the causes being championed by the grantees of these major education reform foundations may differ significantly in some ways from what many people think ed reform foundations support.  These causes being championed may even differ significantly from what the foundation staffs or boards think they are supporting.  The evidence suggests that major ed reform foundation grantees give far higher priority to advocating for diversity than for school choice, seem to favor top-down accountability more than parental empowerment, and sometimes only offer tepid support or even opposition to moderate Republican reformers.