Jay P. Greene's Blog

In my last post I described a method for understanding what ed reform foundations are really pursuing by examining the content of Tweets issued by their grantees.  When some assistants and I conducted this analysis we found that ed reform foundation grantees devote significantly more energy to promoting diversity than promoting school choice. In the prior post I wondered whether this strategy of emphasizing diversity relative to choice is wise given Republican dominance of state governments, where most education policy is formulated and implemented.

The grantees of major ed reform foundations not only give a lower priority to advocating for school choice — both charters and private school choice — but they also seem to prefer top-down accountability approaches over parental empowerment.  It was too difficult for non-expert research assistants to judge whether Tweets championed accountability to regulators as opposed to accountability to parents, but they could reliably count the number of Tweets that mentioned the words accountability, quality, and equity.  When Tweets are advocating top-down accountability they tend to use these words since they typically do not envision having to answer to parents as accountability and because they often argue that quality and equity are the goals of their top-down regulatory efforts.  Of course, some Tweets that use these words are not advocating top-down accountability, but it is also the case that one does not need to specifically mention the words accountability, quality, or equity to be arguing for top-down accountability.  While obviously imprecise, I think the number of Tweets talking about accountability, quality, or equity is a reasonable proxy for support of top-down accountability approaches.

If we compare the number of Tweets using any of these three words to the number of Tweets advocating school choice, we find far greater emphasis on top-down accountability than choice.  Among grantees of the Gates Foundation, Tweets mentioned accountability, quality, or equity 6.7 times as often as they advocated school choice.  Among Broad Foundation grantees the ratio was 3.1.  Arnold Foundation grantees mentioned accountability, quality, or equity 1.9 times as often as they advocated choice.  And at the Walton Foundation the figure was 1.0, representing a relatively even emphasis on top-down accountability and choice.

Another indication of how much foundation grantees favored top-down accountability relative to parental empowerment could be found in how they reacted to Betsy DeVos’ nomination for Secretary of Education.  Keep in mind that the time-period we examined was October 1 to December 15 of 2016, so DeVos had just been nominated toward the end of that period.  In addition, she had not yet testified, so support or opposition of her nomination was a reaction to her perceived position on issues rather than her command (or lack thereof) of the details of education policy.  Much more opposition to DeVos was mobilized and expressed after her confirmation hearings, which was after the time period we examined.  Lastly, it is important to consider that DeVos is a relatively centrist Republican reformer.  Her supporters included moderate advocates of top-down accountability, while opposition to her was marked by hostility to parental empowerment or support for choice only if it was accompanied by fairly strong top-down accountability measures.

When my assistants coded Tweets as supporting or opposing DeVos they found that grantees of the Broad Foundation opposed her 2 to 1, although this was based on a small number of Tweets.  Given that Eli Broad ultimately wrote a public letter opposing DeVos, this result is not surprising but does provide some validation for the method of analyzing Tweets as a window into foundation strategy.  Gates Foundation grantees had slightly more Tweets against DeVos than favoring her.  But among the Arnold and Walton foundation grantees, support for DeVos was much stronger, with Tweets 2.9 and 5.9 times more likely to support than oppose her, respectively.

Foundations can, of course, support whatever causes they prefer.  The major education reform foundations do not have to be enamored with school choice, can devote the bulk of their energy to promoting diversity, and can take whatever positions they like with respect to top-down accountability and Betsy DeVos.  My point in reporting these results is simply that the causes being championed by the grantees of these major education reform foundations may differ significantly in some ways from what many people think ed reform foundations support.  These causes being championed may even differ significantly from what the foundation staffs or boards think they are supporting.  The evidence suggests that major ed reform foundation grantees give far higher priority to advocating for diversity than for school choice, seem to favor top-down accountability more than parental empowerment, and sometimes only offer tepid support or even opposition to moderate Republican reformers.

I’ve been trying to get a better understanding of what strategy major education reform foundations are actually trying to pursue.  I’ve read their mission statements and strategy documents, but it’s hard to know what to make of these vague declarations.  Instead, I decided that we might get a more accurate sense of foundation strategy by examining the social media communications of their grantees.  That is, what the organizations funded by foundations actually advocate on Twitter might tell us more about what those foundations really support.

So I had three research assistants analyze all Tweets from the grantees of four major education reform foundations between October 1 and December 15 of 2016.  We identified the recent grantees of the Arnold, Broad, Gates, and Walton foundations and then found Twitter accounts associated with those grantees.  I then asked those assistants to code tweets for whether they were advocating the expansion of school choice, advocating diversity, supporting or opposing Betsy DeVos, and whether they contained certain words, such as: accountability, quality, social justice, equity, and choice.

Deciding whether tweets were advocating something required a judgment that could distort results.  Despite the fact that my assistants were not experts in education policy, they were still remarkably consistent in their judgements when they coded the same Tweets — generally correlating above .9.  Counting the number of Tweets containing certain words did not require judgments and were even less prone to error.  Despite the consistency of coding Tweets, it’s important to take the results of this analysis with large grains of salt.  Inferring what foundations are actually pursuing based on the Tweets of grantees is a messy enterprise.  Despite that messiness, the results can still be revealing.

The most striking thing we found is that the grantees of these major education reform foundations spend a lot of time Tweeting in support of diversity, especially relative to how often they Tweet in support of school choice.  Grantees of the Broad Foundation advocate for diversity 6.9 times as often as they advocate for choice.  At Gates it’s 7.7 times.  Grantees of the Arnold and Walton foundations pay more attention to choice, but they still advocate for diversity 2.3 and 1.7 times more often, respectively, than for choice.

Keep in mind that supporting choice included any type of choice — charters or private school choice.  And keep in mind that a major referendum on whether to expand charter schools was on the ballot in Massachusetts during the period examined.  Despite the perception that these major education reform foundations are focused on expanding school choice, at least with charters, their grantees appear to be devoting more energy to arguing for greater diversity.

The support expressed for “diversity” generally did not mean anything radical.  Instead, most of the Tweets in support of diversity advocated broadly popular things, like expanded tolerance, greater opportunities for disadvantaged groups, and increased representation of traditionally under-represented groups. For example one Tweet said “Happy International #Tolerance Day. Remember, team always beats individual. Let’s encourage students to embrace #diversity.”  Another said “#LGBT-specific professional development and promotion of acceptance in classrooms can reduce bullying.”  And yet another said, “all students would likely benefit from having teachers from a range of races & backgrounds.”

There’s nothing particularly shocking about foundation grantees expressing these messages.  What’s surprising is how much emphasis they give to these issues relative to issues like school choice.  It’s also surprising given the political realities of education policy-making.  Most education policy-making occurs in states.  And most state governments are dominated by Republicans.  Currently, 25 states have Republican control of the governor and both legislative chambers, compared to just 6 with Democratic trifectas.  Republicans control both legislative chambers in 32 states.  Republican dominance of state governments isn’t a result of the most recent election but has existed since 2010.

So, if foundations wish to exercise influence over education policy (at least in this decade) they had better craft messages that are particularly appealing to state Republicans.  Talking all the time in support of diversity and much less frequently about school choice is unlikely to win over state Republicans.  It’s not that Republicans are necessarily against diversity, it’s just that it’s a wrong set of priorities for addressing Republican concerns and goals.

At times it feels like major ed reform grantees forget who they need to appeal to in order to win.  It’s as if they are competing in a student government election at Oberlin rather than trying to win a legislative battle in Georgia.  At elite colleges you can score points in most debates by emphasizing diversity, but the same tactic is much less effective in Republican dominated state governments.  Part of the problem is that many ed reform grantees and the foundations that fund them are populated by relatively recent graduates of those elite colleges who haven’t adjusted to the fact that what worked back at school and works among their colleagues doesn’t necessarily appeal to the Republican legislators they need to convince.

In the next post I’ll provide a few more results.  None of this should be news to close observers of trends in education advocacy, but it might be useful to have some evidence that documents the shifting focus of ed reform efforts.

It’s really hard to debate policy with Mike Petrilli.  In part it is so difficult because he’s such an amiable guy that you’d rather not disagree with him.  But more importantly, it’s hard to debate Mike because his argument keeps shifting to please whatever audience he is addressing.

To one audience he’s a a champion of top-down accountability who favors everyone taking the state test, using those test results for the “default closure” of charter schools and to exclude private schools from participating in choice programs, and expecting that a lot of schools may need to be closed as a result. To another audience he’s an advocate of parental accountability who doesn’t prefer everyone taking the state test, opposes relying solely on tests when deciding which school options should be closed, and believes that school closures should be rare.

These changing views are not the result of a gradual evolution in his thinking, which everyone may do as they acquire new experiences and evidence.  Instead, this ever-shifting set of positions can change and then change back again within a few hours, a few days, or a few months.

For example, last week Mike argued against relying on test scores to make closure decisions from afar: “If Jason [Bedrick] and Jay and others are saying that those of us in the Ivory Tower shouldn’t sit in our Star Chamber and decide which schools should live and die, based solely on their test scores, I say Amen.” But the next day he tweeted in support of state policies that would automatically close schools based on test results: “I would prefer not to have automatic closures, but some states with terrible authorizers may need them.” This sentiment echoed Fordham publications from December of 2016 and June of 2015 that similarly praised “default closures.”  And just a few weeks earlier Mike was touting the use of state test results to exclude private schools from participating in voucher programs:

In Louisiana, participating private schools that serve more than forty voucher students must administer all of the state tests to them. They then receive a “scholarship cohort index” score that’s used to determine whether they can continue to accept new students. Louisiana state superintendent John White has already triggered the provision to keep several schools from accepting new voucher-bearing pupils.

In Indiana, schools must administer the Indiana Statewide Testing for Educational Progress assessment and report their graduation rates to the states. These data are used to determine each private school’s A–F rating—just like their public school counterparts. If a school is rated a “D” or “F” for two or more consecutive years, it becomes ineligible to accept new scholarship students.

Which is it? Does Mike favor a Star Chamber that decides whether schools live or die based on test scores or not?

Another example: last week Mike argued that closures should be rare: “Appropriately, it’s unusual for charter authorizers to pull the trigger—by one estimate, just three percent of charters are closed for poor performance. (Doing this infrequently is appropriate because Jason and Jay are right that we should defer to parents’ judgments most of the time, because they do have important information, and because we’re talking about their children.) ” But elsewhere Mike brags about the high numbers of closed charter schools: “During the same period, dozens of charter schools have also closed for a variety of reasons, including financial difficulties and academic underperformance. In fact, Ohio’s automatic closure law, which is based on academic results, required twenty-three charters to close during the period of study.”  And even with this high number of closures he complained last week in a tweet that “Having bad schools fester hurt the OH charter sector bigly.”  This was only a day after he wrote that charter closures were appropriately “rare.”

This dizzying change in positions is exacerbated by Mike’s habit of “triangulation.”  It’s very important that Mike position himself as the sensible moderate.  To do so he often caricatures the views of others so they sound extreme and then he positions himself just toward the center of that extreme.  He fashions himself as the “realist” while others are “purists.”  But if Mike is often locating his position relative to someone else’s, his own views will change depending on whom he’s juxtaposing himself against.  If he’s establishing that he has some overlap with but is more centrist than top-down accountability proponents, he comes off sounding more like a top-down accountability advocate.  And if he’s claiming that he has some overlap with but is more centrist that parental accountability supporters, then he comes off sounding more like a parental accountability advocate.

I do think Mike and Fordham deep down have consistent, principled views, but they cloud the picture with this triangulation.  It would also be easier to debate (and sometimes agree) with Mike and Fordham if they devoted less energy to positioning and more just to articulating their worldview and the reasons for it.

(Guest Post by Jason Bedrick)

In a recent blog post, I argued the Mike Petrilli “fundamentally misunderstands accountability” because he sees it as meaning top-down regulations when the reality is that “true accountability is when service providers are directly answerable to the people most affected by their performance.” I further noted that, as Thomas Sowell has written, “It is hard to imagine a more stupid or more dangerous way of making decisions than by putting those decisions in the hands of people who pay no price for being wrong.”

Mike responds by arguing semantics, attacking straw men, and taking my words out of context. He writes:

As Jason knows, I agree with him—and with Sowell, Milton Friedman, and many others—that schools must be accountable to parents via the marketplace, that school choice is a better system than a regulated monopoly, and that I don’t in fact support “top down regulations.” What I and other “choice realists” want is for schools of choice to be lightly regulated yet subject to societal expectations regarding results. That’s what we mean by “accountability.” It’s Jason and his like-minded colleagues who have sought to redefine that word to mean “regulation.” Ironically, the teachers unions play the same game, saying that schools such as charters, which are partially freed from meddlesome government regulations (like tenure rules), are not “accountable.” This strikes me as mildly Orwellian.

Actually, it’s Mike who is trying to redefine the term “regulation.” Merriam-Webster defines “regulation” as “an authoritative rule dealing with details or procedure” or “a rule or order issued by an executive authority or regulatory agency of a government and having the force of law.” When the government issues a rule saying that schools that fail to meet its criteria are ineligible to accept vouchers from parents who would like to choose that school, that is a regulation. That may be a wise or foolish regulation, but it should not be controversial to recognize that it meets the textbook definition of a “regulation.”

If anything is Orwellian, it’s Mike’s insistence that we reserve the term “regulation” only for regulations he doesn’t like.

For the record, I support certain regulations — yes, regulations! — ensuring financial accountability to the taxpayer, such as requiring that ESA parents submit receipts, regular audits of such programs, etc. Taxpayers are owed accountability for funds they are forced to fork over, which means ensuring that they are only used for their intended purposes. However, accountability for academic results should lie primarily with those who have both the greatest incentive to help and most local knowledge about the children we are funding: their parents. In any case, I don’t pretend that the regulations I support somehow aren’t “regulations” just because I support them. Mike shouldn’t either.

Mike continues:

If Jason and Jay and others are saying that those of us in the Ivory Tower shouldn’t sit in our Star Chamber and decide which schools should live and die, based solely on their test scores, I say Amen. But that’s not what we’ve ever proposed. The charter sector has invented an entity called a school authorizer. The best ones know the schools they oversee inside and out. They aren’t “distant”; they build relationships with the schools’ boards, leaders, and parents. They understand the stories behind the test scores. And when reviewing the performance of “their” schools, they look at myriad factors besides those test scores. (As Andy Smarick has argued, voucher programs could use something like authorizers, too.)

Here, Mike is attacking a straw man. Were we talking about charters? No. The article to which I was responding was titled, “Vouchers have changed. Maybe your position should change, too.” In that article, Mike wrote:

In Louisiana, participating private schools that serve more than forty voucher students must administer all of the state tests to them. They then receive a “scholarship cohort index” score that’s used to determine whether they can continue to accept new students. Louisiana state superintendent John White has already triggered the provision to keep several schools from accepting new voucher-bearing pupils.

In Indiana, schools must administer the Indiana Statewide Testing for Educational Progress assessment and report their graduation rates to the states. These data are used to determine each private school’s A–F rating—just like their public school counterparts. If a school is rated a “D” or “F” for two or more consecutive years, it becomes ineligible to accept new scholarship students. (The information about these voucher programs comes from the American Federation for Children’s school choice yearbook.)

So if you oppose vouchers because of lack of accountability, it may be time to change your position.

My response was focused entirely on private school choice (i.e., vouchers, tax-credit scholarships, and education savings accounts). Mike continues to tout policies like those in Louisiana that likely contributed to the first negative outcomes in any random-assignment study of any voucher program, but when I call him out for this, he wants to change the subject and talk about charters instead. I wonder why!

(Note also: I’ve already responded to the Smarick proposal Mike plugged, which I think is creative but unwise.)

By focusing on charters, Mike is able to attack a straw man rather than address my actual points. I do not object to charter authorizers closing schools based on local knowledge, as he described above. I object vigorously to default closures (or expulsion from voucher programs) based on a pre-ordained formula, especially one (like the A-F grading system* he promoted in his previous article) that is heavily dependent on standardized test scores.

So, Mike, which is it? Do you support or oppose default closures? Do you support or oppose the Louisiana-style voucher expulsions? If you support them, then let’s talk about that rather than some non sequiturs.

Mike titled his recent article “Accountability to parents is necessary but not sufficient,” but he doesn’t even attempt to defend this position. Is there any reason to believe that mandating the state test and tossing out low-scoring schools would produce better results, in the long run, than mandating that voucher-accepting schools administer their choice of nationally norm-referenced tests and reporting the results directly to parents? If so, he doesn’t offer any. So-called “choice purists” prefer the latter regulation (there’s that word again!) because it avoids incentivizing the narrowing of curriculum or the refusal of quality schools from participation in voucher programs, as Lindsey Burke and I described in our recent report, “Recalibrating Accountability.” If Mike wants to have a conversation about the points we’ve actually made, instead of the non sequiturs he’d rather raise, I’m happy to talk.

Mike concludes:

Perhaps Jason, Jay, and Thomas Sowell would disagree. They might argue that taking decisions out of the hands of parents can never be justified. But with education, it’s simply not true that the public “pays no price” for being wrong. We all pay a hefty price when kids don’t learn. So, in rare cases, we need to act accordingly.

Here, Mike distorts the meaning and context of what I (and Thomas Sowell) wrote. I said that bureaucrats pay no price for being wrong, not that society pays no price. When a kid graduates high school unable to read his own diploma, no teacher, superintendent, or state education agency employee loses their job, nor does any politician (or, for that matter, any think tank wonks). Better to leave such important decisions in the hands of people who do bear the consequences of their decisions: parents.

In conclusion, I think Mike should update his own flawed choice taxonomy. At heart, he’s really a Super Nanny:

(Just kidding, Mike! You know we love you.)

* UPDATE: Above I called Louisiana’s A-F grading system “heavily dependent” on standardized test scores. Actually, for grades K-6, it is entirely dependent on test scores, as the LA DOE website explains. For grades 7 and 8, 95% of the letter grade is based on those scores. For high schools, half the grade is based on the scores, and half is based on graduation rates. Mike says he likes using LA’s A-F grading system to kick out low-performing schools, but he also offers an “Amen!” to the notion that “those of us in the Ivory Tower shouldn’t sit in our Star Chamber and decide which schools should live and die, based solely on their test scores.”

So which is it, Mike?

My very first academic publication was (I thought) a devastating debunking of election forecasting models.  Several leading political scientists and economists had developed regression models that claimed to have perfect records of predicting presidential election outcomes.  I showed that these models were only perfect in “predicting” elections that had already occurred.  With only a dozen or so observations and five or so independent variables, these models were over-fitted to the data so that they could score a bulls-eye by drawing the circles after the arrows landed.

That was almost a quarter-century ago and despite my warnings election as well as sports forecasting have grown into major industries.  Their methods have become more complicated but the fundamental defect of this enterprise remain the same: most forecasters incredibly overstate the precision of their predictions.  That is, most election and sports forecasters claim to know things that they simply do not know.

Let’s take the recent presidential election to illustrate the problem.  Nate Silver, who has achieved celebrity as a leading forecaster, predicted just prior to the election that Hilary Clinton had a 71.4% chance of winning the presidency.  How did Silver arrive at this incredibly precise sounding forecast?  Basically, he takes a large set of state and national poll results (including their reported margin of error), weights the polls by his assessment of their past accuracy, and then given that each poll provides a probability distribution of the result he runs simulations that draw outcomes from those probability distributions by chance.  In 714 out of 1,000 such simulations Clinton would win the White House.

There are two major flaws in this method.  First, the margin of error reported by polls only captures the errors given that the model is correct.  We also have uncertainty about the composition of the model and that error is never quantified or reported.  To make this less abstract, consider the USC/LA Times poll. Every poll, including USC/LA Times, has to make a series of assumptions and choices to construct its model.  Each one of those assumptions and choices is made with a certain amount of uncertainty, which is not contained in the margin of error but can have a strong effect on the prediction.

So, when polls disproportionately get responses from certain segments of the population, they have to make choices about how to weight the responses they have so that they are representative of the full population for whom they are making the forecast.  But what method should they use for weighting their respondents?  You might not think such a small, technical detail would matter, but as the recent experience of the USC/LA Times poll indicates, the choices you make about weighting would determine the result.

We only know this about the USC/LA Times poll because they happen to make their raw data available in real time so others can re-analyze with different assumptions.  The USC/LA Times poll was also exceptional in that it was one of the only polls predicting a Trump victory (although in the end even they were mistaken in that they were predicting the share of the popular vote, which Trump lost).  Their openness about the data and their outlier prediction attracted Ernie Tedeschi to re-analyze their data using different assumptions about weighting responses.

As it turns out, the USC/LA Times poll used a non-conventional method of weighting their respondents to be similar to the share of people in the Census within “micro-cells.”  They would re-weight their respondents to have the same proportion of small groups as found in the Census, like college-education, African-Americans under 30, rather than large groups, like African-Americans or college-educated people.  Tedeschi showed that if you re-weighted the USC/LA Times poll using large groups you would flip the result, so that Clinton had a large lead rather than Trump.

The point here is that whether you weight by small or large groups does not seem to be theoretically important, but it makes an enormous difference in the predicted result.  The uncertainty we have about which seemingly arbitrary weighting method to use is not captured at all in the reported margin of error.  That margin of error only tells us the uncertainty within the model.  But not knowing how to weight respondents represents uncertainty about the model.  Silver’s simulation method only incorporates the error within models, not the error about models and therefore grossly overstates the precision of his estimates.

The USC/LA Times poll illustrates the second major flaw, which is that the polls are not independent observations as Silver’s method assumes.  That is, errors about model construction are correlated across polls, meaning that if they are wrong about something, they will almost all be wrong together.  For example, one of the criticisms leveled against the USC/LA Times poll is that its weighting method was not the “convention” or “best-practice” among pollsters.  Another way to put that is that pollsters tend to make common assumptions about the construction of their models.  But if any of those seemingly unimportant assumptions turns out to be wrong, not only will the forecasts be off, but the error will be common across polls.

Silver’s simulation method requires that the errors of model construction are random and will balance themselves out across polls so that one will make a wrong assumption in one direction while another makes a wrong assumption in the other direction.  But that isn’t right.  Pollsters are making their decisions mostly in common according to conventions or best-practices.  The USC/LA Times poll stood out because it deviated from that herd mentality among pollsters.

These two errors I’ve described are not small things.  As I’ve shown, a single seemingly unimportant decision about how to weight respondents can change who the predicted winner is.  And pollsters are making dozens of these decisions, any one of which could alter the predicted outcome, without even knowing that they are doing so.  And on top of all of that, the pollsters tend to make common decisions about these matters, so when they are wrong, they and Silver will all be wrong together.

Silver wasn’t just unlucky.  The truth is that neither he nor the pollsters on whom he relies could know a priori what the right assumptions were about weighting, turnout, or a host of other matters.  If he simply said that he thought it was more likely than not that Clinton would win, he might be able to defend his prediction.  But given all of the uncertainty I’ve described taking his prediction to the tenths place just shows him to be a quack.

Unfortunately, we just have a hard time dealing with uncertainty.  It would be comforting to us if the world were orderly and predictable, so we turn to charlatans like Silver, or the UpShot, or ESPN’s in-game prediction model as people once turned to lucky totems or end-of-days preachers.  The hard reality is that there is a lot more uncertainty in the world than these folks would admit and the highly fallible methods of social scientists and charismatic preachers offer little relief from that reality.

We’ve all been fed such a steady diet of Nazi Resistance stories that we likely have no doubt that we’d be placing explosives next to the railroad tracks if we had lived under Nazi occupation.  In my personal variation of this fantasy, I imagine myself in Inglourious Basterds (probably my favorite movie), led by Lieutenant Aldo “The Apache” Raine in a team of Jewish commandos taking Nazi scalps.

But real life is more complicated and it isn’t always obvious what one would or should do under such circumstances.  You’d face competing practical and moral demands.  What concern should you have for your own safety or the safety of your friends and family?  What obligations do you have to resist the occupation of your country relative to minimizing the severity and harms of that occupation by cooperating?

These kinds of issues are explored more deeply than in other films in the new Norwegian series available on Netflix, called Occupied (or Okkupert).  The premise is that an environment-friendly government takes office in Norway and decides to halt all oil and gas production in favor of a new green (and fictional) energy source that is not fully ready.  This decision throws European economies into a crisis, which then spurs the Russians to seize oil and gas facilities to restart production.

You’ll have to just accept parts of this premise that seem implausible to get to the excellent exploration of what people might do if their country’s sovereignty were slipping away to a much stronger power without hope for assistance from the outside.  And given how Russian encroachment into The Ukraine was met with mostly symbolic objections from Europe and the US, this premise might not seem so ridiculous.

While watching I honestly wasn’t entirely sure who I was rooting for (other than pretty clearly against the Russians).  Everyone seems to have their reasons for behaving as they do so that it isn’t easy to judge what the right thing to do would be or to imagine what you would do if you were there. And as crowds gather across the country under the banner “Resist,” Occupied might have something to tell us about our current political crisis.

If you would like to consider another wonderfully complicated work on the competing moral obligations raised by resistance, read Jean Anouilh’s play Antigone, which is inspire by Sophocles’ play of the same name.  I far prefer Anouilh’s version because he allows each side to make its best argument, causing the audience pretty much to switch what it favors based on whoever spoke last. Anouilh’s version is especially amazing because it was first produced in 1944 in Nazi-occupied Paris, so it was speaking directly to the emergency facing its audience.

How Anouilh’s play got passed the German censors isn’t entirely clear.  They may have allowed it to be produced in part because of the strength of the arguments Anouilh gave to Creon about the need for the state to maintain order.  Unlike Sophocles’ Creon, who is clearly the villain to Antigone’s heroism, Anouilh’s Creon is an unwilling ruler who feels as trapped by his obligation to preserve the city as Antigone feels trapped by her obligation to obey the law of the gods.

Below is a taste in which Creon gets the upper hand in the argument before blowing it by urging Antigone to focus on happiness, something that she seems unable and unwilling to do.

(Guest Post by By Kevin Hesla)

“Charter schools are public schools that have flexibility to meet students’ unique needs, while being held accountable for advancing student achievement.” As a charter school researcher, I have copied and pasted this sentence more times then I care to admit over the past five years. But in times like these, I think it is necessary to stop and reflect on what is truly important and unique about this movement.

As Jay P. Greene from the University of Arkansas points out in a recent podcast, the concept of public school choice has reached “escape velocity.” A movement that first gained popularity among low-income parents who were searching for a high-quality educational option is now a concept that the vast majority of parents, from all walks of life, embrace and support. And in 2017, parents don’t just want a single high-quality public school option—they want multiple high-quality public school options.

For parents in neighborhoods with failing district public schools, the concept of school choice is nothing less than a life changing opportunity for their children to receive a high-quality public education, prepare themselves for the demands of a rapidly changing economy, and pursue their dreams. For parents in neighborhoods with high-quality district public schools, school choice is an acknowledgement that district public schools (despite all of their tremendous benefits) do not work for every student. It is an acknowledgement that students are different, that they learn differently and are motivated by different things—and that district public schools are not able to meet the needs and passions of every student.

The National Alliance for Public Charter Schools recently commissioned a survey of parents from across the country and asked them if they favored or opposed the ability of parents and students to choose which public school they attend—regardless of where they live. The results demonstrate that the vast majority of parents (regardless of background) support public school choice:

• Race/ethnicity: 84 percent of Hispanic parents, 82 percent of Black parents, and 76 percent of White parents support public school choice.
• Income: 86 percent of low-income parents, 79 percent of middle-income parents, and 72 percent of high-income parents support public school choice.
• Party Affiliation: 81 percent of Democratic parents, 77 percent of Republican parents, and 76 percent of Independent parents support public school choice.
• Geography: 84 percent of urban parents, 77 percent of suburban parents, and 74 percent of rural parents support public school choice.

Despite all the current rhetoric, it is important to remember that parents across the county are overwhelmingly supportive of public school choice. This false dichotomy between district public schools and charter public schools is in no way helpful to the tremendously important discourse and debate on public education. And education (just like knowledge and economic growth) is not a zero sum game. A new high-quality public school option does not have to come at the expense of another public school option. The vast majority of parents support public school choice because public school choice provides benefits to all students. And all children should be given the opportunity to attend a high-quality public school that meets their needs and inspires them to dream big.

Kevin Hesla is the  Director of Research at the National Alliance for Public Charter Schools

I’d like to continue my review of Michael Lewis’ new book,  The Undoing Project, on the collaboration between Daniel Kahneman and Amos Tversky by speculating about what advice Kahneman and Tversky might offer education foundations.

Foundations have particular challenge in detecting and correcting errors in their own thinking.  Because many people want things from foundations, especially their money, foundations frequently are organized as to limit communication with them.  They generally don’t open their doors, phone lines, and emails to whoever might want to suggest something or ask something of them for fear that they will be overwhelmed.  So they typically hide behind a series of locked doors, don’t make their phone or emails readily available, and insist that all applications follow a specific format, be submitted at a particular time, and be designed to address pre-determined issues.

Insulating themselves from external influence is understandable, but it creates real problems if they ever hope to detect when they are mistaken and need to make changes.  The limited communication that does make it through to foundations tends to re-affirm whatever they are already doing.  Prospective grantees don’t like to tell foundations that they are mistaken and need to change course because that makes getting funded unlikely.  Instead, foundations tend to hear that their farts smell like roses.  To make matters worse, many foundations are run in very top-down ways, which discourage questioning and self-criticism.

The Undoing Project presents a very similar situation having to do with airline pilot errors.  Lewis describes a case in which a commercial airline was experiencing too many accidents caused by pilot error.  The accidents were not fatal, but they were costly and dangerous — things like planes landing at the wrong airport.  So the airline approached Amos Tvsersky and asked for help in improving their training so as to minimize these pilot errors.  They wanted Tversky to design a pilot training method that would make sure pilots had the information and skills to avoid errors.

Tversky told the airline that they were pursuing the wrong goal.  Pilots are going to make errors and no amount of information or training would stop them from committing those mistakes.  We often assume that our errors are always caused by ignorance, but Tversky told them this was not true.  The deeper problem is that once we have a mental model of the world, we tend to downplay or ignore information that is inconsistent with that model and bolster facts that support our model.  If a pilot thinks he is landing at the right airport, he distorts available information to confirm that he is landing in Fort Lauderdale rather than nearby Palm Beach even if that is incorrect.  The problem is not a lack of information, but our tendency to fit information into our pre-conceived beliefs.

Tversky’s advice was to change the cockpit culture to facilitate questioning and self-criticism.  At the time cockpits were very hierarchical based on the belief that co-pilots needed to implement pilot orders quickly and without question lest the delay and doubt promote indecision and disorder.  So the airline implemented Tversky’s suggestions and changed their training to encourage co-pilots to doubt and question and pilots to be more receptive to that criticism.  The results was a marked deline in accidents caused by pilot error.  Apparently we aren’t very good at detecting our own errors, but we are more likely to do so if others are encouraged to point them out.

So what might Tversky suggest to education foundations?  I think he’d recognize that they have exceptional difficulty in detecting their own errors and need intentional, institutional arrangements to address that problem.  In particular, he might suggest that they divide their staff into a Team A and Team B.  Each team would work on a different theory of change — theories that are not necessarily at odds with each other but are also not identical.  For example, one team might focus on promoting school choice and another on promoting test-based accountability.  Or one team may promote tax credits and the other ESAs.  The idea of dividing staff into somewhat competing teams is that they then have incentives to point out shortcomings in the other team’s approach.  Dividing into Team A and Team B could be a useful check on the all-too-common problem of groupthink.

Another potential solution is to hire two or three internal devil’s advocates whose job it is to question the assumptions and evidence believed by foundation staff.  To protect those devil’s advocates, it is probably best to have them report directly to the board rather than the people they are questioning.

Whatever the particular arrangements, the point is that education foundations should strive to promote an internal culture of doubt and self-criticism if they wish to catch and correct their own errors and avoid groupthink.  One foundation that I think has taken steps in this direction is the Arnold Foundation.  They actually hold internal seminars in which they invite outside speakers to come and potentially offer critiques of their work.  Neerav Kingsland, who heads education efforts at Arnold, is also especially available on blogs and twitter for critical discussion.  I don’t always agree with Neerav but I am impressed by his openness to dissent.

The collaboration between Kahneman and Tversky was itself an example of the importance of engaging in tough criticism within an effort.  Like the airline pilots, they developed habits of challenging each other, which made their work together better than it ever could have been individually.

When I was in graduate school I read a lot of what was then new research by Daniel Kahneman and Amos Tversky.  I found their documentation of the systematic ways in which people deviate from rational decision-making fascinating and I was searching for a way to apply that to political science questions.  In the end, I couldn’t figure out how to build a new theory based on systematic irrationality.

Reading Michael Lewis’ excellent new book, The Undoing Project, about the amazing and eventually problematic collaboration between Kahneman and Tversky brought back a flood of nostalgia but also reminded me of some problems with trying to extend their work.  In particular, I was reminded of two things.  First, while Kahneman and Tversky are remarkably persuasive in demonstrating how people regularly deviate from rationality, neither I nor others have had much success in building new theories based on systematic irrationality.  As it turns out, assuming rationality is clearly an inaccurate description of how people think, but it remains quite useful for building theories that yield accurate predictions.  That is, Kahneman and Tversky may have revolutionized social science much less than Lewis suggests.

Second, much of the work that has tried to build on Kahneman and Tversky seems to violate their basic finding that expert judgement is unreliable.  The development of behavioral economics and its application to a variety of fields, including education, mostly seems to consist of trying to devise ways to correct the systematic irrationality of others.  If low-income students are accepted to college but do not enroll after failing to complete the FAFSA financial aid form, we assume they are behaving counter to their long-term interests and propose interventions to induce them to complete the form and enroll.

As I’ve written elsewhere, this approach has a variety of problems, but the chief of which is that it assumes too much rationality on the part of the social scientist devising the solutions.  How do we know that people would be better off if we could nudge them into doing something other than what they had originally decided to do?  Just as other people may be systematically irrational, so may the social scientists devising plans for improving other people’s lives.  I’m not saying that no interventions are helpful.  I’m just saying that we should be extremely cautious and humble when developing plans for how other people should live their lives.

The need for humility among experts and social scientists was a central theme in Kahneman and Tversky’s work.  Their approach was not, as one critic accused them, a psychology of stupid people; it was a psychology of all people, including experts and social scientists.  In fact, one of their first experiments was to give statisticians problems to see if they would update their priors as if they were Bayesians.  As it turns out, even statisticians who you might think would be particularly familiar with Bayes’ Theorem, do not actually think like Bayesians.  In subsequent experiments they found that even warning subjects of the systematic irrationality to which they might be prone does not prevent them from being systematically irrational.  Greater knowledge and expertise do not prevent us from falling into the same intellectual potholes over and over again.

So Kahneman and Tversky’s research demonstrates that there is no priestly class immune to the shortcomings of others and even foreknowledge and confession of one’s sins of irrationality provide little protection against repeating common errors.  And yet, much of behavioral economics seems to pay little heed to this central finding as they move full steam ahead devising solutions for other people’s irrationality.  They seem to forget that devising solutions, building models, and testing them all require human judgements which are also prone to systematic error.

In his seminal volume, Thinking, Fast and Slow, Kahneman admits there is no real solution to our tendency to deviate from rationality.  Instead, he suggests some habits to check the errors, mostly involving slowing down, being more cautious and self-critical, as well as inviting the criticism of others.  Let’s not correct for popular mistakes by installing a technocratic elite because that elite are also prone to common errors.