Tweets as a Window Into Foundation Strategy, Part 2

February 17, 2017

Image result for twitter bird flying through window

In my last post I described a method for understanding what ed reform foundations are really pursuing by examining the content of Tweets issued by their grantees.  When some assistants and I conducted this analysis we found that ed reform foundation grantees devote significantly more energy to promoting diversity than promoting school choice. In the prior post I wondered whether this strategy of emphasizing diversity relative to choice is wise given Republican dominance of state governments, where most education policy is formulated and implemented.

The grantees of major ed reform foundations not only give a lower priority to advocating for school choice — both charters and private school choice — but they also seem to prefer top-down accountability approaches over parental empowerment.  It was too difficult for non-expert research assistants to judge whether Tweets championed accountability to regulators as opposed to accountability to parents, but they could reliably count the number of Tweets that mentioned the words accountability, quality, and equity.  When Tweets are advocating top-down accountability they tend to use these words since they typically do not envision having to answer to parents as accountability and because they often argue that quality and equity are the goals of their top-down regulatory efforts.  Of course, some Tweets that use these words are not advocating top-down accountability, but it is also the case that one does not need to specifically mention the words accountability, quality, or equity to be arguing for top-down accountability.  While obviously imprecise, I think the number of Tweets talking about accountability, quality, or equity is a reasonable proxy for support of top-down accountability approaches.

If we compare the number of Tweets using any of these three words to the number of Tweets advocating school choice, we find far greater emphasis on top-down accountability than choice.  Among grantees of the Gates Foundation, Tweets mentioned accountability, quality, or equity 6.7 times as often as they advocated school choice.  Among Broad Foundation grantees the ratio was 3.1.  Arnold Foundation grantees mentioned accountability, quality, or equity 1.9 times as often as they advocated choice.  And at the Walton Foundation the figure was 1.0, representing a relatively even emphasis on top-down accountability and choice.

Another indication of how much foundation grantees favored top-down accountability relative to parental empowerment could be found in how they reacted to Betsy DeVos’ nomination for Secretary of Education.  Keep in mind that the time-period we examined was October 1 to December 15 of 2016, so DeVos had just been nominated toward the end of that period.  In addition, she had not yet testified, so support or opposition of her nomination was a reaction to her perceived position on issues rather than her command (or lack thereof) of the details of education policy.  Much more opposition to DeVos was mobilized and expressed after her confirmation hearings, which was after the time period we examined.  Lastly, it is important to consider that DeVos is a relatively centrist Republican reformer.  Her supporters included moderate advocates of top-down accountability, while opposition to her was marked by hostility to parental empowerment or support for choice only if it was accompanied by fairly strong top-down accountability measures.

When my assistants coded Tweets as supporting or opposing DeVos they found that grantees of the Broad Foundation opposed her 2 to 1, although this was based on a small number of Tweets.  Given that Eli Broad ultimately wrote a public letter opposing DeVos, this result is not surprising but does provide some validation for the method of analyzing Tweets as a window into foundation strategy.  Gates Foundation grantees had slightly more Tweets against DeVos than favoring her.  But among the Arnold and Walton foundation grantees, support for DeVos was much stronger, with Tweets 2.9 and 5.9 times more likely to support than oppose her, respectively.

Foundations can, of course, support whatever causes they prefer.  The major education reform foundations do not have to be enamored with school choice, can devote the bulk of their energy to promoting diversity, and can take whatever positions they like with respect to top-down accountability and Betsy DeVos.  My point in reporting these results is simply that the causes being championed by the grantees of these major education reform foundations may differ significantly in some ways from what many people think ed reform foundations support.  These causes being championed may even differ significantly from what the foundation staffs or boards think they are supporting.  The evidence suggests that major ed reform foundation grantees give far higher priority to advocating for diversity than for school choice, seem to favor top-down accountability more than parental empowerment, and sometimes only offer tepid support or even opposition to moderate Republican reformers.


Tweets as a Window Into Foundation Strategy

February 16, 2017

Image result for twitter bird flying through window

I’ve been trying to get a better understanding of what strategy major education reform foundations are actually trying to pursue.  I’ve read their mission statements and strategy documents, but it’s hard to know what to make of these vague declarations.  Instead, I decided that we might get a more accurate sense of foundation strategy by examining the social media communications of their grantees.  That is, what the organizations funded by foundations actually advocate on Twitter might tell us more about what those foundations really support.

So I had three research assistants analyze all Tweets from the grantees of four major education reform foundations between October 1 and December 15 of 2016.  We identified the recent grantees of the Arnold, Broad, Gates, and Walton foundations and then found Twitter accounts associated with those grantees.  I then asked those assistants to code tweets for whether they were advocating the expansion of school choice, advocating diversity, supporting or opposing Betsy DeVos, and whether they contained certain words, such as: accountability, quality, social justice, equity, and choice.

Deciding whether tweets were advocating something required a judgment that could distort results.  Despite the fact that my assistants were not experts in education policy, they were still remarkably consistent in their judgements when they coded the same Tweets — generally correlating above .9.  Counting the number of Tweets containing certain words did not require judgments and were even less prone to error.  Despite the consistency of coding Tweets, it’s important to take the results of this analysis with large grains of salt.  Inferring what foundations are actually pursuing based on the Tweets of grantees is a messy enterprise.  Despite that messiness, the results can still be revealing.

The most striking thing we found is that the grantees of these major education reform foundations spend a lot of time Tweeting in support of diversity, especially relative to how often they Tweet in support of school choice.  Grantees of the Broad Foundation advocate for diversity 6.9 times as often as they advocate for choice.  At Gates it’s 7.7 times.  Grantees of the Arnold and Walton foundations pay more attention to choice, but they still advocate for diversity 2.3 and 1.7 times more often, respectively, than for choice.

Keep in mind that supporting choice included any type of choice — charters or private school choice.  And keep in mind that a major referendum on whether to expand charter schools was on the ballot in Massachusetts during the period examined.  Despite the perception that these major education reform foundations are focused on expanding school choice, at least with charters, their grantees appear to be devoting more energy to arguing for greater diversity.

The support expressed for “diversity” generally did not mean anything radical.  Instead, most of the Tweets in support of diversity advocated broadly popular things, like expanded tolerance, greater opportunities for disadvantaged groups, and increased representation of traditionally under-represented groups. For example one Tweet said “Happy International #Tolerance Day. Remember, team always beats individual. Let’s encourage students to embrace #diversity.”  Another said “#LGBT-specific professional development and promotion of acceptance in classrooms can reduce bullying.”  And yet another said, “all students would likely benefit from having teachers from a range of races & backgrounds.”

There’s nothing particularly shocking about foundation grantees expressing these messages.  What’s surprising is how much emphasis they give to these issues relative to issues like school choice.  It’s also surprising given the political realities of education policy-making.  Most education policy-making occurs in states.  And most state governments are dominated by Republicans.  Currently, 25 states have Republican control of the governor and both legislative chambers, compared to just 6 with Democratic trifectas.  Republicans control both legislative chambers in 32 states.  Republican dominance of state governments isn’t a result of the most recent election but has existed since 2010.

So, if foundations wish to exercise influence over education policy (at least in this decade) they had better craft messages that are particularly appealing to state Republicans.  Talking all the time in support of diversity and much less frequently about school choice is unlikely to win over state Republicans.  It’s not that Republicans are necessarily against diversity, it’s just that it’s a wrong set of priorities for addressing Republican concerns and goals.

At times it feels like major ed reform grantees forget who they need to appeal to in order to win.  It’s as if they are competing in a student government election at Oberlin rather than trying to win a legislative battle in Georgia.  At elite colleges you can score points in most debates by emphasizing diversity, but the same tactic is much less effective in Republican dominated state governments.  Part of the problem is that many ed reform grantees and the foundations that fund them are populated by relatively recent graduates of those elite colleges who haven’t adjusted to the fact that what worked back at school and works among their colleagues doesn’t necessarily appeal to the Republican legislators they need to convince.

In the next post I’ll provide a few more results.  None of this should be news to close observers of trends in education advocacy, but it might be useful to have some evidence that documents the shifting focus of ed reform efforts.


Petrilli vs. Petrilli

February 14, 2017

Image result for fight club punching himself

It’s really hard to debate policy with Mike Petrilli.  In part it is so difficult because he’s such an amiable guy that you’d rather not disagree with him.  But more importantly, it’s hard to debate Mike because his argument keeps shifting to please whatever audience he is addressing.

To one audience he’s a a champion of top-down accountability who favors everyone taking the state test, using those test results for the “default closure” of charter schools and to exclude private schools from participating in choice programs, and expecting that a lot of schools may need to be closed as a result. To another audience he’s an advocate of parental accountability who doesn’t prefer everyone taking the state test, opposes relying solely on tests when deciding which school options should be closed, and believes that school closures should be rare.

These changing views are not the result of a gradual evolution in his thinking, which everyone may do as they acquire new experiences and evidence.  Instead, this ever-shifting set of positions can change and then change back again within a few hours, a few days, or a few months.

For example, last week Mike argued against relying on test scores to make closure decisions from afar: “If Jason [Bedrick] and Jay and others are saying that those of us in the Ivory Tower shouldn’t sit in our Star Chamber and decide which schools should live and die, based solely on their test scores, I say Amen.” But the next day he tweeted in support of state policies that would automatically close schools based on test results: “I would prefer not to have automatic closures, but some states with terrible authorizers may need them.” This sentiment echoed Fordham publications from December of 2016 and June of 2015 that similarly praised “default closures.”  And just a few weeks earlier Mike was touting the use of state test results to exclude private schools from participating in voucher programs:

In Louisiana, participating private schools that serve more than forty voucher students must administer all of the state tests to them. They then receive a “scholarship cohort index” score that’s used to determine whether they can continue to accept new students. Louisiana state superintendent John White has already triggered the provision to keep several schools from accepting new voucher-bearing pupils.

In Indiana, schools must administer the Indiana Statewide Testing for Educational Progress assessment and report their graduation rates to the states. These data are used to determine each private school’s A–F rating—just like their public school counterparts. If a school is rated a “D” or “F” for two or more consecutive years, it becomes ineligible to accept new scholarship students.

Which is it? Does Mike favor a Star Chamber that decides whether schools live or die based on test scores or not?

Another example: last week Mike argued that closures should be rare: “Appropriately, it’s unusual for charter authorizers to pull the trigger—by one estimate, just three percent of charters are closed for poor performance. (Doing this infrequently is appropriate because Jason and Jay are right that we should defer to parents’ judgments most of the time, because they do have important information, and because we’re talking about their children.) ” But elsewhere Mike brags about the high numbers of closed charter schools: “During the same period, dozens of charter schools have also closed for a variety of reasons, including financial difficulties and academic underperformance. In fact, Ohio’s automatic closure law, which is based on academic results, required twenty-three charters to close during the period of study.”  And even with this high number of closures he complained last week in a tweet that “Having bad schools fester hurt the OH charter sector bigly.”  This was only a day after he wrote that charter closures were appropriately “rare.”

This dizzying change in positions is exacerbated by Mike’s habit of “triangulation.”  It’s very important that Mike position himself as the sensible moderate.  To do so he often caricatures the views of others so they sound extreme and then he positions himself just toward the center of that extreme.  He fashions himself as the “realist” while others are “purists.”  But if Mike is often locating his position relative to someone else’s, his own views will change depending on whom he’s juxtaposing himself against.  If he’s establishing that he has some overlap with but is more centrist than top-down accountability proponents, he comes off sounding more like a top-down accountability advocate.  And if he’s claiming that he has some overlap with but is more centrist that parental accountability supporters, then he comes off sounding more like a parental accountability advocate.

I do think Mike and Fordham deep down have consistent, principled views, but they cloud the picture with this triangulation.  It would also be easier to debate (and sometimes agree) with Mike and Fordham if they devoted less energy to positioning and more just to articulating their worldview and the reasons for it.



Getting Accountability Right

February 9, 2017

ignitepic1

(Guest Post by Jason Bedrick)

In a recent blog post, I argued the Mike Petrilli “fundamentally misunderstands accountability” because he sees it as meaning top-down regulations when the reality is that “true accountability is when service providers are directly answerable to the people most affected by their performance.” I further noted that, as Thomas Sowell has written, “It is hard to imagine a more stupid or more dangerous way of making decisions than by putting those decisions in the hands of people who pay no price for being wrong.”

Mike responds by arguing semantics, attacking straw men, and taking my words out of context. He writes:

As Jason knows, I agree with him—and with Sowell, Milton Friedman, and many others—that schools must be accountable to parents via the marketplace, that school choice is a better system than a regulated monopoly, and that I don’t in fact support “top down regulations.” What I and other “choice realists” want is for schools of choice to be lightly regulated yet subject to societal expectations regarding results. That’s what we mean by “accountability.” It’s Jason and his like-minded colleagues who have sought to redefine that word to mean “regulation.” Ironically, the teachers unions play the same game, saying that schools such as charters, which are partially freed from meddlesome government regulations (like tenure rules), are not “accountable.” This strikes me as mildly Orwellian.

Actually, it’s Mike who is trying to redefine the term “regulation.” Merriam-Webster defines “regulation” as “an authoritative rule dealing with details or procedure” or “a rule or order issued by an executive authority or regulatory agency of a government and having the force of law.” When the government issues a rule saying that schools that fail to meet its criteria are ineligible to accept vouchers from parents who would like to choose that school, that is a regulation. That may be a wise or foolish regulation, but it should not be controversial to recognize that it meets the textbook definition of a “regulation.”

If anything is Orwellian, it’s Mike’s insistence that we reserve the term “regulation” only for regulations he doesn’t like.

For the record, I support certain regulations — yes, regulations! — ensuring financial accountability to the taxpayer, such as requiring that ESA parents submit receipts, regular audits of such programs, etc. Taxpayers are owed accountability for funds they are forced to fork over, which means ensuring that they are only used for their intended purposes. However, accountability for academic results should lie primarily with those who have both the greatest incentive to help and most local knowledge about the children we are funding: their parents. In any case, I don’t pretend that the regulations I support somehow aren’t “regulations” just because I support them. Mike shouldn’t either.

Mike continues:

If Jason and Jay and others are saying that those of us in the Ivory Tower shouldn’t sit in our Star Chamber and decide which schools should live and die, based solely on their test scores, I say Amen. But that’s not what we’ve ever proposed. The charter sector has invented an entity called a school authorizer. The best ones know the schools they oversee inside and out. They aren’t “distant”; they build relationships with the schools’ boards, leaders, and parents. They understand the stories behind the test scores. And when reviewing the performance of “their” schools, they look at myriad factors besides those test scores. (As Andy Smarick has argued, voucher programs could use something like authorizers, too.)

Here, Mike is attacking a straw man. Were we talking about charters? No. The article to which I was responding was titled, “Vouchers have changed. Maybe your position should change, too.” In that article, Mike wrote:

In Louisiana, participating private schools that serve more than forty voucher students must administer all of the state tests to them. They then receive a “scholarship cohort index” score that’s used to determine whether they can continue to accept new students. Louisiana state superintendent John White has already triggered the provision to keep several schools from accepting new voucher-bearing pupils.

In Indiana, schools must administer the Indiana Statewide Testing for Educational Progress assessment and report their graduation rates to the states. These data are used to determine each private school’s A–F rating—just like their public school counterparts. If a school is rated a “D” or “F” for two or more consecutive years, it becomes ineligible to accept new scholarship students. (The information about these voucher programs comes from the American Federation for Children’s school choice yearbook.)

So if you oppose vouchers because of lack of accountability, it may be time to change your position.

My response was focused entirely on private school choice (i.e., vouchers, tax-credit scholarships, and education savings accounts). Mike continues to tout policies like those in Louisiana that likely contributed to the first negative outcomes in any random-assignment study of any voucher program, but when I call him out for this, he wants to change the subject and talk about charters instead. I wonder why!

(Note also: I’ve already responded to the Smarick proposal Mike plugged, which I think is creative but unwise.)

By focusing on charters, Mike is able to attack a straw man rather than address my actual points. I do not object to charter authorizers closing schools based on local knowledge, as he described above. I object vigorously to default closures (or expulsion from voucher programs) based on a pre-ordained formula, especially one (like the A-F grading system* he promoted in his previous article) that is heavily dependent on standardized test scores.

So, Mike, which is it? Do you support or oppose default closures? Do you support or oppose the Louisiana-style voucher expulsions? If you support them, then let’s talk about that rather than some non sequiturs.

Mike titled his recent article “Accountability to parents is necessary but not sufficient,” but he doesn’t even attempt to defend this position. Is there any reason to believe that mandating the state test and tossing out low-scoring schools would produce better results, in the long run, than mandating that voucher-accepting schools administer their choice of nationally norm-referenced tests and reporting the results directly to parents? If so, he doesn’t offer any. So-called “choice purists” prefer the latter regulation (there’s that word again!) because it avoids incentivizing the narrowing of curriculum or the refusal of quality schools from participation in voucher programs, as Lindsey Burke and I described in our recent report, “Recalibrating Accountability.” If Mike wants to have a conversation about the points we’ve actually made, instead of the non sequiturs he’d rather raise, I’m happy to talk.

Mike concludes:

Perhaps Jason, Jay, and Thomas Sowell would disagree. They might argue that taking decisions out of the hands of parents can never be justified. But with education, it’s simply not true that the public “pays no price” for being wrong. We all pay a hefty price when kids don’t learn. So, in rare cases, we need to act accordingly.

Here, Mike distorts the meaning and context of what I (and Thomas Sowell) wrote. I said that bureaucrats pay no price for being wrong, not that society pays no price. When a kid graduates high school unable to read his own diploma, no teacher, superintendent, or state education agency employee loses their job, nor does any politician (or, for that matter, any think tank wonks). Better to leave such important decisions in the hands of people who do bear the consequences of their decisions: parents.

In conclusion, I think Mike should update his own flawed choice taxonomy. At heart, he’s really a Super Nanny:

screen-shot-2017-02-09-at-8-20-14-am

(Just kidding, Mike! You know we love you.)

* UPDATE: Above I called Louisiana’s A-F grading system “heavily dependent” on standardized test scores. Actually, for grades K-6, it is entirely dependent on test scores, as the LA DOE website explains. For grades 7 and 8, 95% of the letter grade is based on those scores. For high schools, half the grade is based on the scores, and half is based on graduation rates. Mike says he likes using LA’s A-F grading system to kick out low-performing schools, but he also offers an “Amen!” to the notion that “those of us in the Ivory Tower shouldn’t sit in our Star Chamber and decide which schools should live and die, based solely on their test scores.”

So which is it, Mike?


Parents to Technocrats-mind if I cut in?

February 9, 2017

(Guest Post by Matthew Ladner)

Mike Petrilli published a piece on charter school authorizing and oversight in response to Jason and Jay. Before the fun starts, let’s just note the following:

  1. State authorities are going to occasionally close charter schools whether I, Mike, Jay, Jason or anyone likes it or not. Let’s therefore not worry about whether or not we should close a charter school caught having students sacrifice goats to Baal out on the playground- it’s going to happen.
  2. Whether or not this is going to happen or even should happen is not terribly relevant. Mike cites a 3% estimate for charters closed by authorities over academics. I’m personally comfortable with a far higher closure rate. So long as parents take the lead, I’m not going to sweat some authority jumping in front of the parade to close the Baal school.
  3. Ultimately therefore the debate should be about how to get to a policy environment where parents are taking the lead on quality control.

The reasons for this are ultimately very practical. Technocrats make mistakes and many do not develop the close relationships and sweat the details behind test scores the way Mike describes. More to the point all of these schools have access to the legal system, can lawyer up, engage in delaying tactics, get their parents riled up to resist closure etc. It is genuinely worth asking whether the juice is worth the squeeze in many cases.

Meanwhile, when parents close a school there is no resistance, no lawsuits, no delaying tactics. This is the most potent and brutally efficient form of accountability by a very wide margin. Now…breather deeply…

…close your eyes…

…channel your inner Rick Hess and think broadly about what sorts of policies and practices can get you there…

…do you see it?…yes….

Now…you are back on the green…nicely done….yes and now you are doing it again…

Good…very good…yes both of those states scored a 9/33 on NACSA’s ratings but rocked the 2015 NAEP like an 80s hair band trashing a hotel room suite that had it coming. Breathe even deeper…do you see a role for an all-powerful command and control technocrat in this vision?

No? Good-me neither. Light touch stuff inevitable, heavy-handed stuff risky and counterproductive, parent lead highly desirable. I don’t think there is a whole lot to argue about.

Okay open your eyes now. I think we have this all sorted out!


The (Incredibly) False Precision of Election and Sports Forecasting

February 6, 2017

My very first academic publication was (I thought) a devastating debunking of election forecasting models.  Several leading political scientists and economists had developed regression models that claimed to have perfect records of predicting presidential election outcomes.  I showed that these models were only perfect in “predicting” elections that had already occurred.  With only a dozen or so observations and five or so independent variables, these models were over-fitted to the data so that they could score a bulls-eye by drawing the circles after the arrows landed.

That was almost a quarter-century ago and despite my warnings election as well as sports forecasting have grown into major industries.  Their methods have become more complicated but the fundamental defect of this enterprise remain the same: most forecasters incredibly overstate the precision of their predictions.  That is, most election and sports forecasters claim to know things that they simply do not know.

Let’s take the recent presidential election to illustrate the problem.  Nate Silver, who has achieved celebrity as a leading forecaster, predicted just prior to the election that Hilary Clinton had a 71.4% chance of winning the presidency.  How did Silver arrive at this incredibly precise sounding forecast?  Basically, he takes a large set of state and national poll results (including their reported margin of error), weights the polls by his assessment of their past accuracy, and then given that each poll provides a probability distribution of the result he runs simulations that draw outcomes from those probability distributions by chance.  In 714 out of 1,000 such simulations Clinton would win the White House.

There are two major flaws in this method.  First, the margin of error reported by polls only captures the errors given that the model is correct.  We also have uncertainty about the composition of the model and that error is never quantified or reported.  To make this less abstract, consider the USC/LA Times poll. Every poll, including USC/LA Times, has to make a series of assumptions and choices to construct its model.  Each one of those assumptions and choices is made with a certain amount of uncertainty, which is not contained in the margin of error but can have a strong effect on the prediction.

So, when polls disproportionately get responses from certain segments of the population, they have to make choices about how to weight the responses they have so that they are representative of the full population for whom they are making the forecast.  But what method should they use for weighting their respondents?  You might not think such a small, technical detail would matter, but as the recent experience of the USC/LA Times poll indicates, the choices you make about weighting would determine the result.

We only know this about the USC/LA Times poll because they happen to make their raw data available in real time so others can re-analyze with different assumptions.  The USC/LA Times poll was also exceptional in that it was one of the only polls predicting a Trump victory (although in the end even they were mistaken in that they were predicting the share of the popular vote, which Trump lost).  Their openness about the data and their outlier prediction attracted Ernie Tedeschi to re-analyze their data using different assumptions about weighting responses.

As it turns out, the USC/LA Times poll used a non-conventional method of weighting their respondents to be similar to the share of people in the Census within “micro-cells.”  They would re-weight their respondents to have the same proportion of small groups as found in the Census, like college-education, African-Americans under 30, rather than large groups, like African-Americans or college-educated people.  Tedeschi showed that if you re-weighted the USC/LA Times poll using large groups you would flip the result, so that Clinton had a large lead rather than Trump.

The point here is that whether you weight by small or large groups does not seem to be theoretically important, but it makes an enormous difference in the predicted result.  The uncertainty we have about which seemingly arbitrary weighting method to use is not captured at all in the reported margin of error.  That margin of error only tells us the uncertainty within the model.  But not knowing how to weight respondents represents uncertainty about the model.  Silver’s simulation method only incorporates the error within models, not the error about models and therefore grossly overstates the precision of his estimates.

The USC/LA Times poll illustrates the second major flaw, which is that the polls are not independent observations as Silver’s method assumes.  That is, errors about model construction are correlated across polls, meaning that if they are wrong about something, they will almost all be wrong together.  For example, one of the criticisms leveled against the USC/LA Times poll is that its weighting method was not the “convention” or “best-practice” among pollsters.  Another way to put that is that pollsters tend to make common assumptions about the construction of their models.  But if any of those seemingly unimportant assumptions turns out to be wrong, not only will the forecasts be off, but the error will be common across polls.

Silver’s simulation method requires that the errors of model construction are random and will balance themselves out across polls so that one will make a wrong assumption in one direction while another makes a wrong assumption in the other direction.  But that isn’t right.  Pollsters are making their decisions mostly in common according to conventions or best-practices.  The USC/LA Times poll stood out because it deviated from that herd mentality among pollsters.

These two errors I’ve described are not small things.  As I’ve shown, a single seemingly unimportant decision about how to weight respondents can change who the predicted winner is.  And pollsters are making dozens of these decisions, any one of which could alter the predicted outcome, without even knowing that they are doing so.  And on top of all of that, the pollsters tend to make common decisions about these matters, so when they are wrong, they and Silver will all be wrong together.

Silver wasn’t just unlucky.  The truth is that neither he nor the pollsters on whom he relies could know a priori what the right assumptions were about weighting, turnout, or a host of other matters.  If he simply said that he thought it was more likely than not that Clinton would win, he might be able to defend his prediction.  But given all of the uncertainty I’ve described taking his prediction to the tenths place just shows him to be a quack.

Unfortunately, we just have a hard time dealing with uncertainty.  It would be comforting to us if the world were orderly and predictable, so we turn to charlatans like Silver, or the UpShot, or ESPN’s in-game prediction model as people once turned to lucky totems or end-of-days preachers.  The hard reality is that there is a lot more uncertainty in the world than these folks would admit and the highly infallible methods of social scientists and charismatic preachers offer little relief from that reality.