Getting Facts Right

Image result for Everyone is entitled to his own opinion, but not his own facts.

Education policy debates, like national political disputes, are increasingly frustrating because we are having difficulty agreeing on and sticking to a set of facts.  It’s particularly frustrating because people are mis-stating basic facts that quite often are not even necessary for continuing to hold their opinions.  I’m not sure what accounts for this sloppiness in public discourse, but I suspect that it is a general decay in professional norms and standards of behavior encouraged by the stupid brevity and speed of Twitter.

Let me describe a recent example of an obvious, factual error from a debate about charter schools and regulations.  To be clear — there’s nothing wrong with making mistakes.  So what is remarkable about this example is how the people making the demonstrable, factual error never say “Oops, I made a mistake.”  They could easily admit the mistake and still continue to hold their opinion and make their arguments.  The fact that they don’t is what is truly troubling.

A Recent Example — Greg Richmond, who is the head of the National Association of Charter School Authorizers (NACSA), wrote a piece for The 74 in which he claimed: “A recent report from Tulane University’s Education Research Alliance found that a school’s rating from the independent application review process was the only factor that predicted its success.”  But if you look at the report, specifically in Table 7, you will see that the NACSA rating of charter school applications is not a significant predictor of any student outcome.  Richmond’s claim is demonstrably false.  He says that a report found something that it simply does not find.  I had previously noted the falsehood of this claim, but this has not deterred the folks at NACSA from repeating it.

Rather than admit the error, NACSA tries to rescue the claim.  Importantly, they don’t rescue the claim by showing that it is true.  They can’t.  Instead, they shift the discussion to a different point.  NACSA’s Nelson Smith, echoed by their spokesperson Kristen Forbriger, quote a policy brief written by Doug Harris summarizing the findings of the report he co-authored and that they falsely invoke:

“None of the application measures predict the value-added performance of schools, though there are signs of a positive relationship between the NACSA ratings and value-added (emphasis added). It is not surprising that our statistical confidence is weak here because value-added measures are imprecise and the NACSA ratings did not vary much among approved applications.” In other words, it’s hard to detect correlation to specific outcomes when the approved applications all scored at high levels.

Note that this does NOT say that the report “found that a school’s rating from the independent application review process was the only factor that predicted its success.”  In fact, it clearly states that Richmond’s original claim is false: “none of the application measures predict the value-added performance of schools.”  And saying there are “signs of a positive relationship” is just spin, not the actual finding.

Rather than support the truth of the original claim, this quotation provides a rationalization for why the report did not unearth the desired finding — the sample size was too small and there was too little variation in the NACSA rating.  Whether the rationalization is persuasive or not, the fact remains that the report did not find what Richmond claims it found.  His assertion is demonstrably false.  But neither he nor his colleagues at NACSA will say so.  Instead, in Trump-like fashion, they continue to assert that they were right all along even as the evidence contradicts them.

Unfortunately, this recent example of a factual error that is never acknowledged or corrected is becoming part of a pattern.  If people in our field can make demonstrably false claims without having to acknowledge or correct them, it’s unclear to me how we are going to make progress in policy debates.  I am not arguing here what the correct policy should be.  I’m simply arguing that if people in these debates can make false claims with impunity, we have allowed the Trumpization of debates to creep into the area of education policy and will suffer similar highly negative consequences.

 

3 Responses to Getting Facts Right

  1. Jason Bedrick says:

    Bravo

  2. Greg Forster says:

    Excellent work, Jay. A few points about the diagnosis:

    1) Of course professional mendacity has always been with us. I recently learned the following true story:

    In the Alger Hiss trial, the chairman of the Harvard Psychology Department testified under oath that he had diagnosed Whittaker Chambers (the key witness against Hiss) as delusional by reading the articles Chambers had written at Time magazine. The prosecutor asked if he had contacted Chambers’ colleagues at Time to ask about him. He said under oath he had not. The prosecutor informed him they had a whole string of Time staff prepared to testify that he had contacted them and asked if they had any damaging information on Chambers. He recanted his testimony and admitted he had gone fishing for dirt to justify his “diagnosis.”

    2) To the extent that mendacity has become more possible, I think it is less the erosion of professional norms and more the rise of new jobs in which people who do not really belong to the profession are able to pose as members of the profession and usurp its authority (a la “drill and kill”).

    3) The erosion of professional norms is in turn only the local manifestation of a global breakdown in norms of truth-telling; here Twitter and other social media are root causes of the problem.

  3. Agree on “global breakdown in norms of truth-telling. Has little to do with who are presidents are. It’s been going on for years by elected school boards, state and local, and school administrators. What can parents do?

    https://www.amazon.com/dp/0966154894/ref=redir_mobile_desktop?_encoding=UTF8&keywords=indoctrinating%20our%20yourh&qid=1500035092&ref_=mp_s_a_1_fkmr0_1&sr=8-1-fkmr0

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: