(Guest Post by Jason Bedrick)
Yesterday, Jay took apart the factually challenged hit piece the Gray Lady ran on charter schools in Detroit. Matt added his two cents and I piled on as well. Additionally, Tom Gantert of the Mackinac Center for Public Policy identified another dozen or so errors, distortions, or omissions of key facts.
But the most illuminating part of this whole imbroglio was the following Twitter exchange with the NYT reporter:
Jay and others, including Prof. Martin West of Harvard, explained how the article misrepresented the findings of CREDO’s research on charter schools. The NYT reporter, Kate Zernike, defended her piece by noting that she contacted CREDO about the charter she highlighted, which CREDO had found to be low performing. However, Jay pointed out that she cherry-picked that one bad charter and failed to note that CREDO found overall positive results.
Zernike apparently forgot the First Law of Holes: when you find yourself in one, stop digging.
Jay also took her to task for relying only on “claims and anecdotes” instead of data, so she cited data from Excellent Schools Detroit.
However, as Jay explained, those data do not allow for direct comparisons. Zernike is right that the data show the citywide averages in each sector, but looking at the averages is misleading. Zernike seemed to think Jay was objecting to the standardized tests, but the real problem he identified was not the tests themselves but rather how she tried to compare the students taking those tests.
As I noted in my blog post:
The charter schools tend to be mission-based schools that open in the toughest areas and serve the most at-risk students. Comparing city-wide averages fails to take that into account. It would be like comparing the New England Patriots against a championship high school team and concluding that the teenagers are superior athletes because they scored more touchdowns per game.
The appropriate comparison is between the charters and the district schools that serve the same or similar student populations. That is what the CREDO study attempted to do by matching students with similar characteristics and initial test scores in each sector, then tracking and comparing them.
Amazingly, Zernike then claimed that CREDO did not find that Detroit charters had a positive effect overall. In fact, that’s exactly what CREDO found in both its 2013 report on charters in Michigan and its 2015 nationwide charter report.
As Matt noted, the 2013 CREDO study on Michigan found: “on average, charter students in Michigan gain an additional two months of learning in reading and math over their [traditional public school] counterparts. The charter students in Detroit gain over three months per year more than their counterparts at traditional public schools.” [emphasis added]
Nevertheless, Zernike persisted in claiming that CREDO found results that it didn’t find.
That’s right. Zernike says CREDO “does not consider Detroit stellar” — but page 33 of the 2015 CREDO report calls Detroit’s charter sector “a model to other communities.” See for yourself:
“models to other communities”
But apparently some reporters don’t like it when people point out that what they wrote is factually incorrect. Here was her response to my Tweet correcting her:
Blocked!
If the New York Times has any integrity, it will issue a correction.
Jay P. Greene's Blog 
BOOOOOOOOOOOOOOM!
I was not surprised to see Kate Zernike’s name associated with a misleading story about education policy.
In August 2000 Paul Peterson and a team of scholars (William Howell, Pat Wolf, and David Campbell) reported findings of statistically significant test score gains among students in several different school choice programs.
Several prominent stories appeared in various major media outlets. A balanced August 29 story in the New York Times by reporter Edward Wyatt appeared under the headline, “Study Finds Higher Test Scores Among Blacks.”
On September 15, a Times follow-up story (by Kate Zernike) appeared under the headline, “New Doubt Is Cast on Study That Backs Voucher Efforts.” The subheadline added,
“Black Youths’ Gains Are Called Overstated.”
Zernike’s story was about 50% longer than the August 29 report by Wyatt. Its headline, subheadline, length, content, and tenor strongly hinted at questionable conduct by Harvard’s Peterson.
The heart of Zernike’s story was her claim that David Myers of Mathematica had accused Peterson of “overstating” New York City test scores. Zernike reported that Mathematica, “bothered by what it describes as [Peterson’s] exaggerated claims…has taken the unusual step of issuing a statement that cautions against leaping to any policy conclusions.”
On September 16, a day after the Zernike story, Howard Fuller and I interviewed Myers by telephone. With his permission, the call was recorded.
The following excerpt from this conversation (and Mathematica’s actual written statement) show that neither Myers nor Mathematica named Peterson (nor Howell, Wolf, or Campbell) as having “overstated” or “exaggerated” anything.
Myers: We [at Mathematica want] to be very cautious about saying that in New York City there’s an impact [from vouchers]…I want to understand it better before I would make policy about it…That’s the gist of the debate here.
Mitchell: But that’s not what is being reported…[Do] you believe that Paul has inaccurately represented the [NYC] findings. That’s…what the press is…saying. That is the spin. This story [has been] turned from one in which there appears to be substantial agreement to one [which presents] a very
harsh view of Paul Peterson’s integrity…I’m trying to determine as directly as I can whether you associate yourself with that. Has Paul…can you cite something in his report that is inaccurate? That’s my question.
Myers: No…[T]here’s nothing…statistically that’s inaccurate.
Mitchell: [But] has [Peterson] overstated the policy implications of the NYC findings in any [other] way in the report or in the any other venue that you can cite? Because you have stated [in this conversation] — and again I’m trying
to be very precise — [that] these [NYC] results are statistically significant but much more needs to be known before one could argue that [they] should influence policy. Has Paul Peterson said otherwise?
Myers: I don’t know. I don’t have his report in front of me. [In fact, the report included cautions consistent with Myers’ view.]
Mitchell: OK, but you don’t know that he has?
Myers: Right.
Mitchell: OK…David…[to review] you answered no to the question as to whether [Peterson] said anything inaccurate and that you don’t know if elsewhere he may have overstated the impact on public policy. I think that is your belief…?
Myers: Well, right.
Zernike declined to respond to our e-mail inquiries about her story. Ethan Bronner, her supervisor, told us that Zernike’s story only reflected Myers’ concern that the New York City results not be used to “generalize” support for broad findings. However, the report itself emphasized that very point with respect to results from all three cites, a fact Zernike did not report.
The excerpted transcript above is part of a lengthy chapter in a report that Fuller authored with my research assistance. It documents other errors of commission and omission in Zernike’s long story.
After reading the last line of your post (“if the New York Times has any integrity”) I thought of this:
At only the midway point in the year, our 2016 folder is already bulging with deserving nominees:
http://choicemedia.tv/2016/05/10/2015-worst-education-journalism-awards/