NAEP results are being released next week, but state departments of education have already been briefed on their results. State education officials are leaking like sieves, so many edu-pundits have at least some inkling of what’s coming. Rumors from multiple sources suggest that the results generally look bad — with a decline nationally. Aware that they may be blamed for declines, a number of folks are anticipating the release by placing their own spin on the soon-to-be-released results.
Exhibit A in this pre-spinning is John White, who is the superintendent of education in Lousiana. According to Matt Barnum at Chalkbeat White sent a letter to the U.S. Department of Education on March 23 raising concerns about the comparability of NAEP results over time given the transition to computer-administered testing. Although he acknowledges that NCES made adjustments to ensure the comparability of paper and pencil and computer administered tests and found insignificant differential effects by sub-group, White still raises questions about whether differential effects may distort results for certain states. Barnum notes: “Even though researchers warn that it is inappropriate to judge specific policies by raw NAEP results, if White’s letter is a signal that Louisiana’s scores have fallen, that could deal a blow to his controversial tenure, where he’s pushed for vouchers and charter schools, the Common Core, letter grades for schools, and an overhaul of curriculum. White said his state’s results are not what’s driving his concerns.” Hmmm. Maybe it’s just a remarkable coincidence that White has suddenly developed these technical concerns about the validity of NAEP at about the same time that he was briefed on his state’s results. How much do you want to bet that there is a decline in LA?
Exhibit B is Arne Duncan taking to the pages of the Washington Post to defend the idea that ed reform has contributed to significant improvement. He focuses on trends over the last few decades. That would be a smart move to focus on long-term gains if recent trends — you know, in the wake of Duncan’s tenure as U.S. Secretary of Education — have been taking a nose-dive. NAEP results slipped for the first time when 2015 results were released. How much do you want to bet that national results have declined again?
Barnum is right to warn people away from inferring too much from changes in NAEP as an indicator of the success of any particular policy or education leader, but these folks live in a political, not a research, world. Both White and Duncan’s political standing in education policy was built on over-claiming from NAEP results, and those who live by the NAEP sword may die by it. That’s why they better start spinning.