NAEP results are being released next week, but state departments of education have already been briefed on their results. State education officials are leaking like sieves, so many edu-pundits have at least some inkling of what’s coming. Rumors from multiple sources suggest that the results generally look bad — with a decline nationally. Aware that they may be blamed for declines, a number of folks are anticipating the release by placing their own spin on the soon-to-be-released results.
Exhibit A in this pre-spinning is John White, who is the superintendent of education in Lousiana. According to Matt Barnum at Chalkbeat White sent a letter to the U.S. Department of Education on March 23 raising concerns about the comparability of NAEP results over time given the transition to computer-administered testing. Although he acknowledges that NCES made adjustments to ensure the comparability of paper and pencil and computer administered tests and found insignificant differential effects by sub-group, White still raises questions about whether differential effects may distort results for certain states. Barnum notes: “Even though researchers warn that it is inappropriate to judge specific policies by raw NAEP results, if White’s letter is a signal that Louisiana’s scores have fallen, that could deal a blow to his controversial tenure, where he’s pushed for vouchers and charter schools, the Common Core, letter grades for schools, and an overhaul of curriculum. White said his state’s results are not what’s driving his concerns.” Hmmm. Maybe it’s just a remarkable coincidence that White has suddenly developed these technical concerns about the validity of NAEP at about the same time that he was briefed on his state’s results. How much do you want to bet that there is a decline in LA?
Exhibit B is Arne Duncan taking to the pages of the Washington Post to defend the idea that ed reform has contributed to significant improvement. He focuses on trends over the last few decades. That would be a smart move to focus on long-term gains if recent trends — you know, in the wake of Duncan’s tenure as U.S. Secretary of Education — have been taking a nose-dive. NAEP results slipped for the first time when 2015 results were released. How much do you want to bet that national results have declined again?
Barnum is right to warn people away from inferring too much from changes in NAEP as an indicator of the success of any particular policy or education leader, but these folks live in a political, not a research, world. Both White and Duncan’s political standing in education policy was built on over-claiming from NAEP results, and those who live by the NAEP sword may die by it. That’s why they better start spinning.
Jay P. Greene's Blog 
No doubt we’ll be hearing a great deal about the need for more money, smaller classes, etc. A few years ago, choice advocates also would have seized upon a NAEP dip as evidence of the need for choice. Given the new concerns about overusing test scores, which I think have actually gotten a pretty good hearing in the movement, I wonder what will happen. The critique of overusing test scores does leave us either the question of how we make the case to the public that reform is needed. People overestimate the quality of their own schools and some way of evaluating across the board is needed.
Does anyone know if NCES ensures a review of all NAEP math test items by small groups of mathematicians before the items are used? This wasn’t the case before 2007.
https://nces.ed.gov/whatsnew/commissioner/remarks2007/11_23_2007.asp (Response to National Validity Study by Mark Schneider in 2007)
See especially Section 5 A and final section on “marginal” and “flawed” test items
https://files.eric.ed.gov/fulltext/ED499213.pdf (NVS 2007)
And if new math items were added for the 2017 tests?
States have more or less discredited themselves in the labeling schools enterprise, and research shows that parents trust non-profit sources of school information than state sources in any case. A light footprint system with a menu of assessments and the data going to be crunched by multiple non-profits platforms like GS and Niche etc. seems like a big step in the right direction. It’s time for the notion that we should be attempting to standardize children to end.
Amen.
[…] light of the fact that the 2017 NAEP Results will be released next week (and which may be lower than hoped), here at OEP we thought it fitting to examine Arkansas’ performance over time. A […]
[…] Reform at the University of Arkansas, says that it looks like “pre-spin” to him. He wrote on his blog, “Maybe it’s just a remarkable coincidence that White has suddenly developed these […]
[…] a vested interest in the success of Common Core, these officials have been dreading release of the 2017 NAEP scores (the National Assessment of Educational Progress, AKA the […]
[…] National The Pre-Spinning of NAEP Results https://jaypgreene.com/2018/04/02/the-pre-spinning-of-naep-results/ […]