(Guest Post by Matthew Ladner)
Shakespeare’s Henry V includes a scene in the tent of the French nobles in which the Dauphin prattles on an on about how many of the English he plans to slaughter on the following day in the battle of Agincourt. “Will it never be day!?!?!” the Dauphin exclaimed. Careful what you wish for…
NAEP’s 2017 results for Math and Reading for states and select large urban districts will become available tomorrow. I’ve got my basement primed with six laptops and a case of Monster energy drinks. Mrs. Ladner is planning to occasionally open the vault to hurl additional pizza boxes downstairs before quickly re-sealing civilization’s final defense against anarchy. A few notes:
Large score movements either up or down are rare so up is better than down but when examining a statewide average score. On the math and reading tests 10 points roughly equates to an average year worth of progress (e.g. if we did a random assignment exercise and had one group take the 4th grade math test as 4th graders and the other group as 5th graders we would expect the 5th graders to score about 10 points higher). Most gains or declines are of the incremental variety (1-3 points) but they can add up over time like in Arizona, or cancel themselves out like in New York:
Was the 2015 NAEP a disappointing blip or the start of a new trend? A mere quarter century of national math improvement at both the 4th and 8th grade level came to an end in 2015. We’ll find out tomorrow.
If a trend, cry HAVOC and let slip the dogs of spin! Put me down as skeptical on some sort of lagged impact of the great recession given things like:
This will be a big NAEP for the Common Core project. I have not seen any reason to doubt the Tom Loveless analysis that measured the impact as a tiny positive. A tiny positive however was not what was promised. Let’s see what happens next.
Discipline Reform. I’ve been reading the debate on the discipline reform efforts of the Obama administration. I have no idea whether discipline reform had an adverse impact on academic performance, but if we are looking to subject the notion to conjecture and refutation, I would take the most interest in the score trends of low-income students of color in early adopting jurisdictions, and I would take a greater interest in 8th grade scores/trends.
Computerized testing. They piloted it in 2015, and then delayed the release of the 2017 NAEP to further study whether to make additional adjustments. Peggy Carr, the acting commissioner of the National Center for Education Statistics, told reporters last week “We have the full weight of scientific, psychometric evidence behind the release. We are comparing apples to apples. I think we can all be confident without a shadow of a doubt that we are going to be looking at true performance.” Perhaps understandably given the pressures involved some state chiefs took an interest in technical aspects of NAEP testing shortly after being briefed on their state scores, but the National Center for Education Statistics has a head start several years in the running on studying this issue, so I tend to believe them, unless Arizona scores tank, in which case…just kidding.
Stay tuned to this jayblog channel and remember…
Jay P. Greene's Blog 
Beware the desire to reach all conclusions based on standardized testing that is anything but standard.
From one school to the next, the quality of the I.T. network is absurdly disproportionate. It’s impossible to generalize about the level of deficiencies either.
I can take you to a highly successful Scottsdale school that still has IBM PC’s in classrooms.
I can take you to another highly successful SUSD school that proudly points to a huge uptick in school wide test results that was admittedly based solely on how many times they had to reboot the network mid-test and start over. The installation of new carts of smart books made the school and its teachers and students and teachers look brilliant. (Except to those who really knew what happened.)
I can also take you to schools in Yuma with holes in the ceilings where they have a lightning fast network donated by kind souls who cared.
Paper based tests didn’t pose the same issues. Perhaps we need to go back to a box of sharpened #2’s to achieve consistency in analysis.
Mike Norton has raised good questions about one of the two major questions that can be asked about the validity of NAEP’s tests with respect to digital results vs. paper/pencil.
The other is whether there can be any validity at all for NAEP math test results when we don’t know if any mathematicians/scientists vetted all, both old and new, math test items.
http://www.educationviews.org/who-watches-the-watchmen/
Beep you? How long have you been locked in that vault living on pizza boxes, Matt?
There is always a spot for Scooby gang reruns in the vault!
You said you had six laptops, you didn’t say what was on them.
Great pizza in Cleveland Circle, Greg.
Touché! I hope he occasionally gets to eat some of the pizza and not just the (presumably empty) boxes being tossed into the vault.