(Guest Post by Matthew Ladner)
Ok so here is what is going on in this chart: NAEP Math and Reading tests are timed and scaled in such a way as to allow for the calculation of cohort gains. In this case, we’ve tracked the statewide gains for students with disabilities from 4th graders in 2011 to 8th graders in 2015. Both the 2011 and 2015 measurements are a population estimate, and NAEP of course is not tracking the same students over time but rather are sampling both populations. The calculation used here is a straightforward 2015 8th grade scores for students with disabilities minus the 4th grade 2011 scores for students with disabilities, and then calculated as a percentage of improvement between 4th and 8th grade.
Students move in and out of states over time, but this sort of error should be largely random and cancel itself out in the absence of some (relatively implausible) systematic bias (like in this case higher performing students with disabilities fleeing Maryland to live in Hawaii). Given the standard errors, there isn’t much reason to fuss over exactly where you stand if you land say in the middle of the blue blob in the chart above, although one might take an interest in the states landing in the top right or bottom left.
Congrats to Hawaii and Arizona. Bad look for Maryland if taken at face value- having one of the nation’s highest spending per pupil figures but failing to teach students with disabilities much of anything about math and reading over a four years is, ah, terrible. Maryland is a state that had in earlier years flouted the NAEP’s inclusion standards for children with disabilities. It is possible that if they stopped doing so in 2015 that it may explain part of their place on this chart. If I lived in Maryland I would get to the bottom of this, but it’s time to get out of my pajamas.
For Hawaii and Arizona:
We’ll circle back and see how this goes when the new NAEP data is released in January.
Posted by matthewladner 
Jay P. Greene's Blog 