(Guest Post by Matthew Ladner)
The National Center for Education Statistics released the 2013 NAEP this morning for 4th and 8th grade Reading and Math. I will be crunching the numbers for some time to come, but here is a quick look at the net results by state jurisdiction (combined 2013 scores minus combined 2011 scores).
Quick takes: DC and TN crushed the ball with gains four times larger than the national average. These look to be truly historic gains with both DC and TN scoring statistically significantly higher in all four tests.
Indiana scores big with gains almost three times the national average. Florida gets back on track with gains more than twice the national average.
More number crunching to follow but a
BOOOOOOOOOOOOOOOOM!!!!!!!!!
for top gainers DC, TN and IN is already in order.
Jay P. Greene's Blog 
Clearly, DC school reforms have been ineffective. No doubt the spike is attributable to the implementation of Obamacare, just as Florida’s gains were the result of Harry Potter.
Isn’t it amazing that DC banked all of those gains DESPITE half of their kids going to charter schools? Very impressive- the DC union bosses should be very proud.
You sound like the NYT reporter who famously wrote that crime was down despite the fact that many people were in prison. Of course it was sending the criminals to prison that brought the crime rate down. Likewise, giving parents the freedom to choose improves all schools, not just the schools of choice.
Greg, that’s mean-spirited. And racist.
[…] on the right trajectory. Education Week and education policy guru Matt Ladner both pointed out that Washington, D.C., and Tennessee easily showed the most progress since the last test in 2011. Indiana, Washington, Hawaii, and Florida also made significant […]
Thankfully Indiana has chosen to put off implementing the Common Core. Looks like our old standards were starting to pay off.
That was quite a delayed reaction for Indiana’s standards. Indiana NAEP scores were about as flat as the highway between Dallas and Fort Worth until 2013.
Ranking States by gains over their previous performance rewards States that improve over a low baseline. It’s easy for a mal-nourished child to gain weight. The statistic “gain” needs to be paired with the baseline “mean” oe “median”.
The chart shows that there are some malnourished states that are losing weight as well.
I don’t mean to dismiss gains. That’s to be celebrated. It’s strange to see Massachusetts, North Dakota, and Montana, which usually compare favorably to the rest of the US, in the cellar and Hawaii and DC, perennial cellar-dwellers, among the leaders. It’s like a seriously LD 14-year-old has learned to tie his shoes.
[…] continues its steady climb. Yes, there are states like Tennessee and Washington, D.C. that showed far greater improvement from two years ago than the national average, and many will be quick to point to one reform or […]
Matt,
I hope you revise your analysis. You have aggregated the scale score results of four different tests with four different samples reported on four different scales. And each has its own distribution. One way to handle that is to compute z-scores for each of the four tests and then aggregate (or average them). You should also probably adjust the scores–through regression–for demographic changes that may have influenced scores. That analysis still will not get you where you seem to want to go, claiming that particular policies caused a particular NAEP result, but you will be closer than you are now.
Tom Loveless
Not only that, Tom: even a z-score assumes that a “gain” at one part of the distribution is the same as a “gain” at another part in real learning. There is no basis to believe this is the case.
Also: Matt, you work for Jeb Bush, do you not? Doesn’t his Chiefs For Change claim as members ed commissioners in FL, TN, and IN?
Shouldn’t you have mentioned that in this post?
Mark-
Jeb Bush is the chairman of the board of my employer, but currently only the education commissioner of TN is member of Chiefs for Change among the states you list. My association with Governor Bush is common knowledge to regular Jayblog readers, so no I did not see any reason to mention it.
“Currently.” That conveniently leaves out Tony Bennett.
And my point about score gains stands: there is no reason to think a move from 200 to 205 in 4th Grade math is in any way equivalent to a move from 220 to 225 in 8th Grade reading.
And yet that is exactly what your analysis does.
Mark-
I have it good authority that Secretary Arne Duncan called both Mitch Daniels and Tony Bennett to congratulate them on Indiana’s NAEP gains.
If you look hard enough you might be able to find a USDoE website where you can leave comments to complain about z-scores.
Tom-
I haven’t made any causal claims about policies causing gains. The world is a bit too complicated for that on the first day of the data being released. I am in a hotel room in Milwaukee and so not in the best position to calculate z-scores, but if I get the itch to make causal claims I will scratch it as you suggest.
TN and DC had statistically significant gains in all four tests so they seem to be doing something (probably different things) right. I won’t however claim to know what those things are, I’m simply happy with the results.
I wouldn’t get too excited about those DC gains. It appears that they were largely driven by large improvements by affluent students. As DC residents have noted, gentrification in DC has increased. So it can be logically inferred that bigger gains for affluent kids + more affluent kids = big gains.
It also means that DC saw an increase in achievement gaps.
http://www.edtrust.org/dc/press-room/press-release/statement-and-analysis-from-the-education-trust-on-the-2013-naep-reading
Steve-
I have not yet been able to dig into the DC scores in detail, but I can already tell you that both poor and non-poor students have seen gains.
Before this year, DC did indeed see huge and growing achievement gaps, but given that they occurred in a context in which the bottom was coming up in a big way, I really could not care less. When low income scores are increasing, it seems silly to fret about the two attorneys who moved in a few miles away with their bright children to take six figure federal gigs.
This sentiment on my part is based upon 2011 data however, and crunching 2013 is on my to do list.
For your consumption regarding D.C. from WSJ:
http://blogs.wsj.com/economics/2013/09/19/washington-sees-incomes-soar-as-most-of-u-s-declines/
More that just “two attorneys.” If you’re talking averages, this is a reasonable explanation for D.C.’s rise. Even with low-income scores rising, the burden is easily carried by the affluent here. This also reflects an ever-increasing widening of the achievement gap. Or is that not really a concern in this case? Because the focus schools concept is being used to drill several communities where I live.
No praise for charter-free Washington state? 4th best and not far behind “ball-crushing” Indiana.
No criticism for charter-infested Louisiana?
Steve-
I agree that gentrification is going on in DC. In the 2011 numbers DC scores were very strongly improving at the bottom and improving at an even faster pace among the non-poor. In my view it would be unwarranted to worry about this “increasing the achievement gap.”
The opposite phenomenon was going previously in the West Virginia trend. So for instance the Black/White achievement gap “closed” but only because both groups had declining scores but White scores were falling faster than Black scores.
I would take the DC trend every day of the week and twice on Sunday over the West Virginia trend.
[…] These outlets were quick to jump on just the sort of statistic Krstof did. Because some states that have implemented the favored policy reforms had gains in their NAEP scores that outpaced the national average, these “analysts” made the case that “reforms” – such as, using high stakes test scores to rate teachers and schools and increasing the numbers of unregulated, privately operated charter schools – are “working.” […]