Education Reform 2003 to 2017: Modest Success/Epic Failure so What’s Next?

July 23, 2018

(Guest Post by Matthew Ladner)

Having had some time to reflect upon the 2017 NAEP, let’s take stock of things. In this we should keep in mind our broad ignorance between policy changes and state NAEP trends- and the same goes for average school quality. NAEP gives regular 4th and 8th grade scores in math and reading, and all 50 states have participated since 2003. 8th grade scores are more likely to reflect school quality than 4th grade scores in my opinion, as the students have more years of schooling. I’m not sure what to make of positive 4th grade score trends that do not result in higher 8th grade scores for instance. So this in essence a window into what we have to show for American K-12 reform 2003 to 2017 in 8th grade math and reading by state:

So what to make of the above chart? The below chart eliminates a lot of clutter by only including the states with statistically significant gains in both math and reading 2003-2017:


So 19 out of 50 states demonstrate statistically significant gains in both 8th grade math and reading. Notice also the absence in the second chart of mega-states Illinois, New York and Texas (although it is good to see California and Florida making it in). Texas has as many K-12 students as the 20 smallest states combined and annually adds approximately a Wyoming public school system sized number of new students. Florida has half as many students as Texas and California is still larger than Texas.

Since we don’t know the relationship between policy and academic trends, we are limited in the conclusions we can draw with confidence. Having said that, policies that have been broadly applied across all 50 states apparently suffer from severe limitations in their ability to move the needle academically. All 50 states for instance have adopted state academic standards and accountability exams, but most states have failed to move the needle on 8th grade scores. Even if we were feeling incredibly generous and made the wild assumption that none of the second chart gains would have happened in the absence of testing, a failure rate of 62% after 14 years is a far cry from leaving no child behind.

Mike Petrilli and Peter Cunningham recently offered up “where do we go from here” think pieces. I think Mike has some interesting ideas, but Peter’s call for a vast increase in spending is broadly unrealistic imo given the nation’s trillions of dollars in unfunded pension and entitlement liabilities, 10k Baby Boomers per day reaching the age of 65, etc. In normal times, Mike’s incremental adjustments might make a lot of sense, but we don’t live in either normal times, or in times that are going to allow some Great Society on Steroids increase in K-12 spending.

A much more difficult scenario may loom whereby the district system continues to resist reform, reformers continue to push reforms the public does not care for, and severe funding needs for increased health care spending leads to a broad reduction in per pupil spending.  State constitutions guarantee K-12 funding, but whether or not they will be creating schools that the vast majority of parents will continue to entrust with their children, I don’t feel as confident about. There are hopeful signs in the NAEP from state charter sectors:

…but the rate of charter growth has slowed substantially nationwide. Of particular disappointment are the last several state charter laws to pass that produce very few charter schools. Even states with relatively fast growing sectors have large wait lists. There are alas limits to what we can realistically hope for from a charter movement that has to a large degree lost its way by prioritizing cartel behavior over the interests of children on wait lists imo.

The private choice movement enjoyed a strong run earlier in the decade, but has since ran into political headwinds. Many private choice programs exist, but most remain modest in scale. The case for private choice remains as strong as ever, and the need will continue to grow, but the looming state funding crisis is coming fast. In four years, half of the Baby Boom generation will have reached the age of 65, and by 2030 all of them will be there. They have called dibs in advance on all plausible funding increases and a whole lot more.

So what is next? An increasingly likely scenario in my mind is that state district systems retain their flaws but loses a significant part of their funding and that choice systems continue to fail to meet existing much less expanded demand. In such a scenario an increasing percentage of families may decide to fend for themselves. Call them home-schools, home-school co-ops or micro-schools, my spidey-sense tells me that we should expect to see a great many more of them in the years ahead. I’ll write more about this in a follow-up post.

The NCLB Era in One Handy Chart

March 21, 2018

(Guest Post by Matthew Ladner)

Unfortunately a large majority of the nation’s K-12 students are in the tight cluster of meh and sub-meh in the stagnation cluster. Judged by 8th grade math and reading gains 2003 to 2015, Arizona, Hawaii and Tennessee are having the best improvement. New York is still alright if you like saxaphones academic stagnation.

The 2017 NAEP will be released on April 10. Anyone else believe in any of these blue dots enough to dare a prediction?

Time to Bet on Black in 2017

January 11, 2018

(Guest Post by Matthew Ladner)

I am going to be on a panel at the Arizona Townhall tomorrow about racial achievement gaps, so this had me taking a look at data. Arizona’s Black students had the highest NAEP 8th grade math scores in the country in 2015, so this got me curious to how close this would get them to the lowest average statewide scores for White students (not that this bar is high enough but better to pass it than not and sooner rather than later or never).

Put me down for $20 on the 2017 NAEP- I’m betting on Arizona Black baby!

TUDA districts versus states

December 12, 2017

(Guest Post by Matthew Ladner)

So…Chicago…? Anyone?

New York’s Alright If You Like Saxophones, Taxes and Meh School Performance

November 8, 2017

(Guest Post by Matthew Ladner)

NAEP Reading Scores from 2015 along the horizontal axis, NAEP reading cohort gains (2015 8th grade scores minus 2011 4th grade scores). Ok so stare closely at the chart around the 262 score from the bottom to the top. Arizona, Tennessee, Oklahoma, Michigan, Rhode Island, New York, Florida and Delaware all had approximately the same 8th grade math score, but took different paths to get there. Some, like Delaware, Florida and Maryland started above the national average in their 2011 4th grade scores, but had small gains. Others like Arizona and Oklahoma, started below the national average in their 4th grade scores but grinded their way to large gains to catch up.

In 2011, Arizona 4th graders scored a 212 in 4th grade reading, Oklahoma a 215. Maryland’s 4th graders scored a 231 in 4th grade reading., New York stood at 222. Maryland students had an almost 19 point advantage over Arizona students and a 16 point advantage over Oklahoma students. Maryland spends far more than either Arizona or Oklahoma, and New York literally spends more than twice as much per pupil as either of these states. It shouldn’t happen that either Arizona or Oklahoma students would tie Maryland and/or New York by the time those 2011 4th graders became 8th graders.

Keep staring at that middle portion of the chart. Is Tennessee supposed to be neck and neck with Rhode Island? Rhode Island’s 7 point lead in the 2011 4th grade reading scores and almost $7,000 per student spending gap would say no, but the Tennessee kids didn’t get the memo and ended in a dead heat by 8th grade.

Ok so spot NY on the above chart and then look at math:

Arizona, Connecticut, Kansas and Maryland had 2011 4th grade math scores of 235, 242, 246 and 247 respectively. These had current (not total) expenditures that year of $7,782,$16,224, $9,802 and $13,946 per pupil. As an Arizonan, I’m delighted to have closed the gap with Connecticut, Kansas and Maryland. If I were a taxpayer or educator in Connecticut, Kansas and/or Maryland I would not be pleased.

Now locate New York on the math chart. I guess $19,965 per pupil just doesn’t buy what it used to in New York.

Ultimately it is good news that we have examples of states with diverse student bodies making academic progress. Remember- winter is coming to state budgets as 10,000 boomers per day reach the age of 65 and health care costs continue to rise. I hope you can get that sorted out New York but in the meantime both your students and taxpayers are getting horribly short-changed by your K-12 rent-seeking groups. The founders included a solution for you in our constitutional system: federalism. Did I mention that in addition to lower taxes, it is very pleasant here in the winter? As Ling Ving once sang “New York’s alright-if you want to freeze to death!”

Be sure to bring your golf clubs:

As far as where you’ll send your kids to school, Arizona has outstanding options in the public school system in both districts and charters. Here’s some dots to connect on the average performance of Arizona charters:

Additionally if you happen to prefer a private school for your child, Arizona’s policies support your families capacity to make that decision. Tired of having the daylights taxed out of you to pay for a public school system you don’t want to put your kids in, and then paying private school tuition on top of that? I thought you might. Head south until you reach Interstate 10 and then go west young family!

The “Giant” mystery in Maryland NAEP scores- Real Meh or the Appearance of Meh?

October 31, 2017

(Guest Post by Matthew Ladner)

When I think of Maryland, the scene of Liz Taylor riding a horse in the 1956 film classic Giant always comes to mind. Taylor’s character grows up in Maryland but subsequently moves to a desolate western ranch. The film maker used the rolling green hills of Maryland as a way to make a stark visual contrast with the desert southwest.

Likewise here in my arid state my friends on the left yearn to be Maryland. More spending and less choice = more better according to this way of thinking. Well, hmmm….what does the NAEP have to tell us about this POV?

As discussed in the previous post, Maryland had previously not complied with NAEP inclusion standards for students in special programs. They righted the ship in 2015, and it is reasonable to expect that compliance with those standards would have a substantial impact on, for instance, the scores of students with disabilities. Did compliance with inclusion standards also have a large impact on the overall scores for Maryland students? The above chart shows cohort gains from 2011 to 2015 in math and reading. The 2011 scores would have been out of compliance, whereas the 2015 scores were in compliance with NAEP inclusion standards. Did inclusion standards drive these poor results?

The drop in Maryland NAEP scores between 2013 and 2015 looks sudden and sharp. There may be no absolution to be found here for Maryland, as if it is the case that compliance with inclusion standards caused scores to drop precipitously, then the state’s reputation as having a high performing school system may have been built on exclusion of special program students. In other words, even if things are not as bad as they look in the above chart, they may shift to a different type of bad.

To test this question-have Maryland’s inclusion practices inflated their NAEP scores or did they just do poorly in 2015?- I ran cohort gains for general education students. General education students here are neither in ELL or SPED programs, and thus immunized from changes in inclusion standards over time.

For those of you squinting at your iphone, Maryland moves from dead last to merely clumped among the dead-last blob at the bottom left. Thus we conclude that changes in inclusion standards did play a role in the precipitous drop in 2015 NAEP scores, but that the state’s school system has bigger problems with which to wrestle. In other words, there is some real meh, not just the appearance of meh, especially if one were to bring spending into the conversation.

Deprived of the gains of special program students, Arizona slips slightly while Tennessee shows the largest overall gains for general education students. The 2017 NAEP data will be released in January, so let’s see what happens next.


Hawaii and Arizona made the most academic progress with students with disabilities 2011 to 2015

October 29, 2017

(Guest Post by Matthew Ladner)

Ok so here is what is going on in this chart: NAEP Math and Reading tests are timed and scaled in such a way as to allow for the calculation of cohort gains. In this case, we’ve tracked the statewide gains for students with disabilities from 4th graders in 2011 to 8th graders in 2015. Both the 2011 and 2015 measurements are a population estimate, and NAEP of course is not tracking the same students over time but rather are sampling both populations. The calculation used here is a straightforward 2015 8th grade scores for students with disabilities minus the 4th grade 2011 scores for students with disabilities, and then calculated as a percentage of improvement between 4th and 8th grade.

Students move in and out of states over time, but this sort of error should be largely random and cancel itself out in the absence of some (relatively implausible) systematic bias (like in this case higher performing students with disabilities fleeing Maryland to live in Hawaii). Given the standard errors, there isn’t much reason to fuss over exactly where you stand if you land say in the middle of the blue blob in the chart above, although one might take an interest in the states landing in the top right or bottom left.

Congrats to Hawaii and Arizona. Bad look for Maryland if taken at face value- having one of the nation’s highest spending per pupil figures but failing to teach students with disabilities much of anything about math and reading over a four years is, ah, terrible. Maryland is a state that had in earlier years flouted the NAEP’s inclusion standards for children with disabilities. It is possible that if they stopped doing so in 2015 that it may explain part of their place on this chart. If I lived in Maryland I would get to the bottom of this, but it’s time to get out of my pajamas.

For Hawaii and Arizona:

We’ll circle back and see how this goes when the new NAEP data is released in January.

Cohort NAEP Gains by Spending

October 11, 2017


(Guest Post by Matthew Ladner)

So I’ve been cooking up some new charts on NAEP statewide cohort gains for math and reading by state K-12 per pupil spending trends. Imo the cohort gains are pretty good overall measure of statewide school quality, albeit not a perfect one. Student demographics influence all scores, but if school quality is going to assert itself they should have less of an influence on 8th grade scores than 4th grade scores simply because the kids have been in school longer. Thus question addressed along the horizon in this chart is how much math did your state’s students learn between 4th grade in 2009 and 8th grade in 2013? This is plotted against the trend in per capita spending between 2007 and 2014 per NCES.

So let’s note a few things here. First once again there is a lack of a discernible relationship between spending trend and academic outcomes. You had some states that made bid increases that bombed, and others that made big cuts and lead the nation in gains (take a bow Arizona educators and policymakers!)

Maybe this was a fluke. What happens if you do it again for math cohort gains between 2011 4th graders and 2015 8th graders?

Using my Professor X mutant super-power, I am reading your thoughts. You were thinking “Okay Ladner we get it something good is going in math. What about other subjects?” Fair enough- we can only do cohort gains in math and reading, so here is the cohort reading gains:

Well would you look at that- tied for second. Cohort gains are one method for measuring gains, but we can also look at over time gains for different cohorts of students, which allows us to bring in 4th and 8th grade science. Here is what those look like for the entire period we can get readings on all six tests (new results will be released in January 2017):

Can Arizona keep it up? I certainly hope so and we will find out in January.

Texas Implemented a special ed cap, AZ implemented an ESA for special education children. Guess what happened next.

May 15, 2017

(Guest Post by Matthew Ladner)

The Raise Your Hand Texas group has released a white paper opposing an ESA program for special needs students in the Lone Star State. It is alas replete with boiler-plate nostrums etc. but if private choice is terrible for children with disabilities attending district schools, you have an awfully hard time finding evidence for it in the NAEP. We can get NAEP trends for children with disabilities on all six NAEP exams for the 2009 to 2015 period. The Arizona legislature passed a private choice tax credit for special needs children in 2009, and followed that up with the Empowerment Scholarship Accounts program in 2011.

Texas meanwhile during this period had the Texas Education Agency implementing a defacto cap on the number of special needs students in districts, without the slightest apparent protest from Texas districts, who implemented the program quite effectively. Ah, well, at least those Texas districts should have been doing a better job delivering special ed students for the children with disabilities they served, right?


Arizona authorities decided to expand options and increase freedom. As you can see, Arizona students with disabilities have demonstrated academic progress much better than the nation as a whole, which has either been treading water or actually declining. This looks pretty bad until you examine the scores for Texas students with disabilities, which are not only consistently worse, but which failed to show improvement in any of the six subjects covered by NAEP.

These trends obviously have factors other than choice which impact them, but if the theory is that ESAs are terrible for children with disabilities in public schools, we can reject the hypothesis. Texas has a special education disaster on its hands, while Arizona is making progress far and away above the national average. No student group has more to gain from choice than children with disabilities- including those who choose to remain in districts.


%d bloggers like this: