Somebody Stop Me-2017 NAEP predictions

March 2, 2018

Mamma Mia- Here we go again!

(Guest Post by Matthew Ladner)

NAEP has announced that it will release new math and reading results on April 10. I am going to do something truly foolish and dare to make a few predictions.

Making predictions is foolish because the reality is that our understanding of NAEP trends is imperfect at best. In any case, I have a theory about what has been driving NAEP gains here in the Cactus Patch, and I’m willing to make a few calls in advance on the 2017 NAEP based upon that theory.

Prediction 1: Arizona continues to improve.

Arizona’s improvement process is multi-faceted but the elephant in the room in my book is a hyper-active open enrollment market, the nation’s largest and most geographically inclusive charter sector and private choice programs. Arizona lead the nation in academic gains even during a period in which it had the largest cuts in spending due to the Great Recession. If we can do this during a period of funding cuts, we ought to manage it during a period of funding recovery given the broad consistency of policy. Note however that the state’s A-F letter grades has been turned off during the entirety of the 2015 to 2017 period, the demographics of students have continued to move further into majority-minority status, etc. Put me down for Arizona improving anyway based on the AZMerit improving in both 2016 and then again in 2017.

Prediction 2: North Carolina climbs.

Recent NAEP trends have not been great in NC. Notice in the Reardon tables that North Carolina starts with a (good) greenish tint on the top map-showing good early scores-but turns purple on the second map (bad) on cohort NAEP gains during the 2011 to 2015 period.

I suspect North Carolina will do better in 2017 based upon the information in this news report- basically that while statewide enrollment is growing, school district enrollment has been relatively flat due to the growth of choice options. North Carolina lifted a statewide cap on charter schools in 2011, created voucher programs for low-income and special needs children. I’m not sure whether they’ve done enough to start the open-enrollment virtuous cycle, but I think they may have done enough to shake things up a bit.

Third prediction: Indiana improves.

Give Indy a good stare in the Reardon chart, and it looks a bit like the green/purple undesirable combo between the two maps. It looks to me though like the districts are getting into the choice act (until 2007 you had to pay tuition in order to attend an out of zone school) and so I’m willing to buy some Hoosier stock.

Anyone else willing to dare a prediction should spell out their theory/evidence in the comment section.



Why Arizona Charter Leaders Should Feel Confident About the 2017 NAEP

March 2, 2017


(Guest Post by Matthew Ladner)

So the 2017 NAEP is underway as we speak. As the NAEP released all six 2015 exams a cornucopia of good news spilled out for Arizona as a whole and (especially) for Arizona charters. Statewide Arizona has been leading the nation in 4th to 8th grade cohort gains since 2009. That’s a pretty good measure of overall school quality for those grades, as the demographic profile of the cohort isn’t likely to change much for a cohort between (for instance) the time when they were 4th graders in 2011 to when they were 8th graders in 2015. Kids will come and go of course, but absent a DC level of gentrification the academic ability of those coming and going should not skew heavily in a particular direction. You also have a measured amount of sampling error in both the 4th and 8th grade measures, but these are quite modest.

Arizona’s charter schools rocked all 2015 NAEP exams in the fashion of a New England state. This is quite impressive given the very modest level of per-pupil spending in Arizona charters, the majority minority student population etc. Sampling error is a bigger issue with regards to charter schools, but as you can see in the figure above, the state’s AZMerit exam tells us a story very similar to NAEP, and does not involve sampling. AZMerit in short provides backup to the findings in NAEP.

The figure above should lend confidence to Arizona charter leaders that they are likely to rock the NAEP again in 2017. The figure shows 4th and 8th grade NAEP scores from 2015 (proficiency rates) and then the proficiency rates for districts and charters from 2016 on AZMerit. These tests are not scaled exactly the same so you would not expect them to match up exactly, but the similarity in the pattern provides confidence that 2015 was not a fluke. Some of the differences between Arizona charters and districts can be attributed to differences in student demographics, although not nearly so much as sometimes are imagined, but New Hampshire doesn’t have any excuses for losing out to Arizona charter schools.

On the final set of columns on the right, for perspective, Arizona charters scored just a smidge below the highest performing state (MA) while Arizona districts would be right around the national average. Drawing random samples of students leaves room for goofy variation in subgroup scores, but there isn’t any reason to believe such goofiness would skew the same way in six different samples unless one wants to believe the NAEP is stacking the deck for charter schools in Arizona for some nefarious reason. While they were at it, the same conspirators would have to have infiltrated the AZMerit as well.

Absent goofiness and based on what we see in the AZMerit, I’m expecting Arizona charter students to CeleNAEP Good Times again in 2017. The statewide trend will be of even greater importance. Let’s see what happens next.