Why Arizona Charter Leaders Should Feel Confident About the 2017 NAEP

March 2, 2017

azmeritnaep

(Guest Post by Matthew Ladner)

So the 2017 NAEP is underway as we speak. As the NAEP released all six 2015 exams a cornucopia of good news spilled out for Arizona as a whole and (especially) for Arizona charters. Statewide Arizona has been leading the nation in 4th to 8th grade cohort gains since 2009. That’s a pretty good measure of overall school quality for those grades, as the demographic profile of the cohort isn’t likely to change much for a cohort between (for instance) the time when they were 4th graders in 2011 to when they were 8th graders in 2015. Kids will come and go of course, but absent a DC level of gentrification the academic ability of those coming and going should not skew heavily in a particular direction. You also have a measured amount of sampling error in both the 4th and 8th grade measures, but these are quite modest.

Arizona’s charter schools rocked all 2015 NAEP exams in the fashion of a New England state. This is quite impressive given the very modest level of per-pupil spending in Arizona charters, the majority minority student population etc. Sampling error is a bigger issue with regards to charter schools, but as you can see in the figure above, the state’s AZMerit exam tells us a story very similar to NAEP, and does not involve sampling. AZMerit in short provides backup to the findings in NAEP.

The figure above should lend confidence to Arizona charter leaders that they are likely to rock the NAEP again in 2017. The figure shows 4th and 8th grade NAEP scores from 2015 (proficiency rates) and then the proficiency rates for districts and charters from 2016 on AZMerit. These tests are not scaled exactly the same so you would not expect them to match up exactly, but the similarity in the pattern provides confidence that 2015 was not a fluke. Some of the differences between Arizona charters and districts can be attributed to differences in student demographics, although not nearly so much as sometimes are imagined, but New Hampshire doesn’t have any excuses for losing out to Arizona charter schools.

On the final set of columns on the right, for perspective, Arizona charters scored just a smidge below the highest performing state (MA) while Arizona districts would be right around the national average. Drawing random samples of students leaves room for goofy variation in subgroup scores, but there isn’t any reason to believe such goofiness would skew the same way in six different samples unless one wants to believe the NAEP is stacking the deck for charter schools in Arizona for some nefarious reason. While they were at it, the same conspirators would have to have infiltrated the AZMerit as well.

Absent goofiness and based on what we see in the AZMerit, I’m expecting Arizona charter students to CeleNAEP Good Times again in 2017. The statewide trend will be of even greater importance. Let’s see what happens next.

 


AZ Charter Scores are Real and They’re Spectacular!

December 8, 2015

(Guest Post by Matthew Ladner)

AZ charter vs. district

Let’s the speculation shift from “are those AZ charter NAEP scores real?” to “did the NAEP underestimate the scores?” As the lovely Teri Hatcher once said:


A Harsh Apology to the Arizona Class of 2006 and the True Meaning of Accountability

September 2, 2015

(Guest Post by Matthew Ladner)

I had the chance to make a presentation on testing and private choice programs recently, and received a request to share a few slides from that presentation here on JPGB. So the first is from Arizona, circa 2006. I chose 2006 because we have a study that follows the entire public school Class of 2006 through the higher education system.

Slide6

So for those of you squinting at your Ipad- the columns read: Kids attending AZ public schools taking state math and reading tests (100%!), AZ Class of 2006 who read proficiently as 8th graders on the 2002 NAEP 8th grade reading exam (errr, 23%), Percent of Class of 2006 graduating class who went on to earn a Bachelor degree by the end of 2012 (errr 18.6%) and finally the percent of Arizona public schools who earned an “Underperforming” or “Failing” label in 2006 (*cough* 6.5%).

So who was held “accountable” in this slide. Not the Governor she was reelected by a wide margin in 2006. Oh what about the Superintendent of Public Instruction? Nope- he was reelected as well. Did Arizona have a mass culling of ineffective school superintendents in 2006? What about teachers? Nope and nope- it was business as usual.

Let’s compare the accountability for the staff at the 6.5% of schools who received a nice-so-nice label compared to that of the students. Now that it is 2015, what is the chance that any of the adults in those 6.5% of schools carry around a nine-year old label around with them as a burden, even if they remain in education and are still remain employed at the same school? Right- now what about the 81.4 percent of the Class of 2006 who either never attended college or who were among the waves of people who dropped out of college in debt with little to show for it?

The latter scenario constitutes a much harsher form of accountability than Arizona’s former “whip truly terrible schools with a wet noodle accountability.”  Sorry Class of 2006- I know that the state of Arizona gave you the equivalent of a Good Housekeeping Seal of Approval based on your results on a laughably simple AIMS test, then they kicked the can down the road on an high-school exit exam, then they labelled your school “performing” when it was actually anything but academically performing. That “performing” actually meant that it was performing its duty to employ adults and might occasionally facilitate learning if a student had a deep motivation to do it on their own. You were very foolish not to clue into this and now you will have to pay for it.

We the taxpayers and adult policymakers of Arizona feel some regret about all of this Class of 2006 for all these things, but ultimately you should have ignored the fact that the adults in your life were telling you everything was fine, and you should have studied harder especially when you were learning to read in K-3, since 77 percent of you failed to reach proficiency as 8th graders. It was really absurd so many of you thought you could do college level work, but the universities disabused you of that notion quickly didn’t they? Don’t worry the Class of 2007 temporarily filled your spots after you dropped out.

In the next life you should not be so trusting of adults and study harder. Perhaps you will take this lesson to heart as parents.  Go suffer the consequences of your actions and just think of how much worse this might have turned out if Arizona did not hold school systems accountable. The greatest trick the public school lobby ever pulled was convincing the world that the publication of scores on minimal skills math and reading tests constituted “accountability.” And like that **poof**

…meaningful accountability was gone, unless by “accountability” we mean watching helplessly as students suffer the long-term consequences for failing to acquire competitive skills.

Now as a post-script, things have improved somewhat since 2006 in Arizona. Instead of handing out “performing” labels, the state uses letter grades. Grades of C or D are closer to truth in advertising than “Performing.” The wretched AIMS test has finally received the mercy killing it so richly deserved.  Sorry I-hate-CC-with-a-purple-passion tribe, the new test aligns much closer to Arizona’s performance on NAEP so it represents an undeniable upgrade over AIMS, at least so far. Yes, it could have been accomplished by other means etc. etc. but the sad reality is that we sat around indifferently for years as the fraud described above played out.  My humble suggestion at this point would be to offer constructive and rigorous counter proposals to AZ Merit because I hope that if you’ve reached this part of the post you’ll at least acknowledge the true horror of the AIMS regime. I mean it was cooked up by a group of Arizona teachers in 1994, which makes it near sacred and all, but that can’t make up for the system being horribly mismanaged by the AZ Department of Ed and State Board of Education after that. It devolved into a cruel joke on children.

Yes Jay I get it they probably will do the same with the new test sooner or later.  How long do you expect this peace to last?

…as long as it can.

In the end, this too shall pass, so the most enduring accountability going on in Arizona today involves parental choice. Parental choice in fact represents the ultimate form of accountability that no system of aggregate test scores and school labels can ever replace.  Even at its best such accountability is an aggregate phenomenon, whereas parental choice allows parents to hold schools responsible at the individual level by voting with their feet.

Since 2006, AZ charter schools have reached a more meaningful scale. Arizona now has the highest percentage of students attending charter schools (almost 18%) of any state. Parents have used their contacts and Greatschools to figure out that even their allegedly swell schools leave much to be desired and have commenced to pounding on the doors of high quality charter operators, developing huge waiting lists.

The highest rated general enrollment school in the Arizona Board of Regents analysis of higher education outcomes Tempe Prep- was the forerunner of the Great Hearts system of schools that now has 22 campuses and mile long waiting lists. These schools did not appear in the 2006 analysis because they either had not opened yet or did not have a senior class by 2006. Stay tuned-the Board of Regents will soon have an updated analysis. Our private choice programs in the aggregate are mostly helping private schools to remain viable against the rise of charters. We need to do more on that front, and we need to help high quality charters replicate.

Meanwhile, for the first time ever, Arizona districts find their enrollment in decline in absolute terms. Before the great recession charters and choice were simply taking the edge off of district enrollment growth. In the last couple of years there has been district enrollment declines. Enrollment growth will eventually reverse this, but for now the charters are  basically absorbing all of it and more. Oh by the way, while Arizona’s NAEP scores are not high, they were higher than they have ever been in 2013. Sweet are the uses of adversity…

Arizona’s growing choice sector has created a constituency and will not be dispatched as easily as the well-meaning but ultimately failed efforts of the AIMS regime. Keep hope alive!