A Harsh Apology to the Arizona Class of 2006 and the True Meaning of Accountability

September 2, 2015

(Guest Post by Matthew Ladner)

I had the chance to make a presentation on testing and private choice programs recently, and received a request to share a few slides from that presentation here on JPGB. So the first is from Arizona, circa 2006. I chose 2006 because we have a study that follows the entire public school Class of 2006 through the higher education system.

Slide6

So for those of you squinting at your Ipad- the columns read: Kids attending AZ public schools taking state math and reading tests (100%!), AZ Class of 2006 who read proficiently as 8th graders on the 2002 NAEP 8th grade reading exam (errr, 23%), Percent of Class of 2006 graduating class who went on to earn a Bachelor degree by the end of 2012 (errr 18.6%) and finally the percent of Arizona public schools who earned an “Underperforming” or “Failing” label in 2006 (*cough* 6.5%).

So who was held “accountable” in this slide. Not the Governor she was reelected by a wide margin in 2006. Oh what about the Superintendent of Public Instruction? Nope- he was reelected as well. Did Arizona have a mass culling of ineffective school superintendents in 2006? What about teachers? Nope and nope- it was business as usual.

Let’s compare the accountability for the staff at the 6.5% of schools who received a nice-so-nice label compared to that of the students. Now that it is 2015, what is the chance that any of the adults in those 6.5% of schools carry around a nine-year old label around with them as a burden, even if they remain in education and are still remain employed at the same school? Right- now what about the 81.4 percent of the Class of 2006 who either never attended college or who were among the waves of people who dropped out of college in debt with little to show for it?

The latter scenario constitutes a much harsher form of accountability than Arizona’s former “whip truly terrible schools with a wet noodle accountability.”  Sorry Class of 2006- I know that the state of Arizona gave you the equivalent of a Good Housekeeping Seal of Approval based on your results on a laughably simple AIMS test, then they kicked the can down the road on an high-school exit exam, then they labelled your school “performing” when it was actually anything but academically performing. That “performing” actually meant that it was performing its duty to employ adults and might occasionally facilitate learning if a student had a deep motivation to do it on their own. You were very foolish not to clue into this and now you will have to pay for it.

We the taxpayers and adult policymakers of Arizona feel some regret about all of this Class of 2006 for all these things, but ultimately you should have ignored the fact that the adults in your life were telling you everything was fine, and you should have studied harder especially when you were learning to read in K-3, since 77 percent of you failed to reach proficiency as 8th graders. It was really absurd so many of you thought you could do college level work, but the universities disabused you of that notion quickly didn’t they? Don’t worry the Class of 2007 temporarily filled your spots after you dropped out.

In the next life you should not be so trusting of adults and study harder. Perhaps you will take this lesson to heart as parents.  Go suffer the consequences of your actions and just think of how much worse this might have turned out if Arizona did not hold school systems accountable. The greatest trick the public school lobby ever pulled was convincing the world that the publication of scores on minimal skills math and reading tests constituted “accountability.” And like that **poof**

…meaningful accountability was gone, unless by “accountability” we mean watching helplessly as students suffer the long-term consequences for failing to acquire competitive skills.

Now as a post-script, things have improved somewhat since 2006 in Arizona. Instead of handing out “performing” labels, the state uses letter grades. Grades of C or D are closer to truth in advertising than “Performing.” The wretched AIMS test has finally received the mercy killing it so richly deserved.  Sorry I-hate-CC-with-a-purple-passion tribe, the new test aligns much closer to Arizona’s performance on NAEP so it represents an undeniable upgrade over AIMS, at least so far. Yes, it could have been accomplished by other means etc. etc. but the sad reality is that we sat around indifferently for years as the fraud described above played out.  My humble suggestion at this point would be to offer constructive and rigorous counter proposals to AZ Merit because I hope that if you’ve reached this part of the post you’ll at least acknowledge the true horror of the AIMS regime. I mean it was cooked up by a group of Arizona teachers in 1994, which makes it near sacred and all, but that can’t make up for the system being horribly mismanaged by the AZ Department of Ed and State Board of Education after that. It devolved into a cruel joke on children.

Yes Jay I get it they probably will do the same with the new test sooner or later.  How long do you expect this peace to last?

…as long as it can.

In the end, this too shall pass, so the most enduring accountability going on in Arizona today involves parental choice. Parental choice in fact represents the ultimate form of accountability that no system of aggregate test scores and school labels can ever replace.  Even at its best such accountability is an aggregate phenomenon, whereas parental choice allows parents to hold schools responsible at the individual level by voting with their feet.

Since 2006, AZ charter schools have reached a more meaningful scale. Arizona now has the highest percentage of students attending charter schools (almost 18%) of any state. Parents have used their contacts and Greatschools to figure out that even their allegedly swell schools leave much to be desired and have commenced to pounding on the doors of high quality charter operators, developing huge waiting lists.

The highest rated general enrollment school in the Arizona Board of Regents analysis of higher education outcomes Tempe Prep- was the forerunner of the Great Hearts system of schools that now has 22 campuses and mile long waiting lists. These schools did not appear in the 2006 analysis because they either had not opened yet or did not have a senior class by 2006. Stay tuned-the Board of Regents will soon have an updated analysis. Our private choice programs in the aggregate are mostly helping private schools to remain viable against the rise of charters. We need to do more on that front, and we need to help high quality charters replicate.

Meanwhile, for the first time ever, Arizona districts find their enrollment in decline in absolute terms. Before the great recession charters and choice were simply taking the edge off of district enrollment growth. In the last couple of years there has been district enrollment declines. Enrollment growth will eventually reverse this, but for now the charters are  basically absorbing all of it and more. Oh by the way, while Arizona’s NAEP scores are not high, they were higher than they have ever been in 2013. Sweet are the uses of adversity…

Arizona’s growing choice sector has created a constituency and will not be dispatched as easily as the well-meaning but ultimately failed efforts of the AIMS regime. Keep hope alive!

 


I Love It When a Plan Comes Together

July 15, 2008

(Guest Post by Matthew Ladner)

Recently I wrote about Arizona’s system of testing having jumped the shark. The cut scores for the AIMS had been dropped severely, and the state’s version of the Terra Nova exam spun a tale of Arizona students scoring above the national average in every grade and subject tested. Arizonans were asked to believe this, despite having a very difficult to educate K-12 population and NAEP scores below the national average in every test given since the early 1990s.

I am pleased to say that the Arizona legislature, acting in a bipartisan fashion, took corrective action. Essentially they limited the current testing contracts to a single year, and appointed a commission to design a new testing system, specifying the use of a college readiness exam as a graduation exam along the lines of the Michigan model with the ACT.

The challenge now will be for the commission to create a challenging, consistent system of testing providing proper transparency for parents, teachers, administrators and policymakers. The first step was to admit that there was a problem, which the Arizona legislature has now done emphatically.


Standardized Testing Jumps the Shark in AZ

June 4, 2008

(Guest Post by Matthew Ladner)

In an attempt to keep viewers tuning-in after many years on the air, the sitcom Happy Days produced an episode where Fonzie jumped a Great White shark on water-skis. This episode brought the phrase “Jumping the Shark” into the pop-culture lexicon. Jumping the Shark denotes a tipping point in which something becomes absurd and suffers a noticeable decline. Arizona once was a leader in the standards and accountability movement, but those days are long gone. Days ago, Arizona lawmakers dispensed with AIMS as a graduation requirement, making the sad decline of AIMS into farce complete.

The credibility of Arizona’s K-12 testing has suffered the death of a thousand cuts. In 2004, Arizona schools faced a problem in that No Child Left Behind requires schools to be judged by ethnic subgroups, and Hispanic scores were all but certain to force many schools to be ranked failing under federal guidelines.

Instead, the state simply made AIMS much, much easier to pass. Presto-chango, Arizona Hispanic students (and others) were transformed from having been projected to fail the federal standards in almost all subjects at all grade levels in 2005 to passing almost all of them. A study by Peterson and Hess noted that Arizona’s dummy-down was the largest in the country.

Where were you when we needed you Jaws?

Hop on over, the water’s fine!

Around that same time, the Arizona Department of Education recommended replacing the Stanford 9 exam with an Arizona version of the Terra Nova to imbed into AIMS. Happily, the new “Terra Zona” exam found that Arizona students are above the national average in every grade and in every subject tested.

One small problem: the results aren’t the least bit credible. The Arizona Department of Education recently mailed out the latest state report card, and the evidence of the farcical nature of this home-grown exam can be found in ADE’s own booklet.

On the one hand, the ADE touts the above average Terra Nova scores, but in the same booklet, it presents an analysis from the RAND Corporation showing that if you control for student demographics, Arizona’s scores on the Nation’s Report Card are average instead of rock bottom. The Nation’s Report Card- or NAEP- represents the nation’s most highly respected source of K-12 testing data.

The RAND report is entirely credible. Arizona has a far more difficult to educate student body than the national average- with a much higher percentage of low-income students, English language learners and minority students than the national average.

Controlling for demographic factors is a huge step to take. For instance, Arizona has a percentage of children eligible for a free or reduced price lunch more than twice as large as the national average. Our ratio of children who are English Language Learners is almost four times the national average.

If you pretend that Arizona has an ELL population one fourth its actual size, and about half the number of low-income children that we actually have, and some similar heroic assumptions, Arizona’s adjusted scores near the Minnesota middle instead of close to the bottom.

Arizona’s Terra Nova, however, does not control for demographics at all but somehow finds our students above the national average in every single subject without any adjustment whatsoever. If you are willing to buy that, I’ve got a bridge I’d like to sell you in Brooklyn.

Finally, AIMS has suffered what ought to be its final indignity. The legislature passed “AIMS Augmentation” in order to allow 6,000 high school seniors to graduate despite an inability to pass what at most amounts to a test of basic skills.

If you can’t pass a 10th grade level test, the original thinking went, you don’t deserve to graduate. A diploma should mean something. After delaying the graduation requirement several times, the augmentation bill has effectively killed it.

State policymakers should rethink our entire system of testing. Research shows that children who fail to learn to read in the early grades later drop out in huge numbers. Using AIMS as a graduation requirement addresses the problem at the back end. Arizona should look to Florida, which uses testing to require students to repeat grades if they don’t learn to read in the early years. Florida’s 4th grade reading scores used to scrape the bottom with Arizona, but now they greatly exceed us. Florida’s system set kids up to succeed, rather than to fail.

Parents, teachers, administrators and policymakers all require a credible and transparent system of student data. The AIMS/Terra Nova exam is not delivering. ABC eventually cancelled Happy Days and replaced it with another program. Arizona policymakers should do the same with AIMS.