A Harsh Apology to the Arizona Class of 2006 and the True Meaning of Accountability

September 2, 2015

(Guest Post by Matthew Ladner)

I had the chance to make a presentation on testing and private choice programs recently, and received a request to share a few slides from that presentation here on JPGB. So the first is from Arizona, circa 2006. I chose 2006 because we have a study that follows the entire public school Class of 2006 through the higher education system.


So for those of you squinting at your Ipad- the columns read: Kids attending AZ public schools taking state math and reading tests (100%!), AZ Class of 2006 who read proficiently as 8th graders on the 2002 NAEP 8th grade reading exam (errr, 23%), Percent of Class of 2006 graduating class who went on to earn a Bachelor degree by the end of 2012 (errr 18.6%) and finally the percent of Arizona public schools who earned an “Underperforming” or “Failing” label in 2006 (*cough* 6.5%).

So who was held “accountable” in this slide. Not the Governor she was reelected by a wide margin in 2006. Oh what about the Superintendent of Public Instruction? Nope- he was reelected as well. Did Arizona have a mass culling of ineffective school superintendents in 2006? What about teachers? Nope and nope- it was business as usual.

Let’s compare the accountability for the staff at the 6.5% of schools who received a nice-so-nice label compared to that of the students. Now that it is 2015, what is the chance that any of the adults in those 6.5% of schools carry around a nine-year old label around with them as a burden, even if they remain in education and are still remain employed at the same school? Right- now what about the 81.4 percent of the Class of 2006 who either never attended college or who were among the waves of people who dropped out of college in debt with little to show for it?

The latter scenario constitutes a much harsher form of accountability than Arizona’s former “whip truly terrible schools with a wet noodle accountability.”  Sorry Class of 2006- I know that the state of Arizona gave you the equivalent of a Good Housekeeping Seal of Approval based on your results on a laughably simple AIMS test, then they kicked the can down the road on an high-school exit exam, then they labelled your school “performing” when it was actually anything but academically performing. That “performing” actually meant that it was performing its duty to employ adults and might occasionally facilitate learning if a student had a deep motivation to do it on their own. You were very foolish not to clue into this and now you will have to pay for it.

We the taxpayers and adult policymakers of Arizona feel some regret about all of this Class of 2006 for all these things, but ultimately you should have ignored the fact that the adults in your life were telling you everything was fine, and you should have studied harder especially when you were learning to read in K-3, since 77 percent of you failed to reach proficiency as 8th graders. It was really absurd so many of you thought you could do college level work, but the universities disabused you of that notion quickly didn’t they? Don’t worry the Class of 2007 temporarily filled your spots after you dropped out.

In the next life you should not be so trusting of adults and study harder. Perhaps you will take this lesson to heart as parents.  Go suffer the consequences of your actions and just think of how much worse this might have turned out if Arizona did not hold school systems accountable. The greatest trick the public school lobby ever pulled was convincing the world that the publication of scores on minimal skills math and reading tests constituted “accountability.” And like that **poof**

…meaningful accountability was gone, unless by “accountability” we mean watching helplessly as students suffer the long-term consequences for failing to acquire competitive skills.

Now as a post-script, things have improved somewhat since 2006 in Arizona. Instead of handing out “performing” labels, the state uses letter grades. Grades of C or D are closer to truth in advertising than “Performing.” The wretched AIMS test has finally received the mercy killing it so richly deserved.  Sorry I-hate-CC-with-a-purple-passion tribe, the new test aligns much closer to Arizona’s performance on NAEP so it represents an undeniable upgrade over AIMS, at least so far. Yes, it could have been accomplished by other means etc. etc. but the sad reality is that we sat around indifferently for years as the fraud described above played out.  My humble suggestion at this point would be to offer constructive and rigorous counter proposals to AZ Merit because I hope that if you’ve reached this part of the post you’ll at least acknowledge the true horror of the AIMS regime. I mean it was cooked up by a group of Arizona teachers in 1994, which makes it near sacred and all, but that can’t make up for the system being horribly mismanaged by the AZ Department of Ed and State Board of Education after that. It devolved into a cruel joke on children.

Yes Jay I get it they probably will do the same with the new test sooner or later.  How long do you expect this peace to last?

…as long as it can.

In the end, this too shall pass, so the most enduring accountability going on in Arizona today involves parental choice. Parental choice in fact represents the ultimate form of accountability that no system of aggregate test scores and school labels can ever replace.  Even at its best such accountability is an aggregate phenomenon, whereas parental choice allows parents to hold schools responsible at the individual level by voting with their feet.

Since 2006, AZ charter schools have reached a more meaningful scale. Arizona now has the highest percentage of students attending charter schools (almost 18%) of any state. Parents have used their contacts and Greatschools to figure out that even their allegedly swell schools leave much to be desired and have commenced to pounding on the doors of high quality charter operators, developing huge waiting lists.

The highest rated general enrollment school in the Arizona Board of Regents analysis of higher education outcomes Tempe Prep- was the forerunner of the Great Hearts system of schools that now has 22 campuses and mile long waiting lists. These schools did not appear in the 2006 analysis because they either had not opened yet or did not have a senior class by 2006. Stay tuned-the Board of Regents will soon have an updated analysis. Our private choice programs in the aggregate are mostly helping private schools to remain viable against the rise of charters. We need to do more on that front, and we need to help high quality charters replicate.

Meanwhile, for the first time ever, Arizona districts find their enrollment in decline in absolute terms. Before the great recession charters and choice were simply taking the edge off of district enrollment growth. In the last couple of years there has been district enrollment declines. Enrollment growth will eventually reverse this, but for now the charters are  basically absorbing all of it and more. Oh by the way, while Arizona’s NAEP scores are not high, they were higher than they have ever been in 2013. Sweet are the uses of adversity…

Arizona’s growing choice sector has created a constituency and will not be dispatched as easily as the well-meaning but ultimately failed efforts of the AIMS regime. Keep hope alive!


Senators to Emperor Weingarten: What is thy bidding my master?

July 14, 2015

“If we confuse the Senate with populist rhetoric, they could become a powerful ally against state reform efforts…”

(Guest Post by Matthew Ladner)

Not content to let the House have all the fun in gutting state testing data and creating perverse incentives for schools to sweep their low-performing students under the rug, the Senate is getting ready to join in as well. The civil rights community is up in arms-they should be, and we should all be with them.

If you are nursing the hope that school officials would not do such a thing, let me direct you to the following table on the % of schools that actually wound up being including special education students as a part of their accountability subset in 2009-10:

So if you live in either Connecticut, Maine or Utah, then you can hang on to the hope that every child will be counted. If you live in a state education the other 98% of American students, it is time to wake up to the fact that school officials have a long and predictable history of following the path where perverse incentives lead them, and don’t tend to let little things like the interests of children bother them overly much. Given the opportunity to make use of a “parental” opt-out, it is blindingly obvious that school officials will take full advantage of the provision to make themselves look good, just as they have used every available loophole to bury special education scores.

Some in the beltway likes to think that Congress is some sort of gathering of Olympians best positioned to guide the nation towards technocratic K-12 improvement. The House has already provided (additional) recent evidence to demonstrate this to be incredibly misguided, and the Senate seems poised to follow suit.  Anti-common core hysteria “The Devil Made Me Do It!” will not do for an excuse when one is contemplating wrecking state testing systems and creating a Freddie/Fannie level perverse incentive all in one fell swoop.

By the next time this law comes up for reauth some 13+ or so years hence, CC will likely be a distant memory. If you’ve been paying attention, states have been adopting their own tests left and right and they have control over their own cut scores. Oklahoma withdrew from the standards completely, and approximately nothing happened to them. Many states have begun a process to review and revise standards. The best case scenario is that states will choose to use something better than their old My Little Pony Book of Connect the Dots for their new tests, but it will be up to them in any case.

We would likely find a federal opt out of all criterion based tests not so easily dispatched. It would prove far more consequential if Hanushek and Loveless are to be believed. Once put into law, the unions will fiercely defend it, given that it completely thwarts the ability to consider test scores in tenure and retention decisions on the basis of criterion based tests. I would expect it to stay in place until the next reauth. States desiring to have campus level comparable data would have to create new systems to carry it out with non-criterion tests in an era of testing fatigue. Thanks DC!

None of this is likely to happen of course, given the high probability of a presidential veto. It grows ever more obvious however that the unions have outmanoeuvred reformers.

UPDATE: The Senate voted down the opt-out amendment most similar to the House Amendment 32-64. Faith in humanity (temporarily?) restored.

K-12 Reformers Need to stare in the Mirror after 251-178

July 9, 2015

(Guest Post by Matthew Ladner)

Perhaps it can be delayed until after the reauth drama has reached a conclusion, and by no means to I wish to exonerate members of the House, but there is a very real need for K-12 reformers to look at themselves in the mirror and ask “how did we lose a bipartisan super-majority that favored academic transparency in the United States House?”

It’s been clear to me that the transparency consensus has been collapsing since at least 2013.  Even now, many in Washington seemed not to have noticed, and continue to carry supporting poorly considered technocratic tweaks to NCLB as if the consensus still exists.

251-178 folks-it’s gone.  How it was lost is certainly complex and I am not in a position to render definitive judgement. I’m calling it though that the canary just died in the coal mine.

To me there has never been a stronger case made for reducing the federal footprint in K-12 than yesterday’s vote. Yesterday the United State House of Representatives voted in overwhelming fashion to mandate the gathering and collection of student testing data that would be of absolutely no value whatsoever- would compromise the ability of parents to compare schools in a reliable fashion, would never withstand challenge in a legal proceeding like Vegara etc. Both past experience and simple logic would lead one to the conclusion that creating a federal parental opt-out for state testing systems will create a powerful incentive for school officials to nudge low-performing students (read: Black, brown, children with disabilities) out of standardized testing to improve their scores. They managed to do this in an attempt to reauthorize an important piece of legislation 8 years behind schedule. I don’t know about you but if this is the best we can get out of our alleged federal Olympians color me more ready than ever to take my chances with state legislatures.

But I digress. If reformers want there to be a consensus on transparency, we apparently need to rethink our efforts. What we are doing now obviously is not working.

The U.S. House Votes to cram Parental Opt Amendment down the throats of state testing systems

July 8, 2015

What is thy bidding, Master Weingarten?

(Guest Post by Matthew Ladner)

The House adopted the Salmon amendment 251-178. This means that of the 435 members of the United States House of Representatives, only 178 of them thought it would be a bad idea to completely wreck campus level comparability AND to create a federal parental opt-out for STATE testing systems.

So here in my (and Rep. Salmon’s) state of residence, Arizona, our state accountability system predates NCLB. If NCLB got rid of the federal testing mandate, we still have state statutes for state testing. So…the 251 affirmative votes basically thought it was a good idea to have the federal government mandate a parental opt-out wrecking state accountability systems past the point where private organizations like Greatschools can salvage them the way they have in Arizona.

I’ve either gone completely nuts, or a large majority of the House has failed to think about this carefully or perhaps at all. I’ll let you draw your own conclusions.



Tell me, friend, when did a bipartisan super-majority abandon reason for madness?

July 7, 2015

(Guest Post by Matthew Ladner)

So welcome back from your Independence Day holiday break, glad you are still ready to get your wonk on, so let’s talk a little academic transparency. The forces of light and darkness are on the march in this Middle Earth.

A few months ago I had the chance to meet with Bill Jackson from Greatschools here in Phoenix. At that meeting Bill told me something I’ve been trying to wrap my head around ever since- that more than half of parents visited Greatschools last year.

My instant reaction was to pick my jaw off the table and then mumble incoherently about probably far less than that percentage of parents had visited the Arizona Department of Education website ever, and of those that did only some unknown fraction did so to learn information about particular schools.

So then I decided to compare the Great Schools rating system with that of the state of Arizona on schools in my neighborhood. I started with Arcadia High. Arcadia High is a large high school south of Camelback Mountain. You may have heard of a prominent former student by the name of Spielberg- local legend has it that the architecture of the school inspired the space ship design in Close Encounters. The NCES put their FRL % at about half the statewide average at 25% back in 2010.

Great Schools gives Arcadia High a 7 out of 10, while easy grading Arizona gives it an “A” grade. So Greatschools essentially gave the school a C-minus, while the state of Arizona gave it an A. Which one is more accurate? Well one can never be certain (the same school can work very differently for different kids) but the Arizona Board of Regents tracked the academic progress of the entire public school Class of 2006 and found that 39% of the Arcadia High cohort finished a four-year degree within a six-year window.

Hmm, if 39% = “A” then what does a “B” look like? Well Shadow Mountain High School, near where I used to live a few years ago, got a “B” from the state, but saw 29.7% of their Class of 2006 get a BA in six years. The more discerning Greatschools ranking system gave the school a 6 out of 10 rating, which one can reasonably translate to “D-minus” and which seems to fit much more comfortably with the school’s higher education outcomes.

I could go on beating this horse into horseburger, but it misses the point. The (main) point is not that Arizona’s grading system needs a lot of work- although it clearly does- but rather that Greatschools provides an invaluable public service by providing a more rigorous ranking system. In addition, Greatschools also provides a source for student, parent and teacher reviews of schools. Mike McShane cited research showing that parents rate this reviews highly in their decision-making (see page three). This makes it all the better that Greatschools gets more traffic than the state website- it has more of the information they desire. It’s worth noting that the state of Arizona recently took a couple of year time out on school grades as they transition to a new state test. There is precedent for this, as the Arizona Department of Education just stopped publishing school report cards for several years within the last decade. Fortunately Bill Jackson and company have no need for timeouts, and parents are frequenting his website anyway.

Greatschools is performing a very valuable service in a fashion clearly superior to that provided by my state government-may God and His Angels bless Bill Jackson and all of his descendants. Greatschools gives highly valuable information to parents for free in helping them make difficult education decisions here in Arizona, and elsewhere. Greatschools however must make use of public data, and therein lies a growing threat to this invaluable parental choice resource.

It is with great sadness that news has reached my ears that an Arizona Congressman plans to offer an amendment to the ESEA reauthorization bill that would fundamentally undermine Greatschool’s ability to provide comparable data between schools for consideration by parents. The amendment would create a parental opt-out of testing. This would create what seems like a fatal blow to data comparability between schools, and a rather powerful perverse incentive for schools to nudge their weaker students to “opt-out.” Parents and researchers would have little to no way of knowing the rates of strategic “opting out” and thus compatibility between schools would be utterly compromised.

In the midst of anti-testing hysteria being fanned by the unions (the NEA for instance has model opt-out legislation) and others I fear this amendment will in fact pass. I had a front row seat to watch Texas dispatch a 30+ year bipartisan consensus in favor of testing and transparency in a distressingly casual fashion. If it can happen in Austin, it can happen anywhere.

Yes you can argue the federal government should have no role at all in financing schools and/or bossing them around. So why should you care? Notice however that the option of getting the feds out of the schooling business will not be seriously considered anywhere in the process. The federal government will still be sending money, will still be bossing schools around, but just would not get data reliable enough for Greatschools to even pull an Arizona-style rescue manoeuvre, or to consider student learning gains in making more rational retention and tenure decisions. In other words, federal policy will have been entirely captured by adult interests- funding will continue to flow, but transparency will suffer a train wreck.

In fact, if Congress were to pass a law with a parental opt-out, it doesn’t take a very active imagination to foresee a veto by President Obama. If you lead with your chin you should expect your opponent to break your jaw. A veto would allow Secretary Duncan to complete the last two years of rule by lawless administrative fiat in technocratic peace. Some of the latest waivers, by the way, have already stretched into the term of the next administration. They’ve already done waivers for states and districts- why not have the Department of Education hire some seasonal workers and offer waivers to individual schools if they are willing to pant, beg and roll over as commanded? Congress may lack the seriousness to actually put a stop to it.

Jay will argue that reformers brought this down on themselves with naive overreach. If the House votes to kill academic transparency it should indeed be a cause for education reformers to stare long and hard in the mirror. They won’t however be alone. The broad bipartisan consensus that created some level of academic transparency in return for federal funding and bossiness, warts and all, had a better idea than our contemporaries who would provide federal funding and bossiness in return for approximately nothing.

Who can stand against the union of the two towers, when far-left meets far-right at far-gone? Any chance Bill Jackson can take a long hike and throw the One Ring into the lava of Mount Doom?






The Kind of Control You Are Attempting…

May 11, 2015

(Guest Post by Matthew Ladner)

Is there much at stake in the fight over academic standards? Studies from no less than Hanushek and Loveless basically show that the standards movement has largely been pushing on a string. There is some evidence that suggests that states that were doing absolutely nothing on testing before NCLB saw above average math gains, but the fact is most states were testing before NCLB, the gains may have been a one time step increase, and evidence linking the quality of standards and/or tests to academic gains is in short supply.

NCLB’s attempt to test the nation’s kids to 100% proficiency (or as Andy Rotherham insists something more like close to it if you read the fine print, which few outside of Andy did) by a date certain ended in tears waivers.

My impression is that the standards movement basically hangs its hat on the Massachusetts experience. Massachusetts has the highest NAEP scores and thus is a good example to study. Massachusetts however introduced a multifaceted reform strategy in the early 1990s, but scholars seem remarkably incurious about which policy changes helped to drive how much improvement. Of course, like the Florida experience, we can never know what policy changes drove aggregate level improvements, but we have a great deal of micro-level evidence on the impact of individual policies. If any of this exists for Massachusetts, I’ve not seen it discussed. Even if we did have a good sense of this based upon a large body of studies, the question of external validity must be considered. Last time I checked MA was one of four states with an average family income for a family of four in the six figures and I’d wager draws an unusually high number of teachers from selective universities.

Why has the standards movement been pushing on a string? No it is not just that states set the test cut scores at incredibly low levels, although they did that:

It’s not just that states held a repulsive 35% of schools responsible for the scores of their special education kids scores in 2009-10, although they did just that:

After all of those things and others most states took the further step of obscuring the results behind a set of fuzzy labels, like Texas:

Some states have pulled this off much better than others, and a high quality system of transparency should be every policymakers goal. The idea that the country has meaningful, widespread “accountability” through state testing is a demonstrably simplistic notion. The greatest trick the devil ever pulled was conflating minimal skills testing in math and reading with robust accountability. While this is obviously absurd given a moment or two of reflection, it is also deeply ingrained in people’s thinking that you can do things like show a legislative committee a chart like the one immediately above, only to have a member of that committee berate you a mere few minutes later that private schools “lack accountability.”

Er, lack accountability compared to what? I may have missed it but I’m putting the number of people in Texas having been held responsible for the state’s 28% reading proficiency rate over/under at zero unless you want to blame it on the kids themselves, most of whom have been labeled “proficient” on state tests that the Wall Street Stock Picking Chicken might pass on a good day (see Figure 1).

Well yes, but the Common Core will fix all of this. Except of course it won’t. If you’ve been paying attention, you may have noticed that states all over the place have been adopting their own tests and cut scores and discussing withdrawing all together.  Meet the new boss, same as the old boss?

The current chaos shares an origin with the wrecking of the NCLB-era state tests. It is the same reason your tax dollars get used to pay farmers not to grow food so that you can pay higher grocery bills.  Agribusiness is organized and politically active, while eaters are disorganized and politically inactive.  Organized/active beats disorganized/inactive 99 times out of a 100.

So in theory, the state sets out grade level academic standards, and then tests children against those standards. Schools thereby follow a coherent flow of content such that you do simple addition before complex addition etc. In theory teachers and schools that fail to teach the standards get held accountable. In theory, there is no unauthorized breeding on Jurassic Park, but…

As long as you are going to have academic standards and tests, you ought to fight not to have horribly deceptive systems. You should rather fight for informative tests and clear labels, but with the full knowledge that the dinosaurs on your island will constantly be breaking out of your fences in any number of ways. They may even convince some people in the leafy suburbs that the substitution of one set of standards and tests for another constitutes oppression, er, somehow…how? I’m not entirely sure but…ah…stick it to THE MAN!

Bureaucratic accountability, in short, will always face severe political limitations, and even under the best of circumstances is no substitute for parents possessing an exit option. Even under the best theoretical systems there will always be kids who would be better off somewhere else for both academic and non-academic reasons. Decentralized accountability works best with transparency to inform choices, but centralized accountability without choice will inevitably face the gravity well of regulatory capture.

The level of control you are attempting is not possible.



Governor Ducey Calls for Arizona Academic Standards in State Board Address

March 23, 2015

(Guest Post by Matthew Ladner)

Consistent with his campaign position, Arizona Governor Ducey called for the creation of Math and Reading standards that are both high and unique to Arizona in a speech to the Arizona Board of Education.



Get every new post delivered to your Inbox.

Join 2,976 other followers