Vouchers were no Chuck Norris but Megan McArdle Should Hang in There

October 24, 2017

(Guest Post by Matthew Ladner)

Megan McArdle wrote a lament for school vouchers at Bloomberg. I’ll briefly lay out why I think a stronger case for optimism is warranted before trying to beat back insomnia with a youtube documentary of ancient Egyptian engineering (I wish I was making that up).

The first problem involves an over-reliance on studies of short-term test score trends. Our ability to study student test scores within the confines of a random assignment study usually lasts about three years.  During the early part of those three years students are dealing with negative transfer effects in the early going.

So if we take the available evidence from Milwaukee, within the random assignment window the evidence looks to me like this: the normal trajectory for a low-income urban child is to fall further behind over time. The control group of voucher users does not follow suit.  By the time the random assignment study falls apart some of the differences in test scores are statistically significant and in favor of the voucher students.

Is this a failure? It depends largely upon your expectations. If you had expected Milwaukee vouchers to heal the world’s pain, this is indeed disappointing. If you however gather some longer term evidence, find that the voucher kids have meaningful long-term attainment benefits and realized these benefits at a much smaller cost per pupil than the public system, you take a different view. I see Milwaukee vouchers as a success in an evolutionary process and want to find ways to make it more successful.

Milwaukee type programs suffer from design limitations and have hit a ceiling politically. They were basically designed to give families the option to move children into a preexisting set of private schools with empty seats. That’s a wonderful thing for many families, until you run out of empty seats. This doesn’t make these programs bad, just limited. Compared to the district that spends twice as much, has lower test scores and lower graduation rates, it is a bit of a triumph at least until your supply of empty seats runs out.  If we want more than that (and we should) we need more robust programs.

By “more robust” I mean programs with more equitable funding levels, enough to spur the creation of new schools. Programs that allow parents options outside of just private school tuition into a wider array of colleges, tutors, and service providers. Programs open to all children and communities that address with equity issues through funding weights rather than self-defeating exclusion.

Parental choice 2.0 (ESA) programs emerged from the unconstitutional ashes of an Arizona voucher program for children with disabilities just six years ago. Governor Napolitano signed a voucher bill for students with disabilities in 2004, but the Arizona Supreme Court ruled it unconstitutional. Lawmakers subsequently replaced it first with a scholarship tax credit law, and then the first ESA program. How is that whole crazy freedom and opportunity and ability to vote with your feet thing working out for AZ students with disabilities? Thanks for asking:

The ESA concept remains a work in progress with young programs operating in AZ, FL, MS, NC and TN. For a number of reasons, I believe that this model has a much higher academic and political ceiling than version 1.0, but caution is warranted. Part of the reason we see articles like McArdle’s is because we promised that the tears of vouchers would cure cancer like tomorrow.  School vouchers were a vitally necessary step in a process of unpredictable pacing, but they were more like your father’s Oldsmobile vis-a-vis your great grandfather’s Model T. We all want our flying car and we want it now and I can only tell you we are working on it, and the status-quo is both undesirable and unsustainable. The tears of ESAs won’t cure cancer tomorrow either, but the best is yet to come in the evolution of putting families in charge of the education of their children.

 

 

11 Comments | Education Savings Accounts, vouchers | Tagged: , | Permalink
Posted by matthewladner

Arizona Board of Regents Report for the Class of 2010

October 23, 2017

(Guest Post by Matthew Ladner)

I missed the release of the Arizona Board of Regents College Completion report back in April, this time covering the Class of 2010. The chart above shows the four-year college completion rate after six years for the top 10 schools. For those of you scoring at home, that would be 7 charter schools, one district magnet school (University High) and two comprehensive district schools (Catalina Foothills in Tucson and Chaparral in Scottsdale). All 10 of these schools represent choice options, with seven of them using random assignment admission lotteries, one employing placement tests and minimum GPA admission requirements, and the final two mostly for those who can afford pricey real estate.

Overall the news is not good. Only 27.1 percent of the statewide Arizona Class of 2010 earned either a four or two-year degree after six years. This is a lagging indicator, but a very important one. As is the case with the NAEP data, the chart above reinforces a case for a Cheers theme song theory for more small schools, you know, where everybody knows your name:

When you are tracking six year results from cohorts from seven years ago, they are by definition a time capsule. For instance these results come before the recent surge in NAEP scores. It certainly is not the case that only people with degrees contribute to society at all, and it might make sense to track the earning of professional certifications and military service in addition to higher education results. With the state’s age demography challenge unfolding we must continue to find ways to improve the productivity of education spending.

 

5 Comments | charter schools | Tagged: | Permalink
Posted by matthewladner

I Only Know One Truth- It is Time to Discuss AZ School Letter Grades on NPR

October 20, 2017

(Guest Post by Matthew Ladner)

Yours truly did a interview with our Arizona NPR station on Arizona’s preliminary school grades.  I explain why we have school grades, why grades can be better than fuzzy descriptors, why the preliminary Arizona grades lack face validity imo, and mistakes made in other states. Oh and why technological advances and revealed consumer preference may be making the process of state determined grades antiquated.

1 Comment | school accountability | Tagged: | Permalink
Posted by matthewladner

TFA goes hillbilly nuts

October 19, 2017

(Guest Post by Matthew Ladner)

This has been an open secret for a long time now, but now a TFA alumni has gone and spilled the beans.

If, as implied in the article, the organization adopted the habit of lefty virtue signalling in the hope of immunizing itself from criticism from Dianne Ravitch and the pool of AFT interns being whipped with a cat-o-nine tails to run her twitter feed, they chose poorly. Going hillbilly nuts in response to criticism from someone who has herself gone hillbilly nuts does not leave you as fellow hillbillies. It leaves you as both nuts.

How can she expect us to tweet any faster?

The interests of Ravitch’s puppet-masters remain in crushing alternative paths to reaching the classroom and talent pipelines for charter schools. So long as she continues to yearn for AFT’s adulation and receives enough cigarettes and pizza to keep the intern pool tweeting around the clock with a manageable level of grumbling, Ravitch seems unlikely to even pause to take notice of TFA press releases on freeing Mumia or the various lefty cause du jour.

Alienating yourself from most others however is a much easier task than getting AFT to stop pursuing what they believe to be in their interest. TFA’s leadership might want to think a little harder on the cost-benefit analysis of this “strategy.”

 

3 Comments | Uncategorized | Tagged: , | Permalink
Posted by matthewladner

Stanislav Petrov for the Al

October 17, 2017

(Guest Post by Matthew Ladner)

A few years ago I watched a documentary about just how close life as we know it nearly ended in 1983. If not for the fact that Robert Gates, former Secretary of Defense under George W. Bush and Barack Obama had not been interviewed in the piece, I would have had a very hard time believing the story (Gates worked in the CIA during the Reagan administration).

So humanity had a problem back in 1983. Many problems actually, but most acutely was the presence a heavily armed totalitarian communist empire with a vast nuclear arsenal and an ancient and insular leadership that more or less had lost touch with reality. During the first Reagan administration, the Soviet Union’s “intelligence community” and top leadership convinced themselves that President Reagan intended to launch a preemptive nuclear strike against the Soviet Union. It was an amazing and chilling example of group-think as even minor incidents were interpreted fully through the lens of confirmation bias. When for instance the United States increased security at embassies following the Beruit bombing it was seen, among many other things, as an “a-ha!” moment among the Soviets that the dastardly Americans planned to launch a nuclear strike. Mind you, it also shows that the ancient and insular Soviet leadership failed to grasp Reagan’s true and ultimately successful plan to spend their sclerotic economy into bankruptcy.

The Soviet intelligence community felt absolutely certain of this intent to attack despite the fact that it made no sense and despite the fact they had a spy at the highest levels of NATO insisting that there was zero sign of such an attack. The fact that their spy at the highest possible levels of NATO repeatedly insisted that there was no sign of an attack must have simply served of further proof of sinister imperialist intentions. Likewise, a highly placed western spy eventually reported back to NATO that the Soviets were convinced such a strike was coming, and people like Robert Gates had a very difficult time imagining such a thing to be possible. Have the Soviets lost their minds? Sadly they had.

On Sept. 26, 1983, during this hysteria, Lt. Colonel Stanislav Petrov was monitoring the satellite systems at Soviet Union’s Air Defense Forces when an alarm sounded announcing the launch of five American nuclear missiles. Protocol required Petrov to alert his superiors to the launch. The flight time of missles from the United States last only 20 minutes, and the ugly logic of “use it or lose it” in a scenario of Mutual Assured Destruction predominated Soviet military doctrine. “The siren howled, but I just sat there for a few seconds, staring at the big, back-lit, red screen with the word ‘launch’ on it,” Petrov related to the BBC in 2013. “There was no rule about how long we were allowed to think before we reported a strike, but we knew that every second of procrastination took away valuable time, that the Soviet Union’s military and political leadership needed to be informed without delay. All I had to do was to reach for the phone; to raise the direct line to our top commanders — but I couldn’t move. I felt like I was sitting on a hot frying pan.”

Instead of following established protocol and risking global thermonuclear destruction, Petrov utilized his own judgement. Why would the United States launch five missiles? If the United States were going to launch a preemptive strike, five missiles made no sense. He suspected some sort of error- which in fact it was in the form of a unusual cloud formation that the Soviet system had mistaken for an apocalyptic launch. Fortunately for all of us, Petrov decided to turn the alarm off, not to notify his superiors, and to simply wait a few minutes to see whether or not an attack occurred.

Petrov’s reward for saving the planet-an official reprimand from his “superiors.” A badge of honor to be sure, and it beat the living daylights out of the alternative. The Soviets btw kept their insane theory about a preemptive American attack until the completion of a NATO training exercise known as Operation Able Archer.

The conclusion of the NATO training exercise made it abundantly clear, even to the dimmest Soviet bulbs, that they had deluded themselves into a very dangerous hysteria. A few years later a younger Soviet leader began negotiating an end to the Cold War. The world survives a near brush with nuclear apocalypse: roll credits! This however was no Hollywood fable, it actually happened.

Petrov lived in obscurity for most of the remainder of his life, until news of just how close the Soviets were to going full 99 luftballoons came to light, whereupon he became known as “the man who saved the world.” The single sane man managed to thwart the insanity of a totalitarian bureaucracy died earlier this year in a Moscow suburb. Well done Stanislav and thank you. For choosing not to reach for that phone in what may have been the most crucial exercise of grace under pressure in history, I nominate Stanislav Petrov for a posthumous Al.

 

 

7 Comments | Al Copeland Humanitarian Award | Tagged: , , | Permalink
Posted by matthewladner

Connecticut Did Less with More

October 13, 2017

(Guest Post by Matthew Ladner)

A faction of folks in Arizona’s K-12 debate yearns to be Connecticut, but be careful what you wish for-you just might get it. In 2009 CT 4th graders scored a whopping 15 points ahead of Arizona 4th graders in NAEP math. By 2013 that had narrowed to a five point advantage on 8th grade math.

In 2015 only a single point separated AZ and CT on 8th grade math. CT literally spent more than twice as much per pupil as AZ in 2013-14. The 8th grade NAEP shows the two states pretty evenly matched across subgroups in math and reading. Call me crazy but I think it is Connecticut who should envy Arizona on K-12 rather than the other way around.

2 Comments | Uncategorized | Permalink
Posted by matthewladner

Cohort NAEP Gains by Spending

October 11, 2017

 

(Guest Post by Matthew Ladner)

So I’ve been cooking up some new charts on NAEP statewide cohort gains for math and reading by state K-12 per pupil spending trends. Imo the cohort gains are pretty good overall measure of statewide school quality, albeit not a perfect one. Student demographics influence all scores, but if school quality is going to assert itself they should have less of an influence on 8th grade scores than 4th grade scores simply because the kids have been in school longer. Thus question addressed along the horizon in this chart is how much math did your state’s students learn between 4th grade in 2009 and 8th grade in 2013? This is plotted against the trend in per capita spending between 2007 and 2014 per NCES.

So let’s note a few things here. First once again there is a lack of a discernible relationship between spending trend and academic outcomes. You had some states that made bid increases that bombed, and others that made big cuts and lead the nation in gains (take a bow Arizona educators and policymakers!)

Maybe this was a fluke. What happens if you do it again for math cohort gains between 2011 4th graders and 2015 8th graders?


Using my Professor X mutant super-power, I am reading your thoughts. You were thinking “Okay Ladner we get it something good is going in math. What about other subjects?” Fair enough- we can only do cohort gains in math and reading, so here is the cohort reading gains:

Well would you look at that- tied for second. Cohort gains are one method for measuring gains, but we can also look at over time gains for different cohorts of students, which allows us to bring in 4th and 8th grade science. Here is what those look like for the entire period we can get readings on all six tests (new results will be released in January 2017):

Can Arizona keep it up? I certainly hope so and we will find out in January.

3 Comments | Uncategorized | Tagged: , | Permalink
Posted by matthewladner

Two Minute Face Validity Check on AZ Preliminary School Grades 2-Arcadia versus Shadow Mountain

October 7, 2017

(Guest Post by Matthew Ladner)

Yesterday I did a quick face validity test on Arizona’s new school grades by comparing the schools closest to where I live. That didn’t go terrible well for the grading system. This morning before my bike ride I decided to do a second two minute test: comparing the district high school in the neighborhood I used to live (Shadow Mountain High School in the Paradise Valley district) to the one I live near today (Arcadia High School in Scottsdale Unified).

If someone would like to justify the red columns getting a “B” while the blue columns get a “C” the comment section awaits. If you are running Arcadia High, this is going to look to you like Shadow Mountain had a larger swoon on ELA than you from 2016 to 2017, had less math improvement than you (you had a smidge and they were flat). Moreover you outscored them in both math and reading in 2017, but they got a “B” and you got a “C.” Good luck getting the Arcadia High folks on board with this.

Greatschools btw gives both Arcadia and Shadow Mountain a 6 out of 10 for their academics. In other words, Greatschools gives them both the equivalent of a D. With proficiency rates in the twenties for Shadow Mountain and the thirties for Arcadia, it is hard to argue with that assessment. If there is a D-plus to be had here, clearly Arcadia is the school more deserving of it. The state could really, really use higher levels of achievement at both of these schools btw.

The subject at hand however is the grading formula used to create these preliminary grades. It certainly seems to lack face validity to me, but the Greatschools ratings seem to be defensible and thus useful to parents.

3 Comments | Uncategorized | Tagged: , , | Permalink
Posted by matthewladner

Arizona’s New School Grades Lack Face Validity

October 6, 2017

(Guest Post by Matthew Ladner)

Reviewing Arizona’s ESSA plan the Fordham event the other day inspired a growing sense of dread as I trudged through page after page of extreme complexity regarding the state’s plan to grade schools A-F. “If Jurassic Park scientists spliced the DNA of a Franz Kafka nightmare with a Rube Goldberg machine, it would look something like this…” I recall thinking to myself six or so pages in (less than halfway) through the description of the formula.

When we discussed the concept with Arizona lawmakers originally, the idea was a straightforward mix of 50% proficiency rates, 25% gains for all students, and 25% gains for the students who scored at the bottom 25% on the previous year’s test. This formulation is both easily understood and provides an incentive for schools to avoid having students falling hopelessly behind. Why settle for something tried, true and elegant when you can develop your own new and improved version?

An ultra-complex formula provides opportunity for things to go wrong. Grades were released today, they are almost entirely based on AZMerit scores, and the grades failed the very first check of face validity I tried from schools in my own neighborhood. The above charts show AZ Merit scores from Ingleside Middle School (Scottsdale Unified) and Archway Veritas. Respectively these are the closest and the second closest schools to where I live.

Greatschools gives Archway Veritas a 10 out of 10 ranking and Ingleside an 8 out of 10. Hard to quibble with that- Ingleside’s scores are above state averages while Veritas scores are far above state averages.

Under the newly released Arizona grades, Ingleside received a “B” from the state, while Archway Veritas received a “C.”  Despite the fact that Archway students had a 30% proficiency advantage in math, and a 24% proficiency advantage in ELA, they received a lower grade. Hmmm. This alas is simply the first test of face validity I ran, which lasted all of two minutes. If I were to spend a bit more time, I’m confident I could find even more inexplicable results.

Arizona suspended letter grades years ago due to the introduction of new tests. I know many of the people involved in the effort to revise the formula to be outstanding people who care deeply about improving Arizona K-12. Nevertheless, these grades lack face validity in my book and ought to be revised using a straightforward formula. Failing that, the legislature should adopt new school labels along the lines of “Blue” “Green” and “Poka Dot” to satisfy the feds and leave the task of ranking schools to private platforms such as Greatschools and others, which is where the eyeball traffic resides already. The state, alas, seems unequal to this task.

Anyone from either districts or charter schools will find plenty of things to complain about with a bit of examination. They will be upset. They will have a right to be upset. Fortunately Arizona’s nation leading academic progress is being driven from the bottom up.

4 Comments | Uncategorized | Tagged: | Permalink
Posted by matthewladner

Fordham Debate on Future State Gains

October 2, 2017

(Guest Post by Matthew Ladner)

So you’re hanging out in DC, sad because you can’t get your wonk on, when suddenly you remember that Fordham is hosting a debate on whether Arizona, California, Louisiana or Tennessee have the best prospects for academic gains going forward at 3 pm! !!Spoiler alert!! Special nerdy sneak preview chart above.

2 Comments | Uncategorized | Tagged: | Permalink
Posted by matthewladner

« Previous Entries
Next Entries »