If Every Instinct You Have is Wrong…

April 9, 2019

(Guest Post by Matthew Ladner)

In the Seinfeld episode “The Opposite” Jerry discovers that he is “Even Steven” in that setbacks are quickly followed by gains and his life basically remains the same. Meanwhile, George slowly morphs from a loser living in his parents basement to getting hired by the New York Yankees after giving George Steinbrenner a dressing down about the poor management of the franchise. Meanwhile Elaine ruins a merger of her company and finds herself unemployed. “I’ve become George,” she glumly observes. I often think of this episode when reading odd summaries of Arizona’s K-12 which are opposite of reality. Arizona Congressman Ruben Gallego for instance recently wrote a letter to Secretary of Education Betsy DeVos making a variety of claims about education in Arizona. The letter however makes a variety of claims about which are demonstrably mistaken. I’d like to address the following paragraph in particular:

Arizona public schools have improved performance over time rather than seeing performance decline. Student performance, as measured by the National Assessment of Educational Progress (NAEP) has improved both in the aggregate and across a variety of subgroups as shown in the figure below, which shows NAEP data from the first NAEP exam that includes all states (2003) to the most recent data available (2017) in 8th grade reading and math.

NAEP gains have improved Arizona’s subgroup rankings in overall proficiency by subgroup. For instance the chart below shows where Arizona Black students ranked compared to Black students in other states in 2003 on the left, 2017 on the right on 8th grade math. Moreover, as shown in the figure above, the largest Arizona gains were made by Black students in math (16 points), Hispanic students in reading (14 points) and American Indian students in Reading (16 points). Each of these student groups displayed a command of math and reading at 8th graders in 2017 that we might have reasonable expected their 2003 peers to have shown as 10th graders.

 

Here is the breakdown for Hispanic students:

Here is the breakdown for Anglo students:

If these levels of gains and proficiency represented the “fifth worst” school system,  the rest of the world would be looking at America’s international test scores with envy.  It seems profoundly unlikely that Arizona students would be making these enviable academic gains if choice harmed their education. Most Arizona students in Maricopa County attend a school other than their assigned district school. Students attending other district options however outnumber charter students by nearly two to one. Statewide charter student vastly outnumber private choice students in Arizona. Individual district schools both lose and gain students through family decisions. Choice is being done primarily by district schools rather than to district schools. Schools which fail to gain the confidence of families as the best option for their child do lose enrollment, but the positive academic trends show that Arizona schools are rising to the challenge rather than wilting under pressure.

Opponents of choice often conflate it with spending, which is misleading. A great many factors influence public school spending- the wealth of a state, the relative priority placed on K-12 compared to contending priorities like health care and higher education, local and state elections on funding and age demographics-states with lots of elderly and young people. One of the factors influencing per pupil funding trends is enrollment growth. Fast growing states have a harder time in accommodating growth and spending more per pupil at the same time. Arizona had the largest increase in per pupil spending among states with a 20% or more growth in enrollment between 2000 and 2015.

I will however agree with Representative Gallego to this extent: Arizona’s academic improvement is not a coincidence. They represent a huge amount of hard work put in by Arizona students and teachers and some very underestimated policies.  This is not to say there isn’t more progress needed (there is) but these gains don’t have “man-hands” or eat their peas one at time and shouldn’t be taken for granted.


AZ Charters CeleNAEP Good Times Despite District Creaming

August 20, 2018

(Guest Post by Matthew Ladner)

The full implications of this new analysis from the Arizona Charter School Association showing that Arizona charter schools receive district transfers who are below statewide averages on AZMerit, and send students to districts who are on average above the statewide average, can only be fully appreciated in the context of additional data.


Quick play by play on the above chart: 85,000 students transferred between public schools between the 2014-15 school year and the 2015-16 school year. The AZMerit tests were given in the spring of 2015, before the students transferred to a new school in the Fall of 2015 for the 2015-16 school year. The sending sector in other words owns the score.

A huge part of the ongoing Two-Minute Hate against Arizona charter schools has been the notion that they are engaging in systematic creaming. The problem with this story is that the above chart reveals it to not only be false in aggregate, it also reveals it to be the exact opposite of the truth. On net the districts are sending out below average students to charters, and receiving above average students in return. The districts, in short, are guilty of precisely the charge hurled (without evidence) against charters. This is not to say that there aren’t individual schools, district and charter, doing bad things, but the net of everything everyone is doing appears to be district not charter creaming.

The part about charters sending above average performing students to districts deserves a special mention. A part of the litany against charters involves an obsession over high-school attrition rates in BASIS. This has always been off base, as only a small percentage of Arizona charter students attend BASIS, and BASIS is basically the Green Berets of academics. The Green Berets have an attrition rate as well, but the people who complete the training are deadly military professionals. Arizona students are very active (85,000 total transfers in a single year in a ~1.2m student system) and do so for a large variety of reasons- social, athletic, academic etc. I’ve always thought that the students who don’t complete BASIS were likely to have been better off for the experience, and lo and behold that seems to be the case statewide across charters. In other words, when the 8th grade BASIS student chooses to attend a comprehensive school to play football as a 9th grader and brings above average academics with him, the proper response from the district should be one of gratitude rather than condemnation. When the statewide scores show districts to be sending away low-performers in droves (to both charters and other districts btw) the complaint positively reeks of hypocrisy.

But I digress…

Another part of the two-minute hate litany would have us believe that a child with disabilities has never crossed the threshold of an Arizona charter school. If however one goes to the state’s AZMerit data file, you find the statewide percentage of district children with disabilities stood at 11.1% and at 9.73% in charters on statewide ELA exams. The percentages are similar in the math exams. In a similar fashion there is a difference in rates of limited English proficiency, but nothing like what the blaring telescreen would have us believe: 6.3% for districts and 4.4% for charters.

What about the part when the double-plus good duckspeaker screams through the telescreen to tell us that charters are bastions of White segregation? Try again: 55% of charter students are non-Anglos compared to 63% of Arizona district students. There is a difference, but both sectors are majority-minority, and neither looks like either Vermont (or North Scottsdale).

Some of the difference between charter and district performance is certainly explained by differences in student demographics but here is the next shoe to drop in the AZMerit data: every single subgroup available scores higher in charters than they do in districts. Native Americans, Latinos, African-Americans, Asians, Whites, ELL, SPED, FRL etc. students all score higher in AZ charters than in AZ districts. Some of the difference is certainly owing to demographic differences, but nothing close to all of it.

So turning our attention to the above chart and pairing it with NAEP, it makes the NAEP data seem even more impressive when you consider the fact that Arizona districts are sending below average performers to charters, and charters are sending out above average performers. Despite that, NAEP shows us things like this:

So for those scoring at home, Arizona charter schools educate a majority-minority student body, receive only about $8500 per pupil in public funding, receive low-performing students on average from districts, and send higher than average performing students to districts and…scored higher than Massachusetts on the 2017 8th grade NAEP exam plus demonstrated the best 2009-2017 improvement in the country.

I fully expect our friends in the Arizona charter school skeptic community to doubt the AZMerit analysis. My recommendation to them is to file an open-records request with the Arizona Department of Education for the same data file. Crucial findings such as this deserve scrutiny and replication. The evidence currently available leads to only one conclusion: Arizona charters are working extremely for the students fortunate enough to attend them.


Yippie kai yay!

December 19, 2017

(Guest Post by Matthew Ladner)

 

 


NAEP Cohort Gains in Scale Points

November 6, 2017

(Guest Post by Matthew Ladner)

Quick post just to note that the outlines of the story remains broadly similar whether you measure cohort NAEP gains as percentage increases or simply as scale points gained: AZ statewide still leads the field, AZ charters still beats the field like what Clubber Lang did to Rocky in their first encounter in the ring, Michigan charters still edge Louisiana charters, Maryland still needs to invest in ammo and a fortified island complex in hopes of surviving the nuclear zombie apocalypse. The main storyline here is that like the Credo report finding Detroit and New Orleans neck and neck, NAEP finds a similar result in 2011 to 2015 cohort gains for Louisiana and Michigan charters.

The question of which measure is better seems debatable imo so it is easier to simply present both.


#TooMuchWinningAZ

April 24, 2017

(Guest Post by Matthew Ladner)

This weekend, the Arizona Republic editorial board cited a forthcoming report from the Morrison Institute to note that Arizona was the only state to achieve statistically significant increases on all six NAEP exams between 2009 and 2015. I decided to check it out.

The 2009 to 2015 period was not chosen arbitrarily, and has a good deal of historical significance. We can begin to track science achievement under the new framework starting in 2009. It is highly desirable to include science, as it is a “non-tested” subject for state accountability purposes. Starting the clock in 2009 is also useful historically as it tells us which states coped best with the Great Recession.

The chart above is a net of statistically significant gains minus statistically significant declines by test for the 2009 to 2015 period for 4th and 8th grade Math, Reading and Science. Only Arizona hit the maximum of six statistically significant improvements with zero statistically significant declines so got a score of six. South Dakota apparently had the worst overall performance with a minus three. Boring but necessary note: a few states (AK, CO,KS,NE,LA, PA and VT) did not participate in the Science exams, so their scores could only range between -4 and 4.

I think the President may have been referencing AZ NAEP scores when he said:


We are but warriors for the working day, but our hearts are in the trim

March 14, 2017

(Guest Post by Matthew Ladner)

The Arizona Republic was kind enough to run the below letter to the editor from yours truly this morning in response to this editorial. If you are feeling the least bit skeptical, feel free to look these numbers up for yourself. The Republic’s editorial claims that now is not the time to expand parental choice because district schools are vulnerable. My claim is that Arizona district schools have never performed at a higher level than now and that we should in the immortal words of Darrel K. Royal “dance with the one that brung ya” which is to say stick with the strategies that brought success. Letter follows:

On the most recent Nation’s Report Card, Arizona 8th graders tied the state of Maryland in math, and outscored many states including Rhode Island, Delaware and North Carolina. These states spend far more per pupil than Arizona. None of these states has a majority-minority student population (Arizona does) but fortunately our students didn’t get the memo that they weren’t supposed to win. Instead they have been leading the nation in academic gains.

Arizona’s charter schools get still less money overall but scored higher than the statewide averages of 49 states on the same test. Arizona charter schools educate a majority-minority student population, but scored a single point lower than the highly funded and demographically advantaged Massachusetts-the nation’s long-time state academic champion. Again, the “you are supposed to lose” memo apparently went to Arizona’s spam folder, and our students and educators achieved an unprecedented academic triumph.

Arizona is never going to win a spending contest, but that is not the purpose of our investment. Our goal must be to maximize opportunity, not spending.

 

 

 

 

 

 

 

 

9


Tears for Beers

January 11, 2017

(Guest Post by Matthew Ladner)

Arizona Republic columnist Bob Robb wrote a piece on Arizona Governor Doug Ducey’s recent State of the State address. Money quote:

Ducey wasn’t a party to the deep cuts to K-12 education that were made after the bursting of the housing bubble knocked a big hole in state revenues. In fact, during his governorship, per-pupil spending, adjusted for inflation, has gone up, not down. Try to find an acknowledgement of that in the education funding debate.

In his speech, Ducey pointed out that “Arizona students are improving faster in math and reading than any other kids in the country.” That’s true.

Yet, there is a curious lack of curiosity about this development. In fact, Matt Ladner, a scholar with the Foundation for Excellence in Education, is about the only person in the state documenting it and inquiring about its causes. Everyone else is crying in their beer.

Bob’s kind remarks require two clarifications. First I made a professional transition a couple of months ago. Second, it is only bloody well near everyone else crying in their beer in Arizona, rather than actually everyone else. Crying in your beer is a bad look after all. A select few of us are just way too busy celeNAEPing our progress and trying to figure out ways to get more for it.