Arizona Charter Students Aren’t Left Handed Either Part Deux

October 18, 2018

(Guest Post by Matthew Ladner)

So out riding my bike on the canal this morning I had the idea for a new visualization for cohort NAEP gains. Here’s what it looks like:

So a bit of explanation: the shotgun blast at the lower left part of the chart are 4th grade math and reading scores for states in 2013, with Arizona charter school students included. Arizona charter school students didn’t blow anyone away with their math and reading scores as 4th graders in 2013, but this is in the range of what many would expect from a majority minority school system operating with very modest funding.

Fast forward the clock to 2017 and those kids were 8th graders, which are the shot gun blast of dots on the upper right. Lo and behold, that majority minority student population is **ahem** outscoring states that spend more than twice as much per pupil and have the advantaged end of the achievement gap stick. Arizona charter students pulled this off despite spotting such states a head start in the form of higher 4th grade scores.

Wait…I’m picking up a disturbance in the Force. I can feel you thinking “Ok but students come and go from charter schools and this must explain some of those gains.”

Actually kids do come and go from charters, but to the extent this is happening Arizona charters are sending out kids with higher levels of academic achievement and bringing in kids with lower levels of achievement. From the Center for Student Achievement:

So if numeracy and literacy are an important part of what you are looking for in a school for your child, you might want to move to Arizona. Once here you can consider enrolling your child in one of our pluralistic charter school offerings which focus on everything from the arts to equestrianism to the classics. As far as I can tell, it’s the finest system of public education in the country, and it is available to you free of charge delightfully without a crushing level of taxation. Plus…bring your golf clubs:

Advertisements

AZ Charters CeleNAEP Good Times Despite District Creaming

August 20, 2018

(Guest Post by Matthew Ladner)

The full implications of this new analysis from the Arizona Charter School Association showing that Arizona charter schools receive district transfers who are below statewide averages on AZMerit, and send students to districts who are on average above the statewide average, can only be fully appreciated in the context of additional data.


Quick play by play on the above chart: 85,000 students transferred between public schools between the 2014-15 school year and the 2015-16 school year. The AZMerit tests were given in the spring of 2015, before the students transferred to a new school in the Fall of 2015 for the 2015-16 school year. The sending sector in other words owns the score.

A huge part of the ongoing Two-Minute Hate against Arizona charter schools has been the notion that they are engaging in systematic creaming. The problem with this story is that the above chart reveals it to not only be false in aggregate, it also reveals it to be the exact opposite of the truth. On net the districts are sending out below average students to charters, and receiving above average students in return. The districts, in short, are guilty of precisely the charge hurled (without evidence) against charters. This is not to say that there aren’t individual schools, district and charter, doing bad things, but the net of everything everyone is doing appears to be district not charter creaming.

The part about charters sending above average performing students to districts deserves a special mention. A part of the litany against charters involves an obsession over high-school attrition rates in BASIS. This has always been off base, as only a small percentage of Arizona charter students attend BASIS, and BASIS is basically the Green Berets of academics. The Green Berets have an attrition rate as well, but the people who complete the training are deadly military professionals. Arizona students are very active (85,000 total transfers in a single year in a ~1.2m student system) and do so for a large variety of reasons- social, athletic, academic etc. I’ve always thought that the students who don’t complete BASIS were likely to have been better off for the experience, and lo and behold that seems to be the case statewide across charters. In other words, when the 8th grade BASIS student chooses to attend a comprehensive school to play football as a 9th grader and brings above average academics with him, the proper response from the district should be one of gratitude rather than condemnation. When the statewide scores show districts to be sending away low-performers in droves (to both charters and other districts btw) the complaint positively reeks of hypocrisy.

But I digress…

Another part of the two-minute hate litany would have us believe that a child with disabilities has never crossed the threshold of an Arizona charter school. If however one goes to the state’s AZMerit data file, you find the statewide percentage of district children with disabilities stood at 11.1% and at 9.73% in charters on statewide ELA exams. The percentages are similar in the math exams. In a similar fashion there is a difference in rates of limited English proficiency, but nothing like what the blaring telescreen would have us believe: 6.3% for districts and 4.4% for charters.

What about the part when the double-plus good duckspeaker screams through the telescreen to tell us that charters are bastions of White segregation? Try again: 55% of charter students are non-Anglos compared to 63% of Arizona district students. There is a difference, but both sectors are majority-minority, and neither looks like either Vermont (or North Scottsdale).

Some of the difference between charter and district performance is certainly explained by differences in student demographics but here is the next shoe to drop in the AZMerit data: every single subgroup available scores higher in charters than they do in districts. Native Americans, Latinos, African-Americans, Asians, Whites, ELL, SPED, FRL etc. students all score higher in AZ charters than in AZ districts. Some of the difference is certainly owing to demographic differences, but nothing close to all of it.

So turning our attention to the above chart and pairing it with NAEP, it makes the NAEP data seem even more impressive when you consider the fact that Arizona districts are sending below average performers to charters, and charters are sending out above average performers. Despite that, NAEP shows us things like this:

So for those scoring at home, Arizona charter schools educate a majority-minority student body, receive only about $8500 per pupil in public funding, receive low-performing students on average from districts, and send higher than average performing students to districts and…scored higher than Massachusetts on the 2017 8th grade NAEP exam plus demonstrated the best 2009-2017 improvement in the country.

I fully expect our friends in the Arizona charter school skeptic community to doubt the AZMerit analysis. My recommendation to them is to file an open-records request with the Arizona Department of Education for the same data file. Crucial findings such as this deserve scrutiny and replication. The evidence currently available leads to only one conclusion: Arizona charters are working extremely for the students fortunate enough to attend them.


Another Myth Bites the Dust

August 17, 2018

(Guest Post by Matthew Ladner)

The Arizona Charter School Association analyzed public school migration patterns for 85,000 kids who switched among public schools by their prior AZMerit scores. And well this is what it looks like:

So kids transferring from Arizona districts to Arizona charters have below average AZMerit scores. Those transferring from charters to districts have above average AZMerit scores. This is to put things mildly not consistent with the litany against charters that claims that Arizona charters are engaging in massive creaming of students. If anything, this data suggests the opposite: that if there is creaming occurring, it seems to be in the direction of districts creaming charters.

Mind you the creaming thesis already had huge problems, the first of which being math. When you see scores improving in both district and charter schools, it makes it very difficult indeed to argue that charters have circumvented lottery requirements (which districts don’t have) to only get the high flying academic kids. If this were the case it would have been a really neat trick to see Arizona districts showing better gains than the national average despite some of the nation’s largest Great Recession funding cuts, despite a continuing trend into deeper majority-minority states and despite losing tens of thousands of their top performing students.

The average transfer from district to charter having below average scores in fact might in part explain the district gains we see, but then you get back to those charter gains…looks like they are having a lot of success getting kids who were off track back on it.

I would be very interested to see the average grade level by transfer group in this data. My guess would be the average district to charter transfer student is an elementary student, and that the average charter to district student is more of a middle school student. In any case, when you are taking in lower than average kids and sending back higher than average performing kids as a sector, this myth is…


Craig Harris Throws His Other Shoe at the Charter Telescreen

July 12, 2018

(Guest Post by Matthew Ladner)

GOLDSTEIN! Er CHARTER SCHOOLS!

In our last two-minute hate, the Arizona Republic’s Craig Harris fretted about the founders of Basis buying a condo in NYC. In this exciting episode, Harris throws his shoe at charter school facility procurement.

So let me start by saying I’m not sure how other states handle charter school facility procurement laws, and I am curious about it. It is however worth noting that Arizona provides the best access to charter schools by zip code, per this Hamilton Project map from Brookings:

Hmm, maybe other states should be doing what Arizona is doing. Also worth noting is the fact that Arizona charters are not only more proximate than other states, they are also crushing the ball academically (statewide averages in blue, statewide charter averages where available in red):

Arizona just might be doing something right in this space, so my elephant says “Careful about ‘fixing’ something that is not broken.”

Here is the missing context from this article- construction firms make profit from both the construction of charter and district schools. In recent years districts in the state of Arizona have been spending at a half-billion per year annual clip on facilities despite having relatively flat enrollment growth in aggregate.

Let’s put it on the table from the outset that this number could never be zero (air-conditioners die, roofs leak etc.) Moreover, some of this spending involves the districts who are big winners in open enrollment making seats to meet demand. I can’t make any complaint about this portion of the building spree-I like it.

Having said that, $500,000,000 per year is a lot of money (enough to pay every teacher in the state $10,000 per year more) and there are hundreds of thousands of empty seats in Arizona districts, which were badly overbuilt during the boom. The Arizona School Facilities Board lists 1.4m square feet of vacant district space (approximately 35% of the total) and strangely enough almost none of this space is suitable for a charter school (in the estimation of district officials).

The word on the street here in PHX is that a handful of the big construction firms find that $500,000,000 profitable enough to invest in bond and override campaigns. So…we continue building space, including in districts with declining enrollment like Scottsdale. Maybe a reporter should look into that…

Now back to the current Harris piece. It is lacking in context, giving no information about the relative profit margins in charter versus district construction. There are no non-profits building schools in Arizona to my knowledge. Moreover, the profits in this case only come about because of the demand for the school model. Without demand, the CMO in question would lack the funds to buy the buildings-ergo no profit. Contrast this with a district system with 1.4 million empty seats but continuing to build more of them despite flat aggregate demand and at an enormous annual cost despite huge spare capacity and the costs for vacant buildings drawing money out of the classroom.

So all in all, where does charter school construction profits rank in a list of school facility issues in your opinion?

 

 

 

 

 


NAEP- First Report from the Front

April 10, 2018

(Guest Post by Matthew Ladner)

So the data went live at midnight EST. The national data is basically flat on all four tests. Most single cycle differences are not statistically significant, so I’m looking at longer periods of time, but the 2015 to 2017 champ looks to be Florida. More details on this later but I’m resisting the temptation to bring Sally back pending further analysis…but only barely!

The short run (2015 to 2017) state level data doesn’t look terribly exciting. I’m still digesting the longer run state level data, but up top there’s a chart to tide you over. NCLB required all states to take math and reading NAEP starting in 2003, so here is the full period (2003 to 2017) gains for 8th grade math and reading by state. These are numerical gains subtracting 2003 from 2017 scores by state.

The 2003 to 2017 period is a natural one to examine, but so too is the post 2009 period, for a few different reasons. The 2009 NAEP happened both in the early days of the Obama administration and during the outset of the Great Recession. NAEP redid the 4th and 8th grade science exams that year. The inclusion of “non-tested” subjects is healthy in my view given concerns over curriculum narrowing. According to my bleary eyes two states have made statistically significant gains in all six exams for the entire available period- Arizona and Mississippi. For the 2009 to 2015 period it had been AZ alone. Arizona and Mississippi were also the only states to make statistically significant gains on all of the math and reading tests during the 2009 to 2017 period. Several states however had statistically significant gains in 3 out of 4 of the math and reading exams between 2009 and 2017-including California, Hawaii, Indiana and Wyoming. On the downside, I have not yet added up the number of significant statewide declines in scores during this period, but there are many of them.

Arizona had three flat aggregate statewide scores and a decline in 4th grade math between 2015 and 2017. At first blush Arizona and Colorado charter schools crushed the academic ball again in 2017, will dig further into details/subgroups but for now:

There is more good news in statewide charter sectors, haven’t touched the TUDA yet, stay tuned.


Wild West Parents to Ineffective Schools-DRAW!

January 25, 2018

(Guest Post by Matthew Ladner)

Yours truly has a new piece at Ed Next today called “In Defense of Education’s Wild West: Charter Schools in the Four Corner States.” Here’s the punchline:

Just as the country benefits from political diversity, we also benefit from a diversity of policy approaches at the state level. There are those who seek greater uniformity among state charter-school policies—urging that all charters should be for five years and that default closure provisions should be spelled out, among other guidelines. Such advocates should consider the success of these western states, which have chosen not to adopt such policies. The 50 states will become less useful as laboratories of reform if we adopt a single set of policies everywhere.

Many states—including three of the four featured here—have experienced high rates of overall K–12 enrollment growth, which raises the opportunity cost of imposing a stringent charter-authorizing process. It does not follow that every state should rush to amend its charter policies to match those of Arizona, Colorado, New Mexico, or Utah, but the obvious flourishing of the charter sectors there offers food for thought. Questions to consider and debate include: What factors have led to success in these states? What steps can policymakers and philanthropists take to enable parents to take the leading role in closing undesired schools? How important a role does open enrollment in suburban districts play in creating a successful bottom-up accountability system?

We don’t know the answers to these questions. But we do know that relatively freewheeling charter-school systems have prospered in multiple states. Surely we have as much to learn from these success stories as we do from the cautionary tales from states that have experienced difficulties.

Check it out and let me know what you think.

 

 


David Osborne on Charter Policy

December 5, 2017

(Guest Post by Matthew Ladner)

Bob Bowdon interviews David Osborne and around the 8 minute mark the discussion raises the topic as to whether parents can lead the way in school closure. Osborne claims that parents cling to schools for non-academic reasons and should not be trusted with this task. Bowden raises Arizona as a counter-example, and Osborne cites research from five years ago that found Arizona charter schools had less than stellar results.

Osborne is correct that research from five years ago found less than stellar results. What one must appreciate however is that in a sector as dynamic as Arizona charters those results are ancient history. Charter schools are constantly opening and closing in Arizona. The studies referenced by Osborne have data that ended in 2012. In 2013 Arizona educators opened 87 new charter schools and closed 18 other schools. This alone was enough to mean that the Arizona charter school sector of 2013 was a different animal than the 2012 sector, but it was hardly the only change to happen that year. In addition to schools, teachers, students and administrators moved in and out of Arizona charter sector. Younger schools gained a year of experience, moving out of their shake down cruises. A professional football team can win the Superbowl in one year and fail to make the playoffs the next year, and there is a greater degree of year to year continuity in sports than in Arizona charters. Fortunately all the available indicators (NAEP and AZMerit) show over time improvement in Arizona charters.

And then, it all happened again in 2014…and 2015…and 2016…and right now. Any look into academic results in a constantly changing Arizona charter sector is merely a snapshot. This makes it entirely possible for analysts like CREDO and Marty West to have found meh results in 2012 but for NAEP to show this in 2015:

This river of course runs both ways- the 2015 NAEP is just a snapshot as well. Fortunately Arizona’s charter results were also strong in the 2015 AZMerit, got better in the 2016 AZMerit and better still in the 2017 AZMerit.

Although Arizona law requires admission lotteries and Arizona charters educate a majority minority student body, there is obviously room for multiple factors to explain the above chart. Nevertheless, sometimes a cigar is just a cigar. Arizona’s AZMerit exam also shows large advantages for charter school students. In addition, school data of the sort CREDO and West relied upon five years ago is quite messy , especially when it comes to free and reduced lunch status. I won’t go into details but let’s just say that the record keeping at the Arizona Department of Education doesn’t always cover itself in glory and in addition many Arizona charter schools do not participate in the federal program at all. The use of Free and Reduced Lunch as an independent variable has grown increasingly questionable over time, but has been problematic for a long time in Arizona.

So in conclusion, Arizona charters are crushing the academic ball without the benevolent guidance of heavy handed technocrats. The same is true is several other Western states that show either high scores, high over time progress or else both of these things:

If Mr. Osborne would care to explain why westerners should abandon what they are doing to emulate Louisiana, I’m all ears but data from five years ago is of little more than historical interest to discussions of prospective policy.