(Guest Post by Matthew Ladner)
The full implications of this new analysis from the Arizona Charter School Association showing that Arizona charter schools receive district transfers who are below statewide averages on AZMerit, and send students to districts who are on average above the statewide average, can only be fully appreciated in the context of additional data.
Quick play by play on the above chart: 85,000 students transferred between public schools between the 2014-15 school year and the 2015-16 school year. The AZMerit tests were given in the spring of 2015, before the students transferred to a new school in the Fall of 2015 for the 2015-16 school year. The sending sector in other words owns the score.
A huge part of the ongoing Two-Minute Hate against Arizona charter schools has been the notion that they are engaging in systematic creaming. The problem with this story is that the above chart reveals it to not only be false in aggregate, it also reveals it to be the exact opposite of the truth. On net the districts are sending out below average students to charters, and receiving above average students in return. The districts, in short, are guilty of precisely the charge hurled (without evidence) against charters. This is not to say that there aren’t individual schools, district and charter, doing bad things, but the net of everything everyone is doing appears to be district not charter creaming.
The part about charters sending above average performing students to districts deserves a special mention. A part of the litany against charters involves an obsession over high-school attrition rates in BASIS. This has always been off base, as only a small percentage of Arizona charter students attend BASIS, and BASIS is basically the Green Berets of academics. The Green Berets have an attrition rate as well, but the people who complete the training are deadly military professionals. Arizona students are very active (85,000 total transfers in a single year in a ~1.2m student system) and do so for a large variety of reasons- social, athletic, academic etc. I’ve always thought that the students who don’t complete BASIS were likely to have been better off for the experience, and lo and behold that seems to be the case statewide across charters. In other words, when the 8th grade BASIS student chooses to attend a comprehensive school to play football as a 9th grader and brings above average academics with him, the proper response from the district should be one of gratitude rather than condemnation. When the statewide scores show districts to be sending away low-performers in droves (to both charters and other districts btw) the complaint positively reeks of hypocrisy.
But I digress…
Another part of the two-minute hate litany would have us believe that a child with disabilities has never crossed the threshold of an Arizona charter school. If however one goes to the state’s AZMerit data file, you find the statewide percentage of district children with disabilities stood at 11.1% and at 9.73% in charters on statewide ELA exams. The percentages are similar in the math exams. In a similar fashion there is a difference in rates of limited English proficiency, but nothing like what the blaring telescreen would have us believe: 6.3% for districts and 4.4% for charters.
What about the part when the double-plus good duckspeaker screams through the telescreen to tell us that charters are bastions of White segregation? Try again: 55% of charter students are non-Anglos compared to 63% of Arizona district students. There is a difference, but both sectors are majority-minority, and neither looks like either Vermont (or North Scottsdale).
Some of the difference between charter and district performance is certainly explained by differences in student demographics but here is the next shoe to drop in the AZMerit data: every single subgroup available scores higher in charters than they do in districts. Native Americans, Latinos, African-Americans, Asians, Whites, ELL, SPED, FRL etc. students all score higher in AZ charters than in AZ districts. Some of the difference is certainly owing to demographic differences, but nothing close to all of it.
So turning our attention to the above chart and pairing it with NAEP, it makes the NAEP data seem even more impressive when you consider the fact that Arizona districts are sending below average performers to charters, and charters are sending out above average performers. Despite that, NAEP shows us things like this:
So for those scoring at home, Arizona charter schools educate a majority-minority student body, receive only about $8500 per pupil in public funding, receive low-performing students on average from districts, and send higher than average performing students to districts and…scored higher than Massachusetts on the 2017 8th grade NAEP exam plus demonstrated the best 2009-2017 improvement in the country.
I fully expect our friends in the Arizona charter school skeptic community to doubt the AZMerit analysis. My recommendation to them is to file an open-records request with the Arizona Department of Education for the same data file. Crucial findings such as this deserve scrutiny and replication. The evidence currently available leads to only one conclusion: Arizona charters are working extremely for the students fortunate enough to attend them.
Jay P. Greene's Blog 
It’s one thing to generalize to avoid an obvious issue. It’s another to blur the lines so badly you obfuscate. Your data on SPED enrollment is misleading and so badly misleading I have to question your intentions. Did you mean to misstate the situation that badly?
Overall SPED enrollment in District Public Schools exceeds 11% of enrollment. Not even close for Charters. More important, lumping all SPED kids in one basket is ridiculously deceiving. The kid with ADHD issues who achieves well with medication is hardly the same as the spastic quadriplegic Cerebral Palsy victim who is wheel chair bound. We find many of the former in Charters and hardly any of the latter.
The illegal barriers thrown up by Great Hearts, in particular, were published on Respect Our Scottsdale Students Facebook page this weekend. Would love to hear your thoughts on how or why GHA thinks it can force parents to sign away State and Federal statutorily granted rights.
Both sides of this debate are guilty of selecting facts and distorting truth. I am sometimes guilty – but you make me look like a piker when it comes to this blog post, Matt.
Mike-
The AZMerit data file (link provided in the post) allows you to go to the state page, where you can divide the total number of students taking the AZMerit ELA exam by the number of SPED students taking the ELA exam, and the same for the Math exams. This number is not perfect, as it only covers the AZMerit grades (no K-12) but this is what passes for the best academic transparency available in AZ (needs to be better) and the numbers are as I report them. The water cooler talk in the anti-charter circles often casually assumes a near zero SPED rate for charters, but this is clearly not only false but nearing willful ignorance.
The AZMerit file does not provide diagnosis details, and they could vary across sectors, but the more severe disabilities and children with multiple disabilities also receive far more generous state funding. An 11.3% vs. 9.7% SPED rate is a difference, and there may (or may not) be significant differences between diagnoses, but none of this takes away from the big picture: Districts are sending children to charters who have below average AZMerit scores, and are receiving kids from charters who have above average scores when they get there.
You’re proving my point. How many spastic quadriplegic kids with cerebral palsy do you think might have taken the AZ Merits test?
You also deflect the illegal Great Hearts Parent Handbook and agreement. How do you justify forced signing away of State statutory and Federal protection against forcible restraint?
The GHA Handbook is a great big blaring “SPED NOT WELCOME” sign to any parent of a special needs kid.
Back to the AZ Merits data. I already have the AZ Merits test scores download. School by school. Grade by grade. Sliced and diced every way you can slice.
Sometime over beverages, we should gather a couple more data wonks and compare thoughts. There are so many trends within that database it takes a lot of eyes and brain cells to pick out the pepper from the flyshit.
I’d be happy to meet, just got back from the FB page. I don’t have any expertise in this area of IDEA and I am not a spokesman for GH so best for this to be addressed by them.
On your broader point- that dog won’t hunt. NAEP allows you to compare general ed students (neither SPED nor ELL) in AZ districts and charters. Charter general ed students outscored district general ed students by 20 points on the 2017 NAEP 8th grade math test. That difference is both meaningful (worth about two years of avg academic progress) and statistically signficiant at the .0001 level.
Arizona does provide substantial additional funding to children with severe disabilities and I have never seen anything documenting disabilities types across sectors (far beyond our sad state of academic transparency). I don’t find it the least bit credible to assume that differences in special ed rates explains these sort of differences. In addition, if charters were systematically avoiding SPED, wouldn’t we expect to see above average AZMerit scores for those transferring from districts to charters? In fact, we see the opposite.
Opening move: My position is so obviously right, anyone who disagrees must be dishonest (“I have to question your intentions”)!
[Matt provides devastating point by point defense]
Backup plan: Hey, let’s go out for a beer and comb through the data in detail before we jump to any hasty conclusions!
Maybe next time try starting with the reasonable approach? Then you won’t have to backtrack to get there.
I stand by my assessment, Greg. One thing I’ve never done is make stuff up. I have 10 year enrollment histories for every grade level of every school in SUSD. I can prove the White Flight from Northern SUSD upper income areas to BASIS and Great Hearts.
The question is not whether they left SUSD’s upper income schools for GHA or BASIS. They did. We know when, how many, ethnicity, gender, and so on.
The question is what each kid’s baseline was for AIMS/AZ Merits when they left. Were they better than average students? We don’t know the kid by did data, but the AZ Merits test data seems to support that argument.
So with regard to statewide data, I’m not prepared to answer. With respect to SUSD data I am positive you’ll change your mind and admit that in the case of SUSD, Charters took better than average kids.
I could be nuts. But I doubt it.
Mike-
You have not written anything that makes me think you are nuts, but also nothing that makes me think that the data you describe is on point. The vast majority of North Scottsdale kids are well to do and Anglo. Some of them have indeed transferred, but kids all over Arizona are transferring. It isn’t the case that these kids were going to change their socio-demographics before transferring, they were just going to transfer as they were. If they didn’t transfer they were going to stay in highly segregated district schools.
Meanwhile, statewide rather than in an isolated pocket, we see either that districts are systematically divesting themselves of relatively low performing kids or that low-performing kids are systematically seeking the opportunity not to be low-performing elsewhere, or some combination of the two. It is perfectly plausible to me that North Scottsdale Tiger Mom likes BASIS and that this is nothing more than an unrepresentative anecdote obscuring a larger truth- districts not charters are doing most of the creaming.
Nope – not getting drug back to the Statewide sludge bucket.
Solving problems starts with specific micro-economic analysis of strictly controlled data. I can’t solve a problem in Douglas AZ if Scottsdale kids are mixed in to the data – or the opposite.
The 85255 Schools from which BASIS and GHA pulled their 2,200 White Upper Income kids are all 70% plus passing grade schools. By your definition they are 40% above the mean (for those reatding this who didn’t attend a school with competent pre-Algebra instructors that means .40 = (70-50) / 50.
Let’s start here – it is a VERY GOOD thing that Charters challenged Public Schools or Dave Peterson would have turned them all in to pre-post-pubescent day care and juvenile detention centers.
But there are great SUSD schools performing at or above the standard set by Great Hearts. None match BASIS.
But we can do that over time, also. If I have to go run the sumbiches for free working for whoever John Kriekard anoints as his successor, I’ll win that bet.
So let’s imagine a world where all the purple kids in the world live in North Scottsdale. Some of them transfer from district schools. You don’t then get to complain “A huge percentage of North Scottsdale transfer kids were PURPLE!!!!” Or if you do, the proper response is “Yes, okay, and….?” or better still “Why did they want to transfer?”
I am the soothsayer. I know the answer to each of those questions.