Parents Administer Frontier Justice in Wild West Charter Schools

July 18, 2017

(Guest Post by Matthew Ladner)

Parents out west administer quick frontier justice to undesired charter schools and the results are pretty awesome in today’s 74.

Yippie kai yay!


What Went Right?

July 6, 2017

(Guest Post by Matthew Ladner)

I was one of the people who decided to read more about Islamic history after 9/11. Half way through Bernard Lewis’ book What Went Wrong? it occurred to me that Lewis had asked the wrong question. Lewis explored the question of why the Islamic world had gone from arguably the world’s leading civilization to a relative backwater. By the time this had happened however most of the world was a relative backwater in relation to western Europe. The more relevant question therefore in my mind is not “what went wrong” everywhere else but rather “what went right in the West?”

This thought came to mind when reading David Griffith’s paen to regulation in choice programs. David asked “Is there a state in the union with strong oversight, robust market supports, and a low-performing charter sector?” Actually yes there is- NACSA’s top ten state charter sectors are nine miles of bad road interrupted by a unique one-off in Louisiana towards the bottom of the ten.

Griffith writes “Yes, there are a few states where charters have achieved strong results despite a weak framework for intervening in low performing schools, or a dearth of quality authorizers, or limited parental supports. There is an exception to every rule.” Arizona, Colorado and Utah all display the high NAEP/low NACSA score combo. They are not alone btw. By “high NAEP” I mean “near or above Massachusetts scores.” By “low NACSA” I mean a score of 8 or 9 on the NACSA rating before the most recent NAEP. Other flourishing charter sectors, which display either some of these same types of rock star scores in the case of Florida, or else significant advantages over district performance in the case of DC, also dwell outside the top 10 NACSA rated charter laws.

Griffith seems to have mistaken the exception for the rule. It is a simple matter to point to multiple examples of the high NAEP/low NACSA score combo. The high NACSA/high NAEP combo is actually very rare. This is either because top rated states have charter sectors too small to meet NAEP reporting standards-like Indiana and Nevada- or just still struggling after all these years despite the benevolent regulation of the state like Texas.

Now it might be a coincidence that we see high NAEP/low NACSA combos aplenty in the 2015 NAEP. The 2017 NAEP will be released in October. I expect the data to show us more of the same, but time will tell. It could also be a coincidence that voucher programs in Louisiana and Indiana experienced unusually low private school participation rates and struggled academically in the early year evaluations. Some of us started sounding alarm bells on the participation rates before the test score evaluations became available. You don’t need a random assignment study to tell that something is wrong with a voucher program that 70% of private schools choose to avoid, just a bit of common sense. The random assignment studies then did tell us something was indeed wrong, and then a helpful survey of schools pretty much nailed down why it happened. Griffith seems to believe that the problem with LA vouchers is under-regulation. There seems to be an abundance of evidence however that the opposite is true.

So getting back to Lewis, I am convinced that the right question is “what went right in Arizona, Colorado, DC, Florida, Utah etc?” rather than just “what went wrong in Ohio?” Under what set of circumstances can parents take the lead in putting down undesired schools with brutal efficiency? What factors lead this to working in some jurisdictions, but flopping in others? Texas went down the high regulation road in 2001, and well…let’s just say it does not bode well for Ohio.

Even if my friends with a preference for high levels of regulation had evidence to suggest that their approach has benefits (currently lacking) their yearning to apply a one-size-fits-all approach on 50 states with wildly varying needs would still be unwise. Nevada for instance can take little comfort from their high NACSA rating as they continue to suffer extreme levels of public school overcrowding with only a few dozen charter schools. There are hundreds of thousands of children on charter school wait lists in neighboring states with more welcoming charter school laws-why would operators in the surrounding states give Nevada a second thought? This is not a game, and these policies have very real consequences. This fall I will be sending the three Ladner children back to two fantastic charter schools. If either of these schools slips I have other options. Also this fall uncounted thousands of Nevadans will be sending their kids to portable building to meet the first of what will be a series of substitute teachers for the year. These parents have little in the way of other options. What is the case for keeping things this way in Nevada?

 

 

 


LGK on Arizona’s “Wild West” Charter Schooling

June 28, 2017

(Guest Post by Matthew Ladner)

Lisa Graham Keegan takes to the pages of Fordham to talk about lessons learned out here in the Wild West in When Regulating Charter Schools Proceed with Caution. Lisa raises the point that other policies, including A-F school grading, may have contributed to our success. I suspect that she is entirely right about that, but to me this is the money quote:

Moreover, Arizona’s “wild” charter journey led to many low-income, highly performing charter management organizations that can only be found in the Grand Canyon State. Many are community-focused and community-developed, which we all say that we want, but their first priority was on stabilizing the communities they grew from. In other words, they weren’t very good academically to start—but they did transform their neighborhoods, and parents trusted these new schools with their precious children over many other options that went out of business due to lack of enrollment. Years later, many of them, like Academies of Math and Science, Mexicayotl Academy, and Espiritu Schools, are now among the top performing schools in not just the state, but in the country, and were highlighted in last week’s Education Equality Index. The thing is, it took a decade to do that. And we Arizonans let it happen.

Translating this into Ladnerese- if Arizona had five year charters and default closures we might have arbitrarily closed some schools which blossomed into very high performing operations that now do a great job with disadvantaged kids. I use the word might because even if the Arizona Charter School Board had gone hillbilly nuts technocrat (Hey man- hold my beer while I close this school- this gonna be AWESOME!) the schools in question would have got their parents riled up, hired lawyers to engage in delaying actions, etc. I for one am happy that the schools LGK mentions could focus their energies on improving academics rather than fighting a bureaucratic guerrilla war.

Meanwhile these schools faced a much harsher form of accountability- from Arizona parents. Hundreds of Arizona charter schools have closed, and their average length of existence is 4 years, with an average of only 62 students in the final year of operation. If you live to see year 5 as an Arizona charter school, you are probably doing something right because everyone wants your students- your home district, fancy school districts like Scottsdale, Madison and Chandler are playing the open enrollment game, the other charter schools, and the private schools with the assistance of choice programs.

Frontier justice does not allow for parents to appear at a hearing to vent their anger, or for lawyers to file motions, or allies to lobby their political contacts for reprieve. The parents simply never enroll and/or walk away, there are private efforts to explain the reality of the situation to those institutions needing hospice care to wind down, and meh and sub-meh bleeds out on a dusty street full of hot lead. Some of you don’t believe this. Some of you don’t want to believe this. Well…just maybe…

 

 


Deep down in places they don’t talk about at parties, suburbanites want that wall, but broad choice can take walls down

June 12, 2017

(Guest Post by Matthew Ladner)

Important new study from the Fordham Institute on open enrollment in Ohio. The map above shows dark blue show districts not participating in open enrollment, and they just happen to be leafy suburban districts who are both higher income with student bodies that tend to be pale complected that also happen to be near large urban districts with many students who are neither of these things. Feel free to reference this map the next time someone claims that public schools “take everyone.”

Many moons ago I wrote a study for the Mackinac Center about the interaction between charter schools and open enrollment in Michigan. I found a very clear pattern among some of the suburban districts whereby charter schools provided the incentive for early open enrollment participants to opt-in. After one district began taking open enrollment transfers, and some additional charters opened, it created an incentive for additional nearby districts to opt in- they were now losing students to both charters and the opted in district. Through this mechanism, the highly economically and racially segregated walled-off district system began to:

Not every domino fell however. I interviewed a superintendent of a fancy inner ring suburb who related that they saw their competitors as elite private schools, not charter schools. When I asked him why his district chose not to participate in open enrollment, he told me something very close to “I think historically the feeling around here is that we have a good thing going, so they want to keep the unwashed masses out.”

Contrast this as well with Scottsdale Unified in Arizona, which is built for 38,000 students, educates 25,000 students, 4,000 of whom transferred into a Scottsdale Unified school through open enrollment. 4,000 transfer students would rank Scottsdale Unified as the 9th largest CMO in Arizona, and they are far from the only district participating in open enrollment in a big way. Why is Scottsdale willing to participate unlike those fancy Ohio districts? They have 9,000 kids living within their boundaries attending charter and private schools.

Why haven’t choice programs torn down the Berlin walls around suburban districts? Sadly because they have been overly focused on urban areas. The National Alliance for Public Charter Schools Dashboard shows that 72.6% of Ohio charter schools operated in urban areas. The voucher programs likewise started in Cleveland, and then expanded out to include failing schools (and children with disabilities statewide). More recently a broader voucher program has begun the process of phasing in slowly on a means-tested basis, but the combination of adding a single grade per year and means testing promises to unlock a very modest number of walled off suburban seats.

These programs have benefits, but they will not provide an incentive for fancy suburban districts to participate in open enrollment any time soon. Informal conversations I have had with Ohio folks related to me that Ohio suburban and rural dwellers- aka the people who elect the legislative majorities-tend to look at charter schools as a bit of a “Brand Ech” thing for inner city kids. Rest assured that the thousands of Scottsdale moms sitting on BASIS and Great Hearts charter school wait lists do not view charters as “Brand Ech.” Likewise these folks probably see themselves as paying most of the state of Ohio’s bills through their taxes and just might come to wonder why the state’s voucher programs seem so determined to do so little for their kids and communities.

A serious strategic error of the opening act of the parental choice movement was to look out to places like Lakewood Ohio or Scottsdale Arizona and say “those people already have choice.” This point of view is both seriously self-defeating in terms of developing sustaining coalitions, it also fails to appreciate the dynamic interactions between choice programs. Arizona’s choice policies include everyone and have created a virtuous cycle whereby fancy districts compete with charter and private school options for enrollment. This leads to a brutal crucible for new charter schools in Arizona whereby parents quickly shut many down because they have plenty of other options. Educators open lots of schools and parents close lots of schools-leading to world-class Arizona charter scores. Arizona’s charter NAEP score triumph was more or less mathematically inevitable once this process got rolling. Did I mention the part about Arizona leading the nation in statewide cohort NAEP gains since 2009? That too but Ohio not so much.

I’m open to challenge in the comment section from any of my Ohio friends or anyone else, but by contrast to these eyes Ohio’s choice programs look to be mired in an urban quagmire and they need the leafy suburbs to play in order to win. Current policies not only have not unlocked Ohio’s Scottsdale Unified equivalents, they likely never will. NACSA put Ohio’s revised charter school law in their top ten, but allow me to pull up a couch and heat up some popcorn for the next few years as charters lawyer up and parents resist arbitrary bureaucratic closures, and the rate of new schools opening goes glacial.

Competition is by far the best method of quality control and bringing the leafy suburban districts into the melee is crucial if you are in the urban fight to win. The districts currently largely untouched by charters and private choice overlap with those not participating in open enrollment. Regulating urban charters is not going to make your suburban districts into defacto CMOs. This.isn’t.hard.to.figure.out. While counter-intuitive to many if you want to secure improved education options for the poor, you need to include everyone.

 


Los Angeles Charter School Students Crushed the Ball on the 2015 NAEP

May 30, 2017

(Guest Post by Matthew Ladner)

One finds the largest gaps between charter and district performance in the entire NAEP in the Trial Urban District Assessment (TUDA) results for Los Angeles.  On 4th grade math, LA charters scored 249, LA district 221, 4th grade reading 237 for charter students, 200 for LA district students.  On 8th grade math, LA charter students scored 294, while LA district students 261, while in 8th grade reading LA charter students scored 276 while the district students scored 249. Sadly the TUDA did not participate in the 2015 4th and 8th grade Science exams. For you incurable skeptics out there, feel free to look the numbers up for yourselves.

Each of the Los Angeles charter school score averages were in very near vicinity of the highest statewide average scores (Massachusetts). I don’t have demographic information on LA charter students, but the California charter sector as a whole is majority-minority by a wide margin. I would expect the Los Angeles sector to be as well. Massachusetts and the other states clustered towards the top of state NAEP rankings meanwhile tend to have substantial socio-economic student body advantages. Massachusetts stacks up well with the top European and Asian schooling systems in international comparisons, so kudos to Los Angeles charters as your scores dinged the globally competitive bell. Is it worth mentioning that California charters received about $7,800 per pupil while New England states average almost twice that amount? Yes? Ok good well then that too.

As just a small random aside, NACSA’s scoring of state charter school laws from 2014, the most proximate ranking to the 2015 NAEP, gave California an 11 out of a possible 30 (see page 6). So Los Angeles joins Arizona and Colorado in the Low NACSA/High NAEP score combo club. I’m guessing there are other examples out there…

**Nerd Alert Time Out** Only a proper random assignment study would allow us to measure the degree to which school quality is responsible for the academic beat down that Los Angeles charter students administered to LAUSD averages. Let’s just note however that when the gaps in question are larger than the performance gap between Massachusetts and Alabama, you’ve got plenty of space for multiple factors and probably plenty of room left over for school effects. Standard errors for charter sectors are larger than for statewide samples, but sampling error can go either way-meaning that if we had actually tested all students rather than samples the scores could be either lower or higher. Random chance guiding Los Angeles charter students far to the right side of the bell curve on a single test could happen, but random error is very unlikely to behave the same way with four separate samples of students.

**TIME IN!** Congratulations to educators, students and investors in the Los Angeles charter school sector! Hopefully evaluations of long-term outcomes will match test score success in the Los Angeles charter sector, which is no longer something we can take for granted. Students have already taken the 2017 NAEP exams, with results expected in the fall. Let’s see what happens next.

UPDATE (6/1): One of the steps I took to explore Arizona’s charter school NAEP numbers involved looking at subgroup scores. So for instance Hispanic students attending AZ charters score quite well compared to the top statewide averages. I looked these up for Los Angeles charters this morning, but the TUDA does not provide information for some major student subgroups- including Hispanic students-on any of the four exams. Hispanics make up 73% of Los Angeles charter school students so it seems odd for TUDA to provide Anglo subgroups scores but not Hispanic subgroup scores.

I’ve made an inquiry with the NAEP folks and I will report back. Arizona’s case the state exam data displayed an even larger academic advantage for charter students than NAEP. The opposite appears to be the case with regards to Los Angeles charters. For now an * and some additional investigation seems warranted for Los Angeles charter TUDA scores.


The Origin of Arizona’s Nation Leading NAEP gains

May 1, 2017

(Guest Post by Matthew Ladner)

It’s been interesting for me to watch different columnists at the Arizona Republic react differently to the news that Arizona is the only state that has been the only state with statistically significant gains on all six NAEP exams. Bob Robb stated in a recent column that he supports choice but thinks it is limited strategy for improving outcomes. His colleague Joanna Allhands noted the fact that Arizona has lead the nation in NAEP gains, but said we have no idea of why that was the case.

I hope the above chart explains why I think even Robb is selling choice short, why Allhands should reconsider K-12 agnosticism. Formally Allhands is correct that we do not know why Arizona has been leading the nation in gains, but I hope a close examination of the above chart will be fairly persuasive that Arizona’s choice policies had a great deal to do with it.

So let’s peel the above chart like an onion. The first layer- blue columns- are the national public school gains across all six NAEP subjects (4th and 8th grade Math, Reading and Science). These are across time gains rather than cohort gains, calculated by subtracting the 2009 NAEP score from the 2015 score. Looking at the blue columns shows that the national progress falls into the strictly meh: 1 point on 4th grade math, -1 point on 8th grade math, one point on 4th grade reading, 2 points on 8th grade reading, four points on 4th and 8th grade science. Nothing much to celebrate nationally.

Next look at the yellow columns- these are the 2009 to 2015 gains for Arizona school districts (no charters). As you can see, these gains are consistently larger than the blue national public numbers, especially in math and science.

Third, look at the total statewide gains (Arizona flag columns). These are the gains for the combined district and charter schools between 2009 and 2015. As you can see, these gains are consistently larger than the district gains alone (yellow columns) and far, far larger than the national public averages (blue columns). Arizona was the only state to have statistically significant gains on all six NAEP exams between 2009 and 2015.

Finally, in the back in red tower the gains for Arizona charter schools between 2009 and 2015. The over/under for percentage of Arizona students attending charters in 2015 was around 15%, so although these gains are huge, they directly move the statewide needle by the differences between the yellow district columns and the flag columns. *See boring stat nerd note below.

The above gains represent the 2015 minus the 2009 scores-for example Arizona’s 8th grade math score minus Arizona’s 2009 8th grade math scores. A different method for calculating NAEP gains is to follow the progress of a single cohort of students across time. The NAEP math and reading tests have been timed and scaled to allow such comparisons- 4th graders took for instance the 4th grade NAEP math in 2011 and the NAEP 8th grade math exam in 2015. So…which state’s students learned the most about math between their 4th grade test in 2011 and and the 8th grade test in 2015?

Nationally American students gained 41 points between the 2011 4th grade exam and the 2015 8th grade exam-so nationally about 10 points per year. Arizona lead the nation with a 48 point gain. So how did Arizona charter schools fare in this comparison? **See second nerdy statistical note.

Note that the gap between Arizona charters and districts in cohort gains (12 points) is almost as large as the gap between gain leading Arizona and the lowest rated state (Alabama). So what does this mean in practical terms? The faster rate of improvement meant that Arizona charter school students got to do this on the 2015 NAEP, which is pretty cool if you like majority-minority schooling sectors that show globally competitive levels of academic achievement:

Finally, we have a rich set of empirical studies that suggest that parental choice leads to academic gains in traditional district school systems. Going back to the first slide, we have reason to suspect that some of the differences between the yellow and the blue columns relates to parental choice. If you suspect that budget cuts lead to academic gains (I don’t) then okay maybe, or if you can come up with a reason why new standards would have a very unusually large positive impact in Arizona when they flopped around the country, I’m willing to entertain a story to that effect, but it sounds like an implausibly complicated story.

On the choice side, round about 2007, the economy collapsed in a way that made a lot of property available, and Arizona’s charter sector put the peddle to the metal. Arizona charter school sector rose by a rousing 43,000 students in 2013 for example. The number of students exercising private choice also increased during this period, and statewide enrollment growth slowed, but that increase pales next to that of Arizona charters, which increased from 95,000ish students in 2008-09 to 188,000ish students in 2013-14.

We can feel confident that some of the difference between the statewide numbers and the blue columns relates to parental choice. We can feel very confident indeed that some of the difference between the red columns and the blue columns is related to parental choice. I’m open to other interpretations-and the comment section is open-but Occam’s Razor leads me to believe that a huge increase in the prevalence of parental choice that occurred during the Great Recession lead to direct benefits (high charter school scores) and powerful competitive effects (attract students or suffer real consequences- real accountability as opposed to the phony slap on the hand sort).

*The 2009 NAEP Arizona charter school estimates had large standard errors of estimates, owing to the considerably smaller size of the sector at the time. Unless Arizona charter school sleeper agents have infiltrated the NAEP there is little reason to suspect that random error will consistently advantage charter schools across six NAEP exams. Random error in both the 2015 and 2009 estimates means that the red columns in the first chart could be either smaller or bigger if we had actually tested everyone, but I’m at a loss for a reason to think of a reason why the errors across twelve different testing samples (six in 2009 and another six in 2015) would consistently line up to produce a mirage of Arizona charter school academic conquest, again absent sleeper agent infiltration.

** Standard error plays into the calculation of cohort gains as well on both ends of the calculation (in this case 2011 and 2015 scores) such that either could have been higher or lower if we had tested the entire population. Standard errors are larger for sub-population estimates than statewide averages, but again could play either way. For example if the population score for 2011 charter school students was higher than the NAEP estimate, the cohort gain will be overestimated, and if the true population score in 2011 was actually lower, then the cohort gain reported here would be an underestimate. All NAEP scores are estimates based on samples. Arizona’s charter school students displayed larger than any state cohort gains than any other state in both math and reading, but we cannot have the same level of confidence in these estimates as in statewide averages. Again, assuming random error and a lack of Arizona charter school sleeper agents in NAEP, we would not expect random error to consistently advantage Arizona charter schools.

Finally, the state’s AZ Merit exam also shows large advantages for Arizona charter school students vis a vis district students. Sampling is not an issue in AZ Merit, and these results lend reinforcement to the NAEP results. Unless…AZ charter school sleeper agents infiltrated the state’s testing system as well…

 

 

 


It’s Too Much Winning Arizona!

April 25, 2017

(Guest Post by Matthew Ladner)

The Arizona winning just does not stop-BASIS scores five of the top 10 US News and World Report’s Top 10 high schools.

You other 49 states are cordially invited to join in the winning. We’ve yet to find any point of diminishing marginal returns here in the Cactus Patch.