David Osborne on Charter Policy

December 5, 2017

(Guest Post by Matthew Ladner)

Bob Bowdon interviews David Osborne and around the 8 minute mark the discussion raises the topic as to whether parents can lead the way in school closure. Osborne claims that parents cling to schools for non-academic reasons and should not be trusted with this task. Bowden raises Arizona as a counter-example, and Osborne cites research from five years ago that found Arizona charter schools had less than stellar results.

Osborne is correct that research from five years ago found less than stellar results. What one must appreciate however is that in a sector as dynamic as Arizona charters those results are ancient history. Charter schools are constantly opening and closing in Arizona. The studies referenced by Osborne have data that ended in 2012. In 2013 Arizona educators opened 87 new charter schools and closed 18 other schools. This alone was enough to mean that the Arizona charter school sector of 2013 was a different animal than the 2012 sector, but it was hardly the only change to happen that year. In addition to schools, teachers, students and administrators moved in and out of Arizona charter sector. Younger schools gained a year of experience, moving out of their shake down cruises. A professional football team can win the Superbowl in one year and fail to make the playoffs the next year, and there is a greater degree of year to year continuity in sports than in Arizona charters. Fortunately all the available indicators (NAEP and AZMerit) show over time improvement in Arizona charters.

And then, it all happened again in 2014…and 2015…and 2016…and right now. Any look into academic results in a constantly changing Arizona charter sector is merely a snapshot. This makes it entirely possible for analysts like CREDO and Marty West to have found meh results in 2012 but for NAEP to show this in 2015:

This river of course runs both ways- the 2015 NAEP is just a snapshot as well. Fortunately Arizona’s charter results were also strong in the 2015 AZMerit, got better in the 2016 AZMerit and better still in the 2017 AZMerit.

Although Arizona law requires admission lotteries and Arizona charters educate a majority minority student body, there is obviously room for multiple factors to explain the above chart. Nevertheless, sometimes a cigar is just a cigar. Arizona’s AZMerit exam also shows large advantages for charter school students. In addition, school data of the sort CREDO and West relied upon five years ago is quite messy , especially when it comes to free and reduced lunch status. I won’t go into details but let’s just say that the record keeping at the Arizona Department of Education doesn’t always cover itself in glory and in addition many Arizona charter schools do not participate in the federal program at all. The use of Free and Reduced Lunch as an independent variable has grown increasingly questionable over time, but has been problematic for a long time in Arizona.

So in conclusion, Arizona charters are crushing the academic ball without the benevolent guidance of heavy handed technocrats. The same is true is several other Western states that show either high scores, high over time progress or else both of these things:

If Mr. Osborne would care to explain why westerners should abandon what they are doing to emulate Louisiana, I’m all ears but data from five years ago is of little more than historical interest to discussions of prospective policy.

Leave a Comment » | charter schools | Tagged: , , | Permalink
Posted by matthewladner

NAEP Cohort Gains by Scores for State Charter Sectors

December 4, 2017

(Guest Post by Matthew Ladner)

So to get into this chart you had to have a NAEP math score for your charter sector in the 2011 4th grade test, and then again for 8th graders in 2015. Many states either have no charter schools at all, or too few charter schools in 2011 (NC for example), or too few charter schools in 2011 or 2015 to make either sample. Some states fell into this lattercategory despite having venerable charter laws (yes I’m looking at you Indiana and Nevada). If you don’t see your state on the chart, keep calm and open more charter schools.

Otherwise a few notes: Pennsylvania and Maryland both look to have accidentally forgot their charter students any math between 4th grade in 2011 and 8th grade in 2015. I mean there is a few points of gain but one would expect that simply through aging. There might be something odd going on with the sampling or inclusion standards, but if I lived there, I’d be anxious to get to the bottom of it. I don’t so I’m not.

Michigan had the same progress over time as Louisiana despite the fact that Louisiana has been supported with philanthropy and TFA kids to a much larger extent.

Arizona and Colorado are sitting on the bench in the second half eating hot dogs and watching their backups brutalize their hapless opponents.

5 Comments | charter schools, NAEP | Tagged: | Permalink
Posted by matthewladner

In Memoriam for Parental Choice Irrational Exuberance

November 22, 2017

(Guest Post by Matthew Ladner)

I listened to the Federalist podcast this morning on my bike ride in which Megan McArdle debates Joy Pullman on school vouchers. In the discussion McArdle posits that parental choice programs allow parents to sort their kids for the purposes of being around other kids who are likely to want to go to college etc. She acknowledged that her instinct on this is informed by observation in the District of Columbia where she resides and that this has always been the case, it simply used to involve moving the suburbs. McArdle forsees a low-ceiling for choice as McArdle asserts that there are only so many good peer groups to go around, and large influxes of poor kids will lead to upper-middle class parents seeking to segregate their children elsewhere. Thus McArdle puts forward an air of jaded realism for choice as opposed to her optimistic days when she thought of parental choice as a cure for poverty. Worse still, her thesis would imply costs to choice for disadvantaged student groups.

Fortunatley, McArdle’s realism is not terribly realistic. Policy enthusiasts of all sorts and persuasions create problems for themselves when they sell their reform as a wonder drug that will cure the world’s pain and will do it today. People who buy in during the naive enthusiasm stage will invariably feel disappointed. This for instance is starting to happen with technology based personalized learning now.

The course of innovation does not run smoothly or in a predictable fashion. Technology enhanced learning may have a very bright future, but faces a messy process of sorting through things whether it ultimately proves out or not. The dot.com bubble provides a useful example. People of a certain age will recall the naive enthusiasm stage when any idiot who could mumble Silicon Valley lingo was able to get millions in venture capital funding for projects like Dogfood.com or whatever. When a great many of these dubious ventures went bust in the early aughts a fierce backlash mocked the broad notion that the internet was going to profoundly change business.

Give it some time however and no one is mocking now. The internet did profoundly change business and life and continues to do so- it just didn’t do it much by 2003, and not in the way we expected.

McArdle’s thesis combines elements of the disappointed naive early enthusiasm with a theory that those not exercising choice may be harmed by it. A large empirical literature on competitive effects however finds positive benefits from choice from students choosing to remain in district schools. The kids “left behind” in other words benefit academically from the fact that they have the possibility of leaving.

My home state of Arizona is a hot-bed for parental choice. We not only have the highest state percentage of students attending charter schools and private choice programs, recently data has come available showing that inter-and-intra district choice dwarfs both charter and private choice in the state. Arizona, a relatively low spending state (relative to other states not to its own past) that is in the midst of a border-state transition in student demographics (K-12 students ceased being majority Anglo years ago), is an odd state to be leading the nation in academic gains. Oh, well, we went and did it anyway.

Now McArdle’s thesis of tradeoffs and downsides should be visible in the academic trends for disadvantaged students. If the savvy parents are the ones exercising choice, and depriving disadvantaged kids of more ambitious peers, we might expect to see overall scores increasing but scores for disadvantaged kids sliding in Arizona’s results. Instead we see the opposite:

This of course does not prove that school choice lead to the larger than average gains, but good luck explaining how this happened if school choice were harmful to disadvantaged students.

At the 8th grade level, NAEP allows for tracking student achievement by the education status of their parents. This is what it looks like for students whose parents did not finish high school for the entire period we can track all three tests:

If these charts were reversed and we saw disadvantaged children showing less progress than the national average, but stupendous gains for already advantaged kids, that would be consistent with McArdle’s thesis. If I wanted to beat the horse into horse-burger I could put up similar charts by ethnicity and disability status. You have a very, very hard time finding supportive evidence of choice harming the disadvantaged in the academic trends of the nation’s most choice happy state, or in the broad empirical literature.

Improvement of schooling outcomes maybe a necessary but hardly a sufficient step in inter-generational poverty. Choice programs have triggered a process of opening up the suburbs to open enrollment in Arizona, and they seem to be doing something similar in Indiana. The choice movement made a grave error in fixating on a particular type of school in inner cities, and then became obsessed with “accountability” when the urban-only strategy floundered.

I’m confident that the evidence will prove out that broad, inclusive programs work out better for students by hugely broadening available options relative to the necessarily limited efforts to build new charters. If you want to help inner city kids, yes you give them access to private schools and charter schools. If you really want to help them you give them access to all of that and the leafy suburbs (see above charts). You can’t force the burbs to take out of district kids, but you can create the right incentives by subjecting their (all too often complacent and underperforming) schools competition as well. You create enough competition to allow parents to close schools they don’t like. Expect backlash from reactionaries along the way. It’s not quick, it’s not easy, it lacks magical powers. It can however deliver real benefits to students and taxpayers.

 

 

 

1 Comment | charter schools, vouchers | Tagged: , | Permalink
Posted by matthewladner

Riding on the Trainwreck of New Orleans?

November 15, 2017

(Guest Post by Matthew Ladner)

Peter Cook brought bad news about the latest test scores from New Orleans, and used the term “train wreck” to describe the results. This was ironic as the other day I saw a quote from David Osborne’s book that claimed that states that close charter schools have charter school sectors that substantially outperform district schools. In previous looks at NACSA state charter law rankings that came out before the most recent NAEP data, something like six of the top 10 states had too few charter schools to have made it into the NAEP sample, with the top two states in the rankings (Indiana and Nevada) included in the no-scores club despite have charter laws for many years. Of the top 10 rated states, Louisiana looked to be the best of the bunch, and they were towards the bottom of the top 10.

The figure below puts state charter sectors into context by comparing their 2015 NAEP 8th grade math score against their 2011 to 2015 cohort gain in scale points, and also includes all “Wild West” charter sectors. Unlike Nevada, most western states got middling to very low grades from NACSA, but can console themselves with the fact that they actually have “charter schools” generated by their “charter school law.”

Yes, okay, so well that happened. Don’t be looking for many westerners to be dropping everything their doing to emulate either Louisiana or Nevada. I remain a fan of the Louisiana RSD, as in my mind it was a very successful play to leverage the only thing New Orleans had left after the hurricane (empty school buildings) and get a system up and running. However, there is a lot of space between saying that and rushing to embrace the concept as the solution to our all of our problems.

Well, perhaps the reading results are more promising…

Nope- the reading results look very consistent with the math results. New Mexico charter school leaders just filed a petition with the Department of Cosmic Justice to protest Louisiana charters receiving much more hype but less demonstrable academic progress.

So my mutant mind reading power is reading objections in one or more of your minds. These comparisons aren’t fair! Only three of those Wild West sectors (Arizona, California, NM) have majority minority student bodies…given what we know about achievement gaps, we would not expect Louisiana charters to land on the right side of these charts. True enough unless you had a very high rate of improvement. Note that Louisiana charters demonstrate rates of improvement in both reading and math that weren’t bad, but also wasn’t either very high, or very different from the host state.

Again, this does not mean that RSD is bad. It seems to have been brilliant for New Orleans after the hurricane. That is a different question from “has it been so successful that everyone should rush to adopt them?” It is also a different question from “is this model politically sustainable in the face of predictable push-back?” or “what if Katrina hit Houston instead of New Orleans- are there that many TFA kids in the entire country?” or the question “is it possible that RSD would have been more successful without the benevolent guidance of a central command?” or most important of all “wouldn’t we be better off if we got to the point where parents rather than technocrats took the lead in closing schools?”

Some additional problems include the fact that a series of focus groups I saw earlier this year made it clear that people detest the idea of having the government shut down schools based on test scores. Oh, then there is the decisive rejection of an RSD by Georgia voters. The demos do not seem to be buying what the technos want to sell. Then there is the small matter of more recent state scores, which Peter Cook describes as a “train wreck” for RSD. Then the steady and insidious effort to essentially convert RSD back into a normal school district that seems to be going quite well for the reactionaries so far. I’m not sure about the train-wreck take, but I’m also confident that RSD is not a magic carpet made of steel, er, a solution we are likely to see politically sustained at scale.

3 Comments | charter schools, NAEP | Tagged: , , , | Permalink
Posted by matthewladner

Wild West and Loving It

November 9, 2017

NACSA you’ve got some ‘splaining to do!

(Guest Post by Matthew Ladner)

The chart below plots the 2015 8th NAEP grade math scores against the 2011 to 2015 NAEP math cohort gains. The below charts include state averages and the numbers for state charter school sectors in Arizona, Colorado, Idaho, New Mexico and Utah. In the NACSA ratings of state charter school laws most proximate to these data, I recall all of these states with the exception of New Mexico received single digit scores out of a possible 30 something points, clustering towards the bottom of the rankings. New Mexico still ranked pretty low. Generally these states lost points for not having default closure and similar type provisions. How did the charter schools in these state manage?

The high performance/low NACSA phenomenon looks to be a western trend. These charts are not stone tablets handed down from the mountain, but I can’t think of any reason they would systematically favor charter sectors. Those “Wild West” charter sectors look, ah, really good at math. If you recall the international comparisons, Massachusetts ranks up there near the best European and Asian countries. Let’s take a look at the reading results:

Well there you have it- AZ, CO, ID and CO all have Massachusetts like results, and it appears that when it comes to spurring reading gains, New Mexico charters are the ultimate power in the universe…I suggest Enchantment State parents use it.

Please do me a favor and email this post to the next five people you hear use the term “Wild West” as a term of derision in an education conversation. Bless their little hearts, but they generally have not bothered to look at empirical data in order to see whether it can be squared with their regional/ideological prejudices.

 

 

8 Comments | charter schools | Tagged: , , , , , | Permalink
Posted by matthewladner

NAEP Cohort Gains in Scale Points

November 6, 2017

(Guest Post by Matthew Ladner)

Quick post just to note that the outlines of the story remains broadly similar whether you measure cohort NAEP gains as percentage increases or simply as scale points gained: AZ statewide still leads the field, AZ charters still beats the field like what Clubber Lang did to Rocky in their first encounter in the ring, Michigan charters still edge Louisiana charters, Maryland still needs to invest in ammo and a fortified island complex in hopes of surviving the nuclear zombie apocalypse. The main storyline here is that like the Credo report finding Detroit and New Orleans neck and neck, NAEP finds a similar result in 2011 to 2015 cohort gains for Louisiana and Michigan charters.

The question of which measure is better seems debatable imo so it is easier to simply present both.

Leave a Comment » | charter schools | Tagged: , , , , | Permalink
Posted by matthewladner

Michigan Charters Continued

November 5, 2017

(Guest Post by Matthew Ladner)

When Doug Harris took to the pages of the New York Times to denounce Michigan charter schools as “the biggest school reform disaster in the country” he might have profited from an honest reading of the research literature and 10 minutes looking at NAEP data. For example, it took about 10 minutes to calculate the above chart. Michigan charters must be the fastest improving “disaster” yet seen.

Harris went on in the piece not just to trash Michigan, but to boast of the wonders of New Orleans charters. Harris used New Orleans to contrast as the Happy Hunting Grounds of charter schooling in contrast to the Detroit hellscape:

The New Orleans results have been impressive. In the decade after the reforms, the city’s standardized test scores have increased by eight to 15 percentile points and moved the district from the bottom to almost the state average on many measures. High school graduation and college entry rates also seem to have improved significantly, even while suspensions, expulsions and the rate of students switching schools have all dropped. Detroit and New Orleans represent radically different versions of school choice — and the one that seems to work is the one that uses the state oversight that Ms. DeVos opposes.

Well that sounds awfully impressive, but it perhaps less so when you check the only common metric testing data available in both Michigan and Louisiana charters-NAEP. When you plot the cohort gains of Louisiana and Michigan charter schools against state averages, it looks something like:

NAEP cohort gain calculations have their limitations, but the reader should note that similar findings to these were found in research on trends in state test scores. Perhaps someone will be kind enough to provide a link to the Credo dot chart showing nearly indistinguishable growth performance between Detroit and New Orleans in the comment section, but it was reminiscent of this chart as I recall.

Ironically, Harris also denounced Detroit charters as a “Wild West.” Just for the record- here is what a real Wild West charter sector’s results looked like between 2011 and 2015:

 

 

 

 

2 Comments | charter schools | Tagged: , | Permalink
Posted by matthewladner

The LINE to apologize to Michigan CHARTER SCHOOLS forms to the LEFT!

November 4, 2017

(Guest Post by Matthew Ladner)

Regular Jayblog readers may recall the tizzy that some worked themselves into about Michigan charter schools after Betsy DeVos was nominated to serve as Secretary of Education. Oh they are horrible, terrible, no good, “Wild West” etc. Max Eden and others attempted to set the record straight, noting Credo studies and other evidence showing stronger performance for Michigan charters. Not everyone much cared to consider any of that evidence business, and rumors of horrible Michigan charter school performance linger on to this day.

Well it turns out that NAEP cohort gains reinforce the conclusions of the Credo study and two other studies finding positive charter results. Those look like respectable math gains and very strong reading gains for “the biggest school reform disaster in the country” to these eyes. NAEP cohort gains are not perfect or infallible measures, but they are pointing in the same direction as the studies. Moreover, cohort gains rank above merely looking at raw scores as measures of school quality, which is **ahem** precisely one of the mistakes that critics made.

If Michigan charters are a catastrophic failure, what are we to make of the majority of state education systems (aka the blue dots)? Maryland is getting nervous with all of this disaster talk. If Michigan charters are a reform disaster then the Maryland school system just might qualify as a non-reform post-nuclear zombie apocalypse extinction event.

Maryland need brainnnns…and SPF 50,000 sunscreen!

If Michigan’s charter school skeptics would like to take a crack at explaining the above data from the National Assessment of Educational Progress and the three studies showing positive results the comment section courteously awaits your visit. Otherwise the line to offer an apology to educators running Michigan charters and the students making academic progress in them forms to the left.

 

 

9 Comments | charter schools | Tagged: , , | Permalink
Posted by matthewladner

Orlando for Chief Technocat

November 3, 2017

(Guest Post by Matthew Ladner)

Back in 2013 Forbes ran a contest between a group of financial services professionals, a group of school children and a cat named Orlando to see who could do the best job investing in stocks. “While the professionals used their decades of investment knowledge and traditional stock-picking methods, the cat selected stocks by throwing his favorite toy mouse on a grid of numbers allocated to different companies.”

Orlando’s victory ought to have come to the surprise of approximately no one. Stock picking chickens began defeating money managers decades ago. Recently the Wall Street Journal ran a long article demonstrating that the mutual funds rated five stars by the Morningstar service rarely hold on to that rating for very long.

So other than the obvious (buy index funds rather than pay for “expert” advice) what does this have to tell us about a panel of experts donning their lab coats in order to predict which schools will either perform well or flounder?

Things get even more complex with regards to schools when compared to stocks. Stocks have a very straightforward metric of value- a price. A well-meaning panel of school experts lacks a comparable metric, with enrollment trend and wait lists being the closest available analogies to prices. Such panels tend to rely on test scores, but past test scores may not only fail to predict future test scores, K-12 test scores often fail to predict future success in life.

Metrics for instance used to decide chartering and charter renewals in New Orleans for instance failed to predict future test score growth. In other words…

Alas Newt the Alien apocalypse survivor died in a really unwatchable sequel, but Orlando the Cat might be available to make life and death decisions for New Orleans charter schools. Apparently, just as with finance, panels of “experts” are not to be trusted with this task. Even if such metrics did predict future test score growth, we ought not to feel overly assured as the relationship between K-12 test scores and future success seems somewhere on the weak to tenuous spectrum in the currently available research literature.

Now if you don’t like the idea of a cat dropping a toy mouse on a numbered grid to decide which charter schools get approved and which close, we might decide to leave this task primarily to the collective judgments of parents. We’ve been earnestly assured that we can’t do such a thing because it didn’t work out in Cleveland or in X, but given the complete inability of humans to forecast the future, expert panels doesn’t have much of a chance to add value anywhere over the long run.

Rather than flattering itself with the notion that their expertise has prepared someone to exercise technocratic authority properly, the reform movement should spend time investigating the conditions under which bottom-up accountability succeeds and the conditions under which top-down accountability fails. Such an investigation could move the discussion beyond stale polemics such as should the government ever close a school and towards an investigation of the sorts of conditions that lead to success. How many options do parents need before they can effectively take the lead in closing schools?

I’m guessing the answer to this last question requires more than “zoned district school and young urban charters.” Turns out that the howling wind of purifying creative destruction story is a bit much when your only options are your zoned district inner city school and a handful of young inner city charter schools. Here in Arizona zoned district school, suburban district schools, tons of other charters and private school choice seems to be putting down new charters in a mere four years despite the fact that they have a 15 year charter from the state. A large majority of closed charters aren’t lasting long enough to reach a renewal process.

The benefits of such a system as opposed to heavy reliance on a panel of experts or even a cat seem both abundant and apparent.

 

4 Comments | charter schools | Tagged: | Permalink
Posted by matthewladner

« Previous Entries
Next Entries »