Arizona Charters: I’m Not Left Handed Either

April 18, 2018

(Guest Post by Matthew Ladner)

So Arizona charter schools rocked the 2017 NAEP again. In 8th grade math, Arizona charter students narrowly scored below the top scoring state (Massachusetts) in 2015 and above them in 2017. Arizona charter schools educate a majority minority student body, both overall and specifically in 2017’s 8th grade class. It’s really quite extraordinary to see them in the same academic neighborhood as MA, given their majority-Anglo, high income, spend twice as much per pupil combo status. If MA is a good sport, they would fight left handed- what chance does a majority-minority school system with half the spending per pupil have against the highest performing state education system in the nation many years running?

I decided to dig into the details.

The Free and Reduced Price lunch definition has become fairly sketchy, so in the below comparisons I will make use of parental education as a proxy for socio-economic status. Another source for variation between states and sectors involves special program students, so the below comparisons will focus on general education students (neither ELL nor SPED). The first set of scores are for students whose parents did not finish college and are in the general education program:

Massachusetts is still winning the duel, but not comfortably-switch to the right hand? Note for the record that 10 out of 10 of the top performing states have a majority Anglo student population. In fact you don’t spot a majority minority student population state until Texas pulls in at #19. Did I mention before that Arizona charter schools are majority minority? Oh, yes, well good that again then.

Now let’s run the same numbers for general education students with college graduate parents. This should be the MA right handed fighting btw- far more fancy degrees in Massachusetts than Arizona after all, more graduate degrees as well. Well, Arizona charter kids are not left-handed either:

 

Advertisements

I Say the Future is Ours if You Can Count

April 13, 2018

(Guest Post by Matthew Ladner)

Yesterday we went over how states have done very little to improve in 8th grade math since 2009, but how many state charter sectors rocked it. Let’s see about reading, starting with the states:

There are more declines in scores than statistically significant gains in that first chart-some states more lost than others, but a lost decade nationally. Next let’s look at statewide charter sectors-these are the states with charter sectors large enough to make the sample in both 2009 and 2017 in 8th grade reading:


Once again, just as with math, Excel had to move the growth axis scale for several state charter sectors. You will appreciate this better when the following chart combines statewide averages and state charter sector averages:

So in this chart ideally you would like to see high scores and high gains, but there is no shame in just very high scores.

Can you dig it?

Can you dig it?!?

CAN YOU DIG IT?!?!?!?!


John Rawls Rolls in his Grave Upon Viewing NAEP Trends for IEP Students

April 12, 2018

(Guest Post by Matthew Ladner)

…at least for most states.


Charters CeleNAEP Good Times During “Lost Decade”

April 11, 2018

(Guest Post by Matthew Ladner)

The news overall is grim. You want to be in the top right quadrant of this chart. Some states thankfully did land there-including the state with the largest student population- but many only barely due to small reading gains. If you need the dot size to push you in, it doesn’t count-better luck next NAEP.

You don’t want to land in the top left or in the bottom right, and you most of all don’t want to see your state in the bottom left quadrant (declines in both subjects). Mike Petrilli used the phrase “lost decade” to describe the results. Some states seem far more lost than others, but it is hard to find fault with that assessment overall.

The two main reform strategies employed since the 1990s have involved test-based accountability and increased parental choice. During the era covered by the top chart, the test-based folks swung for the fences by creating a federal incentive for states to adopt a preferred set of academic standards and to pass statewide teacher evaluation systems based upon the scores on those tests. Gigantic investments of political and financial capital supported these policy changes, but it is hard to characterize the results as much more than disappointing.

Now some of you will be thinking around about now “oh yeah but we’ve expanded choice during this period as well!” That is true, and while we have numerous studies establishing positive competitive effects on district schools from choice programs, few states have choice programs going at a scale to place a large amount of pressure on district enrolments. NAEP does however allow us to track state charter sector gains over time. Sixteen state charter sectors had scores for 8th grade math and reading in both 2009 and 2017, allowing the following calculation:

Excel had to change the scale of the axis for the above chart. You may not have noticed. Putting state averages and state charter sector averages into the same chart will help:

Suddenly those statewide gains in Arizona, California and Mississippi (i.e. the good ones) from the first chart don’t seem so impressive eh? I’m thinking out loud here and inviting you along for the ride. Gains aren’t everything, so the next iteration will include achievement and gains by subject area, but for the huge gainer sectors (spoiler alert) they didn’t get that way with low 2017 scores. I could go on about standard errors being bigger for charter sectors and whatnot, but who are you going to believe a boring statistics lecture or your own lying eyes? If someone can explain why random error would systematically dramatically favor charter sectors, I’m all ears and the comment section eagerly awaits your thoughtful challenge.

In fact there is a white lie in the above chart-some of the sectors and states that look meh in this chart immediately above had very high scores in both 2009 and 2017, and while not ideal there is no crime in holding your mud with high scores. Last year my friend Robert Pondiscio convinced me that combining achievement and gains to provide a clearer picture, so here goes for 8th grade math:

In this chart you either want to have large gains, or high scores, or preferably both of these things. I’m for instance not inclined to criticize Idaho charter schools for modest gains given that they outscore Massachusetts and all despite spending about half the amount per pupil.

Well…yup it is officially time:

 


Arizona Charters Crushed the Ball Again But They Have Competition Out West

April 10, 2018

(Guest Post by Matthew Ladner)

Arizona charter schools continued to display impressive scores and gains on the 2017 NAEP. Note as always that only a random assignment study properly conducted could hope to isolate the role of school quality in all of this, and that such studies are not only unavailable but also are impractical for a statewide system of schools- not all of which are oversubscribed (a precondition for a random assignment study). Thus the role of average school quality in driving the above results remains a holy mystery- likely to be hotly debated, impossible to be resolved in this world. I and others will be digging into Arizona charter subgroup scores in the days ahead.

Now, behold the nothing burger that has been net American academic progress since 2009 (first set of columns). On the math and reading tests 10 points approximately equals a grade level worth of average academic progress (the science exams have a different scaling). Net American progress equals nothing, nothing, next to nothing a tiny bit of something in 8th grade reading.

The middle blue columns are the statewide numbers for Arizona. As you can see they consistently surpass the American nothing-burger. The final set of gains are the gains for Arizona charter school students.

Some of Arizona’s neighbors also have charter sectors that crushed the ball- starting again with Colorado. I’m happy to say that Nevada’s sector made the minimum reporting requirements for NAEP this year, and the results look good. Give me some time to dig, but a monster story may be California charters:

 


Gains/Scores by State for Anglo Students

April 10, 2018

(Guest Post by Matthew Ladner)

Sorry-nodded off before posting the final piece of the ethnic trilogy. On to other subjects…


On 8th Grade Math, Massachusetts Leads with Hispanic Gains, Virginia with Scores

April 10, 2018

(Guest Post by Matthew Ladner)

A number of state Hispanic sectors closed the gap with the lowest performing statewide Anglo sector (274) in 8th grade math, which is good #MOARRR!