New York’s Alright If You Like Saxophones, Taxes and Meh School Performance

November 8, 2017

(Guest Post by Matthew Ladner)

NAEP Reading Scores from 2015 along the horizontal axis, NAEP reading cohort gains (2015 8th grade scores minus 2011 4th grade scores). Ok so stare closely at the chart around the 262 score from the bottom to the top. Arizona, Tennessee, Oklahoma, Michigan, Rhode Island, New York, Florida and Delaware all had approximately the same 8th grade math score, but took different paths to get there. Some, like Delaware, Florida and Maryland started above the national average in their 2011 4th grade scores, but had small gains. Others like Arizona and Oklahoma, started below the national average in their 4th grade scores but grinded their way to large gains to catch up.

In 2011, Arizona 4th graders scored a 212 in 4th grade reading, Oklahoma a 215. Maryland’s 4th graders scored a 231 in 4th grade reading., New York stood at 222. Maryland students had an almost 19 point advantage over Arizona students and a 16 point advantage over Oklahoma students. Maryland spends far more than either Arizona or Oklahoma, and New York literally spends more than twice as much per pupil as either of these states. It shouldn’t happen that either Arizona or Oklahoma students would tie Maryland and/or New York by the time those 2011 4th graders became 8th graders.

Keep staring at that middle portion of the chart. Is Tennessee supposed to be neck and neck with Rhode Island? Rhode Island’s 7 point lead in the 2011 4th grade reading scores and almost $7,000 per student spending gap would say no, but the Tennessee kids didn’t get the memo and ended in a dead heat by 8th grade.

Ok so spot NY on the above chart and then look at math:

Arizona, Connecticut, Kansas and Maryland had 2011 4th grade math scores of 235, 242, 246 and 247 respectively. These had current (not total) expenditures that year of $7,782,$16,224, $9,802 and $13,946 per pupil. As an Arizonan, I’m delighted to have closed the gap with Connecticut, Kansas and Maryland. If I were a taxpayer or educator in Connecticut, Kansas and/or Maryland I would not be pleased.

Now locate New York on the math chart. I guess $19,965 per pupil just doesn’t buy what it used to in New York.

Ultimately it is good news that we have examples of states with diverse student bodies making academic progress. Remember- winter is coming to state budgets as 10,000 boomers per day reach the age of 65 and health care costs continue to rise. I hope you can get that sorted out New York but in the meantime both your students and taxpayers are getting horribly short-changed by your K-12 rent-seeking groups. The founders included a solution for you in our constitutional system: federalism. Did I mention that in addition to lower taxes, it is very pleasant here in the winter? As Ling Ving once sang “New York’s alright-if you want to freeze to death!”

Be sure to bring your golf clubs:

As far as where you’ll send your kids to school, Arizona has outstanding options in the public school system in both districts and charters. Here’s some dots to connect on the average performance of Arizona charters:

Additionally if you happen to prefer a private school for your child, Arizona’s policies support your families capacity to make that decision. Tired of having the daylights taxed out of you to pay for a public school system you don’t want to put your kids in, and then paying private school tuition on top of that? I thought you might. Head south until you reach Interstate 10 and then go west young family!


The “Giant” mystery in Maryland NAEP scores- Real Meh or the Appearance of Meh?

October 31, 2017

(Guest Post by Matthew Ladner)

When I think of Maryland, the scene of Liz Taylor riding a horse in the 1956 film classic Giant always comes to mind. Taylor’s character grows up in Maryland but subsequently moves to a desolate western ranch. The film maker used the rolling green hills of Maryland as a way to make a stark visual contrast with the desert southwest.

Likewise here in my arid state my friends on the left yearn to be Maryland. More spending and less choice = more better according to this way of thinking. Well, hmmm….what does the NAEP have to tell us about this POV?

As discussed in the previous post, Maryland had previously not complied with NAEP inclusion standards for students in special programs. They righted the ship in 2015, and it is reasonable to expect that compliance with those standards would have a substantial impact on, for instance, the scores of students with disabilities. Did compliance with inclusion standards also have a large impact on the overall scores for Maryland students? The above chart shows cohort gains from 2011 to 2015 in math and reading. The 2011 scores would have been out of compliance, whereas the 2015 scores were in compliance with NAEP inclusion standards. Did inclusion standards drive these poor results?

The drop in Maryland NAEP scores between 2013 and 2015 looks sudden and sharp. There may be no absolution to be found here for Maryland, as if it is the case that compliance with inclusion standards caused scores to drop precipitously, then the state’s reputation as having a high performing school system may have been built on exclusion of special program students. In other words, even if things are not as bad as they look in the above chart, they may shift to a different type of bad.

To test this question-have Maryland’s inclusion practices inflated their NAEP scores or did they just do poorly in 2015?- I ran cohort gains for general education students. General education students here are neither in ELL or SPED programs, and thus immunized from changes in inclusion standards over time.

For those of you squinting at your iphone, Maryland moves from dead last to merely clumped among the dead-last blob at the bottom left. Thus we conclude that changes in inclusion standards did play a role in the precipitous drop in 2015 NAEP scores, but that the state’s school system has bigger problems with which to wrestle. In other words, there is some real meh, not just the appearance of meh, especially if one were to bring spending into the conversation.

Deprived of the gains of special program students, Arizona slips slightly while Tennessee shows the largest overall gains for general education students. The 2017 NAEP data will be released in January, so let’s see what happens next.

 


Hawaii and Arizona made the most academic progress with students with disabilities 2011 to 2015

October 29, 2017

(Guest Post by Matthew Ladner)

Ok so here is what is going on in this chart: NAEP Math and Reading tests are timed and scaled in such a way as to allow for the calculation of cohort gains. In this case, we’ve tracked the statewide gains for students with disabilities from 4th graders in 2011 to 8th graders in 2015. Both the 2011 and 2015 measurements are a population estimate, and NAEP of course is not tracking the same students over time but rather are sampling both populations. The calculation used here is a straightforward 2015 8th grade scores for students with disabilities minus the 4th grade 2011 scores for students with disabilities, and then calculated as a percentage of improvement between 4th and 8th grade.

Students move in and out of states over time, but this sort of error should be largely random and cancel itself out in the absence of some (relatively implausible) systematic bias (like in this case higher performing students with disabilities fleeing Maryland to live in Hawaii). Given the standard errors, there isn’t much reason to fuss over exactly where you stand if you land say in the middle of the blue blob in the chart above, although one might take an interest in the states landing in the top right or bottom left.

Congrats to Hawaii and Arizona. Bad look for Maryland if taken at face value- having one of the nation’s highest spending per pupil figures but failing to teach students with disabilities much of anything about math and reading over a four years is, ah, terrible. Maryland is a state that had in earlier years flouted the NAEP’s inclusion standards for children with disabilities. It is possible that if they stopped doing so in 2015 that it may explain part of their place on this chart. If I lived in Maryland I would get to the bottom of this, but it’s time to get out of my pajamas.

For Hawaii and Arizona:

We’ll circle back and see how this goes when the new NAEP data is released in January.


Cohort NAEP Gains by Spending

October 11, 2017

 

(Guest Post by Matthew Ladner)

So I’ve been cooking up some new charts on NAEP statewide cohort gains for math and reading by state K-12 per pupil spending trends. Imo the cohort gains are pretty good overall measure of statewide school quality, albeit not a perfect one. Student demographics influence all scores, but if school quality is going to assert itself they should have less of an influence on 8th grade scores than 4th grade scores simply because the kids have been in school longer. Thus question addressed along the horizon in this chart is how much math did your state’s students learn between 4th grade in 2009 and 8th grade in 2013? This is plotted against the trend in per capita spending between 2007 and 2014 per NCES.

So let’s note a few things here. First once again there is a lack of a discernible relationship between spending trend and academic outcomes. You had some states that made bid increases that bombed, and others that made big cuts and lead the nation in gains (take a bow Arizona educators and policymakers!)

Maybe this was a fluke. What happens if you do it again for math cohort gains between 2011 4th graders and 2015 8th graders?


Using my Professor X mutant super-power, I am reading your thoughts. You were thinking “Okay Ladner we get it something good is going in math. What about other subjects?” Fair enough- we can only do cohort gains in math and reading, so here is the cohort reading gains:

Well would you look at that- tied for second. Cohort gains are one method for measuring gains, but we can also look at over time gains for different cohorts of students, which allows us to bring in 4th and 8th grade science. Here is what those look like for the entire period we can get readings on all six tests (new results will be released in January 2017):

Can Arizona keep it up? I certainly hope so and we will find out in January.


Texas Implemented a special ed cap, AZ implemented an ESA for special education children. Guess what happened next.

May 15, 2017

(Guest Post by Matthew Ladner)

The Raise Your Hand Texas group has released a white paper opposing an ESA program for special needs students in the Lone Star State. It is alas replete with boiler-plate nostrums etc. but if private choice is terrible for children with disabilities attending district schools, you have an awfully hard time finding evidence for it in the NAEP. We can get NAEP trends for children with disabilities on all six NAEP exams for the 2009 to 2015 period. The Arizona legislature passed a private choice tax credit for special needs children in 2009, and followed that up with the Empowerment Scholarship Accounts program in 2011.

Texas meanwhile during this period had the Texas Education Agency implementing a defacto cap on the number of special needs students in districts, without the slightest apparent protest from Texas districts, who implemented the program quite effectively. Ah, well, at least those Texas districts should have been doing a better job delivering special ed students for the children with disabilities they served, right?

Wrong.

Arizona authorities decided to expand options and increase freedom. As you can see, Arizona students with disabilities have demonstrated academic progress much better than the nation as a whole, which has either been treading water or actually declining. This looks pretty bad until you examine the scores for Texas students with disabilities, which are not only consistently worse, but which failed to show improvement in any of the six subjects covered by NAEP.

These trends obviously have factors other than choice which impact them, but if the theory is that ESAs are terrible for children with disabilities in public schools, we can reject the hypothesis. Texas has a special education disaster on its hands, while Arizona is making progress far and away above the national average. No student group has more to gain from choice than children with disabilities- including those who choose to remain in districts.

 


The 2016 AZ Merit: Improving but Meh versus Mehssachusetts

September 19, 2016

(Guest Post by Matthew Ladner)

The Arizona Department of Education released 2016 AZMerit data last week, and charter school students show an across the board advantage.

az-merit-2016

Breaking down the data by subgroups consistently show charter advantages as well. Let’s start with Anglos:

azmerit-anglo

Move on to Hispanic students:

az-merit-hispanic

African-American students:

azmerit-black

 

Native American students:

azmerit-native-american

Asian students:

azmerit-asian

Special education students (btw the percentages of special education students among district and charter schools were roughly equivalent among tested students):

azmerit-sped

Economically disadvantaged students, but with an *.  A high percentage of Arizona charter schools do not participate in the federal free and reduced lunch program, and about 85% of alternative (dropout recovery) schools in Arizona are charter schools. Having said that, charter schools score higher again:

amerit-ecodis

This next chart required me to dust off my algebra skills and use the existing data to solve for X:

azmerit-non-disadvantaged

What we can take from this: while differences in student populations explains some of the differences between overall charter and district scores, when the charters lead in each and every subgroup it does not explain anything close to all of the difference. Every difference across every subgroup counts, and in the end the add up to:

az-ma

For those of you squinting at your Ipad, that is the statewide averages for Massachusetts (the highest scoring state) on the left, Arizona charters in the middle, and Arizona Districts.  MA of course is much wealthier and spends a great deal more than AZ, but when you break down the subgroups Arizona charter students outscore like students from Massachusetts, which makes me want to CeleNAEP good times here in the cactus patch: