NAEP Cohort Gains for Students with Disabilities, 2011 to 2015

March 22, 2017

(Guest Post by Matthew Ladner)

Which states had the most success teaching math and reading to their students between grades 4 and 8 on the most recent NAEP exams?

Quick (really boring) caveat section- tracking NAEP cohort gains gets more inexact due to the larger standard error of estimate when examining subgroup scores- which exists in both the 4th and 8th grade scores. Accordingly, don’t get too excited if your arch-rival Idaho edges you out by a point, because if we knew a true population score rather than sampled estimate it could easily be the other way around. It is more appropriate to look to see whether the state nearest and dearest to your heart landed near the top, or near the bottom of the list than to obsess about their ranking. Moreover, given the possible role sampling error, it is probably best to consider looking at the math and reading scores together. If a state rocks one of the tests and performs not so great on the other, you may have gotten a lucky sampling error bounce on one test, or a bad bounce on the other, or maybe your state is just better at teaching one of these subjects than the other. You would have to consult other sources of data to figure that out. Conversely, if a state does well on both reading and math it is unlikely that sampling error is driving the result in both cases, assuming randomly distributed error.

Ok so on with the show:

Top ranked Hawaii had gains more than twice as large as bottom ranked Maryland. Note that both of the bottom states (MD and KY) have a noted prior history of high exclusion rates for special needs children in NAEP. If they excluded a high percentage of kids in 2011 4th grade testing but not in 2015 8th grade testing, an appearance of catastrophe could sneak in. I am happy to note that my patch of cactus lies near the top. What about math?

Note that the overall gains for math are smaller than for reading. Also Hawaii is again at the top, and Arizona is near the top. Again Maryland appears to have taught a little more than one year’s worth of math to special needs students in four years, but I suspect that strange things are afoot at the MD Circle K on that one. I could dig into NAEP pdf files to check on exclusion rates but alas it is time for me to get out of my pajamas, so I will leave that to whatever you call people from Maryland. Marylanders?


We are but warriors for the working day, but our hearts are in the trim

March 14, 2017

(Guest Post by Matthew Ladner)

The Arizona Republic was kind enough to run the below letter to the editor from yours truly this morning in response to this editorial. If you are feeling the least bit skeptical, feel free to look these numbers up for yourself. The Republic’s editorial claims that now is not the time to expand parental choice because district schools are vulnerable. My claim is that Arizona district schools have never performed at a higher level than now and that we should in the immortal words of Darrel K. Royal “dance with the one that brung ya” which is to say stick with the strategies that brought success. Letter follows:

On the most recent Nation’s Report Card, Arizona 8th graders tied the state of Maryland in math, and outscored many states including Rhode Island, Delaware and North Carolina. These states spend far more per pupil than Arizona. None of these states has a majority-minority student population (Arizona does) but fortunately our students didn’t get the memo that they weren’t supposed to win. Instead they have been leading the nation in academic gains.

Arizona’s charter schools get still less money overall but scored higher than the statewide averages of 49 states on the same test. Arizona charter schools educate a majority-minority student population, but scored a single point lower than the highly funded and demographically advantaged Massachusetts-the nation’s long-time state academic champion. Again, the “you are supposed to lose” memo apparently went to Arizona’s spam folder, and our students and educators achieved an unprecedented academic triumph.

Arizona is never going to win a spending contest, but that is not the purpose of our investment. Our goal must be to maximize opportunity, not spending.

 

 

 

 

 

 

 

 

9


Why Arizona Charter Leaders Should Feel Confident About the 2017 NAEP

March 2, 2017

azmeritnaep

(Guest Post by Matthew Ladner)

So the 2017 NAEP is underway as we speak. As the NAEP released all six 2015 exams a cornucopia of good news spilled out for Arizona as a whole and (especially) for Arizona charters. Statewide Arizona has been leading the nation in 4th to 8th grade cohort gains since 2009. That’s a pretty good measure of overall school quality for those grades, as the demographic profile of the cohort isn’t likely to change much for a cohort between (for instance) the time when they were 4th graders in 2011 to when they were 8th graders in 2015. Kids will come and go of course, but absent a DC level of gentrification the academic ability of those coming and going should not skew heavily in a particular direction. You also have a measured amount of sampling error in both the 4th and 8th grade measures, but these are quite modest.

Arizona’s charter schools rocked all 2015 NAEP exams in the fashion of a New England state. This is quite impressive given the very modest level of per-pupil spending in Arizona charters, the majority minority student population etc. Sampling error is a bigger issue with regards to charter schools, but as you can see in the figure above, the state’s AZMerit exam tells us a story very similar to NAEP, and does not involve sampling. AZMerit in short provides backup to the findings in NAEP.

The figure above should lend confidence to Arizona charter leaders that they are likely to rock the NAEP again in 2017. The figure shows 4th and 8th grade NAEP scores from 2015 (proficiency rates) and then the proficiency rates for districts and charters from 2016 on AZMerit. These tests are not scaled exactly the same so you would not expect them to match up exactly, but the similarity in the pattern provides confidence that 2015 was not a fluke. Some of the differences between Arizona charters and districts can be attributed to differences in student demographics, although not nearly so much as sometimes are imagined, but New Hampshire doesn’t have any excuses for losing out to Arizona charter schools.

On the final set of columns on the right, for perspective, Arizona charters scored just a smidge below the highest performing state (MA) while Arizona districts would be right around the national average. Drawing random samples of students leaves room for goofy variation in subgroup scores, but there isn’t any reason to believe such goofiness would skew the same way in six different samples unless one wants to believe the NAEP is stacking the deck for charter schools in Arizona for some nefarious reason. While they were at it, the same conspirators would have to have infiltrated the AZMerit as well.

Absent goofiness and based on what we see in the AZMerit, I’m expecting Arizona charter students to CeleNAEP Good Times again in 2017. The statewide trend will be of even greater importance. Let’s see what happens next.

 


Tears for Beers

January 11, 2017

(Guest Post by Matthew Ladner)

Arizona Republic columnist Bob Robb wrote a piece on Arizona Governor Doug Ducey’s recent State of the State address. Money quote:

Ducey wasn’t a party to the deep cuts to K-12 education that were made after the bursting of the housing bubble knocked a big hole in state revenues. In fact, during his governorship, per-pupil spending, adjusted for inflation, has gone up, not down. Try to find an acknowledgement of that in the education funding debate.

In his speech, Ducey pointed out that “Arizona students are improving faster in math and reading than any other kids in the country.” That’s true.

Yet, there is a curious lack of curiosity about this development. In fact, Matt Ladner, a scholar with the Foundation for Excellence in Education, is about the only person in the state documenting it and inquiring about its causes. Everyone else is crying in their beer.

Bob’s kind remarks require two clarifications. First I made a professional transition a couple of months ago. Second, it is only bloody well near everyone else crying in their beer in Arizona, rather than actually everyone else. Crying in your beer is a bad look after all. A select few of us are just way too busy celeNAEPing our progress and trying to figure out ways to get more for it.


Anyone want to bet against Arizona for the 2017 NAEP?

December 13, 2016

(Guest Post by Matthew Ladner)

So Lisa Graham Keegan and I finally had the opportunity to collect on our bet with Mike Petrilli on the 2015 NAEP.  You may recall that Mike bet us before the release of the 2015 NAEP results for Reading and Math that Arizona’s NAEP scores would decline. Using our spidey-sense, LGK and I bet Mike that they would be going up, not down.  Arizona’s NAEP scores did go up. Mike was a good sport and quite appropriately paid his debt to us in copper cups (one of the state nicknames is the Copper State).

Depending upon how you examine the data Arizona is either near or else is at the actual top on gains. Measured by student cohort over time, Arizona’s 4th grade class of 2009 made more progress on Math and Reading between 4th and 8th grade scores in 2013 than any other state. Arizona’s 4th grade class of 2011 achieved the same pinnacle in their 2015 scores as 8th graders. (NAEP Math and Reading exams are both scaled and timed to allow such comparisons). The gains for Arizona charter school students dwarf those of Arizona as a whole, or any other state.

So anyhoo, the term “Wild West” is being thrown around as if it is a term of derision by some of those uncomfortable with the selection of Betsy DeVos as Secretary of Education. Here in the actual Wild West we wear the term with pride. The Arizona charter school sector has a majority minority student population, scored like a New England state on all six NAEP exams, and shows consistent results on the state PARCC exams.

Let me know when your state pulls something like that off, because I will be happy to celeNAEP with you. In the meantime, NAEP will be giving state level exams in Reading, Math and Writing in just a few weeks! Let’s see what happens next…


Where Do You Consistently Find the Highest NAEP Scores? Where Everybody Knows Your Name

October 28, 2016

(Guest Post by Matthew Ladner)

So hidden deep in the NAEP data explorer is a variable for school enrollment.  Yesterday we saw how Arizona charter schools crushed the ball on the 2015 NAEP science exams, but I was curious- would there be evidence suggesting that small schools of choice perform especially well? NAEP provides such a number in a crosstab for Arizona charter/district by school enrollment. Small district schools in Arizona performance is nothing to write home about, and are probably mostly rural. Arizona’s small charter schools-schools of choice-however, well, that is a different story. These are the 8th grade science NAEP scores for Arizona charter schools with 399 or fewer students compared to statewide averages for all students:

small-school-science

I thought that was interesting, so I checked to see how this would look in the 2015 NAEP Reading exam for 8th graders. Well-

small-school-reading

Well but the whole thing would fall apart in the math test. Except, it didn’t:

small-school-mathematics

Obviously this evidence is only suggestive, but do keep in mind that we have a large number of formal studies finding positive outcomes associated with attendance at small high schools. So perhaps high quality education involves authentic community with a shared vision of what constitutes high quality learning, and this process is facilitated by the ability of a child and parent to choose. It certainly appears to be the case out here in the Cactus Patch. Let’s call it the “Cheers theory of learning” in that you want to go where everybody knows your name. If that is you want to learn to read, figure some math, and understand science. If you prefer to fade into the background and then drop out of school- we’ve got plenty of Big Box schools to choose from as well.

So you see dare Normy....

So you see dare Normy dare used to be this big Foundation that had a great idea but then…


Arizona Charters Blow the Doors Off 2015 NAEP Science Gains

October 27, 2016

(Guest Post by Matthew Ladner)

Ok sports fans, I know you are all chomping at the bit, wanting to know “but Ladner, how do Arizona charter school science gains compare to statewide averages?!?” Oh I am glad you asked, here it is for 4th grade:

4th-grade-science-gains-charter

Hmmm…almost twice the gain as top ranking Arizona as a whole. Would it be running up the score to note that Arizona would not have done nearly as well without charters? I’ll just skip that part for now. Here are the NAEP 8th grade science gains:

naep-8th-grade-science-gains-charters

Well, would you look at that-twice as large as the largest state gain.  I’m crunching these out on a Prescott Library computer after taking a mountain bike ride on a “day off” but feel free to run the numbers for yourself here. I’ll breakdown subgroups later when I have more than 16 minutes left on my public computer use.

In the meantime I’m just going to go out on a limb here and say that there just might be something to this whole parental choice thing. Just maybe.