Greeks Bearing Gifts, Bridge Sales in Brooklyn, Confederate Currency and Kentucky NAEP Scores

November 22, 2011

(Guest Post by Matthew Ladner)

So does the data in this chart look fishy to anyone but me? What about after you read this?

Something Rotten in the State of NAEP?

November 10, 2011

(Guest Post by Matthew Ladner)

So if you measure the learning gains for children with disabilities on the four main NAEP exams for the entire period all 50 states and the DC have participated, you get the information in the above chart. Last week, the Bluegrass Institute’s Richard Innes alerted me in the comments and by email about fishy exclusion rates for children with disabilities and English Language Learners. I had only casually examined the exclusion rates, but having examined them more closely, I’m concerned.

The 2011 NAEP included standards for inclusion, which include 95% of all students selected for testing, including 85% of students with disabilities or classified as English Language Learners. One might possibly infer that some states were playing games and tricks with excluding such students in the past, and that simply listing the rates wasn’t doing the trick. This year, they listed expected standards and provided the gory details in an Appendix. On the conference call regarding the results, the NAEP team took pains to note this innovation.

So, as you can see, half of the states in the Top 10 gainers for children with disabilities just so happen to be states that violated the inclusion standards on one or more NAEP exam. Hmmm. Moreover, some of them didn’t just barely miss these standards, but instead chose to commit violence against them.

Maryland led the nation in gains among children with disabilities….or did they? Maryland’s inclusion rate for children with disabilities on the 4th grade reading test in 2011: 31%, which though completely pathetic actually beat the 30% rate for children with disabilities on the 8th grade reading test. The ELL rates were almost as bad.

The only other state to sink into the 30s? That would be second place Kentucky, which also excluded an enormous number of ELL students from NAEP examination. The math exams were better than the reading, but lo and behold- there is Maryland again falling below inclusion standards. Maryland failed to meet the 95% overall inclusion standard on 3 out of the 4 exams in 2011.

I have run the numbers for gains among children who are neither disabled nor ELL, and something real and positive is happening in Maryland: scores are up. It is however obvious that the NAEP created these standards for a reason, and have invited people to make up their own minds about whether to throw a skeptical flag in the air.

I’m throwing my flag. I don’t know if it explains all of the gains in Maryland and Kentucky, but it seems pretty obvious to me the results from those two states and perhaps others ought not to be considered comparable to the other states.

I’ve been told and I find it credible that these exclusions have only a small impact on the statewide numbers. Can we imagine however that very high exclusion rates for ELL students will not heavily bias the Hispanic number? Or that sky-high special ed exclusions won’t inflate a variety of subgroup scores? Or that excluding many of both of these subgroups won’t impact your Free and Reduced lunch eligible sample?

So given that the Congress mandated participation in NAEP as a part of NCLB, a mandate which all the federalist bones in my body find quite reasonable, perhaps it would be a jolly good idea for Congress to mandate minimum inclusion rates along with participation when reauthorization finally rolls around. Caesar’s wife must be above suspicion.

Los Estados que no desea ser reencarnado en si viene como un niño pobre de los hispanos.

November 4, 2011

(Guest Post by Matthew Ladner)

So how did my English to Spanish translator website do? I studied French while a student, which has come in handy about three times in my life, and may never do so again.

But I digress. The chart here ranks states by the percentage of low-income Hispanic students scoring “Below Basic” on the 4th grade NAEP reading exam in 2011.

Like any of these reincarnation charts, there are any number of factors to bear in mind. Some states have more ELL students than others, generational effects are important, and Hispanics are far from monolithic.

Nevertheless, isn’t it interesting that Oregon yet again makes an appearance in the hall of shame. Last time I visited, Oregon was way up in the Pacific Northwest and far from the southern border.

Now that the mandatory Oregon mocking is complete, let’s talk serious business: California is a disaster. The sheer size and low scores of the California Hispanic population ought to be a national concern. While it is fun to poke at Oregon for a being even worse than California, California’s Hispanic population is a sea to Oregon’s pond.

Matters are far better in Texas, the home of the second vast Hispanic population in America, but still very much in need of improvement.

California and Texas educate more than half the nation’s Hispanics, almost 5.5 million students. We need California to wake up, and for Texas to step up.

2011 NAEP: Florida Finally Hits a Wall

November 3, 2011

(Guest Post by Matthew Ladner)

The Florida Age of Public School Improvement hit a wall in the 2011 NAEP. This should not be terribly surprising, as Florida’s improvement seemed certain to plateau in the absence of additional reforms.

Governor Jeb Bush relentlessly pursued a dual strategy- transparency with teeth from the top down, parental choice from the bottom up. Together these reforms drove improvement in the public schools for a number of years.  Accountability measures included school grading (A-F) and earned promotion in the early grades. Parental choice measures included Opportunity Scholarships for children attending F rated schools, the nation’s first special needs voucher program (McKay Scholarships), the nation’s largest scholarship tax credit program (Step Up for Students), a decent charter school law and the nation’s most robust system of digital learning. Florida lawmakers also attempted to thoughtfully incentivize success.

Governor Bush took office in 1999 and left office in 2007. It would be nice if these efforts could indefinitely push progress forward, but there have been plenty of bumps and problems along the way. In 2006, the Florida Supreme Court rendered a logic-free ruling abolishing Opportunity Scholarships (failing school vouchers) for private schools, and followed that up by ruling against a state authorizer for charter schools. Tax-credits, McKay and digital learning continued to incrementally advance, but not at an earth-shattering rate.

The larger problem may have come in the top down measures. The chart below presents the distribution of district and charter school grades, with one line being the A/B grades and the other D/F grades. The dotted lines represent instances when the state board raised school grading standards.

The setting of these standards represents far more of an art than a science. Set them far too high and disaster follows (this happened in Arizona). Set them too low, and you remove the tension in the system needed to drive improvement. Even after the last increase in grading standards, more than 10 times as many Florida schools received A/B grades as D/F grades.

Florida’s policymakers raised standards four times, and last year (wisely) put in an automatic trigger to raise standards by a preset amount when a certain ratio of schools get A or B grades. In addition, a fresh set of reforms passed the Florida legislature in 2011, revamping teaching and increasing charter school and digital learning options.

Just as it is impossible to exactly pinpoint how much of what caused the gains, it is likewise impossible to say exactly what made them stall. Note however that one of the favorite explanations of the anti-reform crowd, the pre-school, finally saw the advent of children old enough to have participated in the program and age into the 4th grade NAEP sample. I hope that someone is carefully studying variation in participation and corresponding trends in FCAT data, but the results at the aggregate level thus far seem underwhelming.

Plenty of other things, however, have been going on- including the collapse of a housing bubble, cutbacks in public school funding (including of some of the incentive funding programs) and a variety of other very bad things. My advice to Florida policymakers: roll up your sleeves and get back at it. Despite the enormous amount of progress seen on NAEP (and no one loves celebrating it more than me) too great of a gulf lies between a state system awarding ten times as many top grades as low grades but still  suffering from large minorities of students scoring below basic on the NAEP exams.

Governor Bush has consistently said for years that success is never final, and reform is never finished. The 2011 pause in progress demonstrates that he called it correctly.  Moving the needle on student learning on a meaningful scale and at a sustained basis represents one of the greatest public policy challenges of our times. Governor Bush has passed the torch to a new generation of Florida reformers, and they must now find new ways, and fine-tune the old ways, to push academic progress forward.

Edited for typos

The New No Excuses: Where Not to be Reincarnated a Rich White Kid

November 3, 2011

(Guest Post by Matthew Ladner)

So the plot thickens, as many JPGB readers (including this author) was born as an American White kid who was not eligible for a Free or Reduced Lunch. In the Great Reincarnation to Come, maybe that is always how it works out!

Or maybe not.

In any case, you ought not to feel overly reassured. Assuming again that you want to learn to read, the above chart shows achievement levels from the 2011 NAEP for non-FRL eligible White students.

Before proceeding to dwell on West Virginia and others, I should note that DC has finally come in first place in something! If you are an ultra-wealthy White student going to one of the highly exclusive public schools in Georgetown, your reading ability rocks. Congratulations to the portion of the DC school into which few poor kids ever step foot much less attend.

Something has been going wrong in West Virginia, as their NAEP scores have been declining. Alaska is a different sort of place that obviously needs to get their act together on K-12. Tennessee can’t be happy to see themselves near the top of this list, and Nevada needs to let go of the idea that you don’t need to be well-educated to deal blackjack.

And then, there’s Oregon. Someone please explain to me why 21% of middle and upper income Anglos in Oregon should be illiterate.



2011 NAEP Guide Where to Avoid Being Reincarnated as a Student with a Disability

November 2, 2011

(Guest Post by Matthew Ladner)

I’m back from SLC, where I had the honor of serving as the opening act to Indiana Superintendent of Public Instruction Tony Bennett. Getting back to the mad science of exploring the 2011 NAEP, and keeping with a sudden OCD fear that I have developed over the possibility of being reincarnated, I present to you the states you want to avoid and those you want to pray to be born into in the next go-around if you happen to be born as a child with a learning disability, and you would like to learn how to read by the 4th grade.

So, whatever you do, try to load the dice to stay out of Washington DC, Hawaii, South Carolina, Alaska and Arizona if you think you might be coming back as a child with a disability.

Conversely, you’ve hit the relative jackpot if you land in Maryland, Massachusetts, Kentucky,New Jersey or Florida.

Seriously DC? 153?!? A mere 60+ point difference between next door neighbor Maryland?

My Cosmic Awareness/Spidey Sense just told me that you were just thinking “yeah, but hey no fair, because Maryland is far wealthier than DC.”

Except, well, it doesn’t really matter so much in terms of the gap. Below you will see a chart for Free and Reduced Lunch Eligible students with disabilities. The top 5 get shuffled a bit, but there is still an appalling gap between DC and the top performing states.

So as you make your reincarnation plans, just remember to stay away from DC, whereas if you have the misfortune of being born with a disability the chances of being academically warehoused seem to approach a near certainty.

Sadly, DC has plenty of company at the bottom.

Make Sure Not to Be Born in Michigan when Poor and Black

November 1, 2011

(Guest Post by Matthew Ladner)

If you want to learn how to read anyway, you need to stay away from Michigan, that is to say Detroit, if you are concerned about being born a poor Black child in the next life. The 2011 NAEP says to stay away from Iowa, Maine and DC for good measure.

On the other end of the scale: MA, NJ, DE, MD and FL are looking relatively good. Low-income Black children in Massachusetts reads a mere 2.5 grade levels ahead of their peers in Michigan on a 4th grade test.

Must run to the airport now. More later…

The 2011 NAEP Guide Where Not to be Reincarnated as a Poor Child

November 1, 2011

(Guest Post by Matthew Ladner)

The chart on the right presents scores for Free and Reduced Lunch Eligible students on the 2011 NAEP 4th grade reading test. Memo to self: remember not to come back as a poor kid in Alaska or DC in the next life. Ten points roughly equals a grade level worth of progress. Low-income kids in Alaska and DC are reading almost as poorly as 1st graders in Massachusetts, which is to say, not much all.

Florida hit a wall in terms of improvement (more on that later), DC saw nice math gains but not much progress in reading, Arizona finally started to move the needle a bit, and it is not entirely isolated to Hispanic children.

The 2009-2011 scores are pretty “meh” so far, and this biggest story I am finding is something big and positive going on with Maryland’s reading scores: 8 point gain for FRL kids between 2009 and 2011, and a nothing to sneeze at five point gain among middle and high income students.

More to come…

Some Thoughts in Advance of NAEP ’11 Release

October 27, 2011

(Guest Post by Matthew Ladner)

NAEP is going to release the 2011 Reading and Mathematics results on November 1st. I thought it would be interesting to boldly make some predictions in advance. Here’s my first one: the 2011 results won’t be all that different from the 2009 results.

I know, I’m going waaaaay out on a limb here, but that’s my prediction and I am sticking to it.

While a number of states have engaged in far-reaching reforms, the vast majority of these efforts still lie in the implementation stage. Possible exceptions in my mind include Washington D.C., Louisiana and Florida.

For DC, the 2011 NAEP will constitute the first plausible check on the tenure of Michelle Rhee. DCPS began making substantial math and reading progress in the mid 1990s, with huge gains but with scores still low. Assuming normal lags between changes and impacts, I believe that the 2009 NAEP arrived a bit early. I’ll be very interested to see what happens with the 2011 scores. Washington DC is also experiencing gentrification, so I will look at the free and reduced lunch numbers.

Louisiana will be a very interesting case, as some important statewide reforms still remain in the implementation phase, but where New Orleans has been in serious reform mode since 2005. I’ll take a look at the trend in urban numbers.

Florida of course enjoyed a steady increase in NAEP scores since 1998. Florida lawmakers also instituted a fresh set of far-reaching reforms in 2011, but the verdict on those will come years down the road. Governor Crist failed to pursue far-reaching reforms of his own, and vetoed some of those that reached his desk. Florida’s scores may rise again, but I won’t be surprised if they hit a plateau.

The Great Recession may also make this NAEP a little less incremental that usual. It will be interesting to see what happens to scores in the “Sand States” with the greatest property crashes (Arizona, California, Florida and Nevada) in addition to other states with acute economic distress like Michigan.

I will look with some interest at Arizona’s scores. Not only is the state face down on the economic canvass, with house building and flipping having been signature industries before the pop, it is possible that the infamous SB 1070 may lead to the illusion of progress in Hispanic scores. To the extent that the already partially overturned SB 1070 convinced undocumented families to leave Arizona, it may create the appearance of academic improvement.

Outside of that, I’ll be looking for pleasant surprises. Tell me what you are interested in seeing from the 2011 NAEP in the comments.

Buckle Up…2011 NAEP release on Nov. 1st

October 18, 2011

(Guest Post by Matthew Ladner)

NAEP is releasing 4th and 8th Grade Reading and Math results for 2011 on November 1st.

I’ll comb through the data and post the results here.

%d bloggers like this: