(Guest Post by Matthew Ladner)
So Texas once had a system of testing and accountability that was the envy of the nation. Texas boasted the highest Hispanic scores on NAEP in the nation not so long ago for instance. The Texas system served as the basis for the No Child Left Behind Act that required testing in grades 3-8 and once in high-school in return for federal education dollars.
The Texas system grew long in the tooth over time. Other states developed better standards and better testing systems and embraced more parental choice while Texas seemed to rest on its laurels. The Houston Chronicle revealed statistical hocus pocus that greatly inflated the number of highly rated schools committed by the Texas Education Agency. Gnomes in the basement of your state education agency can wreak havoc with any centralized system. Still, a system that gets you into the upper-left quadrant of the above chart (relatively low spending increase per pupil, relatively high gains on NAEP) was probably doing something right, especially if you are absorbing a Wyoming public school system sized cohort of additional students every year.
A toxic mixture of reformer overreach, devious alphabet soup group plotting and populist uprising has left this once proud system as a complete train-wreck. I will attempt to summarize this wreck in a single chart:
So only a large minority of Texas students can do grade level work on NAEP in any given subject, 91% of schools got a “met standard” label under the new “pass/fail” accountability system currently used during this brave new world of accountability chaos. This is an accountability system the reminds me of:
These labels are supposed to be transitional, but there will doubtlessly be efforts to codify them into statute during the 2015 legislative session. I could go on at some length about what a mess that the high-school end of course exam system has become, but I will spare you. Go and read the Dallas Morning News series linked to in the previous post if you’d like a detailed blow-by-blow on who all is to blame on this, but my own take is that there is plenty of blame to go around on both the reformer and alphabet soup side. The parents involved had genuine grievances regarding the testing system, but also must share in the blame for what is now a bad joke of a system.
Texans need to engage in a vigorous debate over what it is that they desire out of their system of academic testing and transparency. If the answer is “nothing really we just want to go through the motions of having such a system” then the legislature can codify pass/fail and further dummy down the high-school testing system. Trophies for everyone, a long era of academic stagnation awaits.
If not, then reformers might need to persuade parental activists to exercise greater responsibility to go along with their influence. Hopefully the Texas reform tribe has grown sadder and wiser as well.
Jay P. Greene's Blog 
Great post, Matt, but one quibble. “Full Grade Level Proficiency on NAEP” is probably a misleading title for your chart. What you mean is simply “Proficiency on NAEP.” The mention of grade level will probably make people think you’re referring to the old, totally discredited “performing at grade level” method of measuring outcomes. But that’s not what NAEP actually measures. “Proficient” on NAEP is actually something of a high bar. Nothing wrong with high bars! But we should be clear that the bar is high when we’re using a high bar.
Reblogged this on This Got My Attention and commented:
Texas Train Wreck in student testing.
Proficient on NAEP is a high bar, and those who don’t make it fall into two different NAEP achievement levels- either “Below Basic” or “Basic.” NAEP describes “Basic” as partial mastery of grade level knowledge and skills, and Proficient as solid grade level performance.
Obviously people can, will and have second guessed the levels, and I’m no testing expert, but one would hope that a large majority of your students would not either be hopeless far behind or partially mastering something like reading while your school gets a “pass.”
At any rate “grade level” is not what NAEP is aligned to.
Hard for me to decipher from the descriptions:
http://nces.ed.gov/nationsreportcard/reading/achieveall.asp
If you give an 8th grade math test, more or less describe the lowest level as lost, the next one as partial mastery and the next one as proficient and the one above that as advanced, then it looks to me like proficient or advanced falls into the full mastery of 8th grade math, at least in the opinion of the NAEP people. Advanced doubtlessly includes kids who are far beyond 8th grade math, cut off points are based upon I know not what, etc. but this seems like a reasonable of the descriptors to me.
Grade level seems implicit. A 4th grader score Proficient on the 4th math test would still likely be lost on an 8th grade test unless something odd were going on.
I’m doubtlessly missing something here, but what is it?
The old way of measuring “performing at grade level,” which is not used any more in the era of modern statistical standardization, involved taking the average score of all students in a given grade and then declaring that everyone above the average was “performing at grade level” while everyone below the average was “not performing at grade level.” You can see why they don’t do this any more. Today we either compare you to a benchmark that is set based on where we think you ought to be, without regard to how many students are actually there, or we compare you to where everyone else is but we express that as a percentile rank or in some other appropriate way rather than as a binary cutoff.
However, in the popular mind the superstitious idea that there is such a thing as “performing at grade level” still has a deep, talismanic significance.
So if you were to come up with a way to fix the problems these parents identified, and perpetuated by the TEA gnomes, how would you do it without giving everyone a trophy? Question two (this one is open-book): How would you get that actually passed into law?
Like I said in the post the other day, its not up to me. I am a Texan living abroad in Arizona, and we have plenty of our own problems out here, but since you asked:
The specific problems identified by the Houston Chronicle were eventually corrected. Gnomes are pesky creatures however and bear continual monitoring.
I actually think that the idea of end of course exams for high school students is conceptually sound. If we are going to have a kid take a Calculus class, it seems reasonable to me to find out whether he or she learned any Calculus. The people have spoken their verdict on a 15 end of course exam system required to graduate, however, and their voice is imperial. The transparency function of such tests is separate from the requirements to graduate unless linked.
At the moment however if an 8th grader takes Algebra I it isn’t clear that the district is in compliance with NCLB for testing math once in high-school. Standby while the fiercely independent Texans go to ask Arne Duncan for a waiver to bail them out of a silly mess they made by not including an Algebra II requirement. Que the Benny Hill laugh track and Yakedy Sax if he declines to provide it.
Despite the debacle, in an era of dual enrollment and digital learning, it seems clear to me that end of course exams are the direction we are going to need to go, eventually. I think high school should follow more of a university model of earned credits, but there should be a great deal more options available on how to get there. The default curriculum probably should have followed a more Regents Exam type of model.
On the lower grades, we basically want to know whether kids can read and figure out math. Lots of tests can perform that task, the trick is to keep out item exposure, curtail drilling to test items, etc. The old label system was not great (what’s better-exemplary or recognized?) but beats pass/fail. Grading schools A-F would be better still and actually passed the TX House overwhelmingly last session before getting undermined in the Senate.