Over at RedefinED I dip a toe into podcasting with my old nemesis Sherman Dorn (wait…that makes one of us the bad guy right? Not it!) Dr. Dorn and I used to argue about Florida NAEP scores, but now we both live in the Cactus Patch. Anyway Dorn very kindly hosted me at Arizona State to record the podcast, which is in two parts, and (I think) we basically agree that public schools are over-regulated and seem to reach a consensus on a lighter footprint testing system. Along the way we discuss the 20th anniversary of Jeb Bush’s reforms and other stuff. Jayblog fans should take a listen: part I and part II.
Last spring I was on my bike and came across this in front of a local middle school. I found it striking enough to take a picture with my phone:
In case you are squinting at your iPhone, the sign says “AZ Merit Testing 4/17-5/3.” Now mind you that the test that these students take to determine whether or not they go to college, and if so what sort of college, takes 4 hours. Comprehensive exams for a Ph.D. took me three days. Somehow in the awesome logic of 2017 it came to pass that it would make some sort of “sense” to disrupt schools for two weeks to give…AZMerit.
I think we all know what tends to happen to the school year starting (in this case) 5/4.
Weeks later I had the opportunity to observe a number of focus groups held on parental choice policy. The groups were from different parts of the country, and included parents, teachers and opinion leaders. Despite the fact that the topic of the convening was never testing, everyone made their feelings on the subject clear during conversations. All groups everywhere deeply dislike the current practice of standardized testing.
I can’t emphasize the next point strongly enough: I never once heard anyone use the phrase “Common Core” or burst into a fit of conspiracy mongering. Rather what I saw repeatedly was that people feel that schooling has become overly fixated on test preparation. People have a rather strongly held belief that schooling is supposed to be more than test prep. Something has gone terribly wrong with education in their view, and they want it to stop. Across the groups I saw, the consensus seemed to be that we should drive a stake through the heart of the current system, fill the mouth with holy wafers, and then burn the sarcophagus to fine ash.
I have seen remarkably little evidence that today’s heavy-handed, standards based testing system is of much utility. There is some suggestive evidence that states that had been doing nothing on the testing front before NCLB got a modest bump in results when they started testing. They may however have received a similar bump from a system with a much lighter footprint. Moreover no less than Hanuskek and Loveless have concluded that the heavy-handed Common Core project resulted in approximately nothing in the way of improved student learning. Given that we live in a democracy, a lighter footprint system seems like a fine idea.
So here is mine:
Preserving campus level academic transparency should be the central goal of testing. The Demos would apparently be happy to sacrifice it in return for slaying the testing vampire, but it would be a terrible loss in my view. States can adopt whatever standards they want (I suggest the old Massachusetts standards) but give their students a three-hour national norm reference exam on the second to last day of school. The last day rather than last month of school can now be the write-off. Do a good job teaching the MA standards, your students will do well/show progress on the nnr test.
Some will want to have their state officials grade or otherwise label schools based on the results. Have at it-but it is worth noting that the defacto accountability system in this country has become the Greatschools rating system given that is where the eyeball traffic resides. State ratings have become little more than an obsession internal to the system. Some will want to continue on the troubled path of trying to move the number of teachers fired for low performance from 1% to 1.5%. My view is that this is an unworkable path to hold schools accountable, but if some state or locality wants to keep it up feel free.
I know some of you continue to feel motivated by the idea that standards are going to lead us to profound improvement and narrower achievement gaps. Decades into the project it is time to ask- where’s the beef? If you are willing to impose a deeply unpopular system of testing upon American families I must ask why? The burden of proof lies with you. If you (like me) would like to preserve campus level transparency I ask what is your plan? My plan is to adopt a system that is less intrusive and prescriptive and hold for dear life to campus level data-now tell me your plan. If your plan is to hold onto dear life to a system that the public abhors, I want to suggest that you need a new plan.
In my view, voting with your feet represents the most robust form of accountability by a very wide margin. I would like to have those voting decisions informed by test scores, and a great many other things including parent reviews (score another touchdown for Greatschools). Watching the focus group discussions made me realize that the United States House’s decision to enact a deeply misguided federal opt-out was not a fluke, but rather fit with the democratic sentiments of their constituents.
Opt-outs lead to nudge outs which leads to completely unreliable and thus worthless data. They will be passing at the state level soon unless transparency supporters pull their heads out of the sand. As Corwallis wrote to Clinton before the Battle of Yorktown “What is our plan? If we don’t have one, what are we doing here?”
Perhaps I’ve got this all wrong. If so, the comment section awaits.
In case you missed it, in today’s U.S. News & World Report, the inimitable Robert Pondiscio gently chides fellow school choice advocates for getting caught up in a debate over test scores, which are ancillary to the true value of school choice:
Wonky battles over research studies can be illuminating. They can also be irrelevant or premature. While [school choice] advocates are correct that the preponderance of evidence tends to favor school choice, this entire debate puts the cart before the horse. When we look to test-based evidence – and look no further – to decide whether choice “works,” we are making two rather extraordinary, unquestioned assumptions: that the sole purpose of schooling is to raise test scores, and that district schools have a place of privilege against which all other models must justify themselves.
That’s really not what choice is about. Choice exists to allow parents to educate their children in accordance with their own needs, desires and values. If diversity is a core value of yours, for example, you might seek out a school where your child can learn alongside peers from different backgrounds. If your child is a budding artist, actor or musician, the “evidence” that might persuade you is whether he or she will have the opportunity to study with a working sculptor or to pound the boards in a strong theater or dance program. If your child is an athlete, the number of state titles won by the lacrosse team or sports scholarships earned by graduates might be compelling evidence. If faith is central to your family, you will want a school that allows your child to grow and be guided by your religious beliefs. There can be no doubt that, if you are fortunate enough to select a school based on your child’s talents or interests or your family’s values and traditions, the question of whether school choice “works” has already been answered. It’s working perfectly for you.
Deciding whether or not to permit parents to choose based on test-based evidence is presumptuous. It says, in effect, that one’s values, aspirations and priorities for one’s child amount to nothing. Worse, our evidence-based debate presumes that a single, uniform school structure is and ought to be the norm, and that every departure from that system must justify itself in terms of a narrow set of outcomes that may not reflect parents’ – or society’s – priorities. Academic outcomes matter, of course, but so do civic outcomes, character development, respect for diversity and faith and myriad others.
This isn’t to say that the research on the effect of school choice on test scores is meaningless. But it has to be read and understood in the broader context. Test scores are important, but they’re far from what’s most important about exercising educational choice. As Pondiscio concludes:
School choice proponents who seek to prove that vouchers, tax credits and scholarships “work” by citing test-score-based research have allowed themselves to be lured into argument that can never be completely won. They have tacitly agreed to a reductive frame and a debate over what evidence is acceptable (test scores) and what it means to “win” (better test scores). This is roughly akin to arguing whether to shop at your neighborhood grocery store vs. Wal-Mart based on price alone. Price is important, but you may have reasons for choosing the Main Street Grocery that matter more to you than the 50 cents per pound you’d save on ground beef. Perhaps Main Street’s fresh local produce and personal service are more important to you.
If we limit the frame of this debate to academic outputs alone, every new study provides ammunition, but never a conclusion. The real debate we should be having is, “What kind of system do we want?” Answer that question first, then use evidence to improve the school designs, policies and programs we have agreed deserve public support.
In an attempt to keep viewers tuning-in after many years on the air, the sitcom Happy Days produced an episode where Fonzie jumped a Great White shark on water-skis. This episode brought the phrase “Jumping the Shark” into the pop-culture lexicon. Jumping the Shark denotes a tipping point in which something becomes absurd and suffers a noticeable decline. Arizona once was a leader in the standards and accountability movement, but those days are long gone. Days ago, Arizona lawmakers dispensed with AIMS as a graduation requirement, making the sad decline of AIMS into farce complete.
The credibility of Arizona’s K-12 testing has suffered the death of a thousand cuts. In 2004, Arizona schools faced a problem in that No Child Left Behind requires schools to be judged by ethnic subgroups, and Hispanic scores were all but certain to force many schools to be ranked failing under federal guidelines.
Instead, the state simply made AIMS much, much easier to pass. Presto-chango, Arizona Hispanic students (and others) were transformed from having been projected to fail the federal standards in almost all subjects at all grade levels in 2005 to passing almost all of them. A study by Peterson and Hess noted that Arizona’s dummy-down was the largest in the country.
Hop on over, the water’s fine!
Around that same time, the Arizona Department of Education recommended replacing the Stanford 9 exam with an Arizona version of the Terra Nova to imbed into AIMS. Happily, the new “Terra Zona” exam found that Arizona students are above the national average in every grade and in every subject tested.
One small problem: the results aren’t the least bit credible. The Arizona Department of Education recently mailed out the latest state report card, and the evidence of the farcical nature of this home-grown exam can be found in ADE’s own booklet.
On the one hand, the ADE touts the above average Terra Nova scores, but in the same booklet, it presents an analysis from the RAND Corporation showing that if you control for student demographics, Arizona’s scores on the Nation’s Report Card are average instead of rock bottom. The Nation’s Report Card- or NAEP- represents the nation’s most highly respected source of K-12 testing data.
The RAND report is entirely credible. Arizona has a far more difficult to educate student body than the national average- with a much higher percentage of low-income students, English language learners and minority students than the national average.
Controlling for demographic factors is a huge step to take. For instance, Arizona has a percentage of children eligible for a free or reduced price lunch more than twice as large as the national average. Our ratio of children who are English Language Learners is almost four times the national average.
If you pretend that Arizona has an ELL population one fourth its actual size, and about half the number of low-income children that we actually have, and some similar heroic assumptions, Arizona’s adjusted scores near the Minnesota middle instead of close to the bottom.
Arizona’s Terra Nova, however, does not control for demographics at all but somehow finds our students above the national average in every single subject without any adjustment whatsoever. If you are willing to buy that, I’ve got a bridge I’d like to sell you in Brooklyn.
Finally, AIMS has suffered what ought to be its final indignity. The legislature passed “AIMS Augmentation” in order to allow 6,000 high school seniors to graduate despite an inability to pass what at most amounts to a test of basic skills.
If you can’t pass a 10th grade level test, the original thinking went, you don’t deserve to graduate. A diploma should mean something. After delaying the graduation requirement several times, the augmentation bill has effectively killed it.
State policymakers should rethink our entire system of testing. Research shows that children who fail to learn to read in the early grades later drop out in huge numbers. Using AIMS as a graduation requirement addresses the problem at the back end. Arizona should look to Florida, which uses testing to require students to repeat grades if they don’t learn to read in the early years. Florida’s 4th grade reading scores used to scrape the bottom with Arizona, but now they greatly exceed us. Florida’s system set kids up to succeed, rather than to fail.
Parents, teachers, administrators and policymakers all require a credible and transparent system of student data. The AIMS/Terra Nova exam is not delivering. ABC eventually cancelled Happy Days and replaced it with another program. Arizona policymakers should do the same with AIMS.
Posted by matthewladner 
Jay P. Greene's Blog 