Bring out your Dead!

March 28, 2017

(Guest Post by Matthew Ladner)

Ed Next has a new forum out titled “Is Test-Based Accountability Dead?“that includes Kevin Huffman, Morgan Polikoff and our very own Jay Greene. The crux of the argument (although all the authors make points worthy of consideration) in my view lies between Greene’s political analysis of the situation (school folks + disenchanted parents >> technocrats) and Huffman’s acknowledgement of the difficulties but enduring belief that the old system worked and could yet work if only…

This is where leadership must come into play. It is imperative that governors, state chiefs of education, and other local leaders vocally advocate for the potent change shaper of accountability and convince the public of that power. I am optimistic that state education leaders are availing themselves of the chance to draft stronger, multifaceted measurement systems under ESSA. If voters and parents get behind these systems, and we implement them with fidelity, we will be able to use test results—and other measures—to dramatically improve our public schools.

In my mind this statement recalls the scene in Henry V when the Battle of Agincourt turns decisively against the French. The French noble Orleans exclaims:

We are enough yet living in the field
To smother up the English in our throngs,
If any order might be thought upon.
Go see the play if you need a refresher on how that plan works out for the French. The Huffman piece merits careful examination regardless, as it is actually a good example of how K-12 policy discussions actually occur. Huffman presents evidence on national NAEP scores and attributes a positive trend to testing. Huffman then makes the case that test based accountability systems especially drove improvement in the District of Columbia and Tennessee. Ergo test based systems can drive public school improvement, therefore we must summon up the blood and make them work.

Huffman did not invent this type of reasoning. In the beginning there was Massachusetts-and a nation that strangely lacked curiosity about large package of reforms passed in 1993-It was the testing! Next came North Carolina and Texas to justify NCLB. And then Finland (?!?) and then Florida. Huffman proposes to add DC and Tennessee to the list to make the case for test based improvement strategies “I’m not dead yet!”

Properly trained social scientists of course will pull their hair out and mutter “wacky sassafras!” at such violence being done to the proper ascribing of causality. Given that this is how such conversations take place, we can at a minimum look under the hood of such claims-do they square with more rigorous empirical evidence? Has there been any effort made to explore alternative explanations or look at subgroup trends? Usually not-most often this is “tell me a story and let’s run with it!” So let’s turn some (minimal) attention towards the DC and Tennessee examples.

As has been discussed previously here at Jayblog, DC represents a complex case involving multiple powerful factors other than test based accountability- including massive gentrification and a powerful amount of parental choice. The only positive trend above and beyond the national average for low-income children in DC locates itself in the charter sector. Ergo it is hard to make the case that DC’s testing system is doing something especially impressive in my book.

So let’s consider Tennessee, and a non-tested area of Science under the theory that character is about what you do when you think no one is looking:


Tennessee fares quite well in progress on NAEP Science. Tennessee also scored near the top on 4th to 8th grade Math and Reading cohort gains between 2011 and 2015. Unlike DC, neither a massive dose of gentrification or parental choice has yet made it to Tennessee, so let’s acknowledge that Tennessee students have displayed positive gains on the NAEP, including in areas outside of accountability testing. Tennessee seems to be a case worthy of examination (i.e. closer examination than I can provide in my pajamas) but let’s assume for the moment that Huffman is correct that the test based policies have been driving Tennessee improvement. What then?

Greene’s constituency politics analysis still likely prevails in the medium term. The lack of a “test my child more!” constituency added to the overt hostility of public school employees almost certainly places this strategy on borrowed time. Now if someone made me the Baron of Arizona and the federal government sent down a diktat that we were going to do testings and ratings, I would happily study what TN did and consider imposing it. “You have to have some kind of tests and ratings, why not ones that seem to have driven improvements?” would be a phrase likely to fall from Baron von Arizona’s lips. We however do not live in an imperial system, but rather in a democracy. This makes securing the consent of the governed necessary. Even in an imperial system, the peasants will stage frequent uprisings if and when they are sufficiently motivated.

The title of the Ed Next forum pretty much answers its own question. Whether or not such improvement strategies are “dead” we have reason to suspect have practical limits and constraints and that we have likely hit the ceiling.





The Greene-Polikoff Wager: An Update

July 12, 2016


(Guest Post by Jason Bedrick)

In 2014, Jay made a wager with education policy wonk Morgan Polikoff regarding how many states would, after 10 years, still be a part of Common Core (defined as having “shared standards with shared high stakes tests-even if split between two tests”). The loser owes the winner a cold beer.

At the time the wager was made, the states had almost unanimously adopted Common Core so Morgan was confident but Jay thought political support for CCSS was a mile wide but an inch deep.

Morgan noted that “At last count, 1 state out of 45 has repealed the standards.”  I responded: “I’m sure gay marriage opponents felt similarly triumphant in 2004. How many states have effectively implemented Common Core?” […]

According to Heritage’s count, 15 states have already refused to join Common Core, paused implementation, or downgraded or withdrawn from participation in national tests.  I just need all of these states to continue toward withdrawal from Common Core and 11 more to join them over the next ten years.  I like my chances.

Just a few months later, Jay posted an update:

With the withdrawal of Iowa this week from the Smarter Balanced testing group, there are only 26 states that plan to use one of the two national tests to assess their students during the 2014-15 school year.  It’s true that 35 states remain part of the two testing consortia and some of the 9 states that have delayed implementation of the common tests may begin using one of them in the next few years.  But it’s safe to say that several of those 9 delayed start states will never follow through.  And some of the 26 states actually using a common test in 2015 are already making noises about withdrawing.  See for example reports coming out of Wisconsin and South Carolina.
If one more state that is currently using one of the common tests drops it than decides to follow through on implementation, I will have won the wager.  And we have more than 9 years to see that happen.

So how is the bet looking two years later? Well put it this way: Jay can probably already taste that beer. From Education Next:

State participation in the consortia declined just as implementation of the new standards and tests was set to begin. The pace of withdrawals quickened over time, particularly for PARCC, which five or six states left every year between 2013 and 2015 (see Figure 1). As of May 2016, just six states planned to implement the PARCC-designed assessment in the 2016-17 academic year. SBAC also faced attrition but fared better and still retains 14 states that plan to use the full test. (That figure includes Iowa, where a legislative task force has overwhelmingly recommended the SBAC assessment, though as of early 2016 state officials had yet to formally accept the recommendation.) By early 2016, 38 states had left one or both consortia, short-circuiting the state-by-state comparability that the tests were designed to deliver (see Figure 2).


“Oh, how the mighty have fallen!”


Common Core in retreat.

Note that these charts do not reflect the fact that Illinois has just replaced PARCC with a “revamped” version of the SAT for its high school students. Students in grades 3-8 will still take the PARCC, so perhaps Illinois should count as half a state for purposes of the Greene-Polikoff Wager.

Of course, it’s always possible that the remaining CCSS states will work out the kinks, opposition will fade as people get used to the testing regime, and then the political winds will shift again and states will re-enter one of the CCSS testing consortia. A lot can happen in eight years. But there is no denying that Jay was prescient in his read of the situation.

%d bloggers like this: