Bring out your Dead!

March 28, 2017

(Guest Post by Matthew Ladner)

Ed Next has a new forum out titled “Is Test-Based Accountability Dead?“that includes Kevin Huffman, Morgan Polikoff and our very own Jay Greene. The crux of the argument (although all the authors make points worthy of consideration) in my view lies between Greene’s political analysis of the situation (school folks + disenchanted parents >> technocrats) and Huffman’s acknowledgement of the difficulties but enduring belief that the old system worked and could yet work if only…

This is where leadership must come into play. It is imperative that governors, state chiefs of education, and other local leaders vocally advocate for the potent change shaper of accountability and convince the public of that power. I am optimistic that state education leaders are availing themselves of the chance to draft stronger, multifaceted measurement systems under ESSA. If voters and parents get behind these systems, and we implement them with fidelity, we will be able to use test results—and other measures—to dramatically improve our public schools.

In my mind this statement recalls the scene in Henry V when the Battle of Agincourt turns decisively against the French. The French noble Orleans exclaims:

We are enough yet living in the field
To smother up the English in our throngs,
If any order might be thought upon.
Go see the play if you need a refresher on how that plan works out for the French. The Huffman piece merits careful examination regardless, as it is actually a good example of how K-12 policy discussions actually occur. Huffman presents evidence on national NAEP scores and attributes a positive trend to testing. Huffman then makes the case that test based accountability systems especially drove improvement in the District of Columbia and Tennessee. Ergo test based systems can drive public school improvement, therefore we must summon up the blood and make them work.

Huffman did not invent this type of reasoning. In the beginning there was Massachusetts-and a nation that strangely lacked curiosity about large package of reforms passed in 1993-It was the testing! Next came North Carolina and Texas to justify NCLB. And then Finland (?!?) and then Florida. Huffman proposes to add DC and Tennessee to the list to make the case for test based improvement strategies “I’m not dead yet!”

Properly trained social scientists of course will pull their hair out and mutter “wacky sassafras!” at such violence being done to the proper ascribing of causality. Given that this is how such conversations take place, we can at a minimum look under the hood of such claims-do they square with more rigorous empirical evidence? Has there been any effort made to explore alternative explanations or look at subgroup trends? Usually not-most often this is “tell me a story and let’s run with it!” So let’s turn some (minimal) attention towards the DC and Tennessee examples.

As has been discussed previously here at Jayblog, DC represents a complex case involving multiple powerful factors other than test based accountability- including massive gentrification and a powerful amount of parental choice. The only positive trend above and beyond the national average for low-income children in DC locates itself in the charter sector. Ergo it is hard to make the case that DC’s testing system is doing something especially impressive in my book.

So let’s consider Tennessee, and a non-tested area of Science under the theory that character is about what you do when you think no one is looking:


Tennessee fares quite well in progress on NAEP Science. Tennessee also scored near the top on 4th to 8th grade Math and Reading cohort gains between 2011 and 2015. Unlike DC, neither a massive dose of gentrification or parental choice has yet made it to Tennessee, so let’s acknowledge that Tennessee students have displayed positive gains on the NAEP, including in areas outside of accountability testing. Tennessee seems to be a case worthy of examination (i.e. closer examination than I can provide in my pajamas) but let’s assume for the moment that Huffman is correct that the test based policies have been driving Tennessee improvement. What then?

Greene’s constituency politics analysis still likely prevails in the medium term. The lack of a “test my child more!” constituency added to the overt hostility of public school employees almost certainly places this strategy on borrowed time. Now if someone made me the Baron of Arizona and the federal government sent down a diktat that we were going to do testings and ratings, I would happily study what TN did and consider imposing it. “You have to have some kind of tests and ratings, why not ones that seem to have driven improvements?” would be a phrase likely to fall from Baron von Arizona’s lips. We however do not live in an imperial system, but rather in a democracy. This makes securing the consent of the governed necessary. Even in an imperial system, the peasants will stage frequent uprisings if and when they are sufficiently motivated.

The title of the Ed Next forum pretty much answers its own question. Whether or not such improvement strategies are “dead” we have reason to suspect have practical limits and constraints and that we have likely hit the ceiling.





The Accidental Superpower

March 27, 2017

(Guest Post by Matthew Ladner)

Peter Zeihan manages to build incredibly optimistic and pessimistic visions of the immediate future into a single book. Let’s start with the good news: Zeihan builds a compelling case that the United States has been and will continue to be the most incredibly advantaged nation on the planet. This owes both to geography and other factors. In terms of geography, the United States contains both the largest navigable river system (the Mississippi and six tributaries) that delightfully sits right next to the world’s largest bloc of temperate zone arable land (the Midwest). Add to that a presence on both the Atlantic and Pacific, an abundance of world-class harbors and then a free-wheeling economic culture in which the industrial revolution took root and flourished. Add all of this up, and the United States was in a position to make an incredibly generous offer to most of the rest of the planet towards the end of World War II. The United States offered to secure global maritime shipping and grant access to sell in American market in return for approximately nothing, with approximately nothing quickly morphing into “joining the global military alliance aimed at containing the Soviet Union.”

All in all Pax Americana proved to be a smashing success- the world recovered and prospered. Global poverty dropped, and the number of nations wanting in on the deal steadily grew. Eventually even China wanted in, and then in the early 1990s the Soviet Union collapsed. The end of history!

Not so fast…the story continues.

The end of the Cold War obviously raised questions regarding the United States continuing to bear the imperial burden. Zeihan notes that the United States is actually one of the world’s least internationalized economies- selling and buying in large volumes, but as a relatively small portion of the total economy. Outside the odd BMW most everything you want is available here, especially if you consider Canada and Mexico “here.” A continuing motivation for the United States continuing Pax Americana however had been the importation of oil. Somewhere along the way the United States had switched from being the world’s largest oil exporter to the largest importer. The United States thus had an incentive to secure oil supplies, primarily in the Middle East.

Two things intrude into this state of affairs. First, the United States created entitlement programs that are badly underfunded vis-a-vis the ongoing retirement of the Baby Boom generation, likely impacting both our ability and willingness to bear the imperial burden. Second, the advent of fracking has put the United States in a position to feel rather indifferent about Persian Gulf Oil. Zeihan’s conclusion of the optimistic portion of his book is that while tough days lay ahead, the United States is going to get through it in the end smelling like a rose.

The rest of the world not so much.

The remainder of the book looks at a number of different countries and their challenges, and how they might fare outside of Pax Americana. It’s not pretty. No countries have all of America’s advantages and many countries have age demography challenges more severe than the United States- including Canada, China, most of Western Europe, Japan and Russia. Other regions, like the Middle East, have already begun wars that won’t likely stop anytime soon. If you posit a return to a dog-eats-dog world that existed for most of human history before the imposition of Pax Americana (Zeihan does) the possibilities for armed conflicts are almost limitless. The United States, a defacto continental sized island of prosperity, need not involve itself.

A few years ago I would have thought Zeihan’s global pessimism overwrought. Now I can hope that this is the case, but I am not so sure. A few years ago I would have also thought that it was highly unlikely that the United States would have drawn a “red-line” in a Syrian civil war only to change our minds, and then watch a massive refugee crisis strain European unity and contribute to the departure of the United Kingdom. Or that Russia would start the process of reacquiring their Soviet era buffer states. Or that the United States would effectively abandon the interests of their long-time Middle Eastern allies like Saudi Arabia and Israel in favor of Iran.

All of these things however happened, and it seems to be a good bet that there is more global chaos on the way. Whether or not it unfolds as Zeihan foresaw writing back in 2014 or not remains highly speculative, and there are some more optimistic scenarios than a fast return to international anarchy. Since 2014, American oilmen decisively defeated the OPEC attempt to crush fracking by adopting new technologies and practices that decisively lowered the extraction costs of American tight oil. It’s not going anywhere.

Written in 2014, many things about the Accidental Superpower seem to foreshadow Trump. Trump claims that the United States has signed a huge number of “stupid” deals and that he will strike better deals. While I’m not saying I agree with this sentiment at all, Zeihan’s book does add a new perspective, at least for me. It also makes the choice of the former CEO of the world’s largest oil company to lead American diplomacy even more interesting.

I hope that Zeihan’s global pessimism proves over done. Take the Trump administration’s reading the riot act to our NATO allies regarding their defense spending for instance. If the hobbits of euroshire don’t toughen up pretty quickly, they are going to find themselves unprepared to deal with the legions of orcs heading their way as a fading but desperate Russia grasps at lost buffer zones. Of course this is easier said than done on a continent of stagnant growth, debt crisis and aging populations but failing to deter Russia could prove far more costly. Half of the American Baby Boomers will have reached retirement age by 2020, the United States may soon as Zeihan postulates view this as someone else’s problem. A German army with 63,000 people in uniform does not seem likely to get the deterrence job done, and if you were a Baltic state or Poland you should have your alarm set at “Red Alert.” If the United States can avoid a precipitous cut off, it may be possible for the Europeans to deter aggression on their own.

Similar transitions may be possible elsewhere, but a Saudi-Iranian barely cold war seems already locked in to consume the Middle East for a long time to come. The Congress lifted the ban on oil exports in 2015, and the United States already exports a million barrels of oil per day. Reading Zeihan raises the question of just how long the United States Navy will be protecting the shipping of foreign firms in direct competition with American firms for market share. Indefinitely? Friendly tip- if I were Europe or Japan I would start signing long-term oil and gas contracts with American firms like yesterday. The United States is going to have an interest in protecting its own shipping even in the Zeihan return to anarchy scenario, and other suppliers lack the political and military stability of a Texas or North Dakota.




The Facts about School Choice and Segregation

March 23, 2017

A Century Foundation researcher searching for evidence.

(Guest Post by Jason Bedrick)

ICYMI, our JPG blogger buddy Greg Forster has a new piece up at Education Next debunking a report from the Century Foundation that claims — based on mere conjecture and an ostrich-like ability to bury one’s head in the sand regarding the research — that school choice supposedly increases ethnic segregation.

The Century Foundation has published a report by Halley Potter that claims private school choice will increase ethnic segregation in schools. Although the text of the report constantly invokes words like “evidence,” “studies” and “data,” its conclusions are actually defended almost entirely by appeal to a lengthy recitation of hypothetical, ideological speculation. The report’s actual engagement with empirical research is as scanty as it is misleading. A real review of the evidence shows that private school choice has never been found to increase segregation and often seems to have provided a more integrated classroom experience.

There are a number of serious methodological challenges involved in empirical research on how education policies affect ethnic segregation. I wrote about them at some length in a report for EdChoice a while back. For example, some data don’t permit causal conclusions; some methods of comparison are unfair because they compare elementary grades to secondary grades inappropriately. Reviewing all of the empirical research on school choice last year, I found that 10 studies had been conducted that examine the relationship between school choice and ethnic segregation in some respect. Some are causal, some descriptive; all shed some light on the question. Nine of those studies found that school choice provided a more integrated classroom experience, one found no visible difference, and no empirical study had ever found that a school choice program made ethnic segregation worse.

That is the empirical evidence. Nine out of the 10 studies that have been conducted report positive findings on the actual, real-world impact of school choice programs when it comes to ethnic segregation.

The Century Foundation report mostly ignores the evidence, giving a distorted take on just two of the empirical studies on the effects of school choice on segregation. As Greg notes:

If you dig very, very deep into the report, you do eventually find a discussion of empirical studies. But this doesn’t mean the report gets much better, for Potter examines only two of the 10 studies that exist – and she has described them in a misleading way.

Looking at longitudinal studies in Milwaukee and Louisiana, she describes them in a way that will leave the impression that the results were negative for school choice: “In both cases, programs were used primarily by black students and generally did not exacerbate segregation in public schools; however, students using vouchers did not gain access to integrated private schools, and segregation in private schools actually increased.”

Now, even that misleading description would be enough to call into question the huge mountain of hypothetical, ideological speculation that occupies the overwhelming majority of Potter’s report. However, a more precise description of these two studies would look even worse for Potter, because it would look good for school choice.

The Milwaukee study found the voucher program made no visible difference to segregation, at least during the period under observation. It is the only such study ever to find no visible difference. Other studies in Milwaukee using different methods have found more encouraging results, though because of methodological restrictions, none of these studies can be considered a final word. The longitudinal study’s null finding is not as encouraging as a positive finding would have been, but the nightmare world of increasing school segregation promised by Potter’s lengthy speculations apparently did not come to pass in Milwaukee.

As usual with Greg’s work, I recommend reading the whole thing.

Sadly, the Atlantic ran an entirely uncritical piece parroting the Century Foundation’s “findings.” I’ve reached out to both the Atlantic and the Century Foundation yesterday–and again today–to share with them all the evidence that they ignored, but they have thus far continued to ignore both me and the evidence. I’ll update you if that changes.

NAEP Cohort Gains for Students with Disabilities, 2011 to 2015

March 22, 2017

(Guest Post by Matthew Ladner)

Which states had the most success teaching math and reading to their students between grades 4 and 8 on the most recent NAEP exams?

Quick (really boring) caveat section- tracking NAEP cohort gains gets more inexact due to the larger standard error of estimate when examining subgroup scores- which exists in both the 4th and 8th grade scores. Accordingly, don’t get too excited if your arch-rival Idaho edges you out by a point, because if we knew a true population score rather than sampled estimate it could easily be the other way around. It is more appropriate to look to see whether the state nearest and dearest to your heart landed near the top, or near the bottom of the list than to obsess about their ranking. Moreover, given the possible role sampling error, it is probably best to consider looking at the math and reading scores together. If a state rocks one of the tests and performs not so great on the other, you may have gotten a lucky sampling error bounce on one test, or a bad bounce on the other, or maybe your state is just better at teaching one of these subjects than the other. You would have to consult other sources of data to figure that out. Conversely, if a state does well on both reading and math it is unlikely that sampling error is driving the result in both cases, assuming randomly distributed error.

Ok so on with the show:

Top ranked Hawaii had gains more than twice as large as bottom ranked Maryland. Note that both of the bottom states (MD and KY) have a noted prior history of high exclusion rates for special needs children in NAEP. If they excluded a high percentage of kids in 2011 4th grade testing but not in 2015 8th grade testing, an appearance of catastrophe could sneak in. I am happy to note that my patch of cactus lies near the top. What about math?

Note that the overall gains for math are smaller than for reading. Also Hawaii is again at the top, and Arizona is near the top. Again Maryland appears to have taught a little more than one year’s worth of math to special needs students in four years, but I suspect that strange things are afoot at the MD Circle K on that one. I could dig into NAEP pdf files to check on exclusion rates but alas it is time for me to get out of my pajamas, so I will leave that to whatever you call people from Maryland. Marylanders?

Justice, Equal Opportunity, Diversity and School Choice

March 21, 2017


(Guest post by Greg Forster)

This weekend I was delighted to participate in an hour-long school choice debate on Moody radio’s coast to coast network. The experience left me all the more convinced that choice advocates must continue to de-emphasize the rhetoric of markets and competition, and emphasize instead justice, equal opportunity, diversity and freedom. In that order.

The proponent of the other side had come armed with empty, superficial anti-market talking points. Her argument against choice was basically “we want justice, equal opportunity and diversity, not markets and competition.” So when I opened my case with “we want justice, equal opportunity, diversity and freedom, and here’s how school choice delivers them,” and didn’t use the words market or competition, she was flummoxed.

Her superficial talking points would have been highly effective if I had said “we want justice, equal opportunity, diversity and freedom so we ought to embrace competition and markets.” That is, sadly, because reason and logic are not the only forces in public debate. Her strategy (consciously or unconsciously) seems to have been to use certain trigger words and phrases to prompt emotional responses in the audience – responses unrelated to logic. This is, as JPGB readers know, the nearly universal strategy of choice opponents.

If we stop using the words that allow them to do this to us, we take their toy away.

Of course the quesiton of why choice improves public schools did come up, and here it was necessary to make the point that choice prevents schools from taking students for granted. The body of empirical studies on choice and what they find also came up a number of times. One can make these points without 1) making them the be-all and end-all, or 2) using the specific trigger words that allow the other side to work their emotional trickery.

A final point: personal experience, unfortunately, trumps data. I talked about having visited several private schools in Milwaukee whose existence depends on the voucher program, and the amazing things these particular schools are doing. Then I mentioned the studies finding that choice is impoving education in Milwaukee. I played this card again when a caller raised the inevitable racist talking point “parents in the suburbs are involved with their children’s education but people in those neighborhoods aren’t.” Instead of saying “the data show urban parents make good choices for their kids; that’s a stereotype we shouldn’t be spreading” I said “my experience in urban neighborhoods leads me to believe they care about their children as much as other parents; that’s a stereotype we shouldn’t be spreading.” And then I tried to squeeze in a mention of the data.

Of course if there were a clash between my experience and the data, I’d have to go with the data. But we have to limit our appeals to data when speaking in public. In general, we should present opinions based on experience and then briefly validate them with appeals to data.

Pssst…NACSA…Size DOES matter

March 16, 2017

(Guest Post by Matthew Ladner)

I was looking at NACSA’s 2014 ratings for state charter school laws, as these would have been the most relevant before the most recent NAEP. Hailing from the out west, I noticed that Nevada had a score (26) nearly three times the score of both Arizona (9) and Colorado (9).

Longtime readers of JPGB of course will be aware that charter school students in both Arizona and Colorado rocked the 2015 NAEP exams like nobody’s business. Nevada on the other hand has had a very difficult experience with charter schooling. A decade ago or so ago I wrote a study for the Nevada Policy Research Institute that basically concluded that Nevada was missing out on high quality schools and the opportunity to relieve overcrowding in the public schools, and so should follow the example of their Arizona neighbor and get in the game. I wrote one of the earliest Jayblog posts on the subject called Fear and Loathing in Carson City:

Nevada, by comparison, has been hesitant in expanding parental options. In the five states surround Nevada (Arizona, California, Idaho, Oregon and Utah) and these states have 482, 710, 30, 81 and 60 charter schools respectively, collectively educating hundreds of thousands of students. With only 22 charter schools, Nevada is the tortoise of the region.

On November 30 of 2007, the Nevada Board of Education voted 8-0 to impose a moratorium on the approval of new charter schools. Board members told the press that the freeze was necessary because the state Education Department is being “overwhelmed” by 11 charter applications.

Arizona’s State Board for Charter Schools oversees 482 Arizona charter schools with a staff of 8. Nevada’s board overseeing cosmetology currently has 14 full-time employees.

The fear and loathing in this case referred to the sad fact that many in Carson City obviously feared and loathed charter schools. Imagine then my surprise to see a national charter school organization rank the same law a few years later as nearly three times higher than laws in nearby Western states that had produced far more opportunities for kids. Out of curiosity, I decided to check the 2015 NAEP scores for Nevada charter students.

NAEP yielded no information: the Nevada charter sector remains too small to reliably sample.

Now to give you some perspective on this, NAEP lists 5.6% of students in Nevada as Asians, and the data explorer will give you a number for male Asian students taking the 8th grade math exam in 2015 (Nevada’s Asian males did well on 8th grade math btw) but nothing for charter students of all sorts. NAEP cannot reliably sample charter students in Nevada because in 2013-14 they still only had 34 charter schools according to the National Alliance for Public Charter Schools. Those numbers for Arizona and Colorado btw were 605 and 197 respectively. Presumably the Nevada law would eventually like to have some “charter schools” result from their “charter school” law?

Next I decided to check out the 2015 rankings. The top 10 state charter sectors did not exactly cover themselves in 2015 NAEP glory. Indiana and Nevada tied for first place in their rankings, but neither state would yield NAEP estimates for performance. Alas Indiana’s 75 charter schools were not yet getting the job done in terms of scaling into the NAEP. The Nevada law passed in 1997 and Indiana in 2001. Hopefully this will get better in the future but for now:

The third ranked state (Ohio) has scores reported but those scores are consistently low, but Ohio’s NACSA score only recently increased. NACSA gets a Mulligan on that one, but continuing down the top 10 list however one fails to gain confidence regarding Ohio’s future prospects. Alabama ranks fourth, but is a relative newcomer to charter schooling so also cannot report scores. The Texas charter law ranks fifth on the NACSA 2015 list but has charter scores on NAEP that have yet to impress. I am a Texan who became an Arizonan and I would not swap charter laws with Texas even if they threw a shale formation into the bargain. In the vast majority of Texas schooling is still “take it or leave it” from the districts, whereas in Arizona even our Beverly Hills type districts are anxious for you to open enroll from outside the boundaries.

The same applies to seventh ranked Minnesota- the nation’s oldest charter law. You can get NAEP scores in MN, and I can’t thank them enough for inventing charter schools, but the feeling I get from MN charters is that they are safely contained rather than dynamic. The last three states in NACSA’s top 10-Mississippi, Missouri and South Carolina- all lack enough charter students to meet the minimum NAEP reporting requirements as well. Louisiana comes in 10th.

If you are scoring at home, NACSA’s top 10 is composed of six states with charter sectors that can charitably be described as “wee-tiny” and three others that have yet to flourish like an Arizona or Colorado, and then Louisiana. Tenth rated Louisiana’s charter sector does well in the NAEP, so bully for them, but they obviously have a unique charter history. Notably absent from NACSA’s top 10- very healthy charter sectors like Florida and Washington D.C.

Not to jump to any premature conclusions, but it appears that NACSA’s rating may be overly concerned with bureaucratic compliance rather than performance- either of the academic sort, or the “actually produces charter schools” kind. Arizona and Colorado produced hundreds of charter schools with NAEP scores that compare favorably to New Hampshire (and sometimes Massachusetts) with a 9 score from NACSA. I for one would like to see what they could do with, say, a five score from NACSA. What’s that you say? Three? Ok fine let’s try it out if you insist!

Now maybe I am missing something here, and that is why the comment section is open. I’ll leave you with the following question to consider- Nevada public schools suffer from catastrophic overcrowding. Public education in Las Vegas for a great many students involves sitting in a portable trailer being taught by yet another long-term substitute teacher. Clark County starts each school year with thousands of open teaching spots they are desperate to fill, and their officials told the New York Times they could build 23 new elementary campuses and they would be overcrowded on day one. The United States Census Bureau sees no end in sight for enrollment growth.

Please tell me why any Nevadan in their right mind would prefer Nevada’s charter law to what we see in Arizona and Colorado. I mean maybe scale and great results is not everyone’s cup of tea, but any port in a storm right?


We are but warriors for the working day, but our hearts are in the trim

March 14, 2017

(Guest Post by Matthew Ladner)

The Arizona Republic was kind enough to run the below letter to the editor from yours truly this morning in response to this editorial. If you are feeling the least bit skeptical, feel free to look these numbers up for yourself. The Republic’s editorial claims that now is not the time to expand parental choice because district schools are vulnerable. My claim is that Arizona district schools have never performed at a higher level than now and that we should in the immortal words of Darrel K. Royal “dance with the one that brung ya” which is to say stick with the strategies that brought success. Letter follows:

On the most recent Nation’s Report Card, Arizona 8th graders tied the state of Maryland in math, and outscored many states including Rhode Island, Delaware and North Carolina. These states spend far more per pupil than Arizona. None of these states has a majority-minority student population (Arizona does) but fortunately our students didn’t get the memo that they weren’t supposed to win. Instead they have been leading the nation in academic gains.

Arizona’s charter schools get still less money overall but scored higher than the statewide averages of 49 states on the same test. Arizona charter schools educate a majority-minority student population, but scored a single point lower than the highly funded and demographically advantaged Massachusetts-the nation’s long-time state academic champion. Again, the “you are supposed to lose” memo apparently went to Arizona’s spam folder, and our students and educators achieved an unprecedented academic triumph.

Arizona is never going to win a spending contest, but that is not the purpose of our investment. Our goal must be to maximize opportunity, not spending.