The Next Accountability: What Do We Want from Schools?

August 3, 2016


(Guest post by Greg Forster)

Today the newly renamed EdChoice launches Part 1 of my series of articles on The Next Accoutability, previewed with an introduction a few weeks ago.

I argue that the education reform coalition is coming apart because we don’t agree about what we want from schools:

The movement was well served in many ways by its various edifying impulses: to “close the achievement gap,” to “put parents in charge,” etc. But it has been haunted for decades by a growing awareness that these moral impulses do not always cohere easily.

The question, “What do we do if putting parents in charge doesn’t, by itself, close the achievement gap?” has been debated at every education reform conference I’ve attended. Such debates were lively and interesting intellectual exercises, so long as not much hung on them.

We lack consensus on what we want from schools because in a pluralistic society with religious freedom, we want to respect diverse opinions about the highest questions in life. But this leaves us with an incoherent education policy:

Our freedom to disagree about transcendent things does not mean that public policy can escape the responsibility to ask what is good, true and beautiful. In fact, the very assertion that it is good to have the freedom to disagree about transcendent things is itself an assertion about what is good, i.e. about transcendent things.

The challenge of pluralism is also an opportunity for us to discover a fresh vision of human potential that embraces the freedom to disagree about the highest things:

School accountability should be grounded in an understanding of human potential aimed at building up free communities, open to pluralism under the rule of law and respect for human rights, where people achieve and appreciate the good, the true and the beautiful in the midst of their differences over those very things.

I outline how we can understand educational goals for the head, the hands and the heart in ways that point toward the possibility of coherence in a pluralistic society.

Coming in two weeks: Part 2, looking at how teachers and schools actually carry out the task of educating students in the midst of our uncertainty about the highest goals of education. It is here, I will contend, that we will find clues to how we can hold schools accountable more effectively. Stay tuned!

As always, your comments and feedback are greatly appreciated.


Expert Ratings- A Cautionary Tale

July 15, 2016

GB Screenshot

(Guest Post by Matthew Ladner)

A few months ago, I was on a panel discussing the incorporation of user reviews into the K-12 space for things like digital courses and ESAs. One of my fellow panelists expressed skepticism about the entire project until I noted that Rotten Tomatoes allows both “expert” and “user” reviews of movies. With this, he expressed contentment so long as the experts got their own ratings for separate consideration.

It has long been my experience using Rotten Tomatoes however that the user ratings tended to be much more reliable than the critic reviews. Last night I got a double barrel reminder of this when I allowed a 74% fresh critic rating convince me to go see Ghostbusters before the audience score came in (the studio released the film yesterday).

The 44% rating of the audience is incredibly generous. The 74% critic rating reveals some sort of deep divorce from reality. This movie is about as terrible as:

Star Trek V

Apparently 2o+% of people are just unwilling to admit that they paid to see a bad movie. What more proof? The critics nailed this one:

Highlander 2

Sometimes both the critics and the audience gets it wrong, but the critics get it more wrong. This would be the case with Ghostbusters 2016 and:

Jones Sometimes the critics and audience agree on a stinker:


So in the end, I’m happy to have “expert” reviews included, but if 74% of them thought that the hot mess I saw last night was a good movie, it says something important about relying solely on experts. Like everything else in life “experts” are not to be trusted.

Bending Spoons with Andy Smarick

July 11, 2016

(Guest Post by Matthew Ladner)

Following up on Greg’s post on Andy Smarick’s great post on accountability and how our current systems are based upon premises on their way to being out of date. It brings to mind this scene in the Matrix:


You have to understand that most of these people are not ready to be unplugged, and many of them are so inured, so hopelessly dependent on the system, that they will fight to protect it.

Andy took the red pill- welcome aboard the Nebuchadnezzar!

“Public” Is Not “Government”

July 8, 2016


(Guest post by Greg Forster)

Andy Smarick is starting to have the most important realization about accountability – that the government school monopoly has corrupted not only our systems of accountability but even our conception of what accountability is. There is no need for a single, uniform accountability system in each geographic zone, and there is no need for accountability to be government-controlled.

Go read his post. Then remember that you read it here first. Not only the idea of families holding schools accountable but the idea that many other kinds of social organizations (think tanks, churches, civic groups, etc.) could help them do so by building family-driven structures of accountability.

My only complaint is that the title on his post reinforces the same error he is awakening from. The title is “Public Accountability vs. Consumer Accountability.” Since none of the language in the post itself reflects the extreme myopia we see in that title, I’m betting the title was composed by the smug technocrats at Fordham.

Welcome aboard, Andy! Soon enough you’ll realize these insights aren’t going to find an easy home at Fordham, but we’ll be here to welcome you as you pursue them.

Update: Ironically, I mispelled “error”!

The Next Accountability

July 6, 2016


(Guest post by Greg Forster)

Today the Friedman Foundation carries the introduction to my forthcoming series, The Next Accountability. In this series I will make the case that the education reform movement is being pulled apart by differing visions of “accountability”:

Our forefathers built the education reform movement on a foundation that all reformers shared: We need to hold schools accountable, so they’ll give kids the education we want them to get. Now we’re discovering cracks in the foundation. It turns out we don’t agree on what we want, or on how we get schools to deliver it.

These differing visions of accountablity are really differing visions of what education is for – which are in turn differing visions of what constitutes a good human life:

A few of us, however, think that all this technocracy is precisely what we have been fighting against all along. It is essentially an extension of the old regime’s philosophy: We’re the education experts, and we know best! It’s just as impersonal and unresponsive to the real needs of real people as the blob. It’s as if we defeated the Soviet Union, and then celebrated our victory by imposing communism on Western Europe and North America.

For this reason, supporters of choice are going to have to get beyond talking points and canned rhetoric about “markets” and “competition”:

America needs to rethink what we really want from schools. Whether of the old or new variety, technocratic systems fail not only because they can be manipulated by greedy and incompetent people or because they lack sufficient information about client preferences (although these things are also worth remembering)—technocracy fails more importantly because it is based on a wrong understanding of what education is for.

Knowing what we want requires us to reawaken to who we are. All the great thinkers who have cast big visions for education, from Plato and Aristotle to Aquinas and Locke to Rousseau and Dewey, agreed that knowing something about what it means to educate children ultimately requires us to know something about what it means to be human. If opponents of technocracy can’t say something about that, all our rhetoric about markets and competition is chaff in the wind.

In the forthcoming articles in this series, I will lay out a vision of what we want from schools, and how we can create new approaches to accountability that will help us get what we want, without falling into the technocratic trap.

Follow the link and read the introduction to get a sneak preview of what that will entail.

Championing a different vision of accountability may increase the current conflicts in the education reform movement rather than decrease them. Technocracy is rooted in a vision of what education is for – of what is good for human beings – that many people in the education reform movement really believe in. We shouldn’t expect them to give it up without a fight.

But, as I will argue, a fresh vision can attract new allies. In many cases they will be more powerful allies than our current technocratic coalition partners, who are increasingly dismissive of us and our concerns anyway. And if we’re not willing to fight technocracy, even when it comes from our friends (in many cases former friends, who turned on us as soon as they thought they could) what was the fight against the blob all about in the first place?

It is only through that kind of conflict that really historic progress gets made. Dr. Martin Luther King Jr. was not interested in renegotiating the terms of segregation. He came not to reform it but to exterminate it. We too must fight not for a renegotiation of terms with the technocratic beast, but for its end.

The remainder of this series will launch shortly after Friedman Legacy Day on July 29, when the Friedman Foundation will be making big announcements about its future. One thing I can promise you: we will never leave behind the mission to fight for justice and freedom that has always animated the foundation, which is Milton’s greatest legacy to us.

In the meantime, I welcome your feedback and look forward to the next stage of the fight!

Liberty and Justice for All

June 1, 2016

(Guest Post by Matthew Ladner)

Kingsland writes a hilarious post on five theories of change in K-12 philanthropy.  Over on the Ed Next podcast, Paul Peterson explains why he believes top-down command and control improvement strategies have jumped the shark and calls for a renewed focus on choice as an improvement strategy.


Overregulation Is All You Need: A Response to Paul Bruno

February 29, 2016


(Guest Post by Jason Bedrick)

Over at the Brookings Institution’s education blog, Paul Bruno offers a thoughtful critique of  Overregulation Theory (OT), the idea that government regulations on school choice programs can undermine their positive effects. Bruno argues that although OT is “one of the most plausible explanations” of the negative results that two studies of Louisiana’s voucher program recently found, it is not “entirely consistent with the available evidence” and “does not by itself explain substantial negative effects from vouchers.”

I agree with Bruno–and have stated repeatedly–that the studies’ findings do not conclusively prove OT. That said, I believe both that OT is consistent with the available evidence and that it could explain the substantial negative effects (though I think it’s likely there are other factors at play as well). I’ll explain why below, but first, a shameless plug:

On Friday, March 4th at noon, the Cato Institute will be hosting a debate over the impact of regulations on school choice programs featuring Patrick Wolf, Douglas Harris, Michael Petrilli, and yours truly, moderated by Neal McCluskey. If you’re in the D.C. area, please RSVP at this link and join us! Come for the policy discussion, stay for the sponsored lunch!

Is the evidence consistent with Overregulation Theory?

Bruno notes that the differences in enrollment trends between participating and non-participating private schools is consistent with OT. Participating schools had been experiencing declining enrollment in the decade before the voucher program was enacted whereas non-participating schools had slightly increasing enrollment on average. This is consistent with the OT’s prediction that better schools (which were able to maintain their enrollment or grow) would be more likely eschew the vouchers due to the significant regulatory burden, while the lower-performing schools (which were losing students) were more desperate for students and funding, and were therefore more willing to jump through the voucher program’s regulatory hoops. However, Bruno calls this evidence into question:

For one thing, the authors of the Louisiana study specifically check to see if learning outcomes vary significantly between schools experiencing greater or lesser prior enrollment declines, and find that they do not. (Bedrick acknowledges this, but doubts there was enough variation in the enrollment trends of participating schools to identify differences.)

We should be skeptical of the explanatory value of the study’s enrollment check. There is no good reason to assume that the correlation between enrollment growth or decline among the small sample of participating schools (which had significantly negative growth, on average) is the same as among all private schools in the state. Making such an assumption is like a blind man holding onto the truck of an elephant and assuming that he’s holding a snake.

The study does not show the variation in enrollment trends among the participating and non-participating schools, but we could imagine a scenario where the enrollment trend among participating schools ranged, say, from -25% to +5% while the range at non-participating schools was -5% to +25%. As shown in the following charts (which use hypothetical data), there may be a strong correlation between enrollment trends and outcomes among the entire population, while there is little correlation in the subset of participating schools.

Enrollment Growth and Performance, Participating Private Schools (Hypothetical)

Screen Shot 2016-02-29 at 4.57.05 PM

Enrollment Growth and Performance, All Private Schools (Hypothetical)

Screen Shot 2016-02-29 at 10.05.29 PM.png

In short, looking at the relationship between enrollment growth and performance in the narrow subset of participating schools doesn’t necessarily tell us anything about the relationship between enrollment growth and performance generally. Hence the study’s “check” that Bruno cites does not provide evidence against OT.

Is there evidence that regulations improve performance?

Bruno also cites evidence that regulations can have a positive impact on student outcomes:

Joshua Cowen of Michigan State University also points out that there is previous evidence of positive effects from accountability rules on voucher program outcomes in other states (though regulations may differ in Louisiana).

The Cowen article considers the impact of high-stakes testing imposed on the Milwaukee voucher program during a multi-year study of that program. The “results indicate substantial growth for voucher students in the first high-stakes testing year, particularly in mathematics, and for students with higher levels of earlier academic achievement.” But is this strong evidence that regulations improve performance? One of the authors of both the original Milwaukee study and the cited article–JPG all-star, Patrick Wolf–cautions against over-interpreting these results:

Ours is one study of what happened in one year for one school choice program that switched from low-stakes testing to high-stakes testing.  As we point out in the report, it is entirely possible that the surge in the test scores of the voucher students was a “one-off” due to a greater focus of the voucher schools on test preparation and test-taking strategies that year.  In other words, by taking the standardized testing seriously in that final year, the schools simply may have produced a truer measure of student’s actual (better) performance all along, not necessarily a signal that they actually learned a lot more in the one year under the new accountability regime.

If we had had another year to examine the trend in scores in our study we might have been able to tease out a possible test-prep bump from an effect of actually higher rates of learning due to accountability.  Our research mandate ended in 2010-11, sadly, and we had to leave it there – a finding that is enticing and suggestive but hardly conclusive.

It’s certainly possible that the high-stakes test improved actual learning. But it’s also possible–and, I would argue, more probable–that changing the stakes just meant that the schools responded to the new incentive by focusing more on test-taking strategies to boost their scores.

For that matter, even if it were true that the regulations actually improved student learning, that does not contradict Overregulation Theory. Both advocates and skeptics of the regulations believe that schools respond to incentives. Those of us who are concerned about the impact of the regulations don’t believe that they can’t improve performance. Rather, our concern is that regulations imposed from above are less effective at improving performance than the incentives created by direct accountability to parents in a robust market in education, and may have adverse unintended consequences.

To explain: We’re concerned that regulations forbidding the use of a school’s preferred admissions standards or requiring the state test (which is aligned to the state curriculum) might drive away better-performing schools, leaving parents to choose only among the lower-performing schools. We’re concerned that price controls will inhibit growth, providing schools with an incentive only to fill empty seats rather than to scale up. We’re concerned that mandatory state tests will inhibit innovation and induce conformity. None of these concerns rule out the possibility (or, indeed, the likelihood) that over time, requiring private schools to administer the state test and report the results and/or face sanctions based on test performance will improve the participating schools’ performance on that test.

Again: we agree that schools respond to incentives. We just think the results of top-down incentives are likely to be inferior to the results of bottom-up choice and competition, which have proved to be powerful tools in so many other fields for spurring innovation and improving quality.

Can Overregulation Theory alone explain the negative results in Louisiana?

Finally, Bruno questions whether OT alone explains the Louisiana results:

[E]ven if regulation prevented all but the worst private schools from participating, this would explain why students did not benefit from transferring into them, but not why students would transfer into them in the first place.

So Overregulation Theory might be part of the story in explaining negative voucher effects in Louisiana, but it is not by itself sufficient. To explain the results we see in the study, it is necessary to tell an additional story about why families would sort into these apparently inferior schools.

Bruno offers a few possible stories–that parents select schools “that provide unobserved benefits,” that the voucher program “induced families to select inferior schools,” or that parents merely “assume any private school must be superior to their available public schools”–but any of these can be consistent with OT.  Indeed, the second story Bruno offers is practically an extension of OT: if the voucher regulations truncate supply so that it is dominated by low-quality schools, and the government gives false assurances that they have vetted those schools, then it is likely that we will see parents lured into choosing inferior schools.

That’s not to say that there are no other factors causing the negative results. It’s likely that there are. (I find Douglas Harris’s argument that the private schools’ curricula did not align with the state test in the first year particularly compelling, though I don’t think it entirely explains the magnitude of the negative results.) We just don’t have any compelling evidence that OT is wrong, and OT can suffice to explain the negative results.

I will conclude as I began: expressing agreement. I concur with Bruno’s assessment that “it is likely that the existing evidence will not allow us to fully adjudicate between competing hypotheses.” Indeed, it’s likely that future evidence won’t be conclusive either (it rarely is), but I hope that further research will shed more light on this important question. Bruno concludes by calling for greater efforts to “understand how families determine where their children will be educated,” noting that by understanding how and why parents might make “sub-optimal — or even harmful” decisions will help “maximize the benefits of school choice while mitigating its risks.” These are noble goals and I share Bruno’s desire to pursue them. I just hope that policymakers will approach what we learn with a spirit of humility about what they can accomplish.


Get every new post delivered to your Inbox.

Join 3,281 other followers