Anyone want to bet against Arizona for the 2017 NAEP?

December 13, 2016

(Guest Post by Matthew Ladner)

So Lisa Graham Keegan and I finally had the opportunity to collect on our bet with Mike Petrilli on the 2015 NAEP.  You may recall that Mike bet us before the release of the 2015 NAEP results for Reading and Math that Arizona’s NAEP scores would decline. Using our spidey-sense, LGK and I bet Mike that they would be going up, not down.  Arizona’s NAEP scores did go up. Mike was a good sport and quite appropriately paid his debt to us in copper cups (one of the state nicknames is the Copper State).

Depending upon how you examine the data Arizona is either near or else is at the actual top on gains. Measured by student cohort over time, Arizona’s 4th grade class of 2009 made more progress on Math and Reading between 4th and 8th grade scores in 2013 than any other state. Arizona’s 4th grade class of 2011 achieved the same pinnacle in their 2015 scores as 8th graders. (NAEP Math and Reading exams are both scaled and timed to allow such comparisons). The gains for Arizona charter school students dwarf those of Arizona as a whole, or any other state.

So anyhoo, the term “Wild West” is being thrown around as if it is a term of derision by some of those uncomfortable with the selection of Betsy DeVos as Secretary of Education. Here in the actual Wild West we wear the term with pride. The Arizona charter school sector has a majority minority student population, scored like a New England state on all six NAEP exams, and shows consistent results on the state PARCC exams.

Let me know when your state pulls something like that off, because I will be happy to celeNAEP with you. In the meantime, NAEP will be giving state level exams in Reading, Math and Writing in just a few weeks! Let’s see what happens next…

When Evidence and Science are Really Just Assumptions and Ideology

December 5, 2016

Doug Harris has a new post that attempts to reply to the many critics of his New York Times op-ed, including me.  In the NYT piece Harris claimed that “one well-regarded study found that Detroit’s charter schools performed at about the same dismal level as its traditional public schools.” He also claimed that the relatively light regulatory approach to Detroit’s charter schools has led to “the biggest school reform disaster in the country.”  He prefers instead the heavy-regulation approach adopted in New Orleans, which he says has produced “impressive” results.  He then suggests that failing to believe these claims is “a triumph of ideology over evidence.”

Several critics, including me, noted that the “well-regarded study” Harris cites actually finds that Detroit charter schools are producing significantly greater gains than traditional public school alternatives — gains that are only slightly smaller than those in New Orleans and greater than in another high-regulation darling, Denver.  But Harris wants to continue posturing as the person backed by science and evidence, while describing his opponents as ideologues.  In his reply to critics in Education Next he uses the word “evidence” 16 times.

So, his defense of the claim that Detroit charters perform “at about the same dismal level as its traditional public schools” must be based on evidence, right? Actually, no.  He provides four arguments to rescue his assertion that “the failure of Detroit charter schools to improve student outcomes” is true despite the fact that the CREDO evidence he cited to make that claim shows otherwise.  Each one of those four arguments is based on assumptions, not evidence.  If you believe all of his assumptions, you might believe his claim that Detroit charter schools really are a disaster. But drawing conclusions that depend completely on assumptions and are contrary to the evidence he cites is certainly not science.  It seems more like a faith-based or ideological exercise.

Let’s consider his four arguments.  First he says, “given the lack of oversight in Detroit and evidence from other cities that some charter schools cherry-pick their preferred students, these results may make Detroit’s charter schools look better than they are.”  Got that?  He has no evidence that Detroit charters are cherry-picking students at all, let alone that they are doing so at a higher rate than in New Orleans, but he nevertheless posits that “if it’s happening, then the charter effects on achievement [in Detroit] are inflated.”

Second, given Harris’ assumed concerns about cherry-picking in Detroit charter schools and the inability of the CREDO study to account for that, he examines evidence from the urban NAEP test and finds that the city of Detroit has experienced below average growth in those scores in recent years.  Using NAEP results from the entire city to draw conclusions about Detroit’s charter schools requires a host of assumptions.  He’d have to assume that test results from all schools, charter and traditional, somehow speak to the effectiveness of charter schools.  He’d have to assume that demographic and other non-school factors in Detroit do not affect the comparison of test growth in Detroit relative to other cities.

A number of education analysts have coined a term — misNAEPery —  to capture how unreasonable it is to make the assumptions required to use NAEP to compare policies across jurisdictions.  Oddly, some of those analysts who like to accuse others of misNAEPery, like Morgan Polikoff and Matt Barnum, have somehow failed to denounce Harris’ use of misNAEPery in both the NYT op-ed and in the Ed Next reply to critics.  Both have even “retweeted” Harris’ new post, so we can assume they’ve read it.

And just to anticipate concerns about my own consistency on the use of NAEP, I think comparisons using NAEP that control for observed demographics are about as convincing as the CREDO results, which also rely on comparisons controlling for some observed characteristics.  Information from NAEP or CREDO can be interesting or suggestive, which is why I say that Arizona charters “appear” to be doing very well, even if I am not convinced that any of this is causal.  At the very least, it is useful to offer a disclaimer that neither NAEP nor CREDO provides convincing causal evidence, even if confession does not assure absolution.  Harris does not offer any disclaimer and instead uses his NAEP comparisons to bolster his assumption that Detroit charters may be cherry-picking.

Third, Harris builds on the observation that Detroit city has very low NAEP test scores to assume that “the extraordinarily low standing of the city as a whole, to the degree it is caused by low performance of traditional public schools, should make it easier to improve student outcomes when trying something new.”  It is also quite plausible — perhaps more plausible — that a city with extraordinarily low test scores also has severe social and economic problems that are outside of the control of schools, which would make it harder for charters to improve outcomes.

Lastly, Harris argues that even if Detroit charters have produced gains, the gains are smaller than those in New Orleans, so we should prefer the regulatory approach used in New Orleans to the one in Detroit. But this assumes that any greater gains produced by New Orleans charter schools are caused by the regulatory approach in that city.  In fact, we have no idea whether New Orleans’ regulations helped, hurt, or had no effect on how large the gains in that city were. For all we know, the gains made by New Orleans charters are largely attributable to the importation of top-notch human capital from elite colleges and a huge increase in per pupil spending, and that these gains were made despite the hindrance of burdensome regulations.  The heavy regulations in Denver somehow failed to produce gains as large as those in Detroit.

Other than these four-assumption-dependent arguments, Harris offers very little to defend the strong claims he made in the NYT that Detroit charter schools have been a “failure” and a “disaster.”  He does cite some national “charter-friendly” organizations, like CRPE and NACSA,  as being critical of Detroit charters.  But that is an argument from authority — not evidence.

He also falsely suggests that Detroit has failed to close failing charter schools.  In fact, 30% of Detroit’s charter schools have been shuttered.  And another national charter-friendly organization, NAPCS, gives Michigan higher marks on closure than Louisiana, noting that Michigan has closed more charter schools than Louisiana, 47 to 26.

Harris also cites two voucher studies with negative results — one of which is an RCT and the other not — as proof that lower regulation approaches are less effective.  Leaving aside the distinct possibility that the negative RCT result for Louisiana was actually a function of the over-regulation of that program, it’s important to note that Harris fails to cite the entire literature of rigorous studies on the effects of private school choice programs, which overwhelmingly shows positive outcomes.

I agree with Doug that ideology, which could more kindly be described as “principles” or “values,” has an important role to play in policymaking.  I just disagree with him that the evidence clearly shows the superiority of a high-regulation approach to school choice.  I think it’s more accurate to say that the evidence is unclear on this matter, which means that we may — appropriately — need to rely more on our values, principles, and broader ideology when deciding how to proceed.

[Edited to remove tangential comment that was poorly phrased.]

Putting Kids Before Politics

December 2, 2016

(Guest Post by Jason Bedrick)

Wise words from Rev. Alfred Cockfield II of Brooklyn:

Many members of our congregation live in neighborhoods where their local schools are failing. These parents care deeply about having more than one public school option because it means the difference between their child getting a quality education or just getting passed along.

President Obama and Secretary of Education John King have been strong proponents of public charter schools. They understand what parents in our congregation understand: that public charter schools and school choice are vital to closing the racial achievement gap.

And perhaps surprisingly, so does Donald Trump. For all of his rhetoric during the campaign, he is a strong supporter of public charter schools — and believes the status quo must change.

In the months ahead, we must remain vigilant, and be prepared to challenge the president-elect whenever he proposes a policy that threatens our communities. But we can’t let that stop the progress we know we can make and must make. And nowhere is the need to push forward together clearer than with charter schools.

If President-elect Trump is truly willing to support policies that give the highest-need children the school options that they desperately need, we should be willing to listen. That does not mean compromising our values — rather, it means authentically representing the needs of our communities, whether fighting back against a policy we disagree with or advocating for one we believe will help our children.

That’s the balancing act we face moving forward. And we cannot pretend it will be easy. But we owe it to our children to attempt to navigate the next four years with eyes wide open, and with their interests always front and center.


Pass the Popcorn: We Were Voyagers!

December 1, 2016


(Guest post by Greg Forster)

I am a girl who loves her island; I am the girl who loves the sea

It calls me!

I am the daughter of the village chief; we are descended from voyagers who found their way across the world

They call me!

I have delivered us to where we are; I have journeyed farther; I am everything I’ve learned and more

Still it calls me!

This delightful movie speaks, not always clearly but always movingly, to some of the central tensions of advanced modern life. We need both tradition and discovery; we need both inner-looking integrity and outward-looking responsibility. Ultimately, we need God, who alone can transcend and reconcile these contradictory needs, sweeping aside the artifical barriers we create to divide them for our own purposes.

I’m not sure what’s more amazing, that a movie this good has four directors, or that two of them are the directors of The Little Mermaid – the high water mark of the Bad Old Disney – yet the occasional intrusions of Bad Old Disney thinking are actually swept up and incorporated (probably against rather than according to the directors’ purposes) into a whole that is very much of the New Disney, the Disney that cares about the transcendent.

Only very general spoilers follow, nothing highly specific. But if you intend to see the movie, better to set this aside and come back when you’ve seen it.


“When It’s Time to Find Home, We Know the Way”: Tradition and Discovery

Moana is raised in a closed, tradition-bound society but longs to explore and discover, which she can only do by leaving her island behind. We, living in an open, scientific society, long for stable sources of identity, meaning and purpose, which is why we like to watch movies that take place in ancient times and places, when people knew who they were.

If this had been a Bad, Old Disney movie, the lesson would have been that “tradition” is either a bad thing or, at best, something that must bow the knee to the Brave New World and the quest for knowledge and discovery that will inevitably marginalize tradition. (Remember the great modernist quip in Sleeping Beauty: “After all, this is the fourteenth century!”)

Not here. Moana discovers that her traditionalist father has withheld from her the elements of the tradition that favor exploration and discovery – he has suppressed the part of the tradition that is anti-traditionalist. She discovers that her ancestors were voyagers who explored the world and colonized the empty islands as they found them.

“We are explorers reading every sign,” sing her ancestors, but also: “We tell the stories of our elders in a neverending chain!”

“We set a course to find a brand new island everywhere we roam” but “when it’s time to find home, we know the way!”

“We know where we are, we know who we are!”

Traditions embody commitments that are not themselves traditional, or at least not tradition-bound. The village didn’t just sprout up on the island; the villagers came from somewhere.

In the Bad, Old Disney, tradition was simply a relic to be surpassed by the great voyage of discovery. Here, the voyagers have a tradition – voyaging is the tradition – and it tells them who and where they are.

Her father has suppressed all this because a new danger appeared on the ocean; like all traditionalists, he thinks safety is to be found by retreat into a closed system of tradition. But traditions themselves speak against this; they point outside themselves to the higher things that traditions exist to serve.

Traditionalists always want to burn the boats. But it was our ancestors who made them.


“I Am Moana”: Identity and Purpose

We need tradition because we need identity (“we know who we are”). We need discovery because we need purpose, a calling to which we aspire (“it calls me”). But it’s hard to hold on to both.

Identity requires an inward movement toward integrity, in the literal sense of that term – we need wholeness, a fitting together of all our pieces into a sum greater than the parts.

Purpose requires an outward movement toward responsibility – we need to be called out of ourselves, to serve something higher than ourselves.

Identity without purpose is narcissistic. Purpose without identity destroys our humanity.

The moment of greatest crisis, which I will not reveal, comes when Moana can no longer attach the person she is to the calling for which she has been chosen. The crisis is resolved when an outward calling brings her to an inward realization of their connectedness.

As I’ve said, there’s some intrusion of the Bad, Old Disney in Moana. It comes in the form of “look inside yourself,” “follow your heart,” “be who you are on the inside,” etc.

But here, that language – which remains dangerous – is used in the service of higher things. “Look inside yourself” is playing with fire, but as Moana shows, it is (to borrow a fine phrase from Allan Bloom) fire with which we must play if we are to transcend it.

Family is part of the answer. This is one of those rare (but less rare than they used to be) Disney movies with an intact family at the center. And it is noteworthy that her father, her mother and her grandmother are all necessary to Moana’s story. Without any one of them, the story either wouldn’t happen or wouldn’t happen the way it should.

In the family, both identity and purpose are provided for. Your family loves you and wants you to be a person with wholeness, but also calls you out of yourself to the service of others – at first, the others within your family, but ultimately the community at large.

The unconditional love of the family is essential. One of the lessons of the movie is that efforts to earn love and acceptance are futile; the love and acceptance thus “earned” are not authentic love and acceptance.

However, the family is not enough. A greater love is required.

Moana’s grandmother passes on to her the suppressed part of the tradition, and clarifies for her the calling for which she was chosen, because and only because a higher power to which the grandmother is devoted has provided for her to do so.


“The Ocean Doesn’t Help You”: The Mystery of Divine Calling

Darkness is rising and monsters have appeared because we sought to steal from the gods the power of creation itself.

To redeem the world, the ocean has chosen Moana for a high and dangerous quest. If anyone else attempts to stop her, the ocean intervenes to keep a path to her quest open for her.

Moana herself, however, the one who has been chosen for the calling, can refuse the calling if she chooses.

At first there’s no question what she will choose, because the outward calling that comes to her from the ocean connects so clearly to the inward calling she senses inside herself, that she understands to be the true center of her identity.

But then comes suffering and failure. And the ocean doesn’t help. If the ocean wants these things done, why doesn’t it help?

“The ocean doesn’t help you” says the unwise man, “you help yourself.”

That turns out to be empty. When the unwise man gets wiser, he says of his efforts to help himself: “It was never enough.”

It never is. But the ocean still doesn’t help.

At first, the divine call resolves our tensions – by its transcendent authority it supercedes and breaks down our artificial divisions between tradition and discovery, between identity and purpose. It demands both; because, and only because, it demands both with an authority higher than both, it gets both.

But then the ocean doesn’t help.

At the end of all things, Moana is left floating alone on a raft at night, in the middle of nowhere, unable to find the path to her quest.

She has been chosen for the calling, but she can refuse the calling if she chooses.

She faces the same question she thought she had left behind her: “Moana, do you know who you are?”

The divine call, coming to her from outside, has not resolved that question. She must resolve it inwardly. She must – dare I say it – look inside herself.

Not for narcissistic self-expression but to discover who it is that the ocean is really calling.

And what she discovers leads to a surprising redemption.

The Ministry of Truth

December 1, 2016


(Guest Post by Jason Bedrick)

I’m just going to leave this here:


The Next Accountability: Choice, Polity and a New Definition of Reform

November 30, 2016


(Guest post by Greg Forster)

Today EdChoice has released the final installment of my series on The Next Accountability. A while back Matt said he was curious how I would “land the plane” after the lofty heights to which the early installments soared – canvassing big questions about the meaning of life and the future of democratic pluralism.

Well, here’s how I land it: The next accountability should be grounded in:

  • Empowering parents through school choice and local information systems
  • Devolving polity so principals and local districts govern schools close to communities
  • Reforming our movement’s principles to describe education the right way

The last one will probably be the hardest for the movement to grasp but may be the most important in the end:

Markets and competition as drivers of efficiency and performance are important. But they do not provide the moral norms and narratives needed to inform the next accountability. The best case for universal school choice does not center on them. These should be secondary, not primary themes.

We should develop ways of articulating these principles as the basis of the next accountability:

  • The purpose of education is to help children develop the knowledge, skills and virtues they need to live a good life—achieving and appreciating the true, good and beautiful—and to live as good citizens of a community where we disagree about what is good.
  • To cultivate these, we need teachers who are wise professionals (possessing the qualities they seek to instill, and guided by an independent professional ethic) and schools that are free communities (where shared purpose, not the arbitrary dictates of distant authorities, shape a shared life).
  • Teachers and schools can educate the individual student for free pursuit of the good life as he or she sees it, and also for good citizenship and respect for others’ rights in a diverse community, because of what we share in common as human beings and as fellow Americans.
  • Teachers and schools should be held accountable to do this by parents and local communities—the more local the better—because they are in the closest moral and social connection to schools, and can therefore hold them accountable in ways that support their social fabric rather than disrupting it.

Of course, this series is only a down payment on what needs to be a long-term change in the way we think and speak about accountability. But I had a huge amount of fun writing it and I’m convinced that something like this direction is the only real hope for educational accountability after the coming collapse of technocracy.

As always, I’d love to hear your responses. Thanks for reading!

DeVos and the Education Wars

November 29, 2016

(Guest Post by Jason Bedrick)

President-Elect Trump’s nomination of school choice champion Betsy DeVos has become the latest battleground in both the war between pro- and anti-school choice forces as well as the internecine battle between technocratic reformers and market-oriented reformers within the school choice camp. Jay’s take today is a must-read piece. I also added my two cents over at Cato-at-Liberty, defending market-oriented school choice policies from what I see as unfair attacks from the technocrat crowd while simultaneously cautioning my compatriots against pushing for a federal school choice program (e.g., Title I portability). Here’s a taste:

At the center of the panic over Trump’s nomination of DeVos is their support for school choice. Although light on details, Trump has pledged to devote $20 billion to a federal voucher program. As is so often the case, the most vocal opponents of federal school choice are right for the wrong reasons. Not only does the federal government lack constitutional jurisdiction (outside of Washington, D.C., military installations, and tribal lands), but a federal voucher program poses a danger to school choice efforts nationwide because a less-friendly future administration could attach regulations that undermine choice policies. Such regulations are always a threat to the effectiveness of school choice policies, but when a particular state adopts harmful regulations, the negative effects are localized. Louisiana’s folly does not affect Florida. Not so with a national voucher program. Moreover, harmful regulations are easier to fight at the state level than at the federal level, where the exercise of “pen and phone” executive authority is increasingly (and unfortunately) the norm.

The technocratic crowd wants to blame the mediocre results in the charter sector in Michigan (DeVos’s home state) on its supposedly “unregulated” and “laissez-faire” environment, which raises the question: Do they do know what those terms mean? As I note:

Charter schools in Michigan and Arizona may be subject to fewer government regulations than in other states, but it’s absurd to describe the sectors as “laissez-faire” or “an unregulated free market.” For example, charter school regulations in both states, as elsewhere, limit the ability of charter schools to set their own mission (e.g., they must be secular), mandate that they administer the state standardized test, forbid them from setting their own admissions standards, forbid them from charging tuition, limit who can teach in the schools, limit the growth of the number of schools, and so on.

“Laissez-faire” indeed!

Moreover, as JayBlogger Matt Ladner has frequently pointed out, in the real “Wild West” of Arizona, charter schools are knocking the socks off their district counterparts and showing greater improvement than any state average on the NAEP.

Anyway, while we’re on the topic of Trump and education reform, I’d like to express full-throated agreement with Greg Forster’s two recent posts on bigotry and the choices before us, particularly this:

Trump will be president. All of us who work on policy issues have to live in a world where Trump is president. It’s not necessarily a good idea for every decent person to shun him; that means government will be run by scoundrels like Trump.

Every movement needs its Vaclav Klauses as well as its Vaclav Havels – people who are willing to hold their noses and work for a corrupt regime. You simply can’t get anything done otherwise, because there are no non-corrupt regimes.

Milton went to Chile and advised Pinochet. When challenged, he said: “I gave him good advice.”

But if they forget to hold their noses, if they think the regime is good, the movement dies. And they will forget if no one plays Vaclav Havel and goes to jail for telling the truth about the regime.

My biggest fear is that the school choice issue will become tied to Trump. It can never be said too many times: Donald Trump is a notorious racist who discriminates against blacks in his businesses, said a judge of Mexican ancestry couldn’t judge him impartially, constantly flirted with the alt-right, and refused, three times, to repudiate the KKK when first asked to do so. (Just in case this is unclear, the KKK is a criminal organization that murders people and exists to make war on the US government in the name of white nationalism. If Trump wants to learn more about it, he can ask his attorney general, who had a Klan leader executed.)

We in the school choice movement have spent a generation building bridges between the conservatives and libertarians traditionally associated with the issue and progressives and ethnic minority communities. We can’t afford to throw all that away.

Israeli Prime Minister Yitzchak Rabin once said that he would “fight terrorism as if there is no peace process and pursue peace as if there is no terrorism.” We need a similar approach. We should pursue education reform regardless of the Trump administration’s positions on other issues — as Derrell Bradford’s moving personal account reminds us, the stakes are just too high not to. That will entail, at times, working with anyone at the Trump administration who is willing to listen, and supporting good and decent people who go to work for the administration. However, it also means calling out Trump and/or his administration when they do wrong (like, say, Tweet that people should go to jail for engaging in constitutionally protected speech, to take just one example from the last 24 hours), no matter what progress they have made on education reform.

Navigating the political waters over the next four years will be difficult. Even Odysseus only had to pass between Scylla and Charybdis once. I suspect education reformers will find themselves in the straits on numerous occasions in these coming days. I pray that we will have the wisdom to know and the fortitude to do the right thing.