Jay P. Greene's Blog

Supporters of Portfolio Management have held up New Orleans as a model for what they are advocating.  They argued that every city should be like New Orleans in which a single, super-regulator could oversee a portfolio of schools to ensure that only quality schools are opened and expanded while bad ones are closed.  The Portfolio Manager could also issue regulations to govern school discipline, admissions, special education, and transportation as well as ensure “coordination” among the schools.

It was a beautiful dream until we woke up this month to discover that the Louisiana legislature is transferring responsibility for all schools in New Orleans from the Recovery School District, the city’s Portfolio Manager, to the locally elected Orleans Parish School Board, which has repeatedly declared its hostility toward charter schools.  This hostile school board will assume all responsibility for opening and closing all schools as well as continue regulating discipline, admissions, special education, transportation and other matters it deems necessary.  The fox has officially been awarded the keys to the hen house.

Those in denial about this failure cling to provisions in the state law that say charter schools will maintain their operational autonomy even after the school board takeover. But these declarations protecting autonomy are likely to be as meaningful as the Kellogg–Briand Pact declaring that war is illegal. If the school board can open and close schools as well as issue a host of regulations about their operations, they effectively control their operations.

A big problem with building a centralized authority — a Portfolio Manager — to govern all schools is that you cannot count on the good guys being in charge of that process forever.  Eventually, in this case barely a decade later, forces hostile to school choice will assume control of the Portfolio Manager and begin to strangle choice.

If only someone had warned backers of Portfolio Management about these dangers!  Oh wait…

In general, centralized, monopoly regulators are more susceptible to capture than decentralized, multiple regulators. The problem with portfolio districts is that they are trying to be one ring to rule them all…. The ability to control who operates all types of schools and what regulations govern them is too much power not to attract bad people to it or to corrupt those who possess it.

The solution is to decentralize power so that schools are governed by multiple regulators…. When that power is dispersed, it is too hard to capture all of them and they compete with one another to keep regulations reasonable. This is the logic behind separation of power and federalism. It is the virtue of Tiebout choice. The superiority of dispersing and checking power was understood by the founders. It was understood by Montesquieu. It was really Woodrow Wilson who launched a full-frontal attack on the idea of dispersed power and it is his progressive descendants who continue to this day to believe that they can wield the One Ring for good.

Arizona provides an alternative model for how to create a large and successful charter school sector.  Rather than building centralized machinery for controlling all schools of choice, like the Portfolio Manager in New Orleans, which can be captured by forces hostile to school choice, Arizona has multiple authorizers and light regulation.  No one will be able to capture the authorizers in Arizona and turn that machinery against them.

Of course, because choice creates its own political constituency, the charter schools in new Orleans will not collapse or be strangled immediately.  But the Portfolio Management approach has built the regulatory infrastructure to hassle those schools, prevent new ones from opening, and slowly squeeze them out.  At the very least, the progress of the charter sector in New Orleans will stagnate.  In Arizona, where charter schools will not have to run the gauntlet of a hostile regulator, that sector is likely to thrive.

Despite the fact that Portfolio Management is now one of the most popular reforms and despite the enormity of the setback in New Orleans, there has been relatively little discussion of this development in the blogosphere.  It’s bad enough not to have anticipated the rather predictable failure of Portfolio Management, but reformers can’t even acknowledge the failure.  How are we supposed to learn from this and avoid similar mistakes if we can’t even acknowledge that Portfolio Management has experienced a dramatic political failure in New Orleans?

(Guest post by Greg Forster)

Okay, with the president implementing his new Commode Core program, now can we finally admit it’s unrealistic to expect the federal government to keep its hands off schools?

(Guest Post by Jason Bedrick)

Every now and then, Thomas Sowell writes a column titled “Random Thoughts on the Passing Scene” where he offers up gems like this:

Stupid people can cause problems, but it usually takes brilliant people to create a real catastrophe.

I’m no Thomas Sowell, but here are a few of my own (much less pithy or clever) random thoughts the passing education policy scene:

Montana Department of Revenue: Religious Families Need Not Apply

Last October, the Montana Department of Revenue interpreted the state’s constitution to prohibit it from issuing tax-credit scholarships to students attending religious schools. Fortunately, a judge ordered them to reverse course after the heroes at the Institute for Justice — which deserves its title as the “nation’s pre-eminent courtroom defender of school choice” — filed suit. The injunction is only temporary, pending the outcome of the case, but the MT-DOR bureaucrats just filed an appeal. Can’t let parents get a taste of choice! Why, they might choose something else!

Sadly, they’re far from the only ones working to block educational choice…

The Left v. Educational Choice

Lawsuits against two of the most ambitious educational choice policies are each one step closer to resolution. In Florida, the teachers’ union (joined by the NAACP and others) is challenging the state’s tax-credit scholarship program, which served nearly 70,000 low-income students. A lower court tossed out the challenge based on standing but the union appealed. Earlier this week, an appeals court heard oral arguments in the case, where the union’s attorney struggled to explain how the choice law harms anyone:

Lynn Hearn, arguing for the groups challenging the program, said students receiving scholarships are spread unevenly across the state. The program affects public schools “by drawing students out of the system and sending the funding away,” she said, and schools don’t always reap savings from having fewer children to educate.

“You’ll have a few from this school and a few from that school, and so the school is left with exactly the same expenses,” she said.

Judge Lori Rowe, part of a three-judge panel that heard the case, was skeptical.

“You haven’t alleged that any individual student is suffering,” she said to Hearn. “You haven’t alleged that per-student funding has been reduced. You haven’t even alleged that the [state] education budget has been reduced.”

Meanwhile, the Nevada Supreme Court is preparing to hear oral arguments in a lawsuit challenging the state’s education savings account law. Last month, the attorney general’s office “formally nudged” a judge to rule on a separate challenge by the ACLU that was filed back in August. There’s a decent chance that the state supreme court will merge the two cases.

Why is the left so determined to keep families from exercising choice? In short, it’s about control. And that leads me to…

The Bathroom Wars

It’s frankly incredible that with all the serious problems this country is facing, the president thinks it’s a priority to issue a decree forcing public schools to let children pick which bathroom–or locker room–they want to use. Yes, that’s right, the feds have ordered public schools to give biological males access to the girls’ locker room, where teenage girls undress and shower. The White House press secretary, the Orwellianly named Josh Earnest, claims that the new decree is only “guidance” not a “threat,” but if schools don’t follow the “guidance” then the Obama administration promises to cut their federal funding.

Late last year, the feds intervened when a public school in Illinois required a transgender student who is biologically male to “change and shower separately from her teammates and classmates.” Rather than force the student to use the boys’ locker room, the school came to a compromise that attempted to respect the privacy of all the students involved. Nevertheless, the feds filed suit, demanding the transgender student receive “unfettered access” to the girls’ locker room, even though some teenage girls expressed discomfort undressing and showering with someone who is anatomically male. Eventually, the school caved. Outraged parents have filed a lawsuit seeking to overturn the school’s agreement with the feds, and this week the ACLU announced it was intervening to support the feds.

As Neal McCluskey (my wise and benevolent boss at the Cato Institute) often observes, the zero-sum nature of political control over schooling forces citizens into social conflicts like the Bathroom Wars. A much less divisive alternative would be (you guessed it!) a system of educational choice in which parents could select the schools that have their preferred bathroom policy:

As important, if not more so, is that allowing private entities to choose their own policies is consistent with individual liberty, including freedom of association and religion, while it is much better suited to enabling people with competing values to peacefully co-exist. There is no zero-sum contest: Those who want an open bathroom policy could choose schools in which all the staff and families also embraced it, while those feeling more comfortable with bathrooms and locker rooms restricted by biological sex could go to schools with like-minded people.

Perhaps the best examples of educational choice helping to bring peace and balance rights have been in many European countries, where religious conflicts in schools abated as governments decided to fund choices of Protestant, Catholic, nonsectarian, or other institutions. Getting to the place of greater peace requires something difficult – accepting that all people should be able to live as they want as long as they do not force themselves on others, and even if we do not like the choices they make – but living and letting live is the foundation of a free society.

But forget bathrooms–schools might want to consider separating the genders in the classroom as well…

New Study: Single-Sex Schooling Works

A new study by Prof. C. Kirabo Jackson of Northwestern University finds that single-sex schooling improves student outcomes:

The results show that single-sex education can improve both boys’ and girls’ outcomes. Three years after being assigned to a single-sex secondary school, both boys and girls have higher scores on standardized tests. Five years later, they are more likely to take and pass advanced courses. In the long run, both boys and girls are more likely to have completed secondary school and to have earned the credential required to continue to tertiary education. Importantly, boys are also less likely to have been arrested. Taken as a whole, the results suggest that being in the single-sex cohorts improved test scores and also improved longer-run non test score outcomes such as advanced course taking, high school completion and engaging in criminal activity.

As AEI’s Michael Strain highlighted, “the benefits of single-sex instruction are free to the taxpayer — all you have to do is sort children into the appropriate classrooms or schools.”

You won’t be surprised to learn that, evidence be damned, the ACLU opposes this as well.

(Guest post by Patrick J. Wolf)

Private school choice remains a controversial education reform.  Choice programs, involving school vouchers, tax-credit scholarships, or Education Savings Accounts (ESAs), provide financial support to families who wish to access private schooling for their child.  Once declared dead in the U.S. by professional commentators such as Diane Ravitch and Greg Anrig, there are now 50 private school choice programs in 26 states plus the District of Columbia.  Well over half of the initiatives have been enacted in the past five years.  Private school choice is all the rage.

But does it work?  M. Danish Shakeel, Kaitlin Anderson, and I just released a meta-analysis of 19 “gold standard” experimental evaluations of the test-score effects of private school choice programs around the world.  The sum of the reliable evidence indicates that, on average, private school choice increases the reading scores of choice users by about 0.27 standard deviations and their math scores by 0.15 standard deviations.  These are highly statistically significant, educationally meaningful achievement gains of several months of additional learning from school choice.  The achievement benefits of private school choice appear to be somewhat larger for programs in developing countries than for those in the U.S.  Publicly-funded programs produce larger test-score gains than privately-funded ones.

The clarity of the results from our statistical meta-analysis contrasts with the fog of dispute that often surrounds discussions of the effectiveness of private school choice.  Why does our summing of the evidence identify school choice as a clear success while others have claimed that it is a failure (see here and here)?  Three factors have contributed to the muddled view regarding the effectiveness of school choice:  ideology, the limitations of individual studies, and flawed prior reviews of the evidence.

School choice programs support parents who want access to private schooling for their child.  Some people are ideologically opposed to such programs, regardless of the effects of school choice.  Other people have a vested interest in the public school system and resist the competition for students and funds that comes with private school choice.  No amount of evidence is going to change their opinion that school choice is bad.

A second source of disputes over the effectiveness of choice are the limits of each individual empirical study of school choice.  Some are non-experimental and can’t entirely rule out selection bias as a factor in their results (see here, and here).  Fortunately, over the past 20 years, some education researchers have been able to use experimental methods to evaluate privately- and publicly-funded private school choice programs.  Experimental evaluations take the complete population of students who are eligible for a choice program and motivated to use it, then employ a lottery to randomly assign some students to receive a school-choice voucher or scholarship and the rest to serve in the experimental control group.  Since only random chance, and not parental motivation, determines who gets private school choice and who doesn’t, gold standard experimental evaluations produce the most reliable evidence regarding the effectiveness of choice programs.  We limit our meta-analysis to the 19 gold standard studies of private school choice programs globally.

Each of the gold standard studies, in isolation, has certain limitations.  In the experimental evaluation of the initial DC Opportunity Scholarship Program that I led from 2004 to 2011, the number of students in testing grades dropped substantially from year 3 to year 4, leading to a much noisier estimate of the reading impacts of the program, which were positive but just missed being statistically significant with 95% confidence.  Two experimental studies of the Charlotte privately-funded scholarship program, here and here, reported clear positive effects on student test scores but were limited to just a single year after random assignment.  Two recent experimental evaluations of the Louisiana Scholarship Program found negative effects of the program on student test scores but one study was limited to just a single year of outcome data and the second one (which I am leading) has only analyzed two years of outcome data so far.  The Louisiana program, and the state itself, are unique in certain ways, as are many of the programs and locations that have been evaluated.  What are we to conclude from any of these individual studies?

Meta-analysis is an ideal approach to identifying the common effect of a policy when many rigorous but small and particular empirical studies vary in their individual conclusions.  It is a systematic and scientific way to summarize what we know about the effectiveness of a program like private school choice.  The sum of the evidence points to positive achievement effects of choice.

Finally, most of the previous reviews of the evidence on school choice have generated more fog than light, mainly because they have been arbitrary or incomplete in their selection of studies to review.  The most commonly cited school choice review, by economists Cecilia Rouse and Lisa Barrow, declares that it will focus on the evidence from existing experimental studies but then leaves out four such studies (three of which reported positive choice effects) and includes one study that was non-experimental (and found no significant effect of choice).  A more recent summary, by Epple, Romano, and Urquiola, selectively included only 48% of the empirical private school choice studies available in the research literature.  Greg Forster’s Win-Win report from 2013 is a welcome exception and gets the award for the school choice review closest to covering all of the studies that fit his inclusion criteria – 93.3%.  (Greg for the win!)

Our meta-analysis avoided all three factors that have muddied the waters on the test-score effects of private school choice.  It is a non-ideological scientific enterprise, as we followed strict meta-analytic principles such as including every experimental evaluation of choice produced to date, anywhere in the world.  Our study was accepted for presentation at competitive scientific conferences including those of the Society for Research on Education Effectiveness, the Association for Education Finance and Policy, and the Association for Policy Analysis and Management.  Our study is not limited by small sample sizes or only a few years of outcome data.  It is informed by all the evidence from all the gold standard studies.  Finally, there is nothing arbitrary or selective in our sample of experimental evaluations.  We included all of them, regardless of their findings.  When you do the math, students achieve more when they have access to private school choice.

In a sweeping and persuasive review of the past two decades of education reform, Paul Peterson observes that top-down efforts, like standards, testing, and accountability, have run out of steam educationally and politically.  The best way forward, he argues, is to continue working for the steady expansion of choice and competition.

Here are some highlights:

Vouchers and tax credits are slowly broadening their legal footing. Charter schools are growing in number, improving in quality, and beginning to pose genuine competition to public schools, especially within big cities. Introducing such competition is the best hope for American schools, because today’s public schools are showing little capacity to improve on their own.

And on the failure of top-down reforms:

Admittedly, regulatory reform was not invented in Washington. Calls for higher standards, minimum competency tests, and school accountability had surfaced at the state level as early as the 1970s. Southern governors—James Hunt in North Carolina, Bill Clinton in Arkansas, Jeb Bush in Florida, Ann Richardson in Texas, and others—played major roles. Outside the South, Massachusetts took the lead….

All these steps required a vast number of regulations. But school districts still found ways of undermining federal objectives. They instituted byzantine procedures that parents had to navigate before they could exercise choice. Afterschool programs offered by private providers were frequently denied space at local schools. Reconstitution of low-performing schools often consisted mostly of window dressing.

Leading him to conclude:

As an education reform strategy, federal regulation is dead. The regulations had little long-term effect, and the political opposition crescendoed. The regulated captured the regulators. Nor is there much appetite for new accountability rules at the state level. If reform is to take place as the rest of the 21st century unfolds, it will happen because more competition is being introduced into the American education system.

Be sure to read the entire piece in Education Next.

Editor’s note: This post is the sixth and final entry in an ongoing discussion between Fordham’s Michael Petrilli and the University of Arkansas’s Jay Greene that seeks to answer this question: Are math and reading test results strong enough indicators of school quality that regulators can rely on them to determine which schools should be closed and which should be expanded—even if parental demand is inconsistent with test results? Prior entries can be found here, here, here, here, and here.

Shoot, Jay, maybe I should have quit while we were ahead—or at least while we were closer to rapprochement.

Let me admit to being perplexed by your latest post, which has an Alice in Wonderland aspect to it—a suggestion that down is up and up is down. “Short-term changes in test scores are not very good predictors of success,” you write. But that’s not at all what the research I’ve pointed to shows.

Start with the David Deming study of Texas’s 1990s-era accountability system. Low-performing Lone Star State schools faced low ratings and responded by doing something to boost the achievement of their low-performing students. That yielded short-term test-score gains, which were related to positive long-term outcomes. This is the sort of thing we’d like to see much more of in education, wouldn’t we?

Yet you focus on the negative finding: that higher-performing Texas schools were more likely to keep their low-performing kids from taking the test, and those kids did worse over the long term. Supposing that’s so, it merely indicates a flaw in Texas’s school accountability system, which should have required schools to test virtually all of their students (as No Child Left Behind later did). The reason that this group of low-performers did worse is most likely because their schools failed even to try to raise achievement. If they had, those kids’ long-term outcomes would have likely been better too.

As for your points on test score “fade-out,” you are right in that we see this phenomenon in both the pre-K studies you mentioned and in Project Star. Why it happens is an interesting question for which nobody has a great answer, as far as I know, other than the obvious point that the schools and classrooms those kids enter into don’t know how (or don’t try very hard) to sustain earlier gains. But it doesn’t really matter. For our purposes, what it shows is that short-term test score gains don’t lead to long-term test score gains, but they do lead to long-term success. Which is the Holy Grail!

Let’s take it out of the abstract. Let’s say we want to evaluate preschools on whether their students make progress on cognitive assessments, or judge elementary schools based on student-level gains during grades K–3. The evidence indicates that preschools or elementary schools that knock it out of the park in terms of test score gains will see those impacts fade over time, as gauged by test scores. But the kids enrolled in those preschools and elementary schools will benefit in the long term. Whatever the schools are doing to raise short-term test scores is also helping lead to later success; we can measure the scores, but we can’t measure the other stuff. But remind me again: Why we wouldn’t want to use short-term test scores as one gauge of school or program quality?

You end much as you begin:

Rather than relying on test results anyway and making potentially disastrous decisions to close schools or shutter programs on bad information, we should recognize that local actors—including parents—are in a better position to judge school quality. Their preferences deserve strong deference from more distant authorities.

And as I’ve written previously, we’re of one mind in being “anti-bad-information.” We should absolutely stop using performance levels alone (i.e., proficiency rates) to judge school quality. We should be concerned about accountability systems or authorizing practices that might encourage counterproductive practices—like excluding kids from testing or focusing narrowly on reading and math skills instead of a broad curriculum. And we also agree that parents deserve much deference.

But I don’t agree that short-term achievement gains should be put in the “bad information” bucket. And I think you’re being a tad naïve about the quality of “information” that parents themselves have about their schools, which is often extremely limited or hard to interpret. Most parents (myself included) have only a hazy picture of “school quality” and how to know whether it’s present at our own kids’ schools. You know if your child is happy, if the teacher is welcoming, and if the place is safe. It’s a lot harder to know how much learning is taking place, especially in an age when grade inflation is rampant. (Why else would 90 percent of the nation’s parents think that their own children were on grade level?) The government has a role to play in making sure that all school choices meet a basic threshold for quality, just as it has a role in making sure that all of our choices at the grocery store are safe.

So I return to my proposition: Let’s not make high-stakes decisions about schools or programs based on test scores alone. But let’s not ignore those scores, either, or trivialize their influence to such an extent that we allow persistently low-performing schools to persist, zombie-like, in perpetuity.

Charter school authorizers and other quality monitors should react swiftly when schools post mediocre or worse value-added scores. They should give those schools—and their parents—the chance to demonstrate their quality through other means. They should do what they can to turn failure into success, hard though that is. But for the good of the kids, the public, and the sector, they shouldn’t hesitate to shutter schools that aren’t helping children progress.

***

And with that, let me thank Jay for a great debate. We may not agree on what test scores can tell us, but I’m heartened that we concur that there are times when officials must act to address low performance. Parental choice is necessary, but not sufficient. Q.E.D.

– Mike Petrilli

This first appeared on Flypaper.