Even Seemingly-Harmless Choice Regulations . . . Aren’t

February 9, 2020

C1E966A2-1651-484D-831C-441841C53345
(Guest post by Greg Forster)

OCPA carries my article on an illegal data breach of private data on choice families in Arizona…

The Arizona Department of Education handed over a spreadsheet containing private data on participating families to Save Our Schools Arizona, a group that wants to shut down choice programs and restore the government’s monopoly on education.

The sheet gave the names and email addresses of more than 7,000 parents, the grades their children are in, and the children’s disabilities (if any). As the parent of a special-needs student, I’m not interested in hearing from anybody that this was not a grave violation of the privacy of these families. And we’ll be lucky if ideological fanatics, whipped up by the monopoly system’s generations of irresponsible rhetoric demonizing school choice, don’t use this information to target the families for harassment—or worse.

…that has implications for seemingly harmless regulations on choice programs.

Oklahoma is not obligated to wait for the arousal of a slumbering federal law enforcement to take the hint Arizona is giving it. The more private, personal data the state collects—or requires schools to collect and send it—the less privacy we all have. While the data that were illegally released in Arizona might be data that we can’t prevent the government from having if a choice program is going to exist, the lesson for Oklahoma is that there’s more danger in handing over more data.

No need to keep your opinion private; let me know what you think!


Another Look at Heavy Regulation and Minority Operators

April 25, 2018

MI LA charters 8m

(Guest post by Greg Forster)

EdChoice carries my post using the new NAEP results to bring us back to our earlier discussion of Ian Kingsbury’s finding about what our condescending friends at NACSA do to minority charter operators:

If you’re wondering why the education status quo wants heavy regulation, ask yourself why Facebook CEO Mark Zuckerberg asked Congress to regulate social media: Regulation cements the power of dominant providers, shutting out smaller and less powerful rivals. That’s why heavy regulation does so much damage to minority communities. They have less political power to influence the content of regulations — which more powerful providers can shape in their own favor — and less ability to afford the enormous cost of compliance.

Borrowing Matt’s graphic above to make the point about Louisiana, land of the overregulated NACSA dream.


Charter Regulation Keeps Out Minority Charter Operators

March 9, 2018

Image result for minority charter school operators nacsa

My student, Ian Kingsbury, will be presenting a paper next week at the annual meeting of the Association for Education Finance and Policy examining factors that help explain which applications to operate charter schools are more likely to be approved.  He is still in the early stages of this project and I’m sure will benefit from feedback on how to improve the work, but he has already analyzed nearly 400 applications to operate charter schools in 7 states.  His basic findings, which seem unlikely to change as he gets feedback, should surprise no one but should shock everyone interested in charter schools — the more burdensome the regulatory environment for approving charters, the less likely charters led by minority applicants are to be approved by authorizers.

Using the score that the National Association of Charter School Authorizers (NACSA) gives to each state’s charter policies as a proxy for regulation, Ian finds that for each 1 point increase in a state’s NACSA score (on a scale from 0 to 33), African-American and Hispanic-led charter applications are 1.7 percentage points less likely to be approved.  Given that one state included in the study has had a NACSA score as low as 9 and other states, like Indiana and Nevada, have a 33, the variation in regulatory environments Ian observed is associated with about a 41 percentage point difference in the probability that minority-led charter applications would be approved.

Of course, the charter regulations favored by NACSA, the National Alliance of Public Charter Schools, and most of the charter establishment are meant to promote quality.  Unfortunately, there is no evidence that these regulations are in fact associated with higher quality charter schools. But they clearly make it harder for charter applications to get approved, especially for minority-led charter applicants. Even when Ian controls for whether the minority-led charter applicants attended more selective colleges or were affiliated with CMOs/EMOs, which might be proxies for the quality of those applications, minority-led applicants were still 1.7 percentage points less likely to get approved for each 1 point increase in NACSA score.  In other words, if the purpose of regulations is quality control, those regulations still tend to keep out minority-led charter schools even after adjusting for reasonable proxies of the quality of those proposed charter schools.

This pattern of regulations in the name of quality posing a disproportionate barrier to minorities without actually being related to quality should sound familiar to anyone who has paid attention to the issue of occupational licensure.  A variety of groups, from the Obama White House to the Institute for Justice, have noted that raising requirements to enter many occupations has been an important barrier to opportunity, especially for disadvantaged groups.  Requiring people to spend 2,100 hours and about $22,000 to obtain a cosmetology license before they can braid hair has little to do with quality but is an important obstacle to opportunity.  The same can be said of the type of regulations favored by NACSA and the charter establishment — they have little to do with quality but seem to be large obstacles to minority operated charter schools.

Keeping out minority-led charter schools has potentially serious educational and political implications.  There is some evidence that minority students fare better when educators are of their same race/ethnicity.  Minority-led charter schools may be more likely to provide this type of educational benefit for minority students.  In addition, excluding minority leaders of charter schools severely damages the political prospects for charter schools by making minority community leaders significantly less invested in the growth and success of the charter sector.

If any of you will be at the Association for Education Finance and Policy conference next week, I would encourage you to stop by Ian’s panel on Thursday (March 15) at 10:15.  While Ian’s project is not finished, the evidence is becoming clear enough that NACSA and the rest of the charter establishment need to explain why the policies they favor have such a negative effect on minority-led charter schools.  And if they are going to defend that negative effect by claiming that the policies they favor promote quality, they need to provide evidence to support that claim.  The way it looks now, the types of regulations favored by NACSA and others seem to just keep minorities out without producing any increase in quality.

Update — I’ve added a link to the paper, which is available here.


More Regulation and Less Diverse School Options

November 14, 2017

(Guest Post by Corey A. DeAngelis and Lindsey Burke)

The first experimental evaluation to find negative impacts of a voucher program on student achievement was released to the public over two years ago. Since then, education scholars and public officials have debated whether the initially large negative effects in Louisiana could be explained by the program’s burdensome regulations. While some education policy analysts argued that the state-testing mandates and open-admissions policies deterred higher-performing private schools from participating, others contended that the program might have performed even worse without the rules in place to ensure that parents only chose high-quality educational institutions.

The discussion heated up again three months ago with the release of the Louisiana Scholarship Program’s (LSP) third-year reports showing that students using a voucher caught up to their public school peers on test scores. Just after the public release, John White, Louisiana’s superintendent of education, reasoned that “it may very well be the regulation itself – the accountability system – that is the thing that has promoted the performance.”

However, one of the third-year reports released by researchers at the University of Arkansas addressed this very issue and concluded that the heavy regulatory environment in Louisiana may have driven away higher-quality private schools from participating in the program at all.

Likewise, we just released an analysis examining whether school voucher environments in Louisiana, D.C., and Indiana affected the specialization of the private school market overall. We found that when individual private schools switch into highly-regulated voucher environments such as Louisiana, they are less likely to self-identify as specialized or nontraditional educational institutions.

Theory

Since the voucher programs in question are funded by taxpayers, the public has a legitimate interest in understanding the impact of regulations on private school function and structure. When publicly funded private school choice programs are introduced, most public officials have good intentions when attempting to guarantee that families have access to high-quality educational institutions. In a well-meaning attempt to control levels of school quality and institutional equity, policymakers require that private schools comply with regulations associated with academic quality, financial viability, and access.

Nonetheless, efforts to ensure “accountability” through regulations mirror the various policies governing traditional public schools. If government officials decide to regulate private schools operating within a voucher environment in the same way traditional public schools are regulated, the supply of schools in the private market is expected to become similar to public schools over time. Obviously, private schools that participate in a voucher program will behave more like public schools if they are required to do many of the same things: focus on state standardized tests, fill out supplementary paperwork, and conform admissions processes to the government model.

We expect that private schools in the most-heavily regulated program, the LSP, will be the most likely to experience reductions in specialization. After all, only a third of the private schools chose to participate in the LSP, while between 70 and 78 percent of private schools participated in the programs in D.C. and Indiana. The unusually low LSP participation rate may be because the program is targeted to the least advantaged children in Louisiana and requires students to take the state’s standardized tests and schools to have an open-admissions process.

Results

Every other year, private school leaders provide information about their institutions using the nationally representative Private School Universe Survey (PSS). For our analysis, we examine changes in responses to question eleven which asks school leaders to check one box that best describes their private institution. We examine whether switching into voucher program environments influences the relative likelihoods that private school leaders report that their institutions are regular, specialized, or alternative schools.

As shown in our study, three out of the four effects that were statistically different from zero were in Louisiana. After switching into the voucher program environment in Louisiana, individual private school leaders were around 4-percentage points, or about a tenth of a standard deviation, more likely to identify as regular schools. Furthermore, private school leaders in Louisiana were around 2-percentage points, or about a fifth of a standard deviation, less likely to identify as schools with a “special program or emphasis,” and about 2-percentage points, or around a tenth of a standard deviation, less likely to identify as “schools that offer a curriculum designed to provide alternative or nontraditional education.”

In addition, we found that private school leaders in the nation’s capital were about 10-percentage points, or over a half of a standard deviation, less likely to report that they were alternative schools. This may be because the D.C. voucher program was the only one that required teachers in core subjects to have a bachelor’s degree. It may be that D.C. private schools that provided a nontraditional or alternative education relied more heavily on a robust supply of diverse teachers.

No statistically significant impacts were detected in Indiana, the more lightly regulated voucher program of the three. Although Indiana private schools participating in the state voucher program are required to take the state test, the fact that most did so prior to the introduction of the voucher program in order to be eligible for the high school athletics association may have mitigated the effect such a regulation would have otherwise had on school participation.

But why does this matter? What does this mean for the education of children going forward?

Since individual student needs, learning abilities, interests, and desires are all unique, specialized services ought to lead to improved outcomes. Conversely, access to schools that mimic the district system may not offer much additional choice at all. Consequently, homogenization in the supply of schooling options could have led to the recent negative experimental impacts revealed in Louisiana.

While a hearty set of program regulations may be tempting to policymakers, especially since they give well-intentioned public officials the illusion of quality-control, the disheartening result may be a less meaningful set of choices for parents and their children.

Our results suggest that burdensome packages of regulations likely limit schooling choices in an unfortunate way. Policymakers and school choice advocates interested in establishing a robust universe of education options that are responsive to family needs and preferences should limit red tape and enable private schools to retain their unique identity and character.

###

Corey A. DeAngelis is an Education Policy Analyst at the Cato Institute’s Center for Educational Freedom.

Lindsey Burke is the Director of the Heritage Foundation’s Center for Education Policy.


Playing Ostrich About the Effects of Regulations

July 17, 2017

aaeaaqaaaaaaaanzaaaajdk2mte2y2vkltc1yzctndvkos04njkzlwm0ymi4nzm4mzcwmq

(Guest Post by Jason Bedrick)

There’s plenty to quibble with in Mark Dynarski’s post at EdNext this morning, but his claims about over-regulation theory are downright odd:

Some commenters have concluded that the negative effects in Louisiana were the consequence of the program being ‘over-regulated.’ [6] But the conclusion that the Louisiana program was overregulated relies on unstated premises that private schools that agreed to participate were academically inferior to ones that did not agree but would have if the state did not impose requirements, or that regulation itself impairs academic achievement. Evidence of either is noticeably lacking in the argument.

Far from being an “unstated premise,” the notion that “private schools that agreed to participate were academically inferior to ones that did not agree but would have if the state did not impose requirements” is the explicit argument that I was making in the EdNext post to which he links.

And far from “lacking evidence,” I spelled it out. First, private schools in highly regulated Louisiana were much less likely to accept voucher students than private schools in states with less regulated school choice programs:

Due to the LSP’s high regulatory burden, two-thirds of Louisiana private schools do not accept voucher students. In an American Enterprise Institute survey of private schools, 79 percent of Louisiana school leaders reported that concerns about program regulations played a deciding factor in their decision not to accept LSP students, including 64 percent who listed this as a major factor. In particular, 71 percent worried about the effect on their school’s admissions policies, including 45 percent who stated that this played a major role in their decision. In addition, 54 percent expressed concerns about administering the state test, including 34 percent who said it played a major role in their decision. Other areas of great concern included paperwork and the effect on the schools’ character or identity.

By contrast, the same survey found substantially lower levels of concerns about school choice regulations among school leaders in Indiana and Florida, where the regulatory burdens are considerably lower. While both states limit their vouchers and tax-credit scholarships to low-income students, they do not otherwise restrict admissions criteria, nor do they prevent schools from charging full tuition. Like Louisiana, Indiana requires schools to administer the state test to voucher students, whereas Florida allows schools to choose among many nationally norm-referenced tests.

Unsurprisingly, Florida has the highest level of private school participation among the three states (about two-thirds), followed by Indiana (about half), and Louisiana (one-third). Moreover, Florida schools are the most likely to plan to increase the number of choice students they enroll, while Louisiana schools are the most likely to decrease that number.

Second, there was “suggestive but not conclusive” evidence (as I wrote) that the private schools that did participate were lower performing than those that chose not to:

Low rates of private school participation would not be so troubling if they reflected the decisions of high-performing schools to accept voucher students while the regulations kept low-performing schools away, as proponents of the regulations had desired. However, the regulations may have had the opposite of their intended effect, as Professor Jay P. Greene of the University of Arkansas recently cautioned:

The only schools who are willing to do whatever the state tells them they must do are the schools that are most desperate for money. […] If you don’t have enough kids in your private school and your finances are in bad shape, you’re in danger of closing — probably because you’re not very good — then you’re willing to do whatever the state says.

Indeed, Greene’s concern is borne out by the data. According to the NBER study, “LSP schools open in both 2000 and 2012 experienced an average enrollment loss of 13 percent over this time period, while other private schools grew 3 percent on average.” The authors note that this “indicat[es] that the LSP may attract private schools struggling to maintain enrollment,” and they conclude that these results “suggest caution in the design of voucher systems aimed at expanding school choice for disadvantaged students.”

And, indeed, the recent study by Wolf, DeAngelis, and Sude lends further evidence to the Over-regulation Theory:

Our results largely confirm our hypothesis that higher tuition levels and larger cohort enrollments, conditions normally associated with high quality schools, identify schools that are less likely to participate in voucher programs. We also find a consistent negative relationship between Great Schools Review score and the school participation decision, indicating lower quality schools have a higher tendency of participating in voucher programs in all three states, however the coefficients are not significantly different from zero. State fixed effects reveal private schools in D.C. and Louisiana, the two states that have higher regulatory burdens, are less likely to participate in voucher programs.

The evidence is still merely suggestive, not conclusive, but it’s the best evidence we have. Dynarski might not be persuaded by it, but he can’t ignore that it exists.


Real Accountability Is Choice, Not Regulation

January 27, 2017

carl

(Guest Post by Jason Bedrick)

Imagine that a lobbyist from the taxi industry argued that rideshare companies like Uber and Lyft are “unaccountable” because they aren’t subject to the same regulations as the taxi industry.

The right response, of course, would be to laugh at the absurdity. Clearly Uber and Lyft drivers are much more accountable than taxi drivers because they are directly accountable to the consumer. Passengers rate their drivers based on the quality of their experience, so drivers tend to work hard to ensure that passengers have a good experience. When was the last time you got into a taxi and were offered candy or your choice of music?

Now imagine that someone responded to the taxi lobbyist with something like the following:

That may have been the case years ago, when Uber first burst onto the scene. Rideshare companies were not generally subject to price controls — not even to prohibit “surge pricing” —and even if they were, there were no requirements to purchase a taxi medallion or obtain a commercial license. And, to be fair, that’s still the case for some rideshare companies, where commercial insurance requirements remain light to nonexistent.

But what some rideshare-doubters might not know is that some of the newest and biggest rideshare markets—like those in New York and Maryland—now have significant accountability provisions that are arguably even stronger than those found in many taxi laws. That’s no accident. Pro-rideshare lawmakers adopted these taxi-like requirements because some of us accountability hawks and advocacy groups pushed for them.

In Austin, Texas, participating rideshare companies must run fingerprint background checks on all their drivers. In New York City, drivers must obtain a Taxi and Limousine Commission permit and license plate that’s used to show that the driver meets the city’s standards. Mayor Bill De Blasio has also pushed for imposing a medallion-like system that would keep the growth in the number of rideshare drivers down to manageable levels.

So if you oppose Uber and Lyft because of lack of accountability, it may be time to change your position.

Sadly, that’s almost exactly what Fordham’s Mike Petrilli wrote this week in attempting to defend choice programs from the spurious charge that they are “unaccountable.” The above paragraphs are merely a revised version of what he actually wrote (as Matt highlighted earlier this week). Rather than explain that the very act of choosing is, itself, a strong form of accountability, Mike instead lists all the ways in which some voucher programs subject schools to top-down government regulations–“just like their public school counterparts.”

This fundamentally misunderstands accountability. As I explained at the Heritage Foundation earlier this week, true accountability is when service providers are directly answerable to the people most affected by their performance. When that isn’t possible, as when a utility company has a monopoly, top-down regulations may be necessary instead. But we shouldn’t confuse the inferior alternative accountability regime for the ideal form of accountability just because that’s what we’re used to. As Thomas Sowell has written, “It is hard to imagine a more stupid or more dangerous way of making decisions than by putting those decisions in the hands of people who pay no price for being wrong.”

Sadly, we’ve become so accustomed to the “accountability” regime imposed on the monopoly district school system that many people have not only come to confuse it for true accountability, but they no longer recognize true accountability when they see it.

UPDATE: For my comments at Heritage this week, plus even wiser commentary from Jay Greene and Yuval Levin, see here:

 


Overregulation Is All You Need: A Response to Paul Bruno

February 29, 2016

6-blind-men-hans

(Guest Post by Jason Bedrick)

Over at the Brookings Institution’s education blog, Paul Bruno offers a thoughtful critique of  Overregulation Theory (OT), the idea that government regulations on school choice programs can undermine their positive effects. Bruno argues that although OT is “one of the most plausible explanations” of the negative results that two studies of Louisiana’s voucher program recently found, it is not “entirely consistent with the available evidence” and “does not by itself explain substantial negative effects from vouchers.”

I agree with Bruno–and have stated repeatedly–that the studies’ findings do not conclusively prove OT. That said, I believe both that OT is consistent with the available evidence and that it could explain the substantial negative effects (though I think it’s likely there are other factors at play as well). I’ll explain why below, but first, a shameless plug:

On Friday, March 4th at noon, the Cato Institute will be hosting a debate over the impact of regulations on school choice programs featuring Patrick Wolf, Douglas Harris, Michael Petrilli, and yours truly, moderated by Neal McCluskey. If you’re in the D.C. area, please RSVP at this link and join us! Come for the policy discussion, stay for the sponsored lunch!

Is the evidence consistent with Overregulation Theory?

Bruno notes that the differences in enrollment trends between participating and non-participating private schools is consistent with OT. Participating schools had been experiencing declining enrollment in the decade before the voucher program was enacted whereas non-participating schools had slightly increasing enrollment on average. This is consistent with the OT’s prediction that better schools (which were able to maintain their enrollment or grow) would be more likely eschew the vouchers due to the significant regulatory burden, while the lower-performing schools (which were losing students) were more desperate for students and funding, and were therefore more willing to jump through the voucher program’s regulatory hoops. However, Bruno calls this evidence into question:

For one thing, the authors of the Louisiana study specifically check to see if learning outcomes vary significantly between schools experiencing greater or lesser prior enrollment declines, and find that they do not. (Bedrick acknowledges this, but doubts there was enough variation in the enrollment trends of participating schools to identify differences.)

We should be skeptical of the explanatory value of the study’s enrollment check. There is no good reason to assume that the correlation between enrollment growth or decline among the small sample of participating schools (which had significantly negative growth, on average) is the same as among all private schools in the state. Making such an assumption is like a blind man holding onto the truck of an elephant and assuming that he’s holding a snake.

The study does not show the variation in enrollment trends among the participating and non-participating schools, but we could imagine a scenario where the enrollment trend among participating schools ranged, say, from -25% to +5% while the range at non-participating schools was -5% to +25%. As shown in the following charts (which use hypothetical data), there may be a strong correlation between enrollment trends and outcomes among the entire population, while there is little correlation in the subset of participating schools.

Enrollment Growth and Performance, Participating Private Schools (Hypothetical)

Screen Shot 2016-02-29 at 4.57.05 PM

Enrollment Growth and Performance, All Private Schools (Hypothetical)

Screen Shot 2016-02-29 at 10.05.29 PM.png

In short, looking at the relationship between enrollment growth and performance in the narrow subset of participating schools doesn’t necessarily tell us anything about the relationship between enrollment growth and performance generally. Hence the study’s “check” that Bruno cites does not provide evidence against OT.

Is there evidence that regulations improve performance?

Bruno also cites evidence that regulations can have a positive impact on student outcomes:

Joshua Cowen of Michigan State University also points out that there is previous evidence of positive effects from accountability rules on voucher program outcomes in other states (though regulations may differ in Louisiana).

The Cowen article considers the impact of high-stakes testing imposed on the Milwaukee voucher program during a multi-year study of that program. The “results indicate substantial growth for voucher students in the first high-stakes testing year, particularly in mathematics, and for students with higher levels of earlier academic achievement.” But is this strong evidence that regulations improve performance? One of the authors of both the original Milwaukee study and the cited article–JPG all-star, Patrick Wolf–cautions against over-interpreting these results:

Ours is one study of what happened in one year for one school choice program that switched from low-stakes testing to high-stakes testing.  As we point out in the report, it is entirely possible that the surge in the test scores of the voucher students was a “one-off” due to a greater focus of the voucher schools on test preparation and test-taking strategies that year.  In other words, by taking the standardized testing seriously in that final year, the schools simply may have produced a truer measure of student’s actual (better) performance all along, not necessarily a signal that they actually learned a lot more in the one year under the new accountability regime.

If we had had another year to examine the trend in scores in our study we might have been able to tease out a possible test-prep bump from an effect of actually higher rates of learning due to accountability.  Our research mandate ended in 2010-11, sadly, and we had to leave it there – a finding that is enticing and suggestive but hardly conclusive.

It’s certainly possible that the high-stakes test improved actual learning. But it’s also possible–and, I would argue, more probable–that changing the stakes just meant that the schools responded to the new incentive by focusing more on test-taking strategies to boost their scores.

For that matter, even if it were true that the regulations actually improved student learning, that does not contradict Overregulation Theory. Both advocates and skeptics of the regulations believe that schools respond to incentives. Those of us who are concerned about the impact of the regulations don’t believe that they can’t improve performance. Rather, our concern is that regulations imposed from above are less effective at improving performance than the incentives created by direct accountability to parents in a robust market in education, and may have adverse unintended consequences.

To explain: We’re concerned that regulations forbidding the use of a school’s preferred admissions standards or requiring the state test (which is aligned to the state curriculum) might drive away better-performing schools, leaving parents to choose only among the lower-performing schools. We’re concerned that price controls will inhibit growth, providing schools with an incentive only to fill empty seats rather than to scale up. We’re concerned that mandatory state tests will inhibit innovation and induce conformity. None of these concerns rule out the possibility (or, indeed, the likelihood) that over time, requiring private schools to administer the state test and report the results and/or face sanctions based on test performance will improve the participating schools’ performance on that test.

Again: we agree that schools respond to incentives. We just think the results of top-down incentives are likely to be inferior to the results of bottom-up choice and competition, which have proved to be powerful tools in so many other fields for spurring innovation and improving quality.

Can Overregulation Theory alone explain the negative results in Louisiana?

Finally, Bruno questions whether OT alone explains the Louisiana results:

[E]ven if regulation prevented all but the worst private schools from participating, this would explain why students did not benefit from transferring into them, but not why students would transfer into them in the first place.

So Overregulation Theory might be part of the story in explaining negative voucher effects in Louisiana, but it is not by itself sufficient. To explain the results we see in the study, it is necessary to tell an additional story about why families would sort into these apparently inferior schools.

Bruno offers a few possible stories–that parents select schools “that provide unobserved benefits,” that the voucher program “induced families to select inferior schools,” or that parents merely “assume any private school must be superior to their available public schools”–but any of these can be consistent with OT.  Indeed, the second story Bruno offers is practically an extension of OT: if the voucher regulations truncate supply so that it is dominated by low-quality schools, and the government gives false assurances that they have vetted those schools, then it is likely that we will see parents lured into choosing inferior schools.

That’s not to say that there are no other factors causing the negative results. It’s likely that there are. (I find Douglas Harris’s argument that the private schools’ curricula did not align with the state test in the first year particularly compelling, though I don’t think it entirely explains the magnitude of the negative results.) We just don’t have any compelling evidence that OT is wrong, and OT can suffice to explain the negative results.

I will conclude as I began: expressing agreement. I concur with Bruno’s assessment that “it is likely that the existing evidence will not allow us to fully adjudicate between competing hypotheses.” Indeed, it’s likely that future evidence won’t be conclusive either (it rarely is), but I hope that further research will shed more light on this important question. Bruno concludes by calling for greater efforts to “understand how families determine where their children will be educated,” noting that by understanding how and why parents might make “sub-optimal — or even harmful” decisions will help “maximize the benefits of school choice while mitigating its risks.” These are noble goals and I share Bruno’s desire to pursue them. I just hope that policymakers will approach what we learn with a spirit of humility about what they can accomplish.


Over-regulation backfires on Voucher Supporters

January 4, 2016

(Guest Post by Matthew Ladner)

Almost a decade ago I posed a Moynihan Challenge to choice opponents: I could produce a pile of random assignment voucher studies with significant positive results, but if you could produce two random assignment studies with statistically significant and negative results I’d buy you a steak dinner. I started here with skeptics in Arizona and when I got cricket noises extended it nationwide and got more cricket noises. Before your gut gets too greedy, yes it had a time limit. If you keep doing these studies long enough you will eventually get false negatives by random chance.

What we have now does not look like a false negative but rather a very poorly designed program in Louisiana with the release of a new study on the first year results. They look **ahem** decidedly negative.

We’ve covered this ground before on JPGB so I will avoid beating the equine corpse, but over the years there have always been concerns that voucher programs would become overly regulated. The response has always been that if this were to occur, that private schools would choose not to participate. Well lo and behold Louisiana’s heavily regulated voucher program comes along, 70% of private schools choose to sit it out, including a large majority of Catholic schools in Orleans Parish. Catholic schools have a long history of putting up with a lot of red tape around the country and around the world, so their choice to pass on the Louisiana Scholarship Program speaks volumes about the program.

What do we know about the 30% of private schools who did participate? Well now we know that their students had a very rough first year and we also know from the study:

Survey data show that LSP-eligible private schools experience rapid enrollment declines prior to entering the program, indicating that the LSP may attract private schools struggling to maintain enrollment. These results suggest caution in the design of voucher systems aimed at expanding school choice for disadvantaged students.

So…..many of the desperate on their way to folding schools decided to grit their teeth and participate in the program. We can also infer from the contrast between this evaluation and all the others that a disproportionate number of stable and successful Louisiana private schools read over the red-tape of the program and decided “thanks but no thanks.”

The ironic dagger cutting deep here- this program was designed to help some of the most disadvantaged students in the country. If you are a low-income student attending low-rated schools in one of the lowest performing states, you got the short end of the stick in life. The very good people who designed this program had every intention of this program being a path out. Tragically in designing to keep bad schools out, they ironically kept the good schools out and invited the bad schools in. The road to this hell was built out of the cobblestones of good intentions, but it still led straight to this debacle.

Those students, among the lowest of the low academically, have been victimized by the design of this program. The high quality private schools among the non-participating schools stand every bit as out of their reach as they had been before the creation of the program. In theory it could have worked out differently, but now we must rid ourselves of all illusions: every system is perfectly designed to produce the results associated with it.

You live and learn. This is a very bitter lesson but one we choice supporters need to take to heart in order to correct our mistake and to prevent future missteps.


Strawmen and Choice Regulation

October 21, 2015

Neerav Kingsland is smart, honest, and, in a positive development for ed philanthropy, is now leading education efforts at the Arnold Foundation.  I enjoyed his response to my series on the dangers of high regulation of school choice (it started here, and then had parts 1, 2, 3, and 4).  But the bulk of the critical response from Neerav and others seemed to focus on defeating a strawman rather than what I actually wrote.  They depict me as arguing against any regulation when my post was explicitly against “high-regulation” of school choice.

Maybe the mistake was on my part.  Maybe I just wasn’t clear enough, so let me succinctly reiterate here my concerns.  Some of the big reform-oriented foundations have drifted toward a high-regulation approach to school choice that I think is very dangerous and counter-productive.  Many of the features of this high-regulation approach can be seen in Louisiana and New Orleans.  They are:

  1. Families should have choices, but they should only choose among quality schools.  So, a portfolio manager, harbor master, or other type of regulator should use test scores to identify who is and is not a quality school operator and eliminate from the set of options a large number of schools that appear to be sub-par.
  2. All choice schools should administer state achievement tests so that regulators and families can more easily make comparisons.  Besides, it is normal to expect that government funds bring with them the requirement to demonstrate performance to the government.
  3. To ensure equity of access, choice schools should not be allowed to use their own admissions criteria but should be required to take all applicants or admit by lottery.  They should also be required to accept the amount of state funding as payment in full.  And equity would be further enhanced if we targeted choice programs toward low income students in low performing traditional public schools.
  4. If private schools are reluctant to go along with this high-regulation approach, maybe it is best just to concentrate on charter schools which have no alternative but to accept whatever regulations come with state funding.  Besides, private schools seem to bring with them other political problems and things uncomfortable to foundations, like religion.

In describing this high-regulation approach increasingly preferred by the big reform foundations I don’t think I am making a strawman out of their views.  This seems like a fair summary of what they believe and it describes what they have put into practice in choice programs in New Orleans and Louisiana.

But I think my objections are being turned into a strawman.  Neerav asks in the title of his post: “To Regulate or Not?”  Joel Klein similarly tweeted: “read you as saying choice per se, no regulation, is solution.”  This is not what I argued.  My argument was against high-regulation, not all regulation. Here is what I was actually trying to argue:

  1. Test scores are useful but are not strong enough predictors of later life outcomes to determine which are the “quality” schools that should be among the options available to families.  So, regulators relying on test scores will experience false positives and false negatives if they try to actively manage a portfolio of schools.  This doesn’t mean that regulators should never close a school or should ignore test results (both of which are strawman arguments), but they should be significantly more humble about what they really know about school quality.  It means that they should give a fair amount of deference to parent preference and should only eliminate schools from choice programs if there are multiple indicators of failure, and exercise some human judgment about those indicators.  In the end I think I agree with Ashley Jochim who argued: “It may be that government will be more effective at establishing performance floors – much like they do in the arena of auto safety – rather than driving continuous improvement in school quality.”
  2. High regulation does not improve equity of access because it drives away many of the highest quality schools.  Heavy regulation caused two-thirds of the private schools in Louisiana to refuse to participate in the state’s voucher program, including most of the best private schools in the state.  Neerav is right that I should have addressed how equity could be achieved without these regulations. Rather than trying to compel equity of access through regulations that instead drive schools out of the program, we should incentivize equity by having student-weighted voucher amounts.  That is, students who are more expensive to educate (disabled, ELL, low-income) should be provided with larger voucher amounts so that schools are compensated for taking on more expensive to educate students.  Most of our concerns about equity of access are likely to be addressed if we simply empower disadvantaged families with higher resource amounts.
  3. Focusing only on charter schools because they are more compliant with a high-regulation approach is a serious mistake.  The evidence suggests that private school choice programs may have stronger later-life outcomes for students than charters.  But more importantly, I argued: “No one knows the ideal political strategy or regulatory scheme, so having a variety of different approaches [vouchers, charters, ESAs, etc…] allows us to learn about how these different methods for expanding choice are doing.  We need choice among choice.”

Just as we should be humble about using test scores to identify quality schools, we should be humble about knowing the ideal political or regulatory strategy.  While I don’t exactly know the right regulation for choice, I’m pretty confident that the high regulation strategy being pursued in LA and New Orleans is a really bad idea.  It deters quality schools from participating, it unnecessarily excludes schools from being options, and it is counter-productive in how it interferes with the operations of choice schools.


Does regulation improve the political prospects for choice?

October 6, 2015

In this series of post against the high-regulation approach to school choice, I have demonstrated that performance accountability is not typical of government programs and that heavy regulation drives away quality supply, hurting rather than protecting the students these regulations are meant to help.  If high-regulation is not the norm and does not help children, supporters of this approach might still favor it if they think it has certain political advantages.

For those interested in private school choice, two political advantages are claimed: 1) High-regulation addresses some  objections, winning votes among skeptics to improve the political prospects of passing and sustaining those programs; 2) High-regulation protects private school choice programs from the political damage caused by scandals and embarrassing outcomes.

Neither of these arguments is supported by experience.  Conceding regulatory measures to skeptics and opponents has hardly changed a single vote.  Backers of the Milwaukee voucher program thought they would get relief from legislative opposition if they accepted more burdensome regulation.  No votes have changed as a result and the program remains as precarious as ever.  Nor has regulation protected programs from scandal.  Judging from the steady stream of news reports about teachers in traditional public schools sleeping with students, it appears that no amount of background checks or government oversight can eliminate rare but regular instances of misconduct.  I’m not arguing against a reasonable and light regulatory framework, I’m just suggesting that higher levels of regulation provide little or no additional political protection.  Determined opponents can always find scandals to exploit and cannot be appeased with anything short of preserving the traditional public system.

I’m actually more worried that key backers of school choice are starting to abandon private school choice and focus all of their energies on charters.  High-regulation is the norm in charter programs.  You don’t have to worry about charter schools refusing to participate in a heavily regulated program since they have no alternatives.  And charters seem to be flourishing.  Charter programs exist in more states with more schools serving more students than do private choice programs.  Many important backers of school choice seem to believe that charters are also getting better results.  As Neerav Kingsland of the Arnold Foundation tweeted yesterday: “why is it the over-regulated charter sector that has had the most breakthroughs with low income students?”

Unfortunately, Neerav is mistaken.  Charters are not producing better results than private school choice.  High-regulation comes with a cost to quality.  Let’s consider rigorous evidence on how charter and private school choice affect educational attainment.  For reasons I will discuss at greater length in the next post, I think attainment is a more meaningful indicator of long-term benefits than achievement test results.  I’m aware of 4 rigorous studies of the effect of charter schools on attainment.  The general pattern among them is that programs producing large gains in achievement test outcomes are producing little or no increase in educational attainment.

Angrist, et al examined Boston charter schools and found significant benefits for charter students on MCAS, SAT, and AP performance.  On attainment they write:

Does charter attendance also increase high school graduation rates? Perhaps surprisingly given the gains in test score graduation requirements reported in Table 4, the estimates in Table 7 suggest not. In fact, charter attendance reduces the likelihood a student graduates on time by 12.5 percentage points, a statistically significant effect. This negative estimate falls to zero when the outcome is graduation within five years of 9th-grade entry. (p. 15)

Nor are results much better for attending college: “While the estimated effect of charter attendance on college attendance is positive, it is not large enough to generate a statistically significant finding.” (p. 16)  Angrist, et al do find a significant shift of students from attending 2 year to 4 year colleges, but we don’t know yet if that shift represents a positive development until we see whether they complete their degrees.  Shifting students to 4 year college for which they are ill-suited and from which they drop out does them no favor.

Dobbie and Fryer examine the results of a single charter school in Harlem, the Promise Academy.  Like Angrist, et al, they find large achievement test gains but little benefit for attainment.  Dobbie and Fryer find a higher high school graduation rate after 4 years of the start of 9th grade, but it disappears by 6 years. (p, 18)  College attendance benefits are also fleeting: “Similar to the results for high school graduation,however, control students eventually catch up and make the treatment effects on college enrollment insignificant.”  Dobbie and Fryer similarly find a shift toward 4 year colleges, but again this result is ambiguous. Four year college should help students obtain more schooling but they report “The number of total semesters enrolled in college between lottery winners and lottery losers is small and statistically insignificant.” (p. 19)

Tuttle, et al’s recent evaluation of KIPP charter schools also finds large achievement test gains for charter students but little or no attainment benefit.  Tuttle and her team at Mathematica make two types of comparisons to assess the progress of KIPP high school students.  In one they find: “For new entrants to KIPP high schools, we also examine the probability of graduating within four years of entry. We find that this group of KIPP high schools did not significantly affect four-year graduation rates among new entrants.” (p. 36)  When they examine students who continued from KIPP middle schools into KIPP high schools, they find a small but statistically significant drop in the rate at which students drop out — about 2 percentage points. (p. 39)

Booker, et al examine charter schools in Chicago and Florida and find significant benefits in educational attainment as well as higher earnings later in the workforce — at least for Florida charter students.  They write: “In Florida, the charter high school students show a consistent advantage in absolute terms of 8 to 11 percentage points from high school graduation through a second year of college enrollment.” (p. 22)  On later earnings they find: “Charter high school attendance is
associated with an increase in maximum annual earnings for students between ages 23 and 25 of $2,347—or about 12.7 percent higher earnings than for comparable students who attended a charter middle school but matriculated to a traditional high school.”

Before the high-regulation folks get too excited about the Booker, et al results as vindication of their approach, they should note that these charter schools did not produce impressive achievement test results.  Booker, et al write:

The substantial positive impacts of charter high schools on attainment and earnings are especially striking, given that charter schools in the same jurisdictions have not been shown to have large positive impacts on students’ test scores (Sass, 2006; Zimmer et al., 2012)…. Positive impacts on long-term attainment outcomes and earnings are, of course, more consequential than outcomes on test scores in school. It is possible that charter schools’ full long-term impacts on their students have been underestimated by studies that examine only test scores. More broadly, the findings suggest that the research examining the efficacy of educational programs should examine a broader array of outcomes than just student achievement. (pp. 27-8)

In the high-regulation approach, these charter schools might well be identified as the “bad” schools for failing to improve test scores, and yet they are the ones that produce long-term success for their students.  In the high-regulation approach a portfolio manager or harbor master might kick these schools out of the program or restrict their growth for failing to produce achievement gains.

Let’s briefly review the results from the three rigorous examinations of the effect of private school choice on educational attainment.  Unlike the charter research, they all show significant benefits for attainment.  Wolf, et al examined the federally funded DC voucher program.  They found little benefit for voucher students on achievement tests but those students enjoyed a 21 percentage point increase in the rate at which they graduated high school.  Cowen, et al examined the public funded voucher program in Milwaukee and found a 5 to 7 percentage point increase in the rate at which voucher students attended college.  And Peterson and Chingos examined a privately funded voucher program in New York City and found that African-American voucher recipients experienced a 9 percentage point increase in attending college.  There was no significant benefit for Hispanic students.

If the high-regulation folks wanted to ditch private school choice to go all-in on charters, they would be making a horrible mistake.  The evidence suggests private school choice is producing stronger long-term results.  In addition, among charter schools, the kinds of schools that high-regulation folks like the most are the ones producing weaker long-term outcomes.  Focusing only on charters making the biggest achievement score gains would miss those charters with more modest achievement results but truly impressive attainment outcomes.  Charter schools offer the illusion of getting the benefits from choice without too much of the messiness markets.  As it turns out, central planning among charter schools is no easier than central planning among traditional public schools.

In addition to losing quality if key choice backers were to support charters to the exclusion of private school choice, there are obvious political advantages to backing both types of choice.  Private school choice has helped make the world safe for charters by taking more of the political heat.  We wouldn’t have the same expanding charter sector were it not for the credible threat of even more private school choice.  And the choice movement would be wise to spread its bets across a variety of approaches to expanding school choice.  No one knows the ideal political strategy or regulatory scheme, so having a variety of different approaches allows us to learn about how these different methods for expanding choice are doing.  We need choice among choice.