Another Look at Heavy Regulation and Minority Operators

April 25, 2018

MI LA charters 8m

(Guest post by Greg Forster)

EdChoice carries my post using the new NAEP results to bring us back to our earlier discussion of Ian Kingsbury’s finding about what our condescending friends at NACSA do to minority charter operators:

If you’re wondering why the education status quo wants heavy regulation, ask yourself why Facebook CEO Mark Zuckerberg asked Congress to regulate social media: Regulation cements the power of dominant providers, shutting out smaller and less powerful rivals. That’s why heavy regulation does so much damage to minority communities. They have less political power to influence the content of regulations — which more powerful providers can shape in their own favor — and less ability to afford the enormous cost of compliance.

Borrowing Matt’s graphic above to make the point about Louisiana, land of the overregulated NACSA dream.


More Regulation and Less Diverse School Options

November 14, 2017

(Guest Post by Corey A. DeAngelis and Lindsey Burke)

The first experimental evaluation to find negative impacts of a voucher program on student achievement was released to the public over two years ago. Since then, education scholars and public officials have debated whether the initially large negative effects in Louisiana could be explained by the program’s burdensome regulations. While some education policy analysts argued that the state-testing mandates and open-admissions policies deterred higher-performing private schools from participating, others contended that the program might have performed even worse without the rules in place to ensure that parents only chose high-quality educational institutions.

The discussion heated up again three months ago with the release of the Louisiana Scholarship Program’s (LSP) third-year reports showing that students using a voucher caught up to their public school peers on test scores. Just after the public release, John White, Louisiana’s superintendent of education, reasoned that “it may very well be the regulation itself – the accountability system – that is the thing that has promoted the performance.”

However, one of the third-year reports released by researchers at the University of Arkansas addressed this very issue and concluded that the heavy regulatory environment in Louisiana may have driven away higher-quality private schools from participating in the program at all.

Likewise, we just released an analysis examining whether school voucher environments in Louisiana, D.C., and Indiana affected the specialization of the private school market overall. We found that when individual private schools switch into highly-regulated voucher environments such as Louisiana, they are less likely to self-identify as specialized or nontraditional educational institutions.

Theory

Since the voucher programs in question are funded by taxpayers, the public has a legitimate interest in understanding the impact of regulations on private school function and structure. When publicly funded private school choice programs are introduced, most public officials have good intentions when attempting to guarantee that families have access to high-quality educational institutions. In a well-meaning attempt to control levels of school quality and institutional equity, policymakers require that private schools comply with regulations associated with academic quality, financial viability, and access.

Nonetheless, efforts to ensure “accountability” through regulations mirror the various policies governing traditional public schools. If government officials decide to regulate private schools operating within a voucher environment in the same way traditional public schools are regulated, the supply of schools in the private market is expected to become similar to public schools over time. Obviously, private schools that participate in a voucher program will behave more like public schools if they are required to do many of the same things: focus on state standardized tests, fill out supplementary paperwork, and conform admissions processes to the government model.

We expect that private schools in the most-heavily regulated program, the LSP, will be the most likely to experience reductions in specialization. After all, only a third of the private schools chose to participate in the LSP, while between 70 and 78 percent of private schools participated in the programs in D.C. and Indiana. The unusually low LSP participation rate may be because the program is targeted to the least advantaged children in Louisiana and requires students to take the state’s standardized tests and schools to have an open-admissions process.

Results

Every other year, private school leaders provide information about their institutions using the nationally representative Private School Universe Survey (PSS). For our analysis, we examine changes in responses to question eleven which asks school leaders to check one box that best describes their private institution. We examine whether switching into voucher program environments influences the relative likelihoods that private school leaders report that their institutions are regular, specialized, or alternative schools.

As shown in our study, three out of the four effects that were statistically different from zero were in Louisiana. After switching into the voucher program environment in Louisiana, individual private school leaders were around 4-percentage points, or about a tenth of a standard deviation, more likely to identify as regular schools. Furthermore, private school leaders in Louisiana were around 2-percentage points, or about a fifth of a standard deviation, less likely to identify as schools with a “special program or emphasis,” and about 2-percentage points, or around a tenth of a standard deviation, less likely to identify as “schools that offer a curriculum designed to provide alternative or nontraditional education.”

In addition, we found that private school leaders in the nation’s capital were about 10-percentage points, or over a half of a standard deviation, less likely to report that they were alternative schools. This may be because the D.C. voucher program was the only one that required teachers in core subjects to have a bachelor’s degree. It may be that D.C. private schools that provided a nontraditional or alternative education relied more heavily on a robust supply of diverse teachers.

No statistically significant impacts were detected in Indiana, the more lightly regulated voucher program of the three. Although Indiana private schools participating in the state voucher program are required to take the state test, the fact that most did so prior to the introduction of the voucher program in order to be eligible for the high school athletics association may have mitigated the effect such a regulation would have otherwise had on school participation.

But why does this matter? What does this mean for the education of children going forward?

Since individual student needs, learning abilities, interests, and desires are all unique, specialized services ought to lead to improved outcomes. Conversely, access to schools that mimic the district system may not offer much additional choice at all. Consequently, homogenization in the supply of schooling options could have led to the recent negative experimental impacts revealed in Louisiana.

While a hearty set of program regulations may be tempting to policymakers, especially since they give well-intentioned public officials the illusion of quality-control, the disheartening result may be a less meaningful set of choices for parents and their children.

Our results suggest that burdensome packages of regulations likely limit schooling choices in an unfortunate way. Policymakers and school choice advocates interested in establishing a robust universe of education options that are responsive to family needs and preferences should limit red tape and enable private schools to retain their unique identity and character.

###

Corey A. DeAngelis is an Education Policy Analyst at the Cato Institute’s Center for Educational Freedom.

Lindsey Burke is the Director of the Heritage Foundation’s Center for Education Policy.


Playing Ostrich About the Effects of Regulations

July 17, 2017

aaeaaqaaaaaaaanzaaaajdk2mte2y2vkltc1yzctndvkos04njkzlwm0ymi4nzm4mzcwmq

(Guest Post by Jason Bedrick)

There’s plenty to quibble with in Mark Dynarski’s post at EdNext this morning, but his claims about over-regulation theory are downright odd:

Some commenters have concluded that the negative effects in Louisiana were the consequence of the program being ‘over-regulated.’ [6] But the conclusion that the Louisiana program was overregulated relies on unstated premises that private schools that agreed to participate were academically inferior to ones that did not agree but would have if the state did not impose requirements, or that regulation itself impairs academic achievement. Evidence of either is noticeably lacking in the argument.

Far from being an “unstated premise,” the notion that “private schools that agreed to participate were academically inferior to ones that did not agree but would have if the state did not impose requirements” is the explicit argument that I was making in the EdNext post to which he links.

And far from “lacking evidence,” I spelled it out. First, private schools in highly regulated Louisiana were much less likely to accept voucher students than private schools in states with less regulated school choice programs:

Due to the LSP’s high regulatory burden, two-thirds of Louisiana private schools do not accept voucher students. In an American Enterprise Institute survey of private schools, 79 percent of Louisiana school leaders reported that concerns about program regulations played a deciding factor in their decision not to accept LSP students, including 64 percent who listed this as a major factor. In particular, 71 percent worried about the effect on their school’s admissions policies, including 45 percent who stated that this played a major role in their decision. In addition, 54 percent expressed concerns about administering the state test, including 34 percent who said it played a major role in their decision. Other areas of great concern included paperwork and the effect on the schools’ character or identity.

By contrast, the same survey found substantially lower levels of concerns about school choice regulations among school leaders in Indiana and Florida, where the regulatory burdens are considerably lower. While both states limit their vouchers and tax-credit scholarships to low-income students, they do not otherwise restrict admissions criteria, nor do they prevent schools from charging full tuition. Like Louisiana, Indiana requires schools to administer the state test to voucher students, whereas Florida allows schools to choose among many nationally norm-referenced tests.

Unsurprisingly, Florida has the highest level of private school participation among the three states (about two-thirds), followed by Indiana (about half), and Louisiana (one-third). Moreover, Florida schools are the most likely to plan to increase the number of choice students they enroll, while Louisiana schools are the most likely to decrease that number.

Second, there was “suggestive but not conclusive” evidence (as I wrote) that the private schools that did participate were lower performing than those that chose not to:

Low rates of private school participation would not be so troubling if they reflected the decisions of high-performing schools to accept voucher students while the regulations kept low-performing schools away, as proponents of the regulations had desired. However, the regulations may have had the opposite of their intended effect, as Professor Jay P. Greene of the University of Arkansas recently cautioned:

The only schools who are willing to do whatever the state tells them they must do are the schools that are most desperate for money. […] If you don’t have enough kids in your private school and your finances are in bad shape, you’re in danger of closing — probably because you’re not very good — then you’re willing to do whatever the state says.

Indeed, Greene’s concern is borne out by the data. According to the NBER study, “LSP schools open in both 2000 and 2012 experienced an average enrollment loss of 13 percent over this time period, while other private schools grew 3 percent on average.” The authors note that this “indicat[es] that the LSP may attract private schools struggling to maintain enrollment,” and they conclude that these results “suggest caution in the design of voucher systems aimed at expanding school choice for disadvantaged students.”

And, indeed, the recent study by Wolf, DeAngelis, and Sude lends further evidence to the Over-regulation Theory:

Our results largely confirm our hypothesis that higher tuition levels and larger cohort enrollments, conditions normally associated with high quality schools, identify schools that are less likely to participate in voucher programs. We also find a consistent negative relationship between Great Schools Review score and the school participation decision, indicating lower quality schools have a higher tendency of participating in voucher programs in all three states, however the coefficients are not significantly different from zero. State fixed effects reveal private schools in D.C. and Louisiana, the two states that have higher regulatory burdens, are less likely to participate in voucher programs.

The evidence is still merely suggestive, not conclusive, but it’s the best evidence we have. Dynarski might not be persuaded by it, but he can’t ignore that it exists.


Overregulation Is All You Need: A Response to Paul Bruno

February 29, 2016

6-blind-men-hans

(Guest Post by Jason Bedrick)

Over at the Brookings Institution’s education blog, Paul Bruno offers a thoughtful critique of  Overregulation Theory (OT), the idea that government regulations on school choice programs can undermine their positive effects. Bruno argues that although OT is “one of the most plausible explanations” of the negative results that two studies of Louisiana’s voucher program recently found, it is not “entirely consistent with the available evidence” and “does not by itself explain substantial negative effects from vouchers.”

I agree with Bruno–and have stated repeatedly–that the studies’ findings do not conclusively prove OT. That said, I believe both that OT is consistent with the available evidence and that it could explain the substantial negative effects (though I think it’s likely there are other factors at play as well). I’ll explain why below, but first, a shameless plug:

On Friday, March 4th at noon, the Cato Institute will be hosting a debate over the impact of regulations on school choice programs featuring Patrick Wolf, Douglas Harris, Michael Petrilli, and yours truly, moderated by Neal McCluskey. If you’re in the D.C. area, please RSVP at this link and join us! Come for the policy discussion, stay for the sponsored lunch!

Is the evidence consistent with Overregulation Theory?

Bruno notes that the differences in enrollment trends between participating and non-participating private schools is consistent with OT. Participating schools had been experiencing declining enrollment in the decade before the voucher program was enacted whereas non-participating schools had slightly increasing enrollment on average. This is consistent with the OT’s prediction that better schools (which were able to maintain their enrollment or grow) would be more likely eschew the vouchers due to the significant regulatory burden, while the lower-performing schools (which were losing students) were more desperate for students and funding, and were therefore more willing to jump through the voucher program’s regulatory hoops. However, Bruno calls this evidence into question:

For one thing, the authors of the Louisiana study specifically check to see if learning outcomes vary significantly between schools experiencing greater or lesser prior enrollment declines, and find that they do not. (Bedrick acknowledges this, but doubts there was enough variation in the enrollment trends of participating schools to identify differences.)

We should be skeptical of the explanatory value of the study’s enrollment check. There is no good reason to assume that the correlation between enrollment growth or decline among the small sample of participating schools (which had significantly negative growth, on average) is the same as among all private schools in the state. Making such an assumption is like a blind man holding onto the truck of an elephant and assuming that he’s holding a snake.

The study does not show the variation in enrollment trends among the participating and non-participating schools, but we could imagine a scenario where the enrollment trend among participating schools ranged, say, from -25% to +5% while the range at non-participating schools was -5% to +25%. As shown in the following charts (which use hypothetical data), there may be a strong correlation between enrollment trends and outcomes among the entire population, while there is little correlation in the subset of participating schools.

Enrollment Growth and Performance, Participating Private Schools (Hypothetical)

Screen Shot 2016-02-29 at 4.57.05 PM

Enrollment Growth and Performance, All Private Schools (Hypothetical)

Screen Shot 2016-02-29 at 10.05.29 PM.png

In short, looking at the relationship between enrollment growth and performance in the narrow subset of participating schools doesn’t necessarily tell us anything about the relationship between enrollment growth and performance generally. Hence the study’s “check” that Bruno cites does not provide evidence against OT.

Is there evidence that regulations improve performance?

Bruno also cites evidence that regulations can have a positive impact on student outcomes:

Joshua Cowen of Michigan State University also points out that there is previous evidence of positive effects from accountability rules on voucher program outcomes in other states (though regulations may differ in Louisiana).

The Cowen article considers the impact of high-stakes testing imposed on the Milwaukee voucher program during a multi-year study of that program. The “results indicate substantial growth for voucher students in the first high-stakes testing year, particularly in mathematics, and for students with higher levels of earlier academic achievement.” But is this strong evidence that regulations improve performance? One of the authors of both the original Milwaukee study and the cited article–JPG all-star, Patrick Wolf–cautions against over-interpreting these results:

Ours is one study of what happened in one year for one school choice program that switched from low-stakes testing to high-stakes testing.  As we point out in the report, it is entirely possible that the surge in the test scores of the voucher students was a “one-off” due to a greater focus of the voucher schools on test preparation and test-taking strategies that year.  In other words, by taking the standardized testing seriously in that final year, the schools simply may have produced a truer measure of student’s actual (better) performance all along, not necessarily a signal that they actually learned a lot more in the one year under the new accountability regime.

If we had had another year to examine the trend in scores in our study we might have been able to tease out a possible test-prep bump from an effect of actually higher rates of learning due to accountability.  Our research mandate ended in 2010-11, sadly, and we had to leave it there – a finding that is enticing and suggestive but hardly conclusive.

It’s certainly possible that the high-stakes test improved actual learning. But it’s also possible–and, I would argue, more probable–that changing the stakes just meant that the schools responded to the new incentive by focusing more on test-taking strategies to boost their scores.

For that matter, even if it were true that the regulations actually improved student learning, that does not contradict Overregulation Theory. Both advocates and skeptics of the regulations believe that schools respond to incentives. Those of us who are concerned about the impact of the regulations don’t believe that they can’t improve performance. Rather, our concern is that regulations imposed from above are less effective at improving performance than the incentives created by direct accountability to parents in a robust market in education, and may have adverse unintended consequences.

To explain: We’re concerned that regulations forbidding the use of a school’s preferred admissions standards or requiring the state test (which is aligned to the state curriculum) might drive away better-performing schools, leaving parents to choose only among the lower-performing schools. We’re concerned that price controls will inhibit growth, providing schools with an incentive only to fill empty seats rather than to scale up. We’re concerned that mandatory state tests will inhibit innovation and induce conformity. None of these concerns rule out the possibility (or, indeed, the likelihood) that over time, requiring private schools to administer the state test and report the results and/or face sanctions based on test performance will improve the participating schools’ performance on that test.

Again: we agree that schools respond to incentives. We just think the results of top-down incentives are likely to be inferior to the results of bottom-up choice and competition, which have proved to be powerful tools in so many other fields for spurring innovation and improving quality.

Can Overregulation Theory alone explain the negative results in Louisiana?

Finally, Bruno questions whether OT alone explains the Louisiana results:

[E]ven if regulation prevented all but the worst private schools from participating, this would explain why students did not benefit from transferring into them, but not why students would transfer into them in the first place.

So Overregulation Theory might be part of the story in explaining negative voucher effects in Louisiana, but it is not by itself sufficient. To explain the results we see in the study, it is necessary to tell an additional story about why families would sort into these apparently inferior schools.

Bruno offers a few possible stories–that parents select schools “that provide unobserved benefits,” that the voucher program “induced families to select inferior schools,” or that parents merely “assume any private school must be superior to their available public schools”–but any of these can be consistent with OT.  Indeed, the second story Bruno offers is practically an extension of OT: if the voucher regulations truncate supply so that it is dominated by low-quality schools, and the government gives false assurances that they have vetted those schools, then it is likely that we will see parents lured into choosing inferior schools.

That’s not to say that there are no other factors causing the negative results. It’s likely that there are. (I find Douglas Harris’s argument that the private schools’ curricula did not align with the state test in the first year particularly compelling, though I don’t think it entirely explains the magnitude of the negative results.) We just don’t have any compelling evidence that OT is wrong, and OT can suffice to explain the negative results.

I will conclude as I began: expressing agreement. I concur with Bruno’s assessment that “it is likely that the existing evidence will not allow us to fully adjudicate between competing hypotheses.” Indeed, it’s likely that future evidence won’t be conclusive either (it rarely is), but I hope that further research will shed more light on this important question. Bruno concludes by calling for greater efforts to “understand how families determine where their children will be educated,” noting that by understanding how and why parents might make “sub-optimal — or even harmful” decisions will help “maximize the benefits of school choice while mitigating its risks.” These are noble goals and I share Bruno’s desire to pursue them. I just hope that policymakers will approach what we learn with a spirit of humility about what they can accomplish.


What Else Could Explain Negative Result from Louisiana?

January 7, 2016

(Guest Post by Jason Richwine)

The new experimental evaluation of the Louisiana voucher program poses a challenge to school-choice advocates such as myself. How do we explain the voucher students’ negative test outcomes – including a massive 0.4 SD drop in math scores – when evaluations in other cities showed neutral to mildly positive effects? Supporters have quickly coalesced around the explanation that Louisiana imposed regulations so suffocating that only the worst private schools participated in the voucher program.

I find that explanation unsatisfactory for a few reasons. First, it feels like a post-hoc excuse. Yes, choice advocates have been warning about burdensome regulation for years, including in Louisiana, but how many predicted that the state’s voucher system would go down in flames because of it? The magnitude of the score declines must surprise even the most vociferous critics of regulation.

More importantly, if the participating private schools are so bad – and other people apparently knew they were bad, given the declining enrollments – then why did the voucher recipients choose them? Did the parents fail to research their options? Do they not value academics much at all? Blaming the results on an unusually bad set of private schools is tempting, but it creates the new problem of having to explain why parents made such dubious choices.

Personally, I do not find it plausible that school quality alone could have so much impact, especially in one year. The traits that students bring with them to school – natural abilities, resilience, family support networks  – generally explain much more of the variance in student achievement than school quality. Only the absolute worst schools could have such deleterious effects, but there is no indication that the Louisiana voucher schools were the bottom of the barrel. Even if the one third of private schools that participated really were the worst third in the state, we are still talking about schools that are below average – not uniformly awful.

In trying to reconcile Louisiana with the successful experiments in DC, Milwaukee, Charlotte, etc., I suggest exploring other explanations. In particular, how well did the private schools align their curricula with the demands of the state tests? Maybe the private schools were simply teaching different material rather than teaching the state’s curriculum badly. Also worth examining is whether the randomization process, which was done within a complicated set of priority levels for admission, was conducted appropriately. Another issue is how schools adapt after the first year of statewide implementation. And, remember, it is not uncommon for studies to change significantly from the working paper phase to publication. So let’s be patient. Explaining this anomalous study will require more research.


Over-regulation backfires on Voucher Supporters

January 4, 2016

(Guest Post by Matthew Ladner)

Almost a decade ago I posed a Moynihan Challenge to choice opponents: I could produce a pile of random assignment voucher studies with significant positive results, but if you could produce two random assignment studies with statistically significant and negative results I’d buy you a steak dinner. I started here with skeptics in Arizona and when I got cricket noises extended it nationwide and got more cricket noises. Before your gut gets too greedy, yes it had a time limit. If you keep doing these studies long enough you will eventually get false negatives by random chance.

What we have now does not look like a false negative but rather a very poorly designed program in Louisiana with the release of a new study on the first year results. They look **ahem** decidedly negative.

We’ve covered this ground before on JPGB so I will avoid beating the equine corpse, but over the years there have always been concerns that voucher programs would become overly regulated. The response has always been that if this were to occur, that private schools would choose not to participate. Well lo and behold Louisiana’s heavily regulated voucher program comes along, 70% of private schools choose to sit it out, including a large majority of Catholic schools in Orleans Parish. Catholic schools have a long history of putting up with a lot of red tape around the country and around the world, so their choice to pass on the Louisiana Scholarship Program speaks volumes about the program.

What do we know about the 30% of private schools who did participate? Well now we know that their students had a very rough first year and we also know from the study:

Survey data show that LSP-eligible private schools experience rapid enrollment declines prior to entering the program, indicating that the LSP may attract private schools struggling to maintain enrollment. These results suggest caution in the design of voucher systems aimed at expanding school choice for disadvantaged students.

So…..many of the desperate on their way to folding schools decided to grit their teeth and participate in the program. We can also infer from the contrast between this evaluation and all the others that a disproportionate number of stable and successful Louisiana private schools read over the red-tape of the program and decided “thanks but no thanks.”

The ironic dagger cutting deep here- this program was designed to help some of the most disadvantaged students in the country. If you are a low-income student attending low-rated schools in one of the lowest performing states, you got the short end of the stick in life. The very good people who designed this program had every intention of this program being a path out. Tragically in designing to keep bad schools out, they ironically kept the good schools out and invited the bad schools in. The road to this hell was built out of the cobblestones of good intentions, but it still led straight to this debacle.

Those students, among the lowest of the low academically, have been victimized by the design of this program. The high quality private schools among the non-participating schools stand every bit as out of their reach as they had been before the creation of the program. In theory it could have worked out differently, but now we must rid ourselves of all illusions: every system is perfectly designed to produce the results associated with it.

You live and learn. This is a very bitter lesson but one we choice supporters need to take to heart in order to correct our mistake and to prevent future missteps.


Strawmen and Choice Regulation

October 21, 2015

Neerav Kingsland is smart, honest, and, in a positive development for ed philanthropy, is now leading education efforts at the Arnold Foundation.  I enjoyed his response to my series on the dangers of high regulation of school choice (it started here, and then had parts 1, 2, 3, and 4).  But the bulk of the critical response from Neerav and others seemed to focus on defeating a strawman rather than what I actually wrote.  They depict me as arguing against any regulation when my post was explicitly against “high-regulation” of school choice.

Maybe the mistake was on my part.  Maybe I just wasn’t clear enough, so let me succinctly reiterate here my concerns.  Some of the big reform-oriented foundations have drifted toward a high-regulation approach to school choice that I think is very dangerous and counter-productive.  Many of the features of this high-regulation approach can be seen in Louisiana and New Orleans.  They are:

  1. Families should have choices, but they should only choose among quality schools.  So, a portfolio manager, harbor master, or other type of regulator should use test scores to identify who is and is not a quality school operator and eliminate from the set of options a large number of schools that appear to be sub-par.
  2. All choice schools should administer state achievement tests so that regulators and families can more easily make comparisons.  Besides, it is normal to expect that government funds bring with them the requirement to demonstrate performance to the government.
  3. To ensure equity of access, choice schools should not be allowed to use their own admissions criteria but should be required to take all applicants or admit by lottery.  They should also be required to accept the amount of state funding as payment in full.  And equity would be further enhanced if we targeted choice programs toward low income students in low performing traditional public schools.
  4. If private schools are reluctant to go along with this high-regulation approach, maybe it is best just to concentrate on charter schools which have no alternative but to accept whatever regulations come with state funding.  Besides, private schools seem to bring with them other political problems and things uncomfortable to foundations, like religion.

In describing this high-regulation approach increasingly preferred by the big reform foundations I don’t think I am making a strawman out of their views.  This seems like a fair summary of what they believe and it describes what they have put into practice in choice programs in New Orleans and Louisiana.

But I think my objections are being turned into a strawman.  Neerav asks in the title of his post: “To Regulate or Not?”  Joel Klein similarly tweeted: “read you as saying choice per se, no regulation, is solution.”  This is not what I argued.  My argument was against high-regulation, not all regulation. Here is what I was actually trying to argue:

  1. Test scores are useful but are not strong enough predictors of later life outcomes to determine which are the “quality” schools that should be among the options available to families.  So, regulators relying on test scores will experience false positives and false negatives if they try to actively manage a portfolio of schools.  This doesn’t mean that regulators should never close a school or should ignore test results (both of which are strawman arguments), but they should be significantly more humble about what they really know about school quality.  It means that they should give a fair amount of deference to parent preference and should only eliminate schools from choice programs if there are multiple indicators of failure, and exercise some human judgment about those indicators.  In the end I think I agree with Ashley Jochim who argued: “It may be that government will be more effective at establishing performance floors – much like they do in the arena of auto safety – rather than driving continuous improvement in school quality.”
  2. High regulation does not improve equity of access because it drives away many of the highest quality schools.  Heavy regulation caused two-thirds of the private schools in Louisiana to refuse to participate in the state’s voucher program, including most of the best private schools in the state.  Neerav is right that I should have addressed how equity could be achieved without these regulations. Rather than trying to compel equity of access through regulations that instead drive schools out of the program, we should incentivize equity by having student-weighted voucher amounts.  That is, students who are more expensive to educate (disabled, ELL, low-income) should be provided with larger voucher amounts so that schools are compensated for taking on more expensive to educate students.  Most of our concerns about equity of access are likely to be addressed if we simply empower disadvantaged families with higher resource amounts.
  3. Focusing only on charter schools because they are more compliant with a high-regulation approach is a serious mistake.  The evidence suggests that private school choice programs may have stronger later-life outcomes for students than charters.  But more importantly, I argued: “No one knows the ideal political strategy or regulatory scheme, so having a variety of different approaches [vouchers, charters, ESAs, etc…] allows us to learn about how these different methods for expanding choice are doing.  We need choice among choice.”

Just as we should be humble about using test scores to identify quality schools, we should be humble about knowing the ideal political or regulatory strategy.  While I don’t exactly know the right regulation for choice, I’m pretty confident that the high regulation strategy being pursued in LA and New Orleans is a really bad idea.  It deters quality schools from participating, it unnecessarily excludes schools from being options, and it is counter-productive in how it interferes with the operations of choice schools.


Does regulation protect kids and improve outcomes from choice?

October 5, 2015

In my last piece in this series against the high-regulation approach to school choice, I observed that accountability to the government does not automatically follow from receiving government funds.  In fact, most government programs, including Food Stamps, Social Security, Pell Grants, and the Day Care Tuition Tax Credit, have no requirements for performance accountability to the state.

Even if government accountability is not the norm for government programs, some people may still favor requiring choice schools to take the state test and comply with other components of the high-regulation approach to school choice, such as mandating that schools accept voucher amounts as payment in full, prohibiting schools from applying their own admissions requirements, and focusing programs on low-income students in low-performing schools.  Some people, including many of the most powerful backers of school choice, seem to believe that these regulations help protect kids and improve outcomes.

Let’s leave aside for now discussion of whether this set of heavy regulation negatively affects the quality of participating schools.  And let’s also leave aside whether these regulations are even effective in promoting equity of access to participating schools for disadvantaged students.  The real problem is that heavy regulation dramatically reduces the number of participating schools.  Arizona’s choice programs have light regulation and near-universal participation among private schools.  Florida’s tax credit program has more regulation, although it does not require taking the state test.  It has almost two-thirds of private schools willing to take students.  But in Indiana’s heavy-regulation program the private school participation rate drops to around 50%.  At least in Indiana, many private schools were accustomed to administering the state test as a requirement for participating in inter-scholastic athletics.  In Louisiana, where the heavy regulation and state-testing requirement were new, only about 1/3 of private schools are willing to participate in the voucher program.  Survey research by Brian Kisida, Pat Wolf, and Evan Rhinesmith confirms that heavy regulation is driving private schools away from these programs.

The only equity of access that is promoted by the heavy-regulation approach is that everyone is equally unable to access schools that refuse to participate in the programs.  In their desire to protect disadvantaged students, the backers of this heavy-regulation approach have ironically done serious harm to these students by driving away most of the supply.  And the minority of private schools that are willing to participate are likely to include many of the lower quality schools.  Who is most likely to be willing to abandon control over their admissions, accept tiny voucher amounts as payment in full for serving the lowest achieving students, and is willing to take the state achievement tests?  Financially desperate private schools with a lot of empty seats are likely to be first in line to accept these terms.  High-quality private schools may at most make a token number of seats available.  Rather than protecting access and ensuring quality, heavy regulation is having the opposite effect.  Heavy regulations are eliminating the bulk of options and especially driving away the highest-quality private schools.

It should come as no surprise to anyone if we see some very disappointing academic outcomes in Louisiana’s voucher program.  A heavy regulation program that some major backers of school choice believe represents the “ideal” approach is actually designed to give us the worst outcomes.  If we do see bad results, the first impulse of the backers of heavy regulation will be to double-down on regulation.  They’ll wonder who the bad schools are and call for regulators to remove them from the program.

If education reform could be accomplished simply by identifying and closing bad schools while expanding good ones, everything could be fixed already without any need for school choice.  We would just issue regulations to forbid bad schools and to mandate good ones.  See?  Problem solved.  But real education reform requires using the power of choice and competition to provide incentives to create more good and to reduce bad.  The whole problem with the high-regulation approach is that it falsely believes regulators can define, identify, and require good outcomes.  If that were in fact possible, we would have already solved the problem and we could have done so without any school choice.  The enduring troubles of the traditional public system tell me that is not possible.


State Regulation of Private Schools: the Good, the Bad and the Ugly

April 30, 2008

(Guest post by Greg Forster)

 

Today, the Friedman Foundation for Educational Choice releases a report that evaluates how each of the 50 states regulates private schools. While all states regulate things like health and safety, most states go further and impose unreasonable and unnecessary burdens on private schools. This creates barriers to entry, hindering competition and thereby reducing the quality of both public and private schools; it also limits the freedom of parents to choose how their children will be educated. Friedman Foundation Senior Fellow Christopher Hammons graded each state based on how good a job it does of regulating private schools. Scroll down to see the grades.  

 

Accompanying the report, we have compiled lists of all the laws and regulations governing private schools in each of the 50 states. The lists are now available on our website.  

 

Our goal is to educate the public on two fronts. First, we often hear private schools described as “unregulated” by forces hostile to school choice. Private schools are in fact regulated and are accountable to the public for following a large body of laws and regulations. Second, there is wide variation from state to state in the quality of private school regulation. We hope to make the public aware of these disparities so that states with poor regulatory systems will themselves be accountable to the public.  

 

To help ensure the accuracy of our list of private school laws and regulations in each state, we contacted each of the 50 state departments of education, asking them to review our lists and let us know if we had anything missing or incorrect. Each state has an extremely large body of laws and regulations, so any effort to locate all the laws and regulations on a particular topic is very difficult, and we wanted to do everything possible to make sure we didn’t miss anything. As you will see below, some states were more helpful than others.

The Good

The Good #1: About one third of the states (18 ) earned a grade in the A or B range. Florida and New Jersey were tied for having the nation’s best regulatory systems for private schools, followed closely by Connecticut and Delaware.  

 

The Good #2: I will admit that I expected most of our e-mails to the state departments of education would be ignored. As it turned out, most of the states – 29 of them – not only got back to us but went over our lists and either said they were OK as is or offered corrections. In fact, publication of the report was delayed so that we would have time to process all the constructive input we were getting from state departments of education. So let me pour myself a big, delicious bowl of crow and apologize to the departments of education in Connecticut, Delaware, Florida, Georgia, Iowa, Illinois, Kentucky, Louisiana, Maryland, Michigan, Missouri, Nevada, New Hampshire, New Jersey, New Mexico, New York, North Carolina, Ohio, Oregon, South Carolina, South Dakota, Tennessee, Texas, Virginia, Washington, West Virginia, Wisconsin, and Wyoming. I’m sorry I doubted you, and we greatly appreciate your help.  

 

In addition, Arkansas and Arizona deserve recognition for getting back to us and letting us know that they were unable to help us with our request.

The Bad 

The Bad #1: Almost half the states (22) receive D or F grades for the unnecessary burdens imposed on private schools by their laws and regulations. North Dakota ranked the worst in the nation by a large margin, followed by South Dakota, Alabama, Maryland, New York and Tennessee.  

 

The Bad #2: The departments of education in 17 states did not respond to our attempts to contact them. California, Colorado, Hawaii, Idaho, Indiana, Kansas, Maine, Massachusetts, Minnesota, Missouri Mississippi, [oops – apologies to the DOE of Missouri and the schoolchildren of Mississippi] Montana, Nebraska, North Dakota, Oklahoma, Pennsylvania, Rhode Island, and Utah, please check whether you still have a department of education.  

 

Mysteriously, Alaska responded to our initial inquiry, but then didn’t respond to our follow-up communications. 

The Ugly 

Alabama’s department of education deserves special recognition for its efforts to help us. Our request was considered so important that it was ultimately handled by no less than the department’s general counsel.  

 

The department’s first response was to ask where we had gotten our list of Alabama’s private school laws and regulations, and how we were planning to publish it.  

 

I did not ask why they wanted to know, or whom they were planning to pass the information on to once I told them. Instead, I replied that we had compiled our list from the state’s publicly available laws and regulations, and that we were going to post the list on our website and publish a report looking at the laws and regulations in all 50 states.  

 

Their response to that was: “After continued review by appropriate persons and because of the depth of information that you have forwarded to us, it has been determined that this request needs to be reviewed by our SDE Legal Department.” They also asked for more time, which we were happy to give them, as we did for every department that asked for it.  

 

The next and final communication we received was this, which I reprint in its entirety:

I am the General Counsel for the State Department of Education. I have been asked by the Deputy Superintendent of Education, Dr. Eddie Johnson, to review and respond to your request. There are numerous errors contained in the four page document titled ALABAMA. I submit that a further review of our laws and regulations might be helpful. You can access our statutes at www.legislature.state.al.us. The Administrative Code for the Alabama Department of Education can be found at our website, www.alsde.edu/html/home.asp. Thank you for your interest in Alabama.

 

The message was signed “Larry Craven.” Really.  

 

I offer no speculation as to why Mr. Craven would tell us that our document contained numerous errors, but decline to specify any of them.  

 

If at any time he or any other party will be so kind as to specify anything in our list of laws and regulations for Alabama or any other state that’s wrong or missing, we will gladly make any necessary corrections. In a project of this size, combing through countless thousands of laws and regulations to find the ones relevant to private schools, there would be no shame in having missed some. We make a point of saying so both in the report itself and in a disclaimer that appears on each of the 50 state lists we compiled and put on our website.  

 

That said, this also should be said: we wouldn’t have to comb through countless thousands of laws and regulations, a process inherently subject to this kind of difficulty, if the 50 state departments of education provided this information to the public in an easily accessible format. (Some do, but most don’t.) Our only goal here is to get public-domain information actually delivered to the public. We wish we could say that goal was shared by everyone in charge of running the nation’s education system. 

Grades for State Laws and Regulations Governing Private Schools

Alabama F
Alaska B
Arizona A-
Arkansas A-
California B
Colorado B
Connecticut A
Delaware A
Florida A
Georgia A-
Hawaii C+
Idaho C+
Illinois C+
Indiana D-
Iowa D
Kansas F
Kentucky B
Louisiana D
Maine D+
Maryland F
Massachusetts C-
Michigan C-
Minnesota B+
Mississippi F
Missouri A-
Montana F
Nebraska F
Nevada F
New Hampshire C+
New Jersey A
New Mexico C+
New York F
North Carolina D
North Dakota F
Ohio C-
Oklahoma B
Oregon C+
Pennsylvania D
Rhode Island D
South Carolina F
South Dakota F
Tennessee F
Texas B-
Utah A-
Vermont D
Virginia B
Washington F
West Virginia C-
Wisconsin A-
Wyoming F

 Edited for typos


What Went Right?

July 6, 2017

(Guest Post by Matthew Ladner)

I was one of the people who decided to read more about Islamic history after 9/11. Half way through Bernard Lewis’ book What Went Wrong? it occurred to me that Lewis had asked the wrong question. Lewis explored the question of why the Islamic world had gone from arguably the world’s leading civilization to a relative backwater. By the time this had happened however most of the world was a relative backwater in relation to western Europe. The more relevant question therefore in my mind is not “what went wrong” everywhere else but rather “what went right in the West?”

This thought came to mind when reading David Griffith’s paen to regulation in choice programs. David asked “Is there a state in the union with strong oversight, robust market supports, and a low-performing charter sector?” Actually yes there is- NACSA’s top ten state charter sectors are nine miles of bad road interrupted by a unique one-off in Louisiana towards the bottom of the ten.

Griffith writes “Yes, there are a few states where charters have achieved strong results despite a weak framework for intervening in low performing schools, or a dearth of quality authorizers, or limited parental supports. There is an exception to every rule.” Arizona, Colorado and Utah all display the high NAEP/low NACSA score combo. They are not alone btw. By “high NAEP” I mean “near or above Massachusetts scores.” By “low NACSA” I mean a score of 8 or 9 on the NACSA rating before the most recent NAEP. Other flourishing charter sectors, which display either some of these same types of rock star scores in the case of Florida, or else significant advantages over district performance in the case of DC, also dwell outside the top 10 NACSA rated charter laws.

Griffith seems to have mistaken the exception for the rule. It is a simple matter to point to multiple examples of the high NAEP/low NACSA score combo. The high NACSA/high NAEP combo is actually very rare. This is either because top rated states have charter sectors too small to meet NAEP reporting standards-like Indiana and Nevada- or just still struggling after all these years despite the benevolent regulation of the state like Texas.

Now it might be a coincidence that we see high NAEP/low NACSA combos aplenty in the 2015 NAEP. The 2017 NAEP will be released in October. I expect the data to show us more of the same, but time will tell. It could also be a coincidence that voucher programs in Louisiana and Indiana experienced unusually low private school participation rates and struggled academically in the early year evaluations. Some of us started sounding alarm bells on the participation rates before the test score evaluations became available. You don’t need a random assignment study to tell that something is wrong with a voucher program that 70% of private schools choose to avoid, just a bit of common sense. The random assignment studies then did tell us something was indeed wrong, and then a helpful survey of schools pretty much nailed down why it happened. Griffith seems to believe that the problem with LA vouchers is under-regulation. There seems to be an abundance of evidence however that the opposite is true.

So getting back to Lewis, I am convinced that the right question is “what went right in Arizona, Colorado, DC, Florida, Utah etc?” rather than just “what went wrong in Ohio?” Under what set of circumstances can parents take the lead in putting down undesired schools with brutal efficiency? What factors lead this to working in some jurisdictions, but flopping in others? Texas went down the high regulation road in 2001, and well…let’s just say it does not bode well for Ohio.

Even if my friends with a preference for high levels of regulation had evidence to suggest that their approach has benefits (currently lacking) their yearning to apply a one-size-fits-all approach on 50 states with wildly varying needs would still be unwise. Nevada for instance can take little comfort from their high NACSA rating as they continue to suffer extreme levels of public school overcrowding with only a few dozen charter schools. There are hundreds of thousands of children on charter school wait lists in neighboring states with more welcoming charter school laws-why would operators in the surrounding states give Nevada a second thought? This is not a game, and these policies have very real consequences. This fall I will be sending the three Ladner children back to two fantastic charter schools. If either of these schools slips I have other options. Also this fall uncounted thousands of Nevadans will be sending their kids to portable building to meet the first of what will be a series of substitute teachers for the year. These parents have little in the way of other options. What is the case for keeping things this way in Nevada?