Jay P. Greene's Blog

In this series of post against the high-regulation approach to school choice, I have demonstrated that performance accountability is not typical of government programs and that heavy regulation drives away quality supply, hurting rather than protecting the students these regulations are meant to help.  If high-regulation is not the norm and does not help children, supporters of this approach might still favor it if they think it has certain political advantages.

For those interested in private school choice, two political advantages are claimed: 1) High-regulation addresses some  objections, winning votes among skeptics to improve the political prospects of passing and sustaining those programs; 2) High-regulation protects private school choice programs from the political damage caused by scandals and embarrassing outcomes.

Neither of these arguments is supported by experience.  Conceding regulatory measures to skeptics and opponents has hardly changed a single vote.  Backers of the Milwaukee voucher program thought they would get relief from legislative opposition if they accepted more burdensome regulation.  No votes have changed as a result and the program remains as precarious as ever.  Nor has regulation protected programs from scandal.  Judging from the steady stream of news reports about teachers in traditional public schools sleeping with students, it appears that no amount of background checks or government oversight can eliminate rare but regular instances of misconduct.  I’m not arguing against a reasonable and light regulatory framework, I’m just suggesting that higher levels of regulation provide little or no additional political protection.  Determined opponents can always find scandals to exploit and cannot be appeased with anything short of preserving the traditional public system.

I’m actually more worried that key backers of school choice are starting to abandon private school choice and focus all of their energies on charters.  High-regulation is the norm in charter programs.  You don’t have to worry about charter schools refusing to participate in a heavily regulated program since they have no alternatives.  And charters seem to be flourishing.  Charter programs exist in more states with more schools serving more students than do private choice programs.  Many important backers of school choice seem to believe that charters are also getting better results.  As Neerav Kingsland of the Arnold Foundation tweeted yesterday: “why is it the over-regulated charter sector that has had the most breakthroughs with low income students?”

Unfortunately, Neerav is mistaken.  Charters are not producing better results than private school choice.  High-regulation comes with a cost to quality.  Let’s consider rigorous evidence on how charter and private school choice affect educational attainment.  For reasons I will discuss at greater length in the next post, I think attainment is a more meaningful indicator of long-term benefits than achievement test results.  I’m aware of 4 rigorous studies of the effect of charter schools on attainment.  The general pattern among them is that programs producing large gains in achievement test outcomes are producing little or no increase in educational attainment.

Angrist, et al examined Boston charter schools and found significant benefits for charter students on MCAS, SAT, and AP performance.  On attainment they write:

Does charter attendance also increase high school graduation rates? Perhaps surprisingly given the gains in test score graduation requirements reported in Table 4, the estimates in Table 7 suggest not. In fact, charter attendance reduces the likelihood a student graduates on time by 12.5 percentage points, a statistically significant effect. This negative estimate falls to zero when the outcome is graduation within five years of 9th-grade entry. (p. 15)

Nor are results much better for attending college: “While the estimated effect of charter attendance on college attendance is positive, it is not large enough to generate a statistically significant finding.” (p. 16)  Angrist, et al do find a significant shift of students from attending 2 year to 4 year colleges, but we don’t know yet if that shift represents a positive development until we see whether they complete their degrees.  Shifting students to 4 year college for which they are ill-suited and from which they drop out does them no favor.

Dobbie and Fryer examine the results of a single charter school in Harlem, the Promise Academy.  Like Angrist, et al, they find large achievement test gains but little benefit for attainment.  Dobbie and Fryer find a higher high school graduation rate after 4 years of the start of 9th grade, but it disappears by 6 years. (p, 18)  College attendance benefits are also fleeting: “Similar to the results for high school graduation,however, control students eventually catch up and make the treatment effects on college enrollment insignificant.”  Dobbie and Fryer similarly find a shift toward 4 year colleges, but again this result is ambiguous. Four year college should help students obtain more schooling but they report “The number of total semesters enrolled in college between lottery winners and lottery losers is small and statistically insignificant.” (p. 19)

Tuttle, et al’s recent evaluation of KIPP charter schools also finds large achievement test gains for charter students but little or no attainment benefit.  Tuttle and her team at Mathematica make two types of comparisons to assess the progress of KIPP high school students.  In one they find: “For new entrants to KIPP high schools, we also examine the probability of graduating within four years of entry. We find that this group of KIPP high schools did not significantly affect four-year graduation rates among new entrants.” (p. 36)  When they examine students who continued from KIPP middle schools into KIPP high schools, they find a small but statistically significant drop in the rate at which students drop out — about 2 percentage points. (p. 39)

Booker, et al examine charter schools in Chicago and Florida and find significant benefits in educational attainment as well as higher earnings later in the workforce — at least for Florida charter students.  They write: “In Florida, the charter high school students show a consistent advantage in absolute terms of 8 to 11 percentage points from high school graduation through a second year of college enrollment.” (p. 22)  On later earnings they find: “Charter high school attendance is
associated with an increase in maximum annual earnings for students between ages 23 and 25 of $2,347—or about 12.7 percent higher earnings than for comparable students who attended a charter middle school but matriculated to a traditional high school.”

Before the high-regulation folks get too excited about the Booker, et al results as vindication of their approach, they should note that these charter schools did not produce impressive achievement test results.  Booker, et al write:

The substantial positive impacts of charter high schools on attainment and earnings are especially striking, given that charter schools in the same jurisdictions have not been shown to have large positive impacts on students’ test scores (Sass, 2006; Zimmer et al., 2012)…. Positive impacts on long-term attainment outcomes and earnings are, of course, more consequential than outcomes on test scores in school. It is possible that charter schools’ full long-term impacts on their students have been underestimated by studies that examine only test scores. More broadly, the findings suggest that the research examining the efficacy of educational programs should examine a broader array of outcomes than just student achievement. (pp. 27-8)

In the high-regulation approach, these charter schools might well be identified as the “bad” schools for failing to improve test scores, and yet they are the ones that produce long-term success for their students.  In the high-regulation approach a portfolio manager or harbor master might kick these schools out of the program or restrict their growth for failing to produce achievement gains.

Let’s briefly review the results from the three rigorous examinations of the effect of private school choice on educational attainment.  Unlike the charter research, they all show significant benefits for attainment.  Wolf, et al examined the federally funded DC voucher program.  They found little benefit for voucher students on achievement tests but those students enjoyed a 21 percentage point increase in the rate at which they graduated high school.  Cowen, et al examined the public funded voucher program in Milwaukee and found a 5 to 7 percentage point increase in the rate at which voucher students attended college.  And Peterson and Chingos examined a privately funded voucher program in New York City and found that African-American voucher recipients experienced a 9 percentage point increase in attending college.  There was no significant benefit for Hispanic students.

If the high-regulation folks wanted to ditch private school choice to go all-in on charters, they would be making a horrible mistake.  The evidence suggests private school choice is producing stronger long-term results.  In addition, among charter schools, the kinds of schools that high-regulation folks like the most are the ones producing weaker long-term outcomes.  Focusing only on charters making the biggest achievement score gains would miss those charters with more modest achievement results but truly impressive attainment outcomes.  Charter schools offer the illusion of getting the benefits from choice without too much of the messiness markets.  As it turns out, central planning among charter schools is no easier than central planning among traditional public schools.

In addition to losing quality if key choice backers were to support charters to the exclusion of private school choice, there are obvious political advantages to backing both types of choice.  Private school choice has helped make the world safe for charters by taking more of the political heat.  We wouldn’t have the same expanding charter sector were it not for the credible threat of even more private school choice.  And the choice movement would be wise to spread its bets across a variety of approaches to expanding school choice.  No one knows the ideal political strategy or regulatory scheme, so having a variety of different approaches allows us to learn about how these different methods for expanding choice are doing.  We need choice among choice.

In my last piece in this series against the high-regulation approach to school choice, I observed that accountability to the government does not automatically follow from receiving government funds.  In fact, most government programs, including Food Stamps, Social Security, Pell Grants, and the Day Care Tuition Tax Credit, have no requirements for performance accountability to the state.

Even if government accountability is not the norm for government programs, some people may still favor requiring choice schools to take the state test and comply with other components of the high-regulation approach to school choice, such as mandating that schools accept voucher amounts as payment in full, prohibiting schools from applying their own admissions requirements, and focusing programs on low-income students in low-performing schools.  Some people, including many of the most powerful backers of school choice, seem to believe that these regulations help protect kids and improve outcomes.

Let’s leave aside for now discussion of whether this set of heavy regulation negatively affects the quality of participating schools.  And let’s also leave aside whether these regulations are even effective in promoting equity of access to participating schools for disadvantaged students.  The real problem is that heavy regulation dramatically reduces the number of participating schools.  Arizona’s choice programs have light regulation and near-universal participation among private schools.  Florida’s tax credit program has more regulation, although it does not require taking the state test.  It has almost two-thirds of private schools willing to take students.  But in Indiana’s heavy-regulation program the private school participation rate drops to around 50%.  At least in Indiana, many private schools were accustomed to administering the state test as a requirement for participating in inter-scholastic athletics.  In Louisiana, where the heavy regulation and state-testing requirement were new, only about 1/3 of private schools are willing to participate in the voucher program.  Survey research by Brian Kisida, Pat Wolf, and Evan Rhinesmith confirms that heavy regulation is driving private schools away from these programs.

The only equity of access that is promoted by the heavy-regulation approach is that everyone is equally unable to access schools that refuse to participate in the programs.  In their desire to protect disadvantaged students, the backers of this heavy-regulation approach have ironically done serious harm to these students by driving away most of the supply.  And the minority of private schools that are willing to participate are likely to include many of the lower quality schools.  Who is most likely to be willing to abandon control over their admissions, accept tiny voucher amounts as payment in full for serving the lowest achieving students, and is willing to take the state achievement tests?  Financially desperate private schools with a lot of empty seats are likely to be first in line to accept these terms.  High-quality private schools may at most make a token number of seats available.  Rather than protecting access and ensuring quality, heavy regulation is having the opposite effect.  Heavy regulations are eliminating the bulk of options and especially driving away the highest-quality private schools.

It should come as no surprise to anyone if we see some very disappointing academic outcomes in Louisiana’s voucher program.  A heavy regulation program that some major backers of school choice believe represents the “ideal” approach is actually designed to give us the worst outcomes.  If we do see bad results, the first impulse of the backers of heavy regulation will be to double-down on regulation.  They’ll wonder who the bad schools are and call for regulators to remove them from the program.

If education reform could be accomplished simply by identifying and closing bad schools while expanding good ones, everything could be fixed already without any need for school choice.  We would just issue regulations to forbid bad schools and to mandate good ones.  See?  Problem solved.  But real education reform requires using the power of choice and competition to provide incentives to create more good and to reduce bad.  The whole problem with the high-regulation approach is that it falsely believes regulators can define, identify, and require good outcomes.  If that were in fact possible, we would have already solved the problem and we could have done so without any school choice.  The enduring troubles of the traditional public system tell me that is not possible.

(Guest Post by Matthew Ladner)

…especially if they have to do with per student funding in Arizona public schools, as I had the chance to explain in today’s Arizona Republic.

Yesterday I promised to rebut the case for high-regulation of school choice programs over a series of posts.  In this first post of that series I address the argument that state funding of programs requires accountability to the state for performance.  It’s the taxpayers’ money, the argument goes, so the public deserves to know if students are doing well.  That’s the price — if you take the government’s money, you are accountable to the government.

Unfortunately, the people who make this argument are just repeating a political slogan.  If they bothered to think about it, even for a few minutes, they would quickly realize that the vast majority of government programs do not require accountability to the government for performance.  When the government provides food stamps it does not require recipients to submit BMI measurements or other indicators of adequate nutrition.  Yes, food stamps have some restriction on the items that may be purchased, but the program does not require accountability for performance.  Social Security was developed to ensure that senior citizens would be able to buy necessities, like housing and food.  But we do not demand an accounting from seniors of the use of those funds.  If they want to blow it at the casino and not pay their rent or buy groceries, they are free to do so.

Even in the area of education, government funds do not typically require accountability for performance.  We do not require recipients of Pell Grants to take a state test.  Beneficiaries of the Day Care Tuition Tax Credit similarly do not have to demonstrate progress toward school readiness in exchange for the government subsidy.  Repeating that government funds require accountability to the government is just mindless sloganeering, not an accurate description of how government programs typically operate.

Why do most government programs not require accountability for performance?  The simple answer is that in most cases we trust that the private interests of program participants are aligned with the public interest in providing them with the benefit.  We trust that food stamp recipients want to avoid being malnourished, which is why we provide them with this assistance.  We trust that seniors don’t want to be homeless or go hungry, so are unlikely to blow their money at the casino if a rent payment is due.  We trust that college students want an education.  And we trust that families with children in pre-school want them to be prepared for later schooling.

We don’t demand performance accountability in any of these programs because we believe that people are likely to use funds in ways that are consistent with the public purpose in providing them with assistance.  Of course, that is not always true.  Some people would rather trade food stamps for drugs and go hungry.  Some people will spend their Social Security checks foolishly and fail to pay the rent.  Some college students would rather embark on an alcohol-fueled journey of self-discovery than receive an education.  And some families just want their pre-schoolers to be warehoused conveniently somewhere while they go to work rather than improve their children’s school-readiness.

While we are fully aware that some people will abuse these programs and fail to use the funds efficiently in a way that is aligned with the public interest, we recognize that demanding performance measures would undermine the public purpose of these programs even more.  Requiring performance measures distorts and narrows the behavior of program participants.  It is also costly, burdensome, and highly intrusive.

The same is true for school choice programs.  As long as we believe that most program participants have interests that are aligned with those of the taxpayer, let’s design school choice programs like we design most government programs — without performance accountability requirements.

Many of the most powerful backers of school choice are embracing a high-regulation approach.  Their interests have shifted from promoting choice as the goal to using choice as a mechanism for obtaining more quality schools.  They don’t trust that choice produces quality.  They want a fairly heavy dose of regulation to prevent bad schools from being included among the options available to families.  They want to control key aspects of school operations to prevent schools from becoming bad.  And they want a powerful regulator — a portfolio manager or harbor master — who will identify and remove bad schools from choice programs.

I think this approach is deeply flawed.  I understand that political reality requires some amount of reasonable regulation.  But the view that regulation, not choice itself, is the main driver of quality improvement is completely wrong.  My fear is that just when school choice is achieving escape velocity as a self-sustaining and expanding policy, the love for high-regulation may do serious harm to these programs and the children they intend to help.

Over a series of blog posts I intend to describe the arguments folks have for high-regulation and why I think those arguments are mistaken.  First let me describe the high-regulation approach to school choice.  Their ideal model has the following central features:

1. Choice programs should not allow private schools to use their regular admissions standards and procedures.  Instead, they should be requires to accept all applicants or use a lottery if over-subscribed.
2. Participating schools should be required to accept the voucher amount as payment in full even though that amount is almost always less than their regular tuition, less than their cost to educate each student, and far less than what is provided to students in traditional public schools.
3. Choice programs should focus on low-income students in low-performing public schools.
4. Participating private schools should be required to administer and report results from the state achievement tests.

This model is not a program that powerful backers of school choice would be willing to accept.  It is their ideal.  It is the starting point for their political negotiations, not what they would be willing to accept after compromises to win political support.

Why do they favor all of these regulations?  I think they have four main arguments:

1. State funds require accountability to the state for performance.
2. Regulation protects kids and improves outcomes from choice.
3. Regulation improves the political prospects for choice.
4. Achievement tests are a reasonable proxy for school quality that a regulator could use to decide which schools should be included or excluded from the set of options available to parents.

I think all four of these arguments are mistaken.  In subsequent posts I’ll consider and rebut each of them.