October 9, 2015

(Guest Post by Matthew Ladner)

I spent yesterday up in Prescott attending a meeting of the Classrooms First initiative, which is a panel put together by Governor Ducey to discuss school finance reform in Arizona. Arizona has both online learning programs and CTE programs that basically entail dividing the per pupil funding between a home school and an outside provider. As you might imagine, this gets messy.

On the digital side for instance schools and districts are not required to accept online courses by outside providers for credit. “How are we supposed to know whether the course was any good?” goes the refrain. Of course districts will happily accept online courses which they provided themselves- these of course have all of the necessary quality control and all. An outside observer however would struggle to discern that a district online course was any more effective than one provided by an outside provider.

So Arizona taxpayers foot the bill for all of this, and we have kids completing coursework but finding themselves denied credit for said coursework. Delightful.

CTE training has a different but related set of problems involving division of per student funding.

During this discussion it occurred to me that our state is about a third of the way through the birth process leading to an a la carte education for high school students.

Some people want our mom to stop pushing while we, er, figure things out or something. The contractions however will not agree to a pause. Lobbyist Jay Kaprosy noted that testing is moving in the direction of end of course exams, so we should consider moving the funding from the providers to the student to allow them to divvy it up among providers. A great many details would need working out to move to such a system and the Classrooms First council was understandably cautious (this topic is related but not central to their task) but my reaction:



Frank Zappa, the Hogs, and Me

October 8, 2015

Here is Frank Zappa during a 1975 concert at the University of Arkansas wearing a hog hat.  And below I am doing the same.  I knew we had a lot in common. (h/t Arkansas Newswire)

Are achievement tests a reasonable proxy for school quality?

October 7, 2015

In my final piece of this series arguing against the high-regulation approach to school choice, I’m going to discuss whether achievement test results are a reasonable proxy for school quality.  Achievement tests are at the center of the high-regulation approach.  They are used by regulators — whether authorizers, portfolio managers, or harbor masters — to identify good and bad schools, to determine whether they should be included as choice options, and to shape the goals schools should pursue.

There is no question that growth in student learning provides us with some useful information.  The problem is that school quality is much broader than just test score results.  I always understood that achievement tests were only a partial and imperfect indicator of school quality, but I used to believe that other aspects of school quality not captured by achievement tests were largely correlated with those test results.  That is, I used to think that if a school raised scores it probably meant that students were safer, more students would graduate, more students would learn productive values, and more students would go on to become successful adults.

Unfortunately, the evidence is increasingly clear that test scores are only weakly correlated with all of these other desirable outcomes from schools.  All you have to do is look at yesterday’s post.  Schools that produce the largest achievement test gains are not necessarily the ones that produce higher graduation, or college-attendance rates.  And sometimes schools with unimpressive achievement gains make significant contributions to attainment and annual earnings when students join the workforce.  I used to think that this couldn’t be possible.  All of these happy outcomes had to be aligned.  They just aren’t.

If you are not persuaded by the evidence I reviewed yesterday on the disconnect between achievement results and other outcomes, I suggest you read an excellent book written by Nobel Prize winning economist James Heckman and his students called The Myth of Achievement Tests.

The problem is that the high-regulation approach needs achievement tests to be correlated with all of these other good outcomes.  If they are going to pick the school choice winners and losers based on test scores, then test scores need to be strongly predictive of other things we care about.  People have been very slow to accept the fact that test scores are only weakly correlated with later life outcomes because it would be so convenient if readily available and relatively inexpensive test scores could capture something as complex as school quality.  The fact that they don’t throws a monkey wrench into the entire high-regulation machinery.

The reality is that the average low-income parent has more complete information about their kid’s school quality than does the highly-trained regulator armed only with test scores.  When we wonder why parents are choosing schools that regulators and other distant experts deem to be “bad,” it is almost certainly because the parents know more about what is good and bad than do the experts.

The wrong response to recognizing that test scores fail to capture school quality sufficiently is to increase the set of high-stakes measures we collect.  We can’t fix the limits of math and reading achievement tests by adding mandatory “grit” surveys or other measures.  Even informed by a variety of measures, Chinese officials are no more effective in telling state-controlled banks how to allocate capital than portfolio managers are in determining how to allocate school options.  Decentralized decision-making is simply better than central planning.

The school choice movement has to remember that choice is what makes this reform work, not the regulation.  I’m perfectly willing to accept that some regulation is necessary and inevitable.  And I’m willing to make compromises to get programs adopted.  But the cardinal sin of the high-regulation school choice folks is that they believe that heavy regulation is the ideal and should be the starting point for political compromises.

Does regulation improve the political prospects for choice?

October 6, 2015

In this series of post against the high-regulation approach to school choice, I have demonstrated that performance accountability is not typical of government programs and that heavy regulation drives away quality supply, hurting rather than protecting the students these regulations are meant to help.  If high-regulation is not the norm and does not help children, supporters of this approach might still favor it if they think it has certain political advantages.

For those interested in private school choice, two political advantages are claimed: 1) High-regulation addresses some  objections, winning votes among skeptics to improve the political prospects of passing and sustaining those programs; 2) High-regulation protects private school choice programs from the political damage caused by scandals and embarrassing outcomes.

Neither of these arguments is supported by experience.  Conceding regulatory measures to skeptics and opponents has hardly changed a single vote.  Backers of the Milwaukee voucher program thought they would get relief from legislative opposition if they accepted more burdensome regulation.  No votes have changed as a result and the program remains as precarious as ever.  Nor has regulation protected programs from scandal.  Judging from the steady stream of news reports about teachers in traditional public schools sleeping with students, it appears that no amount of background checks or government oversight can eliminate rare but regular instances of misconduct.  I’m not arguing against a reasonable and light regulatory framework, I’m just suggesting that higher levels of regulation provide little or no additional political protection.  Determined opponents can always find scandals to exploit and cannot be appeased with anything short of preserving the traditional public system.

I’m actually more worried that key backers of school choice are starting to abandon private school choice and focus all of their energies on charters.  High-regulation is the norm in charter programs.  You don’t have to worry about charter schools refusing to participate in a heavily regulated program since they have no alternatives.  And charters seem to be flourishing.  Charter programs exist in more states with more schools serving more students than do private choice programs.  Many important backers of school choice seem to believe that charters are also getting better results.  As Neerav Kingsland of the Arnold Foundation tweeted yesterday: “why is it the over-regulated charter sector that has had the most breakthroughs with low income students?”

Unfortunately, Neerav is mistaken.  Charters are not producing better results than private school choice.  High-regulation comes with a cost to quality.  Let’s consider rigorous evidence on how charter and private school choice affect educational attainment.  For reasons I will discuss at greater length in the next post, I think attainment is a more meaningful indicator of long-term benefits than achievement test results.  I’m aware of 4 rigorous studies of the effect of charter schools on attainment.  The general pattern among them is that programs producing large gains in achievement test outcomes are producing little or no increase in educational attainment.

Angrist, et al examined Boston charter schools and found significant benefits for charter students on MCAS, SAT, and AP performance.  On attainment they write:

Does charter attendance also increase high school graduation rates? Perhaps surprisingly given the gains in test score graduation requirements reported in Table 4, the estimates in Table 7 suggest not. In fact, charter attendance reduces the likelihood a student graduates on time by 12.5 percentage points, a statistically significant effect. This negative estimate falls to zero when the outcome is graduation within five years of 9th-grade entry. (p. 15)

Nor are results much better for attending college: “While the estimated effect of charter attendance on college attendance is positive, it is not large enough to generate a statistically significant finding.” (p. 16)  Angrist, et al do find a significant shift of students from attending 2 year to 4 year colleges, but we don’t know yet if that shift represents a positive development until we see whether they complete their degrees.  Shifting students to 4 year college for which they are ill-suited and from which they drop out does them no favor.

Dobbie and Fryer examine the results of a single charter school in Harlem, the Promise Academy.  Like Angrist, et al, they find large achievement test gains but little benefit for attainment.  Dobbie and Fryer find a higher high school graduation rate after 4 years of the start of 9th grade, but it disappears by 6 years. (p, 18)  College attendance benefits are also fleeting: “Similar to the results for high school graduation,however, control students eventually catch up and make the treatment effects on college enrollment insignificant.”  Dobbie and Fryer similarly find a shift toward 4 year colleges, but again this result is ambiguous. Four year college should help students obtain more schooling but they report “The number of total semesters enrolled in college between lottery winners and lottery losers is small and statistically insignificant.” (p. 19)

Tuttle, et al’s recent evaluation of KIPP charter schools also finds large achievement test gains for charter students but little or no attainment benefit.  Tuttle and her team at Mathematica make two types of comparisons to assess the progress of KIPP high school students.  In one they find: “For new entrants to KIPP high schools, we also examine the probability of graduating within four years of entry. We find that this group of KIPP high schools did not significantly affect four-year graduation rates among new entrants.” (p. 36)  When they examine students who continued from KIPP middle schools into KIPP high schools, they find a small but statistically significant drop in the rate at which students drop out — about 2 percentage points. (p. 39)

Booker, et al examine charter schools in Chicago and Florida and find significant benefits in educational attainment as well as higher earnings later in the workforce — at least for Florida charter students.  They write: “In Florida, the charter high school students show a consistent advantage in absolute terms of 8 to 11 percentage points from high school graduation through a second year of college enrollment.” (p. 22)  On later earnings they find: “Charter high school attendance is
associated with an increase in maximum annual earnings for students between ages 23 and 25 of $2,347—or about 12.7 percent higher earnings than for comparable students who attended a charter middle school but matriculated to a traditional high school.”

Before the high-regulation folks get too excited about the Booker, et al results as vindication of their approach, they should note that these charter schools did not produce impressive achievement test results.  Booker, et al write:

The substantial positive impacts of charter high schools on attainment and earnings are especially striking, given that charter schools in the same jurisdictions have not been shown to have large positive impacts on students’ test scores (Sass, 2006; Zimmer et al., 2012)…. Positive impacts on long-term attainment outcomes and earnings are, of course, more consequential than outcomes on test scores in school. It is possible that charter schools’ full long-term impacts on their students have been underestimated by studies that examine only test scores. More broadly, the findings suggest that the research examining the efficacy of educational programs should examine a broader array of outcomes than just student achievement. (pp. 27-8)

In the high-regulation approach, these charter schools might well be identified as the “bad” schools for failing to improve test scores, and yet they are the ones that produce long-term success for their students.  In the high-regulation approach a portfolio manager or harbor master might kick these schools out of the program or restrict their growth for failing to produce achievement gains.

Let’s briefly review the results from the three rigorous examinations of the effect of private school choice on educational attainment.  Unlike the charter research, they all show significant benefits for attainment.  Wolf, et al examined the federally funded DC voucher program.  They found little benefit for voucher students on achievement tests but those students enjoyed a 21 percentage point increase in the rate at which they graduated high school.  Cowen, et al examined the public funded voucher program in Milwaukee and found a 5 to 7 percentage point increase in the rate at which voucher students attended college.  And Peterson and Chingos examined a privately funded voucher program in New York City and found that African-American voucher recipients experienced a 9 percentage point increase in attending college.  There was no significant benefit for Hispanic students.

If the high-regulation folks wanted to ditch private school choice to go all-in on charters, they would be making a horrible mistake.  The evidence suggests private school choice is producing stronger long-term results.  In addition, among charter schools, the kinds of schools that high-regulation folks like the most are the ones producing weaker long-term outcomes.  Focusing only on charters making the biggest achievement score gains would miss those charters with more modest achievement results but truly impressive attainment outcomes.  Charter schools offer the illusion of getting the benefits from choice without too much of the messiness markets.  As it turns out, central planning among charter schools is no easier than central planning among traditional public schools.

In addition to losing quality if key choice backers were to support charters to the exclusion of private school choice, there are obvious political advantages to backing both types of choice.  Private school choice has helped make the world safe for charters by taking more of the political heat.  We wouldn’t have the same expanding charter sector were it not for the credible threat of even more private school choice.  And the choice movement would be wise to spread its bets across a variety of approaches to expanding school choice.  No one knows the ideal political strategy or regulatory scheme, so having a variety of different approaches allows us to learn about how these different methods for expanding choice are doing.  We need choice among choice.

Does regulation protect kids and improve outcomes from choice?

October 5, 2015

In my last piece in this series against the high-regulation approach to school choice, I observed that accountability to the government does not automatically follow from receiving government funds.  In fact, most government programs, including Food Stamps, Social Security, Pell Grants, and the Day Care Tuition Tax Credit, have no requirements for performance accountability to the state.

Even if government accountability is not the norm for government programs, some people may still favor requiring choice schools to take the state test and comply with other components of the high-regulation approach to school choice, such as mandating that schools accept voucher amounts as payment in full, prohibiting schools from applying their own admissions requirements, and focusing programs on low-income students in low-performing schools.  Some people, including many of the most powerful backers of school choice, seem to believe that these regulations help protect kids and improve outcomes.

Let’s leave aside for now discussion of whether this set of heavy regulation negatively affects the quality of participating schools.  And let’s also leave aside whether these regulations are even effective in promoting equity of access to participating schools for disadvantaged students.  The real problem is that heavy regulation dramatically reduces the number of participating schools.  Arizona’s choice programs have light regulation and near-universal participation among private schools.  Florida’s tax credit program has more regulation, although it does not require taking the state test.  It has almost two-thirds of private schools willing to take students.  But in Indiana’s heavy-regulation program the private school participation rate drops to around 50%.  At least in Indiana, many private schools were accustomed to administering the state test as a requirement for participating in inter-scholastic athletics.  In Louisiana, where the heavy regulation and state-testing requirement were new, only about 1/3 of private schools are willing to participate in the voucher program.  Survey research by Brian Kisida, Pat Wolf, and Evan Rhinesmith confirms that heavy regulation is driving private schools away from these programs.

The only equity of access that is promoted by the heavy-regulation approach is that everyone is equally unable to access schools that refuse to participate in the programs.  In their desire to protect disadvantaged students, the backers of this heavy-regulation approach have ironically done serious harm to these students by driving away most of the supply.  And the minority of private schools that are willing to participate are likely to include many of the lower quality schools.  Who is most likely to be willing to abandon control over their admissions, accept tiny voucher amounts as payment in full for serving the lowest achieving students, and is willing to take the state achievement tests?  Financially desperate private schools with a lot of empty seats are likely to be first in line to accept these terms.  High-quality private schools may at most make a token number of seats available.  Rather than protecting access and ensuring quality, heavy regulation is having the opposite effect.  Heavy regulations are eliminating the bulk of options and especially driving away the highest-quality private schools.

It should come as no surprise to anyone if we see some very disappointing academic outcomes in Louisiana’s voucher program.  A heavy regulation program that some major backers of school choice believe represents the “ideal” approach is actually designed to give us the worst outcomes.  If we do see bad results, the first impulse of the backers of heavy regulation will be to double-down on regulation.  They’ll wonder who the bad schools are and call for regulators to remove them from the program.

If education reform could be accomplished simply by identifying and closing bad schools while expanding good ones, everything could be fixed already without any need for school choice.  We would just issue regulations to forbid bad schools and to mandate good ones.  See?  Problem solved.  But real education reform requires using the power of choice and competition to provide incentives to create more good and to reduce bad.  The whole problem with the high-regulation approach is that it falsely believes regulators can define, identify, and require good outcomes.  If that were in fact possible, we would have already solved the problem and we could have done so without any school choice.  The enduring troubles of the traditional public system tell me that is not possible.

Some of the truths we cling to depend greatly upon our point of view…

October 3, 2015

(Guest Post by Matthew Ladner)

…especially if they have to do with per student funding in Arizona public schools, as I had the chance to explain in today’s Arizona Republic.


Protecting School Choice from the State

October 2, 2015

[Guest Post by Jason Bedrick]

As economists have understood for more than half a century, government agencies charged with regulating industries are often subject to regulatory capture. Rather than protect consumers from bad actors in the industries they were created to oversee, regulators too often develop cozy relationships with industry leaders and work at their behest to advance their interests. In Free to Choose, Milton and Rose Friedman detailed a particularly egregious example: the Interstate Commerce Commission (ICC).

Established in 1887, the ICC’s mission was to regulate the powerful railroad industry, which critics accused of engaging in cartel-like price fixing and market sharing. Instead, the railroad industry took almost immediate control of the ICC. The ICC’s first commissioner, Thomas Cooley, was a lawyer who had long represented the railroads and, as the Friedmans explained, many of the agency’s the bureaucrats “were drawn from the railroad industry, their day-to-day business tended to be with railroad people, and their chief hope of a lucrative future was with railroads.”

Rather than protect the consumers from the railroads, the ICC primarily protected the railroads. The ICC raised prices on consumers, shielded the railroads from state and local regulations, and even protected the railroads from outside competition. In the 1920s, the nascent trucking industry was emerging as the railroads’ most serious competitors. Like Uber against the taxi cartel, the lower-cost trucking industry benefited from the artificially high prices of the railroad cartel. And also like the taxi cartel, rather than seek deregulation, the railroad cartel turned to their friends in government to put the brakes on their “unregulated” competition. The bureaucrats all-too-happily complied. In 1933, the Motor Carrier Act gave the ICC authority over the trucking industry, which it used to limit the number of trucks that could operate on the roads and otherwise constrain the trucking industry. Fortunately, the railroad industry was significantly deregulated in the early 1980s and the ICC was abolished in 1995.

If regulatory capture is a common problem among the regulators of private industries, it can be even more acute when one government agency is overseeing other agencies. Bureaucrats at the various state departments of education tend to identify with the district schools they oversee and seek to protect them from outside competition. Lawmakers who support greater educational choice should keep this in mind when crafting choice policies. It is unwise to give an agency the power to regulate the primary competition to its core constituency.

Examples of state education agencies trying to undermine school choice initiatives abound. In Wisconsin, home to the nation’s first school voucher program, the Department of Public Instruction is currently subjecting private schools accepting vouchers to intrusive audits:

“I’ve been a CPA for 25 years and I’ve never seen anything like DPI’s approach to the audits of choice schools,” Noel Williams told Wisconsin Watchdog.

Williams, managing partner of Williams CPA in Milwaukee, has worked as an auditor for MPCP schools for 10 years. […]

“Although the law does allow DPI to follow-up with the auditor to clarify things that weren’t clear, in my opinion, DPI has grossly abused that power. Every one of the audit firms I’m acquainted with has gotten countless phone calls and emails on every audit report, requesting copies of data, clarification as to how they arrived at a conclusion.”

“It seems DPI’s intent has been to make it difficult and unpleasant to work with choice schools,” Williams said.

Indeed, the Wisconsin Institute for Law & Liberty and EAG News issued a report in 2013 detailing the Wisconsin DPI’s history of abuses against choice schools, including the use of audits to “harass and intimidate” them, withholding funding intended for private schools, forcing applicants to jump through hoops and provide lots of information and documentation not required by law, and more.

This summer, the Mississippi Department of Education initially limited the application period for the state’s new education savings account program to just 10 days in a move choice supporters called “unworkable” and “inconsistent with the law” which not only imposed no such application window, but actually stated that applications must be accepted on a rolling basis. The department eventually relented due to public outcry, but it’s unlikely to be the last of the DOE’s shenanigans.

Meanwhile, the New Hampshire Department of Education is trying to eliminate the town tuitioning program created by the village of Croydon (population 764). The village is too small to maintain its own school system so after 4th grade, it contracts with neighboring towns to provide schooling. In 2007, as its contract neared expiration, the village school board decided to pay for students to attend the district or private school of their family’s choice, similar to arrangements in nearby Vermont and Maine. However, the state DOE demanded that they cease and desist last fall. The town’s attorney, former NH supreme court justice Charles Douglas, argues that the state’s attempt to terminate the program is on shaky legal grounds. After nearly a year of negotiations, the state issued an ultimatum: terminate the program or the state will withhold its funding. The village is now trying to crowdfund a legal defense fund.

There are exceptions to the rule. The Florida Department of Education has a strong track record of supporting the state’s school choice programs. However, that could change as the political pendulum swings, as Indiana has demonstrated. And even where the top official at a state education agency support educational choice, there is no guarantee that the bureaucrats who preceded him or her will share that view or competently manage choice programs. Under John Huppenthal, Arizona’s DOE was ostensibly pro-school choice yet the agency repeatedly botched implementation of the state’s education savings account program.

Where some regulation or state program implementation is necessary, wise lawmakers have invested that authority in the state department of revenue, where the green-eyeshade bureaucrats are less likely to have an axe to grind against school choice than the state education establishment. Several states have already taken this approach, particularly for scholarship tax credit laws. Likewise, Nevada’s new education savings account is administered by the State Treasurer.

In “Fiddler on the Roof,” someone asks the town’s rabbi if there is a proper blessing for the tsar. “Of course!” replies the rabbi, “May G-d bless and keep the tsar… far away from us!” Educational choice policies should be similarly blessed.

[Originally posted at Cato-at-Liberty.]


Get every new post delivered to your Inbox.

Join 2,929 other followers