Justice, Equal Opportunity, Diversity and School Choice

March 21, 2017

Little-sprouts_-Grow-bean-sprouts-in-your-back-garden

(Guest post by Greg Forster)

This weekend I was delighted to participate in an hour-long school choice debate on Moody radio’s coast to coast network. The experience left me all the more convinced that choice advocates must continue to de-emphasize the rhetoric of markets and competition, and emphasize instead justice, equal opportunity, diversity and freedom. In that order.

The proponent of the other side had come armed with empty, superficial anti-market talking points. Her argument against choice was basically “we want justice, equal opportunity and diversity, not markets and competition.” So when I opened my case with “we want justice, equal opportunity, diversity and freedom, and here’s how school choice delivers them,” and didn’t use the words market or competition, she was flummoxed.

Her superficial talking points would have been highly effective if I had said “we want justice, equal opportunity, diversity and freedom so we ought to embrace competition and markets.” That is, sadly, because reason and logic are not the only forces in public debate. Her strategy (consciously or unconsciously) seems to have been to use certain trigger words and phrases to prompt emotional responses in the audience – responses unrelated to logic. This is, as JPGB readers know, the nearly universal strategy of choice opponents.

If we stop using the words that allow them to do this to us, we take their toy away.

Of course the quesiton of why choice improves public schools did come up, and here it was necessary to make the point that choice prevents schools from taking students for granted. The body of empirical studies on choice and what they find also came up a number of times. One can make these points without 1) making them the be-all and end-all, or 2) using the specific trigger words that allow the other side to work their emotional trickery.

A final point: personal experience, unfortunately, trumps data. I talked about having visited several private schools in Milwaukee whose existence depends on the voucher program, and the amazing things these particular schools are doing. Then I mentioned the studies finding that choice is impoving education in Milwaukee. I played this card again when a caller raised the inevitable racist talking point “parents in the suburbs are involved with their children’s education but people in those neighborhoods aren’t.” Instead of saying “the data show urban parents make good choices for their kids; that’s a stereotype we shouldn’t be spreading” I said “my experience in urban neighborhoods leads me to believe they care about their children as much as other parents; that’s a stereotype we shouldn’t be spreading.” And then I tried to squeeze in a mention of the data.

Of course if there were a clash between my experience and the data, I’d have to go with the data. But we have to limit our appeals to data when speaking in public. In general, we should present opinions based on experience and then briefly validate them with appeals to data.


The View Sure Looks Good from Here

January 19, 2017

h79yxk

(Guest post by Greg Forster)

My Win-Win findings read out loud on The View (check it out at the 9:30 mark).

I promise to remember y’all now that I’ve come into my kingdom.


Competition and Core Knowledge-Let’s Keep an Eye on a North Phoenix Neighborhood

June 3, 2016

(Guest Post by Matthew Ladner)

So I have been reading with interest the comment section discussion on Jay’s missed the point piece, which inspired me to follow up on a post I wrote in 2012. Your humble author took the above photograph of Shea Middle School in Arizona’s Paradise Valley School District. This was the neighborhood I used to live in, and I was zoned to send my children to this school (they were elementary age at the time). In the 2012 piece I relate circumstantial evidence that leads me to believe that the 9000 pt font banner you see in the photograph was prompted by the announcement of both a BASIS and a Great Hearts school opening in the area.

The following chart, making use of the AZ Merit results, gives a flavor of why long before the banner hung that my wife and I had already decided that we would explore other options for middle and high school.

MM Shea SM

Mercury Mine Elementary School feeds into Shea Middle School, which in turn feeds into Shadow Mountain High School. These schools are all within easy walking/biking distance from each other, with the middle and the high school being literally next door.  Mercury Mine had a larger number of open enrollment transfers attending when my children attended, which speaks well of the school’s reputation (the neighborhood had a larger number of empty nest residents so transfers were welcome.)  These schools operate in a relatively advantaged area overall, but the slippage over time from elementary to middle to high school scores is evident in both the AZ Merit and the older AIMS data. An examination of parent reviews in Great Schools failed to convince me that my lying eyes were leading me astray in examining test scores. I cast no aspersions here-I’m entirely confident that these schools have a great many dedicated people working very hard in them, but as a parent the trend you see in the chart above was gravely concerning.

So in 2012, BASIS opened a new school within walking/biking distance. What does that look like?

MM Shea SM Basis

BASIS has a dedicated band of internet detractors who complain that BASIS does this/that/and the other thing to pick and choose their students. Some of the enormous difference in scores you see here may indeed be due to factors other than great instruction and hard work by students, but as you can see there is a lot of room to give. Such arguments are ultimately moot without a random assignment study (which we lack) and the discussion is off mission. My intention in showing you this chart is to suggest why neighborhood school leaders seemed to be freaking out in the email to parents referenced in the 2012 post. Er, wouldn’t you?

Now this brings us back to Shea Middle School, which proudly and loudly adopted Core Knowledge starting in 2013.  Due to changes in the testing regime during this period, it is difficult to assess the overall direction of scores, but let’s stipulate that neither Core Knowledge nor the advent of competition proved to be a wonder cure than instantly transformed Shea Middle School overnight. That is not how the real world works after all. It could be the CK and competition will lead to incremental gains over time. Alternatively, maybe the losses to competition come to be viewed a sunk cost and the district schools fail to realize the potential of the curriculum.

Have a good summer- I’ll update this next year with new data and we’ll see how this little neighborhood experiment goes.

Disclosure: your humble author formerly served on the board of BASIS, and sends two of his three children to Great Hearts schools (although not the one mentioned in the post).


Overregulation Is All You Need: A Response to Paul Bruno

February 29, 2016

6-blind-men-hans

(Guest Post by Jason Bedrick)

Over at the Brookings Institution’s education blog, Paul Bruno offers a thoughtful critique of  Overregulation Theory (OT), the idea that government regulations on school choice programs can undermine their positive effects. Bruno argues that although OT is “one of the most plausible explanations” of the negative results that two studies of Louisiana’s voucher program recently found, it is not “entirely consistent with the available evidence” and “does not by itself explain substantial negative effects from vouchers.”

I agree with Bruno–and have stated repeatedly–that the studies’ findings do not conclusively prove OT. That said, I believe both that OT is consistent with the available evidence and that it could explain the substantial negative effects (though I think it’s likely there are other factors at play as well). I’ll explain why below, but first, a shameless plug:

On Friday, March 4th at noon, the Cato Institute will be hosting a debate over the impact of regulations on school choice programs featuring Patrick Wolf, Douglas Harris, Michael Petrilli, and yours truly, moderated by Neal McCluskey. If you’re in the D.C. area, please RSVP at this link and join us! Come for the policy discussion, stay for the sponsored lunch!

Is the evidence consistent with Overregulation Theory?

Bruno notes that the differences in enrollment trends between participating and non-participating private schools is consistent with OT. Participating schools had been experiencing declining enrollment in the decade before the voucher program was enacted whereas non-participating schools had slightly increasing enrollment on average. This is consistent with the OT’s prediction that better schools (which were able to maintain their enrollment or grow) would be more likely eschew the vouchers due to the significant regulatory burden, while the lower-performing schools (which were losing students) were more desperate for students and funding, and were therefore more willing to jump through the voucher program’s regulatory hoops. However, Bruno calls this evidence into question:

For one thing, the authors of the Louisiana study specifically check to see if learning outcomes vary significantly between schools experiencing greater or lesser prior enrollment declines, and find that they do not. (Bedrick acknowledges this, but doubts there was enough variation in the enrollment trends of participating schools to identify differences.)

We should be skeptical of the explanatory value of the study’s enrollment check. There is no good reason to assume that the correlation between enrollment growth or decline among the small sample of participating schools (which had significantly negative growth, on average) is the same as among all private schools in the state. Making such an assumption is like a blind man holding onto the truck of an elephant and assuming that he’s holding a snake.

The study does not show the variation in enrollment trends among the participating and non-participating schools, but we could imagine a scenario where the enrollment trend among participating schools ranged, say, from -25% to +5% while the range at non-participating schools was -5% to +25%. As shown in the following charts (which use hypothetical data), there may be a strong correlation between enrollment trends and outcomes among the entire population, while there is little correlation in the subset of participating schools.

Enrollment Growth and Performance, Participating Private Schools (Hypothetical)

Screen Shot 2016-02-29 at 4.57.05 PM

Enrollment Growth and Performance, All Private Schools (Hypothetical)

Screen Shot 2016-02-29 at 10.05.29 PM.png

In short, looking at the relationship between enrollment growth and performance in the narrow subset of participating schools doesn’t necessarily tell us anything about the relationship between enrollment growth and performance generally. Hence the study’s “check” that Bruno cites does not provide evidence against OT.

Is there evidence that regulations improve performance?

Bruno also cites evidence that regulations can have a positive impact on student outcomes:

Joshua Cowen of Michigan State University also points out that there is previous evidence of positive effects from accountability rules on voucher program outcomes in other states (though regulations may differ in Louisiana).

The Cowen article considers the impact of high-stakes testing imposed on the Milwaukee voucher program during a multi-year study of that program. The “results indicate substantial growth for voucher students in the first high-stakes testing year, particularly in mathematics, and for students with higher levels of earlier academic achievement.” But is this strong evidence that regulations improve performance? One of the authors of both the original Milwaukee study and the cited article–JPG all-star, Patrick Wolf–cautions against over-interpreting these results:

Ours is one study of what happened in one year for one school choice program that switched from low-stakes testing to high-stakes testing.  As we point out in the report, it is entirely possible that the surge in the test scores of the voucher students was a “one-off” due to a greater focus of the voucher schools on test preparation and test-taking strategies that year.  In other words, by taking the standardized testing seriously in that final year, the schools simply may have produced a truer measure of student’s actual (better) performance all along, not necessarily a signal that they actually learned a lot more in the one year under the new accountability regime.

If we had had another year to examine the trend in scores in our study we might have been able to tease out a possible test-prep bump from an effect of actually higher rates of learning due to accountability.  Our research mandate ended in 2010-11, sadly, and we had to leave it there – a finding that is enticing and suggestive but hardly conclusive.

It’s certainly possible that the high-stakes test improved actual learning. But it’s also possible–and, I would argue, more probable–that changing the stakes just meant that the schools responded to the new incentive by focusing more on test-taking strategies to boost their scores.

For that matter, even if it were true that the regulations actually improved student learning, that does not contradict Overregulation Theory. Both advocates and skeptics of the regulations believe that schools respond to incentives. Those of us who are concerned about the impact of the regulations don’t believe that they can’t improve performance. Rather, our concern is that regulations imposed from above are less effective at improving performance than the incentives created by direct accountability to parents in a robust market in education, and may have adverse unintended consequences.

To explain: We’re concerned that regulations forbidding the use of a school’s preferred admissions standards or requiring the state test (which is aligned to the state curriculum) might drive away better-performing schools, leaving parents to choose only among the lower-performing schools. We’re concerned that price controls will inhibit growth, providing schools with an incentive only to fill empty seats rather than to scale up. We’re concerned that mandatory state tests will inhibit innovation and induce conformity. None of these concerns rule out the possibility (or, indeed, the likelihood) that over time, requiring private schools to administer the state test and report the results and/or face sanctions based on test performance will improve the participating schools’ performance on that test.

Again: we agree that schools respond to incentives. We just think the results of top-down incentives are likely to be inferior to the results of bottom-up choice and competition, which have proved to be powerful tools in so many other fields for spurring innovation and improving quality.

Can Overregulation Theory alone explain the negative results in Louisiana?

Finally, Bruno questions whether OT alone explains the Louisiana results:

[E]ven if regulation prevented all but the worst private schools from participating, this would explain why students did not benefit from transferring into them, but not why students would transfer into them in the first place.

So Overregulation Theory might be part of the story in explaining negative voucher effects in Louisiana, but it is not by itself sufficient. To explain the results we see in the study, it is necessary to tell an additional story about why families would sort into these apparently inferior schools.

Bruno offers a few possible stories–that parents select schools “that provide unobserved benefits,” that the voucher program “induced families to select inferior schools,” or that parents merely “assume any private school must be superior to their available public schools”–but any of these can be consistent with OT.  Indeed, the second story Bruno offers is practically an extension of OT: if the voucher regulations truncate supply so that it is dominated by low-quality schools, and the government gives false assurances that they have vetted those schools, then it is likely that we will see parents lured into choosing inferior schools.

That’s not to say that there are no other factors causing the negative results. It’s likely that there are. (I find Douglas Harris’s argument that the private schools’ curricula did not align with the state test in the first year particularly compelling, though I don’t think it entirely explains the magnitude of the negative results.) We just don’t have any compelling evidence that OT is wrong, and OT can suffice to explain the negative results.

I will conclude as I began: expressing agreement. I concur with Bruno’s assessment that “it is likely that the existing evidence will not allow us to fully adjudicate between competing hypotheses.” Indeed, it’s likely that future evidence won’t be conclusive either (it rarely is), but I hope that further research will shed more light on this important question. Bruno concludes by calling for greater efforts to “understand how families determine where their children will be educated,” noting that by understanding how and why parents might make “sub-optimal — or even harmful” decisions will help “maximize the benefits of school choice while mitigating its risks.” These are noble goals and I share Bruno’s desire to pursue them. I just hope that policymakers will approach what we learn with a spirit of humility about what they can accomplish.


Competition Is Healthy for Public Schools

December 9, 2015

Competition Benefits Public Schools

[Guest Post by Jason Bedrick]

As more North Carolina families are using school vouchers, enrolling their children in charter schools, or homeschooling, some traditional district schools are experiencing slower growth in enrollment than anticipated. The News & Observer reports:

Preliminary numbers for this school year show that charter, private and home schools added more students over the past two years than the Wake school system did. Though the school system has added 3,880 students over the past two years, the growth has been 1,000 students fewer than projected for each of those years.

This growth at alternatives to traditional public schools has accelerated in the past few years since the General Assembly lifted a cap on the number of charter schools and provided vouchers under the Opportunity Scholarship program for families to attend private schools.

Opponents of school choice policies often claim that they harm traditional district schools. Earlier this year, the News & Observer ran an op-ed comparing choice policies to a “Trojan horse” and quoting a union official claiming that “public schools will be less able to provide a quality education than they have in the past” because they’re “going to be losing funds” and “going to be losing a great many of the students who are upper middle-class… [who] receive the most home support.”

Setting aside the benefits to the students who receive vouchers or scholarships (and the fact that North Carolina’s vouchers are limited to low-income students and students with special needs), proponents of school choice argue that the students who remain in their assigned district schools benefit from the increased competition. Monopolies don’t have to be responsive to a captive audience, but when parents have other alternatives, district schools must improve if they want to retain their students. But don’t take their word for it. Here’s what a North Carolina public school administrator had to say about the impact of increased competition:

New Wake County school board Chairman Tom Benton said the district needs to be innovative to remain competitive in recruiting and keeping families in North Carolina’s largest school system. At a time when people like choice, he said Wake must provide options to families.

“In the past, public schools could assign students to wherever they wanted to because parents couldn’t make a choice to leave the public schools,” Benton said. “Now we’re trying to make every school a choice of high quality so that parents don’t want to leave

Wake County is not unique in this regard. As readers of this blog surely know (and as I’ve written elsewhere), there have been 23 empirical studies investigating the impact of school choice laws on the students at district schools. As shown in the chart below, 22 of those studies found that the performance of students at district schools improved after a school choice law was enacted. One study found no statistically significant difference and none found any harm.

Academic Outcomes of Public Schools in Response to Competition

Beating district schools over the head with more and more top-down regulations has done little to improve quality. A better approach is bottom-up: empower parents with alternatives and give district schools the freedom to figure out how to provide a quality education that will persuade parents to choose them.

[A version of this post was originally published at Cato-at-Liberty. Hat tip to Dr. Terry Stoops of the John Locke Foundation for the story from New Wake County. Thanks to Bob Bowdon of Choice Media for the image.]


Response to Charter Competition

August 6, 2013

Check out the new article in Education Next by my current and former students, Marc Holley, Anna Egalite, and Marty Lueken on how traditional public school systems respond to competition from charters.  Using an innovative technique they gauge the types of responses exhibited by a dozen school districts.  While traditional districts engage in some non-constructive reactions, like focusing on blocking or regulating charters to minimize the competitive threat, overall they find districts rising to the challenge in positive ways.  In particular they are finding that districts often respond to charter competition by  replicating charter practices, collaborating with charters, developing innovative schools and programs, and expanding school offerings.

Of course, this analysis does not demonstrate that these positive reactions are resulting in improved student outcomes.  But responding positively is the first step that we hope will lead to better outcomes.  It is certainly a big change from earlier analyses that found districts focused almost exclusively on fighting and blocking charters.  It appears school districts have come to realize that charter competition is here to stay and it is best to try to rise to the challenge rather than squash the competition.


We Win Pop Culture! Also, a Podcast on Win-Win

May 2, 2013

Sci-Fi fest poster

(Guest post by Greg Forster)

In a major news development, today the Heartland Institute described JPGB as a “widely read education reform-pop culture blog.” After all these years of struggling for recognition as a major voice in the pop culture world, at long last our toil and struggle has been vindicated.

Oh, and they have this podcast I did on the Win-Win report showing that the research consistently supports school choice. If you’re, you know, into that kind of thing.

Win-Win 3.0 chart

In case you forgot what that column of zeros on the right looks like, here it is again.