The Case for a Broader Approach to Education

March 9, 2016

The Americans for the Arts recently released a poll finding that the vast majority of Americans agree that “the arts are part of a well-rounded education for K-12 students.”  Over half of respondents “strongly agree.”  Unfortunately, the current trend in ed reform is out of sync with this popular support for the arts in education.

The narrow focus on math and reading achievement is driving out other subjects, including the arts.  Ed reformers may offer rhetorical praise for the arts and a broader education, but most quietly believe that math and reading are of such primary importance that shifting away from the arts to attend more to math and reading might actually be a good thing.

Like most other ed reformers, I used to believe this too.  But more research is beginning to show that a broader education, including the arts, may be essential for later success in math and reading as well as the proper development of civic values and character skills, including tolerance, empathy, and self-regulation.  The narrow focus on math and reading may goose math and reading test scores in the short term but at the expense of the longer-term and broader goals of education.  Parents seem to understand how essential the arts and a broader approach to education are even if this has escaped the highly-credentialed minds of “policy experts” trying to manage schools from afar through test results.

Let me briefly provide some evidence to support the claims above.  First, we have good reason to believe that the arts are being squeezed out of the curriculum.  For example, research by Daphna Bassok, Scott Latham, and Anna Rorem has found a dramatic decline in the role of the arts in school between 1998 and 2011.  Using teacher surveys from the ECLS-K, they find that far fewer teachers in Kindergarten and 1st Grade report teaching Music, Art, Dance, and Theater on a daily or weekly basis, and far more teachers report never covering these subjects at all.  (See their Table reproduced below)

Bassok Table

What evidence do we have that this shift away from the arts to focus more narrowly on math and reading has negative consequences?  David Grissmer and his colleagues are producing a series of studies that suggest how much later success in math, reading, and science depends on early acquisition of the kind of “general knowledge” and fine-motor skills learned through art and other subjects.

In one of these studies they find: “Whereas the early math and reading tests focused mainly on procedural knowledge, the general knowledge test focused mainly on declarative knowledge (i.e., elementary knowledge or comprehension of the external world). General knowledge was the strongest predictor of later reading and science and, along with earlier math, was a strong predictor of later math. General knowledge measured at kindergarten entrance may reflect early comprehension skills that are necessary when reading changes from a more procedural task in early grades (learning to read) to incorporating more comprehension around third through fifth grades (reading to learn).” This is essentially empirical support for the type of argument E.D. Hirsch, Robert Pondiscio, and the folks at Core Knowledge have been making for years.  It’s important for students to know a lot of things about the world, including about Art, History, etc…, to progress academically.  If we narrow education to the mechanics of math and reading as captured by yearly testing, we short-change the broader knowledge that is the key to academic success later.

Less intuitively, they find that the development of fine motor and other “visuo-spatial” skills are also very strong predictors of later academic success.  These are the kinds of things students learn by playing musical instruments or making art projects — activities disappearing from the early school curriculum.  Yet these fine-motor and coordination skills seem to be an important part of brain development that improves math and reading achievement years later.

Grissmer and his colleagues summarize the implications of their findings better than I could:

Our results suggest that the focus of interventions should shift from a primary emphasis on changing the direct math and reading instructional environment to interventions that build better foundational skills of attention and fine motor skills and a better understanding of the world outside schools. The results suggest that current direct math and reading instruction is insufficient to build attention and fine motor skills. Building these skills may rely more on subjects and curricula that have been deemphasized to provide more math and reading instruction: the arts, music, dance, physical education, and free play. Each of these subjects and curricula may need to be redesigned to focus on building foundational skills in the same way that math and reading have been redesigned in recent years. Building stronger knowledge of the external world also suggests that improving early science and social studies curricula are important. Paradoxically, higher long-term achievement in math and reading may require reduced direct emphasis on math and reading and more time and stronger curricula outside math and reading.

“Building stronger knowledge of the external world” might include going on culturally enriching field trips, which have also been disappearing from schools.  In addition to the general knowledge these experiences convey, the research I’ve done with others at the University of Arkansas on the effects of field trips to art museums and to see live theater suggests that these culturally enriching experiences change student values to promote greater tolerance and empathy.

Given that short-term gain in math and reading achievement are only weakly related to later life outcomes, while a broad education that includes the arts and culturally enriching activities may be associated with long-term success, education reformers should wonder whether they are simply rediscovering what most Americans already know — “the arts are part of a well-rounded education for K-12 students.”

(Edited for typos)

35 Comments | Uncategorized | Permalink
Posted by Jay P. Greene

Regulating School Choice: The Debate Continues

March 9, 2016

Design vs Experience

(Guest Post by Jason Bedrick)

Last week, the Cato Institute held a policy forum on school choice regulations (video here). Two of our panelists, Dr. Patrick Wolf and Dr. Douglas Harris, were part of a team that authored one of the recent studies finding that Louisiana’s voucher program had a negative impact on participating students’ test scores. Why that was the case – especially given the nearly unanimously positive previous findings – was the main topic of our discussion. Wolf and I argued that there is reason to believe that the voucher program’s regulations might have played a role in causing the negative results, while Harris and Michael Petrilli of the Fordham Institute pointed to other factors.

The debate continued after the forum, including a blog post in which Harris raises four “problems” with my arguments. I respond to his criticisms below.

The Infamous Education Productivity Chart

Problem #1: Trying to discredit traditional public schools by placing test score trends and expenditure changes on one graph. These graphs have been floating around for years. They purport to show that spending has increased much faster than expenditures [sic], but it’s obvious that these comparisons make no sense. The two things are on different scales. Bedrick tried to solve this problem by putting everything in percentage terms, but this only gives the appearance of a common scale, not the reality. You simply can’t talk about test scores in terms of percentage changes.

The more reasonable question is this: Have we gotten as much from this spending as we could have? This one we can actually answer and I think libertarians and I would probably agree: No, we could be doing much better than we are with current spending. But let’s be clear about what we can and cannot say with these data.

Harris offers a reasonable objection to the late, great Andrew Coulson’s infamous chart (shown below). Coulson already addressed critics of his chart at length, but Harris is correct that the test scores and expenditures do not really have a common scale. That said, the most important test of a visual representation of data is whether the story it tells is accurate. In this case, it is, as even Harris seems to agree. Adjusted for inflation, spending per pupil in public schools has nearly tripled in the last four decades while the performance of 17-year-olds on the NAEP has been flat.

U.S. Education Spending and Productivity

Producing a similar chart with data from the scores of younger students on the NAEP would be misleading because the scale would mask their improvement. But for 17-year-olds, whose performance has been flat on the NAEP and the SAT, the story the chart tells is accurate.

Voucher Regulations Are Keeping Private Schools Away

Problem #2: Repeating arguments that have already been refuted. Bedrick’s presentation repeated arguments about the Louisiana voucher case that I already refuted in a prior post. Neither the NBER study nor the survey by Pat Wolf and his colleagues provide compelling evidence that existing regulations are driving out potentially more effective private schools in the Louisiana voucher program, which was a big focus of the panel.

Here Harris attacks a claim I did not make. He is correct that there is no compelling evidence that regulations are driving out higher-quality private schools, but no one claimed that there was. Rather, I have repeatedly argued that the evidence was “suggestive but not conclusive” and speculated in my presentation that “if the enrollment trends are a rough proxy [for quality], though we can’t prove this, then it would suggest that the higher-quality schools chose not to participate” while lower-quality schools did.

Moreover, what Harris claims he refuted he actually merely disputed – and not very persuasively. In the previous post he mentions, he minimized the role that regulation played in driving away private schools:

As I wrote previously, the study he cites, by Patrick Wolf and colleagues, actually says that what private schools nationally most want changed is the voucher’s dollar value. In Louisiana, the authors reported that “the top concern was possible future regulations, followed by concerns about the amount of paperwork and reports. When asked about their concerns relating to student testing requirements, a number of school leaders expressed a strong preference for nationally normed tests” (italics added). These quotes give a very different impression that [sic] Bedrick states. The supposedly burdensome current regulations seem like less of a concern than funding levels and future additional regulations–and no voucher policy can ever insure against future changes in policy.

Actually, the results give a very different impression than Harris states. The quote Harris cites from the report is regarding the concerns of participating schools, but the question at hand is why the nonparticipating schools opted out of the voucher program. Future regulations was still the top concern for nonparticipating schools, but current regulations were also major concerns. Indeed, the study found that 9 of the 11 concerns that a majority of nonparticipating private schools said played a role their decision not to participate in the voucher program related to current regulations, particularly around admissions and the state test.

"Views from Private Schools," by Brian Kisida, Patrick J. Wolf, and Evan Rhinesmith, American Enterprise Institute

Source: ”Views from Private Schools,” by Brian Kisida, Patrick J. Wolf, and Evan Rhinesmith, American Enterprise Institute (page 19)

Nearly all of the nonparticipating schools’ top concerns related to the voucher program’s ban on private schools using their own admissions criteria (concerns 2, 3, 5, 7, 8 and 11) or requiring schools to administer the state test (concerns 6, 9, 10, and possibly 7 again). It is clear that these regulations played a significant role in keeping private schools away from the voucher program. The open question is whether the regulations were more likely to drive away higher-quality private schools. I explained why that might be the case, but I have never once claimed that we know it is the case.

Market vs. Government Regulations in Education

Problem #3: Saying that unregulated free markets are good in education because they have been shown to work in other non-education markets. […] For example, the education market suffers from perhaps the worst information problem of any market–many complex hard-to-measure outcomes most of which consumers (parents) cannot directly observe even after they’ve chosen a school for their child. Also, since students can realistically only attend schools near their homes, and there are economies of scale in running schools, that means there will generally be few practical options (unless you happen to live in a large city with great public transportation–very rare in the U.S.). And the transaction costs are very high to switch schools. And there are equity considerations. And … I could go on.

Harris claims that a free market in education wouldn’t work because education is uniquely different from other markets. However, the challenges he lists – information asymmetry, difficulty measuring intangible outcomes, difficulties providing options in rural areas, transaction costs for switching schools – aren’t unique to K-12 education at all. Moreover, there is no such thing as an “unregulated” free market because market forces regulate. As I describe below, while not perfect, these market forces are better suited than the government to address the challenges Harris raises.

Information asymmetry and hard-to-measure/intangible outcomes:

Parents need information in order to select quality education providers for their children. But are government regulations necessary to provide that information? Harris has provided zero evidence that it is, but there is much evidence to the contrary. Here the disparity between K-12 and higher education is instructive. Compared to K-12, colleges and universities operate in a relatively free market. Certainly, there are massive public subsidies, but they are mostly attached to students, and colleges have maintained meaningful independence. Even Pell vouchers do not require colleges to administer particular tests or set a single standard that all colleges must follow.

So how do families determine if a college is a good fit or not? There are three primary mechanisms they use: expert reviews, user reviews, and private certification.

The first category includes the numerous organizations that rate colleges, including U.S. News & World Report, the Princeton Review, Forbes, the Economist, and numerous others like them. These are similar to sorts of expert reviews, like Consumer Reports, that consumers regularly consult when buying cars, computers, electronics, or even hiring lawyers – all industries where the non-expert consumer faces a significant information asymmetry problem.

The second category includes the dozens of websites that allow current students and alumni to rate and review their schools. These are similar to Yelp, Amazon.com, Urban Spoon and numerous other platforms for end-users to describe their personal experience with a given product or service.

Finally, there are numerous national and regional accreditation agencies that certify that colleges meet a certain standard, similar to Underwriters Laboratories for consumer goods. This last category used to be private and voluntary, although now it is de factomandatory because accreditation is needed to get access to federal funds.

None of these are perfect, but then again, neither are government regulations. Moreover, the market-based regulators have at least four major advantages over the government. First, they provide more comprehensive information about all those hard-to-measure and intangible outcomes that Harris was concerned about. State regulators tend to measure only narrow and more objective outcomes, like standardized test scores in math and English or graduation rates. By contrast, the expert and user reviews consider return-on-investment, campus life, how much time students spend studying, teaching quality, professor accessibility, career services assistance, financial aid, science lab facilities, study abroad options, and much more.

Second, the diversity of options means parents and students can better identify the best fit for them. As Malcolm Gladwell observed, different people give different weights to different criteria. A family’s preferences might align better with the Forbes rankings than the U.S. News rankings, for example. Alternatively, perhaps no single expert reviewer captures a particular family’s preferences, in which case they’re still better off consulting several different reviews and then coming to their own conclusion. A single government-imposed standard would only make sense if there was a single best way to provide (or at least measure) education, we knew what it was, and there was a high degree of certainty that the government would actually implement it well. However, that is not the case.

Third, a plethora of private certifiers and expert and user reviews are less likely to create systemic perverse incentives than a single, government standard. As it is, the hegemony of U.S. News & World Report’s rankings created perverse incentives for colleges to focus on inputs rather than outputs, monkey around with class sizes, send applications to students who didn’t qualify to increase their “selectivity” rating, etc. If the government imposed a single standard and then rewarded or punished schools based on their performance according to that standard, the perverse incentives would be exponentially worse. The solution here is more competing standards, not a single standard.

Fourth, as Dr. Howard Baetjer Jr. describes in a recent edition of Cato Journal, whereas “government regulations have to be designed based on the limited, centralized knowledge of legislators and bureaucrats, the standards imposed by market forces are free to evolve through a constant process of evaluation and adjustment based on the dispersed knowledge, values, and judgment of everyone operating in the marketplace.” As Baetjer describes, the incentives to provide superior standards are better aligned in the market than for the government:

Incentives and accountability also play a central role in the superiority of regulation by market forces. First, government regulatory agencies face no competition from alternative suppliers of quality and safety assurance, because the regulated have no right of exit from government regulation: they cannot choose a better supplier of regulation, even if they want to. Second, government regulators are paid out of tax revenue, so their budget, job security, and status have little to do with the quality of the “service” they provide. Third, the public can only hold regulators to account indirectly, via the votes they cast in legislative elections, and such accountability is so distant as to be almost entirely ineffectual. These factors add up to a very weak set of incentives for government regulators to do a good job. Where market forces regulate, by contrast, both goods and service providers and quality-assurance enterprises must continuously prove their value to consumers if they are to be successful. In this way, regulation by market forces is itself regulated by market forces; it is spontaneously self-improving, without the need for a central, organizing authority.

In K-12, there are many fewer private certifiers, expert reviewers, or websites for user reviews, despite a significantly larger number of students and schools. Why? Well, first of all, the vast majority of students attend their assigned district school. To the extent that those schools’ outcomes are measured, it’s by the state. In other words, the government is crowding out private regulators. Even still, there is a small but growing number of organizations like GreatSchools, Private School Review, School Digger, andNiche that are providing parents with the information they desire.

Options in rural areas:

First, it should be noted that, as James Tooley has amply documented, private schools regularly operate – and outperform their government-run counterparts – even in the most remote and impoverished areas in the world, including those areas that lack basic sanitation or electricity, let alone public transportation. (For that matter, even the numerous urban slums where Tooley found a plethora of private schools for the poor lack the “great public transportation” that Harris claims is necessary for a vibrant education market.) Moreover, to the extent rural areas do, indeed, present challenges to providing education, such challenges are far from unique. Providers of other goods and services also must contend with reduced economies of scale, transportation issues, etc.

That said, innovations in communication and transportation mean these obstacles are less difficult to overcome than ever before. Blended learning and course access are already expanding educational opportunities for students in rural areas, and the rise of “tiny schools” and emerging ride-sharing operations like Shuddle (“Uber for kids”) may soon expand those opportunities even further. These innovations are more likely to be adopted in a free-market system than a highly government-regulated one.

Test Scores Matter But Parents Should Decide 

Problem #4: Using all this evidence in support of the free market argument, but then concluding that the evidence is irrelevant. For libertarians, free market economics is mainly a matter of philosophy. They believe individuals should be free to make choices almost regardless of the consequences. In that case, it’s true, as Bedrick acknowledged, that the evidence is irrelevant. But in that case, you can’t then proceed to argue that we should avoid regulation because it hasn’t worked in other sectors, especially when those sectors have greater prospects for free market benefits (see problem #3 above). And it’s not clear why we should spend a whole panel talking about evidence if, in the end, you are going to conclude that the evidence doesn’t matter.

Once again, Harris misconstrues what I actually said. In response to a question from Petrilli regarding whether I would support “kicking schools out of the [voucher] program” if they performed badly on the state test, I answered:

No, because I don’t think it’s a wise move to eliminate a school that parents chose, which may be their least bad option. We don’t know why a parent chose that school. Maybe their kid was being bullied at their local public school. Maybe their local public school that they were assigned to was not as good. Maybe there was a crime problem or a drug problem.

We’re never going to have a perfect system. Libertarians are not under the illusion that all private schools are good and all public schools are bad… Given the fact that we’ll never have a perfect system, what sort of mechanism is more likely to produce a wide diversity of options, and foster quality and innovation? We believe that the market – free choice among parents and schools having the ability to operate as they see best – has proven over and over again in a variety of industries to have better outcomes than Mike Petrilli sitting in an office deciding what quality is… as opposed to what individual parents think [quality] is.

Harris then responded by claiming that I was saying the evidence was “irrelevant,” to which I replied:

It’s irrelevent in terms of how we should design the policy, in terms of whether we should kick [schools] out or not, but I think it’s very important that we know how well these programs are working. Test scores do measure something. They are important. They’re not everything, but I think they’re a pretty decent proxy for quality…

In other words, yes, test scores matter. But they are far from the only things that matter. Test scores should be one of many factors that inform parents so that they can make the final decision about what’s best for their children, rather than having the government eliminate what might well be their least bad option based on a single performance measure.

I am grateful that Dr. Harris took the time both to attend our policy forum and to continue the debate on his blog afterward. I look forward to continued dialogue regarding our shared goal of expanding educational opportunity for all children.

(Cross-posted at Cato-at-Liberty.)

 

Leave a Comment » | research reports, Uncategorized, vouchers | Tagged: , , , , , , , | Permalink
Posted by Jason Bedrick

Virginia Legislature Passes Education Savings Accounts Bill

March 7, 2016

(Guest Post by Jason Bedrick)

Great news in Virginia via Virginia Delegate Dave LaRock’s office:

Parental Choice Education Savings Accounts HB389 has passed the Senate with a 20-19 vote, and is heading to the Governor!

Richmond, VA – Today, the Senate voted 20-19 to pass​ House Bill 389,​ which​ will​ establish Parental Choice Education Savings Accounts​ for children with special needs. After hearing impassioned debate​,​ ​legislators ​asking for​ help​ for​ children with special needs, who are sometimes ill-served in a public school setting​, won the day. The Governor will now decide if this will become law in Virginia.

Dave LaRock, the patron of House Bill 389 commented, “Education Savings Accounts offer educational opportunity to children with special needs, regardless of their circumstances in life. These unique children often face challenges which most will never fully appreciate. We owe it to them to provide access to academic resources best suited for their particular needs. We are working to lower the barriers so these kids can reach their full potential.”

LaRock added, “Parental Choice Education Savings Accounts are a proven example of problem solving through smart innovation, not endless appropriations. American voters overwhelmingly support school choice programs for good reason. It’s time to give the people of Virginia what they are asking for, more education freedom through Education Savings Accounts.” He added, “Most important of all, through these accounts, countless parents and children with disabilities in Virginia will be given the help they desperately need.”

Education Savings Accounts are supported by former Virginia Secretary of Education, Gerard Robinson; former U.S. Deputy Assistant Secretary of Education, Dr. Barry Stern; former Virginia State Board of Education Vice President and former Virginia Delegate, Winsome Sears; current and former School Board members from around the Commonwealth; as well as several candidates for statewide office in Virginia.

While the victories in both chambers of the legislature are deserving of Matt Ladner’s patented BOOOOOOOOOOOOOOOOOMs, the bill still faces one very large obstacle: Gov. Terry McAuliffe. Will he do what’s right for Virginia students with special needs? We shall soon find out…

 

1 Comment | Education Savings Accounts, Uncategorized | Tagged: , , | Permalink
Posted by Jason Bedrick

Why Relinquishment Would be a Good but Insufficient Idea for DCPS

March 7, 2016

(Guest Post by Matthew Ladner)

Andy Smarick has written about the next phase of reform for DC, calling for a relinquishment model. Given that DC charters receive a lot less revenue per pupil than DCPS and get substantially better results, this idea has merit.

Ladner DC 4

So scoring about a grade level worth of average progress higher for considerably less money constitutes an impressive accomplishment. However when we rank both DCPS and DC charters against the other urban districts in TUDA (for students with parents having similar education backgrounds) we find:

Ladner DC 5

The same pattern appears if you slice by family income etc. Not only to DC charters have a clear advantage over DCPS- they also demonstrate the only impressive gains for free and reduced lunch eligible students to be found in the NAEP data. Don’t go diving too far into the punch bowl however because those overall scores, even for charter students, are still trying to catch up to Mississippi:

Ladner DC 6

So- I agree with Andy that DC policymakers should consider pursuing an achievement district type strategy. DC policymakers and students should not turn their noses up at any strategy with a clear theory of action for improving outcomes at a lower cost. It’s not as if $29k per child in revenue will last a lifetime, but these terrible DCPS results will.

The data however seem to strongly suggest the approximate limits of such a strategy. We have no evidence as yet for instance suggesting that DC charter schools can produce a 20 percent boost in graduation rates the way DC OSP managed. Moreover, the status quo in DC is heading straight towards the extinction of private schools rather than utilizing their potential as partners. An achievement district approach would not have to finish this sad job as it looks likely to be finished long before.

DCPS has lost half of its students. It receives an unsustainable amount of revenue per pupil and does very little with it.  The appearance of improvement in the first figure above is mostly a mirage driven by gentrification. That same gentrification has helped to make DC the achievement gap capital of the nation as the one thing that DCPS has seemed to master is educating kids born on third base. Even if one mistakenly takes DC gains at face value, it still leaves DC ranked near the bottom as seen in the second figure and chart above.

As a resident of a distant patch of cactus, I don’t get a vote on any of this. DC residents will ultimately work this out for themselves. Call me crazy-it’s been too long since anyone has- but I would not be looking for incremental change.

 

 

 

3 Comments | Education Savings Accounts | Tagged: , , | Permalink
Posted by matthewladner

What If “Program Design” Misses the Point?

March 1, 2016

Bob+the+robber

Hi. My name is Bob. I’ll be your robber.

(Guest post by Greg Forster)

While we’re on the subject of “Overregulation Theory,” what if that’s a misnomer? What if we’re missing the point by drawing the lesson from Louisiana that “program design matters,” because the real problem isn’t design, it’s arbitrary threats by federal and state officials? The kind of thing you can’t stop with better program design?

I can’t seem to shake these possibly heretical thoughts, so I will submit them to my spiritual directors here on JPGB in hopes of recieving guidance.

I took a fresh look recently at one of the loci classicus for Overregulation Theory in Louisiana, Jason’s post “The Folly of Overregulating School Choice.” What struck me is how the “regulations” on the Louisiana program, while they are very bad regulations, are not so different from regulations on other school choice programs that have not failed this miserably. To participate in the voucher program, Louisiana schools give up control over admissions and price – but they do the same in Milwaukee and in other voucher programs. Why is Louisiana such a big loser when other programs operating under similar hindrances have been winners – moderate winners, to be sure, but still?

Then I noticed that in the AEI survey of Louisiana private school officials, the top reason for not participating in the program was not current regulations but “future regulations that might come with participation.” This was also the top concern about the program for school leaders whose schools were participating.

And then I took a fresh look at Lindsey Burke and Jonathan Butcher’s NRO article about Louisiana vouchers, and was reminded of a couple items that helped this all click for me:

  • The federal government attempted to gain sweeping authority over private schools participating in the voucher by manipulating desegregation law.
  • The state superintendent issued vague, arbitrary threats of interference with private schools based on his personal judgment of their performance.

I have not seen as much attention to that second point as I would have expected. Lindsey and Jonathan write:

Schools risk losing access to the Louisiana Scholarship Program if results on state tests “don’t meet expectations,” according to a letter from John White, Louisiana’s state superintendent of education. And, he adds, schools “are not permitted to accept new Scholarship students until their results align with program requirements.”

Now, I don’t think – but I’m open to hearing new information – that the Louisiana voucher program authorizes the state superintendent to swagger around making these kinds of intimidating statements to participating private schools. And that seems to me like a really big deal, much bigger than any aspect of program design as such.

It’s one thing to have a set of clear rules to follow. The rules may be stupid, but if you follow them, you’re safe. It’s quite another thing to be subject to vauge and arbitrary commands from a person in authority – commands that aren’t anchored in any clear set of rules, which could therefore be expected to change at any moment and be enforced based on whim. Or, more likely, malicious design.

As Al Copeland Nominee Charles Montesquieu explains, you cannot have freedom unless you have a well-grounded, reasonable expectation that you will continue to have freedom in the future, indefinitely. If it is even just somewhat possible that your freedom may be taken away tomorrow, you are not free today. You cannot plan your life as a free life; moreover, those who might become your rulers tomorrow can start nudging you to do things their way now. Why do you think some Republicans are selling their souls to Donald Trump?

Comparing the list of formal regulations in the Louisiana program with the federal attempt to gain sweeping power by climbing in the back window of desegregation law, and the vague and arbitrary threats from the state super, I know which would scare me more if I were running a private school.

Am I missing something? Does the state super have some stronger ground in the voucher law than I’m aware of for his threats? Or might these arbitrary intimidation tactics be something we should consider over and above “regulation” and “program design” as a cause of the Louisiana program’s failure?

I leave it to you, my spiritual directors, to guide me.

10 Comments | Uncategorized | Permalink
Posted by Greg Forster

Overregulation Is All You Need: A Response to Paul Bruno

February 29, 2016

6-blind-men-hans

(Guest Post by Jason Bedrick)

Over at the Brookings Institution’s education blog, Paul Bruno offers a thoughtful critique of  Overregulation Theory (OT), the idea that government regulations on school choice programs can undermine their positive effects. Bruno argues that although OT is “one of the most plausible explanations” of the negative results that two studies of Louisiana’s voucher program recently found, it is not “entirely consistent with the available evidence” and “does not by itself explain substantial negative effects from vouchers.”

I agree with Bruno–and have stated repeatedly–that the studies’ findings do not conclusively prove OT. That said, I believe both that OT is consistent with the available evidence and that it could explain the substantial negative effects (though I think it’s likely there are other factors at play as well). I’ll explain why below, but first, a shameless plug:

On Friday, March 4th at noon, the Cato Institute will be hosting a debate over the impact of regulations on school choice programs featuring Patrick Wolf, Douglas Harris, Michael Petrilli, and yours truly, moderated by Neal McCluskey. If you’re in the D.C. area, please RSVP at this link and join us! Come for the policy discussion, stay for the sponsored lunch!

Is the evidence consistent with Overregulation Theory?

Bruno notes that the differences in enrollment trends between participating and non-participating private schools is consistent with OT. Participating schools had been experiencing declining enrollment in the decade before the voucher program was enacted whereas non-participating schools had slightly increasing enrollment on average. This is consistent with the OT’s prediction that better schools (which were able to maintain their enrollment or grow) would be more likely eschew the vouchers due to the significant regulatory burden, while the lower-performing schools (which were losing students) were more desperate for students and funding, and were therefore more willing to jump through the voucher program’s regulatory hoops. However, Bruno calls this evidence into question:

For one thing, the authors of the Louisiana study specifically check to see if learning outcomes vary significantly between schools experiencing greater or lesser prior enrollment declines, and find that they do not. (Bedrick acknowledges this, but doubts there was enough variation in the enrollment trends of participating schools to identify differences.)

We should be skeptical of the explanatory value of the study’s enrollment check. There is no good reason to assume that the correlation between enrollment growth or decline among the small sample of participating schools (which had significantly negative growth, on average) is the same as among all private schools in the state. Making such an assumption is like a blind man holding onto the truck of an elephant and assuming that he’s holding a snake.

The study does not show the variation in enrollment trends among the participating and non-participating schools, but we could imagine a scenario where the enrollment trend among participating schools ranged, say, from -25% to +5% while the range at non-participating schools was -5% to +25%. As shown in the following charts (which use hypothetical data), there may be a strong correlation between enrollment trends and outcomes among the entire population, while there is little correlation in the subset of participating schools.

Enrollment Growth and Performance, Participating Private Schools (Hypothetical)

Screen Shot 2016-02-29 at 4.57.05 PM

Enrollment Growth and Performance, All Private Schools (Hypothetical)

Screen Shot 2016-02-29 at 10.05.29 PM.png

In short, looking at the relationship between enrollment growth and performance in the narrow subset of participating schools doesn’t necessarily tell us anything about the relationship between enrollment growth and performance generally. Hence the study’s “check” that Bruno cites does not provide evidence against OT.

Is there evidence that regulations improve performance?

Bruno also cites evidence that regulations can have a positive impact on student outcomes:

Joshua Cowen of Michigan State University also points out that there is previous evidence of positive effects from accountability rules on voucher program outcomes in other states (though regulations may differ in Louisiana).

The Cowen article considers the impact of high-stakes testing imposed on the Milwaukee voucher program during a multi-year study of that program. The “results indicate substantial growth for voucher students in the first high-stakes testing year, particularly in mathematics, and for students with higher levels of earlier academic achievement.” But is this strong evidence that regulations improve performance? One of the authors of both the original Milwaukee study and the cited article–JPG all-star, Patrick Wolf–cautions against over-interpreting these results:

Ours is one study of what happened in one year for one school choice program that switched from low-stakes testing to high-stakes testing.  As we point out in the report, it is entirely possible that the surge in the test scores of the voucher students was a “one-off” due to a greater focus of the voucher schools on test preparation and test-taking strategies that year.  In other words, by taking the standardized testing seriously in that final year, the schools simply may have produced a truer measure of student’s actual (better) performance all along, not necessarily a signal that they actually learned a lot more in the one year under the new accountability regime.

If we had had another year to examine the trend in scores in our study we might have been able to tease out a possible test-prep bump from an effect of actually higher rates of learning due to accountability.  Our research mandate ended in 2010-11, sadly, and we had to leave it there – a finding that is enticing and suggestive but hardly conclusive.

It’s certainly possible that the high-stakes test improved actual learning. But it’s also possible–and, I would argue, more probable–that changing the stakes just meant that the schools responded to the new incentive by focusing more on test-taking strategies to boost their scores.

For that matter, even if it were true that the regulations actually improved student learning, that does not contradict Overregulation Theory. Both advocates and skeptics of the regulations believe that schools respond to incentives. Those of us who are concerned about the impact of the regulations don’t believe that they can’t improve performance. Rather, our concern is that regulations imposed from above are less effective at improving performance than the incentives created by direct accountability to parents in a robust market in education, and may have adverse unintended consequences.

To explain: We’re concerned that regulations forbidding the use of a school’s preferred admissions standards or requiring the state test (which is aligned to the state curriculum) might drive away better-performing schools, leaving parents to choose only among the lower-performing schools. We’re concerned that price controls will inhibit growth, providing schools with an incentive only to fill empty seats rather than to scale up. We’re concerned that mandatory state tests will inhibit innovation and induce conformity. None of these concerns rule out the possibility (or, indeed, the likelihood) that over time, requiring private schools to administer the state test and report the results and/or face sanctions based on test performance will improve the participating schools’ performance on that test.

Again: we agree that schools respond to incentives. We just think the results of top-down incentives are likely to be inferior to the results of bottom-up choice and competition, which have proved to be powerful tools in so many other fields for spurring innovation and improving quality.

Can Overregulation Theory alone explain the negative results in Louisiana?

Finally, Bruno questions whether OT alone explains the Louisiana results:

[E]ven if regulation prevented all but the worst private schools from participating, this would explain why students did not benefit from transferring into them, but not why students would transfer into them in the first place.

So Overregulation Theory might be part of the story in explaining negative voucher effects in Louisiana, but it is not by itself sufficient. To explain the results we see in the study, it is necessary to tell an additional story about why families would sort into these apparently inferior schools.

Bruno offers a few possible stories–that parents select schools “that provide unobserved benefits,” that the voucher program “induced families to select inferior schools,” or that parents merely “assume any private school must be superior to their available public schools”–but any of these can be consistent with OT.  Indeed, the second story Bruno offers is practically an extension of OT: if the voucher regulations truncate supply so that it is dominated by low-quality schools, and the government gives false assurances that they have vetted those schools, then it is likely that we will see parents lured into choosing inferior schools.

That’s not to say that there are no other factors causing the negative results. It’s likely that there are. (I find Douglas Harris’s argument that the private schools’ curricula did not align with the state test in the first year particularly compelling, though I don’t think it entirely explains the magnitude of the negative results.) We just don’t have any compelling evidence that OT is wrong, and OT can suffice to explain the negative results.

I will conclude as I began: expressing agreement. I concur with Bruno’s assessment that “it is likely that the existing evidence will not allow us to fully adjudicate between competing hypotheses.” Indeed, it’s likely that future evidence won’t be conclusive either (it rarely is), but I hope that further research will shed more light on this important question. Bruno concludes by calling for greater efforts to “understand how families determine where their children will be educated,” noting that by understanding how and why parents might make “sub-optimal — or even harmful” decisions will help “maximize the benefits of school choice while mitigating its risks.” These are noble goals and I share Bruno’s desire to pursue them. I just hope that policymakers will approach what we learn with a spirit of humility about what they can accomplish.

2 Comments | competitive effects, economics, methodology, research reports, school accountability, vouchers | Tagged: , , , , , | Permalink
Posted by Jason Bedrick

Great Spoof, Kid!

February 26, 2016

(Guest post by Greg Forster)

If you liked Emo Kylo Ren you’ll love this. You’re welcome.

Leave a Comment » | Uncategorized | Permalink
Posted by Greg Forster

NAEP Reading Gains and Spending Trends by State

February 26, 2016

(Guest Post by Matthew Ladner)

2003 to 2015 NAEP reading by spending

New York and North Dakota are your team captains of the Washington Generals in the spending up NAEP scores down sweepstakes. Meanwhile out west AZ and California did more with less.

Leave a Comment » | NAEP | Tagged: | Permalink
Posted by matthewladner

NAEP Gains by Spending Trend- Some States are the Harlem Globetrotters, Others the Washington Generals

February 24, 2016

(Guest Post by Matthew Ladner)

Notice the large number of states on the negative side of zero on the spending bar. In the immortal words of Lee Ving it’s already started.

Winning

Once again let me note with insufferable state pride that Arizona is your ROI champion doing things that no one would have thought possible.

Go majority-minority, cut funding and improve scores? No problem-nothing but net!

So it is tough to pick a Washington Generals, but I’ve got mine narrowed down between Alaska, New York and Wyoming.

 

Leave a Comment » | charter schools, Demographic change, school funding | Tagged: , | Permalink
Posted by matthewladner

Arizona Senate Votes to Phase In Universal Public School Student ESA Eligiblity

February 22, 2016

(Guest Post by Matthew Ladner)

Congrats to Senator Debbie Lesko and the band of happy school choice warriors for passing SB 1279 today, which would phase in making all Arizona public school students eligible to participate in the Empowerment Scholarship Account program.

3 Comments | Education Savings Accounts | Tagged: , , | Permalink
Posted by matthewladner

« Previous Entries
Next Entries »