Playing Ostrich About the Effects of Regulations

July 17, 2017


(Guest Post by Jason Bedrick)

There’s plenty to quibble with in Mark Dynarski’s post at EdNext this morning, but his claims about over-regulation theory are downright odd:

Some commenters have concluded that the negative effects in Louisiana were the consequence of the program being ‘over-regulated.’ [6] But the conclusion that the Louisiana program was overregulated relies on unstated premises that private schools that agreed to participate were academically inferior to ones that did not agree but would have if the state did not impose requirements, or that regulation itself impairs academic achievement. Evidence of either is noticeably lacking in the argument.

Far from being an “unstated premise,” the notion that “private schools that agreed to participate were academically inferior to ones that did not agree but would have if the state did not impose requirements” is the explicit argument that I was making in the EdNext post to which he links.

And far from “lacking evidence,” I spelled it out. First, private schools in highly regulated Louisiana were much less likely to accept voucher students than private schools in states with less regulated school choice programs:

Due to the LSP’s high regulatory burden, two-thirds of Louisiana private schools do not accept voucher students. In an American Enterprise Institute survey of private schools, 79 percent of Louisiana school leaders reported that concerns about program regulations played a deciding factor in their decision not to accept LSP students, including 64 percent who listed this as a major factor. In particular, 71 percent worried about the effect on their school’s admissions policies, including 45 percent who stated that this played a major role in their decision. In addition, 54 percent expressed concerns about administering the state test, including 34 percent who said it played a major role in their decision. Other areas of great concern included paperwork and the effect on the schools’ character or identity.

By contrast, the same survey found substantially lower levels of concerns about school choice regulations among school leaders in Indiana and Florida, where the regulatory burdens are considerably lower. While both states limit their vouchers and tax-credit scholarships to low-income students, they do not otherwise restrict admissions criteria, nor do they prevent schools from charging full tuition. Like Louisiana, Indiana requires schools to administer the state test to voucher students, whereas Florida allows schools to choose among many nationally norm-referenced tests.

Unsurprisingly, Florida has the highest level of private school participation among the three states (about two-thirds), followed by Indiana (about half), and Louisiana (one-third). Moreover, Florida schools are the most likely to plan to increase the number of choice students they enroll, while Louisiana schools are the most likely to decrease that number.

Second, there was “suggestive but not conclusive” evidence (as I wrote) that the private schools that did participate were lower performing than those that chose not to:

Low rates of private school participation would not be so troubling if they reflected the decisions of high-performing schools to accept voucher students while the regulations kept low-performing schools away, as proponents of the regulations had desired. However, the regulations may have had the opposite of their intended effect, as Professor Jay P. Greene of the University of Arkansas recently cautioned:

The only schools who are willing to do whatever the state tells them they must do are the schools that are most desperate for money. […] If you don’t have enough kids in your private school and your finances are in bad shape, you’re in danger of closing — probably because you’re not very good — then you’re willing to do whatever the state says.

Indeed, Greene’s concern is borne out by the data. According to the NBER study, “LSP schools open in both 2000 and 2012 experienced an average enrollment loss of 13 percent over this time period, while other private schools grew 3 percent on average.” The authors note that this “indicat[es] that the LSP may attract private schools struggling to maintain enrollment,” and they conclude that these results “suggest caution in the design of voucher systems aimed at expanding school choice for disadvantaged students.”

And, indeed, the recent study by Wolf, DeAngelis, and Sude lends further evidence to the Over-regulation Theory:

Our results largely confirm our hypothesis that higher tuition levels and larger cohort enrollments, conditions normally associated with high quality schools, identify schools that are less likely to participate in voucher programs. We also find a consistent negative relationship between Great Schools Review score and the school participation decision, indicating lower quality schools have a higher tendency of participating in voucher programs in all three states, however the coefficients are not significantly different from zero. State fixed effects reveal private schools in D.C. and Louisiana, the two states that have higher regulatory burdens, are less likely to participate in voucher programs.

The evidence is still merely suggestive, not conclusive, but it’s the best evidence we have. Dynarski might not be persuaded by it, but he can’t ignore that it exists.

Getting Facts Right

July 13, 2017

Image result for Everyone is entitled to his own opinion, but not his own facts.

Education policy debates, like national political disputes, are increasingly frustrating because we are having difficulty agreeing on and sticking to a set of facts.  It’s particularly frustrating because people are mis-stating basic facts that quite often are not even necessary for continuing to hold their opinions.  I’m not sure what accounts for this sloppiness in public discourse, but I suspect that it is a general decay in professional norms and standards of behavior encouraged by the stupid brevity and speed of Twitter.

Let me describe a recent example of an obvious, factual error from a debate about charter schools and regulations.  To be clear — there’s nothing wrong with making mistakes.  So what is remarkable about this example is how the people making the demonstrable, factual error never say “Oops, I made a mistake.”  They could easily admit the mistake and still continue to hold their opinion and make their arguments.  The fact that they don’t is what is truly troubling.

A Recent Example — Greg Richmond, who is the head of the National Association of Charter School Authorizers (NACSA), wrote a piece for The 74 in which he claimed: “A recent report from Tulane University’s Education Research Alliance found that a school’s rating from the independent application review process was the only factor that predicted its success.”  But if you look at the report, specifically in Table 7, you will see that the NACSA rating of charter school applications is not a significant predictor of any student outcome.  Richmond’s claim is demonstrably false.  He says that a report found something that it simply does not find.  I had previously noted the falsehood of this claim, but this has not deterred the folks at NACSA from repeating it.

Rather than admit the error, NACSA tries to rescue the claim.  Importantly, they don’t rescue the claim by showing that it is true.  They can’t.  Instead, they shift the discussion to a different point.  NACSA’s Nelson Smith, echoed by their spokesperson Kristen Forbriger, quote a policy brief written by Doug Harris summarizing the findings of the report he co-authored and that they falsely invoke:

“None of the application measures predict the value-added performance of schools, though there are signs of a positive relationship between the NACSA ratings and value-added (emphasis added). It is not surprising that our statistical confidence is weak here because value-added measures are imprecise and the NACSA ratings did not vary much among approved applications.” In other words, it’s hard to detect correlation to specific outcomes when the approved applications all scored at high levels.

Note that this does NOT say that the report “found that a school’s rating from the independent application review process was the only factor that predicted its success.”  In fact, it clearly states that Richmond’s original claim is false: “none of the application measures predict the value-added performance of schools.”  And saying there are “signs of a positive relationship” is just spin, not the actual finding.

Rather than support the truth of the original claim, this quotation provides a rationalization for why the report did not unearth the desired finding — the sample size was too small and there was too little variation in the NACSA rating.  Whether the rationalization is persuasive or not, the fact remains that the report did not find what Richmond claims it found.  His assertion is demonstrably false.  But neither he nor his colleagues at NACSA will say so.  Instead, in Trump-like fashion, they continue to assert that they were right all along even as the evidence contradicts them.

Unfortunately, this recent example of a factual error that is never acknowledged or corrected is becoming part of a pattern.  If people in our field can make demonstrably false claims without having to acknowledge or correct them, it’s unclear to me how we are going to make progress in policy debates.  I am not arguing here what the correct policy should be.  I’m simply arguing that if people in these debates can make false claims with impunity, we have allowed the Trumpization of debates to creep into the area of education policy and will suffer similar highly negative consequences.


If You Give the Feds a Cookie…

July 12, 2017


(Guest Post by Jason Bedrick)

Proponents of the Every Student Succeeds Act (ESSA) assured us that it would curb executive overreach by the U.S. Department of Education. Skeptics like Neal McCluskey and Lindsey Burke warned us that the law didn’t go nearly far enough. In their view, ESSA contained enough ambiguous language that a clever EdSec or her staff could mold it into practically anything.

It’s becoming increasingly clear that the skeptics were right. (Neal jokes that he was wrong, but he was only wrong about which ambiguous language the DOE would exploit first.) Earlier this month, the feds rejected Delaware’s ESSA compliance plan because aiming to cut the number of low-performing students in half by 2030 is “not ambitious enough.” You see, ESSA calls for “ambitious” goals, but doesn’t define the term “ambitious.” Acting assistant secretary Jason Botel explained to the New York Times, “Because the statute does not define the word ‘ambitious,’ the secretary has the responsibility of determining whether a state’s long-term goals are ambitious.”

As the skeptics said, give the feds an ambiguously worded cookie and soon they’ll be redefining that cookie to mandate that all children achieve proficiency in cookie baking by 2030. Or, y’know, something “ambitious” like that.

To Rick Hess, the whole thing sounds eerily similar to Soviet central planning:

General Secretary: Comrades, I call to order this meeting of Politburo Bureau of the Communist Party. We will start by turning to the implementation of our new Every Farmer Succeeds Act. Comrade Minister?

Minister of Agriculture Petrovsky: Thank you, comrade General Secretary. Comrades, you will recall that we adopted the Every Farmer Succeeds Act—EFSA, for short—after various complaints with the No Farmer Left Behind Act. That effort, initiated by the former regime, did not work out as intended.

Minister of the Interior Kirov: I’ll say, Mikhail! When we told farmers that they were to ensure that 100% of acres delivered adequate production, we got many complaints. We sent many to Siberia, but others still complained. Just to avoid running low on farmers, is good that we changed strategy.

Petrovsky: Thus, new Every Farmer Succeeds Act. We are working hard to implement. Instead of telling farmers that 100% of acres must produce adequate grain, we now tell them we realize world more complicated than that.

General Secretary: So, what is problem whereof you spoke?

Petrovksy: In giving collectives more flexibility, we still told them to set ambitious targets. Yes? Yet we have encountered resistance from some collectives. In its wisdom, the Politburo told farms they needed to be ambitious. However, my deputy, Comrade Botelinksi, informs me that some of the collectives are only proposing to double production. We have decided that production must go up tenfold. Anything less is not ambitious.

General Secretary: Da. And so?

Petrovsky: We have told collectives to change their plans. They must promise to increase production tenfold.

Kirov: But, is that not what caused problem with No Farmer Left Behind in first place? That no one took targets seriously?

Petrovsky: Comrade, do you want people to go hungry? And, that was last time. This time will be different.

Lesson: don’t give the feds any cookies. They make a big enough mess without the crumbs.



Pass the Popcorn: The Wedding Plan

July 6, 2017

Despite living in Fayetteville, AR, which is the greatest center of Yiddishkeit in all of the Ozarks, I had to travel to Boston to see the wonderful new film, The Wedding Plan, in a theater.  You should make an effort to see it if you live in a city where it is now showing and, if not, you should plan to see it when it arrives eventually on Netflix.

The Wedding Plan is the second film from writer/director Rama Burshtein, following her debut movie that I also loved and previously reviewed on JPGB, Fill the Void. Like the previous film, The Wedding Plan is set in an Orthodox Jewish community in Israel and focuses on questions of love and marriage.  But unlike Fill the Void, The Wedding Plan is not primarily about love and marriage.  It is really about faith and whether it is reasonable to expect that good things will happen — perhaps even miracles.  In this sense it is more like another fantastic film set in an Israeli Orthodox community, Ushpizin.

The premise of The Wedding Plan is that Michal is finally set to be married when her her fiance gets cold feet and backs out a month before the nuptials.  She has the dress and the wedding hall is reserved, so she decides to go ahead with the plan.  All she needs is a groom.  I know this sounds like the sort of Rom-Com plot that might feature Julia Roberts hilariously racing from one bad date to another until she finds Mr. Right just in time (and the American trailer has that feeling), but this movie is about much more.

The Hebrew title of the film means Through the Wall, which highlights a scene in which Michal explains that her plan to get married by the end of the month is like a karate chop.  She says she has to believe 100% to break through the wall.  If it is 99% she’ll break her hand.

Her friends and family fear that she will break her hand regardless.  They bring a rabbi to advise her. “What you are doing is counting on miracles,” he tells her. “Who gave you the right?” She answers quoting the Talmud: “The world was created for me.”  Is this just hubristic entitlement or should each of us feel like there is a plan for oneself in the world that includes enjoying the good?

Given the excessively cynical — even nihilistic — tilt of our culture and politics, it is worth considering whether we might be better off just having confidence that good things will somehow work out.  Or does believing this just set us up for even greater disappointment?  As Michal says, sitting alone in her wedding dress, “The bubble is about to burst.”  Should we protect ourselves from disappointment by not expecting anything good to come from the world?

These are questions that a Julia Roberts movie would not ask, but The Wedding Plan does.

And here is the American trailer:


What Went Right?

July 6, 2017

(Guest Post by Matthew Ladner)

I was one of the people who decided to read more about Islamic history after 9/11. Half way through Bernard Lewis’ book What Went Wrong? it occurred to me that Lewis had asked the wrong question. Lewis explored the question of why the Islamic world had gone from arguably the world’s leading civilization to a relative backwater. By the time this had happened however most of the world was a relative backwater in relation to western Europe. The more relevant question therefore in my mind is not “what went wrong” everywhere else but rather “what went right in the West?”

This thought came to mind when reading David Griffith’s paen to regulation in choice programs. David asked “Is there a state in the union with strong oversight, robust market supports, and a low-performing charter sector?” Actually yes there is- NACSA’s top ten state charter sectors are nine miles of bad road interrupted by a unique one-off in Louisiana towards the bottom of the ten.

Griffith writes “Yes, there are a few states where charters have achieved strong results despite a weak framework for intervening in low performing schools, or a dearth of quality authorizers, or limited parental supports. There is an exception to every rule.” Arizona, Colorado and Utah all display the high NAEP/low NACSA score combo. They are not alone btw. By “high NAEP” I mean “near or above Massachusetts scores.” By “low NACSA” I mean a score of 8 or 9 on the NACSA rating before the most recent NAEP. Other flourishing charter sectors, which display either some of these same types of rock star scores in the case of Florida, or else significant advantages over district performance in the case of DC, also dwell outside the top 10 NACSA rated charter laws.

Griffith seems to have mistaken the exception for the rule. It is a simple matter to point to multiple examples of the high NAEP/low NACSA score combo. The high NACSA/high NAEP combo is actually very rare. This is either because top rated states have charter sectors too small to meet NAEP reporting standards-like Indiana and Nevada- or just still struggling after all these years despite the benevolent regulation of the state like Texas.

Now it might be a coincidence that we see high NAEP/low NACSA combos aplenty in the 2015 NAEP. The 2017 NAEP will be released in October. I expect the data to show us more of the same, but time will tell. It could also be a coincidence that voucher programs in Louisiana and Indiana experienced unusually low private school participation rates and struggled academically in the early year evaluations. Some of us started sounding alarm bells on the participation rates before the test score evaluations became available. You don’t need a random assignment study to tell that something is wrong with a voucher program that 70% of private schools choose to avoid, just a bit of common sense. The random assignment studies then did tell us something was indeed wrong, and then a helpful survey of schools pretty much nailed down why it happened. Griffith seems to believe that the problem with LA vouchers is under-regulation. There seems to be an abundance of evidence however that the opposite is true.

So getting back to Lewis, I am convinced that the right question is “what went right in Arizona, Colorado, DC, Florida, Utah etc?” rather than just “what went wrong in Ohio?” Under what set of circumstances can parents take the lead in putting down undesired schools with brutal efficiency? What factors lead this to working in some jurisdictions, but flopping in others? Texas went down the high regulation road in 2001, and well…let’s just say it does not bode well for Ohio.

Even if my friends with a preference for high levels of regulation had evidence to suggest that their approach has benefits (currently lacking) their yearning to apply a one-size-fits-all approach on 50 states with wildly varying needs would still be unwise. Nevada for instance can take little comfort from their high NACSA rating as they continue to suffer extreme levels of public school overcrowding with only a few dozen charter schools. There are hundreds of thousands of children on charter school wait lists in neighboring states with more welcoming charter school laws-why would operators in the surrounding states give Nevada a second thought? This is not a game, and these policies have very real consequences. This fall I will be sending the three Ladner children back to two fantastic charter schools. If either of these schools slips I have other options. Also this fall uncounted thousands of Nevadans will be sending their kids to portable building to meet the first of what will be a series of substitute teachers for the year. These parents have little in the way of other options. What is the case for keeping things this way in Nevada?




Granite State Enacts Second School Choice Program

June 29, 2017


NH Gov. Chris Sununu speaks at a rally for school choice. (Credit: Union Leader)

(Guest Post by Jason Bedrick)

In a week that’s already loaded with good news for school choice, Gov. Chris Sununu of New Hampshire decided to add some more with the signing of a town tuitioning bill. Check out this blog post at EdChoice for more info:

SB 8 clarifies that any town—not just towns that share a border with other states—may include non-religious private schools in their town tuitioning plans. Town tuitioning, a longstanding tradition in New Hampshire and other New England states, occurs when a district “tuitions out” students to public or private schools in surrounding towns and districts because their home district does not provide schools with the grades those students need.

An equivalent bill was sent to the governor’s desk last year; however, then-Gov. Maggie Hassan vetoed it, claiming that a failure to exclude religious schools violated the state’s Blaine amendment. The current bill excludes religious schools, freeing legislators to deal with that issue in a separate fight.

The exclusion of religious schools is a serious defect, but I noted earlier this week, Blaine’s days might be numbered. The Live Free or Die State already has a tax-credit scholarship program and the legislature is currently considering a universal ESA.

Arizona ESA Momma Bear Mauls Arizona Republic in LTE

June 29, 2017

(Guest Post by Matthew Ladner)


I’ll give the Republic credit for running the letter, but totally agree with Mrs. Visser.