Jay P. Greene's Blog

Image: “Rally in Tally” to support school choice on March 24, 2011.

(Guest Post by Jason Bedrick)

I still frequently see the canard that “poor families can’t benefit from school choice programs.”

Oh really? Tell that to the 100,000+ kids from low-income families in Florida receiving tax-credit scholarships through Step Up for Students. Average family income: $24,000.

Also, contra the false “minorities don’t benefit” narrative, about seven in 10 scholarship students are black or Hispanic.

Chutzpah is jokingly defined as murdering one’s parents and then complaining about being an orphan. Atila Abdulkadiroglu, Parag Pathak, and Christopher Walters ( hereafter APW or the MIT team) sure show some chutzpah when complaining about not having continued access to data regarding the Louisiana Scholarshp Program (LSP) in a recent article. While I don’t know for sure why they were denied continued access to data, I believe that it is related to their rush to release 1st year results from their evaluation. Why they were rushing is an incredibly depressing story about how status and power in our field contributes to academic abuse and dishonesty– a story the reporter who wrote the article entirely missed.

It is not widely known or acknowledged, but the original analysis of 1st year result from LSP was conducted by Jonathan Mills when he was a doctoral student along with his advisor, Patrick Wolf, at the University of Arkansas.  They presented those findings at academic conferences 8 times during 2014 and 2015 and they were contained in Jon’s dissertation published in July 2015. APW were at some of those conferences.  Atila actually had lunch at one conference with Jon and Pat during which they discussed that study in June of 2015.  Atila never indicated that he was conducting or planning to conduct a similar study.  He offered to help and they sent him some materials.  He never responded with help but he did move forward with his own study with the MIT team without informing Pat or Jon that they were doing so.

APW released their own study as an NBER report in December 2015.  Nowhere in that report did they acknowledge or cite Jon and Pat’s earlier work of which they were almost certainly aware, having discussed it with them. Nor did APW acknowledge that their study was essentially a replication of Jon and Pat’s earlier study. The research designs were nearly identical.  The data were almost the same.  The only difference was that Jon and Pat had a more complete data set and as a result reported more negative results.

That’s right.  Jon and Pat had more negative results.  They released those results along with the negative 2nd year results in February 2016.  So the fact that Jon and Pat continued getting access to LA data while APW did not does not appear to have anything to do with reporting negative results.  It seems to be related to the fact that APW were rushing to release results.  They didn’t take the time like Jon and Pat did to solve missing data issues.  Instead they were determined to move fast to get their results out first.

Why did  it matter that they be first?  By being first to release they could act like they had the original analysis rather than a replication.  Top Econ journals tend not to be as  interested in replications of a grad student’s dissertation.  And by being first to release and not citing Jon’s work they could act like theirs was the original analysis.

Failing to credit and cite earlier work is a form of academic fraud.  I have not come forward earlier with this story because Jon was entering the academic job market and did not want to get on the wrong side of high status and powerful people in the field.  Pat and I, as his advisers, deferred to his wishes and remained quiet.  Now that Jon has a secure job ( with us) and a news article wrongly implies that APW were denied access because (presumably unlike us) they wouldn’t withhold negative result, I felt compelled to tell this story.  It’s an ugly one.

UPDATE: Pat Wolf checked his records and found that he also had a discussion at a conference in April 2015 with Atila regarding the Louisiana evaluation that he and Jon were doing. The materials he sent, however, were following that conversation, not following the June conversation as Pat had earlier remembered, and those materials were not directly related to the study.  In any event, it is clear from multiple conversations and multiple conference presentations that APW were aware of the existence of prior research.

2nd Update:  APW have a statement here.  My response to it is here.

(Edited for typos and to add links)

Read the rest of this entry »

(Guest Post by Jason Bedrick)

Last month, as detailed here, the U.S. Department of Education rejected Delaware’s ESSA plan for being insufficiently “ambitious.” You see, Delaware was merely attempting to do something that no state had ever accomplished before.

The administration’s actions flew in the face of their frequently stated commitment to federalism. Now, however, it seems they have reversed course:

After some serious drama, U.S. Secretary of Education Betsy DeVos on Tuesday gave Delaware the green light for its Every Student Succeeds Act plan.

You read that right. Delaware, aka the state whose Feedback Shook the World, is the first state to get the all-clear to proceed on ESSA.

What drama are we talking about? Here’s some quick background: DeVos had been hitting the local control theme hard in speeches since taking office. But her team’s response to the submitted plan from Delaware, one of the first states to get ESSA plan feedback from the Trump education department, seemed out of line with that rhetoric.

The department questioned the ambitiousness of the First State’s student achievement goals and criticized the state for wanting to use Advanced Placement tests to gauge college and career readiness. (The department said this was a no-go because the tests and courses aren’t available in every school.)

That got many important people pretty upset, including Sen. Lamar Alexander, R-Tenn., the Senate education chairman and an ESSA architect. Chris Minnich, the executive director of the Council of Chief State School Officers, also said he was disappointed. Both said that DeVos’ team had essentially overstepped the bounds of the law.

In a press release, Secretary DeVos noted that she believed Delaware’s plan “adhered to the law” but she stopped short of recommitting her department to the principle of federalism:

“Delaware has always been a state of firsts, so it should be no surprise that theirs was both the first state plan submitted and the first approved under ESSA,” said Secretary DeVos.

“My criteria for approval is clear: does the state’s plan adhere to the law? Delaware demonstrated their plan does, and so I am happy to approve it. I hope it will give the students, families and educators in the state a strong foundation for a great education.

“Throughout the process, Delaware’s leaders have been terrific partners. I want to thank Gov. Carney, Secretary of Education Bunting and State Board President Loftus for their work and collaboration on putting forth a plan that embraces ESSA’s spirit of flexibility and creative thinking.”

All in all, this is a positive development. Nevertheless, this episode should serve to remind education reformers that even an administration that talks the federalist talk doesn’t necessarily always walk the walk. Those who respect subsidiarity and value local control — especially those who understand that the most local form of control is in the hands of parents — have good reason to be wary about giving the feds any power.

(Guest post by Greg Forster)

OCPA’s Perspective carries my new article on school choice and segregation. I wrote it before the recent silliness from our friends at CAP, so my expectation that the recent increase in focus on this issue would only continue is holding up well so far:

The accusation that school choice will increase ethnic segregation in schools, after a long period on the rhetorical back-burner (during the age of test-score obsessions), has suddenly returned to the forefront of public debate. That’s no surprise, given rising levels of ethnic tension and polarization.

Last time the other side tried to make hay out of this, it failed, largely because the empirical evidence we have on this question is in favor of choice:

In fact, that body of research is the reason it’s been a while since we heard much talk about segregation in the debate over school choice. I remember hearing this talking point much more in the early 2000s, when fewer of these studies had been done. As the evidence piled up, the talking point went away.

In the new article I go into some of the politics of schools and ethnic segregation, arguing that while “most parents aren’t racist” is one possible explanation of the evidence for choice, you can also believe race is an important factor in school selection and still believe school choice will reduce ethnic segregation as compared with the status quo:

To whatever extent parents are racist, consciously or unconsciously, the government monopoly system is perfectly designed to cater to those racist preferences. Segregation flourishes under the government monopoly, both because schools are tied to ZIP codes and because power brokers draw the attendance lines.

So the strongest argument for choice is not “parents aren’t racist.” It’s “under the government monopoly, segregation happens by default, regardless of what parents prefer; only school choice creates the opportunity for integration.”

Check it out and let me know what you think!

(Guest Post by Matthew Ladner)

Shot from Denver of some of my favorite folks…

(Guest Post by Jason Bedrick)

There’s plenty to quibble with in Mark Dynarski’s post at EdNext this morning, but his claims about over-regulation theory are downright odd:

Some commenters have concluded that the negative effects in Louisiana were the consequence of the program being ‘over-regulated.’ [6] But the conclusion that the Louisiana program was overregulated relies on unstated premises that private schools that agreed to participate were academically inferior to ones that did not agree but would have if the state did not impose requirements, or that regulation itself impairs academic achievement. Evidence of either is noticeably lacking in the argument.

Far from being an “unstated premise,” the notion that “private schools that agreed to participate were academically inferior to ones that did not agree but would have if the state did not impose requirements” is the explicit argument that I was making in the EdNext post to which he links.

And far from “lacking evidence,” I spelled it out. First, private schools in highly regulated Louisiana were much less likely to accept voucher students than private schools in states with less regulated school choice programs:

Due to the LSP’s high regulatory burden, two-thirds of Louisiana private schools do not accept voucher students. In an American Enterprise Institute survey of private schools, 79 percent of Louisiana school leaders reported that concerns about program regulations played a deciding factor in their decision not to accept LSP students, including 64 percent who listed this as a major factor. In particular, 71 percent worried about the effect on their school’s admissions policies, including 45 percent who stated that this played a major role in their decision. In addition, 54 percent expressed concerns about administering the state test, including 34 percent who said it played a major role in their decision. Other areas of great concern included paperwork and the effect on the schools’ character or identity.

By contrast, the same survey found substantially lower levels of concerns about school choice regulations among school leaders in Indiana and Florida, where the regulatory burdens are considerably lower. While both states limit their vouchers and tax-credit scholarships to low-income students, they do not otherwise restrict admissions criteria, nor do they prevent schools from charging full tuition. Like Louisiana, Indiana requires schools to administer the state test to voucher students, whereas Florida allows schools to choose among many nationally norm-referenced tests.

Unsurprisingly, Florida has the highest level of private school participation among the three states (about two-thirds), followed by Indiana (about half), and Louisiana (one-third). Moreover, Florida schools are the most likely to plan to increase the number of choice students they enroll, while Louisiana schools are the most likely to decrease that number.

Second, there was “suggestive but not conclusive” evidence (as I wrote) that the private schools that did participate were lower performing than those that chose not to:

Low rates of private school participation would not be so troubling if they reflected the decisions of high-performing schools to accept voucher students while the regulations kept low-performing schools away, as proponents of the regulations had desired. However, the regulations may have had the opposite of their intended effect, as Professor Jay P. Greene of the University of Arkansas recently cautioned:

The only schools who are willing to do whatever the state tells them they must do are the schools that are most desperate for money. […] If you don’t have enough kids in your private school and your finances are in bad shape, you’re in danger of closing — probably because you’re not very good — then you’re willing to do whatever the state says.

Indeed, Greene’s concern is borne out by the data. According to the NBER study, “LSP schools open in both 2000 and 2012 experienced an average enrollment loss of 13 percent over this time period, while other private schools grew 3 percent on average.” The authors note that this “indicat[es] that the LSP may attract private schools struggling to maintain enrollment,” and they conclude that these results “suggest caution in the design of voucher systems aimed at expanding school choice for disadvantaged students.”

And, indeed, the recent study by Wolf, DeAngelis, and Sude lends further evidence to the Over-regulation Theory:

Our results largely confirm our hypothesis that higher tuition levels and larger cohort enrollments, conditions normally associated with high quality schools, identify schools that are less likely to participate in voucher programs. We also find a consistent negative relationship between Great Schools Review score and the school participation decision, indicating lower quality schools have a higher tendency of participating in voucher programs in all three states, however the coefficients are not significantly different from zero. State fixed effects reveal private schools in D.C. and Louisiana, the two states that have higher regulatory burdens, are less likely to participate in voucher programs.

The evidence is still merely suggestive, not conclusive, but it’s the best evidence we have. Dynarski might not be persuaded by it, but he can’t ignore that it exists.