The Chutzpah of Abdulkadiroglu, Pathak, and Walters

August 4, 2017

Chutzpah is jokingly defined as murdering one’s parents and then complaining about being an orphan. Atila Abdulkadiroglu, Parag Pathak, and Christopher Walters ( hereafter APW or the MIT team) sure show some chutzpah when complaining about not having continued access to data regarding the Louisiana Scholarshp Program (LSP) in a recent article. While I don’t know for sure why they were denied continued access to data, I believe that it is related to their rush to release 1st year results from their evaluation. Why they were rushing is an incredibly depressing story about how status and power in our field contributes to academic abuse and dishonesty– a story the reporter who wrote the article entirely missed.

It is not widely known or acknowledged, but the original analysis of 1st year result from LSP was conducted by Jonathan Mills when he was a doctoral student along with his advisor, Patrick Wolf, at the University of Arkansas.  They presented those findings at academic conferences 8 times during 2014 and 2015 and they were contained in Jon’s dissertation published in July 2015. APW were at some of those conferences.  Atila actually had lunch at one conference with Jon and Pat during which they discussed that study in June of 2015.  Atila never indicated that he was conducting or planning to conduct a similar study.  He offered to help and they sent him some materials.  He never responded with help but he did move forward with his own study with the MIT team without informing Pat or Jon that they were doing so.

APW released their own study as an NBER report in December 2015.  Nowhere in that report did they acknowledge or cite Jon and Pat’s earlier work of which they were almost certainly aware, having discussed it with them. Nor did APW acknowledge that their study was essentially a replication of Jon and Pat’s earlier study. The research designs were nearly identical.  The data were almost the same.  The only difference was that Jon and Pat had a more complete data set and as a result reported more negative results.

That’s right.  Jon and Pat had more negative results.  They released those results along with the negative 2nd year results in February 2016.  So the fact that Jon and Pat continued getting access to LA data while APW did not does not appear to have anything to do with reporting negative results.  It seems to be related to the fact that APW were rushing to release results.  They didn’t take the time like Jon and Pat did to solve missing data issues.  Instead they were determined to move fast to get their results out first.

Why did  it matter that they be first?  By being first to release they could act like they had the original analysis rather than a replication.  Top Econ journals tend not to be as  interested in replications of a grad student’s dissertation.  And by being first to release and not citing Jon’s work they could act like theirs was the original analysis.

Failing to credit and cite earlier work is a form of academic fraud.  I have not come forward earlier with this story because Jon was entering the academic job market and did not want to get on the wrong side of high status and powerful people in the field.  Pat and I, as his advisers, deferred to his wishes and remained quiet.  Now that Jon has a secure job ( with us) and a news article wrongly implies that APW were denied access because (presumably unlike us) they wouldn’t withhold negative result, I felt compelled to tell this story.  It’s an ugly one.

UPDATE: Pat Wolf checked his records and found that he also had a discussion at a conference in April 2015 with Atila regarding the Louisiana evaluation that he and Jon were doing. The materials he sent, however, were following that conversation, not following the June conversation as Pat had earlier remembered, and those materials were not directly related to the study.  In any event, it is clear from multiple conversations and multiple conference presentations that APW were aware of the existence of prior research.

2nd Update:  APW have a statement here.  My response to it is here.

(Edited for typos and to add links)


Read the rest of this entry »

USDOE Rediscovers Federalism

August 2, 2017

(Guest Post by Jason Bedrick)

Last month, as detailed here, the U.S. Department of Education rejected Delaware’s ESSA plan for being insufficiently “ambitious.” You see, Delaware was merely attempting to do something that no state had ever accomplished before.

The administration’s actions flew in the face of their frequently stated commitment to federalism. Now, however, it seems they have reversed course:

After some serious drama, U.S. Secretary of Education Betsy DeVos on Tuesday gave Delaware the green light for its Every Student Succeeds Act plan.

You read that right. Delaware, aka the state whose Feedback Shook the World, is the first state to get the all-clear to proceed on ESSA.

What drama are we talking about? Here’s some quick background: DeVos had been hitting the local control theme hard in speeches since taking office. But her team’s response to the submitted plan from Delaware, one of the first states to get ESSA plan feedback from the Trump education department, seemed out of line with that rhetoric.

The department questioned the ambitiousness of the First State’s student achievement goals and criticized the state for wanting to use Advanced Placement tests to gauge college and career readiness. (The department said this was a no-go because the tests and courses aren’t available in every school.)

That got many important people pretty upset, including Sen. Lamar Alexander, R-Tenn., the Senate education chairman and an ESSA architectChris Minnich, the executive director of the Council of Chief State School Officers, also said he was disappointed. Both said that DeVos’ team had essentially overstepped the bounds of the law.

In a press release, Secretary DeVos noted that she believed Delaware’s plan “adhered to the law” but she stopped short of recommitting her department to the principle of federalism:

“Delaware has always been a state of firsts, so it should be no surprise that theirs was both the first state plan submitted and the first approved under ESSA,” said Secretary DeVos.

“My criteria for approval is clear: does the state’s plan adhere to the law? Delaware demonstrated their plan does, and so I am happy to approve it. I hope it will give the students, families and educators in the state a strong foundation for a great education.

“Throughout the process, Delaware’s leaders have been terrific partners. I want to thank Gov. Carney, Secretary of Education Bunting and State Board President Loftus for their work and collaboration on putting forth a plan that embraces ESSA’s spirit of flexibility and creative thinking.”

All in all, this is a positive development. Nevertheless, this episode should serve to remind education reformers that even an administration that talks the federalist talk doesn’t necessarily always walk the walk. Those who respect subsidiarity and value local control — especially those who understand that the most local form of control is in the hands of parents — have good reason to be wary about giving the feds any power.

School Choice and Segregation in Perspective

July 31, 2017


(Guest post by Greg Forster)

OCPA’s Perspective carries my new article on school choice and segregation. I wrote it before the recent silliness from our friends at CAP, so my expectation that the recent increase in focus on this issue would only continue is holding up well so far:

The accusation that school choice will increase ethnic segregation in schools, after a long period on the rhetorical back-burner (during the age of test-score obsessions), has suddenly returned to the forefront of public debate. That’s no surprise, given rising levels of ethnic tension and polarization.

Last time the other side tried to make hay out of this, it failed, largely because the empirical evidence we have on this question is in favor of choice:

In fact, that body of research is the reason it’s been a while since we heard much talk about segregation in the debate over school choice. I remember hearing this talking point much more in the early 2000s, when fewer of these studies had been done. As the evidence piled up, the talking point went away.

In the new article I go into some of the politics of schools and ethnic segregation, arguing that while “most parents aren’t racist” is one possible explanation of the evidence for choice, you can also believe race is an important factor in school selection and still believe school choice will reduce ethnic segregation as compared with the status quo:

To whatever extent parents are racist, consciously or unconsciously, the government monopoly system is perfectly designed to cater to those racist preferences. Segregation flourishes under the government monopoly, both because schools are tied to ZIP codes and because power brokers draw the attendance lines.

So the strongest argument for choice is not “parents aren’t racist.” It’s “under the government monopoly, segregation happens by default, regardless of what parents prefer; only school choice creates the opportunity for integration.”

Check it out and let me know what you think!

Wonk Action Shot!

July 28, 2017


(Guest Post by Matthew Ladner)

Shot from Denver of some of my favorite folks…


The Ambition of Delaware’s ESSA plan

July 26, 2017

(Guest Post by Matthew Ladner)

Taking up where Jason left off the topic of USDoE labeling Delaware’s ESSA goal to cut the number of low-performing students as “not ambitious” I decided to look at NAEP gains in Arizona. You may or may not have heard, but Arizona students have been leading the nation in NAEP gains in recent years basically regardless of how you measure gains. Arizona was the only state that saw a statistically significant gains on all six NAEP exams for the entire period we can track all of them (2009-2015). When you subtract declines from gains the average state landed at +1. Measured on a cohort basis, Arizona students lead in gains between 4th grade in 2009 and 8th grade in 2013, and lo and behold they did it again between 2011 and 2015.

Arizona students did not however come remotely close to reducing the percentage of students scoring “Below Basic” by half on any of the NAEP exams on a statewide basis.

Not.even.close. You can track NAEP scores for Arizona’s super-high flying (on average) charter sector from 2005 to 2015, and even they don’t show that kind of progress and those folks moved a majority-minority student body into New England NAEP score range on all six tests.

The ESSA statute calls for “ambitious” rather than “completely fantastic” goals.



The Future of School Choice: Bickering about Words!

July 25, 2017


Will Flanders is right that school choice is not welfare (you heard it here first) and more broadly that school choice has not benefitted from appropriating the Rawlsian language of fairness (ditto). But he is wrong to think we would be better off making big investments in the free market movement’s language of markets and competition. I’m as big a fan of Milton as anyone (proof) but that language has all the wrong non-cognitive associations for the present moment. Flanders cites Jonathan Haidt but doesn’t seem to have learned the biggest lesson Haidt has to teach, which is that the non-cognitive content of language is more politically important than its cognitive content.

What we need is a new language of justice, equal opportunity, diversity and freedom that both Rawlsianism and the free-market movement used to have, say, fifty years ago, but that neither currently has in a very robust form. Much, much more about that here.

Parents Administer Frontier Justice in Wild West Charter Schools

July 18, 2017

(Guest Post by Matthew Ladner)

Parents out west administer quick frontier justice to undesired charter schools and the results are pretty awesome in today’s 74.

Yippie kai yay!