Chutzpah is jokingly defined as murdering one’s parents and then complaining about being an orphan. Atila Abdulkadiroglu, Parag Pathak, and Christopher Walters ( hereafter APW or the MIT team) sure show some chutzpah when complaining about not having continued access to data regarding the Louisiana Scholarshp Program (LSP) in a recent article. While I don’t know for sure why they were denied continued access to data, I believe that it is related to their rush to release 1st year results from their evaluation. Why they were rushing is an incredibly depressing story about how status and power in our field contributes to academic abuse and dishonesty– a story the reporter who wrote the article entirely missed.
It is not widely known or acknowledged, but the original analysis of 1st year result from LSP was conducted by Jonathan Mills when he was a doctoral student along with his advisor, Patrick Wolf, at the University of Arkansas. They presented those findings at academic conferences 8 times during 2014 and 2015 and they were contained in Jon’s dissertation published in July 2015. APW were at some of those conferences. Atila actually had lunch at one conference with Jon and Pat during which they discussed that study in June of 2015. Atila never indicated that he was conducting or planning to conduct a similar study. He offered to help and they sent him some materials. He never responded with help but he did move forward with his own study with the MIT team without informing Pat or Jon that they were doing so.
APW released their own study as an NBER report in December 2015. Nowhere in that report did they acknowledge or cite Jon and Pat’s earlier work of which they were almost certainly aware, having discussed it with them. Nor did APW acknowledge that their study was essentially a replication of Jon and Pat’s earlier study. The research designs were nearly identical. The data were almost the same. The only difference was that Jon and Pat had a more complete data set and as a result reported more negative results.
That’s right. Jon and Pat had more negative results. They released those results along with the negative 2nd year results in February 2016. So the fact that Jon and Pat continued getting access to LA data while APW did not does not appear to have anything to do with reporting negative results. It seems to be related to the fact that APW were rushing to release results. They didn’t take the time like Jon and Pat did to solve missing data issues. Instead they were determined to move fast to get their results out first.
Why did it matter that they be first? By being first to release they could act like they had the original analysis rather than a replication. Top Econ journals tend not to be as interested in replications of a grad student’s dissertation. And by being first to release and not citing Jon’s work they could act like theirs was the original analysis.
Failing to credit and cite earlier work is a form of academic fraud. I have not come forward earlier with this story because Jon was entering the academic job market and did not want to get on the wrong side of high status and powerful people in the field. Pat and I, as his advisers, deferred to his wishes and remained quiet. Now that Jon has a secure job ( with us) and a news article wrongly implies that APW were denied access because (presumably unlike us) they wouldn’t withhold negative result, I felt compelled to tell this story. It’s an ugly one.
UPDATE: Pat Wolf checked his records and found that he also had a discussion at a conference in April 2015 with Atila regarding the Louisiana evaluation that he and Jon were doing. The materials he sent, however, were following that conversation, not following the June conversation as Pat had earlier remembered, and those materials were not directly related to the study. In any event, it is clear from multiple conversations and multiple conference presentations that APW were aware of the existence of prior research.
(Edited for typos and to add links)