Fun With Peer Review

PHD Comics

(Guest post by Greg Forster)

I may have to revise my opinion of Vox; they seem to have taken an interest in the weaknesses of the peer review system. Of course there are a lot of responsible peer-reviewed journals and, well, peers. But there a lot of the other kind as well, and we are long past the point where simply having gone through something called “peer review” ought to count for anything.

One story details how unscrupulous researchers can manipulate journals, including – amazingly – posing as their own reviewers. In highly specialized fields, journal editors may not know who the appropriate reviewers would be, so they rely – apparently uncritically in some cases – on the “recommended reviewers” supplied by the article authors. Who in some cases are simply the authors themselves using another email address. One scientist used 130 email accounts to create a vast, self-validating “peer review and citation ring”; 60 papers were recently retracted after a 14-month investigation uncovered the fraud. A total of at least 110 articles have been pulled in the last two years due to this type of fraud.

Get me off your email list

Figure 1 from the article “Get Me Off Your Fucking Mailing List”

Accepted for publication by the highly reputable International Journal of Advanced Computer Technology

But the other story is a lot better. It details how some journals now survive not by selling subscriptions or getting institutional support, but by charging a fee to publish your paper. They are apparently known as “predatory journals” because they spam the email universe looking for gullible (or, presumably, unscrupulous) people looking to break into publication. “Article mills” (after the analogous “diploma mills”) would seem a more appropriate name.

As you can see above, the “peer review” process becomes somewhat lax in these cases. One pair of scientists slapped the above-referenced article and began submitting it to peer review spammers. They were amused to discover that one journal accepted their article for publication. Another journal not only accepted but published an article (consisting of nonsense text) by Maggie Simpson and Edna Krabappel. It now sends the authors regular demands that they pay their $459 bill.

But it’s not just spam scammers – peer review controls are easy to get past even at some highly reputable publishers.

2 Responses to Fun With Peer Review

  1. You raise some good issues about peer review, but please don’t revise your opinion about Vox. They are truly awful. It is the new form of media — super-opinionated, 20-somethings, who have never done anything, studied anything, or know anything to justify their arrogant faux-expertise on all matters.

    Of course, this was always the case among the front-line writers at opinion magazines, like The New Republic and The Atlantic, but they were always policed by some grown-ups who had some wisdom and could restrain their insufferable over-confidence.

    But now the grown-ups are gone. Vox is run by Ezra Klein and Matt Yglesias who epitomize the perpetual windbags with no experience or expertise to justify their confident spewing of opinions on all issues. Having barely turned 30, they are the grey beards at Vox.

    Until just last week the New Republic at least had Franklin Foer and Leon Wieseltier to act as the grown-ups. That was until barely 30 year old Chris Hughes destroyed the New Republic and drove out them and the rest of the staff.

    I’ve been thinking of writing a post on this. Policy discourse is becoming so divisive and coarse not because Congress has gotten worse. The problem is that journalism has gotten worse. And it has gotten worse because it has lost all of the restraint of grown-ups and grown-up institutions.

  2. Greg Forster says:

    Verily, there is a good deal of juvenile crap on Vox. And Chris Hughes was obviously sent by central casting to play the role of William Higginbotham. If he isn’t nominated for The Higgy, it will only be because we all look at him and say, “too obvious.”

    But . . . these awesome stories didn’t come to my attention until Vox aggregated them and added suitably clickholish titles. Couldn’t they serve as a sort of BuzzFeed for educated people?

Leave a comment