Illustration by Dale Edwin Murray
Most journal editors know how much effort it takes to persuade busy researchers to review a paper. That is why the editor of The Journal of Enzyme Inhibition and Medicinal Chemistry was puzzled by the reviews for manuscripts by one author — Hyung-In Moon, a medicinal-plant researcher then at Dongguk University in Gyeongju, South Korea.
The reviews themselves were not remarkable: mostly favourable, with some suggestions about how to improve the papers. What was unusual was how quickly they were completed — often within 24 hours. The turnaround was a little too fast, and Claudiu Supuran, the journal's editor-in-chief, started to become suspicious.
In 2012, he confronted Moon, who readily admitted that the reviews had come in so quickly because he had written many of them himself. The deception had not been hard to set up. Supuran's journal and several others published by Informa Healthcare in London invite authors to suggest potential reviewers for their papers. So Moon provided names, sometimes of real scientists and sometimes pseudonyms, often with bogus e-mail addresses that would go directly to him or his colleagues. His confession led to the retraction of 28 papers by several Informa journals, and the resignation of an editor.
Moon's was not an isolated case. In the past 2 years, journals have been forced to retract more than 110 papers in at least 6 instances of peer-review rigging. What all these cases had in common was that researchers exploited vulnerabilities in the publishers' computerized systems to dupe editors into accepting manuscripts, often by doing their own reviews. The cases involved publishing behemoths Elsevier, Springer, Taylor & Francis, SAGE and Wiley, as well as Informa, and they exploited security flaws that — in at least one of the systems — could make researchers vulnerable to even more serious identity theft. “For a piece of software that's used by hundreds of thousands of academics worldwide, it really is appalling,” says Mark Dingemanse, a linguist at the Max Planck Institute for Psycholinguistics in Nijmegen, the Netherlands, who has used some of these programs to publish and review papers.
But even the most secure software could be compromised. That is why some observers argue for changes to the way that editors assign papers to reviewers, particularly to end the use of reviewers suggested by a manuscript's authors. Even Moon, who accepts the sole blame for nominating himself and his friends to review his papers, argues that editors should police the system against people like him. “Of course authors will ask for their friends,” he said in August 2012, “but editors are supposed to check they are not from the same institution or co-authors on previous papers.”
Peer-review ring
Moon's case is by no means the most spectacular instance of peer-review rigging in recent years. That honour goes to a case that came to light in May 2013, when Ali Nayfeh, then editor-in-chief of the Journal of Vibration and Control, received some troubling news. An author who had submitted a paper to the journal told Nayfeh that he had received e-mails about it from two people claiming to be reviewers. Reviewers do not normally have direct contact with authors, and — strangely — the e-mails came from generic-looking Gmail accounts rather than from the professional institutional accounts that many academics use (see 'Red flags in review').
Red flags in review
Signs that an author might be trying to game the system
A handful of researchers have exploited loopholes in peer-review systems to ensure that they review their own papers. Here are a few signs that should raise suspicions.
- The author asks to exclude some reviewers, then provides a list of almost every scientist in the field.
- The author recommends reviewers who are strangely difficult to find online.
- The author provides Gmail, Yahoo or other free e-mail addresses to contact suggested reviewers, rather than e-mail addresses from an academic institution.
- Within hours of being requested, the reviews come back. They are glowing.
- Even reviewer number three likes the paper.
Nayfeh alerted SAGE, the company in Thousand Oaks, California, that publishes the journal. The editors there e-mailed both the Gmail addresses provided by the tipster, and the institutional addresses of the authors whose names had been used, asking for proof of identity and a list of their publications. One scientist responded — to say that not only had he not sent the e-mail, but he did not even work in the field.
This sparked a 14-month investigation that came to involve about 20 people from SAGE's editorial, legal and production departments. It showed that the Gmail addresses were each linked to accounts with Thomson Reuters' ScholarOne, a publication-management system used by SAGE and several other publishers, including Informa. Editors were able to track every paper that the person or people behind these accounts had allegedly written or reviewed, says SAGE spokesperson Camille Gamboa. They also checked the wording of reviews, the details of author-nominated reviewers, reference lists and the turnaround time for reviews (in some cases, only a few minutes). This helped the investigators to ferret out further suspicious-looking accounts; they eventually found 130.
As they worked through the list, SAGE investigators realized that authors were both reviewing and citing each other at an anomalous rate. Eventually, 60 articles were found to have evidence of peer-review tampering, involvement in the citation ring or both. “Due to the serious nature of the findings, we wanted to ensure we had researched all avenues as carefully as possible before contacting any of the authors and reviewers,” says Gamboa.
When the dust had settled, it turned out that there was one author in the centre of the ring: Peter Chen, an engineer then at the National Pingtung University of Education (NPUE) in Taiwan, who was a co-author on practically all of the papers in question. After “a series of unsatisfactory responses” from Chen, says Gamboa, SAGE contacted the NPUE, which joined the investigation into Chen's work. Chen resigned from his post in February 2014.
In May, Nayfeh resigned over the scandal at his journal, and SAGE contacted the authors of all 60 affected articles to let them know that the papers would be retracted. Chen could not be reached for comment for this story, but Taiwan's state-run news agency said in July that he had issued a statement taking sole responsibility for the peer-review and citation ring, and admitting to the “indiscreet practice” of adding Taiwan's education minister as a co-author on five of the papers without his knowledge. That minister, Chiang Wei-ling, denies any involvement, but nevertheless resigned “to uphold his own reputation and avoid unnecessary disturbance of the work of the education ministry”, according to a public statement.
The collateral damage did not stop there. A couple of authors have asked SAGE to reconsider and reinstate their papers, Gamboa says, but the publisher's decision is final — even if the authors in question knew nothing of Chen or the peer-review ring.
Password loophole
Moon and Chen both exploited a feature of ScholarOne's automated processes. When a reviewer is invited to read a paper, he or she is sent an e-mail with login information. If that communication goes to a fake e-mail account, the recipient can sign into the system under whatever name was initially submitted, with no additional identity verification. Jasper Simons, vice-president of product and market strategy for Thomson Reuters in Charlottesville, Virginia, says that ScholarOne is a respected peer-review system and that it is the responsibility of journals and their editorial teams to invite properly qualified reviewers for their papers.
Nature Publishing Group (NPG) owns a few journals that use ScholarOne, but Nature itself and Nature-branded journals use different software, developed by eJournalPress of Rockville, Maryland. Véronique Kiermer, Nature's executive editor and director of author and reviewer services for NPG in New York City, says that NPG does not seem to have been the victim of any such peer-review-rigging schemes.
Free Podcast
An interview with Ivan Oransky from Retraction Watch.
But ScholarOne is not the only publishing system with vulnerabilities. Editorial Manager, built by Aries Systems in North Andover, Massachusetts, is used by many societies and publishers, including Springer and PLOS. The American Association for the Advancement of Science in Washington DC uses a system developed in-house for its journals Science, Science Translational Medicine and Science Signaling, but its open-access offering, Science Advances, uses Editorial Manager. Elsevier, based in Amsterdam, uses a branded version of the same product, called the Elsevier Editorial System.
Editorial Manager's main issue is the way it manages passwords. When users forget their password, the system sends it to them by e-mail, in plain text. For PLOS ONE, it actually sends out a password, without prompting, whenever it asks a user to sign in, for example to review a new manuscript. Most modern web services, such as Google, hide passwords under layers of encryption to prevent them from being intercepted. That is why they require users to reset a password if they forget it, often coupled with checking identity in other ways.
Security loopholes can do more than compromise peer review. Because people often use the same or similar passwords for many of their online activities — including banking and shopping — e-mailing out the password presents an opportunity for hackers to do more than damage the research record. Dingemanse, who has published in a number of journals that use Editorial Manager, including PLOS ONE, says: “It's quite amazing that they haven't got around to implementing a safe system.” Neither Aries nor PLOS ONE responded to several requests for comment.
Safety measures
Lax password protection has resulted in breaches. In 2012, the Elsevier journal Optics & Laser Technology retracted 11 papers after an unknown party gained access to an editor's account and assigned papers to fake reviewer accounts. The authors of the retracted papers were not implicated in the hack, and were offered the chance to resubmit.
Elsevier has since taken steps to prevent reviewer fraud, including implementing a pilot programme to consolidate accounts across 100 of its journals. The rationale is that reducing the number of accounts in its system might help to reveal those that are fraudulent, says Tom Reller, a spokesperson for Elsevier. If it is successful, consolidation will roll out to all journals in early 2015. Furthermore, passwords are no longer included in most e-mails from the editorial system. And to verify reviewers' identities, the system now integrates the Open Researcher and Contributor ID (ORCID) at various points. ORCID identifiers, unique numbers assigned to individual researchers, are designed to track researchers through all of their publications, even if they move institutions.
ScholarOne also allows ORCID integration, but it is up to each journal to decide how to use it. Gamboa says that not enough scientists have adopted the system to make it possible to require an ORCID for each reviewer. And there is another problem: “Unfortunately, like any online verification system, ORCID is also open to the risk of unethical manipulation,” says Gamboa — for example, through hacking.
That is a common refrain. “As you make the system more technical and more automated, there are more ways to game it,” says Bruce Schneier, a computer-security expert at Harvard Law School's Berkman Center for Internet and Society in Cambridge, Massachusetts. “There are almost never technical solutions to social problems.”
“As you make the system more technical and more automated, there are more ways to game it.”
It ultimately falls to editors and publishers to be on the alert, particularly when contacting potential reviewers. Carefully checking e-mail addresses is one way to ferret out fakes: a non-institutional e-mail address such as a free account from Gmail is a red flag, say sources. But at the same time, it could also be a perfectly legitimate address.
Jigisha Patel, associate editorial director of BioMed Central in London, says that it is definitely possible to catch cheaters by being on the alert for dubious e-mail addresses. “We've had some cases where we've caught them tweaking the e-mail addresses to try to steal someone's identity,” she says. But such screening is imperfect. In September, the publisher retracted a paper in BMC Systems Biology, stating that it believed that “the peer-review process was compromised and inappropriately influenced by the authors”.
Some scientists and publishers say that journals should not allow authors to recommend reviewers in the first place. John Loadsman, an editor of Anaesthesia and Intensive Care, which is published by the Australian Society of Anaesthetists in Sydney, calls the practice “bizarre” and “completely nuts”, and says that his journal does not permit it.
It is unclear exactly what proportion of journals allows the practice, but as fields become more specialized it provides an easy way for busy editors to find relevant expertise. Jennifer Nyborg, a biochemist at Colorado State University in Fort Collins, says that most of the journals to which she submits articles request at least five potential reviewers.
For most of the 60 articles retracted by SAGE, the original peer review had used only author-nominated reviewers. Despite this experience, the Journal of Vibration and Control still allows authors to suggest peer reviewers (and provide their contact e-mails) when they submit a manuscript — although more safeguards are now in place, says Gamboa.
The Committee on Publication Ethics (COPE), which serves as a kind of moral compass for scientific publishing (but has no authority to enforce its advice) has no guidance on the practice, but urges journals to vet reviewers adequately. Good practice is always to check the names, addresses and e-mail contacts of reviewers, says Natalie Ridgeway, operations manager for COPE in London. “Editors should never use only the preferred reviewer.”
NPG journals do allow authors to suggest independent reviewers. “But these suggestions are not necessarily followed,” says Kiermer. “The editors select reviewers and the selection includes checking for the absence of conflict of interests.” On the flip side, authors can ask an editor to exclude reviewers who they believe to have unmanageable conflicts, such as competing research. The publisher usually honours such requests, as long as authors do not ask to exclude more than three people or labs, Kiermer says.
Sometimes, recommending reviewers can backfire. Robert Lindsay, one of two editors-in-chief of the Springer-published journal Osteoporosis International, says that his publication allows authors to recommend up to two reviewers — but that he often uses this information to rule those reviewers out. This is based on past experience, in which he has seen authors recommend their own contacts, or worse: “We have had family members, folks in the same department, postgraduate students being supervised by an author,” he says. The journal generally uses suggested reviewers — who have passed screening — only if it runs into trouble finding other scientists to perform the task.
But screening can be difficult. Usually, editors in the United States and Europe know the scientific community in those regions well enough to catch potential conflicts of interest between authors and reviewers. But Lindsay says that Western editors can find this harder with authors from Asia — “where often none of us knows the suggested reviewers”. In these cases, the journal insists on at least one independent reviewer, identified and invited by the editors.
In what Lindsay calls the worst case that he has seen, an author suggested a reviewer who shared her first name but not her surname. Some investigation revealed that the surname was the author's maiden name — she was recommending that she review her own paper. “I don't think she is going to submit anything to us again,” says Lindsay.
- Journal name:
- Nature
- Volume:
- 515,
- Pages:
- 480–482
- Date published:
- ()
- DOI:
- doi:10.1038/515480a
Comments for this thread are now closed.