NEWS FEATURE

The fight against fake-paper factories that churn out sham science

Some publishers say they are battling industrialized cheating. A Nature analysis examines the 'paper mill' problem — and how editors are trying to cope.
Cartoon of researchers looking at a stack of papers with Western Blots. A magnifying glass shows the blots are tadpoles.

Illustration by Ana Kova

When Laura Fisher noticed striking similarities between research papers submitted to RSC Advances, she grew suspicious. None of the papers had authors or institutions in common, but their charts and titles looked alarmingly similar, says Fisher, the executive editor at the journal. “I was determined to try to get to the bottom of what was going on.”

A year later, in January 2021, Fisher retracted 68 papers from the journal, and editors at two other Royal Society of Chemistry (RSC) titles retracted one each over similar suspicions; 15 are still under investigation. Fisher had found what seemed to be the products of paper mills: companies that churn out fake scientific manuscripts to order. All the papers came from authors at Chinese hospitals. The journals’ publisher, the RSC in London, announced in a statement that it had been the victim of what it believed to be “the systemic production of falsified research”.

What was surprising about this was not the paper-mill activity itself: research-integrity sleuths have repeatedly warned that some scientists buy papers from third-party firms to help their careers. Rather, it was extraordinary that a publisher had publicly announced something that journals generally keep quiet about. “We believe that it is a paper mill, so we want to be open and transparent,” Fisher says.

The RSC wasn’t alone, its statement added: “We are one of a number of publishers to have been affected by such activity.” Since last January, journals have retracted at least 370 papers that have been publicly linked to paper mills, an analysis by Nature has found, and many more retractions are expected to follow.

Much of this literature cleaning has come about because, last year, outside sleuths publicly flagged papers that they think came from paper mills owing to their suspiciously similar features. Collectively, the lists of flagged papers total more than 1,000 studies, the analysis shows. Editors are so concerned by the issue that last September, the Committee on Publication Ethics (COPE), a publisher-advisory body in London, held a forum dedicated to discussing “systematic manipulation of the publishing process via paper mills”. Their guest speaker was Elisabeth Bik, a research-integrity analyst in California known for her skill in spotting duplicated images in papers, and one of the sleuths who posts their concerns about paper mills online.

Bik thinks there are thousands more of these papers in the literature. The RSC’s announcement is significant for its openness, she says. “It is pretty embarrassing that so many papers are fake. Kudos to them to admit that they have been fooled.”

At some journals that have had a spate of apparent paper-mill submissions, editors have now revamped their review processes, aiming not to be fooled again. Combating industrialized cheating requires stricter review: telling editors to ask for raw data, for instance, and hiring people specifically to check images. Science publishing needs a “concerted, coordinated effort to stamp out falsified research”, the RSC said.

Paper-mill detectives

In January 2020, Bik and other image detectives who work under pseudonyms — Smut Clyde, Morty and Tiger BB8 — posted, on a blog run by science journalist Leonid Schneider, a list of more than 400 published papers they said probably came from a paper mill. Bik dubbed it the ‘tadpole’ paper mill, because of the shapes that appeared in the papers’ western blot analyses, a type of test used to detect proteins in biological samples. A spate of media headlines followed. Throughout the year, the sleuths (not always working together) posted spreadsheets of other suspect papers — picking up on similar features across multiple studies. By March 2021, they had collectively listed more than 1,300 articles, by Nature’s tally, as possibly coming from paper mills.

Journals started to look at the papers. According to Nature’s analysis, around 26% of the articles that the sleuths alleged came from paper mills have so far been retracted or labelled with expressions of concern. Many others are still under investigation. The Journal of Cellular Biochemistry (JCB), for instance, announced in February1 that, last year, editors investigated and retracted 23 of 137 papers alleged to contain image manipulation.

Journals did not identify problems with all of the papers that had been flagged. Chris Graf, director of research integrity at Wiley, which publishes JCB, said in January that the publisher had completed investigations into 73 papers identified by Bik and others, and had found no reason to act on 11 of them. Seven others required corrections and 55 have been retracted or will be retracted.

Publishers almost never explicitly declare on retraction notices that a particular study is fraudulent or was created by a company to order, because it is difficult to prove. None of the RSC’s retraction notices, for instance, mentions a paper mill — despite the RSC’s announcement that it thinks the articles did come from one. But Nature has tallied 370 articles retracted since January 2020, all from authors at Chinese hospitals, that either publishers or independent sleuths have alleged to come from paper mills (see ‘Fraud allegations’). Most were published in the past three years (see ‘Chinese hospital papers on the rise’). Publishers have added expressions of concern to another 45 such articles.

FRAUD ALLEGATIONS: barchart showing the number of published papers potentially linked to companies that produce fraudulent work.

Sources: forbetterscience.com, scienceintegritydigest.com and Nature analysis

Nature has identified a further 197 retractions of papers from authors at Chinese hospitals since the start of last year. These are not ones that have made it onto lists of potential publication-mill products, although some were flagged by sleuths for image concerns, often on the post-publication peer-review website PubPeer.

Industrialized cheating

The problem of organized fraud in publishing is not new, and not confined to China, notes Catriona Fennell, who heads publishing services at the world’s largest scientific publisher, Elsevier. “We’ve seen evidence of industrialized cheating from several other countries, including Iran and Russia,” she told Nature last year. Others have also reported on Iranian and Russian paper-mill activities.

In a statement this year to Nature, Elsevier said that its journal editors detect and prevent the publication of thousands of probable paper-mill submissions each year, although some do get through.

China has long been known to have a problem with firms selling papers to researchers, says Xiaotian Chen, a librarian at Bradley University in Peoria, Illinois. As far back as 2010, a team led by Shen Yang, a management-studies researcher then at Wuhan University in China, warned of websites offering to ghostwrite papers on fictional research, or to bypass peer-review systems for payment. In 2013, Science reported on a market for authorships on research papers in China. In 2017, China’s Ministry of Science and Technology (MOST) said it would crack down on misconduct after a scandal in which 107 papers were retracted at the journal Tumor Biology; their peer reviews had been fabricated and a MOST investigation concluded that some had been produced by third-party companies.

Physicians in China are a particular target market because they typically need to publish research articles to gain promotions, but are so busy at hospitals that they might not have time to do the science, says Chen. Last August, the Beijing municipal health authority published a policy stipulating that an attending physician wanting to be promoted to deputy chief physician must have at least two first-author papers published in professional journals; three first-author papers are required to become a chief physician. These titles affect a physician’s salary and authority, as well as the surgeries they are allowed to perform, says Changqing Li, a former senior physician and gastroenterology researcher at a Chinese hospital who now lives in the United States.

“The effect is devastating,” says Li, about the impacts on Chinese science. “The literature environment published in Chinese is already ruined, since hardly anyone believes them or references studies from them.”

“Now this plague has eroded into the international medical journals,” he adds. The fact that people use paper mills also affects China’s reputation globally, says Futao Huang, a Chinese researcher working at Hiroshima University in Japan.

The prevalence of problem papers is leading some journal editors to doubt the submissions they get from Chinese hospital researchers. “The increasing volume of this ‘junk science’ is wreaking havoc on the credibility of the research emanating out of China and increasingly casting doubt upon legitimate science from the region,” said a February 2021 editorial2 in the journal Molecular Therapy.

Several other editors echo these concerns about the impact of paper mills. “They are undermining our confidence in the other manuscripts received from Chinese groups,” says Frank Redegeld, editor in chief of the European Journal of Pharmacology, published by Elsevier.

CHINESE HOSPITAL PAPERS ON THE RISE: chart showing the rise in English language articles with authors from Chinese hospitals.

Source: lens.org

China’s science and education ministries have taken steps to curb problematic publication incentives. They published a notice last February telling research institutions — including hospitals — not to promote or recruit researchers solely on the basis of the numbers of papers they publish, and also told them to stop paying cash bonuses for papers. And in August, China announced the introduction of measures to crack down on research misconduct, including attempts to curb independent contractors who fabricate data on others’ behalf. (MOST didn’t respond to Nature’s request for comment on the scale of the problem or the impact of its measures.)

Some Chinese researchers think these measures are beginning to work. Li Tang, who researches science policy at Fudan University in Shanghai, China, is hopeful that submissions from paper mills in China will fall in the future — although she notes that the issue isn’t confined to Chinese research.

Redegeld says he hasn’t yet seen a decrease in the number of suspected paper-mill manuscripts his journal receives, which he estimates to be around 15 a month.

Problem signs

Image-integrity sleuths and journal editors have identified a range of features in manuscripts that could be fingerprints of a paper mill. “We’re wondering how we protect ourselves from publishing this stuff,” says Jana Christopher, an image-integrity analyst at the publisher FEBS Press in Heidelberg, Germany, who screens incoming manuscripts for a number of journals, and helped the RSC with its investigation.

Potential signs of trouble include papers from different authors at different institutions sharing similar features: western blots with identical-looking backgrounds and suspiciously smooth outlines, titles that seem to be variations on a theme, bar charts with identical layouts that supposedly represent different experiments, or identical plots of flow cytometry analyses, which are used in studying cells. It seems that these manuscripts are produced from common templates, with words and images slightly tweaked to make the papers look a little different.

A particular problem is biomedical articles that claim to investigate understudied genetic regions that might be involved in cancers. Jennifer Byrne, a molecular oncology researcher at the University of Sydney, Australia, specializes in exposing flawed papers of this type, by spotting that their experimental details sometimes list incorrect nucleotide sequences or reagents, so that the experiments described cannot have taken place. Many of these papers are probably doctored simply by switching around the type of cancer or the genes involved in the study, says Byrne, although it’s hard to prove they’re from paper mills. “This problem of incorrect nucleotide sequences in the literature is rampant,” she says.

At last September’s COPE forum, Bik rattled off other red flags for editors to watch out for, including papers from Chinese hospitals and manuscripts with e-mail addresses that don’t seem to be linked to any of the author names. “Individually, these factors may not be problematic, but taken together they raise concerns and could be part of a pattern,” she said. Editors at the forum also noted that a manuscript-processing system, ScholarOne, can flag up unusual activity when it picks up on submissions from the same computer. A ScholarOne alert was also instrumental in the RSC’s investigation.

Portrait of Elisabeth Bik

Elisabeth Bik.Credit: Gabriela Hasbun

In February, Naunyn-Schmiedeberg’s Archives of Pharmacology said it had been affected by paper mills. The journal published an editorial3 listing important features of paper-mill articles. These included non-academic e-mail addresses (which happen to be common with Chinese scientists), authors’ inability to supply raw data when asked, and poor English. The journal is retracting 10 studies, and it reports that around 5% of all its submissions are from paper mills.

Publishers and others battling paper mills suspect they are only seeing the tip of the iceberg in the published literature. In part, that’s because similarities between images across studies might become obvious only when many papers are compared. Sleuths also know that features such as similar western blots and flawed nucleotide sequences might be the most obvious signs of paper-mill activity, says Bik. “There may be tonnes of other paper mills that have done a better job of hiding it,” she says. Editors at the COPE forum said they’d seen paper mills in areas such as computer sciences, engineering, humanities and social sciences, for instance.

The overall size of the paper-mill problem probably runs to thousands or tens of thousands of papers, Bik, Byrne and others think4. Graf, at Wiley, says it’s hard to estimate. “I don’t think it should be understated, I can’t say how big it is,” he says. “We have very little information about the people or companies doing this. I am exasperated by the situation, and that is being polite.”

“It’s detrimental to science as a whole because it makes science and scientists look unreliable,” says Christopher. Byrne has identified a different concern: she worries that by simply appearing in journals, fake studies that link genes to particular cancers can give the perception of activity in an area where there is none, and might be included in meta-analyses. “People die from cancer — it is not a game. It is important that the literature describes the work that takes place,” she adds.

Zombie papers

Journal editors know that if they reject manuscripts they suspect to be fabricated, that might not kill the paper forever. Fraudulent manuscripts can be submitted to multiple journals at the same time: so even if an editor rejects it during peer review, they might see it published elsewhere.

This has happened to Christopher, who 3 years ago saw alarming similarities in a cluster of 13 research manuscripts submitted to 2 journals published by FEBS Press, where she worked. Their western blots seemed to be not only fabricated, but also similar, as if they’d been created by tweaking a template. The journals rejected the manuscripts on her advice. Christopher published a 2018 paper5 warning of “systematic fabrication of scientific images” and urged journals to invest in pre-publication image screening. She also noted that she’d seen some papers appear in other journals.

Christopher told Nature that she tried to privately raise the alarm about the papers. In 2018, for instance, she and FEBS Letters’ managing editor advised the journal Cellular Physiology and Biochemistry that a paper it had published that year was probably fabricated; it had been simultaneously submitted to FEBS Letters, which had rejected it. But the journal’s publisher at the time, Karger in Basel, Switzerland, did not hear of any problem until 2020, when the paper was flagged up again in Bik and others’ ‘tadpole paper mill’ collection, along with other papers in the journal. Karger is now investigating all these papers together with the journal’s current publisher, says Christna Chap, Karger’s head of editorial development.

This year, Christopher again looked into the 13 manuscripts that had been submitted to her journals. She found they had all been published in other journals; so far, only three have been retracted and one has an expression of concern.

Many journals have changed their editorial-review processes to try to combat organized fraud. Some Elsevier journals, for instance, have changed their scope to avoid subject areas that seem to be a particular focus of paper mills, the publisher says. And several publishers say many of their journals have updated their policies to require that authors present the raw data behind their western blots at the time of submission. Asking for raw data is one of the main ways that publishers tell editors to follow up when they think there might be something wrong with a manuscript. But editors are aware that even raw data can be faked, especially if paper-mill firms catch on that such requests are being made.

“Asking for raw data is not an absolute guarantee, as you can fake the data. It is a deterrent,” says Sabina Alam, director of publishing integrity and ethics at Taylor and Francis. One of its journals, Artificial Cells, Nanomedicine, and Biotechnology, is investigating almost 100 published papers alleged to be from paper mills.

Alam also says that once they started investigations, some authors quickly asked to withdraw their papers. Some sent raw data in unreadable formats or without labels. In all these cases, journal editors say they’re not sure whether it’s correct to withdraw such articles, or to do something else — and are hoping for guidance on this from COPE. Bik has pointed out that some journals have already allowed authors to withdraw papers without stating the reason for retraction.

COPE says it will update its existing guidance on how journals should deal with systematic manipulation of the publication process, and is also creating a task force of editors from its membership to determine how the organization can provide better support on the issue.

Path forwards

Publishers say that they are limited in what they can do to share information between journals because even titles within the same stable are editorially independent of one another. They’re wary of sharing information between titles or publishers about an author that could be defamatory, and data-protection rules hinder the sharing of authors’ personal data.

Once fraudsters know they can get a paper into a particular title, they might continue to publish there, which could be why some journals seem to be more affected than others. One journal, the European Review for Medical and Pharmacological Sciences, has retracted 186 articles since January 2020, most of them flagged by Bik and Smut Clyde. “We were shocked by these investigations,” says one of its editors in chief, Antonio Gasbarrini.

Many journals are starting to employ analysts to try to spot problems in manuscripts as they come in. Graf, for instance, says that last year Wiley employed and trained 11 people to try to spot manipulated images across 24 journals — focusing on the papers most likely to be published. It hopes to expand the programme to more titles.

Publishers would like to automate some of this screening process. Many have teamed up with research groups to develop software that could detect duplicated images across published papers, and, last May, an industry group formed to try to set standards for these checks. Software is improving but isn’t yet capable of looking through many papers on a massive scale, says IJsbrand Jan Aalbersberg, the head of research integrity at Elsevier, who chairs the group. To do this would also require a giant shared database of images that publishers could check for duplication between papers. That will come when software can handle it, Aalbersberg predicts.

Suzanne Farley, Springer Nature’s research-integrity director, based in London, says she thinks that there will be a fall in the proportion of paper-mill submissions. “The paper mills are aware that publishers are getting better at detecting their submissions, and potential paper-mill customers are aware that there are now more serious consequences of using the services,” she says. (Nature’s news team is editorially independent of its publisher.) In the meantime, Farley says, there will be more retractions and expressions of concern. “We are committed to cleaning house,” she says.

But Christopher worries that an arms race could develop if fraudsters get better at avoiding obvious mistakes. One preprint posted to bioRxiv last year6, for instance, suggested that artificial intelligence techniques could generate fake western blots that were indistinguishable from real ones. “I’m really worried about the sophistication going up,” she says.

Nature 591, 516-519 (2021)

References

  1. 1.

    Behl, C. J. Cell. Biochem. https://doi.org/10.1002/jcb.29906 (2021).

  2. 2.

    Frederickson, R. M. & Herzog, R. W. Mol. Ther. https://doi.org/10.1016/j.ymthe.2021.02.011 (2021).

  3. 3.

    Seifert, R. Naunyn Schmiedebergs Arch. Pharmacol. 394, 431–436 (2021).

  4. 4.

    Byrne, J. A. & Christopher, J. FEBS Lett. 594, 583–589 (2020).

  5. 5.

    Christopher, J. FEBS Lett. 592, 3027–3029 (2018).

  6. 6.

    Qi, C., Zhang, J. & Luo, P. Preprint at bioRxiv https://doi.org/10.1101/2020.11.24.395319 (2020).

Download references

Nature Briefing

An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday.

Related Articles

Subjects

Sign up to Nature Briefing

An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday.

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing