The US National Institutes of Health (NIH) announced on 1 February that it will discontinue PubMed Commons, a tool that allows scientists to comment on articles indexed in the agency’s popular PubMed database of biomedical literature. The agency said that low levels of engagement and the growth of alternate venues for public discussion of published papers contributed to its decision to close PubMed Commons after five years.
Those who want to hold forth on the merits or shortcomings of published papers — or who simply want to add links to related papers or blog posts — have until 15 February to make their mark. After that, they can post comments on an increasing number of individual journal websites. Or they can use PubPeer, a platform that launched around the same time as PubMed Commons but, unlike the NIH tool, allows commenters to remain anonymous (see ‘Little-used commons?’).
“We gave it a fair shot,” says Jerry Sheehan, deputy director of the NIH’s National Library of Medicine (NLM) in Bethesda, Maryland. “It just wasn’t turning into a major point of discussion for the research community.”
Advocates for the platform were quick to express their dismay, however. “It’s a terrible blow to responsible post-publication peer review,” says Jim Coyne, a health psychologist at the University of Pennsylvania in Philadelphia, who helped to beta-test PubMed Commons in 2013.
Alexander Tsai, a psychiatrist at Massachusetts General Hospital in Boston, says that the other venues that allow commenting won’t help him. He used PubMed Commons to post a correction to his own 2014 meta-analysis1 of rates of depression among people with HIV. But after 2 March, comments won't be visible — although people can still download them. Tsai worries that readers who find his paper through the NIH portal will have no way of knowing there’s a problem. “It does concern me,” Tsai says.
Christopher Southan, a curator for the Guide to Pharmacology database at the University of Edinburgh, UK, likes that PubMed Commons requires commenters to use their real names, saying that it provides “an important filter” and brings “quality” to the discussion. Southan, one of PubMed Commons’ top-ten most prolific commenters, also worries that sites such as PubPeer cater mainly to a “niche” group of highly motivated users — thus reinforcing the kinds of echo chambers that are seen on other Internet commenting forums, such as Reddit or 4Chan, and leaving out the vast majority of the biomedical-research community. “Everyone goes to PubMed,” he says.
Yet, only a small fraction of the people that used PubMed ever posted a comment. According to an analysis presented by NLM staff at last year’s International Congress on Peer Review and Scientific Publication in Chicago, Illinois, only around 1,400 individuals posted at least one comment on PubMed Commons between 2014 and 2016 — and those who did were overwhelmingly men from North America and Europe.
It’s impossible to know how many users posted to PubPeer, given that they can remain anonymous. But in terms of volume, PubPeer has definitely generated much more engagement — and for good reason, says Jaime Teixeira da Silva, an independent plant scientist in Takamatsu, Japan.
“Let’s be honest — academics don’t want to be seen criticizing their colleagues’ work by name,” says Teixeira da Silva, who published an analysis last year of post-publication peer-review platforms2. PubMed Commons, he argues, “had to eventually die, not because the idea wasn’t good, but because it just wasn’t really useful, let alone essential”.
The platform did serve as a flashpoint for intense debate on a couple of occasions, however. The first was when two of the most prominent CRISPR-gene-editing researchers posted comments on a controversial 2016 perspective article3, alleging that what was purported to be the definitive history of the technology was “factually incorrect” and “incomplete”. Another instance involved two editorials4,5 by Michael Blatt, editor-in-chief of the journal Plant Physiology and a plant-cell biologist at the University of Glasgow, UK, that called out the motivations of anonymous commenters on sites such as PubPeer. His articles generated the most comments of all time of any PubMed article.
PubPeer probably pulled many would-be users away from the NIH service, says Coyne, as did in-house commenting platforms from publications such as The BMJ and journals put out by the Public Library of Science, which now allow readers to post thoughts that are directly appended to the article under discussion. And just this week, eLife announced its own annotation tool.
“Rapid comments have become almost like Amazon reviews,” Coyne says. Just as Amazon pushed out brick-and-mortar bookstores, these other commenting platforms seem to have doomed PubMed Commons. The NIH platform had credibility, Coyne says. But “they could have done a lot better job publicizing”.