Scientists are an opinionated bunch. From the cutting criticisms they make during peer review to bold questioning after a conference presentation, the rough and tumble of academic debate is seen as a crucial part of scientific progress. So where is the online equivalent for published papers?

To be sure, there are lively debates on blogs and social media about the merits of published work. Individual communities have formed their own central areas for engaged and informed criticism of peer-reviewed results. For example, Haldane’s Sieve, a site for evolutionary geneticists, is always busy and encourages authors to write short explanations of preprint abstracts. But click on the published homes of many of these papers — the websites of the journals in which they appear — and you will find digital tumbleweed.

In recent years, authors and readers have been able to post online comments about Nature papers on our site. Few bother. At the Public Library of Science, where the commenting system is more successful, only 10% of papers have comments, and most of those have only one.

Last week, the US National Center for Biotechnology Information became the latest publisher to attempt to corral this online discussion. It is opening up PubMed, its online repository of abstracts of some 23 million scientific papers, to comments. If online commenting on research papers or abstracts can flourish anywhere, surely it will be at PubMed, the site at which so many biomedical researchers begin their investigation of the literature. Visibility should be high, after the site emerges from its restricted pilot stage. And next week will see the launch of bioRxiv, the latest attempt to mimic the physics preprint site arXiv for biomedical sciences; this site will also encourage comments.

The early signs for the PubMed trial are promising. Among the first 200 comments, there has been some apparently useful self-criticism: for example, Andrew Kniss of the University of Wyoming in Laramie, posted on his own 2006 paper, saying that “our conclusions with respect to field management of the disease went beyond what the limited data could support”.

What PubMed has going for it is high traffic. But what has yet to emerge, and what could dictate how scientists use it, are the boundaries of online conduct that it will set, and how it will tune into the constructive criticism needed for genuine post-publication review of findings amid the noise and static that passes for much online debate.

It makes sense to digitally tie the research and the comments together.

Alarmed at the tenor of criticism when it concerns misconduct, some editors have tried to rein in online discussion, and to bring it within the limits of conventional debate. But attempts to dictate terms are likely to backfire. In a recent editorial in the journal ACS Nano, for example, the editors asserted that “the numbers of blogs, twitter messages, etc. in which individuals accuse others of academic fraud are steadily rising” — although they did not provide evidence for this. And they asked that suspicions of plagiarism or data manipulation be reported directly to a journal, rather than posted openly online (W. J. Parak et al. ACS Nano 7, 8313–8316; 2013). It was others’ “privilege” to be able to comment on a journal’s decision on a blog afterwards, the editorial added.

Although written with concern for the fair treatment of scientists who suffer damage to their reputation when comments are made irresponsibly, the editorial raised the hackles of chemistry bloggers who have pointed out egregious examples of image manipulation in papers — and who understandably consider that it is they, as much as the journals, who are doing the community the service (see go.nature.com/cplnfd). It is better to ask that debate be civil, responsible and courteous, than that it not appear online at all.

Does it matter just where online these discussions take place? Not in the short term — even the most obscure blog discussion can catch a wave and throw an academic debate into the mainstream. But if a longer-term goal is to leave some permanent signpost to help others navigate the scientific landscape, then it does make sense to digitally tie the research and the comments together.

For this to happen, PubMed Commons and digital publishers either need to become a hub for separate online discussions, or to generate community engagement directly at the sites of research papers or abstracts — something that has so far been lacking. Ultimately, the success of this worthy effort is incumbent on us all: so do visit and comment on your paper and others. Help the experiment to work.