Supplementary information has its place in science publishing, but deploying it effectively requires effort from journals and scientists.
It is often assumed that the more information one has, the better. But in the publication of research papers, what is critical for some readers is superfluous for others. Publishers began hosting online supplementary information as a mechanism to convey information that is crucial for full assessment and reproduction of a study without deterring the more casual reader.
Like many actions driven by good intentions, this has had unintended consequences. The supplement has opened the floodgates for considerable unnecessary information to make its way into published papers. Authors use it to publish ancillary data they don't intend to develop into a full manuscript, and some referees may be less inclined to curb their demands for tangential data because there is almost unlimited space in which to put them.
There has therefore been a move by publishers, including NPG, to limit supplementary information or to move it into the body of the manuscript. But it is important to recognize that the supplement comprises different kinds of information with diverse roles. A one-size-fits-all approach to reducing it could cause more harm than good.
Methods papers, for instance, have different needs than biological results papers. In fact, the most common complaints related to Nature Methods papers are from readers saying they want more details to reproduce a method. Some reviewers, on the other hand, understandably complain about the time it takes to scrutinize excessive supplementary information. Further complicating the situation are increasing concerns about research irreproducibility and the resulting calls for even more information to be provided with published papers (Nat. Biotechnol. 30, 806, 2012). Finally, the supplement has not expanded in isolation: research studies are typically much more complex than they were in the past.
Reconciling these conflicts requires first of all an acknowledgment that supplementary information is, as the name states, supplementary: not all of it need be subject to the same level of editorial and peer review.
Detailed results behind figures in the main paper, positive and negative controls, detailed performance data, algorithms and software, the steps and results of chemical synthesis, instrument schematics, and all the information necessary to evaluate and reproduce a method should be provided with any research paper. Nearly all of this belongs in supplementary information, unless specialized repositories exist for a particular data type, in which case the data should be deposited there.
Ultimately, it falls to editors and reviewers to discriminate between the parts of the supplementary material that do and do not need careful evaluation. It is virtually impossible to make general pronouncements on this issue, given the variety of supplementary material and the fact that its importance depends on the particular paper. But a situation in which the supplement is a repository for less-reliable or otherwise problematic data is clearly to be avoided.
Editors already on occasion select referees to evaluate only specific parts of a manuscript: in the review of methods papers, we regularly select a mix of technical and biological experts and request their opinion only on particular aspects of the work. Similarly, referees should (and undoubtedly often do) judge for themselves what supplementary material requires their attention. Ideally, entirely tangential pieces of the supplement should be pointed out so they can be removed.
Even if a supplementary item does not need strict peer review, it can still be essential for some readers. Good examples are the reagent catalog numbers for the methods applied in a paper, particularly when these are nonstandard, and the step-by-step protocols. This information is often important to reproduce the research but is unlikely to substantially benefit from peer review.
Another example is image data. The DataViewer at the Journal of Cell Biology provides an excellent mechanism for readers to access, manipulate and download the original image files associated with a research paper (providing them is optional for the author). An examination of these images allows readers to make their own judgment about the interpretations in the paper; this, too, does not require prior peer review.
Authors, with the help of editors, should organize the supplement with a view to making it as easy as possible for a reader to grasp the information. Publishers, for their part, should better integrate the supplementary information into the paper and provide portals for easy access to data in repositories. Massive PDF files, although convenient for printing, are very impractical for finding a specific piece of data. Efforts to move critical supplementary material into the online version of the manuscript or improve linkages are already underway, but more work is needed.
Integrating all supplementary information into the main manuscript or prohibiting it altogether is a poor strategy. It is worth recognizing that any research paper has multiple audiences; an intelligent use of supplementary information improves the chance that it will be accessible to all of them.