Much has been written about the reproducibility crisis in scientific research. The pressures on researchers to publish novel and exciting results can lead some to poor scientific practices ranging from cherry picking, HARKing (hypothesizing after the results are known), P-hacking and, at worst, outright fraud.

One approach some journals are taking to avoid such poor practices is by offering an article format known as a Registered Report. On the surface, a published Registered Report appears much like a traditional research paper, but this format differs radically in the approach to peer review: the review of the experimental design plan and acceptance ‘in principle’ by the journal occurs at an early stage, often before any experiments have been carried out. This helps shift reviewers’ and editors’ focus to the soundness of the research question and experimental design, rather than on the perceived importance of the results. By its very nature, this format encourages greater transparency in reporting and the publication of ‘negative’ results.

Registered Reports were initially introduced in 2013 by the journals Cortex and Perspectives in Psychological Science. This format has since become relatively commonplace in the social sciences. It is not well known in the life sciences community, though interest is growing. Several journals publishing basic biology research now offer the Registered Reports format, including eLife, PLoS Biology, PLoS One, Scientific Reports and several BioMed Central journals.

We have been particularly inspired by our colleagues at Nature Human Behavior, who have offered this format since they launched in 2017. By adopting their workflow and adapting their guidelines to suit the unique needs of life sciences research, we are now introducing the Registered Reports format at Nature Methods for papers describing comparative analyses of tools or methods.

Although Registered Reports were initially designed for hypothesis-driven research and replication studies, we realized that they are also ideal for comparative analyses. The key contributions of such papers are whether a comparison is valuable for a research community and whether it is scientifically robust — not whether a particular method performs best. (Note that the Registered Reports format is not suitable for method development papers themselves.)

In traditional peer review, if reviewers identify a fundamental flaw in a large-scale comparative analysis study, there is usually little the authors can do to address this: coordinating a completely revised analysis, often performed by researchers at different institutes, would be a logistical nightmare. However, when peer review takes place before any experiments have been carried out, this enables authors to rework their design plan to ensure that it is robust and meets standards in the field.

The general process for peer review of Registered Reports at Nature Methods is as follows. First, authors submit a ‘Stage 1’ manuscript, which should include an Introduction that justifies the value of the comparative study and a detailed experimental plan, including a data analysis plan. If the Stage 1 manuscript meets our editorial criteria for scope, novelty, potential interest and comprehensiveness, it will be peer reviewed. If, over one or more rounds, reviewers find the experimental plan to be valuable and scientifically sound, the editors will offer an ‘accepted in principle’ decision. At this point, authors must register their Stage 1 paper in an appropriate repository, such as Figshare. Next, authors carry out their experiments and then resubmit the full ‘Stage 2’ manuscript, now including Results and Discussion. Reviewers perform a final technical evaluation of the Stage 2 manuscript, but the editors will not reject papers at this stage for reasons such as scooping or the perceived importance of the results.

The ‘accepted in principle’ decision is conditional on the assumption that authors will not substantially deviate from their Stage 1 manuscript. Further exploratory analysis of the results is allowed, but must be clearly stated as such in the Stage 2 submission. Should authors with an ‘accepted in principle’ agreement realize that they need to make significant changes to their experimental plan, they should contact the editors as soon as possible. To avoid this, we encourage authors (especially for wet lab studies) to include pilot data in the Stage 1 manuscript that demonstrate the feasibility of the proposed broader analysis. Authors must also agree in writing at Stage 1 to make their data, code and unique materials available upon publication.

Over the past two years we have closely engaged with several research groups and sent three Registered Reports out for peer review. We are very pleased to report that one of these papers, reporting an experimental comparison of near-infrared fluorescent proteins, is now accepted in principle, and the Stage 1 manuscript is available via Figshare.

We’ve learned several valuable lessons during this trial period: for example, that our guidelines need to be sufficiently flexible to allow for minor changes to experimental design. We’ve also realized that Registered Reports really do need to be submitted for review before a large-scale study gets underway — otherwise, as with a regular Analysis paper, it becomes too logistically difficult to address reviewer concerns about experimental design. We’ve also found that reviewers may have a kid-in-a-candy-shop tendency to request experiments that go beyond the reasonable scope of a study; thus, the role of the editor in giving clear advice to authors about what experiments we expect to see in a revision is essential.

We encourage all authors interested in submitting comparative analyses to Nature Methods to familiarize themselves with our guidelines for Registered Reports and reach out to us at the early stages of a project with presubmission inquiries and questions.

We hope the expert guidance authors will receive from Stage 1 manuscript peer review and the ability for the editors to provide an ‘accepted in principle’ decision at Stage 1 will make it more attractive for researchers to pursue scientifically valuable and sound comparisons of tools or methods. We also hope with this announcement that we may raise awareness in the life sciences community of the many benefits of this alternative approach to peer review.