Editorial

Promoting reproducibility with registered reports

Authors who wish to publish their work with us have the option of a registered report. With this format, acceptance in principle happens before the research outcomes are known. As a result, publication bias is neutralized, as are incentives for practices that undermine the validity of scientific research.

Across the social and biomedical sciences, there is a pressing realization that, in order to increase the reliability of scientific results, we need to change how research is carried out, evaluated, reported, and incentivized. Ten leading advocates for the reform of scientific practices set out ‘A manifesto for reproducible science’ in this first issue of Nature Human Behaviour (article no. 0021). They put forward a set of proposals for the optimization of the scientific process that require engagement and adoption by all stakeholders — researchers, institutions, funders, and journals — in order to succeed.

Nature Human Behaviour is committed to supporting robust scientific practices and the adoption of the registered report format is one step in this direction. Registered reports shift the emphasis from the results of research to the questions that guide the research and the methods used to answer them. As such, the format aims to counteract a host of problems associated with results-based publication that threaten the integrity of the scientific method.

Basing decisions to publish on the direction and significance of research results has led to a biased scientific record that does not represent the actual state of knowledge and undermines the ability of science to self-correct1. The quest for positive results encourages numerous questionable research practices — “the steroids of scientific competition”2 — such as HARKing (hypothesizing after the results are known) and P-hacking (collecting or selecting data or statistical analyses until non-significant results become significant).

With registered reports, initial peer review and in-principle acceptance occur prior to data collection (Fig. 1). As a result, the decision to publish is not affected by the significance or direction of the results, but rather by the importance of the research question and the strength of the methods. Peer review after data collection ensures that the agreed protocol has been adhered to, but cannot be used to revoke a positive decision based on the outcomes. The format was originally introduced in 2013 at Cortex3 and Perspectives in Psychological Science (http://go.nature.com/2hmrKjH), and is now offered in some form by more than 40 journals (http://go.nature.com/2grvm1W).

Figure 1: The registered report workflow.
Figure 1

Figure adapted from https://cos.io/rr/.

The format in Nature Human Behaviour is available for hypothesis-driven quantitative research where the data have not yet been collected (and are not already available for secondary analyses). Authors interested in having their work considered in this format should submit their study protocol that includes an introduction, methods (including an analysis plan), and the results of any pilot experiments that motivate the research proposal. Protocols that meet the journal's criteria for significance and broad relevance will be sent for in-depth peer review. As for all other research formats in the journal (see companion Editorial), novel studies and high-value replications have equal priority for publication. If the reviewers' assessment of the protocol is positive, we will be offering acceptance in principle. The authors can then collect their data over a period of time agreed with the journal and submit a full manuscript that includes the original submission with added results and discussion. The full manuscript is reviewed, but acceptance cannot be revoked unless the authors failed to adhere to the agreed protocol.

A prerequisite for publication is that authors agree to share publicly their raw data, as well as their materials and any code (through deposition in a suitable repository or inclusion as supplementary material). Further details and guidelines for authors and reviewers appear on our website (http://www.nature.com/nathumbehav/about/content).

By offering the registered report format, it is the journal's intention to support the research community's efforts for transparency, reproducibility, and open sharing. Some have questioned whether the format curtails creativity and the exploration of serendipitous findings4. However, this is by no means the case: although authors are required to adhere to the accepted protocol, there are no restrictions on the reporting of unplanned, unregistered analyses. We only ask that unregistered analyses be flagged as such, so that readers can discriminate between pre-registered and unregistered analyses. We further offer incremental registration (with fast-track peer review), so that authors have the option to add experiments to approved submissions. This is particularly appropriate in cases where an initial experiment leads to a serendipitous finding that warrants follow-up in the same paper.

The registered report format is not suitable for all types of research and we will be offering it alongside our other research formats. However, we strongly encourage those scientists engaged in hypothesis-driven quantitative research to consider submitting their work as a registered report — there is nothing to lose, but much to gain.

References

  1. 1.

    Persp. Psychol. Sci. 7, 645–654 (2012).

  2. 2.

    , , Psychol. Sci. 23, 524–532 (2012).

  3. 3.

    Cortex 49, 609–610 (2013).

  4. 4.

    , , & AIMS Neurosci. 1, 4–17 (2014).

Download references

Authors