Registered Reports

A Registered Report offered at Nature Methods describes a comprehensive comparison of the performance of established, related methods or tools in which the experimental design and data analysis plan is peer-reviewed and registered in a suitable repository prior to data collection. Stage 1 submissions meeting the journal’s editorial criteria for scope, novelty, potential interest and comprehensiveness are provisionally accepted for publication before data collection commences. This format is designed to minimize publication bias and research bias in method and tool performance comparison studies.

Note that the Registered Reports format is not suitable for method development papers; such papers should be submitted as Articles or Brief Communications. Comparative analyses that have already been completed prior to journal submission should be submitted as Analyses.

Guidelines for authors

The initial Stage 1 submission should include a description of the key research question(s), motivation and background literature, a detailed experimental design plan, benchmarking datasets (if applicable), a detailed data analysis plan (including any unique code), descriptions of statistical metrics that will be applied, and pilot data (if applicable).

Stage 1 submissions that pass the editorial triage stage will be sent for peer review. Following one or more rounds of review, the manuscript will be either rejected or accepted in principle for publication. After acceptance in principle (AIP), the authors will proceed to conduct the study, adhering closely to the experimental design plan described in the Stage 1 manuscript. At this point, the Stage 1 manuscript must be registered in an appropriate repository (such as Figshare).

When the study is complete the authors will submit their finalized manuscript for re-review (Stage 2) and must upload their raw data and code (if relevant) to publicly accessible repositories. Pending quality checks and a sensible interpretation of the findings, the manuscript will be published by Nature Methods regardless of the results of the study.

Stage 1: Initial manuscript submission and review

Stage 1 submissions should include the manuscript and a cover letter. Authors are strongly encouraged to submit presubmission inquiries for advice on the likely suitability of a study as a Registered Report at Nature Methods. However, please note that we cannot commit to sending a manuscript for peer review until a complete Stage 1 submission has been evaluated by the editors.

The Stage 1 cover letter should include:

A brief scientific case for consideration. The journal aims to publish manuscripts that are of interest and relevance to a broad, multidisciplinary audience.
A statement confirming that all necessary support (e.g. funding, facilities) and approvals (e.g. ethics) are in place for the proposed study.
An anticipated timeline for completing the study if the Stage 1 submission is accepted.
A statement confirming that the authors agree to share their raw data and code (if relevant) for all published results.
A statement confirming that, following Stage 1 acceptance in principle, the authors agree to register their approved Stage 1 manuscript on a recognized repository (such as Figshare), either publicly or under private embargo until submission of the Stage 2 manuscript.
A statement confirming that if the authors later withdraw their paper, they agree to the journal publishing a short summary of the study under a section called Withdrawn Registrations.

Manuscript preparation guidelines – Stage 1

Initial Stage 1 submissions should include the following sections:

Introduction

A review of the relevant literature that motivates the comparative analysis and a full description of the experimental aims. Please note that following AIP, the Introduction section should not be altered apart from correction of factual errors, typographic errors and altering of tense from future to past (see below).

Methods

A description of experimental procedures in sufficient detail to allow another researcher to repeat the methodology exactly, without requiring further information. These procedures must be closely adhered to in the subsequent experiments; we reserve the right to reject Stage 2 manuscripts if the experimental plan deviates significantly from the originally proposed plan.
Proposed data analysis plan, including a precise description of all planned analyses and statistical tests. If applicable to the study, benchmark datasets and unique code should be described and provided for peer review.
Full description of proposed sample characteristics, including criteria for data inclusion and exclusion (e.g. outlier extraction). Procedures for objectively defining exclusion criteria due to technical errors or for any other reasons must be specified, including details of how and under what conditions data would be replaced.
Timeline for completion of the study and proposed resubmission date if Stage 1 review is successful. Extensions to this deadline should be discussed with the handling editor.

Pilot Data

Optional, but strongly encouraged for wet lab experimental comparisons. Can be included to establish proof of concept, effect size estimations, or feasibility of proposed methods. Any pilot experiments will be published with the final version of the manuscript and should be clearly distinguished from data obtained during Stage 2.

Stage 1 submissions that are judged by the editors to be of sufficient quality and scientific importance will be sent for in-depth peer review. Reviewers will be asked to assess:

The importance of the comparative analysis and relevance for a broad, multidisciplinary audience.
The rationale, plausibility and comprehensiveness of the proposed comparative analysis.
The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis where appropriate).
Whether the clarity and degree of methodological detail is sufficient to exactly replicate the proposed experimental procedures and analysis pipeline.
Whether the proposed comparative analysis is subject to any bias.

Following Stage 1 peer review, manuscripts will be rejected outright, offered the opportunity to revise, or accepted in principle (AIP). Manuscripts that receive an AIP decision will be published pending completion of the approved experimental design plan, passing of all quality checks, and a defensible interpretation of the results. Stage 1 manuscripts are not published in the journal following AIP. Instead they are registered by the authors in a recognized repository (either publicly or under embargo until Stage 2) and integrated into a single completed article following approval of the final Stage 2 manuscript.

Any major deviations from the stated experimental procedures could lead to rejection of the manuscript at Stage 2. In cases where the registered experimental design is altered after AIP due to unforeseen circumstances (e.g. change of equipment or unanticipated technical error), the authors should consult the editors immediately for advice, prior to the completion of data collection. Minor changes to the experimental design may be permitted according to editorial discretion. In such cases, AIP would be preserved and the deviation reported in the Stage 2 submission. If the authors wish to alter the experimental design more substantially following AIP but still wish to publish their article as a Registered Report, they must contact the handling editor to discuss the situation.

Stage 2: Full manuscript review

Once the study is complete, authors prepare and resubmit their full Stage 2 manuscript for review, which should include the following sections:

Introduction

Apart from minor stylistic revisions, the Introduction should not be altered from the approved Stage 1 submission. Any relevant literature that appeared following the date of AIP should instead be cited in the Discussion.
A link to the registered Stage 1 manuscript should be provided.

Results

The outcome of all registered analyses must be reported in the manuscript, except in rare instances where a registered and approved analysis is subsequently shown to be logically flawed or unfounded. In such cases, the authors, reviewers, and editors must agree that a collective error of judgment was made and that the analysis is inappropriate. In such cases the analysis would still be mentioned in the Methods but omitted with justification from the Results.
Only pre-planned analyses should be reported in the main Results section of the Stage 2 submission.
Authors may wish to include additional analyses that were not included in the registered submission. For instance, a new approach might become available between AIP and Stage 2 review, or a particularly interesting and unexpected finding may emerge. Such analyses are admissible but must be clearly justified in the text, appropriately caveated, and described in a separate section of the Results titled “Exploratory analyses”. Authors should be careful not to base their conclusions entirely on the outcome of these post hoc analyses.

Discussion

Authors should discuss their findings, including any limitations of the study.

Methods

The Methods should be the same as in the Stage 1 manuscript, except any descriptions written in future tense within the Stage 1 manuscript should be changed to past tense.
Any changes or deviations from the original Stage 1 experimental plan should be clearly marked in the Stage 2 submission.
Data Availability, Materials Availability, and Code Availability statements should be included.

Supplementary Information, Data and Code

Supplementary figures, tables, data, and other files should be included as standard supplementary information that accompanies the paper.
Raw data and code (if relevant) should be made freely available in a public repository. Authors are encouraged to use any repository that renders data and materials freely and publicly accessible and provides a digital object identifier (DOI) to ensure that the data remain persistent, unique and citable.
Data files should be appropriately time stamped to show that data was collected after AIP and not before. Other than registered and approved pilot data, no data acquired prior to the date of AIP is admissible in the Stage 2 submission. Raw data must be accompanied by guidance notes, where required, to assist other scientists in replicating the analysis procedure.

The resubmission will most likely be considered by the same reviewers as in Stage 1, but could also be assessed by new reviewers. In considering papers at Stage 2, reviewers will be asked to assess:

Whether the data reported are accurate and statistically sound.
Whether the authors have adhered to the experimental plan described in the approved Stage 1 submission.
Whether any unregistered post hoc analyses added by the authors are justified, methodologically sound, and informative.
Whether the authors’ conclusions are justified given the data.

Reviewers are informed that editorial decisions will not be based on the perceived importance, novelty or conclusiveness of the results. Thus while reviewers are free to enter such comments on the record, they will not influence editorial decisions. Reviewers at Stage 2 may suggest that authors perform additional post hoc tests on their data; however authors are not obliged to do so unless such tests are necessary to satisfy one or more of the Stage 1 review criteria.

Manuscript withdrawal and Withdrawn Registrations

It is possible that authors with an AIP may wish to withdraw their manuscript following or during data collection. Possible reasons could include major technical error, an inability to complete the study due to other unforeseen circumstances, or the desire to submit the results to a different journal. In all such cases, manuscripts can of course be withdrawn at the authors’ discretion. However, the journal will publicly record each case in a section called Withdrawn Registrations. This section will include the authors, proposed title, the abstract from the approved Stage 1 submission, and brief reason(s) for the failure to complete the study. Partial withdrawals are not possible; i.e. authors cannot publish part of a registered study by selectively withdrawing one of the planned experiments. Such cases must lead to withdrawal of the entire paper.

Guidelines for reviewers

The review process for Registered Reports is divided into two stages. At Stage 1, reviewers assess the experimental design plan before data are collected and analyzed. At Stage 2, reviewers consider the full study, including results and interpretation.

Stage 1 manuscripts will include only an Introduction, Methods (including proposed analyses), and Pilot Data (where applicable). In considering papers at Stage 1, reviewers will be asked to assess:

The importance of the comparative analysis and relevance for a broad, multidisciplinary audience.
The rationale, plausibility and comprehensiveness of the proposed comparative analysis.
The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis where appropriate).
Whether the clarity and degree of methodological detail is sufficient to exactly replicate the proposed experimental procedures and analysis pipeline.
Whether the proposed comparative analysis is subject to any bias.

Following completion of the study, authors will complete the manuscript, including Results and Discussion sections. The manuscript will then be returned to the reviewers, who will be asked to appraise:

Whether the data reported are accurate and statistically sound.
Whether the authors have adhered to the experimental plan described in the approved Stage 1 submission.
Whether any unregistered post hoc analyses added by the authors are justified, methodologically sound, and informative.
Whether the authors’ conclusions are justified given the data.

Reviewers at Stage 2 may suggest that authors report additional post hoc tests on their data; however authors are not obliged to do so unless such tests are necessary to satisfy one or more of the Stage 2 review criteria. Please note that editorial decisions will not be based on the perceived importance, novelty, or conclusiveness of the results.