Registered Reports

A Registered Report is an original research article that minimizes publication bias and research bias in hypothesis-driven research, while also allowing the flexibility to conduct exploratory (unregistered) analyses and report serendipitous findings. It also allows for methodological issues with the proposal to be resolved before the study is initiated, potentially leading to a more robust study design from the outset.

Registered Reports undergo a two-stage peer review process. In the first stage, the methods and proposed analyses are peer reviewed prior to the research being conducted. Protocols meeting stringent standards of methodological rigour are then provisionally accepted for publication in Communications Psychology and pre-registered before data collection commences. In the second stage, once the study is complete, the authors finalise the article to include results and discussion. The complete manuscript is then peer-reviewed again to ensure there is no unjustified deviation from the pre-registered protocol. The final acceptance of the manuscript does not depend on the results obtained.

Communications Psychology considers Registered Reports of any hypothesis-driven research of relevance to a specialized field of Psychology. We also welcome submissions of clinical trials under this format (Study protocol as Stage 1 manuscript and the completed trial as Stage 2 manuscript). We encourage submissions proposing high-value scientifically-justified replication studies, and in particular seek to publish generalization studies, i.e. large-scale replications that expand the research to include large and diverse samples. We likewise welcome replication and generalization studies of work published in Communications Psychology.

Guidelines for Authors

The initial Stage 1 submission will include a description of the key research question(s) and background literature, hypotheses, experimental procedures, outcome measures, analysis pipeline (including any unique code), sampling plan (statistical power analysis or Bayesian equivalent), and pilot data (if applicable). Use of the following template to prepare your submission is highly recommended.

Stage 1 submissions will be triaged by the editorial team for the strength of the research design and the importance of the research question for a specialized field of psychology. Submissions that pass the editorial triage stage will be sent for peer review (Stage 1). Following review, the article will be either rejected, returned to the authors for revision, or accepted in principle for publication. After acceptance in principle (AIP), the authors will proceed to conduct the study, adhering exactly to the peer-reviewed procedures. At this point, the Stage 1 manuscript must be registered in an appropriate repository (such as Figshare).

When the study is complete the authors will submit their finalised manuscript for re-review (Stage 2) and will upload their raw data, study materials, computer code (if relevant) and laboratory log to a publicly accessible file-sharing service. Pending quality checks and a sensible interpretation of the findings, the manuscript will be published by Communications Psychology regardless of the significance or direction of the results.

Stage 1: Initial manuscript submission and review

Stage 1 submissions should include the manuscript and a brief cover letter. 

The Stage 1 cover letter should include:

• A brief scientific case for consideration. The journal aims to publish manuscripts that are relevant to a specialized field of Psychology. High-value scientifically-justified replication studies and in particular direct replications that include a degree of generalization are welcome.
• A statement confirming that all necessary support (e.g. funding, facilities) and approvals (e.g. ethics) are in place for the proposed study. Note that manuscripts will be generally considered only for studies that are able to commence immediately. However, authors with alternative plans are encouraged to contact the Editorial office for advice.
• An anticipated timeline for completing the study if the Stage 1 submission is accepted.
• A statement confirming that the authors agree to share their raw data, any digital study materials, computer code (if relevant), and laboratory log for all published results.
• A statement confirming that, following Stage 1 acceptance in principle, the authors agree to register their approved Stage 1 manuscript on a recognized repository (such as Figshare), either publicly or under private embargo until submission of the Stage 2 manuscript.
• A statement confirming that if the authors later withdraw their paper, they agree to the journal publishing a short summary of the study as a Withdrawn Registration.

Manuscript preparation guidelines – Stage 1

Initial Stage 1 submissions should include the following sections:

Introduction

• A review of the relevant literature that motivates the research question and a full description of the experimental aims and hypotheses. Please note that following AIP, the Introduction section cannot be altered apart from correction of factual errors, typographic errors and altering of tense from future to past (see below).

Methods

• Full description of proposed sample characteristics, including criteria for data inclusion and exclusion (e.g. outlier extraction). Procedures for objectively defining exclusion criteria due to technical errors or for any other reasons must be specified, including details of how and under what conditions data would be replaced.
• A description of experimental procedures in sufficient detail to allow another researcher to repeat the methodology exactly, without requiring further information. These procedures must be adhered to exactly in the subsequent experiments or any Stage 2 manuscript can be rejected.
• Proposed analysis pipeline, including all preprocessing steps, and a precise description of all planned analyses, including appropriate correction for multiple comparisons. Any covariates or regressors must be stated. Where analysis decisions are contingent on the outcome of prior analyses, these contingencies must be specified and adhered to. Only pre-planned analyses can be reported in the main Results section of Stage 2 submissions. However, unplanned exploratory analyses will be admissible in a separate section of the Results (see below).
• Studies involving Neyman-Pearson inference must include a statistical power analysis. Estimated effect sizes should be justified with reference to the existing literature. Since publication bias overinflates published estimates of effect size, power analysis must be based on the lowest available or meaningful estimate of the effect size. For frequentist analysis plans, the a priori power must be 0.95 or higher for all proposed hypothesis tests. In the case of highly uncertain effect sizes, a variable sample size and interim data analysis is permissible but with inspection points stated in advance, appropriate Type I error correction for ‘peeking’ employed, and a final stopping rule for data collection outlined.
• Methods involving Bayesian hypothesis testing are encouraged. For studies involving analyses with Bayes factors, the predictions of the theory must be specified so that a Bayes factor can be calculated. Authors should indicate what distribution will be used to represent the predictions of the theory and how its parameters will be specified. For example, will you use a uniform up to some specified maximum, or a normal/half-normal to represent a likely effect size, or a JZS/Cauchy with a specified scaling constant? For inference by Bayes factors, authors must be able to guarantee data collection until the Bayes factor is at least 10 times in favour of the experimental hypothesis over the null hypothesis (or vice versa). Authors with resource limitations are permitted to specify a maximum feasible sample size at which data collection must cease regardless of the Bayes factor; however to be eligible for advance acceptance this number must be sufficiently large that inconclusive results at this sample size would nevertheless be an important message for the field. For further advice on Bayes factors or Bayesian sampling methods, prospective authors are encouraged to read this key article by Schönbrodt and Wagenmakers.
• Full descriptions must be provided of any outcome-neutral criteria that must be met for successful testing of the stated hypotheses. Such quality checks might include the absence of floor or ceiling effects in data distributions, positive controls, or other quality checks that are orthogonal to the experimental hypotheses.
• Timeline for completion of the study and proposed resubmission date if Stage 1 review is successful. Extensions to this deadline can be negotiated with the editorial office.
• Any description of prospective methods or analysis plans should be written in future tense.

Pilot Data (Optional)

• Can be included to establish proof of concept, effect size estimations, or feasibility of proposed methods. Must be included to demonstrate feasibility for studies implementing a novel desig Any pilot experiments will be published with the final version of the manuscript and will be clearly distinguished from data obtained for the pre-registered experiment(s).

Stage 1 submissions that are judged by the editors to be of sufficient quality and scientific importance will be sent for peer review. Reviewers will be asked to assess:

1. The importance of the research question(s) and the relevance of the research for a specialised field of Medicine or Public Health.
2. The extent to which the proposed study can satisfactorily answer the research question(s).
3. The logic, rationale, and plausibility of the proposed hypotheses.
4. The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis where appropriate).
5. Whether the clarity and degree of methodological detail is sufficient to exactly replicate the proposed experimental procedures and analysis pipeline.
6. Whether the authors have pre-specified sufficient outcome-neutral tests for ensuring that the results obtained are able to test the stated hypotheses, including positive controls and quality checks.

Following Stage 1 peer review, manuscripts will be rejected outright, offered the opportunity to revise, or accepted in principle (AIP). Manuscripts that receive an AIP decision will be published pending completion of the approved experimental design plan, passing of all quality checks, and a defensible interpretation of the results. Stage 1 manuscripts are not published in the journal following AIP. Instead they are registered by the authors in a recognized repository (either publicly or under embargo until Stage 2) and integrated into a single completed article following approval of the final Stage 2 manuscript.

Any major deviations from the stated experimental procedures could lead to rejection of the manuscript at Stage 2. In cases where the registered experimental design is altered after AIP due to unforeseen circumstances (e.g. change of equipment or unanticipated technical error), the authors should consult the editors immediately for advice, prior to the completion of data collection. Minor changes to the experimental design may be permitted according to editorial discretion. In such cases, AIP would be preserved and the deviation reported in the Stage 2 submission. If the authors wish to alter the experimental procedures more substantially following AIP but still wish to publish their article as a Registered Report, they must contact the handling editor to discuss the situation. If the changes are deemed too significant, authors might be asked to withdraw their Registered Report and resubmit it as a new Stage 1 submission. Note that the already-registered analyses must be undertaken, but additional unregistered analyses can also be included in a final manuscript (see below).

Stage 2: Full manuscript review

Once the study is complete, authors prepare and resubmit their full Stage 2 manuscript for review with a brief cover letter.

The Stage 2 cover letter should include confirmation that:

• the manuscript includes a link to a public repository containing anonymized study data, digital experimental materials, computer code (if applicable), and the laboratory log. Within the manuscript, this information should appear in two separate sections, entitled ‘Data availability’ and ‘Code availability’ (if applicable).
• the manuscript contains a link to the approved Stage 1 protocol in a standalone section entitled ‘Protocol Registration’.
• for primary Registered Reports, no data for any pre-registered study (other than pilot data included at Stage 1) was collected prior to the date of AIP.

Manuscript preparation guidelines – Stage 2

Stage 2 submissions should include the following sections:

Introduction

• Apart from minor stylistic revisions, the Introduction should not be altered from the approved Stage 1 submission. At Stage 2, any description of the rationale or proposed methodology that was written in future tense within the Stage 1 manuscript should be changed to past tense. Any textual changes to the Introduction or Methods (e.g. correction of typographic errors) must be clearly marked in the Stage 2 submission. Any relevant literature that appeared following the date of AIP should instead be cited in the Discussion.

Methods

• The Methods should be the same as in the Stage 1 manuscript, except any descriptions written in future tense within the Stage 1 manuscript should be changed to past tense.
• Any changes or deviations from the original Stage 1 experimental plan should be clearly marked in the Stage 2 submission.
• Data Availability and Code Availability statements should be included.
• The Stage 2 manuscript must also contain a link to the registered protocol (deposited following AIP) in a standalone subsection entitled ‘Protocol Registration’.

Results

• The outcome of all registered analyses must be reported in the manuscript, except in rare instances where a registered and approved analysis is subsequently shown to be logically flawed or unfounded. In such cases, the authors, reviewers, and editors must agree that a collective error of judgment was made and that the analysis is inappropriate. In such cases the analysis would still be mentioned in the Methods but omitted with justification from the Results.
• Only pre-planned analyses should be reported in the main Results section of the Stage 2 submission.
• Authors may wish to include additional analyses that were not included in the registered submission. For instance, a new approach might become available between AIP and Stage 2 review, or a particularly interesting and unexpected finding may emerge. Such analyses are admissible but must be clearly justified in the text, appropriately caveated, and described in a separate section of the Results titled “Exploratory analyses”. Authors should be careful not to base their conclusions entirely on the outcome of these post hoc analyses.
• Authors reporting null hypothesis significance tests are required to report exact p values, effect sizes and confidence intervals for all inferential analyses.

The Stage 2 submission will most likely be considered by the same reviewers as in Stage 1, but could also be assessed by new reviewers. In considering papers at Stage 2, reviewers will be asked to decide:

1. Whether the data are able to test the authors’ proposed hypotheses by satisfying the approved outcome-neutral conditions (such as quality checks, positive controls)
2. Whether the Introduction, rationale and stated hypotheses are the same as the approved Stage 1 submission (required)
3. Whether the authors adhered precisely to the registered experimental procedures
4. Whether any unregistered post hoc analyses added by the authors are justified, methodologically sound, and informative
5. Whether the authors’ conclusions are justified given the data

Reviewers are informed that editorial decisions will not be based on the perceived importance, novelty or conclusiveness of the results. Thus while reviewers are free to enter such comments on the record, they will not influence editorial decisions. Reviewers at Stage 2 may suggest that authors perform additional post hoc tests on their data. However, authors are not obliged to do so unless such tests are necessary to satisfy one or more of the Stage 1 review criteria.

Discussion

• Authors should briefly discuss their findings, including any limitations of the study.

Supplementary Information, laboratory log, data and code

• Supplementary figures, tables, or other text (such as supplementary methods) should be included as standard supplementary information that accompanies the paper. The raw data itself should be archived in a public repository (see above) rather than submitted to the journal as supplementary material.
• Anonymized raw data, any digital experimental materials, and computer code (if applicable) must be made freely available in a public repository. Authors are free to use any repository that renders data and materials freely and publicly accessible and provides a digital object identifier (DOI) to ensure that the data remain persistent, unique and citable.
• Data files should be appropriately time-stamped to show that data was collected after AIP and not before. Other than pre-registered and approved pilot data, no data acquired prior to the date of AIP is admissible in the Stage 2 submission. Raw data must be accompanied by guidance notes, where required, to assist other scientists in replicating the analysis pipeline. Authors are also expected to upload any relevant analysis scripts and other experimental materials that would assist in replication.
• A basic laboratory log must also be provided outlining the range of dates during which data collection took place. This log should be uploaded to the same public repository as the data and materials.

Manuscript withdrawal and Withdrawn Registrations

It is possible that authors with an AIP may wish to withdraw their manuscript following or during data collection. Possible reasons could include major technical error, an inability to complete the study due to other unforeseen circumstances, or the desire to submit the results to a different journal. In all such cases, manuscripts can of course be withdrawn at the authors’ discretion. However, the journal will publicly record each case in a section called Withdrawn Registrations. This section will include the authors, proposed title, the abstract from the approved Stage 1 submission, and brief reasons for the failure to complete the study. Partial withdrawals are not possible; i.e. authors cannot publish part of a registered study by selectively withdrawing one of the planned experiments. Such cases must lead to withdrawal of the entire paper.

Guidelines for Reviewers

The review process for Registered Reports is divided into two stages. At Stage 1, reviewers assess study proposals before data are collected. At Stage 2, reviewers consider the full study, including the results and its interpretation.

Stage 1 manuscripts will include only an Introduction, Methods (including proposed analyses), and Pilot Data (where applicable). In considering papers at Stage 1, reviewers will be asked to assess:

1. The importance of the research question(s) and relevance for a specialist audience within Psychology
2. The extent to which the proposed study can satisfactorily answer the research question(s)
3. The logic, rationale, and plausibility of the proposed hypotheses
4. The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis)
5. Whether the clarity and degree of methodological detail would be sufficient to replicate exactly the proposed experimental procedures and analysis pipeline
6. Whether the authors provide a sufficiently clear and detailed description of the methods to prevent undisclosed flexibility in the experimental procedures or analysis pipeline
7. Whether the authors have considered sufficient outcome-neutral conditions (e.g. absence of floor or ceiling effects; positive controls) for ensuring that the results obtained are able to test the stated hypotheses.

Following Stage 1 peer review, manuscripts will be accepted in principle, offered the opportunity to revise, or rejected outright. Acceptance in principle indicates that the article will be published pending successful completion of the study according to the pre-registered methods and analytic procedures, as well as a defensible and evidence-based interpretation of the results.

Following completion of the study, authors will complete the manuscript, including Results and Discussion sections. These Stage 2 manuscripts will more closely resemble a regular article format. The manuscript will then be returned to the same reviewers (if available), who will be asked to appraise:

1. Whether the data are able to test the authors’ proposed hypotheses by satisfying the approved outcome-neutral conditions (such as quality checks, positive controls)
2. Whether the Introduction, rationale and stated hypotheses are the same as the approved Stage 1 submission (required)
3. Whether the authors adhered precisely to the registered experimental procedures
4. Whether any unregistered post hoc analyses added by the authors are justified, methodologically sound, and informative
5. Whether the authors’ conclusions are justified given the data

Reviewers at Stage 2 may suggest that authors report additional post hoc tests on their data. However, authors are not obliged to do so unless such tests are necessary to satisfy one or more of the Stage 2 review criteria. Please note that editorial decisions will not be based on the perceived importance, novelty, or conclusiveness of the results.