Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • COMMENT

One statistical analysis must not rule them all

Any single analysis hides an iceberg of uncertainty. Multi-team analysis can reveal it.
  1. Eric-Jan Wagenmakers

    1. Eric-Jan Wagenmakers is a methodologist at the University of Amsterdam, the Netherlands.

    View author publications

    You can also search for this author in PubMed  Google Scholar

  2. Alexandra Sarafoglou

    1. Alexandra Sarafoglou is a postdoctoral fellow at the University of Amsterdam, the Netherlands.

    View author publications

    You can also search for this author in PubMed  Google Scholar

  3. Balazs Aczel

    1. Balazs Aczel is vice-dean for science at Eötvös Loránd University in Budapest, Hungary.

    View author publications

    You can also search for this author in PubMed  Google Scholar

Cartoon showing a lone researcher in a boat looking at an iceberg with many researchers under the water also inspecting it

Illustration by David Parkins

A typical journal article contains the results of only one analysis pipeline, by one set of analysts. Even in the best of circumstances, there is reason to think that judicious alternative analyses would yield different outcomes.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Nature 605, 423-425 (2022)

doi: https://doi.org/10.1038/d41586-022-01332-8

References

  1. Scientific Pandemic Influenza Group on Modelling. SPI-M-O: Consensus statement on COVID-19, 8 October 2020 (2020).

  2. Botvinik-Nezer, R. et al. Nature 582, 84–88 (2020).

    Article  PubMed  Google Scholar 

  3. Menkveld, A. J. et al. Preprint at SSRN https://doi.org/10.2139/ssrn.3961574 (2021).

  4. Silberzahn, R. et al. Adv. Methods Pract. Psychol. Sci. 1, 337–356 (2018).

    Article  Google Scholar 

  5. Steegen, S., Tuerlinckx, F., Gelman, A. & Vanpaemel, W. Perspect. Psychol. Sci. 11, 702–712 (2016).

    Article  PubMed  Google Scholar 

  6. Hoogeveen, S. et al. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/pbfye (2022).

  7. Marek, S. et al. Nature 603, 654–660 (2022).

    Article  PubMed  Google Scholar 

  8. Wagenmakers, E.-J. et al. Nature Hum. Behav. 5, 1473–1480 (2021).

    Article  PubMed  Google Scholar 

  9. Aczel, B. et al. eLife 10, e72185 (2021).

    Article  PubMed  Google Scholar 

  10. van Dongen, N. N. N. et al. Am. Stat. 73, 328–339 (2019).

    Article  Google Scholar 

  11. Button, K. Nature 561, 287 (2018).

    Article  PubMed  Google Scholar 

Download references

Supplementary Information

  1. Supplementary table

Competing Interests

The authors declare no competing interests.

Subjects

Latest on:

Nature Careers

Jobs

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing

Search

Advanced search

Quick links