Special |

Challenges in irreproducible research

Science moves forward by corroboration – when researchers verify others’ results. Science advances faster when people waste less time pursuing false leads. No research paper can ever be considered to be the final word, but there are too many that do not stand up to further study.

There is growing alarm about results that cannot be reproduced.  Explanations include increased levels of scrutiny, complexity of experiments and statistics, and pressures on researchers. Journals, scientists, institutions and funders all have a part in tackling reproducibility. Nature has taken substantive steps to improve the transparency and robustness in what we publish, and to promote awareness within the scientific community. We hope that the articles contained in this collection will help.

Key reads

  • Nature Physics | Perspective | open

    The solutions adopted by the high-energy physics community to foster reproducible research are examples of best practices that could be embraced more widely. This first experience suggests that reproducibility requires going beyond openness.

    • Xiaoli Chen
    • , Sünje Dallmeier-Tiessen
    • , Robin Dasler
    • , Sebastian Feger
    • , Pamfilos Fokianos
    • , Jose Benito Gonzalez
    • , Harri Hirvonsalo
    • , Dinos Kousidis
    • , Artemis Lavasa
    • , Salvatore Mele
    • , Diego Rodriguez Rodriguez
    • , Tibor Šimko
    • , Tim Smith
    • , Ana Trisovic
    • , Anna Trzcinska
    • , Ioannis Tsanaktsidis
    • , Markus Zimmermann
    • , Kyle Cranmer
    • , Lukas Heinrich
    • , Gordon Watts
    • , Michael Hildreth
    • , Lara Lloret Iglesias
    • , Kati Lassila-Perini
    •  &  Sebastian Neubert
  • Nature News | Comment

    Replicating our work took four years and 100,000 worms but brought surprising discoveries, explain Gordon J. Lithgow, Monica Driscoll and Patrick Phillips.

    • Gordon J. Lithgow
    • , Monica Driscoll
    •  &  Patrick Phillips
  • Nature News | Editorial

    A survey of Nature readers revealed a high level of concern about the problem of irreproducible results. Researchers, funders and journals need to work together to make research more reliable.

Latest

All articles

  • Nature Methods | Analysis | open

    A multi-laboratory study finds that single-molecule FRET is a reproducible and reliable approach for determining accurate distances in dye-labeled DNA duplexes.

    • Björn Hellenkamp
    • , Sonja Schmid
    • , Olga Doroshenko
    • , Oleg Opanasyuk
    • , Ralf Kühnemuth
    • , Soheila Rezaei Adariani
    • , Benjamin Ambrose
    • , Mikayel Aznauryan
    • , Anders Barth
    • , Victoria Birkedal
    • , Mark E. Bowen
    • , Hongtao Chen
    • , Thorben Cordes
    • , Tobias Eilert
    • , Carel Fijen
    • , Christian Gebhardt
    • , Markus Götz
    • , Giorgos Gouridis
    • , Enrico Gratton
    • , Taekjip Ha
    • , Pengyu Hao
    • , Christian A. Hanke
    • , Andreas Hartmann
    • , Jelle Hendrix
    • , Lasse L. Hildebrandt
    • , Verena Hirschfeld
    • , Johannes Hohlbein
    • , Boyang Hua
    • , Christian G. Hübner
    • , Eleni Kallis
    • , Achillefs N. Kapanidis
    • , Jae-Yeol Kim
    • , Georg Krainer
    • , Don C. Lamb
    • , Nam Ki Lee
    • , Edward A. Lemke
    • , Brié Levesque
    • , Marcia Levitus
    • , James J. McCann
    • , Nikolaus Naredi-Rainer
    • , Daniel Nettels
    • , Thuy Ngo
    • , Ruoyi Qiu
    • , Nicole C. Robb
    • , Carlheinz Röcker
    • , Hugo Sanabria
    • , Michael Schlierf
    • , Tim Schröder
    • , Benjamin Schuler
    • , Henning Seidel
    • , Lisa Streit
    • , Johann Thurn
    • , Philip Tinnefeld
    • , Swati Tyagi
    • , Niels Vandenberk
    • , Andrés Manuel Vera
    • , Keith R. Weninger
    • , Bettina Wünsch
    • , Inna S. Yanez-Orozco
    • , Jens Michaelis
    • , Claus A. M. Seidel
    • , Timothy D. Craggs
    •  &  Thorsten Hugel
  • Nature Methods | This Month

    Customize the experiment for the setting instead of adjusting the setting to fit a classical design.

    • Byran Smucker
    • , Martin Krzywinski
    •  &  Naomi Altman
  • Nature Methods | Technology Feature

    Many factors can skew the results of a widely used amplification technique for microbiome analysis, but researchers are finding strategies for getting at the truth.

    • Michael Eisenstein
  • Nature | Comment

    The publishing system builds in resistance to replication. Paul Gertler, Sebastian Galiani and Mauricio Romero surveyed economics journals to find out how to fix it.

    • Paul Gertler
    • , Sebastian Galiani
    •  &  Mauricio Romero

Research & Reviews

  • Nature Methods | Analysis | open

    A multi-laboratory study finds that single-molecule FRET is a reproducible and reliable approach for determining accurate distances in dye-labeled DNA duplexes.

    • Björn Hellenkamp
    • , Sonja Schmid
    • , Olga Doroshenko
    • , Oleg Opanasyuk
    • , Ralf Kühnemuth
    • , Soheila Rezaei Adariani
    • , Benjamin Ambrose
    • , Mikayel Aznauryan
    • , Anders Barth
    • , Victoria Birkedal
    • , Mark E. Bowen
    • , Hongtao Chen
    • , Thorben Cordes
    • , Tobias Eilert
    • , Carel Fijen
    • , Christian Gebhardt
    • , Markus Götz
    • , Giorgos Gouridis
    • , Enrico Gratton
    • , Taekjip Ha
    • , Pengyu Hao
    • , Christian A. Hanke
    • , Andreas Hartmann
    • , Jelle Hendrix
    • , Lasse L. Hildebrandt
    • , Verena Hirschfeld
    • , Johannes Hohlbein
    • , Boyang Hua
    • , Christian G. Hübner
    • , Eleni Kallis
    • , Achillefs N. Kapanidis
    • , Jae-Yeol Kim
    • , Georg Krainer
    • , Don C. Lamb
    • , Nam Ki Lee
    • , Edward A. Lemke
    • , Brié Levesque
    • , Marcia Levitus
    • , James J. McCann
    • , Nikolaus Naredi-Rainer
    • , Daniel Nettels
    • , Thuy Ngo
    • , Ruoyi Qiu
    • , Nicole C. Robb
    • , Carlheinz Röcker
    • , Hugo Sanabria
    • , Michael Schlierf
    • , Tim Schröder
    • , Benjamin Schuler
    • , Henning Seidel
    • , Lisa Streit
    • , Johann Thurn
    • , Philip Tinnefeld
    • , Swati Tyagi
    • , Niels Vandenberk
    • , Andrés Manuel Vera
    • , Keith R. Weninger
    • , Bettina Wünsch
    • , Inna S. Yanez-Orozco
    • , Jens Michaelis
    • , Claus A. M. Seidel
    • , Timothy D. Craggs
    •  &  Thorsten Hugel
  • Nature Human Behaviour | Letter

    Camerer et al. carried out replications of 21 Science and Nature social science experiments, successfully replicating 13 out of 21 (62%). Effect sizes of replications were about half of the size of the originals.

    • Colin F. Camerer
    • , Anna Dreber
    • , Felix Holzmeister
    • , Teck-Hua Ho
    • , Jürgen Huber
    • , Magnus Johannesson
    • , Michael Kirchler
    • , Gideon Nave
    • , Brian A. Nosek
    • , Thomas Pfeiffer
    • , Adam Altmejd
    • , Nick Buttrick
    • , Taizan Chan
    • , Yiling Chen
    • , Eskil Forsell
    • , Anup Gampa
    • , Emma Heikensten
    • , Lily Hummer
    • , Taisuke Imai
    • , Siri Isaksson
    • , Dylan Manfredi
    • , Julia Rose
    • , Eric-Jan Wagenmakers
    •  &  Hang Wu
  • Nature Methods | This Month

    Customize the experiment for the setting instead of adjusting the setting to fit a classical design.

    • Byran Smucker
    • , Martin Krzywinski
    •  &  Naomi Altman
  • Nature | News & Views

    The results of in vitro and in vivo screens to identify genes that are essential for the survival of a type of brain cancer show almost no overlap, underlining the need for caution when interpreting in vitro studies. See Letter p355.

    • Paul A. Northcott
  • Nature Methods | Technology Feature

    Many factors can skew the results of a widely used amplification technique for microbiome analysis, but researchers are finding strategies for getting at the truth.

    • Michael Eisenstein