Nature | World View
Challenges in irreproducible research
Science moves forward by corroboration – when researchers verify others’ results. Science advances faster when people waste less time pursuing false leads. No research paper can ever be considered to be the final word, but there are too many that do not stand up to further study.
There is growing alarm about results that cannot be reproduced. Explanations include increased levels of scrutiny, complexity of experiments and statistics, and pressures on researchers. Journals, scientists, institutions and funders all have a part in tackling reproducibility. Nature has taken substantive steps to improve the transparency and robustness in what we publish, and to promote awareness within the scientific community. We hope that the articles contained in this collection will help.
Key reads
-
-
Nature | Editorial
Checklists work to improve science
Nature authors say a reproducibility checklist is a step in the right direction, but more needs to be done.
-
Nature | World View
Give every paper a read for reproducibility
I was hired to ferret out errors and establish routines that promote rigorous research, says Catherine Winchester.
- Catherine Winchester
-
Nature News | Comment
A long journey to reproducible results
Replicating our work took four years and 100,000 worms but brought surprising discoveries, explain Gordon J. Lithgow, Monica Driscoll and Patrick Phillips.
- Gordon J. Lithgow
- , Monica Driscoll
- & Patrick Phillips
-
Nature News | Editorial
Reality check on reproducibility
A survey of Nature readers revealed a high level of concern about the problem of irreproducible results. Researchers, funders and journals need to work together to make research more reliable.
-
Nature News | News Feature
1,500 scientists lift the lid on reproducibility
Survey sheds light on the ‘crisis’ rocking research.
- Monya Baker
Latest
-
Nature | Technology Feature
Cryo-electron microscopy shapes up
As the imaging technique produces ever-sharper protein structures, researchers are racing to develop tools to assess how accurate they are.
- Monya Baker
-
Nature | World View
Reboot undergraduate courses for reproducibility
Collaboration across institutes can train students in open, team science, which better prepares them for challenges to come, says Katherine Button.
- Katherine Button
-
Nature | News
High-profile journals put to reproducibility test
Researchers replicated 62% of social-behaviour findings published in Science and Nature — a result matched almost exactly by a prediction market.
- Philip Ball
-
Nature | World View
No more excuses for non-reproducible methods
Online technologies make it easy to share precise experimental protocols — and doing so is essential to modern science, says Lenny Teytelman.
- Lenny Teytelman
-
Nature | Technology Feature
A toolkit for data transparency takes shape
A simple software toolset can help to ease the pain of reproducing computational analyses.
- Jeffrey M. Perkel
-
Nature | World View
Before reproducibility must come preproducibility
Instead of arguing about whether results hold up, let’s push to provide enough information for others to repeat the experiments, says Philip Stark.
- Philip B. Stark
All articles
-
Nature Methods | Analysis | open
Precision and accuracy of single-molecule FRET measurements—a multi-laboratory benchmark study
A multi-laboratory study finds that single-molecule FRET is a reproducible and reliable approach for determining accurate distances in dye-labeled DNA duplexes.
- Björn Hellenkamp
- , Sonja Schmid
- , Olga Doroshenko
- , Oleg Opanasyuk
- , Ralf Kühnemuth
- , Soheila Rezaei Adariani
- , Benjamin Ambrose
- , Mikayel Aznauryan
- , Anders Barth
- , Victoria Birkedal
- , Mark E. Bowen
- , Hongtao Chen
- , Thorben Cordes
- , Tobias Eilert
- , Carel Fijen
- , Christian Gebhardt
- , Markus Götz
- , Giorgos Gouridis
- , Enrico Gratton
- , Taekjip Ha
- , Pengyu Hao
- , Christian A. Hanke
- , Andreas Hartmann
- , Jelle Hendrix
- , Lasse L. Hildebrandt
- , Verena Hirschfeld
- , Johannes Hohlbein
- , Boyang Hua
- , Christian G. Hübner
- , Eleni Kallis
- , Achillefs N. Kapanidis
- , Jae-Yeol Kim
- , Georg Krainer
- , Don C. Lamb
- , Nam Ki Lee
- , Edward A. Lemke
- , Brié Levesque
- , Marcia Levitus
- , James J. McCann
- , Nikolaus Naredi-Rainer
- , Daniel Nettels
- , Thuy Ngo
- , Ruoyi Qiu
- , Nicole C. Robb
- , Carlheinz Röcker
- , Hugo Sanabria
- , Michael Schlierf
- , Tim Schröder
- , Benjamin Schuler
- , Henning Seidel
- , Lisa Streit
- , Johann Thurn
- , Philip Tinnefeld
- , Swati Tyagi
- , Niels Vandenberk
- , Andrés Manuel Vera
- , Keith R. Weninger
- , Bettina Wünsch
- , Inna S. Yanez-Orozco
- , Jens Michaelis
- , Claus A. M. Seidel
- , Timothy D. Craggs
- & Thorsten Hugel
-
Nature Methods | This Month
Optimal experimental design
Customize the experiment for the setting instead of adjusting the setting to fit a classical design.
- Byran Smucker
- , Martin Krzywinski
- & Naomi Altman
-
Nature Methods | Technology Feature
Microbiology: making the best of PCR bias
Many factors can skew the results of a widely used amplification technique for microbiome analysis, but researchers are finding strategies for getting at the truth.
- Michael Eisenstein
-
Nature Methods | Correspondence
What's in a sample? Increasing transparency in biospecimen procurement methods
- Joshua LaBaer
- , Joseph F Miceli III
- & Leonard P Freedman
-
Nature Methods | Analysis
Bias, robustness and scalability in single-cell differential expression analysis
An extensive evaluation of differential expression methods applied to single-cell expression data, using uniformly processed public data in the new conquer resource.
- Charlotte Soneson
- & Mark D Robinson
-
Nature | Comment
How to make replication the norm
The publishing system builds in resistance to replication. Paul Gertler, Sebastian Galiani and Mauricio Romero surveyed economics journals to find out how to fix it.
- Paul Gertler
- , Sebastian Galiani
- & Mauricio Romero
Research & Reviews
-
Nature Methods | Analysis | open
Precision and accuracy of single-molecule FRET measurements—a multi-laboratory benchmark study
A multi-laboratory study finds that single-molecule FRET is a reproducible and reliable approach for determining accurate distances in dye-labeled DNA duplexes.
- Björn Hellenkamp
- , Sonja Schmid
- , Olga Doroshenko
- , Oleg Opanasyuk
- , Ralf Kühnemuth
- , Soheila Rezaei Adariani
- , Benjamin Ambrose
- , Mikayel Aznauryan
- , Anders Barth
- , Victoria Birkedal
- , Mark E. Bowen
- , Hongtao Chen
- , Thorben Cordes
- , Tobias Eilert
- , Carel Fijen
- , Christian Gebhardt
- , Markus Götz
- , Giorgos Gouridis
- , Enrico Gratton
- , Taekjip Ha
- , Pengyu Hao
- , Christian A. Hanke
- , Andreas Hartmann
- , Jelle Hendrix
- , Lasse L. Hildebrandt
- , Verena Hirschfeld
- , Johannes Hohlbein
- , Boyang Hua
- , Christian G. Hübner
- , Eleni Kallis
- , Achillefs N. Kapanidis
- , Jae-Yeol Kim
- , Georg Krainer
- , Don C. Lamb
- , Nam Ki Lee
- , Edward A. Lemke
- , Brié Levesque
- , Marcia Levitus
- , James J. McCann
- , Nikolaus Naredi-Rainer
- , Daniel Nettels
- , Thuy Ngo
- , Ruoyi Qiu
- , Nicole C. Robb
- , Carlheinz Röcker
- , Hugo Sanabria
- , Michael Schlierf
- , Tim Schröder
- , Benjamin Schuler
- , Henning Seidel
- , Lisa Streit
- , Johann Thurn
- , Philip Tinnefeld
- , Swati Tyagi
- , Niels Vandenberk
- , Andrés Manuel Vera
- , Keith R. Weninger
- , Bettina Wünsch
- , Inna S. Yanez-Orozco
- , Jens Michaelis
- , Claus A. M. Seidel
- , Timothy D. Craggs
- & Thorsten Hugel
-
Nature Human Behaviour | Letter
Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015
Camerer et al. carried out replications of 21 Science and Nature social science experiments, successfully replicating 13 out of 21 (62%). Effect sizes of replications were about half of the size of the originals.
- Colin F. Camerer
- , Anna Dreber
- , Felix Holzmeister
- , Teck-Hua Ho
- , Jürgen Huber
- , Magnus Johannesson
- , Michael Kirchler
- , Gideon Nave
- , Brian A. Nosek
- , Thomas Pfeiffer
- , Adam Altmejd
- , Nick Buttrick
- , Taizan Chan
- , Yiling Chen
- , Eskil Forsell
- , Anup Gampa
- , Emma Heikensten
- , Lily Hummer
- , Taisuke Imai
- , Siri Isaksson
- , Dylan Manfredi
- , Julia Rose
- , Eric-Jan Wagenmakers
- & Hang Wu
-
Nature Methods | This Month
Optimal experimental design
Customize the experiment for the setting instead of adjusting the setting to fit a classical design.
- Byran Smucker
- , Martin Krzywinski
- & Naomi Altman
-
Nature | News & Views
Cancer: Keeping it real to kill glioblastoma
The results of in vitro and in vivo screens to identify genes that are essential for the survival of a type of brain cancer show almost no overlap, underlining the need for caution when interpreting in vitro studies. See Letter p355.
- Paul A. Northcott
-
Nature Methods | Correspondence
What's in a sample? Increasing transparency in biospecimen procurement methods
- Joshua LaBaer
- , Joseph F Miceli III
- & Leonard P Freedman
-
Nature Methods | Technology Feature
Microbiology: making the best of PCR bias
Many factors can skew the results of a widely used amplification technique for microbiome analysis, but researchers are finding strategies for getting at the truth.
- Michael Eisenstein
Before reproducibility must come preproducibility
Checklists work to improve science
Give every paper a read for reproducibility
A long journey to reproducible results
1,500 scientists lift the lid on reproducibility
Cryo-electron microscopy shapes up
Reboot undergraduate courses for reproducibility
High-profile journals put to reproducibility test
No more excuses for non-reproducible methods
A toolkit for data transparency takes shape
Precision and accuracy of single-molecule FRET measurements—a multi-laboratory benchmark study
Optimal experimental design
Microbiology: making the best of PCR bias
Bias, robustness and scalability in single-cell differential expression analysis
How to make replication the norm
Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015
Cancer: Keeping it real to kill glioblastoma