Special |

Challenges in irreproducible research

Science moves forward by corroboration – when researchers verify others’ results. Science advances faster when people waste less time pursuing false leads. No research paper can ever be considered to be the final word, but there are too many that do not stand up to further study.

There is growing alarm about results that cannot be reproduced.  Explanations include increased levels of scrutiny, complexity of experiments and statistics, and pressures on researchers. Journals, scientists, institutions and funders all have a part in tackling reproducibility. Nature has taken substantive steps to improve the transparency and robustness in what we publish, and to promote awareness within the scientific community. We hope that the articles contained in this collection will help.

Key reads

The solutions adopted by the high-energy physics community to foster reproducible research are examples of best practices that could be embraced more widely. This first experience suggests that reproducibility requires going beyond openness.

Perspective | open | | Nature Physics

Replicating our work took four years and 100,000 worms but brought surprising discoveries, explain Gordon J. Lithgow, Monica Driscoll and Patrick Phillips.

Comment | | Nature News

A survey of Nature readers revealed a high level of concern about the problem of irreproducible results. Researchers, funders and journals need to work together to make research more reliable.

Editorial | | Nature News

Latest

Software that uncovers suspicious papers will do little for a community that does not confront organized research fraud, says Jennifer Byrne.

World View | | Nature

As the imaging technique produces ever-sharper protein structures, researchers are racing to develop tools to assess how accurate they are.

Technology Feature | | Nature

All articles

Software that uncovers suspicious papers will do little for a community that does not confront organized research fraud, says Jennifer Byrne.

World View | | Nature

Customize the experiment for the setting instead of adjusting the setting to fit a classical design.

This Month | | Nature Methods

The publishing system builds in resistance to replication. Paul Gertler, Sebastian Galiani and Mauricio Romero surveyed economics journals to find out how to fix it.

Comment | | Nature

As debate rumbles on about how and how much poor statistics is to blame for poor reproducibility, Nature asked influential statisticians to recommend one change to improve science. The common theme? The problem is not our maths, but ourselves.

Comment | | Nature

Efforts to reduce irreproducibility in research must also tackle the temptation to cheat, argue Donald S. Kornfeld and Sandra L. Titus.

Comment | | Nature News

Start-up firms say robotics and software that autonomously record every detail of an experiment can transform the efficiency and reliability of research.

Toolbox | | Nature News

Papers in Nature journals should make computer code accessible where possible.

Editorial | | Nature News

Scientists and journals must work together to ensure that eye-catching artefacts are not trumpeted as genomic insights, says Daniel MacArthur.

Comment | | Nature

Biologists must realize the pitfalls of work on massive amounts of data.

Editorial | | Nature

Research & Reviews

Customize the experiment for the setting instead of adjusting the setting to fit a classical design.

This Month | | Nature Methods

The results of in vitro and in vivo screens to identify genes that are essential for the survival of a type of brain cancer show almost no overlap, underlining the need for caution when interpreting in vitro studies. See Letter p355.

News & Views | | Nature

Quality control of cell lines used in biomedical research is essential to ensure reproducibility. Although cell line authentication has been widely recommended for many years, misidentification, including cross-contamination, remains a serious problem. We outline a multi-stakeholder, incremental approach and policy-related recommendations to facilitate change in the culture of cell line authentication.

Commentary | | Nature Methods

The reliability and reproducibility of science are under scrutiny. However, a major cause of this lack of repeatability is not being considered: the wide sample-to-sample variability in the P value. We explain why P is fickle to discourage the ill-informed practice of interpreting analyses based predominantly on this statistic.

Commentary | | Nature Methods

'Irreproducibility' is symptomatic of a broader challenge in measurement in biomedical research. From the US National Institute of Standards and Technology (NIST) perspective of rigorous metrology, reproducibility is only one aspect of establishing confidence in measurements. Appropriate controls, reference materials, statistics and informatics are required for a robust measurement process. Research is required to establish these tools for biological measurements, which will lead to greater confidence in research results.

Commentary | | Nature Methods

Low-powered studies lead to overestimates of effect size and low reproducibility of results. In this Analysis article, Munafò and colleagues show that the average statistical power of studies in the neurosciences is very low, discuss ethical implications of low-powered studies and provide recommendations to improve research practices.

Analysis | | Nature Reviews Neuroscience

A wealth of microarray gene expression data and a growing volume of RNA sequencing data are now available in public databases. The authors look at how these data are being used and discuss considerations for how such data should be analysed and deposited and how data reuse could be improved.

Review Article | | Nature Reviews Genetics

Scientific reproducibility now very often depends on the computational method being available to duplicate, so here it is argued that all source code should be freely available.

Perspective | | Nature

Too many sloppy mistakes are creeping into scientific papers. Lab heads must look more rigorously at the data — and at themselves.

Editorial | | Nature