Reproducibility, rigour, transparency and independent verification are cornerstones of the scientific method. Of course, just because a result is reproducible does not make it right, and just because it is not reproducible does not make it wrong. A transparent and rigorous approach, however, will almost always shine a light on issues of reproducibility. This light ensures that science moves forward, through independent verifications as well as the course corrections that come from refutations and the objective examination of the resulting data.
It was with the goal of strengthening such approaches in the biomedical sciences that a group of editors representing more than 30 major journals; representatives from funding agencies; and scientific leaders assembled at the American Association for the Advancement of Science’s headquarters in June 2014 to discuss principles and guidelines for preclinical biomedical research. The gathering was convened by the US National Institutes of Health, Nature and Science (see Science 346, 679; 2014).
The discussion ranged from what journals were already doing to address reproducibility — and the effectiveness of those measures — to the magnitude of the problem and the cost of solutions. The attendees agreed on a common set of Principles and Guidelines in Reporting Preclinical Research (see go.nature.com/ezjl1p) that list proposed journal policies and author reporting requirements in order to promote transparency and reproducibility.
“The guidelines encourage using a checklist to ensure reporting of important experimental parameters.”
The guidelines recommend that journals include in their information for authors their policies for statistical analysis and how they review the statistical accuracy of work under consideration. Any imposed page limits should not discourage reproducibility. The guidelines encourage using a checklist to ensure reporting of important experimental parameters, such as standards used, number and type of replicates, statistics, method of randomization, whether experiments were blinded, how the sample size was determined and what criteria were used to include or exclude any data. Journals should recommend deposition of data in public repositories, where available, and link data bidirectionally when the paper is published. Journals should strongly encourage, as appropriate, that all materials used in the experiment be shared with those who wish to replicate the experiment. Once a journal publishes a paper, it assumes the obligation to consider publication of a refutation of that paper, subject to its usual standards of quality.
The more open-ended portion of the guidelines suggests that journals establish best practices for dealing with image-based data (for example, screening for manipulation, storing full-resolution archival versions) and for describing experiments in full. An example for animal experiments is to report the source, species, strain, sex, age, husbandry and inbred and strain characteristics for transgenic animals. For cell lines, one might report the source, authentication and mycoplasma contamination status. The existence of these guidelines does not obviate the need for replication or independent verification of research results, but should make it easier to perform such replication.
Some of the journals at the meeting had already had all or most of these principles and guidelines in place. But the point is that a large number of scientific journals are standing together in their conviction that reproducibility and transparency are important issues. As partners to the research enterprise in the communication and dissemination of research results, we want to do our part to raise the standards for the benefit of scientists and of society. The hope is that these guidelines will be viewed not as onerous, but as part of the quality control that justifies the public trust in science.
- Journal name:
- Date published: