In 2013, Nature began asking the authors of life-sciences papers to provide extra information in a bid to tackle the pressing problem of poor reproducibility in research. According to one survey of Nature authors conducted in 2016–17, 86% of respondents considered poor reproducibility to be a growing challenge in the life sciences.
Researchers in these fields are now asked to use a structured reporting summary for their manuscript submissions. Among other things, the checklist requires authors to state whether their experimental findings have been replicated; how they determined an appropriate sample size; whether they randomized samples; and whether data have been assessed by researchers who did not know which group they were assessing.
Such a checklist, which is provided to peer reviewers and published with each life-sciences paper, has helped to improve transparency in the reporting of research1,2. But editors from many journals and researchers recognize that there is still work to be done.
In 2017, a group met to discuss how such a systematic approach to transparency and reproducibility could be improved and adopted across more journals. The result is the MDAR (Materials Design Analysis Reporting) Framework, which has just been published3.
The MDAR initiative is the result of an effort by editors at Science, Cell Press, the Public Library of Science, eLife, Wiley and in the Nature Portfolio, working with experts in reproducibility and research improvement. The objective is to encourage more-detailed disclosures in four areas of life-sciences manuscripts: materials (such as reagents, laboratory animals and model organisms); data; analysis (including code and statistics); and reporting (adhering to discipline-specific guidelines). Nature’s standards cover most of the MDAR initiative’s objectives, but there are plans for further alignment. At the same time, the group is encouraging other journals beyond the founding members to sign up.
Publishers are not the only important players in this arena, however. A key test will be the extent to which funders and universities also support the new framework. Any initiative that improves transparency and reproducibility should be welcomed. But MDAR comes at a time when some of Europe’s largest funders have announced plans to reduce what they regard as burdens and bureaucracy in research. The European Commission, for example, is undertaking a review of its pharmaceuticals legislation, partly in an effort to reduce red tape. And the UK government has appointed Adam Tickell, vice-chancellor of the University of Sussex in Brighton, to lead a review with the explicit aim of reducing red tape for researchers.
For these funders, such measures are designed, in part, to remove perceived obstacles to innovation and competitiveness in science. But if the result is reduced funding for research management and administrative support — which are essential to the success of implementing quality measures — that will have an impact on efforts to improve transparency and reproducibility.
All of those involved — funders, publishers and research managers and administrators — need to be on the same page in this respect. Europe’s national and regional funders, in particular, must not forget that efforts to enhance transparency and reproducibility are fundamental to the scientific process — and to scientific integrity — and are far from being red tape.
Fortunately, many researchers appreciate this. In a pilot study in 2019, the MDAR checklist was tested by 33 journal editors and 211 authors working on 289 manuscripts. Most respondents from both groups said they found the expanded checklist helpful. And in response to Nature’s 2016–17 survey, some three-quarters of respondents said that they would use the journal’s checklist to some extent, whether or not they were planning to submit their draft to a Nature journal.
In a parallel and welcome development, researchers and publishers, including the Nature journals, are embracing a format called Registered Reports in which scientists submit a detailed plan for a research project, including the question or questions being asked, study design and methodology. If editors approve it for peer review, and reviewers think the proposal is sufficiently robust, the journal commits to publishing the work, regardless of the outcome.
All participants involved in the research process know that good research starts long before papers get written. Progress in science comes not with the sparkle of glitter or the crash of cymbals, but in carefully crafted prose after years of deliberations, experimental testing and continuous refinement. The MDAR Framework is one such achievement. The time has come for science institutions to catch up with the growing desire among researchers for greater transparency and reproducibility. MDAR won’t solve everything, but, if it can make research more reliable, then it will go some way towards achieving its promise.
Nature 593, 8 (2021)