Francis Collins, director of the US National Institutes of Health, recently highlighted in Nature the need to identify and correct systematic problems in biomedical research. One such effort, the Stanford Meta-Research Innovation Center, will monitor the practice of research and suggest policies for improvement. We commend this initiative that supports our commitment to publishing scientifically rigorous research.
The success rates of grant applications to the US funding agencies the National Institutes of Health (NIH) and the National Science Foundation (NSF) are at an all-time low, with little relief in sight. This makes the allocation of grant money to poorly planned, badly executed research all the more frustrating. Not all research findings can turn out to be true—that's the nature of research—but there are ways to avoid sloppy science.
Nearly a decade has passed since John Ioannidis wrote “Why most published research findings are false” (PLoS Med. 2, e124, 2005), and the scientific community is still struggling with how to improve reproducibility and minimize false positives in the literature. Outright fraud is still, thankfully, rare (Proc. Natl. Acad. Sci. USA 109, 17028–17033, 2012), but, in a world of omics data and small effect sizes, the design and execution of statistically robust studies that account for multiple hypotheses remains a substantial challenge.
In response to growing concern over the inability to replicate many studies (Nature 505, 612–613, 2014 and http://www.nature.com/nature/focus/reproducibility/, but see Nature 503, 333–334, 2013 for a compelling counterargument), John Ioannidis and Steven Goodman have founded a center in the Department of Health Research and Policy at Stanford University. The newly minted Meta-Research Innovation Center (METRICS) has set an ambitious agenda for itself: to identify and minimize “persistent threats to biomedical/scientific research quality” (http://med.stanford.edu/metrics/). METRICS will serve as the home base for the new field of meta-research, which will benefit from a bird's-eye view of trends across a wide range of research fields to identify what works and what does not. These trends can then be used to inform new models and policies for data sharing and the standardization of research practices.
The list of goals for the new center is admirable but will take the participation of the scientific community to achieve. Journals have an integral role in applying best standards to publications. We are constantly improving the transparency and rigor of the research we publish and have recently implemented, with all Nature journals, a transparency checklist for authors as part of this goal. Many of the directives set forth by METRICS—sharing of protocols and analysis, making data accessible and interpretable, such as through Scientific Data, and declaring conflicts of interest—are already implemented across Nature journals. But there is much more we can do, and we look forward to the recommendations that METRICS will develop through the course of their 'research on research'.
In the meantime, there are ways that we as a journal can help our authors to develop strong publications that will remain useful research tools for a considerable time, from the study design stage through to presentation and data sharing. First, communication is key. Many of our policies and standards can already be found in our Guide to Authors, but these can and should be communicated much more clearly and prominently. We will soon be interactively discussing our basic criteria, such as journal standards and referee recommendations in each field of genetics and requirements for statistical reporting and data access policies, on our journal's blog Free Association (http://blogs.nature.com/freeassociation/). We hope to receive feedback that will allow us to strengthen and clarify our standards. One measure of success in this project will be the number of skeptical researchers we persuade to submit papers to the journal as a consequence of this open discussion of our scientific values.
In addition to these changes, we will continue to update our standards on the basis of interactions with researchers at conferences and institutional visits. We are also happy to provide editorial advice to authors at all stages of research through presubmission inquiries and one-on-one conversations. And, importantly in this age of genomic analysis, we will continue to improve data access and transparency, including a requirement that, whenever possible, all data be available to peer reviewers.
Biomedical research has led to some of the most important advances in human health. The application of scientific objectivity to improve research practice will ensure that it continues to do so.
About this article
Cite this article
Researching the researchers. Nat Genet 46, 417 (2014). https://doi.org/10.1038/ng.2972