In 1969, I skipped school to watch the Moon landing from home. Fifty years later, I struggle to think of an event that would justify truancy today. It’s not for lack of stunning breakthroughs in research, but rather their frequency: if children neglected their work every time the television reported another scientific milestone that my generation scarcely dared to contemplate, they’d end up with no education at all.
Yet there is a growing rumble of concern about the rigour and reproducibility of published research. Problems of over-hyped analysis and puffed-up CVs are well recognized. Financial and career incentives keep researchers on a treadmill, churning out papers.
We cannot know how many of the 1.6 million or so papers now added every year to the Web of Science database are flawed as a consequence, but we can agree that our focus has to shift from quantity to quality if we are to safeguard against shoddy work.
As Australia’s chief scientist, I am charged both with advising our government officials and with advocating for better science nationally and globally. Late last year, I organized a high-level round table in Canberra to discuss approaches that have been tried, and how the best of those might be upgraded to commitments. We comprised just over a dozen vice-chancellors, heads of granting agencies, deputy vice-chancellors, chief scientists, senior researchers and publications specialists, including Philip Campbell, former editor-in-chief of Nature. We spoke honestly about difficult topics.
I came away with several ideas on how to leap from good intentions to better research.
More than anything, we must abandon the assumption that a passive apprentice system works. I was lucky to train under a great scientist, neuroscientist Steve Redman at the Australian National University in Canberra. He would throw us into deep water to teach us to swim. But we were hardly left to drown. We were immersed in an environment in which papers took time, and questions and self-reflection were encouraged. Redman gave me leeway and introductions to gain the expertise I needed, encouraging me to take undergraduate courses, spend weeks in a colleague’s lab and learn essentials from technicians. In that swimming pool, I developed a deep understanding of what it means to be a scientist, without any specialized training from the scientific system.
Not so today. We can no longer rely on a model that assumes time and resources our mentors do not have. Institutions must provide explicit instruction in research integrity, data management and professional expectations. Training is already compulsory in some jurisdictions and applications. For example, in the United States, training fellowships often require courses in responsible conduct in research. People working with animals in Australia must complete an animal-handling course. However, all this training varies widely in quality and is often seen as a pro forma exercise. To ensure that training is time well spent, the programmes should be accredited, practical and respected.
Institutions should also be required to train PhD supervisors in mentorship, and on leaders’ roles in creating a healthy research culture. Furthermore, supervisors and mentors should be judged not by head count, but by impact statements about the projects and career progression of at least two PhD students; ideally, at least one woman and one man. I know of no institutions that currently require such a practice.
Next, institutions must heed growing calls to abandon paper counting and similar metrics for evaluating researchers. One alternative approach, the Rule of Five, demonstrates a clear commitment to quality: candidates present their best five papers over the past five years, accompanied by a description of the research, its impact and their individual contribution. The exact numbers are immaterial: what matters is the focus on quality. A handful of institutions have required reviewers to consider individual contributions rather than lists of publications, and the shift has not been easy. Reviewers should be admonished for Googling individuals’ h-indices and citation lists, for example. Perseverance and self-reflection are essential.
Funders should take the lead in pushing these changes. Only applicants who have completed an accredited integrity training programme should be eligible for grants. Agencies such as Australia’s National Health and Medical Research Council have already adopted the Rule of Five for some programmes. They should go further, and ensure that evaluations give weight to an applicant’s contribution to professional practices — such as mentoring and peer review.
Journals need to shift from being knowledge distributors to being knowledge custodians. Custodians do not publish and forget. They ensure that data remain accessible and prioritize concerns about research quality. This includes prompt investigations when published research is suspected to be flawed. A retracted paper should not vanish. It should be clearly marked ‘RETRACTED’ on journals’ websites, and some effort should be made to notify readers and citers of it. These standards have been codified by the Committee on Publication Ethics and the Center for Open Science. Their adoption lags across many publishers and, as a result, retracted claims propagate endlessly. Analyses have found that many papers receive more citations after retraction than before.
People respond to incentives. Change will come only when grants and promotions are contingent on best practice. If this doesn’t happen, we will still be talking about this issue during the next Moon landings.
Nature 566, 297 (2019)