Give every paper a read for reproducibility

I was hired to ferret out errors and establish routines that promote rigorous research, says Catherine Winchester.
Catherine Winchester is the grants- and research-integrity adviser at the Cancer Research UK Beatson Institute in Glasgow.

Search for this author in:

In 2012, I saw an advertisement for an unusual new role, and knew I was the one for the job. Leaders at the Cancer Research UK Beatson Institute, a non-profit organization in Glasgow, had created a position to weave discussion of and practices for research integrity into the daily routines of its roughly 300 researchers.

I had some ideas about rigorous research from working as a molecular biologist for almost 20 years, supervising graduate students and running a team. I think the best way to boost research quality is to discuss it often and freely. Intentional misconduct is rare, but even raising the topic of bad practices and sloppy mistakes can be sensitive — the elephant in the room that no one wants to talk about.

In addition to helping with grant submissions, my role is to support data curation and improve manuscript quality, and to provide training and a first point of contact for concerns about integrity. I thoroughly read every research manuscript that our scientists submit to journals, about 50 primary papers each year. I receive warm thanks much more often than cold shoulders.

I strive to be the friendly face behind a serious issue, raising awareness, changing behaviour and working to create an environment in which discussing research integrity is a normal, non-confrontational part of doing science. I meet everyone personally, including postdocs, PhD students and technicians — ideally, during their first week on the job — and I have integrated myself as a scientist at the institute. People expect to see me at seminars and social events.

Perhaps the most complex undertaking so far has been developing practices for curating and preserving all the data that underpin a paper, including replicates. This took about a year, working with senior faculty members, the information-technology team and another research manager. We trialled our data-archiving system with a couple of groups, implemented it across our institute for a year and then amended it on the basis of feedback. Instead of squirrelling away data in individual folders and lab books, researchers now archive all published data in a designated central drive, so that the information is accessible for the long haul. Initially, people thought the process was just extra bureaucratic work, or that it had been invented so I could police their data. Now, it has become the norm, and researchers tell me they save time and worry by having their data organized and archived.

Feedback about my reviews has been positive, especially because, as a fresh set of eyes, I can sometimes spot mistakes that someone closer to the work might not see. I’ve pointed out duplicated image panels, missing data and mislabelled images, among other problems. I also check manuscript texts for plagiarism, using software such as iThenticate, and check figures to look for inadvertent duplications or inappropriate manipulation. I have incorporated elements of journal checklists, including Nature’s technical and statistical checklist, into my reviews for reporting accuracy, experimental design and analyses. Occasionally, I suggest using a different statistical test or way of presenting data (plotting individual points rather than bar charts, for instance). The goal is to improve a manuscript’s quality without trying to emulate peer review.

Checks are compulsory but informal. I promote my work as an extra pair of eyes, and I can usually complete checks in a week or two, because I have already read researchers’ grant applications and have access to the data to answer most queries myself. Sometimes, I get a response that reads like a rebuttal to a peer reviewer; most often, it feels collaborative.

Training provides an informal forum in which to discuss research-integrity issues. Face-to-face, tailored training brings integrity matters into the open and provides relevant, practical guidance. For example, I run 90-minute workshops on data management, responsible image preparation, statistical considerations and avoiding plagiarism. Training is mandatory for everyone. For new principal investigators, it is one on one, whereas postdocs, graduate students, technicians and research assistants meet in groups of up to ten people. Graduate students receive academic credit; they are, after all, learning the essentials for being a scientist. I update curricula regularly to incorporate new policies and news stories, but the sessions work best when people talk about a publication issue or problem they have experienced themselves.

Five years on, we’ve had no retractions or serious issues with publications from our institute, and I feel that my and my colleagues’ efforts have improved practice much more than would just putting a policy on a webpage. My door is always open. If someone does raise an issue, our approach allows for informal discussions and discrete enquiries before any formal investigation is initiated. This can prevent things from escalating unnecessarily, and it also takes some of the pressures off researchers who might want to raise concerns.

Could most university departments have research-integrity advisers? Yes — but these people need to have a research background in that discipline, and be embedded in researchers’ day-to-day affairs. It would add to headcount, but in the long run it would save money: high-quality research is easier to build from, and misconduct investigations get pricey quickly. More importantly, they consume many investigators’ lives, and damage credibility and public trust in science. Hopefully, initiatives such as this one will all go some way towards creating a culture in which more scientists are willing to talk about — and tame — the elephant in the room.

Nature 557, 281 (2018)

doi: 10.1038/d41586-018-05140-x
Nature Briefing

Sign up for the daily Nature Briefing email newsletter

Stay up to date with what matters in science and why, handpicked from Nature and other publications worldwide.

Sign Up