Five years ago, I was part of a small group of ‘activists’ who convinced the Berlin Institute of Health (BIH), where I work, to try out a set of reforms intended to improve the trustworthiness, usefulness and ethics of research. Things grew from there: three years ago, with the help of government grants and some nudging by a retired local politician, we secured €2.5 million (US$2.9 million) per year for efforts to build up incentives and technologies that increase rigour.
We were inspired by initiatives at other universities, such as the reforms that Frank Miedema introduced during his deanship at the University Medical Center Utrecht in the Netherlands. But when the QUEST Center (QUEST stands for Quality, Ethics, Open Science and Translation) launched at the BIH, there was no precedent or blueprint for a programme of this scale.
From the beginning, we presumed that researchers and clinician–scientists are skilled professionals who want to ‘do the right thing’ but are also under pressure to accrue publications to advance their careers. Doing quality research takes time and humility, so unless we changed the system, researchers who pursued quality-enhancing practices could have found themselves at a disadvantage.
What was the solution? We made sure that we were viewed as a resource, not a policing unit. We selected interventions that we thought we could implement. Alongside introducing courses on experimental design and methods aimed at reducing bias, we focused on practices to increase the transparency of research. One push was for the use of electronic laboratory notebooks (ELNs), which improve research documentation and make collaboration easier. We made sure that QUEST, and not individual labs, covered the licence fees and provided plenty of support. So far, nearly 2,000 of our 7,000 researchers, PhD students and technicians are registered ELN users; my guess is that about half of these have an ELN as their primary lab notebook. For many, ELNs are a necessary first step towards systematically managing their research data, which QUEST also supports.
We simultaneously adjusted the incentive and reward system. When hiring professors and awarding institutional funds, we now consider how thoroughly and quickly people share their results. Those who make original data available in publications are rewarded with a financial bonus that can be spent on research. QUEST works with the BIH and the leadership of the Charité, Berlin’s university medical centre, to ensure that evaluation criteria encompass responsible research practices, including publication of null results, provision of open data and community engagement. A QUEST good-evaluation-practice officer has sat as an independent assessor on hiring commissions for 10 of the past 29 hiring calls.
We tried to craft a system designed for its own improvement. For example, we have developed an anonymous online tool through which researchers have reported hundreds of errors and worrying incidents (U. Dirnagl et al. PLoS Biol. 14, e2000705; 2016). This has allowed us to learn from errors — for example, a technician realized that ambiguous labelling of cell-culture media by a manufacturer had spoiled her experiment. Her swift reporting prevented others from making the same mistake. The company changed the labels on its flasks and alerted other customers. After we saw many errors stemming from the use of pipettes outside the calibrated range, we set up ‘pipetting exercises’ and saw the rate of these errors fall.
Three years in, we’re seeing more papers published open access and with open data. We’re also seeing greater participation in educational activities and in intramural programmes using responsible selection criteria, such as engagement with patient communities, reuse of data or preregistration. Of course, funders and journals are also pulling in the same direction, so it is impossible to know to which changes are due to the efforts of QUEST.
However, we still have a long way to go. Our benchmarking study found that, within 2 years of completion, only 40% of studies sponsored by the Charité had reported results (S. Wieschowski et al. J. Clin. Epidemiol. 115, 37–45; 2019). Furthermore, 5 years after completion, more than 30% of results remained unavailable. But we hope to correct this. We use counselling and web tools to offer guidance on how to publish null, inconclusive, negative and other ‘nonstandard’ results, and award monetary research bonuses for the publication of negative results or replication studies.
Most faculty members welcome our activities, and we are working to expand student and researcher engagement.
For example, using funding from the biomedical research charity Wellcome in London, we have established fellowships for mid-career researchers who collaborate to develop and track initiatives for improving science in their own research groups. Our experience shows that structured programmes can be rolled out by any academic institution that is willing and able to improve its research in a systematic fashion. The budget of QUEST is less than 1% of our institution’s state funding for research and teaching, not including monies from third-party funders.
QUEST started from scratch. But many institutions already promote activities such as open science, data management and responsible research. If they align their efforts, they can expand them and incorporate scientific ideals into incentive structures. The quality of science and the culture of the workplace will be better off.
Nature 588, 197 (2020)