Millions of mice and rats are used in research each year. But one-third to one-half of animal experiments are never published, and of those that are, many are too poorly conducted to be reliable. Advocates for better animal research and reproducibility are promoting a strategy established in other fields to counter publication bias, improve investigations and increase transparency: study registries.
Registries ask researchers to detail their hypotheses, experimental strategy and analytical plans before studies begin. The intention is to prevent teams from simply cherry-picking significant or desirable findings and to supply the scientific community with a way of learning about experiments that would otherwise go unpublished.
The best-known registry, clinicaltrials.gov, has logged more than 300,000 human clinical trials since it launched in 2000, amid outrage over drug companies burying unfavourable clinical-trial results. Regulatory authorities around the world now require registration for drugs and devices approved for market, and medical journals require it for publication.
The Open Science Framework is an example of a voluntary registration system. Researchers, mainly psychologists and social scientists, input or ‘preregister’ research plans before starting a project, which they can keep private, or ‘embargoed’, for up to four years. More than 30,500 preregistrations have been entered since 2012, but few of these involve animals.
The first registry specifically set up for animal studies, preclinicaltrials.eu, was launched in April 2018. Registry co-founder Mira van der Naald and her colleagues at the University Medical Center Utrecht in the Netherlands were carrying out systematic reviews in cardiac regenerative medicine, and found themselves frustrated by the consistently poor quality of preclinical evidence. They felt a dedicated registry would help, and were surprised that none existed. “We thought, ‘Hey, let’s just start it. We’re not getting anywhere just talking about it’.”
Unknown to them, Germany’s centre for the protection of laboratory animals, Bf3R in Berlin, had taken on a similar project. Animalstudyregistry.org launched in January. Together, the two registries have only a few dozen entries.
The registries use templates specifically designed for animal experiments, with fields for species as well as several experimental design parameters described in a set of reporting guidelines known as Animal Research: Reporting of In Vivo Experiments, or ARRIVE. (In 2017, M.B. served on a working group to update these guidelines, which ask authors to state whether they have preregistered their experiment.) Curators at both registries review entries and can ask for more detail. Registration is open to researchers worldwide.
A tough sell
Malcolm Macleod, a stroke researcher at the University of Edinburgh, UK, who has documented research quality and bias in preclinical work, says that for journal editors and peer reviewers, registration can boost a study’s credibility. “Registries and preregistration are pretty essential in terms of being able to demonstrate the rigour with which the research was done, and to reassure research users that you answered the questions that you set out to answer,” he says.
But convincing researchers to use animal-study registries could prove to be a tough sell, he says. “We are going to ask a group of researchers who have not had any experience with this at all to suddenly change what they do.” Researchers are used to communicating their work as a final manuscript that describes experiments and findings as if everything went to plan, notes physiologist Kieron Rooney, a registry advocate at the University of Sydney, Australia. “You don’t see any battle scars of my project, where I had to change direction.”
Scientists largely agree that registration would yield communal advantages by reducing cherry-picking, publication bias and duplication, says Daniel Strech, a bioethicist at Charité Medical University in Berlin who studies animal researchers’ attitudes to study registration. But they also worry about individual disadvantages such as increased administrative burden, the possibility of having their ideas stolen and being targeted by animal-rights activists (S. Wieschowski et al. PLoS Biol. 14, e2000391; 2016). “They think, on average, animal registries will have no impact on efficiencies,” Strech says.
Researchers who have submitted protocols to their animal-ethics committees or funding agencies can simply paste relevant portions into the registry, upload supporting files (animalstudyregistry.org) or provide URLs (preclinicaltrials.eu). Finalized registrations are time-stamped, but researchers can add annotations to explain deviations from the plan, or to flag that further experiments have been done. The registries also provide secure embargo periods.
Still unclear, however, is which types of study should be registered. Bf3R head Gilbert Schönfelder encourages researchers to log any study requiring approval from an institute’s ethical advisory board. This helps to advance the ethical aim that any experiment using animals should increase the overall level of knowledge. Bioethicist Jonathan Kimmelman at McGill University in Montreal, Canada, counters that the push should be for researchers doing preclinical trials — highly structured studies that serve as the basis for deciding whether to test a drug in people.
To Macleod, the optimal registration process would target confirmatory studies that set out to test (rather than generate) hypotheses and require no more than half an hour to complete, even if that means omitting some details. If researchers documented half a dozen items including the hypothesis, experimental intervention, primary outcome and how it will be measured, and statistical parameters, “you deal with 95% of the problems that arise”. It would also increase the number of entries for those studies in which registration is most advantageous, he says. “If people get 90% of the benefits for ten minutes, I think that would be much more likely to happen than getting 100% of the benefits for two hours.”
Also worth logging are animal housing and handling details, says Adrian Smith, secretary of Norecopa, an organization in Oslo that aims to improve and reduce the use of animals in research. Isolating mice or picking them up by the tail can strongly impact certain types of study, he notes. “It is unthinkable to try and solve the reproducibility crisis without also attending to these ‘non-mathematical’ factors.”
Broad participation and fully described experiments are key, says Deborah Zarin, who from 2005 to 2018 directed clinicaltrials.gov. Yet it could prove difficult to get researchers to provide sufficient detail in their registrations to really know whether they are cherry-picking results, she warns. Also, the fewer researchers who participate in a registry, the less valuable it will be for helping others to identify collaborators, or to know whether anyone else has tried to address similar questions. And separate, uncoordinated registries will make searching for particular kinds of study inefficient, further undermining their use.
Even incomplete registries could promote “good researcher hygiene” that would improve individual studies, says Kimmelman. Still, the availability of a registry is just one piece of the puzzle, says Manoj Lalu, an anaesthesiologist at Ottawa Hospital Research Institute, who is working to improve translational research. Many researchers do not understand why techniques to reduce bias are necessary or how they should be implemented. This means that even if they do register a study, they might do so inaccurately. Thus, registries must be combined with educational resources, he says. Incentives are also essential, adds Roberta Scherer, who studies clinical-trial methodology at Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland. “If researchers go to the site, they may become educated, but they have to get there first.” Funders, journals and institutions will have to require or reward registration for it to become common practice, she predicts.
Rooney says that a better strategy would be to show that registries can benefit researchers by helping them to find collaborators or determine whether and how to repeat studies other researchers have tried. “We have to say we want this not because we want to make science difficult, but because we want to fix some issues,” he says. “Give it a few years, and it just becomes part of the process.”
Nature 573, 297-298 (2019)