Low statistical power reduces the reliability of animal research; yet, increasing sample sizes to increase statistical power is problematic for both ethical and practical reasons. We present an alternative solution using Bayesian priors based on historical control data, which capitalizes on the observation that control groups in general are expected to be similar to each other. In a simulation study, we show that including data from control groups of previous studies could halve the minimum sample size required to reach the canonical 80% power or increase power when using the same number of animals. We validated the approach on a dataset based on seven independent rodent studies on the cognitive effects of early-life adversity. We present an open-source tool, RePAIR, that can be widely used to apply this approach and increase statistical power, thereby improving the reliability of animal experiments.
Subscribe to Journal
Get full journal access for 1 year
only $17.42 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
The data that support the findings of the current study can be downloaded from https://osf.io/wvs7m/.
All code used in this manuscript is available at https://osf.io/wvs7m/.
Ioannidis, J. P. A. Why most published research findings are false. PLoS Med. 2, e124 (2005).
Button, K. S. et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365–376 (2013).
Smaldino, P. E. & McElreath, R. The natural selection of bad science. R. Soc. Open Sci. 3, 160384 (2016).
Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1, 0021 (2017).
Crabbe, J. C., Wahlsten, D. & Dudek, B. C. Genetics of mouse behavior: interactions with laboratory environment. Science 284, 1670–1672 (1999).
Prinz, F., Schlange, T. & Asadullah, K. Believe it or not: how much can we rely on published data on potential drug targets? Nat. Rev. Drug Discov. 10, 712–713 (2011).
Voelkl, B. et al. Reproducibility of animal research in light of biological variation. Nat. Rev. Neurosci. 21, 384–393 (2020).
Macleod, M. & Mohan, S. Reproducibility and rigor in animal-based research. ILAR J. 60, 17–23 (2019).
Bonapersona, V. et al. The behavioral phenotype of early life adversity: a 3-level meta-analysis of rodent studies. Neurosci. Biobehav. Rev. 102, 299–307 (2019).
Gurevitch, J., Koricheva, J., Nakagawa, S. & Stewart, G. Meta-analysis and the science of research synthesis. Nature 555, 175–182 (2018).
Richter, S. H., Garner, J. P. & Würbel, H. Environmental standardization: cure or cause of poor reproducibility in animal experiments? Nat. Methods 6, 257–261 (2009).
Karp, N. A. Reproducible preclinical research—is embracing variability the answer? PLoS Biol. 16, e2005413 (2018).
Richter, S. H. et al. Effect of population heterogenization on the reproducibility of mouse behavior: a multi-laboratory study. PLoS ONE 6, e16461 (2011).
Kramer, M. & Font, E. Reducing sample size in experiments with animals: historical controls and related strategies. Biol. Rev. Philos. Soc. 92, 431–445 (2017).
Brakenhoff, T., Roes, K. & Nikolakopoulos, S. Bayesian sample size re-estimation using power priors. Stat. Methods Med. Res. 28, 1664–1675 (2018).
Spiegelhalter, D. J., Abrams, K. R. & Myles, J. P. Bayesian Approaches to Clinical Trials and Health-Care Evaluation 1 (Wiley, 2004).
Galwey, N. W. Supplementation of a clinical trial by historical control data: is the prospect of dynamic borrowing an illusion? Stat. Med. 36, 899–916 (2017).
Mutsvari, T., Tytgat, D. & Walley, R. Addressing potential prior-data conflict when using informative priors in proof-of-concept studies. Pharm. Stat. 15, 28–36 (2016).
Walley, R. et al. Using Bayesian analysis in repeated preclinical in vivo studies for a more effective use of animals. Pharm. Stat. 15, 277–285 (2016).
Nord, C. L., Valton, V., Wood, J. & Roiser, J. P. Power-up: a reanalysis of ‘power failure’ in neuroscience using mixture modeling. J. Neurosci. 37, 8051–8061 (2017).
Cohen, J. Statistical Power Analysis for the Behavioral Sciences (Lawrence Erlbaum Associates, 1977).
Ibrahim, J. G. & Chen, M. H. Power prior distributions for regression models. Stat. Sci. 15, 46–60 (2000).
Rice, C. J., Sandman, C. A., Lenjavi, M. R. & Baram, T. Z. A novel mouse model for acute and long-lasting consequences of early life stress. Endocrinology 149, 4892–4900 (2008).
Percie du Sert, N. et al. The ARRIVE guidelines 2.0: updated guidelines for reporting animal research. PLoS Biol. 18, e3000410 (2020).
Ioannidis, J. P. A. Why most discovered true associations are inflated. Epidemiology 19, 640–648 (2008).
Rubin, E. J. & Fortune, S. M. Misunderstanding the goals of animal research. BMJ 360, 29321149 (2018).
Gelman, A. Objections to Bayesian statistics. Bayesian Anal. 3, 445–450 (2008).
Viele, K. et al. Use of historical control data for assessing treatment effects in clinical trials. Pharm. Stat. 13, 41–54 (2014).
Neuenschwander, B., Capkun-Niggli, G., Branson, M. & Spiegelhalter, D. J. Summarizing historical information on controls in clinical trials. Clin. Trials 7, 5–18 (2010).
O’Hagan, A. Expert knowledge elicitation: subjective but scientific. Am. Stat. 73, 69–81 (2019).
van der Naald, M., Wenker, S., Doevendans, P. A., Wever, K. E. & Chamuleau, S. A. J. Publication rate in preclinical research: a plea for preregistration. BMJ Open Sci. 4, e100051 (2020).
Du Sert, N. P. et al. The experimental design assistant. Nat. Methods 14, 1024–1025 (2017).
Crabbe, J. C. & Phillips, T. J. Mother nature meets mother nurture. Nat. Neurosci. 6, 440–442 (2003).
Kafkafi, N. et al. Addressing reproducibility in single-laboratory phenotyping experiments. Nat. Methods 14, 462–464 (2017).
Shansky, R. M. Are hormones a ‘female problem’ for animal research? Science 364, 825–826 (2019).
Bonapersona, V., Joëls, M. & Sarabdjitsingh, R. A. Effects of early life stress on biochemical indicators of the dopaminergic system: a 3 level meta-analysis of rodent studies. Neurosci. Biobehav. Rev. 95, 1–16 (2018).
Wickham, H. et al. Welcome to the Tidyverse. J. Open Source Softw. 4, 1686 (2019).
Chang, W., Cheng, J., Allaire, J., Xie, Y. & McPherson, J. shiny: Web Application Framework for R. R Package version 1.5.0 https://CRAN.R-project.org/package=shiny (2020).
Ekstrøm, C. T. MESS: Miscellaneous Esoteric Statistical Scripts. R package version 0.5.6 https://CRAN.R-project.org/package=MESS (2019).
Faul, F., Erdfelder, E., Lang, A. G. & Buchner, A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 39, 175–191 (2007).
Lenth, R. V. Some practical guidelines for effective sample size determination. Am. Stat. 55, 187–193 (2001).
Gelman, A., Carlin, J. B., Stern, H. S. & Rubin, D. B. Bayesian Data Analysis. (Chapman & Hall, 1995).
We thank J. Knop and M. Sep for helpful discussions and R. de Kloet for critically reviewing the manuscript. This work was supported by the Consortium of Individual Development (CID), which is funded by the Gravitation program of the Dutch Ministry of Education, Culture and Science and the Netherlands Organization for Scientific Research (NWO grant no. 024.001.003).
The authors declare no competing interests.
Peer review information Nature Neuroscience thanks Stanley Lazic, Malcolm MacLeod, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Bonapersona, V., Hoijtink, H., RELACS Consortium. et al. Increasing the statistical power of animal experiments with historical control data. Nat Neurosci (2021). https://doi.org/10.1038/s41593-020-00792-3