Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Environmental standardization: cure or cause of poor reproducibility in animal experiments?

Abstract

It is widely believed that environmental standardization is the best way to guarantee reproducible results in animal experiments. However, mounting evidence indicates that even subtle differences in laboratory or test conditions can lead to conflicting test outcomes. Because experimental treatments may interact with environmental conditions, experiments conducted under highly standardized conditions may reveal local 'truths' with little external validity. We review this hypothesis here and present a proof of principle based on data from a multilaboratory study on behavioral differences between inbred mouse strains. Our findings suggest that environmental standardization is a cause of, rather than a cure for, poor reproducibility of experimental outcomes. Environmental standardization can contribute to spurious and conflicting findings in the literature and unnecessary animal use. This conclusion calls for research into practicable and effective ways of systematic environmental heterogenization to attenuate these scientific, economic and ethical costs.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Study design.
Figure 2: Variance between replicate experiments.
Figure 3: False positive and false negative rate.

References

  1. 1

    US Department of Agriculture, Animal and Plant Health Inspection Service. Animal Welfare Act 22 (Riverdale: U.S. Department of Agriculture, 1990).

  2. 2

    NRC (National Research Council). Guide for the Care and Use of Laboratory Animals (Washington, National Academy Press, 1996).

  3. 3

    Beynen, A.C., Gärtner, K. & van Zutphen, L.F.M. Standardization of animal experimentation In Principles of Laboratory Animal Science (eds., van Zutphen, L.F.M., Baumans, V. & Beynen, A.C.) 103–110 (Elsevier, Amsterdam, 2003).

    Google Scholar 

  4. 4

    Festing, M.F.W. Good experimental design and statistics can save animals, but how can it be promoted? Altern. Lab. Anim. 32, 133–135 (2004).

    CAS  PubMed  Google Scholar 

  5. 5

    Festing, M.F.W. Refinement and reduction through the control of variation. Altern. Lab. Anim. 32, 259–263 (2004).

    CAS  PubMed  Google Scholar 

  6. 6

    Quinn, G.P. & Keough, M.J. Hypothesis testing. In Experimental Design and Data Analysis for Biologists (eds., Quinn, G.P. & Keough, M.J.) 32–57 (Cambridge University Press, 2002).

    Chapter  Google Scholar 

  7. 7

    Crabbe, J.C., Wahlsten, D. & Dudek, B.C. Genetics of mouse behaviour: interactions with laboratory environment. Science 284, 1670–1672 (1999).

    CAS  Article  Google Scholar 

  8. 8

    Wahlsten, D. Standardizing tests of mouse behavior: reasons, recommendations, and reality. Physiol. Behav. 73, 695–705 (2001).

    CAS  Article  Google Scholar 

  9. 9

    Würbel, H. Behavioral phenotyping enhanced—beyond (environmental) standardization. Genes Brain Behav. 1, 3–8 (2002).

    Article  Google Scholar 

  10. 10

    Campbell, D.T. Factors relevant to the validity of experiments in social settings. Psychol. Bull. 54, 297–312 (1957).

    CAS  Article  Google Scholar 

  11. 11

    Lehner, P.N. Handbook of Ethological Methods, 2nd edn (Cambridge University Press, Cambridge, 1996).

    Google Scholar 

  12. 12

    Guala, F. Experimental localism and external validity. Philos. Sci. 70, 1195–1205 (2003).

    Article  Google Scholar 

  13. 13

    Van der Staay, F.J. Animal models of behavioral dysfunctions: Basic concepts and classifications, and an evaluation strategy. Brain Res. Brain Res. Rev. 52, 131–159 (2006).

    Article  Google Scholar 

  14. 14

    Crestani, F., Martin, J.R., Möhler, H. & Rudolph, U. Resolving differences in GABAA receptor mutant mouse studies. Nat. Neurosci. 3, 1059 (2000).

    CAS  Article  Google Scholar 

  15. 15

    Wahlsten, D., et al. Different data from different labs: lessons from studies of gene-environment interaction. J. Neurobiol. 54, 283–311 (2003).

    Article  Google Scholar 

  16. 16

    Wolfer, D.P. et al. Laboratory animal welfare: cage enrichment and mouse behaviour. Nature 432, 821–822 (2004).

    CAS  Article  Google Scholar 

  17. 17

    Kafkafi, N., Benjamini, Y., Sakov, A., Elmer, G.I. & Golani, I. Genotype-environment interactions in mouse behavior: A way out of the problem. Proc. Natl. Acad. Sci. USA 102, 4619–4624 (2005).

    CAS  Article  Google Scholar 

  18. 18

    Lewejohann, L. et al. Environmental bias? Effects of housing conditions, laboratory environment and experimenter on behavioral tests. Genes Brain Behav. 5, 64–72 (2006).

    CAS  Article  Google Scholar 

  19. 19

    Mandillo, S. et al. Reliability, robustness and reproducibility in mouse behavioral phenotyping: a cross-laboratory study. Physiol. Genomics 34, 243–255 (2008).

    Article  Google Scholar 

  20. 20

    Chesler, E.J., Wilson, S.G., Lariviere, W.R., Rodriguez-Zas, S.L. & Mogil, J.S. Identification and ranking of genetic and laboratory environment factors influencing a behavioral trait, thermal nociception, via computational analysis of a large data archive. Neurosci. Biobehav. Rev. 26, 907–923 (2002).

    Article  Google Scholar 

  21. 21

    Valdar, W. et al. Genetic and environmental effects on complex traits in mice. Genetics 174, 959–984 (2006).

    CAS  Article  Google Scholar 

  22. 22

    Wahlsten, D., Bachmanov, A., Finn, D.A. & Crabbe, J.C. Stability of inbred mouse strain differences in behavior and brain size between laboratories and across decades. Proc. Natl. Acad. Sci. USA 103, 16364–16369 (2006).

    CAS  Article  Google Scholar 

  23. 23

    Andrews, N. & File, S.E. Handling history of rats modifies behavioural effects of drugs in the elevated plus-maze test of anxiety. Eur. J. Pharmacol. 235, 109–112 (1993).

    CAS  Article  Google Scholar 

  24. 24

    Rampon, C. et al. Enrichment induces structural changes and recovery from nonspatial memory deficits in CA1 NMDAR1-knockout mice. Nat. Neurosci. 3, 238–244 (2000).

    CAS  Article  Google Scholar 

  25. 25

    Cabib, S., Orsini, C., Le Moal, M. & Piazza, P.V. Abolition and reversal of strain differences in behavioural responses to drugs of abuse after brief experience. Science 289, 463–465 (2000).

    CAS  Article  Google Scholar 

  26. 26

    Kas, M.J.H. & Van Ree, J.M. Dissecting complex behaviours in the post-genomic era. Trends Neurosci. 27, 366–369 (2004).

    CAS  Article  Google Scholar 

  27. 27

    Izidio, G.S., Lopes, D.M., Spricigio, L. & Ramos, A. Common variations in the pretest environment influence genotypic comparisons in models of anxiety. Genes Brain Behav. 4, 412–419 (2005).

    CAS  Article  Google Scholar 

  28. 28

    Öbrink, K.J. & Rehbinder, C. Animal definition: a necessity for the validity of animal experiments. Lab. Anim. 34, 121–130 (2000).

    Article  Google Scholar 

  29. 29

    van der Staay, F.J. & Steckler, T. Behavioural phenotyping of mouse mutants. Behav. Brain Res. 125, 3–12 (2001).

    CAS  Article  Google Scholar 

  30. 30

    van der Staay, F.J. & Steckler, T. The fallacy of behavioral phenotyping without standardisation. Genes Brain Behav. 1, 9–13 (2002).

    CAS  Article  Google Scholar 

  31. 31

    Würbel, H. Behaviour and the standardization fallacy. Nat. Genet. 26, 263 (2000).

    Article  Google Scholar 

  32. 32

    De Witt, T.J. Scheiner, S.M. Phenotypic Plasticity. Functional and Conceptual Approaches. (Oxford University Press, 2004).

    Google Scholar 

  33. 33

    Latham, N. & Mason, G. From house mouse to mouse house: the behavioural biology of free-living Mus musculus and its implications in the laboratory. Appl. Anim. Behav. Sci. 86, 261–289 (2004).

    Article  Google Scholar 

  34. 34

    Silver, L.M. Mouse genetics: concepts and applications (Oxford University Press, New York, 1995).

  35. 35

    Silva, A. J., et al. Mutant mice and neuroscience: recommendations concerning genetic background. Neuron 19, 755–759 (1997).

    Article  Google Scholar 

  36. 36

    Beynen, A.C., Festing, M.F.M. & van Montfort, M.A.J. Design of animal experiments In Principles of Laboratory Animal Science (eds., van Zutphen, L.F.M., Baumans, V. & Beynen, A.C.) 219–249 (Elsevier, Amsterdam, 2003).

    Google Scholar 

  37. 37

    Baggerly, K.A. Reproducibility of SELDI-TOF protein patterns in serum: comparing datasets from different experiments. Bioinformatics 20, 777 (2004).

    CAS  Article  Google Scholar 

  38. 38

    Anderson, C.A., Lindsay, J.J. & Bushman, B.J. Research in the psychological laboratory: truth or triviality? Curr. Dir. Psychol. Sci. 8, 3–9 (1999).

    Article  Google Scholar 

  39. 39

    Vissers, G., Heyne, G., Peters, V. & Guerts, J. The validity of laboratory research in social and behavioral science. Qual. Quant. 35, 129–145 (2001).

    Article  Google Scholar 

  40. 40

    Boehm, G.W. et al. Learning and memory in autoimmune BXSB mouse: Effects of neocorticaol ectopias and environmental enrichment. Brain Res. 726, 11–22 (1996).

    CAS  Article  Google Scholar 

  41. 41

    Chapillon, P., Manneche, C., Belzung, C. & Caston, J. Rearing environmental enrichment in two inbred strain of mice: 1. Effects on emotional reactivity. Behav. Genet. 29, 41–46 (1999).

    CAS  Article  Google Scholar 

  42. 42

    Cudilo, E., Al Naemi, H., Marmorstein, L. & Baldwin, A.L. Knockout mice: is it just genetics? Effect of enriched housing on Fibulin-4+/− mice. PLoS ONE 2, e229, (2007).

    Article  Google Scholar 

  43. 43

    Hascoet, M., Colombel, M.-C. & Bourin, M. Influence of age on behavioural response in the light/dark paradigm. Physiol. Behav. 66, 567–570 (1999).

    CAS  Article  Google Scholar 

  44. 44

    Imhof, J.T., Coelho, Z.M.I., Schmitt, M.L., Morato, G.S. & Carobrez, A.P. Influence of age and gender on performance of rats in the elevated plus maze apparatus. Behav. Brain Res. 56, 177–180 (1993).

    CAS  Article  Google Scholar 

  45. 45

    Poon, A.M. et al. Effect of cage size on ultradian locomotor rhythms of laboratory mice. Physiol. Behav. 62, 1253–1258 (1997).

    CAS  Article  Google Scholar 

  46. 46

    Soffie, M., Hahn, K., Terao, E. & Eclancher, F. Behavioural and glial changes in old rats following environmental enrichment. Behav. Brain Res. 101, 37–49 (1999).

    CAS  Article  Google Scholar 

  47. 47

    Van de Weerd, H.A., Baumans, V., Koolhaas, J.M. & van Zutphen, L.F. Strain specific behavioral response to environmental enrichment in the mouse. J. Exp. Anim. Sci. 36, 117–127 (1994).

    CAS  PubMed  Google Scholar 

  48. 48

    Russell, W.M.S. & Burch, R.L. The principles of humane experimental technique (Methuen, London, 1959).

Download references

Acknowledgements

This study was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft Project Wu 494/2-1) and the 3R Research Foundation Switzerland (3R Project 77-01). We thank K. Failing for help with data analysis, and M. Dawkins and P. Bateson for their comments on an earlier version of this paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hanno Würbel.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Richter, S., Garner, J. & Würbel, H. Environmental standardization: cure or cause of poor reproducibility in animal experiments?. Nat Methods 6, 257–261 (2009). https://doi.org/10.1038/nmeth.1312

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing