Automated home-cage behavioural phenotyping of mice

  • Nature Communications volume 1, Article number: 68 (2010)
  • doi:10.1038/ncomms1064
  • Download Citation
Published online:


Neurobehavioural analysis of mouse phenotypes requires the monitoring of mouse behaviour over long periods of time. In this study, we describe a trainable computer vision system enabling the automated analysis of complex mouse behaviours. We provide software and an extensive manually annotated video database used for training and testing the system. Our system performs on par with human scoring, as measured from ground-truth manual annotations of thousands of clips of freely behaving mice. As a validation of the system, we characterized the home-cage behaviours of two standard inbred and two non-standard mouse strains. From these data, we were able to predict in a blind test the strain identity of individual animals with high accuracy. Our video-based software will complement existing sensor-based automated approaches and enable an adaptable, comprehensive, high-throughput, fine-grained, automated analysis of mouse behaviour.

Additional access options:

Already a subscriber?  Log in  now or  Register  for online access.

Change history

  • Updated online 31 January 2012

    A correction has been published and is appended to both the HTML and PDF versions of this paper. The error has not been fixed in the paper.


  1. 1.

    & The European dimension for the mouse genome mutagenesis program. Nat. Genet. 36, 925–927 (2004).

  2. 2.

    , & Genetics of mouse behavior: interactions with laboratory environment. Science 284, 1670–1672 (1999).

  3. 3.

    & Hoxb8 is required for normal grooming behavior in mice. Neuron 33, 23–34 (2002).

  4. 4.

    et al. Autism-like behavioral phenotypes in BTBR T1tf/J mice. Genes Brain Behav. 7, 152–163 (2008).

  5. 5.

    , & Automated analysis of postoperative behaviour: assessment of HomeCageScan as a novel method to rapidly identify pain and analgesic effects in mice. Lab. Anim. 43, 17–26 (2008).

  6. 6.

    , , & Increase in activity during calorie restriction requires Sirt1. Science 310, 1641 (2005).

  7. 7.

    , , & The power of automated high-resolution behavior analysis revealed by its application to mouse models of Huntington's and prion diseases. Proc. Natl Acad. Sci. 104, 1983–1988 (2007).

  8. 8.

    et al. A robust automated system elucidates mouse home cage behavioral structure. Proc. Natl Acad. Sci. 105, 20575–20582 (2008).

  9. 9.

    et al. Early behavioural changes in mice infected with BSE and scrapie: automated home cage monitoring reveals prion strain differences. Eur. J. Neurosci. 16, 735–742 (2002).

  10. 10.

    et al. Heat shock factor 1 regulates lifespan as distinct from disease onset in prion disease. Proc. Natl Acad. Sci. 105, 13626–13631 (2008).

  11. 11.

    , , & Nucleocytoplasmic transport signals affect the age at onset of abnormalities in knock-in mice expressing polyglutamine within an ectopic protein context. Hum. Mol. Gen. 12, 1621–1629 (2003).

  12. 12.

    , & EthoVision: a versatile video tracking system for automation of behavioral experiments. Behav. Res. Meth. Ins. C. 33, 398–414 (2001).

  13. 13.

    , , & Detection of early behavioral markers of Huntington's disease in R6/2 mice employing an automated social home cage. Behav. Brain Res. 203, 188–199 (2009).

  14. 14.

    & Histograms of oriented gradients for human detection. Proc. IEEE Comp. Vision and Patt. Recogn. 1, 886–893 (2005).

  15. 15.

    & Robust real-time object detection. Int. J. Comp. Vision 57, 137–154 (2002).

  16. 16.

    , & A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104, 90–126 (2006).

  17. 17.

    , & Shape-and-behavior encoded tracking of bee dances. IEEE Trans. Pattern Anal. Mach. Intell. 30, 463–476 (2008).

  18. 18.

    , , & TrackFly: virtual reality for a behavioral system analysis in free-ying fruit flies. J. Neurosci. Methods 171, 110–117 (2008).

  19. 19.

    , & MCMC-based particle filtering for tracking a variable number of interacting targets. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1805–1819 (2005).

  20. 20.

    , , , & High-throughput ethomics in large groups of Drosophila. Nat. Methods 6, 451–457 (2009).

  21. 21.

    , , , & Automated monitoring and analysis of social behavior in Drosophila. Nat. Methods 6, 297–303 (2009).

  22. 22.

    & Feature fusion for basic behavior unit segmentation from video sequences. Robot. Auton. Syst. 57, 239–248 (2009).

  23. 23.

    , , & Behavior recognition via sparse spatio-temporal features. Proc. IEEE Int. Workshop on VS-PETS 1, 65–72 (2005).

  24. 24.

    & Neural mechanisms for the recognition of biological movements. Nat. Rev. Neurosci. 4, 179–192 (2003).

  25. 25.

    , , & A biologically inspired system for action recognition. Proc. IEEE Int. Conf. on Comp. Vision 1, 1–8 (2007).

  26. 26.

    & Structure and function of visual area MT. Annu. Rev. Neurosci. 28, 157–189 (2005).

  27. 27.

    & A model of neuronal responses in visual area MT. Vision Res. 38, 743–761 (1998).

  28. 28.

    , & Hidden Markov support vector machines. Proc. Int. Conf. on Mach. Learn. 1, 3–10 (2003).

  29. 29.

    , & Cutting-plane training of structural SVMs. Mach. Learn 76, 27–59 (2009).

  30. 30.

    & Neurobehavioral assessment in the information age. Nat. Neurosci. 7, 462–466 (2004).

Download references


This project was sponsored by the McGovern Institute for Brain Research. A.D.S., L.Y. and V.K. were funded by the Broad Fellows Program in Brain Circuitry at Caltech. H.J. was founded by the Taiwan National Science Council (TMS-094-1-A032). We thank Jessi Ambrose, Andrea Farbe, Cynthia Hsu, Alexandra Jiang, Grant Kadokura, Xuying Li, Anjali Patel, Kelsey von Tish, Emma Tolley, Ye Yao and Eli T Williams for their efforts in annotating the videos for this project. We are grateful to Susan Lindquist for allowing us to use her facilities and mice to generate much of the training video database and to Walker Jackson for assistance with operating Home Cage Scan. We thank Piotr Dollar for providing source code. The algorithm and software development and its evaluation were conducted at the Center for Biological and Computational Learning, which is in the McGovern Institute for Brain Research at MIT, as well as in the Department of Brain and Cognitive Sciences, and which is affiliated with the Computer Sciences and Artificial Intelligence Laboratory.

Author information

Author notes

    • Thomas Serre

    Present address: Department of Cognitive, Linguistic and Psychological Sciences, Brown Institute for Brain Sciences, Brown University, Providence, RI 02912, USA.


  1. Department of Brain and Cognitive Sciences, McGovern Institute, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.

    • Hueihan Jhuang
    • , Estibaliz Garrote
    • , Tomaso Poggio
    •  & Thomas Serre
  2. Broad Fellows in Brain Circuitry Program, Division of Biology, California Institute of Technology, Pasadena, California 91125, USA.

    • Xinlin Yu
    • , Vinita Khilnani
    •  & Andrew D. Steele


  1. Search for Hueihan Jhuang in:

  2. Search for Estibaliz Garrote in:

  3. Search for Xinlin Yu in:

  4. Search for Vinita Khilnani in:

  5. Search for Tomaso Poggio in:

  6. Search for Andrew D. Steele in:

  7. Search for Thomas Serre in:


H.J., T.P., A.D.S. and T.S. designed the research; H.J., E.G., A.D.S. and T.S. conducted the research; H.J., E.G., X.Y., V.K., A.D.S. and T.S analysed data; H.J., T.P., A.D.S. and T.S. wrote the paper.

Competing interests

The authors declare no competing financial interests.

Corresponding authors

Correspondence to Andrew D. Steele or Thomas Serre.

Supplementary information


  1. 1.

    Supplementary Movie 1

    Supplementary Movie 1 demonstrating the automatic scoring of 8 types of behaviour at day by the system

  2. 2.

    Supplementary Movie 2

    Supplementary Movie 2 demonstrating the automatic scoring of 8 types of behaviour at night by the system.

  3. 3.

    Supplementary Movie 3

    Supplementary Movie 3 showing the automatic scoring of “wheel-interaction” behaviour by the system.

PDF files

  1. 1.

    Supplementary Information

    Supplementary Figures S1-S3, Supplementary Tables S1-S2, Supplementary Note

Zip files

  1. 1.

    Supplementary Software

    Supplementary Software


By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.