Preclinical studies of psychiatric disorders use animal models to investigate the impact of environmental factors or genetic mutations on complex traits such as decision-making and social interactions. Here, we introduce a method for the real-time analysis of the behaviour of mice housed in groups of up to four over several days and in enriched environments. The method combines computer vision through a depth-sensing infrared camera, machine learning for animal and posture identification, and radio-frequency identification to monitor the quality of mouse tracking. It tracks multiple mice accurately, extracts a list of behavioural traits of both individuals and the groups of mice, and provides a phenotypic profile for each animal. We used the method to study the impact of Shank2 and Shank3 gene mutations—mutations that are associated with autism—on mouse behaviour. Characterization and integration of data from the behavioural profiles of Shank2 and Shank3 mutant female mice revealed their distinctive activity levels and involvement in complex social interactions.
Subscribe to Journal
Get full journal access for 1 year
only $4.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
The authors declare that all the data supporting the findings of this study are available within the paper and Supplementary Information. Full datasets (databases and films) generated during and/or analysed during the current study are available in the LMT website repository, https://livemousetracker.org.
Full source code is available at http://icy.bioimageanalysis.org/plugins/livemousetracker. This includes Java code and also CAD hardware resource files. Python analysis scripts are available at https://github.com/fdechaumont/lmt-analysis.
Tecott, L. H. & Nestler, E. J. Neurobehavioral assessment in the information age. Nat. Neurosci. 7, 462–466 (2004).
Silverman, J. L., Yang, M., Lord, C. & Crawley, J. N. Behavioural phenotyping assays for mouse models of autism. Nat. Rev. Neurosci. 11, 490–502 (2010).
Crawley, J. N. Designing mouse behavioral tasks relevant to autistic-like behaviors. Ment. Retard. Dev. Disabil. Res. Rev. 10, 248–258 (2004).
Olsson, I. A. S. et al. Understanding behaviour: the relevance of ethological approaches in laboratory animal science. Appl. Anim. Behav. Sci. 81, 245–264 (2018).
Belzung, C. & Griebel, G. Measuring normal and pathological anxiety-like behaviour in mice: a review. Behav. Brain Res. 125, 141–149 (2001).
Wiltschko, A. B. et al. Mapping sub-second structure in mouse behavior. Neuron 88, 1121–1135 (2015).
Schmeisser, M. J. et al. Autistic-like behaviours and hyperactivity in mice lacking ProSAP1/Shank2. Nature 486, 256–260 (2012).
Ey, E. et al. The autism ProSAP1/Shank2 mouse model displays quantitative and structural abnormalities in ultrasonic vocalisations. Behav. Brain Res. 256, 677–689 (2013).
Berkel, S. et al. Mutations in the SHANK2 synaptic scaffolding gene in autism spectrum disorder and mental retardation. Nat. Genet. 42, 489–491 (2010).
Leblond, C. S. et al. Genetic and functional analyses of SHANK2 mutations suggest a multiple hit model of autism spectrum disorders. PLoS Genet. 8, e100252 (2012).
Leblond, C. S. et al. Meta-analysis of SHANK mutations in autism spectrum disorders: a gradient of severity in cognitive impairments. PLoS Genet. 10, e1004580 (2014).
Durand, C. M. et al. Mutations in the gene encoding the synaptic scaffolding protein SHANK3 are associated with autism spectrum disorders. Nat. Genet. 39, 25–27 (2007).
Bernardin, K., Elbs, A. & Stiefelhagen, R. Multiple object tracking performance metrics and evaluation in a smart room environment. In Proc. of IEEE International Workshop on Visual Surveillance (2006).
Matthews, S. G., Miller, A. L., Plötz, T. & Kyriazakis, I. Automated tracking to measure behavioural changes in pigs for health and welfare monitoring. Sci. Rep. 7, 17582 (2017).
Peca, J. et al. Shank3 mutant mice display autistic-like behaviours and striatal dysfunction. Nature 472, 437–442 (2011).
Drapeau, E., Riad, M., Kajiwara, Y. & Buxbaum, J. D. Behavioral phenotyping of an improved mouse model of Phelan-McDermid syndrome with a complete deletion of the Shank3 gene. eNeuro 5, e0046-18 (2018).
Ohayon, S., Avni, O., Taylor, A. L., Perona, P. & Roian Egnor, S. E. Automated multi-day tracking of marked mice for the analysis of social behaviour. J. Neurosci. Methods 219, 10–19 (2013).
Ey, E. et al. Shank2 mutant mice display hyperactivity insensitive to methylphenidate and reduced flexibility in social motivation, but normal social recognition. Front. Mol. Neurosci. 11, 365 (2018).
Pappas, A. L. et al. Deficiency of Shank2 causes mania-like behavior that responds to mood stabilizers. JCI Insight 2, 92052 (2017).
Holly, K. S., Orndorff, C. O. & Murray, T. A. MATSAP: an automated analysis of stretch-attend posture in rodent behavioral experiments. Sci. Rep. 6, 31286 (2016).
Torquet, N., de Chaumont, F., Faure, P., Bourgeron, T. & Ey, E. mouseTube—a database to collaboratively unravel mouse ultrasonic communication. F1000 Res. 5, 2332 (2016).
Esler, A. & Grissom, N. Why we need a mouse version of a diagnostic test for autism. Spectrum (21 March 2018); https://spectrumnews.org/opinion/viewpoint/need-mouse-version-diagnostic-test-autism/
Maubourguet, N., Lesne, A., Changeux, J.-P., Maskos, U. & Faure, P. Behavioral sequence analysis reveals a novel role for β2* nicotinic receptors in exploration. PLoS Comput. Biol. 4, e1000229 (2008).
R Core Team R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2017); http://www.R-project.org/
The authors thank Y. Archambeau and P. Ollivon at the workshop of the Institut Pasteur for building the first 12 setups and advising on hardware, W. Meiniel for the mathematical proof for decisions of head/tail probability, Microsoft France for their technical support, P. Spinicelli for optical engineering and reading of the paper, R. Marée for machine learning support, B. König for advice and reading of biological experiments, J. N. Crawley for reading and providing comments on the manuscript, A. Barmpoutis for providing us with the early Kinect 2 driver and support, N. Chenouard for driving the use of the machine learning solution, P. Dugast for drawing the mice in the different behavioural events, A. Engelberg for checking the English, S. Wagner and R. Accardi for RFID advice, M. Marim for website development, and X. Montagutelli and M. Bérard for animal facility support. This work was partially funded by the Institut Pasteur, the Bettencourt-Schueller Foundation, the Cognacq–Jay Foundation, the Conny–Maeva Foundation, the ERANET–NEURON SYNPATHY program, the Agence Nationale de la Recherche through grant number ANR-10-LABX-62-IBEID, France-BioImaging infrastructure through grant number ANR-10-INBS-04 and the INCEPTION program through grant number ANR-16-CONV-0005, the Centre National de la Recherche Scientifique, the University Paris Diderot, the BioPsy Labex, the Institut National du Cancer through grant number TABAC-16–022, the Foundation for Medical Research (Equipe DEQ20130326488), the Innovative Medicines Initiative Joint Undertaking through grant agreement number 115300, resources of which are composed of financial contributions from the European Union’s Seventh Framework Program (FP7/2007–2013) and EFPIA companies in kind contribution. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
The authors declare no competing interests.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary figures, tables, methods, results, video captions and references.
Assembly instructions for the mouse live tracker.
The different steps of the method, and a general overview of the system.
3D animations of the assembly, and how to perform the calibration of the system.
A case study of a typical problem: a nest that is built by the animal at an unexpected location.
Background height map (a view of the cage without the animals).
Live 3D rendering demo.
Data used to validate the method.
How the system copes with different appearances, and also with a mix of animals having different coat colours.
About this article
Cite this article
de Chaumont, F., Ey, E., Torquet, N. et al. Real-time analysis of the behaviour of groups of mice via a depth-sensing camera and machine learning. Nat Biomed Eng 3, 930–942 (2019). https://doi.org/10.1038/s41551-019-0396-1
Animal Cognition (2021)
Neuroscience Bulletin (2021)
Nature Methods (2020)
Deep learning-based behavioral analysis reaches human accuracy and is capable of outperforming commercial solutions