Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Technical Report
  • Published:

Simple Behavioral Analysis (SimBA) as a platform for explainable machine learning in behavioral neuroscience

Abstract

The study of complex behaviors is often challenging when using manual annotation due to the absence of quantifiable behavioral definitions and the subjective nature of behavioral annotation. Integration of supervised machine learning approaches mitigates some of these issues through the inclusion of accessible and explainable model interpretation. To decrease barriers to access, and with an emphasis on accessible model explainability, we developed the open-source Simple Behavioral Analysis (SimBA) platform for behavioral neuroscientists. SimBA introduces several machine learning interpretability tools, including SHapley Additive exPlanation (SHAP) scores, that aid in creating explainable and transparent behavioral classifiers. Here we show how the addition of explainability metrics allows for quantifiable comparisons of aggressive social behavior across research groups and species, reconceptualizing behavior as a sharable reagent and providing an open-source framework. We provide an open-source, graphical user interface (GUI)-driven, well-documented package to facilitate the movement toward improved automation and sharing of behavioral classification tools across laboratories.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: SimBA workflow and outside integrations.
Fig. 2: Classifier construction workflow and classifier performance metrics.
Fig. 3: SHAP attack classifier consortium data.
Fig. 4: SHAP cross-species attack classifier data.
Fig. 5: Social stress experience influences aggression and coping behaviors differently in males and females.
Fig. 6: Environment and experience influence male aggression and coping behaviors.

Similar content being viewed by others

Data availability

The data used in this study are available as a branch of the SimBA GitHub repository: https://github.com/sgoldenlab/simba/tree/simba-data/data.

The CRIM13 database is publicly available at https://paperswithcode.com/dataset/crim13. Source data are provided with this paper.

Code availability

Code and documentation are available at https://github.com/sgoldenlab/simba. SimBA was built with Python version 3.6.

References

  1. Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A. & Poeppel, D. Neuroscience needs behavior: correcting a reductionist bias. Neuron 93, 480–490 (2017).

    Article  CAS  PubMed  Google Scholar 

  2. Anderson, D. J. & Perona, P. Toward a science of computational ethology. Neuron 84, 18–31 (2014).

    Article  CAS  PubMed  Google Scholar 

  3. Egnor, S. E. R. & Branson, K. Computational analysis of behavior. Annu. Rev. Neurosci. 39, 217–236 (2016).

    Article  CAS  PubMed  Google Scholar 

  4. Datta, S. R., Anderson, D. J., Branson, K., Perona, P. & Leifer, A. Computational neuroethology: a call to action. Neuron 104, 11–24 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Falkner, A. L., Grosenick, L., Davidson, T. J., Deisseroth, K. & Lin, D. Hypothalamic control of male aggression-seeking behavior. Nat. Neurosci. 19, 596–604 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Ferenczi, E. A. et al. Prefrontal cortical regulation of brainwide circuit dynamics and reward-related behavior. Science 351, aac9698 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  7. Kim, Y. et al. Mapping social behavior-induced brain activation at cellular resolution in the mouse. Cell Rep. 10, 292–305 (2015).

    Article  CAS  PubMed  Google Scholar 

  8. Gunaydin, L. A. et al. Natural neural projection dynamics underlying social behavior. Cell 157, 1535–1551 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Graving, J. M. et al. DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning. eLife 8, e47994 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Mathis, A. et al. DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21, 1281–1289 (2018).

    Article  CAS  PubMed  Google Scholar 

  11. Pereira, T. D. et al. Fast animal pose estimation using deep neural networks. Nat. Methods 16, 117–125 (2019).

    Article  CAS  PubMed  Google Scholar 

  12. Geuther, B. Q. et al. Robust mouse tracking in complex environments using neural networks. Commun. Biol. 2, 124 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  13. Gris, K. V., Coutu, J.-P. & Gris, D. Supervised and unsupervised learning technology in the study of rodent behavior. Front. Behav. Neurosci. 11, 141 (2017).

  14. Schaefer, A. T. & Claridge-Chang, A. The surveillance state of behavioral automation. Curr. Opin. Neurobiol. 22, 170–176 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Robie, A. A., Seagraves, K. M., Egnor, S. E. R. & Branson, K. Machine vision methods for analyzing social interactions. J. Exp. Biol. 220, 25–34 (2017).

    Article  PubMed  Google Scholar 

  16. Vu, M.-A. T. et al. A shared vision for machine learning in neuroscience. J. Neurosci. 38, 1601–1607 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Goodwin, N. L., Nilsson, S. R. O., Choong, J. J. & Golden, S. A. Toward the explainability, transparency, and universality of machine learning for behavioral classification in neuroscience. Curr. Opin. Neurobiol. 73, 102544 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Newton, K. C. et al. Lateral line ablation by ototoxic compounds results in distinct rheotaxis profiles in larval zebrafish. Commun. Biol. 6, 1–15 (2023).

    Article  Google Scholar 

  19. Jernigan, C. M., Stafstrom, J. A., Zaba, N. C., Vogt, C. C. & Sheehan, M. J. Color is necessary for face discrimination in the Northern paper wasp, Polistes fuscatus. Anim. Cogn. 26, 589–598 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Dahake, A. et al. Floral humidity as a signal – not a cue – in a nocturnal pollination system. Preprint at bioRxiv https://doi.org/10.1101/2022.04.27.489805 (2022).

  21. Dawson, M. et al. Hypocretin/orexin neurons encode social discrimination and exhibit a sex-dependent necessity for social interaction. Cell Rep. 42, 112815 (2023).

    Article  CAS  PubMed  Google Scholar 

  22. Baleisyte, A., Schneggenburger, R. & Kochubey, O. Stimulation of medial amygdala GABA neurons with kinetically different channelrhodopsins yields opposite behavioral outcomes. Cell Rep. 39, 110850 (2022).

    Article  CAS  PubMed  Google Scholar 

  23. Cruz-Pereira, J. S. et al. Prebiotic supplementation modulates selective effects of stress on behavior and brain metabolome in aged mice. Neurobiol. Stress 21, 100501 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Linders, L. E. et al. Stress-driven potentiation of lateral hypothalamic synapses onto ventral tegmental area dopamine neurons causes increased consumption of palatable food. Nat. Commun. 13, 6898 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Slivicki, R. A. et al. Oral oxycodone self-administration leads to features of opioid misuse in male and female mice. Addiction Biol. 28, e13253 (2023).

    Article  CAS  Google Scholar 

  26. Miczek, K. A. et al. Excessive alcohol consumption after exposure to two types of chronic social stress: intermittent episodes vs. continuous exposure in C57BL/6J mice with a history of drinking. Psychopharmacology (Berl.) 239, 3287–3296 (2022).

    Article  CAS  PubMed  Google Scholar 

  27. Cui, Q. et al. Striatal direct pathway targets Npas1+ pallidal neurons. J. Neurosci. 41, 3966–3987 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Chen, J. et al. A MYT1L syndrome mouse model recapitulates patient phenotypes and reveals altered brain development due to disrupted neuronal maturation. Neuron 109, 3775–3792 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Rigney, N., Zbib, A., de Vries, G. J. & Petrulis, A. Knockdown of sexually differentiated vasopressin expression in the bed nucleus of the stria terminalis reduces social and sexual behaviour in male, but not female, mice. J. Neuroendocrinol. 34, e13083 (2021).

  30. Winters, C. et al. Automated procedure to assess pup retrieval in laboratory mice. Sci. Rep. 12, 1663 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Neira, S. et al. Chronic alcohol consumption alters home-cage behaviors and responses to ethologically relevant predator tasks in mice. Alcohol Clin. Exp. Res. 46, 1616–1629 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  32. Kwiatkowski, C. C. et al. Quantitative standardization of resident mouse behavior for studies of aggression and social defeat. Neuropsychopharmacology 46, 1584–1593 (2021).

  33. Yamaguchi, T. et al. Posterior amygdala regulates sexual and aggressive behaviors in male mice. Nat. Neurosci. 23, 1111–1124 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Nygaard, K. R. et al. Extensive characterization of a Williams syndrome murine model shows Gtf2ird1-mediated rescue of select sensorimotor tasks, but no effect on enhanced social behavior. Genes Brain Behav. 22, e12853 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Ojanen, S. et al. Interneuronal GluK1 kainate receptors control maturation of GABAergic transmission and network synchrony in the hippocampus. Mol. Brain 16, 43 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Hon, O. J. et al. Serotonin modulates an inhibitory input to the central amygdala from the ventral periaqueductal gray. Neuropsychopharmacology 47, 2194–2204 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Murphy, C. A. et al. Modeling features of addiction with an oral oxycodone self-administration paradigm. Preprint at bioRxiv https://doi.org/10.1101/2021.02.08.430180 (2021).

  38. Neira, S. et al. Impact and role of hypothalamic corticotropin releasing hormone neurons in withdrawal from chronic alcohol consumption in female and male mice. J. Neurosci. 43, 7657–7667 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Lapp, H. E., Salazar, M. G. & Champagne, F. A. Automated maternal behavior during early life in rodents (AMBER) pipeline. Sci. Rep. 13, 18277 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Barnard, I. L. et al. High-THC cannabis smoke impairs incidental memory capacity in spontaneous tests of novelty preference for objects and odors in male rats. eNeuro 10, ENEURO.0115-23.2023 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  41. Ausra, J. et al. Wireless battery free fully implantable multimodal recording and neuromodulation tools for songbirds. Nat. Commun. 12, 1968 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Friard, O. & Gamba, M. BORIS: a free, versatile open-source event-logging software for video/audio coding and live observations. Methods Ecol. Evol. 7, 1325–1330 (2016).

    Article  Google Scholar 

  43. Spink, A. J., Tegelenbosch, R. A. J., Buma, M. O. S. & Noldus, L. P. J. J. The EthoVision video tracking system—a tool for behavioral phenotyping of transgenic mice. Physiol. Behav. 73, 731–744 (2001).

    Article  CAS  PubMed  Google Scholar 

  44. Lundberg, S. shap. https://github.com/shap/shap

  45. Lauer, J. et al. Multi-animal pose estimation, identification and tracking with DeepLabCut. Nat. Methods 19, 496–504 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Pereira, T. D. et al. SLEAP: a deep learning system for multi-animal pose tracking. Nat Methods 19, 486–495 (2022).

  47. Segalin, C. et al. The Mouse Action Recognition System (MARS) software pipeline for automated analysis of social behaviors in mice. eLife 10, e63720 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001).

    Article  Google Scholar 

  49. Liaw, A. & Wiener, M. Classification and regression by randomForest. R News 2/3 https://journal.r-project.org/articles/RN-2002-022/RN-2002-022.pdf (2022).

  50. Goodwin, N. L., Nilsson, S. R. O. & Golden, S. A. Rage against the machine: advancing the study of aggression ethology via machine learning. Psychopharmacology 237, 2569–2588 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Lundberg, S. M. et al. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2, 56–67 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  52. Ribeiro, M. T., Singh, S., & Guestrin, C. ‘Why should I trust you?’: explaining the predictions of any classifier. Preprint at arXiv https://doi.org/10.48550/arXiv.1602.04938 (2016).

  53. Sundararajan, M., Taly, A. & Yan, Q. Axiomatic attribution for deep networks. In Proc. of the 34th International Conference on Machine Learning 3319–3328 (MLR Press, 2017).

  54. Hatwell, J., Gaber, M. M. & Azad, R. M. A. CHIRPS: explaining random forest classification. Artif. Intell. Rev. 53, 5747–5788 (2020).

    Article  Google Scholar 

  55. Lundberg, S. & Lee, S.-I. A unified approach to interpreting model predictions. Preprint at arXiv https://doi.org/10.48550/arXiv.1705.07874 (2017).

  56. Verma, S., Dickerson, J. & Hines, K. Counterfactual explanations for machine learning: a review. Preprint at arXiv https://doi.org/10.48550/arXiv.2010.10596 (2020).

  57. Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206–215 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  58. Takahashi, A. et al. Establishment of a repeated social defeat stress model in female mice. Sci. Rep. 7, 12838 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  59. Hashikawa, K. et al. Esr1+ cells in the ventromedial hypothalamus control female aggression. Nat. Neurosci. 20, 1580–1590 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Newman, E. L. et al. Fighting females: neural and behavioral consequences of social defeat stress in female mice. Biol. Psychiatry 86, 657–668 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  61. Aubry, A. V. et al. Sex differences in appetitive and reactive aggression. Neuropsychopharmacology 47, 1746–1754 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  62. Golden, S. A., Covington, H. E., Berton, O. & Russo, S. J. A standardized protocol for repeated social defeat stress in mice. Nat. Protoc. 6, 1183–1191 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  63. Shemesh, Y. & Chen, A. A paradigm shift in translational psychiatry through rodent neuroethology. Mol. Psychiatry 28, 993–1003 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  64. Kabra, M., Robie, A. A., Rivera-Alba, M., Branson, S. & Branson, K. JAABA: interactive machine learning for automatic annotation of animal behavior. Nat. Methods 10, 64–67 (2013).

    Article  CAS  PubMed  Google Scholar 

  65. Bordes, J. et al. Automatically annotated motion tracking identifies a distinct social behavioral profile following chronic social defeat stress. Nat. Commun. 14, 4319 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  66. Winters, C., Gorssen, W., Wöhr, M. & D’Hooge, R. BAMBI: a new method for automated assessment of bidirectional early-life interaction between maternal behavior and pup vocalization in mouse dam-pup dyads. Front. Behav. Neurosci. 17, 1139254 (2023).

  67. Lundberg, S. M., Erion, G. G. & Lee, S.-I. Consistent individualized feature attribution for tree ensembles. Preprint at arXiv https://doi.org/10.48550/arXiv.1802.03888 (2019).

  68. Covert, I. C., Lundberg, S. & Lee, S.-I. Explaining by removing: a unified framework for model explanation. J. Mach. Learn. Res. 22, 1–90 (2021).

  69. Lorbach, M., Poppe, R. & Veltkamp, R. C. Interactive rodent behavior annotation in video using active learning. Multimed. Tools Appl. 78, 19787–19806 (2019).

    Article  Google Scholar 

  70. Tillmann, J. F., Hsu, A. I., Schwarz, M. K. & Yttri, E. A. A-SOiD, an active-learning platform for expert-guided, data-efficient discovery of behavior. Nat. Methods 21, 703–711 (2024).

    Article  CAS  PubMed  Google Scholar 

  71. Whiteway, M. R. et al. Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders. PLoS Comput. Biol. 17, e1009439 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Sun, J. J. et al. Task Programming: Learning Data Efficient Behavior Representations. In Proc IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2875–2884 (2021).

  73. MABe 2022. Multi-agent behavior: representation, modeling, measurement, and applications. https://sites.google.com/view/mabe22/home

  74. Sun, J. J. et al. The multi-agent behavior dataset: mouse dyadic social interactions. Preprint at arXiv https://doi.org/10.48550/arXiv.2104.02710 (2021).

  75. OpenBehavior. About the OpenBehavior Project and the open source movement. https://edspace.american.edu/openbehavior/

  76. Mouse Phenome Database. https://phenome.jax.org/about

  77. Kapoor, S. & Narayanan, A. Leakage and the reproducibility crisis in machine-learning-based science. Patterns (N Y) 4, 100804 (2023).

    Article  PubMed  Google Scholar 

  78. Dankert, H., Wang, L., Hoopfer, E. D., Anderson, D. J. & Perona, P. Automated monitoring and analysis of social behavior in Drosophila. Nat. Methods 6, 297–303 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  79. de Chaumont, F. et al. Real-time analysis of the behaviour of groups of mice via a depth-sensing camera and machine learning. Nat. Biomed. Eng. 3, 930–942 (2019).

    Article  PubMed  Google Scholar 

  80. Giancardo, L. et al. Automatic visual tracking and social behaviour analysis with multiple mice. PLoS ONE 8, e74557 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  81. Hong, W. et al. Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning. Proc. Natl Acad. Sci. USA 112, E5351–E5360 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  82. Goodwin, N. L. et al. Simple behavioral analysis (SimBA) as a platform for explainable machine learning in behavioral neuroscience. Nat. Neurosci. (in the press).

  83. Bohnslav, J. P. et al. DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels. eLife 10, e63377 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  84. Gerós, A., Magalhães, A. & Aguiar, P. Improved 3D tracking and automated classification of rodents’ behavioral activity using depth-sensing cameras. Behav. Res. 52, 2156–2167 (2020).

    Article  Google Scholar 

  85. Harris, C., Finn, K. R., Kieseler, M.-L., Maechler, M. R. & Tse, P. U. DeepAction: a MATLAB toolbox for automated classification of animal behavior in video. Sci. Rep. 13, 2688 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  86. Hu, Y. et al. LabGym: quantification of user-defined animal behaviors using learning-based holistic assessment. Cell Rep. Methods 3, 100415 (2023).

  87. Marks, M. et al. Deep-learning based identification, tracking, pose estimation, and behavior classification of interacting primates and mice in complex environments. Nat. Mach. Intell. 4, 331–340 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  88. Branson, K., Robie, A. A., Bender, J., Perona, P. & Dickinson, M. H. High-throughput ethomics in large groups of Drosophila. Nat. Methods 6, 451–457 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  89. Jia, Y. et al. Selfee, self-supervised features extraction of animal behaviors. eLife 11, e76218 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  90. Berman, G. J., Choi, D. M., Bialek, W. & Shaevitz, J. W. Mapping the stereotyped behaviour of freely moving fruit flies. J. R. Soc. Interface 11, 20140672 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  91. Arakawa, T. et al. Automated estimation of mouse social behaviors based on a hidden Markov model. In Hidden Markov Models: Methods and Protocols (eds Westhead, D. R. & Vijayabaskar, M. S.) 185–197 (Humana Press, 2017).

  92. Chen, Z. et al. AlphaTracker: a multi-animal tracking and behavioral analysis tool. Front. Behav. Neurosci. 17, 1111908 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  93. Huang, K. et al. A hierarchical 3D-motion learning framework for animal spontaneous behavior mapping. Nat. Commun. 12, 2784 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  94. Luxem, K. et al. Identifying behavioral structure from deep variational embeddings of animal motion. Commun. Biol. 5, 1267 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  95. Nandi, A., Virmani, G., Barve, A. & Marathe, S. DBscorer: an open-source software for automated accurate analysis of rodent behavior in forced swim test and tail suspension test. eNeuro 8, ENEURO.0305-21.2021 (2021).

  96. Gabriel, C. J. et al. BehaviorDEPOT is a simple, flexible tool for automated behavioral detection based on markerless pose tracking. eLife 11, e74314 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  97. Golden, S. A. et al. Epigenetic regulation of RAC1 induces synaptic remodeling in stress disorders and depression. Nat. Med. 19, 337–344 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  98. Burgos-Artizzu, X. P., Dollar, P., Lin D., Anderson, D. J. & Perona, P. CRIM13 (Caltech Resident-Intruder Mouse 13) (1.0). CaltechDATA. https://doi.org/10.22002/D1.1892 (2021).

  99. Karashchuk, P., Tuthill, J. C. & Brunton, B. W. The DANNCE of the rats: a new toolkit for 3D tracking of animal behavior. Nat. Methods 18, 460–462 (2021).

    Article  CAS  PubMed  Google Scholar 

  100. Branson, K. APT. https://github.com/kristinbranson/APT

  101. Lee, W., Fu, J., Bouwman, N., Farago, P. & Curley, J. P. Temporal microstructure of dyadic social behavior during relationship formation in mice. PLoS ONE 14, e0220596 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This research was supported by National Institute on Drug Abuse (NIDA) R00DA045662 (S.A.G.), R01DA059374 (S.A.G.) and P30DA048736 (M.H. and S.A.G.); NARSAD Young Investigator Award 27082 (S.A.G.); National Institute of Mental Health 1F31MH125587 (N.L.G.), F31AA025827 (E.L.N.) and F32MH125634 (E.L.N.); National Institute of General Medical Sciences R35GM146751 (M.H.); National Institutes of Health K08MH123791 (N.E.); the Burroughs Wellcome Fund Career Award for Medical Scientists (N.E.); the Simons Foundation Bridge to Independence Award (N.E.); and the Washington Research Foundation Postdoctoral Fellowship (E.R.S.). We thank V. Tsai, R. Vrooman and C. Xu for their skillful technical contributions. We thank B. Bentzley and D. Lin for contributing to aggression consortium data. S.R.O.N. has continued development and maintenance of SimBA independent of other funding sources. Figures were created with BioRender.

Author information

Authors and Affiliations

Authors

Contributions

Conceptualization: S.A.G., S.R.O.N. and N.L.G. Data curation: K.P., L.B., A.I., Y.Y.Z., E.S., X.T., E.L.N., K.M., H.R.W., R.J.M., Z.C.N., N.E., M.H., S.R.O.N., N.L.G. and S.A.G. Methodology: S.R.O.N. and N.L.G. Formal analysis: S.R.O.N., N.L.G. and S.A.G. Visualization: N.L.G. and S.R.O.N. Writing—original draft: S.A.G., S.R.O.N. and N.L.G. Funding acquisition: S.A.G., S.R.O.N., N.L.G., M.H., E.L.N., N.E. and E.R.S. Writing—reviewing and editing: S.A.G., S.R.O.N., N.L.G., K.P., L.B., A.I., Y.Y.Z., E.S., X.T., E.L.N., K.M., H.R.W., R.J.M., Z.C.N., N.E. and M.H. Software: S.R.O.N., J.J.C. and S.H. Supervision: S.A.G. Project administration: S.R.O.N. and N.L.G.

Corresponding authors

Correspondence to Simon R. O. Nilsson or Sam A. Golden.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Neuroscience thanks Joshua Shaevitz and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Example of DeepLabCut pose-estimation model for mouse resident-intruder behavior.

These data were calculated from 12,686 images from n = 101 mice. (a) The 16 body-parts labeled. (b) Schematic depiction of the location of each of the 16 body part labels. (c) Evaluations of three models (rgb, clahe, greyscale) using the DeepLabCut evaluation tool. Pixel distances were converted to millimeter by using the lowest resolution images in the dataset (1000x1544px; 4.6px/millimeter). (d) Median millimeter error per body part. (e) Image representing the relative standard error (RSE) of the median millimeter error across all test images. The labelled images and DeepLabCut generated weights are available to download on the Open Science Framework, osf.io/mutws. (f) SimBA supports a range of alternative body-part settings for single animals and dyadic protocols through the File-> Create Project menu. Note: tail end tracking performance was insufficient for a tail rattle classifier, and the tail end body parts were dropped for all analysis in the main figures. Data are presented as mean +/− SEM.

Extended Data Fig. 2 SimBA outlier correction options.

(a) SimBA calculates the mean or median distance between two user-defined body-parts across the frames of each video. We set the user-defined body-parts to be the nose and the tail-base of each animal. The user also defines a movement criterion value, and a location criterion value. We set the movement criterion to 0.7, and location criterion to 1.5. Two different outlier criteria are then calculated by SimBA. These criteria are the mean length between the two user-defined body parts in all frames of the video, multiplied by the either user-defined movement criterion value or location criterion value. SimBA corrects movement outliers prior to correcting location outliers. (b) Schematic representations of a pose-estimation body-part ‘movement outlier’ (top) and a ‘location outlier’ (bottom). A body-part violates the movement criterion when the movement of the body-part across sequential frames is greater than the movement outlier criterion. A body-part violates the location criteria when its distance to more than one other body-part in the animals’ hull (except the tail-end) is greater than the location outlier criterion. Any body part that violates either the movement or location criterion is corrected by placing the body-part at its last reliable coordinate. (c) The ratio of body-part movements (top) and body-part locations (bottom) detected as outliers and corrected by SimBA in the RGB-format mouse resident-intruder data-set. For the outlier corrected in rat and the CRIM13 datasets, see the SimBA GitHub repository. We also offer (d) interpolation options for frames with missing body parts and (e) smoothing options to reduce frame-to-frame jitter.

Source data

Extended Data Fig. 3 Training set information for original mouse, rat, and CRIM13 classifiers.

Training set information for mouse, rat, and CRIM13 mouse resident intruder behavioral classifiers.

Extended Data Fig. 4 Feature binning for SHAP calculations.

Classifiers for the same behavior using different pose estimation schemes will have different feature lists, but can be directly compared via feature binning through the SHAP additivity axiom.

Extended Data Fig. 5 UW versus Stanford scoring and SHAP scores.

UW and Stanford manual scoring of the same dataset for attack behavior. (a) Manual annotations (n = 9 videos) were highly correlated (R2 = 0.998). (b) Gantt plot of UW versus Stanford scores for a high-attack video. (c) SHAP scores for UW positive or Stanford positive attack frames. UW scores rely more on longer rolling windows of behavior than Stanford does.

Extended Data Fig. 6 SHAP values for rat attack classifier.

SHAP values across feature bins and rolling windows for rat attack classifier.

Extended Data Fig. 7 SHAP values for positive frames of UW mouse classifiers used in Figs. 5, 6.

SHAP values for attack, pursuit, anogenital sniffing, defensive, and escape behavioral classifiers used in Figs. 5, 6.

Extended Data Fig. 8 Attack SHAP values across groups and throughout testing sessions.

We calculated SHAP values for 1250 attack frames and 1250 non-attack frames within each experimental protocol. (a) We used these values to calculate delta shap values, where we evaluated the female CSDS and male RI SHAP values against male CSDS SHAP value baseline. The SHAP analyses revealed large similarities in how feature values affected attack classification probabilities in the three experiments (all feature sub-category delta shap < 0.044). The most notable experiment difference was the importance of animal distance features within the current frame, which was associated with higher attack classification probabilities in the RI experiment than in the male CSDS experiment. Attack classification probabilities in the RI experiments were also less affected by features of the resident shape than in the males CSDS experiment. These differences may relate to the different attack strategies and experimental setup used in the experimental protocols. (b) Next, we analyzed SHAP vales for classifying attack and non-attack events in the male and female CSDS experiments within 1 min bins and showed that SHAP values are not affected by time of session.

Supplementary information

Supplementary Information

Supplementary Figs. 1–6, Supplementary Notes 1 and 2 and Supplementary Table 1: SimBA feature list for 16 body parts and two animal tracking.

Reporting Summary

Supplementary Table 2: SHAP feature bins for 14 body parts and two animal tracking.

Source data

Source Data Extended Data Fig. 2

Mean and s.e.m. data for subfigure d.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Goodwin, N.L., Choong, J.J., Hwang, S. et al. Simple Behavioral Analysis (SimBA) as a platform for explainable machine learning in behavioral neuroscience. Nat Neurosci (2024). https://doi.org/10.1038/s41593-024-01649-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41593-024-01649-9

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing