Until now, most brain studies have focused on small numbers of neurons that interact in limited circuits, allowing analysis of individual computations or steps of neural processing. During behaviour, however, brain activity must integrate multiple circuits in different brain regions. Whole-brain recording with cellular resolution provides a new opportunity to dissect the neural basis of behaviour, but whole-brain activity is mutually contingent on behaviour itself, especially for natural behaviours such as navigation, mating or hunting, which require dynamic interaction between the animal, its environment and other animals. Many of the signalling and feedback pathways that animals use to guide behaviour only occur in freely moving animals. Recent technological advances have enabled whole-brain recording in small behaving animals including the nematode Caenorhabditis elegans, the fruit fly Drosophila melanogaster and the larval zebrafish Danio rerio. These whole-brain experiments capture neural activity with cellular resolution spanning sensory, decision-making and motor circuits, and thereby demand new theoretical approaches that integrate brain dynamics with behavioural dynamics. We review the experimental and theoretical methods used to understand animal behaviour and whole-brain activity, and the opportunities for physics to contribute to this emerging field of systems neuroscience.
Advances in optical microscopy allow brain-wide imaging with cellular resolution throughout the sensory, decision-making and motor circuits of behaving animals.
A complete understanding brain-wide dynamics requires requires the context provided by behavioural dynamics: ongoing behaviour emerges from brain activity, and brain activity itself is contingent on past behaviours and experiences.
Brain activity is organized by structural, functional and physiological mechanisms. The wiring diagram of the brain (the connectome) represents pathways of synaptic information flow. The molecular properties of synapses and cells determine the neuronal responses to sensory and synaptic inputs. Non-synaptic mechanisms organize brain-wide activities corresponding to different behavioural states.
Small animals like nematodes, insects and larval fish are tractable models for comprehensively exploring and modelling the mechanisms of brain-wide activity and behaviour.
Modelling brain-wide activity is a multiscale problem from synapses to cells to circuits, across brain areas and across behaviours.
Both top-to-bottom modelling — posing a theory of neural computation and modelling biological mechanisms that might carry it out — and bottom-to-top modelling — looking for structure in high-dimensional activity patterns that might explain correlated behavioural patterns — are important strategies for building towards an understanding of brain-wide dynamics and animal behaviour.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
Single-Shot Light-Field Microscopy: An Emerging Tool for 3D Biomedical Imaging
BioChip Journal Open Access 27 July 2022
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 per month
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$99.00 per year
only $8.25 per issue
Rent or buy this article
Get just this article for as long as you need it
Prices may be subject to local taxes which are calculated during checkout
Dupre, C. & Yuste, R. Non-overlapping neural networks in Hydra vulgaris. Curr. Biol. 27, 1085–1097 (2017).
Chen, J. L., Andermann, M. L., Keck, T., Xu, N. L. & Ziv, Y. Imaging neuronal populations in behaving rodents: paradigms for studying neural circuits underlying behavior in the mammalian cortex. J. Neurosci. 33, 17631–17640 (2013).
Hong, G. & Lieber, C. M. Novel electrode technologies for neural recordings. Nat. Rev. Neurosci. 20, 330–345 (2019).
Allen, W. E. Thirst regulates motivated behavior through modulation of brainwide neural population dynamics. Science 364, 253 (2019).
Allen, W. E. et al. Global representations of goal-directed behavior in distinct cell types of mouse neocortex. Neuron 94, 891–907 (2017).
Stringer, C. et al. Spontaneous behaviors drive multidimensional, brainwide activity. Science 364, 255 (2019).
Steinmetz, N. A. et al. Neuropixels 2.0: a miniaturized high-density probe for stable, long-term brain recordings. Science 372, eabf4588 (2021).
Steinmetz, N. A., Koch, C., Harris, K. D. & Carandini, M. Challenges and opportunities for large-scale electrophysiology with neuropixels probes. Curr. Opin. Neurobiol. 50, 92–100 (2018).
DeYoe, E. A., Bandettini, P., Neitz, J., Miller, D. & Winans, P. Functional magnetic resonance imaging (FMRI) of the human brain. J. Neurosci. Methods 54, 171–187 (1994).
Lee, J. H. et al. Global and local FMRI signals driven by neurons defined optogenetically by type and wiring. Nature 465, 788–792 (2010).
Calhoun, A. J. & Murthy, M. Quantifying behavior to solve sensorimotor transformations: advances from worms and flies. Curr. Opin. Neurobiol. 46, 90–98 (2017).
Kandel, E. R., Schwartz, J. H., Jessell, T. M., Siegelbaum, S. A. & Hudspeth, A. J. Principles of Neural Science 5th edn (McGraw-Hill, 2013).
Chen, Z., Truong, T. M. & Ai, H.-w. Illuminating brain activities with fluorescent protein-based biosensors. Chemosensors 5, 32 (2017).
Deo, C. & Lavis, L. D. Synthetic and genetically encoded fluorescent neural activity indicators. Curr. Opin. Neurobiol. 50, 101–108 (2018).
Pal, A. & Tian, L. Imaging voltage and brain chemistry with genetically encoded sensors and modulators. Curr. Opin. Chem. Biol. 57, 166–176 (2020).
Laviv, T. & Yasuda, R. Imaging neuronal protein signaling dynamics in vivo. Curr. Opin. Neurobiol. 69, 68–75 (2021).
Stephens, D. & Allan, V. Light microscopy techniques for live cell imaging. Science 300, 82–86 (2003).
Huisken, J. & Stainier, D. Y. R. Selective plane illumination microscopy techniques in developmental biology. Development 136, 1963–1975 (2009).
Svoboda, K. & Yasuda, R. Principles of two-photon excitation microscopy and its applications to neuroscience. Neuron 50, 823–839 (2006).
Bouchard, M. B. et al. Swept confocally-aligned planar excitation (SCAPE) microscopy for high-speed volumetric imaging of behaving organisms. Nat. Photonics 9, 113–119 (2015).
Voleti, V. et al. Real-time volumetric microscopy of in vivo dynamics and large-scale samples with SCAPE 2.0. Nat. Methods 16, 1054–1062 (2019).
Karagyozov, D., Mihovilovic Skanata, M., Lesar, A. & Gershow, M. Recording neural activity in unrestrained animals with three-dimensional tracking two-photon microscopy. Cell Reports 25, 1371–1383 (2018).
Griffiths, V. A. et al. Real-time 3D movement correction for two-photon imaging in behaving animals. Nat. Methods 17, 741–748 (2020).
Power, R. M. & Huisken, J. A guide to light-sheet fluorescence microscopy for multiscale imaging. Nat. Methods 14, 360–373 (2017).
Prabhat, P., Ram, S., Ward, E. S. & Ober, R. J. Simultaneous imaging of different focal planes in fluorescence microscopy for the study of cellular dynamics in three dimensions. IEEE Trans. Nanobiosci. 3, 237–242 (2004).
Abrahamsson, S. et al. Multifocus microscopy with precise color multi-phase diffractive optics applied in functional neuronal imaging.Biomed. Opt. Express 7, 855–869 (2016).
Bimber, O. & Schedl, D. C. Light-field microscopy: a review. J. Neurol. Neuromed. 4, 1–6 (2019).
Aimon, S. et al. Fast near-whole-brain imaging in adult Drosophila during responses to stimuli and behavior. PLoS Biol. 17, e2006732 (2019).
Cong, L. et al. Rapid whole brain imaging of neural activity in freely behaving larval zebrafish (Danio rerio). eLife 6, e28158 (2017).
Zhang, Z. et al. Imaging volumetric dynamics at high speed in mouse and zebrafish brain with confocal light field microscopy. Nat. Biotechnol. 39, 74–83 (2021).
Wang, Z. et al. Real-time volumetric reconstruction of biological dynamics with light-field microscopy and deep learning. Nat. Methods 18, 551–556 (2021).
Aragon, M. J. et al. Multiphoton imaging of neural structure and activity in Drosophila through the intact cuticle. eLife 11, e69094 (2022).
Mahou, P., Vermot, J., Beaurepaire, E. & Supatto, W. Multicolor two-photon light-sheet microscopy. Nat. Methods 11, 600–601 (2014).
Venkatachalam, V. et al. Pan-neuronal imaging in roaming Caenorhabditis elegans. Proc. Natl Acad. Sci. USA 113, 1082–8 (2016).
Nguyen, J. P., Linder, A. N., Plummer, G. S., Shaevitz, J. W. & Leifer, A. M. Automatically tracking neurons in a moving and deforming brain.PLoS Comput. Biol. 13, e1005517 (2017).
Schaffer, E. S. et al. Flygenvectors: the spatial and temporal structure of neural activity across the fly brain. Preprint at bioRxiv https://doi.org/10.1101/2021.09.25.461804 (2021).
Brown, A. E. & De Bivort, B. Ethology as a physical science. Nat. Phys. 14, 653–657 (2018).
Kim, D. H. et al. Pan-neuronal calcium imaging with cellular resolution in freely swimming zebrafish. Nat. Methods 14, 1107–1114 (2017).
Wu, Y. et al. Rapid detection and recognition of whole brain activity in a freely behaving Caenorhabditis elegans. Preprint at https://arxiv.org/abs/2109.10474 (2021).
Yu, X. et al. Fast deep neural correspondence for tracking and identifying neurons in C. elegans using semi-synthetic training. eLife 10, e66410 (2021).
Randlett, O. et al. Whole-brain activity mapping onto a zebrafish brain atlas. Nat. Methods 12, 1039–1046 (2015).
Pereira, T. D. et al. Fast animal pose estimation using deep neural networks. Nat. Methods 16, 117–125 (2019).
Pereira, T. D. et al. SLEAP: multi-animal pose tracking. Preprint at bioRxiv https://doi.org/10.1101/2020.08.31.276246 (2020).
Graving, J. M. et al. DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning. eLife 8, e47994 (2019).
Berman, G. J., Choi, D. M., Bialek, W. & Shaevitz, J. W. Mapping the stereotyped behaviour of freely moving fruit flies. J. R. Soc. Interface 11, 20140672 (2014).
Datta, S. R., Anderson, D. J., Branson, K., Perona, P. & Leifer, A. Computational neuroethology: a call to action. Neuron 104, 11–24 (2019).
White, J., Southgate, E., Thomson, J. & Brenner, S. The structure of the nervous system of the nematode Caenorhabditis elegans. Phil. Trans. R Soc. Lond. B 314, 1–340 (1986).
Witvliet, D., Mulcahy, B., Mitchell, J.K. et al. Connectomes across development reveal principles of brain maturation. Nature 596, 257–261 (2021).
Barr, M. M. & Garcia, L. R. Male mating behavior. WormBook https://doi.org/10.1895/wormbook.1.78.1 (2006).
Schrödel, T., Prevedel, R., Aumayr, K., Zimmer, M. & Vaziri, A. Brain-wide 3D imaging of neuronal activity in Caenorhabditis elegans with sculpted light. Nat. Methods 10, 1013–1020 (2013).
Kato, S. et al. Global brain dynamics embed the motor command sequence of Caenorhabditis elegans. Cell 163, 656–669 (2015).
Zhen, M. & Samuel, A. D. C. elegans locomotion: small circuits, complex functions. Curr. Opin. Neurobiol. 33, 117–126 (2015).
Kaplan, H. S., Salazar Thula, O., Khoss, N. & Zimmer, M. Nested neuronal dynamics orchestrate a behavioral hierarchy across timescales. Neuron 105, 562–576 (2020).
Nguyen, J. P. et al. Whole-brain calcium imaging with cellular resolution in freely behaving Caenorhabditis elegans. Proc. Natl Acad. Sci. USA 113, 1074–81 (2016).
Hallinen, K. M. et al. Decoding locomotion from population neural activity in moving C. elegans. eLife 10, e66135 (2021).
Yemini, E. et al. NeuroPAL: a multicolor atlas for whole-brain neuronal identification in C. elegans. Cell 184, 272–288 (2021).
Sulston, J. E., Albertson, D. G. & Thomson, J. N. The Caenorhabditis elegans male: postembryonic development of nongonadal structures. Dev. Biol. 78, 542–576 (1980).
Susoy, V. et al. Natural sensory context drives diverse brain-wide activity during C. elegans mating. Cell 184, 5122–5137 (2021).
Nejatbakhsh, A. et al. Demixing calcium imaging data in C. elegans via deformable non-negative matrix factorization. Med. Image Comput. Comput. Assist. Interv. 12265, 14–24 (2020).
Chaudhary, S., Lee, S. A., Li, Y., Patel, D. S. & Lu, H. Automated annotation of cell identities in dense cellular images. eLife 10, e60321 (2021).
Yu, X. et al. Fast deep learning correspondence for neuron tracking and identification in C. elegans using synthetic training. eLife 10, e66410 (2021).
Hendricks, M., Ha, H., Maffey, N. & Zhang, Y. Compartmentalized calcium dynamics in a C. elegans interneuron encode head movement. Nature 487, 99–103 (2012).
Ouellette, M. H., Desrochers, M. J., Gheta, I., Ramos, R. & Hendricks, M. A gate-and-switch model for head orientation behaviors in C. elegans. eNeuro 5, ENEURO.0121-18.2018 (2018).
Cuentas-Condori, A. et al. C. elegans neurons have functional dendritic spines. eLife 8, e47918 (2019).
Mullins, O. J., Hackett, J. T., Buchanan, J. T. & Friesen, W. O. Neuronal control of swimming behavior: comparison of vertebrate and invertebrate model systems. Prog. Neurobiol. 93, 244–269 (2011).
Kiehn, O. Development and functional organization of spinal locomotor circuits. Curr. Opin. Neurobiol. 21, 100–109 (2011).
Rybak, I. A., Dougherty, K. J. & Shevtsova, N. A. Organization of the mammalian locomotor CPG: review of computational model and circuit architectures based on genetically identified spinal interneurons. eNeuro 2, ENEURO.0069-15.2015 (2015).
Falgairolle, M., Puhl, J. G., Pujala, A., Liu, W. & O’Donovan, M. J. Motoneurons regulate the central pattern generator during drug-induced locomotor-like activity in the neonatal mouse. eLife 6, e26622 (2017).
Cohen, N. & Sanders, T. Nematode locomotion: nissecting the neuronal-environmental loop. Curr. Opin. Neurobiol. 25, 99–106 (2014).
Scheffer, L. K. et al. A connectome and analysis of the adult Drosophila central brain. eLife 9, e57443 (2020).
Seelig, J. D. et al. Two-photon calcium imaging from head-fixed Drosophila during optomotor walking behavior. Nat. Methods 7, 535–540 (2010).
Ito, K. et al. A systematic nomenclature for the insect brain. Neuron 81, 755–765 (2014).
Mann, K., Gallen, C. L. & Clandinin, T. R. Whole-brain calcium imaging reveals an intrinsic functional network in Drosophila. Curr. Biol. 27, 2389–2396 (2017).
Mann, K., Deny, S., Ganguli, S. et al. Coupling of activity, metabolism and behaviour across the Drosophila brain. Nature 593, 244–248 (2021).
Isaacman-Beck, J. et al. SPARC enables genetic manipulation of precise proportions of cells. Nat. Neurosci. 23, 1168–1175 (2020).
Pacheco, D. A., Thiberge, S. Y., Pnevmatikakis, E. & Murthy, M. Auditory activity is diverse and widespread throughout the central brain of Drosophila. Nat. Neurosci 24, 93–104 (2021).
Deutsch, D. et al. The neural basis for a persistent internal state in Drosophila females. eLife 9, e59502 (2020).
Lemon, W. C. et al. Whole-central nervous system functional imaging in larval Drosophila. Nat. Commun. 6, 7924 (2015).
Sun, X. & Heckscher, E. S. Using linear agarose channels to study Drosophila larval crawling behavior. J. Vis. Exp. 2016, 1–7 (2016).
Hinsch, K. & Zupanc, G. Generation and long-term persistence of new neurons in the adult zebrafish brain: a quantitative analysis. Neuroscience 146, 679–696 (2007).
Orger, M. B. & De Polavieja, G. G. Zebrafish behavior: opportunities and challenges. Ann. Rev. Neurosci. 40, 125–147 (2017).
Mu, Y., Narayan, S., Mensh, B. D. & Ahrens, M. B. Brain-wide, scale-wide physiology underlying behavioral flexibility in zebrafish. Curr. Opin. Neurobiol. 64, 151–160 (2020).
Kunst, M. et al. A cellular-resolution atlas of the larval zebrafish brain. Neuron 103, 21–38.e5 (2019).
Ahrens, M. B., Orger, M. B., Robson, D. N., Li, J. M. & Keller, P. J. Whole-brain functional imaging at cellular resolution using light-sheet microscopy. Nat. Methods 10, 413–420 (2013).
Vanwalleghem, G. C., Ahrens, M. B. & Scott, E. K. Integrative whole-brain neuroscience in larval zebrafish. Curr. Opin. Neurobiol. 50, 136–145 (2018).
Ahrens, M. B., Huang, K. H., Narayan, S., Mensh, B. D. & Engert, F. Two-photon calcium imaging during fictive navigation in virtual environments. Front. Neural Circuits 7, 104 (2013).
Dunn, T. W. et al. Brain-wide mapping of neural activity controlling zebrafish exploratory locomotion. eLife 5, e12741 (2016).
Haesemeyer, M., Robson, D. N., Li, J. M., Schier, A. F. & Engert, F. A brain-wide circuit model of heat-evoked swimming behavior in larval zebrafish. Neuron 98, 817–831 (2018).
Lovett-Barron, M. et al. Multiple convergent hypothalamus-brainstem circuits drive defensive behavior. Nat. Neurosci. 23, 959–967 (2020).
Bahl, A. & Engert, F. Neural circuits for evidence accumulation and decision making in larval zebrafish. Nat. Neurosci. 23, 94–102 (2020).
Dragomir, E. I., Štih, V. & Portugues, R. Evidence accumulation during a sensorimotor decision task revealed by whole-brain imaging. Nat. Neurosci. 23, 85–93 (2020).
Kawashima, T., Zwart, M. F., Yang, C. T., Mensh, B. D. & Ahrens, M. B. The serotonergic system tracks the outcomes of actions to mediate short-term motor learning. Cell 167, 933–946.e20 (2016).
Lin, Q. et al. Cerebellar neurodynamics predict decision timing and outcome on the single-trial level. Cell 180, 536–551.e17 (2020).
Mu, Y. et al. Glia accumulate evidence that actions are futile and suppress unsuccessful behavior. Cell 178, 27–43 (2019).
Wolf, S. et al. Sensorimotor computation underlying phototaxis in zebrafish. Nat. Commun. 8, 651 (2017).
Chen, X. et al. Brain-wide organization of neuronal activity and convergent sensorimotor transformations in larval zebrafish. Neuron 100, 876–890 (2018).
Oikonomou, G. et al. The serotonergic raphe promote sleep in zebrafish and mice. Neuron 103, 686–701.e8 (2019).
Andalman, A. S. et al. Neuronal dynamics regulating brain and behavioral state transitions. Cell 177, 970–985.e20 (2019).
Lovett-Barron, M. et al. Ancestral circuits for the coordinated modulation of brain state. Cell 171, 1411–1423.e17 (2017).
Migault, G. et al. Whole-brain calcium imaging during physiological vestibular stimulation in larval zebrafish. Curr. Biol. 28, 3723–3735.e6 (2018).
Marques, J. C., Li, M., Schaak, D., Robson, D. N. & Li, J. M. Internal state dynamics shape brainwide activity and foraging behaviour. Nature 577, 239–243 (2020).
Chalfie, M. et al. The neural circuit for touch sensitivity in Caenorhabditis elegans. J. Neurosci. 5, 956–964 (1985).
Berck, M. E. et al. The wiring diagram of a glomerular olfactory system. eLife 5, e14859 (2016).
Gerhard, S., Andrade, I., Fetter, R. D., Cardona, A. & Schneider-Mizell, C. M. Conserved neural circuit structure across Drosophila larval development revealed by comparative connectomics. eLife 6, e29089 (2017).
Zheng, Z. et al. A complete electron microscopy volume of the brain of adult Drosophila melanogaster. Cell 174, 730–743.e22 (2018).
Dorkenwald, S. et al. FlyWire: online community for whole-brain connectomics. Nat. Methods 19, 119–128 (2022).
Turner, M. H., Mann, K. & Clandinin, T. R. The connectome predicts resting-state functional connectivity across the Drosophila brain. Curr. Biol. 31, 2386–2394 (2021).
Hildebrand, D. G. C. et al. Whole-brain serial-section electron microscopy in larval zebrafish. Nature 545, 345–349 (2017).
Lichtman, J. W. & Denk, W. The big and the small: challenges of imaging the brain’s circuits. Science 334, 618–623 (2011).
Swanson, L. W. & Lichtman, J. W. From Cajal to connectome and beyond. Annu. Rev. Neurosci. 39, 197–216 (2016).
Pnevmatikakis, E. A. et al. Simultaneous denoising, deconvolution, and demixing of calcium imaging data. Neuron 89, 285–299 (2016).
Zhou, P. et al. Efficient and accurate extraction of in vivo calcium signals from microendoscopic video data. eLife 7, e28728 (2018).
Pachitariu, M., Stringer, C. & Harris, K. D. Robustness of spike deconvolution for neuronal calcium imaging. J. Neurosci. 38, 7976–7985 (2018).
Giovannucci, A. et al. CaImAn an open source tool for scalable calcium imaging data analysis. eLife 8, e38173 (2019).
Mathis, A. et al. DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21, 1281–1289 (2018).
Wu, A. et al. Deep graph pose: a semi-supervised deep graphical model for improved animal pose tracking. In Adv. Neural Inf. Proc. Syst. 33 (NeurIPS, 2020).
Bala, P. C. et al. Automated markerless pose estimation in freely moving macaques with openmonkeystudio. Nat. Commun. 11, 4560 (2020).
Zhang, L., Dunn, T., Marshall, J., Olveczky, B. & Linderman, S. Animal pose estimation from video data with a hierarchical von Mises–Fisher–Gaussian model. Proc. Machine Learning Res. 130, 2800–2808 (2021).
Abbott, L. F. et al. The mind of a mouse. Cell 182, 1372–1376 (2020).
Linderman, S. W. & Gershman, S. J. Using computational theory to constrain statistical models of neural data. Curr. Opin. Neurobiol. 46, 14–24 (2017).
Tukey, J. W. Exploratory Data Analysis Vol. 2 (Addison-Wesley, 1977).
Cunningham, J. P. & Yu, B. M. Dimensionality reduction for large-scale neural recordings. Nat. Neurosci. 17, 1500–1509 (2014).
York, R. A., Carreira-Rosario, A., Giocomo, L. M. & Clandinin, T. R. Flexible analysis of animal behavior via time-resolved manifold embedding. Preprint at bioRxiv https://doi.org/10.1101/2020.09.30.321406 (2021).
Wiltschko, A. B. et al. Mapping sub-second structure in mouse behavior. Neuron 88, 1121–1135 (2015).
Calhoun, A. J., Pillow, J. W. & Murthy, M. Unsupervised identification of the internal states that shape natural behavior. Nat. Neurosci. 22, 2040–2049 (2019).
Markowitz, J. E. et al. The striatum organizes 3D behavior via moment-to-moment action selection. Cell 174, 44–58.e17 (2018).
Linderman, S. W., Nichols, A. L. A., Blei, D. M., Zimmer, M. & Paninski, L. Hierarchical recurrent state space models reveal discrete and continuous dynamics of neural activity in C. elegans. Preprint at bioRxiv https://doi.org/10.1101/621540 (2019).
Paninski, L. et al. A new look at state-space models for neural data. J. Comput. Neurosci. 29, 107–126 (2010).
Murphy, K. P. Machine Learning: A Probabilistic Perspective Ch. 17–18 (MIT Press, 2012).
Eyjolfsdottir, E., Branson, K., Yue, Y. & Perona, P. Learning recurrent representations for hierarchical behavior modeling. Preprint at https://arxiv.org/abs/1611.00094 (2016).
Zhao, Y. & Park, I. M. Variational latent Gaussian process for recovering single-trial dynamics from population spike trains. Neural Comput. 29, 1293–1316 (2017).
Linderman, S. W. et al. Bayesian learning and inference in recurrent switching linear dynamical systems. PMLR 54, 914–922 (2017).
Pandarinath, C. et al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nat. Methods 15, 805–815 (2018).
Hernandez, D. et al. A novel variational family for hidden nonlinear Markov models. Preprint at https://arxiv.org/abs/1811.02459 (2018).
Duncker, L., Bohner, G., Boussard, J. & Sahani, M. Learning interpretable continuous-time models of latent stochastic dynamical systems. Preprint at https://arxiv.org/abs/1902.04420 (2019).
Batty, E. et al. BehaveNet: nonlinear embedding and Bayesian neural decoding of behavioral videos. In Adv. Neural Inf. Process. Syst. 32 (NeurIPS, 2019).
Nassar, J., Linderman, S., Bugallo, M. & Park, I. Tree-structured recurrent switching linear dynamical systems for multi-scale modeling. ICLR https://openreview.net/forum?id=HkzRQhR9YX (2019).
Glaser, J. I., Whiteway, M., Cunningham, J. P., Paninski, L. & Linderman, S. W. Recurrent switching dynamical systems models for multiple interacting neural populations. In Adv. Neural Inf. Proc. Syst. 33 (NeurIPS, 2020).
Zoltowski, D., Pillow, J. & Linderman, S. A general recurrent state space framework for modeling neural dynamics during decision-making. PMLR 119, 11680–11691 (2020).
Sussillo, D. & Barak, O. Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural Comput. 25, 626–649 (2013).
Vyas, S., Golub, M. D., Sussillo, D. & Shenoy, K. V. Computation through neural population dynamics. Annu. Rev. Neurosci. 43, 249–275 (2020).
Ben-Yishai, R., Bar-Or, R. L. & Sompolinsky, H. Theory of orientation tuning in visual cortex. Proc. Natl Acad. Sci. USA 92, 3844–3848 (1995).
Skaggs, W. E., Knierim, J. J., Kudrimoti, H. S. & McNaughton, B. L. A model of the neural basis of the rat’s sense of direction. Adv. Neural Inf. Process. Syst. 7, 173–180 (1995).
Kim, S. S., Rouault, H., Druckmann, S. & Jayaraman, V. Ring attractor dynamics in the Drosophila central brain. Science 356, 849–853 (2017).
Yamins, D. L. K. & DiCarlo, J. J. Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. 19, 356–365 (2016).
Nayebi, A. et al. Task-driven convolutional recurrent models of the visual system. Adv. Neural Inf. Proc. Syst. 31, 5290–5301 (2018).
Sussillo, D., Churchland, M. M., Kaufman, M. T. & Shenoy, K. V. A neural network that finds a naturalistic solution for the production of muscle activity. Nat. Neurosci. 18, 1025–1033 (2015).
Kriegeskorte, N., Mur, M. & Bandettini, P. Representational similarity analysis — connecting the branches of systems neuroscience. Front. Syst. Neurosci. 2, 4 (2008).
Wadhams, G. H. & Armitage, J. P. Making sense of it all: bacterial chemotaxis. Nat. Rev. Mol. Cell Biol. 5, 1024–1037 (2004).
Frankel, N. W. et al. Adaptability of non-genetic diversity in bacterial chemotaxis. eLife 3, e03526 (2014).
Waite, A. J. et al. Non-genetic diversity modulates population performance. Mol. Syst Biol. 12, 895 (2016).
Vogels, T. P., Rajan, K. & Abbott, L. F. Neural network dynamics. Annu. Rev. Neurosci. 28, 357–376 (2005).
Chen, X., Randi, F., Leifer, A. M. & Bialek, W. Searching for collective behavior in a small brain. Phys. Rev. E 99, 052418 (2019).
Williams, A. H., Degleris, A., Wang, Y. & Linderman, S. W. Point process models for sequence detection in high-dimensional neural spike trains. Adv. Neural Inf. Proc. Syst. 33, 14350–14361 (NeurIPS, 2020).
The authors thank J. Kanwal, D. Zimmerman and V. Susoy for discussions. This work was partially supported by funding from the Simons Foundation SCGB 697092 and US National Institutes of Health (NIH) Brain Initiatives U19NS113201 and R01NS11311 awarded to S.W.L., US National Science Foundation (IOS-1452593) and NIH (R01 NS082525, R01 GM130842-01 and U01-NS111697) grants to A.D.T.S., and a Burroughs Wellcome Career Award and American Federation for Aging Research Junior Faculty Award to V.V.
The authors declare no competing interests.
Peer review information
Nature Reviews Physics thanks Joshua Shaevitz, Ralph Greenspan and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lin, A., Witvliet, D., Hernandez-Nunez, L. et al. Imaging whole-brain activity to understand behaviour. Nat Rev Phys 4, 292–305 (2022). https://doi.org/10.1038/s42254-022-00430-w
This article is cited by
Single-Shot Light-Field Microscopy: An Emerging Tool for 3D Biomedical Imaging
BioChip Journal (2022)