Biological organisms learn from interactions with their environment throughout their lifetime. For artificial systems to successfully act and adapt in the real world, it is desirable to similarly be able to learn on a continual basis. This challenge is known as lifelong learning, and remains to a large extent unsolved. In this Perspective article, we identify a set of key capabilities that artificial systems will need to achieve lifelong learning. We describe a number of biological mechanisms, both neuronal and non-neuronal, that help explain how organisms solve these challenges, and present examples of biologically inspired models and biologically plausible mechanisms that have been applied to artificial systems in the quest towards development of lifelong learning machines. We discuss opportunities to further our understanding and advance the state of the art in lifelong learning, aiming to bridge the gap between natural and artificial intelligence.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
Sleep-like unsupervised replay reduces catastrophic forgetting in artificial neural networks
Nature Communications Open Access 15 December 2022
Three types of incremental learning
Nature Machine Intelligence Open Access 05 December 2022
Learning plastic matching of robot dynamics in closed-loop central pattern generators
Nature Machine Intelligence Open Access 18 July 2022
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 per month
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$99.00 per year
only $8.25 per issue
Rent or buy this article
Get just this article for as long as you need it
Prices may be subject to local taxes which are calculated during checkout
Kandel, E. R. & Hawkins, R. D. The biological basis of learning and individuality. Sci. Am. 267, 78–87 (1992).
Carlson, A. et al. Toward an architecture for never-ending language learning. In 24th AAAI Conf. on Artificial Intelligence (2010).
Mitchell, T. et al. Never-ending learning. Commun. ACM 61, 103–115 (2018).
Wen, Y., Tran, D. & Ba, J. Batchensemble: an alternative approach to efficient ensemble and lifelong learning. In Int. Conf. Learning Representations (2019).
Lopez-Paz, D. & Ranzato, M. Gradient episodic memory for continual learning. Adv. Neural Inf. Process. Syst. 30, 6467–6476 (2017).
Rebuffi, S.-A., Kolesnikov, A., Sperl, G. & Lampert, C. H. icarl: Incremental classifier and representation learning. In Proc. IEEE Conf. Computer Vision and Pattern Recognition 2001–2010 (2017).
Nguyen, C. V., Li, Y., Bui, T. D. & Turner, R. E. Variational continual learning. In Int. Conf. Learning Representations (2018).
Javed, K. & White, M. Meta-learning representations for continual learning. In Proc. 33rd Int. Conf. Neural Information Processing Systems 1820–1830 (2019).
Xie, A., Harrison, J. & Finn, C. Deep reinforcement learning amidst continual structured non-stationarity. In Int. Conf. Machine Learning 11393–11403 (PMLR, 2021).
Khetarpal, K., Riemer, M., Rish, I. & Precup, D. Towards continual reinforcement learning: A review and perspectives. Preprint at https://arxiv.org/abs/2012.13490 (2020).
Chaudhry, A. et al. Continual learning with tiny episodic memories. Preprint at https://arxiv.org/abs/1902.10486 (2019).
Hayes, T. L., Cahill, N. D. & Kanan, C. Memory efficient experience replay for streaming learning. In 2019 Int. Conf. Robotics and Automation (ICRA) 9769–9776 (IEEE, 2019).
Smith, J. et al. Always be dreaming: A new approach for data-free class-incremental learning. In Int. Conf. Computer Vision (ICCV) (2021).
Ebrahimi, S., Meier, F., Calandra, R., Darrell, T. & Rohrbach, M. Adversarial continual learning. In European Conference on Computer Vision (2020).
Rusu, A. A. et al. Progressive neural networks. Preprint at https://arxiv.org/abs/1606.04671 (2016).
Schwarz, J. et al. Progress & compress: a scalable framework for continual learning. In Int. Conf. Machine Learning 4528–4537 (PMLR, 2018).
van de Ven, G. M., Li, Z. & Tolias, A. S. Class-incremental learning with generative classifiers. In Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR) Workshops 3611–3620 (2021).
Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M. & Tuytelaars, T. Memory aware synapses: learning what (not) to forget. In European Conference on Computer Vision (ECCV) (eds Ferrari, V. et al.) vol 11207, 144–161 (Springer, 2018).
Castro, F. M., Marín-Jiménez, M. J., Guil, N., Schmid, C. & Alahari, K. End-to-end incremental learning. In European Conference on Computer Vision (ECCV) 241–257 (2018).
Li, Z. & Hoiem, D. Learning without forgetting. IEEE Trans. Pattern Anal. Mach. Intell. 40, 2935–2947 (2017).
Sun, Q., Liu, Y., Chua, T.-S. & Schiele, B. Meta-transfer learning for few-shot learning. In Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition 403–412 (2019).
Hospedales, T. M., Antoniou, A., Micaelli, P. & Storkey, A. J. Meta-learning in neural networks: A survey. In IEEE Trans. Pattern Analysis and Machine Intelligence (2021).
Najarro, E. & Risi, S. Meta-learning through Hebbian plasticity in random networks. In Advances in Neural Information Processing Systems 33, 20719–20731 (2020).
Grossberg, S. Competitive learning - from interactive activation to adaptive resonance. Cognit. Sci. 11, 23–63 (1987).
Jaderberg, M. et al. Reinforcement learning with unsupervised auxiliary tasks. Preprint at https://arxiv.org/abs/1611.05397 (2016).
Wang, Y., Yao, Q., Kwok, J. & Ni, L. M. Generalizing from a few examples: A survey on few-shot learning. https://arxiv.org/abs/1904.05046 (2020).
Caruana, R. Multitask learning. Mach. Learn. 28, 41–75 (1997).
Van de Ven, G. M. & Tolias, A. S. Three scenarios for continual learning. Preprint at https://arxiv.org/abs/1904.07734 (2019).
Andrychowicz, O. M. et al. Learning dexterous in-hand manipulation. Int. J. Rob. Res. 39, 3–20 (2020).
Schwarz, J. et al. Progress and compress: a scalable framework for continual learning. In Proc. 35th Int. Conf. Machine Learning (eds. Dy, J. & Krause, A.) 80, 4528–4537 (PMLR, 2018).
Kaplanis, C., Shanahan, M. & Clopath, C. Policy consolidation for continual reinforcement learning. In Proc. 36th Int. Conf. Machine Learning (eds. Chaudhuri, K. & Salakhutdinov, R.) 97, 3242–3251 (PMLR, 2019).
Traoré, K. R. et al. DisCoRL: continual reinforcement learning via policy distillation. Preprint at https://arxiv.org/abs/1907.05855 (2019).
Neumaier, A. Solving ill-conditioned and singular linear systems: a tutorial on regularization. SIAM Rev. 40, 636–666 (1998).
Jing, L. & Tian, Y. Self-supervised visual feature learning with deep neural networks: a survey. In IEEE Trans. Pattern Analysis and Machine Intelligence (2020).
Burda, Y. et al. Large-scale study of curiosity-driven learning. In Int. Conf. Learning Representations (2019).
Wang, R. et al. Enhanced poet: open-ended reinforcement learning through unbounded invention of learning challenges and their solutions. In Int. Conf. Machine Learning 9940–9951 (PMLR, 2020).
Kuhn, H. G., Dickinson-Anson, H. & Gage, F. H. Neurogenesis in the dentate gyrus of the adult rat: age-related decrease of neuronal progenitor proliferation. J. Neurosci. 16, 2027–2033 (1996).
Lim, D. A. & Alvarez-Buylla, A. The adult ventricular–subventricular zone (V-SVZ) and olfactory bulb (OB) neurogenesis. Cold Spring Harbor Perspect. Biol. 8, a018820 (2016).
Kempermann, G., Kuhn, H. G. & Gage, F. H. Experience-induced neurogenesis in the senescent dentate gyrus. J. Neurosci. 18, 3206–3212 (1998).
Taliaz, D. Skills development in infants: a possible role for widespread neurogenesis? Front. Behav. Neurosci. 7, 178 (2013).
Saumweber, T. et al. Functional architecture of reward learning in mushroom body extrinsic neurons of larval drosophila. Nat. Commun. 9, 1104 (2018).
Blackiston, D. J., Silva Casey, E. & Weiss, M. R. Retention of memory through metamorphosis: can a moth remember what it learned as a caterpillar? PLoS ONE 3, e1736 (2008).
Wilson, M. A. & McNaughton, B. L. Reactivation of hippocampal ensemble memories during sleep. Science 265, 676–679 (1994).
Ji, D. & Wilson, M. A. Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat. Neurosci. 10, 100–107 (2007).
McClelland, J. L., McNaughton, B. L. & O’Reilly, R. C. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol. Rev. 102, 419 (1995).
Rasch, B. & Born, J. Maintaining memories by reactivation. Curr. Opin. Neurobiol. 17, 698–703 (2007).
Girardeau, G., Benchenane, K., Wiener, S. I., Buzsáki, G. & Zugaro, M. B. Selective suppression of hippocampal ripples impairs spatial memory. Nat. Neurosci. 12, 1222 (2009).
Oudiette, D. & Paller, K. A. Upgrading the sleeping brain with targeted memory reactivation. Trends Cognit. Sci. 17, 142–149 (2013).
van de Ven, G. M., Trouche, S., McNamara, C. G., Allen, K. & Dupret, D. Hippocampal offline reactivation consolidates recently formed cell assembly patterns during sharp wave-ripples. Neuron 92, 968–974 (2016).
Gridchyn, I., Schoenenberger, P., O’Neill, J. & Csicsvari, J. Assembly-specific disruption of hippocampal replay leads to selective memory deficit. Neuron 106, 291–300 (2020).
Maquet, P. The role of sleep in learning and memory. Science 294, 1048–1052 (2001).
Stella, F., Baracskay, P., O’Neill, J. & Csicsvari, J. Hippocampal reactivation of random trajectories resembling Brownian diffusion. Neuron 102, 450–461 (2019).
Robins, A. Catastrophic forgetting, rehearsal and pseudorehearsal. Connect. Sci. 7, 123–146 (1995).
van de Ven, G. M., Siegelmann, H. T. & Tolias, A. S. Brain-inspired replay for continual learning with artificial neural networks. Nat. Commun. 11, 4069 (2020).
Rasch, B. & Born, J. About sleep’s role in memory. Physiol. Rev. 93, 681–766 (2013).
Stickgold, R. Parsing the role of sleep in memory processing. Curr. Opin. Neurobiol. 23, 847–853 (2013).
O’Donnell, C. & Sejnowski, T. J. Selective memory generalization by spatial patterning of protein synthesis. Neuron 82, 398–412 (2014).
Langille, J. J. & Brown, R. E. The synaptic theory of memory: a historical survey and reconciliation of recent opposition. Front. Syst. Neurosci. 12, 52 (2018).
Abraham, W. C. & Bear, M. F. Metaplasticity: the plasticity of synaptic plasticity. Trends Neurosci. 19, 126–130 (1996).
Abraham, W. C. Metaplasticity: tuning synapses and networks for plasticity. Nat. Rev. Neurosci. 9, 387 (2008).
Dudai, Y. & Eisenberg, M. Rites of passage of the engram: reconsolidation and the lingering consolidation hypothesis. Neuron 44, 93–100 (2004).
Finnie, P. S. B. & Nader, K. The role of metaplasticity mechanisms in regulating memory destabilization and reconsolidation. Neurosci. Biobehav. Rev. 36, 1667–1707 (2012).
Bailey, C. H., Giustetto, M., Huang, Y.-Y., Hawkins, R. D. & Kandel, E. R. Is heterosynaptic modulation essential for stabilizing Hebbian plasiticity and memory. Nat. Rev. Neurosci. 1, 11–20 (2000).
Bartol Jr, T. M. et al. Nanoconnectomic upper bound on the variability of synaptic plasticity. eLife 4, e10778 (2015).
Fusi, S. Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol. Cybern. 87, 459–470 (2002).
Fusi, S., Drew, P. & Abbott, L. F. Cascade models of synaptically stored memories. Neuron 45, 599–611 (2005).
Benna, M. & Fusi, S. Computational principles of synaptic memory consolidation. Nat. Neurosci. 19, 1697–1706 (2016).
Dayan, P. & Yu, A. Phasic norepinephrine: a neural interrupt signal for unexpected events. Network Comput. Neural Syst. 17, 335–350 (2006).
Hasselmo, M. & McGaughy, J. High acetylcholine levels set circuit dynamics for attention and encoding and low acetylcholine levels set dynamics for consolidation. Prog. Brain Res. 145, 207–231 (2004).
Zou, X., Kolouri, S., Pilly, P. K. & Krichmar, J. L. Neuromodulated attention and goal-driven perception in uncertain domains. Neural Networks 125, 56–69 (2020).
Xiao, C. et al. Cholinergic mesopontine signals govern locomotion and reward through dissociable midbrain pathways. Neuron 90, 333–347 (2016).
Miyazaki, K. et al. Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience. Nat. Commun. 9, 2048 (2018).
Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Sutton, R. & Barto, A. Reinforcement Learning: An Introduction 2nd edn (MIT Press, 1998).
Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015).
Yu, A. J. & Dayan, P. Uncertainty, neuromodulation, and attention. Neuron 46, 681–692 (2005).
Hangya, B., Ranade, S. P., Lorenc, M. & Kepecs, A. Central cholinergic neurons are rapidly recruited by reinforcement feedback. Cell 162, 1155–1168 (2015).
Brna, A. P. et al. Uncertainty-based modulation for lifelong learning. Neural Networks 120, 129–142 (2019).
Hwu, T. & Krichmar, J. L. A neural model of schemas and memory encoding. Biol. Cybern. 114, 169–186 (2020).
Cho, J. R. et al. Dorsal raphe dopamine neurons modulate arousal and promote wakefulness by salient stimuli. Neuron 94, 1205–1219 (2017).
Matthews, G. A. et al. Dorsal raphe dopamine neurons represent the experience of social isolation. Cell 164, 617–631 (2016).
Otani, S., Daniel, H., Roisin, M.-P. & Crepel, F. Dopaminergic modulation of long-term synaptic plasticity in rat prefrontal neurons. Cereb. Cortex 13, 1251–1256 (2003).
Li, A., Rao, X., Zhou, Y. & Restrepo, D. Complex neural representation of odour information in the olfactory bulb. Acta Physiol. 228, e13333 (2020).
Beaulieu, S. et al. Learning to continually learn. In ECAI 2020 992–1001 (IOS Press, 2020).
Ellefsen, K. O., Mouret, J.-B. & Clune, J. Neural modularity helps organisms evolve to learn new skills without forgetting old skills. PLoS Comput. Biol. 11, e1004128 (2015).
Velez, R. & Clune, J. Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks. PLoS ONE 12, e0187736 (2017).
Miconi, T., Rawal, A., Clune, J. & Stanley, K. O. Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity. In Int. Conf. Learning Representations (2019).
Daram, A., Yanguas-Gil, A. & Kudithipudi, D. Exploring neuromodulation for dynamic learning. Front. Neurosci. 14, 928 (2020).
Madireddy, S., Yanguas-Gil, A. & Balaprakash, P. Neuromodulated neural architectures with local error signals for memory-constrained online continual learning. Preprint at https://arxiv.org/abs/2007.08159 (2021).
Kay, L. M. & Laurent, G. Odor- and context-dependent modulation of mitral cell activity in behaving rats. Nat. Neurosci. 2, 1003–1009 (1999).
Hermer-Vazquez, R., Hermer-Vazquez, L., Srinivasan, S. & Chapin, J. K. Beta- and gamma-frequency coupling between olfactory and motor brain regions prior to skilled olfactory-driven reaching. Exp. Brain Res. 180, 217–235 (2007).
Kiselycznyk, C. L., Zhang, S. & Linster, C. Role of centrifugal projections to the olfactory bulb in olfactory processing. Learn. Mem. 13, 575–579 (2006).
Levinson, M. et al. Context-dependent odor learning requires the anterior olfactory nucleus. Behav. Neurosci. 134, 332–343 (2020).
Linster, C. & Kelsch, W. A computational model of oxytocin modulation of olfactory recognition memory. eNeuro 6, ENEURO.0201-19.2019 (2019).
Benn, Y. et al. The neural basis of monitoring goal progress. Front. Hum. Neurosci. 8, 688 (2014).
Wiederman, S. D., Fabian, J. M., Dunbier, J. R. & O’Carroll, D. C. A predictive focus of gain modulation encodes target trajectories in insect vision. eLife 6, e26478 (2017).
Baluch, F. & Itti, L. Mechanisms of top-down attention. Trends Neurosci. 34, 210–224 (2011).
Baxter, M. G. & Chiba, A. A. Cognitive functions of the basal forebrain. Curr. Opin. Neurobiol. 9, 178–183 (1999).
Oros, N., Chiba, A. A., Nitz, D. A. & Krichmar, J. L. Learning to ignore: a modeling study of a decremental cholinergic pathway and its influence on attention and learning. Learn. Mem. 21, 105–118 (2014).
Duszkiewicz, A. J., McNamara, C. G., Takeuchi, T. & Genzel, L. Novelty and dopaminergic modulation of memory persistence: a tale of two systems. Trends Neurosci. 42, 102–114 (2019).
Tse, D. et al. Schemas and memory consolidation. Science 316, 76–82 (2007).
Tse, D. et al. Schema-dependent gene activation and memory encoding in neocortex. Science 333, 891–895 (2011).
van Kesteren, M. T., Ruiter, D. J., Fernández, G. & Henson, R. N. How schema and novelty augment memory formation. Trends Neurosci. 35, 211–219 (2012).
Swanson, L. Brain Maps: Structure of the Rat Brain (Gulf Professional Publishing, 2004).
Scheffer, L. K. & Meinertzhagen, I. A. The Fly Brain Atlas. Annu. Rev. Cell Dev. Biol. 35, 637–653 (2019).
Pipkin, J. Connectomes: mapping the mind of a fly. eLife 9, e62451 (2020).
Schwab, I. & Coates, M. Is the brain overrated? Br. J. Ophthalmol. 87, 525–525 (2003).
Healy, S. D. & Rowe, C. A critique of comparative studies of brain size. Proc. R. Soc. B 274, 453–464 (2007).
Borrelli, L. Testing the Contribution of Relative Brain Size and Learning Capabilities on the Evolution of Octopus vulgaris and Other Cephalopods. PhD thesis, Open Univ. (2007).
Aflalo, T. et al. Decoding motor imagery from the posterior parietal cortex of a tetraplegic human. Science 348, 906–910 (2015).
Rongala, U. B. et al. Intracellular dynamics in cuneate nucleus neurons support self-stabilizing learning of generalizable tactile representations. Front. Cell. Neurosci. 12, 210 (2018).
Kwiatkowski, R. & Lipson, H. Task-agnostic self-modeling machines. Sci. Robot. 4, eaau9354 (2019).
Marjaninejad, A., Urbina-Meléndez, D., Cohn, B. A. & Valero-Cuevas, F. J. Autonomous functional movements in a tendon-driven limb via limited experience. Nat. Mach. Intell. 1, 144–154 (2019).
Jalaleddini, K. et al. Neuromorphic meets neuromechanics, part II: the role of fusimotor drive. J. Neural Eng. 14, 025002 (2017).
Kawato, M., Furukawa, K. & Suzuki, R. A hierarchical neural-network model for control and learning of voluntary movement. Biol. Cybern. 57, 169–185 (1987).
Kawato, M., Uno, Y., Isobe, M. & Suzuki, R. Hierarchical neural network model for voluntary movement with application to robotics. IEEE Control Syst. Mag. 8, 8–15 (1988).
Merel, J., Botvinick, M. & Wayne, G. Hierarchical motor control in mammals and machines. Nat. Commun. 10, 5489 (2019).
Brooks, R. A. Intelligence without representation. Artif. Intell. 47, 139–159 (1991).
Grillner, S. & Wallen, P. Central pattern generators for locomotion, with special reference to vertebrates. Annu. Rev. Neurosci. 8, 233–261 (1985).
Grillner, S. Locomotion in vertebrates: central mechanisms and reflex interaction. Physiol. Rev. 55, 247–304 (1975).
Daun, S., Rubin, J. E. & Rybak, I. A. Control of oscillation periods and phase durations in half-center central pattern generators: a comparative mechanistic analysis. J. Comput. Neurosci. 27, 3–36 (2009).
Raphael, G., Tsianos, G. A. & Loeb, G. E. Spinal-like regulator facilitates control of a two-degree-of-freedom wrist. J. Neurosci. 30, 9431–9444 (2010).
Markin, S. N. et al. In Neuromechanical Modeling of Posture and Locomotion (eds Prilutsky, B. I. & Edwards D. H.) 21–65 (Springer, 2016).
Kandel, E. R. et al. Principles of Neural Science Vol. 4 (McGraw-Hill, 2000).
Valero-Cuevas, F. J. Fundamentals of Neuromechanics Vol. 8 (Series in Biosystems & Biorobotics, Springer, 2016).
Ijspeert, A. J. Biorobotics: using robots to emulate and investigate agile locomotion. Science 346, 196–203 (2014).
Treweek, J. B. & Gradinaru, V. Extracting structural and functional features of widely distributed biological circuits with single cell resolution via tissue clearing and delivery vectors. Curr. Opin. Biotechnol. 40, 193–207 (2016).
Chung, K. & Deisseroth, K. CLARITY for mapping the nervous system. Nat. Methods 10, 508–513 (2013).
Oh, S. W. et al. A mesoscale connectome of the mouse brain. Nature 508, 207–214 (2014).
Flash, T. & Hochner, B. Motor primitives in vertebrates and invertebrates. Curr. Opin. Neurobiol. 15, 660–666 (2005).
Baluška, F. & Levin, M. On having no head: cognition throughout biological systems. Front. Psychol. 7, 902 (2016).
Pezzulo, G. & Levin, M. Re-membering the body: applications of computational neuroscience to the top-down control of regeneration of limbs and other complex organs. Integr. Biol. 7, 1487–1517 (2015).
Levin, M., Pezzulo, G. & Finkelstein, J. M. Endogenous bioelectric signaling networks: exploiting voltage gradients for control of growth and form. Annu. Rev. Biomed. Eng. 19, 353–387 (2017).
Biswas, S., Manicka, S., Hoel, E. & Levin, M. Gene regulatory networks exhibit several kinds of memory: Quantification of memory in biological and random transcriptional networks. iScience 24, 102131 (2021).
Manicka, S. & Levin, M. Modeling somatic computation with non-neural bioelectric networks. Sci. Rep. 9, 18612 (2019).
Emmons-Bell, M. et al. Regenerative adaptation to electrochemical perturbation in planaria: A molecular analysis of physiological plasticity. iScience 22, 147–165 (2019).
Blackiston, D. J., Shomrat, T. & Levin, M. The stability of memories during brain remodeling: a perspective. Commun. Integr. Biol. 8, e1073424 (2015).
Blackiston, D. J. & Levin, M. Ectopic eyes outside the head in xenopus tadpoles provide sensory data for light-mediated learning. J. Exp. Biol. 216, 1031–1040 (2013).
Kriegman, S., Blackiston, D., Levin, M. & Bongard, J. A scalable pipeline for designing reconfigurable organisms. Proc. Natl Acad. Sci. USA 117, 1853–1859 (2020).
Kriegman, S. et al. Automated shapeshifting for function recovery in damaged robots. In Proc. Robotics: Science and Systems (RSS) (2019).
Purnick, P. E. & Weiss, R. The second wave of synthetic biology: from modules to systems. Nat. Rev. Molecular Cell Biol. 10, 410–422 (2009).
Pezzulo, G. & Levin, M. Top-down models in biology: explanation and control of complex living systems above the molecular level. J. R. Soc. Interface 13, 20160555 (2016).
Vandenberg, L. N., Adams, D. S. & Levin, M. Normalized shape and location of perturbed craniofacial structures in the xenopus tadpole reveal an innate ability to achieve correct morphology. Dev. Dyn. 241, 863–878 (2012).
Lipchik, E., Cohen, E. & Mewissen, M. Transvenous liver biopsy in critically ill patients: adequacy of tissue samples. Radiology 181, 497–499 (1991).
Oviedo, N. J. et al. Long-range neural and gap junction protein-mediated cues control polarity during planarian regeneration. Dev. Biol. 339, 188–199 (2010).
Fields, C., Bischof, J. & Levin, M. Morphological coordination: a common ancestral function unifying neural and non-neural signaling. Physiology 35, 16–30 (2020).
Blackiston, D. et al. A cellular platform for the development of synthetic living machines. Sci. Robot. 6, eabf1571 (2021).
Ernst, M. O. & Bülthoff, H. H. Merging the senses into a robust percept. Trends Cognit. Sci. 8, 162–169 (2004).
Stein, B. E., Stanford, T. R. & Rowland, B. A. Multisensory integration and the society for neuroscience: then and now. J. Neurosci. 40, 3–11 (2020).
Stevenson, R. A. et al. Identifying and quantifying multisensory integration: a tutorial review. Brain Topogr. 27, 707–730 (2014).
Necker, R., Janßen, A. & Beissenhirtz, T. Behavioral evidence of the role of lumbosacral anatomical specializations in pigeons in maintaining balance during terrestrial locomotion. J. Comp. Physiol. A 186, 409–412 (2000).
Urbina-Meléndez, D., Jalaleddini, K., Daley, M. A. & Valero-Cuevas, F. J. A physical model suggests that hip-localized balance sense in birds improves state estimation in perching: implications for bipedal robots. Front. Robot. AI 5, 38 (2018).
Holmes, N. P. & Spence, C. Multisensory integration: space, time and superadditivity. Curr. Biol. 15, R762–R764 (2005).
Berry, J. A. & Valero-Cuevas, F. J. Sensory-motor gestalt: Sensation and action as the foundations of identity, agency, and self. In Artificial Life Conf. Proc. 130–138 (MIT Press, 2020).
Tan, H., Zhou, Y., Tao, Q., Rosen, J. & van Dijken, S. Bioinspired multisensory neural network with crossmodal integration and recognition. Nat. Commun. 12, 1120 (2021).
Silver, D. et al. Mastering the game of go with deep neural networks and tree search. Nature 529, 484–489 (2016).
Risi, S. & Stanley, K. O. A unified approach to evolving plasticity and neural geometry. In The 2012 Int. Joint Conference on Neural Networks (IJCNN) (IEEE, 2012).
Imam, N. & Cleland, T. A. Rapid online learning and robust recall in a neuromorphic olfactory circuit. Nat. Mach. Intell. 2, 181–191 (2020).
Soltoggio, A., Bullinaria, J. A., Mattiussi, C., Dürr, P. & Floreano, D. Evolutionary advantages of neuromodulated plasticity in dynamic, reward-based scenarios. In Proc. 11th International Conference on Artificial Life (Alife XI) 569–576 (MIT Press, 2008).
Soltoggio, A., Stanley, K. O. & Risi, S. Born to learn: the inspiration, progress, and future of evolved plastic artificial neural networks. Neural Networks 108, 48–67 (2018).
Tsuda, B., Tye, K. M., Siegelmann, H. T. & Sejnowski, T. J. A modeling framework for adaptive lifelong learning with transfer and savings through gating in the prefrontal cortex. Proc. Natl Acad. Sci. USA 117, 29872–29882 (2020).
Warner, J., Devaraj, A. & Miikkulainen, R. Using context to make gas classifiers robust to sensor drift. Preprint at https://arxiv.org/abs/2003.07292 (2020).
Tutum, C. C., Abdulquddos, S. & Miikkulainen, R. Generalization of agent behavior through explicit representation of context. In Proc. 3rd IEEE Conference on Games (2021).
Hwu, T., Kashyap, H. & Krichmar, J. A neurobiological schema model for contextual awareness in robotics. In IEEE International Joint Conference on Neural Networks (2020).
Higgins, I. et al. beta-VAE: learning basic visual concepts with a constrained variational framework. In ICLR (2017).
Mathieu, E., Rainforth, T., Siddharth, N. & Teh, Y. W. Disentangling disentanglement in variational autoencoders. In Proc. 36th International Conference on Machine Learning Vol. 97, 4402–4412 (PMLR, 2019).
Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T. & Wang, X.-J. Task representations in neural networks trained to perform many cognitive tasks. Nat. Neurosci. 22, 297–306 (2019).
Masse, N. Y., Grant, G. D. & Freedman, D. J. Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization. Proc. Natl Acad. Sci. USA 115, E10467–E10475 (2018).
Aimone, J. B., Wiles, J. & Gage, F. H. Potential role for adult neurogenesis in the encoding of time in new memories. Nat. Neurosci. 9, 723–727 (2006).
Aimone, J. B., Wiles, J. & Gage, F. H. Computational influence of adult neurogenesis on memory encoding. Neuron 61, 187–202 (2009).
Stanley, K. O., Clune, J., Lehman, J. & Miikkulainen, R. Designing neural networks through neuroevolution. Nat. Mach. Intell. 1, 24–35 (2019).
Lee, S., Ha, J., Zhang, D. & Kim, G. A neural dirichlet process mixture model for task-free continual learning. In Int. Conf. Learning Representations (2020).
Aimone, J. B., Deng, W. & Gage, F. H. Resolving new memories: a critical look at the dentate gyrus, adult neurogenesis, and pattern separation. Neuron 70, 589–596 (2011).
Pandit, T. & Kudithipudi, D. Relational neurogenesis for lifelong learning agents. In Proc. Neuro-Inspired Computational Elements Workshop (Association for Computing Machinery, 2020).
González, O. C., Sokolov, Y., Krishnan, G. P., Delanois, J. E. & Bazhenov, M. Can sleep protect memories from catastrophic forgetting? eLife 9, e51005 (2020).
Krishnan, G. P., Tadros, T., Ramyaa, R. & Bazhenov, M. Biologically inspired sleep algorithm for artificial neural networks. Preprint at https://arxiv.org/abs/1908.02240 (2019).
Tadros, T., Krishnan, G. P., Ramyaa, R. & Bazhenov, M. Biologically inspired sleep algorithm for increased generalization and adversarial robustness in deep neural networks. In Int. Conf. Learning Representations (2019).
Tadros, T., Krishnan, G., Ramyaa, R. & Bazhenov, M. Biologically inspired sleep algorithm for reducing catastrophic forgetting in neural networks. AAAI Conf. Artif. Intell. 34, 13933–13934 (2020).
Shin, H., Lee, J. K., Kim, J. & Kim, J. Continual learning with deep generative replay. In Proc. 31st Int. Conf. Neural Information Processing Systems 2994–3003 (2017).
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T. P. & Wayne, G. Experience replay for continual learning. In Advances in Neural Information Processing Systems (2019).
Laborieux, A., Ernoult, M., Hirtzlin, T. & Querlioz, D. Synaptic metaplasticity in binarized neural networks. Nat. Commun. 12, 2549 (2021).
Kaplanis, C., Shanahan, M. & Clopath, C. Continual reinforcement learning with complex synapses. In Int. Conf. Machine Learning 2497–2506 (PMLR, 2018).
Zenke, F., Poole, B. & Ganguli, S. Continual learning through synaptic intelligence. In Proc. 34th Int. Conf. Machine Learning Vol. 70, 3987–3995 (JMLR, 2017).
Kirkpatrick, J. et al. Overcoming catastrophic forgetting in neural networks. Proc. Natl Acad. Sci. USA 114, 3521–3526 (2017).
Soures, N., Helfer, P., Daram, A., Pandit, T. & Kudithipudi, D. Tacos: task agnostic continual learning in spiking neural networks. In Theory and Foundation of Continual Learning Workshop at ICML’2021 (2021).
Kosiorek, A. R., Sabour, S., Teh, Y. & Hinton, G. E. Stacked capsule autoencoders. In NeurIPS (2019).
Lake, B. M., Salakhutdinov, R. & Tenenbaum, J. B. Human-level concept learning through probabilistic program induction. Science 350, 1332–1338 (2015).
Chiang, M., Low, S. H., Calderbank, A. R. & Doyle, J. C. Layering as optimization decomposition: a mathematical theory of network architectures. Proc. IEEE 95, 255–312 (2007).
Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pastor, P. & Schaal, S. Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput. 25, 328–373 (2013).
Schaal, S. In Adaptive Motion of Animals and Machines 261–280 (Springer, 2006).
Papadimitriou, C. H., Vempala, S. S., Mitropolsky, D., Collins, M. & Maass, W. Brain computation by assemblies of neurons. Proc. Natl Acad. Sci. USA 117, 14464–14472 (2020).
Zeng, T., Tang, F., Ji, D. & Si, B. Neurobayesslam: Neurobiologically inspired bayesian integration of multisensory information for robot navigation. Neural Networks 126, 21–35 (2020).
Wijesinghe, L. P., Triesch, J. & Shi, B. E. Robot end effector tracking using predictive multisensory integration. Front. Neurorobot. 12, 66 (2018).
Wang, H., Dong, S. & Shao, L. Measuring structural similarities in finite mdps. In Int. Joint Conferences on Artificial Intelligence 3684–3690 (2019).
Levin, M. Bioelectric signaling: reprogrammable circuits underlying embryogenesis, regeneration, and cancer. Cell 184, 1971–1989 (2021).
Harris, M. P. Bioelectric signaling as a unique regulator of development and regeneration. Development 148, dev180794 (2021).
Pietak, A. & Levin, M. Bioelectric gene and reaction networks: computational modelling of genetic, biochemical and bioelectrical dynamics in pattern regulation. J. R. Soc. Interface 14, 20170425 (2017).
Zohora, F. T., Karia, V., Daram, A. R., Zyarah, A. M. & Kudithipudi, D. Metaplasticnet: Architecture with probabilistic metaplastic synapses for continual learning. In 2021 IEEE International Symposium on Circuits and Systems (IEEE, 2021).
Soltoggio, A. Short-term plasticity as cause–effect hypothesis testing in distal reward learning. Biol. Cybernet. 109, 75–94 (2015).
Cui, Y., Ahmad, S. & Hawkins, J. Continuous online sequence learning with an unsupervised neural network model. Neural Comput. 28, 2474–2504 (2016).
Draelos, T. J. et al. Neurogenesis deep learning: Extending deep networks to accommodate new classes. In 2017 International Joint Conference on Neural Networks 526–533 (IEEE, 2017).
Parisi, G. I., Tani, J., Weber, C. & Wermter, S. Lifelong learning of spatiotemporal representations with dual-memory recurrent self-organization. Front. Neurorobot. 12, 78 (2018).
McClelland, J. L., McNaughton, B. L. & Lampinen, A. K. Integration of new information in memory: new insights from a complementary learning systems perspective. Philos. Trans. R. Soc. B 375, 20190637 (2020).
French, R. M. Pseudo-recurrent connectionist networks: an approach to the ‘sensitivity-stability’ dilemma. Connect. Sci. 9, 353–379 (1997).
Vogelstein, J. T. et al. Representation ensembling for synergistic lifelong learning with quasilinear complexity. Preprint at https://arxiv.org/abs/2004.12908v12 (2020).
Mead, C. How we created neuromorphic engineering. Nat. Electron. 3, 434–435 (2020).
Boahen, K. A neuromorph’s prospectus. Comput. Sci. Eng. 19, 14–28 (2017).
Davies, M. et al. Advancing neuromorphic computing with loihi: A survey of results and outlook. In Proc. IEEE (2021).
Indiveri, G. et al. Neuromorphic silicon neuron circuits. Front. Neurosci. 5, 73 (2011).
Furber, S. B., Galluppi, F., Temple, S. & Plana, L. A. The spinnaker project. Proc. IEEE 102, 652–665 (2014).
Yue, K., Liu, Y., Lake, R. K. & Parker, A. C. A brain-plausible neuromorphic on-the-fly learning system implemented with magnetic domain wall analog memristors. Sci. Adv. 5, eaau8170 (2019).
Akopyan, F. et al. Truenorth: Design and tool flow of a 65 mW 1 million neuron programmable neurosynaptic chip. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 34, 1537–1557 (2015).
Schuman, C. D. et al. A survey of neuromorphic computing and neural networks in hardware. Preprint at https://arxiv.org/abs/1705.06963 (2017).
Yanguas-Gil, A. Memristor design rules for dynamic learning and edge processing applications. APL Mater. 7, 091102 (2019).
Daram, A. R., Kudithipudi, D. & Yanguas-Gil, A. Task-based neuromodulation architecture for lifelong learning. In 20th International Symposium on Quality Electronic Design 191–197 (2019).
Soures, N., Zyarah, A., Carlson, K. D., Aimone, J. B. & Kudithipudi, D. How Neural Plasticity Boosts Performance of Spiking Neural Networks (Sandia National Lab, 2017).
Zyarah, A. M., Gomez, K. & Kudithipudi, D. Neuromorphic system for spatial and temporal information processing. IEEE Trans. Comput. 69, 1099–1112 (2020).
Hardt, O., Nader, K. & Nadel, L. Decay happens: the role of active forgetting in memory. Trends Cognit. Sci. 17, 111–120 (2013).
Bouton, M. E. Context and behavioral processes in extinction. Learn. Mem. 11, 485–494 (2004).
Hardt, O., Einarsson, E. Ö. & Nader, K. A bridge over troubled water: reconsolidation as a link between cognitive and neuroscientific memory research traditions. Annu. Rev. Psychol. 61, 141–167 (2010).
Cabessa, J. & Siegelmann, H. T. The super-turing computational power of plastic recurrent neural networks. Int. J. Neural Syst. 24, 1450029 (2014).
Lifelong Learning Machines. https://www.darpa.mil/program/lifelong-learning-machines (DARPA, accessed 25 February 2022).
Lennington, J. B., Yang, Z. & Conover, J. C. Neural stem cells and the regulation of adult neurogenesis. Reprod. Biol. Endocrinol. 1, 99 (2003).
Diba, K. & Buzsáki, G. Forward and reverse hippocampal place-cell sequences during ripples. Nat. Neurosci. 10, 1241–1242 (2007).
Lee, A. K. & Wilson, M. A. Memory of sequential experience in the hippocampus during slow wave sleep. Neuron 36, 1183–1194 (2002).
Drieu, C. & Zugaro, M. Hippocampal sequences during exploration: mechanisms and functions. Front. Cell. Neurosci. 13, 232 (2019).
Liu, T.-Y. & Watson, B. O. Patterned activation of action potential patterns during offline states in the neocortex: replay and non-replay. Phil. Trans. R. Soc. B 375, 20190233 (2020).
Doya, K. Metalearning and neuromodulation. Neural Networks 15, 495–506 (2002).
Krichmar, J. L. The neuromodulatory system: a framework for survival and adaptive behavior in a challenging world. Adapt. Behav. 16, 385–399 (2008).
Kandel, E. R. et al. (eds.) Principles of Neural Science 5th edn (McGraw-Hill Education, 2013).
Buchler, N. E. & Cross, F. R. Protein sequestration generates a flexible ultrasensitive response in a genetic network. Mol. Syst. Biol. 5, 272 (2009).
Moorman, A., Samaniego, C. C., Maley, C. & Weiss, R. A dynamical biomolecular neural network. In 2019 IEEE 58th Conf. Decision and Control 1797–1802 (IEEE, 2019).
Cuba Samaniego, C., Giordano, G., Kim, J., Blanchini, F. & Franco, E. Molecular titration promotes oscillations and bistability in minimal network models with monomeric regulators. ACS Synth. Biol. 5, 321–333 (2016).
Mendez, J. & Eaton, E. Lifelong learning of compositional structures. In Int. Conf. Learning Representations (2021).
This work was partly supported by the DARPA Lifelong Learning Machines programme. We wish to express our thanks to the technical leadership team of DARPA L2M, specifically R. McFarland, B. Epstein, R. McFarland and T. Senator. R. McFarland and B. Epstein offered several insights on organization of the paper, contributed in brainstorming sessions, and provided graphics suggestions. T. Senator seeded the idea to develop a review article. R. McFarland and other members of the L2M team spurred insightful discussions and provided feedback on the Perspective. We thank G. Vallabha, E. Johnson, M. Peot, F. Sha for reviewing the manuscript.
The authors declare no competing interests.
Peer review information
Nature Machine Intelligence thanks Nabil Imam and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
A brief explanation of the biologically inspired models mentioned in the article; and metrics that have been used to assess specific aspects of L2 performance.
Rights and permissions
About this article
Cite this article
Kudithipudi, D., Aguilar-Simon, M., Babb, J. et al. Biological underpinnings for lifelong learning machines. Nat Mach Intell 4, 196–210 (2022). https://doi.org/10.1038/s42256-022-00452-0
This article is cited by
Bio-robots step towards brain–body co-adaptation
Nature Machine Intelligence (2022)
Three types of incremental learning
Nature Machine Intelligence (2022)
Learning plastic matching of robot dynamics in closed-loop central pattern generators
Nature Machine Intelligence (2022)
Sleep-like unsupervised replay reduces catastrophic forgetting in artificial neural networks
Nature Communications (2022)