Despite rich behavioral evidence, it is unclear how the brain expands its behavior repertoire. By building theoretical models with a deep reinforcement learning algorithm, I show that the brain composes a behavior to solve a novel task by combining previously acquired skills and augmenting their variability.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$189.00 per year
only $15.75 per issue
Rent or buy this article
Get just this article for as long as you need it
$39.95
Prices may be subject to local taxes which are calculated during checkout
References
Kohler, W. The Mentality of Apes (trans Winter, E.) (Liveright, 1976). This book presents the intelligent behavior repertoires of chimpanzees.
Epstein, R., Kirshnit, C. E., Lanza, R. P. & Rubin, L. C. ‘Insight’ in the pigeon: antecedents and determinants of an intelligent performance. Nature 308, 61–62 (1984). This paper reports the sudden acquisition of a problem solution in pigeons by combining pre-learned motor skills.
Arthur, W. B. The Nature of Technology: What It is and How It Evolves (Free Press, 2009). This book presents the concept of combinatorial evolution of technology.
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J. & Kurth-Nelson, Z. Deep reinforcement learning and its neuroscientific implications. Neuron 107, 603–616 (2020). This paper emphasizes the benefits of collaborative efforts between deep reinforcement learning and neuroscience.
Haarnoja, T. et al. Composable deep reinforcement learning for robotic manipulation. 2018 IEEE International Conf. on Robotics and Automation (ICRA) 6244–6251 (2018). This paper reports a deep reinforcement learning-based theoretical framework for combinatorial policy composition in machines.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This is a summary of: Makino, H. Arithmetic value representation for hierarchical behavior composition. Nat. Neurosci. https://doi.org/10.1038/s41593-022-01211-5 (2022).
Rights and permissions
About this article
Cite this article
Deep reinforcement learning to study combinatorial expansion of a behavior repertoire. Nat Neurosci 26, 2–3 (2023). https://doi.org/10.1038/s41593-022-01235-x
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41593-022-01235-x
