Abstract
Decisions made by mammals and birds are often temporally extended. They require planning and sampling of decision-relevant information. Our understanding of such decision-making remains in its infancy compared with simpler, forced-choice paradigms. However, recent advances in algorithms supporting planning and information search provide a lens through which we can explain neural and behavioral data in these tasks. We review these advances to obtain a clearer understanding for why planning and curiosity originated in certain species but not others; how activity in the medial temporal lobe, prefrontal and cingulate cortices may support these behaviors; and how planning and information search may complement each other as means to improve future action selection.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Gold, J. I. & Shadlen, M. N. The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–574 (2007).
Niv, Y. Reinforcement learning in the brain. J. Math. Psychol. 53, 139–154 (2009).
Glimcher, P. W. & Fehr, E. Neuroeconomics: Decision Making and the Brain 2nd edn (Elsevier/Academic Press, 2014).
Mobbs, D., Trimmer, P. C., Blumstein, D. T. & Dayan, P. Foraging for foundations in decision neuroscience: insights from ethology. Nat. Rev. Neurosci. 19, 419–427 (2018).
Daw, N. D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711 (2005).
Dolan, R. J. & Dayan, P. Goals and habits in the brain. Neuron 80, 312–325 (2013).
Redish, A. D. Vicarious trial and error. Nat. Rev. Neurosci. 17, 147–159 (2016).
Jones, J. L. et al. Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 338, 953–956 (2012).
Doll, B. B., Duncan, K. D., Simon, D. A., Shohamy, D. & Daw, N. D. Model-based choices involve prospective neural activity. Nat. Neurosci. 18, 767–772 (2015).
Schmidt, B., Duin, A. A. & Redish, A. D. Disrupting the medial prefrontal cortex alters hippocampal sequences during deliberative decision making. J. Neurophysiol. 121, 1981–2000 (2019).
Wilkinson, A. & Huber, L. Cold-blooded cognition: Reptilian cognitive abilities. in The Oxford Handbook of Comparative Evolutionary Psychology (eds Vonk, J. & Shackelford, T. K) 129–143 (Oxford Univ. Press, 2012).
Burghardt, G. M. Environmental enrichment and cognitive complexity in reptiles and amphibians: concepts, review, and implications for captive populations. Appl. Anim. Behav. Sci. 147, 286–298 (2013).
Broglio, C. et al. Hippocampal pallium and map-like memories through vertebrate evolution. J. Behav. Brain Sci. 05, 109–120 (2015).
MacIver, M. A., Schmitz, L., Mugan, U., Murphey, T. D. & Mobley, C. D. Massive increase in visual range preceded the origin of terrestrial vertebrates. Proc. Natl Acad. Sci. USA 114, E2375–E2384 (2017).
Stein, W. E., Berry, C. M., Hernick, L. V. & Mannolini, F. Surprisingly complex community discovered in the mid-Devonian fossil forest at Gilboa. Nature 483, 78–81 (2012).
Mugan, U. & MacIver, M. A. Spatial planning with long visual range benefits escape from visual predators in complex naturalistic environments. Nat. Commun. 11, 3057 (2020).
Tolman, E. C. Cognitive maps in rats and men. Psychol. Rev. 55, 189–208 (1948).
Tse, D. et al. Schemas and memory consolidation. Science 316, 76–82 (2007).
Raby, C. R., Alexis, D. M., Dickinson, A. & Clayton, N. S. Planning for the future by western scrub-jays. Nature 445, 919–921 (2007).
Wimpenny, J. H., Weir, A. A., Clayton, L., Rutz, C. & Kacelnik, A. Cognitive processes associated with sequential tool use in New Caledonian crows. PLoS ONE 4, e6471 (2009).
Tanji, J., Shima, K. & Mushiake, H. Concept-based behavioral planning and the lateral prefrontal cortex. Trends Cogn. Sci. 11, 528–534 (2007).
Clutton-Brock, T. H. & Harvey, P. H. Primates, brains and ecology. J. Zool. 190, 309–323 (1980).
Conway, C. M. & Christiansen, M. H. Sequential learning in non-human primates. Trends Cogn. Sci. 5, 539–546 (2001).
Le Fur, S., Fara, E., Mackaye, H. T., Vignaud, P. & Brunet, M. The mammal assemblage of the hominid site TM266 (Late Miocene, Chad Basin): ecological structure and paleoenvironmental implications. Naturwissenschaften 96, 565–574 (2009).
Dunbar, R. I. M. & Shultz, S. Why are there so many explanations for primate brain evolution? Philos. Trans. R Soc. Lond. B Biol. Sci. https://doi.org/10.1098/rstb.2016.0244 (2017).
Lee, D. & Seo, H. Neural basis of strategic decision making. Trends Neurosci. 39, 40–48 (2016).
Gottlieb, J. & Oudeyer, P. Y. Towards a neuroscience of active sampling and curiosity. Nat. Rev. Neurosci. 19, 758–770 (2018).
Glickman, S. E. & Sroges, R. W. Curiosity in zoo animals. Behaviour 26, 151–188 (1966).
Montgomery, S. H. The relationship between play, brain growth and behavioural flexibility in primates. Anim. Behav. 90, 281–286 (2014).
Wimpenny, J. H., Weir, A. A. & Kacelnik, A. New Caledonian crows use tools for non-foraging activities. Anim. Cogn. 14, 459–464 (2011).
Callaway, F. et al. Human planning as optimal information seeking. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/byaqd (2021).
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P. & Dolan, R. J. Model-based influences on humans’ choices and striatal prediction errors. Neuron 69, 1204–1215 (2011).
Mattar, M. G. & Daw, N. D. Prioritized memory access explains planning and hippocampal replay. Nat. Neurosci. 21, 1609–1617 (2018).
Keramati, M., Dezfouli, A. & Piray, P. Speed/accuracy trade-off between the habitual and the goal-directed processes. PLoS Comput. Biol. 7, e1002055 (2011).
Behrens, T. E. J. et al. What Is a cognitive map? Organizing knowledge for flexible behavior. Neuron 100, 490–509 (2018).
Johnson, A. & Redish, A. D. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 27, 12176–12189 (2007).
Kay, K. et al. Constant sub-second cycling between representations of possible futures in the hippocampus. Cell 180, 552–567.e25 (2020).
Diba, K. & Buzsaki, G. Forward and reverse hippocampal place-cell sequences during ripples. Nat. Neurosci. 10, 1241–1242 (2007).
Buzsaki, G. Hippocampal sharp wave-ripple: a cognitive biomarker for episodic memory and planning. Hippocampus 25, 1073–1188 (2015).
Gupta, A. S., van der Meer, M. A., Touretzky, D. S. & Redish, A. D. Segmentation of spatial experience by hippocampal theta sequences. Nat. Neurosci. 15, 1032–1039 (2012).
Zielinski, M. C., Shin, J. D. & Jadhav, S. P. Coherent coding of spatial position mediated by theta oscillations in the hippocampus and prefrontal cortex. J. Neurosci. 39, 4550–4565 (2019).
van der Meer, M. A. & Redish, A. D. Expectancies in decision making, reinforcement learning, and ventral striatum. Front. Neurosci. 4, 6 (2010).
Gardner, R. S. et al. A secondary working memory challenge preserves primary place strategies despite overtraining. Learn. Mem. 20, 648–656 (2013).
Steiner, A. P. & Redish, A. D. The road not taken: neural correlates of decision making in orbitofrontal cortex. Front. Neurosci. 6, 131 (2012).
Powell, N. J. & Redish, A. D. Complex neural codes in rat prelimbic cortex are stable across days on a spatial decision task. Front. Behav. Neurosci. 8, 120 (2014).
Stott, J. J. & Redish, A. D. A functional difference in information processing between orbitofrontal cortex and ventral striatum during decision-making behaviour. Philos. Trans. R. Soc. Lond. B Biol. Sci. https://doi.org/10.1098/rstb.2013.0472 (2014).
Hu, D. & Amsel, A. A simple test of the vicarious trial-and-error hypothesis of hippocampal function. Proc. Natl Acad. Sci. USA 92, 5506–5509 (1995).
Meyer-Mueller, C. et al. Dorsal, but not ventral, hippocampal inactivation alters deliberation in rats. Behav. Brain Res. 390, 112622 (2020).
Kreher, M. A. et al. The perirhinal cortex supports spatial intertemporal choice stability. Neurobiol. Learn. Mem. 162, 36–46 (2019).
Procyk, E., Tanaka, Y. L. & Joseph, J. P. Anterior cingulate activity during routine and non-routine sequential behaviors in macaques. Nat. Neurosci. 3, 502–508 (2000).
Stoll, F. M., Fontanier, V. & Procyk, E. Specific frontal neural dynamics contribute to decisions to check. Nat. Commun. 7, 11990 (2016).
Singer, A. C. & Frank, L. M. Rewarded outcomes enhance reactivation of experience in the hippocampus. Neuron 64, 910–921 (2009).
Papale, A. E., Zielinski, M. C., Frank, L. M., Jadhav, S. P. & Redish, A. D. Interplay between hippocampal sharp-wave-ripple events and vicarious trial and error behaviors in decision making. Neuron 92, 975–982 (2016).
Gupta, A. S., van der Meer, M. A., Touretzky, D. S. & Redish, A. D. Hippocampal replay is not a simple function of experience. Neuron 65, 695–705 (2010).
Singer, A. C., Carr, M. F., Karlsson, M. P. & Frank, L. M. Hippocampal SWR activity predicts correct decisions during the initial learning of an alternation task. Neuron 77, 1163–1173 (2013).
Jadhav, S. P., Kemere, C., German, P. W. & Frank, L. M. Awake hippocampal sharp-wave ripples support spatial memory. Science 336, 1454–1458 (2012).
Foster, D. J. & Wilson, M. A. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440, 680–683 (2006).
Ambrose, R. E., Pfeiffer, B. E. & Foster, D. J. Reverse replay of hippocampal place cells is uniquely modulated by changing reward. Neuron 91, 1124–1136 (2016).
Davidson, T. J., Kloosterman, F. & Wilson, M. A. Hippocampal replay of extended experience. Neuron 63, 497–507 (2009).
Olafsdottir, H. F., Barry, C., Saleem, A. B., Hassabis, D. & Spiers, H. J. Hippocampal place cells construct reward related sequences through unexplored space. eLife 4, e06063 (2015).
Miller, K.J. & Venditto, S. J. C. Multi-step planning in the brain. Curr. Opin. Behav. Sci. 38, 29–39 (2021).
Kurth-Nelson, Z., Economides, M., Dolan, R. J. & Dayan, P. Fast sequences of non-spatial state representations in humans. Neuron 91, 194–204 (2016).
Momennejad, I., Otto, A. R., Daw, N. D. & Norman, K. A. Offline replay supports planning in human reinforcement learning. eLife https://doi.org/10.7554/eLife.32548 (2018).
Schuck, N. W. & Niv, Y. Sequential replay of nonspatial task states in the human hippocampus. Science https://doi.org/10.1126/science.aaw5181 (2019).
Liu, Y., Dolan, R. J., Kurth-Nelson, Z. & Behrens, T. E. J. Human replay spontaneously reorganizes experience. Cell 178, 640–652.e14 (2019).
Liu, Y., Mattar, M. G., Behrens, T. E. J., Daw, N. D. & Dolan, R. J. Experience replay is associated with efficient nonlocal learning. Science 372, eabf1357 (2021).
van Opheusden, B. & Ma, W. J. Tasks for aligning human and machine planning. Curr. Opin. Behav. Sci. 29, 127–133 (2019).
Kemp, C. & Tenenbaum, J. B. The discovery of structural form. Proc. Natl Acad. Sci. USA 105, 10687–10692 (2008).
Bengio, Y., Courville, A. & Vincent, P. Representation Learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798–1828 (2013).
Radulescu, A., Niv, Y. & Ballard, I. Holistic reinforcement learning: the role of structure and attention. Trends Cogn. Sci. 23, 278–292 (2019).
Schrittwieser, J. et al. Mastering Atari, Go, chess and shogi by planning with a learned model. Nature 588, 604–609 (2020).
Baram, A. B., Muller, T. H., Nili, H., Garvert, M. M. & Behrens, T. E. J. Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems. Neuron 109, 713–723.e7 (2021).
Schulz, E., Franklin, N. T. & Gershman, S. J. Finding structure in multi-armed bandits. Cogn. Psychol. 119, 101261 (2020).
Wu, C. M., Schulz, E., Speekenbrink, M., Nelson, J. D. & Meder, B. Generalization guides human exploration in vast decision spaces. Nat. Hum. Behav. 2, 915–924 (2018).
Harlow, H. F. The formation of learning sets. Psychol. Rev. 56, 51–65 (1949).
Wang, J. X. et al. Prefrontal cortex as a meta-reinforcement learning system. Nat. Neurosci. 21, 860–868 (2018).
Browning, P. G., Easton, A. & Gaffan, D. Frontal-temporal disconnection abolishes object discrimination learning set in macaque monkeys. Cereb. Cortex 17, 859–864 (2007).
M’Harzi, M. et al. Effects of selective lesions of fimbria-fornix on learning set in the rat. Physiol. Behav. 40, 181–188 (1987).
Schuck, N. W., Cai, M. B., Wilson, R. C. & Niv, Y. Human orbitofrontal cortex represents a cognitive map of state space. Neuron 91, 1402–1412 (2016).
Dayan, P. Improving generalization for temporal difference learning: the successor representation. Neural Comput. 5, 613–624 (1993).
Singh, S., James, M. R. & Rudary, M. R. Predictive state representations: a new theory for modeling dynamical systems. In Proc. 20th Conference on Uncertainty in Artificial Intelligence 512–519 (AUAI Press, 2004).
Gershman, S. J. The successor representation: its computational logic and neural substrates. J. Neurosci. 38, 7193–7200 (2018).
Stachenfeld, K. L., Botvinick, M. M. & Gershman, S. J. The hippocampus as a predictive map. Nat. Neurosci. 20, 1643–1653 (2017).
Mehta, M. R., Quirk, M. C. & Wilson, M. A. Experience-dependent asymmetric shape of hippocampal receptive fields. Neuron 25, 707–715 (2000).
Momennejad, I. et al. The successor representation in human reinforcement learning. Nat. Hum. Behav. 1, 680–692 (2017).
Russek, E. M., Momennejad, I., Botvinick, M. M., Gershman, S. J. & Daw, N. D. Predictive representations can link model-based reinforcement learning to model-free mechanisms. PLoS Comput. Biol. 13, e1005768 (2017).
Gardner, M. P. H., Schoenbaum, G. & Gershman, S. J. Rethinking dopamine as generalized prediction error. Proc. Biol. Sci. https://doi.org/10.1098/rspb.2018.1645 (2018).
Barnes, T. D., Kubota, Y., Hu, D., Jin, D. Z. & Graybiel, A. M. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158–1161 (2005).
van der Meer, M. A., Johnson, A., Schmitzer-Torbert, N. C. & Redish, A. D. Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 67, 25–32 (2010).
Koechlin, E. & Summerfield, C. An information theoretical approach to prefrontal executive function. Trends Cogn. Sci. 11, 229–235 (2007).
Brunec, I. K. & Momennejad, I. Predictive representations in hippocampal and prefrontal hierarchies. Preprint at bioRxiv https://doi.org/10.1101/786434 (2019).
Mahadevan, S. & Maggioni, M. Proto-value Functions: a Laplacian framework for learning representation and control in Markov decision processes. J. Mach. Learn. Res. 8, 2169–2231 (2007).
Machado, M. C., Bellemare, M. G. & Bowling, M. Count-based exploration with the successor representation. in Proceedings of the AAAI Conference on Artificial Intelligence 34, 5125–5133 (2020).
Schapiro, A. C., Turk-Browne, N. B., Botvinick, M. M. & Norman, K. A. Complementary learning systems within the hippocampus: a neural network modelling approach to reconciling episodic memory with statistical learning. Philos. Trans. R. Soc. Lond. B Biol. Sci. https://doi.org/10.1098/rstb.2016.0049 (2017)
Whittington, J. C. R. et al. The Tolman–Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation. Cell 183, 1249–1263.e23 (2020).
Piray, P. & Daw, N. D. Linear reinforcement learning: flexible reuse of computation in planning, grid fields, and cognitive control. Preprint at bioRxiv https://doi.org/10.1101/856849 (2021).
Russo, E. & Durstewitz, D. Cell assemblies at multiple time scales with arbitrary lag constellations. eLife https://doi.org/10.7554/eLife.19428 (2017).
Durstewitz, D. A state space approach for piecewise-linear recurrent neural networks for identifying computational dynamics from neural measurements. PLoS Comput. Biol. 13, e1005542 (2017).
Pipa, G., Wheeler, D. W., Singer, W. & Nikolic, D. NeuroXidence: reliable and efficient analysis of an excess or deficiency of joint-spike events. J. Comput. Neurosci. 25, 64–88 (2008).
Benchenane, K. et al. Coherent theta oscillations and reorganization of spike timing in the hippocampal–prefrontal network upon learning. Neuron 66, 921–936 (2010).
Quaglio, P., Yegenoglu, A., Torre, E., Endres, D. M. & Grun, S. Detection and evaluation of spatio-temporal spike patterns in massively parallel spike train data with SPADE. Front. Comput. Neurosci. 11, 41 (2017).
Grossberger, L., Battaglia, F. P. & Vinck, M. Unsupervised clustering of temporal patterns in high-dimensional neuronal ensembles using a novel dissimilarity measure. PLoS Comput. Biol. 14, e1006283 (2018).
van de Ven, G. M., Trouche, S., McNamara, C. G., Allen, K. & Dupret, D. Hippocampal offline reactivation consolidates recently formed cell assembly patterns during sharp wave-ripples. Neuron 92, 968–974 (2016).
Gridchyn, I., Schoenenberger, P., O’Neill, J. & Csicsvari, J. Assembly-specific disruption of hippocampal replay leads to selective memory deficit. Neuron 106, 291–300.e6 (2020).
Oettl, L. L. et al. Phasic dopamine reinforces distinct striatal stimulus encoding in the olfactory tubercle driving dopaminergic reward prediction. Nat. Commun. 11, 3460 (2020).
Gomperts, S. N., Kloosterman, F. & Wilson, M. A. VTA neurons coordinate with the hippocampal reactivation of spatial experience. eLife https://doi.org/10.7554/eLife.05360 (2015).
Kurth-Nelson, Z. & Redish, A. D. Don’t let me do that!—models of precommitment. Front. Neurosci. 6, 138 (2012).
Kolling, N., Scholl, J., Chekroud, A., Trier, H. A. & Rushworth, M. F. S. Prospection, perseverance, and insight in sequential behavior. Neuron 99, 1069–1082.e7 (2018).
Goel, V. & Grafman, J. Are the frontal lobes implicated in ‘planning’ functions? Interpreting data from the Tower of Hanoi. Neuropsychologia 33, 623–642 (1995).
Burgess, P. W. Strategy application disorder: the role of the frontal lobes in human multitasking. Psychol. Res. 63, 279–288 (2000).
Holroyd, C. B., Ribas-Fernandes, J. J. F., Shahnazian, D., Silvetti, M. & Verguts, T. Human midcingulate cortex encodes distributed representations of task progress. Proc. Natl Acad. Sci. USA 115, 6398–6403 (2018).
Averbeck, B. B., Sohn, J. W. & Lee, D. Activity in prefrontal cortex during dynamic selection of action sequences. Nat. Neurosci. 9, 276–282 (2006).
Collins, A. & Koechlin, E. Reasoning, learning, and creativity: frontal lobe function and human decision-making. PLoS Biol. 10, e1001293 (2012).
Donoso, M., Collins, A. G. & Koechlin, E. Human cognition. Foundations of human reasoning in the prefrontal cortex. Science 344, 1481–1486 (2014).
Juechems, K. et al. A network for computing value equilibrium in the human medial prefrontal cortex. Neuron 101, 977–987.e3 (2019).
Huys, Q. J. et al. Bonsai trees in your head: how the pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput. Biol. 8, e1002410 (2012).
Lally, N. et al. The neural basis of aversive pavlovian guidance during planning. J. Neurosci. 37, 10215–10229 (2017).
Huys, Q. J. et al. Interplay of approximate planning strategies. Proc. Natl Acad. Sci. USA 112, 3098–3103 (2015).
Keramati, M., Smittenaar, P., Dolan, R. J. & Dayan, P. Adaptive integration of habits into depth-limited planning defines a habitual-goal-directed spectrum. Proc. Natl Acad. Sci. USA 113, 12868–12873 (2016).
Gittins, J. C. Bandit processes and dynamic allocation indices. J. R. Stat. Soc.: Ser. B (Methodol.) 41, 148–164 (1979).
Russo, D. J., Roy, B. V., Kazerouni, A., Osband, I. & Wen, Z. A tutorial on Thompson sampling. Found. Trends Mach. Learn. 11, 1–96 (2018).
MacIver, M. A., Patankar, N. A. & Shirgaonkar, A. A. Energy–information trade-offs between movement and sensing. PLoS Comput. Biol. https://doi.org/10.1371/journal.pcbi.1000769 (2010).
Chen, C., Murphey, T. D. & MacIver, M. A. Tuning movement for sensing in an uncertain world. eLife 9, e52371 (2020).
Khamassi, M., Quilodran, R., Enel, P., Dominey, P. F. & Procyk, E. Behavioral regulation and the modulation of information coding in the lateral prefrontal and cingulate cortex. Cereb. Cortex 25, 3197–3218 (2015).
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A. & Cohen, J. D. Humans use directed and random exploration to solve the explore–exploit dilemma. J. Exp. Psychol. Gen. 143, 2074–2081 (2014).
Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).
Boorman, E. D., Behrens, T. E., Woolrich, M. W. & Rushworth, M. F. How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 62, 733–743 (2009).
Blanchard, T. C. & Gershman, S. J. Pure correlates of exploration and exploitation in the human brain. Cogn. Affect. Behav. Neurosci. 18, 117–126 (2018).
Badre, D., Doll, B. B., Long, N. M. & Frank, M. J. Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration. Neuron 73, 595–607 (2012).
Zajkowski, W. K., Kossut, M. & Wilson, R. C. A causal role for right frontopolar cortex in directed, but not random, exploration. eLife https://doi.org/10.7554/eLife.27430 (2017).
Warren, C. M. et al. The effect of atomoxetine on random and directed exploration in humans. PLoS ONE 12, e0176034 (2017).
Tervo, D. G. R. et al. Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. Cell 159, 21–32 (2014).
Blanchard, T. C., Hayden, B. Y. & Bromberg-Martin, E. S. Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity. Neuron 85, 602–614 (2015).
White, J. K. et al. A neural network for information seeking. Nat. Commun. 10, 5168 (2019).
Hunt, L. T. et al. Triple dissociation of attention and decision computations across prefrontal cortex. Nat. Neurosci. 21, 1471–1481 (2018).
Ayala-Orozco, B. et al. Lévy walk patterns in the foraging movements of spider monkeys (Ateles geoffroyi). Behav. Ecol. Sociobiol. 55, 223–230 (2004).
Sims, D. W. et al. Scaling laws of marine predator search behaviour. Nature 451, 1098–1102 (2008).
Viswanathan, G. M., Raposo, E. P. & da Luz, M. G. E. Lévy flights and superdiffusion in the context of biological encounters and random searches. Phys. Life Rev. 5, 133–150 (2008).
McNamee, D. C., Stachenfeld, K. L., Botvinick, M. M. & Gershman, S. J. Flexible modulation of sequence generation in the entorhinal–hippocampal system. Nat. Neurosci. 24, 851–862 (2021).
Stella, F., Baracskay, P., O’Neill, J. & Csicsvari, J. Hippocampal reactivation of random trajectories resembling brownian diffusion. Neuron 102, 450–461.e7 (2019).
Wosniack, M. E., Santos, M. C., Raposo, E. P., Viswanathan, G. M. & da Luz, M. G. E. The evolutionary origins of Levy walk foraging. PLoS Comput. Biol. 13, e1005774 (2017).
Skaggs, W. E., McNaughton, B. L., Wilson, M. A. & Barnes, C. A. Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences. Hippocampus 6, 149–172 (1996).
Fiebelkorn, I. C., Pinsk, M. A. & Kastner, S. A dynamic interplay within the frontoparietal network underlies rhythmic spatial attention. Neuron 99, 842–853.e8 (2018).
Kleinfeld, D., Deschenes, M. & Ulanovsky, N. Whisking, sniffing, and the hippocampal theta-rhythm: a tale of two oscillators. PLoS Biol. 14, e1002385 (2016).
Stewart, N., Hermens, F. & Matthews, W. J. Eye movements in risky choice. J. Behav. Decis. Mak. 29, 116–136 (2016).
Hunt, L. T., Rutledge, R. B., Malalasekera, W. M., Kennerley, S. W. & Dolan, R. J. Approach-induced biases in human information sampling. PLoS Biol. 14, e2000638 (2016).
Kobayashi, K., Ravaioli, S., Baranes, A., Woodford, M. & Gottlieb, J. Diverse motives for human curiosity. Nat. Hum. Behav. 3, 587–595 (2019).
Hayden, B. Y. Economic choice: the foraging perspective. Curr. Opin. Behav. Sci. 24, 1–6 (2018).
Sweis, B. M. et al. Sensitivity to ‘sunk costs’ in mice, rats, and humans. Science 361, 178–181 (2018).
Juechems, K. & Summerfield, C. Where does value come from? Trends Cogn. Sci. 23, 836–850 (2019).
Nilsson, D. E. Evolution: an irresistibly clear view of land. Curr. Biol. 27, R715–R717 (2017).
Acknowledgements
L.T.H. was supported by a Henry Dale Fellowship from the Royal Society and Wellcome Trust (208789/Z/17/Z). N.D.D. was supported by NIDA R01DA038891 and NSF IIS-1822571, both part of the CRCNS program. M.A.M. was funded by NSF Brain Initiative ECCS-1835389. E.R. was supported by a Ch. and H. Schaller Foundation and the Boehringer Ingelheim Foundation grant ‘Complex Systems’. E.P. and C.R.E.W. are supported by the French National Research Agency within the framework of the labex CORTEX ANR-11-LABX-0042 of Université de Lyon, and grant ANR-19-CE37-0008 NORAD and ANR-18-CE37-0016-01 PREDYCT. E.P. is employed by the Centre National de la Recherche Scientifique. J.S. is funded by a MRC Skills Development Fellowship (MR/N014448/1). N.K. is funded by a fellowship from the BBSRC (BB/R010803/1).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature Neuroscience thanks Daeyeol Lee and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Hunt, L.T., Daw, N.D., Kaanders, P. et al. Formalizing planning and information search in naturalistic decision-making. Nat Neurosci 24, 1051–1064 (2021). https://doi.org/10.1038/s41593-021-00866-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41593-021-00866-w