Neuronal basis of sequential foraging decisions in a patchy environment

Article metrics

Abstract

Deciding when to leave a depleting resource to exploit another is a fundamental problem for all decision makers. The neuronal mechanisms mediating patch-leaving decisions remain unknown. We found that neurons in primate (Macaca mulatta) dorsal anterior cingulate cortex, an area that is linked to reward monitoring and executive control, encode a decision variable signaling the relative value of leaving a depleting resource for a new one. Neurons fired during each sequential decision to stay in a patch and, for each travel time, these responses reached a fixed threshold for patch-leaving. Longer travel times reduced the gain of neural responses for choosing to stay in a patch and increased the firing rate threshold mandating patch-leaving. These modulations more closely matched behavioral decisions than any single task variable. These findings portend an understanding of the neural basis of foraging decisions and endorse the unification of theoretical and experimental work in ecology and neuroscience.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Patch-leaving task.
Figure 2: Monkeys obey the marginal value theorem in a virtual patchy foraging task.
Figure 3: Firing rates of dACC neurons integrate patch residence time and travel time in computations occurring over multiple actions.
Figure 4: Firing rates of dACC neurons rise to a threshold associated with patch abandonment.
Figure 5: Travel time governs both neuronal response gain and threshold.

References

  1. 1

    Prins, H.H.T. Ecology and Behavior of the African Buffalo: Social Inequality and Decision Making (Chapman and Hall, London, 1996).

  2. 2

    Charnov, E.L. Optimal foraging, the marginal value theorem. Theor. Popul. Biol. 9, 129–136 (1976).

  3. 3

    Stephens, D.W. & Krebs, J.R. Foraging Theory (Princeton University Press, Princeton, NJ, 1986).

  4. 4

    Bendesky, A., Tsunozaki, M., Rockman, M.V., Kruglyak, L. & Bargmann, C.I. Catecholamine receptor polymorphisms affect decision-making in C. elegans. Nature 472, 313–318 (2011).

  5. 5

    Thompson, D. & Fedak, M.A. How long should a dive last? A simple model of foraging decisions by breath-hold divers in a patchy environment. Anim. Behav. 61, 287–296 (2001).

  6. 6

    McNickle, G.G. & Cahill, J.F. Jr . Plant root growth and the marginal value theorem. Proc. Natl. Acad. Sci. USA 106, 4747–4751 (2009).

  7. 7

    Smith, E.A. & Winterhalder, B. Evolutionary Ecology and Human Behavior (de Gruyer, New York, 1992).

  8. 8

    Agetsuma, N. Simulation of patch use by monkeys using operant conditioning. J. Ethology 16, 49–55 (1999).

  9. 9

    Gold, J.I. & Shadlen, M.N. The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–574 (2007).

  10. 10

    Gold, J.I. & Shadlen, M.N. Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36, 299–308 (2002).

  11. 11

    Hanes, D.P. & Schall, J.D. Neural control of voluntary movement initiation. Science 274, 427–430 (1996).

  12. 12

    Schall, J.D. On building a bridge between brain and behavior. Annu. Rev. Psychol. 55, 23–50 (2004).

  13. 13

    Carpenter, R.H.S. Movements of the Eyes (Pion, London, 1988).

  14. 14

    Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006).

  15. 15

    Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008).

  16. 16

    Holroyd, C.B. & Coles, M.G. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).

  17. 17

    Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).

  18. 18

    Quilodran, R., Rothe, M. & Procyk, E. Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57, 314–325 (2008).

  19. 19

    Ito, S., Stuphorn, V., Brown, J.W. & Schall, J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).

  20. 20

    Williams, Z.M., Bush, G., Rauch, S.L., Cosgrove, G.R. & Eskandar, E.N. Human anterior cingulate neurons and the integration of monetary reward with motor responses. Nat. Neurosci. 7, 1370–1375 (2004).

  21. 21

    Hayden, B.Y. & Platt, M.L. Neurons in anterior cingulate cortex multiplex information about reward and action. J. Neurosci. 30, 3339–3346 (2010).

  22. 22

    Hayden, B.Y., Pearson, J.M. & Platt, M.L. Fictive reward signals in the anterior cingulate cortex. Science 324, 948–950 (2009).

  23. 23

    Shima, K. & Tanji, J. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282, 1335–1338 (1998).

  24. 24

    Shidara, M. & Richmond, B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296, 1709–1711 (2002).

  25. 25

    Baler, R.D. & Volkow, N.D. Drug addiction: the neurobiology of disrupted self-control. Trends Mol. Med. 12, 559–566 (2006).

  26. 26

    Stern, E. et al. A functional neuroanatomy of tics in Tourette syndrome. Arch. Gen. Psychiatry 57, 741–748 (2000).

  27. 27

    Devinsky, O., Morrell, M.J. & Vogt, B.A. Contributions of anterior cingulate cortex to behavior. Brain 118, 279–306 (1995).

  28. 28

    Stephens, D.W. & Anderson, D. The adaptive value of preference for immediacy: when shortsighted rules have farsighted consequences. Behav. Ecol. 12, 330–339 (2001).

  29. 29

    Evans, T.A. & Beran, M.J. Delay of gratification and delay maintenance by rhesus macaques (Macaca mulatta). J. Gen. Psychol. 134, 199–216 (2007).

  30. 30

    Kim, S., Hwang, J. & Lee, D. Prefrontal coding of temporally discounted values during intertemporal choice. Neuron 59, 161–172 (2008).

  31. 31

    Louie, K. & Glimcher, P.W. Separating value from choice: delay discounting activity in the lateral intraparietal area. J. Neurosci. 30, 5498–5507 (2010).

  32. 32

    Kahneman, D., Knetsch, J.L. & Thaler, R.H. Anomalies: the endowment effect, loss aversion, and the status quo bias. J. Econ. Perspect. 5, 193–206 (1991).

  33. 33

    Mazur, J.E. An adjusting procedure for studying delayed reinforcement. in Quantitative Analyses of Behavior, vol 5. The Effect of Delay and Intervening Events on Reinforcement Value (eds. Commons, M.L., Mazur, J.E., Nevin, J.A. & Rachlin, H.) (Erlbaum, Mahway, New Jersey, 1987).

  34. 34

    Roitman, J.D. & Shadlen, M.N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002).

  35. 35

    Horwitz, G.D., Batista, A.P. & Newsome, W.T. Representation of an abstract perceptual decision in macaque superior colliculus. J. Neurophysiol. 91, 2281–2296 (2004).

  36. 36

    Tobler, P.N., Fiorillo, C.D. & Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642–1645 (2005).

  37. 37

    Yu, A.J. & Dayan, P. Uncertainty, neuromodulation, and attention. Neuron 46, 681–692 (2005).

  38. 38

    Marr, D.C. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (Freeman, New York, 1982).

  39. 39

    Latty, T. & Beekman, M. Food quality affects search strategy in the acellular slime mold, Physarum polycephalum. Behav. Ecol. 20, 1160–1167 (2009).

  40. 40

    Bonser, R., Wright, P.J., Bament, S. & Chukwu, U.O. Optimal patch use by foraging workers of Lasius fuliginosus, L. niger and Myrmica ruginodis. Ecol. Entomol. 23, 15–21 (1998).

  41. 41

    Wallis, J.D. & Kennerley, S.W. Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191–198 (2010).

  42. 42

    Sallet, J. et al. Expectations, gains, and losses in the anterior cingulate cortex. Cogn. Affect. Behav. Neurosci. 7, 327–336 (2007).

  43. 43

    Kennerley, S.W., Dahmubed, A.F., Lara, A.H. & Wallis, J.D. Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162–1178 (2008).

  44. 44

    Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647–656 (2007).

  45. 45

    Amiez, C., Joseph, J.P. & Procyk, E. Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex 16, 1040–1055 (2006).

  46. 46

    Hayden, B.Y., Heilbronner, S.R., Pearson, J.M. & Platt, M.L. Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J. Neurosci. 31, 4178–4187 (2011).

  47. 47

    Stephens, D.W., Brown, J.S. & Ydenberg, R.C. Foraging: Behavior and Ecology (University of Chicago Press, Chicago, 2007).

  48. 48

    Wilson, E.O. Consilience: The Unity of Knowledge (Knopf, New York, 1998).

  49. 49

    Brainard, D.H. The Psychophysics Toolbox. Spat. Vis. 10, 433–436 (1997).

  50. 50

    Cornelissen, F.W., Peters, E. & Palmer, J. The Eyelink Toolbox: eye tracking with MATLAB and the Psychophysics Toolbox. Behav. Res. Methods Instrum. Comput. 34, 613–617 (2002).

Download references

Acknowledgements

We thank S. Heilbronner for comments on design, analysis and writing. This research was supported by US National Institutes of Health grant R01EY013496 (M.L.P.), a Fellowship from the Tourette Syndrome Association (B.Y.H.) and US National Institutes of Health grant K99 DA027718-01 (B.Y.H.).

Author information

B.Y.H. designed the experiment and collected the data. B.Y.H. and J.M.P. contributed to data analysis. B.Y.H., J.M.P. and M.L.P. wrote the manuscript.

Correspondence to Benjamin Y Hayden.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–8 and Supplementary Refereneces (PDF 4065 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Hayden, B., Pearson, J. & Platt, M. Neuronal basis of sequential foraging decisions in a patchy environment. Nat Neurosci 14, 933–939 (2011) doi:10.1038/nn.2856

Download citation

Further reading