Article | Published:

Optimal decision making and the anterior cingulate cortex

Abstract

Learning the value of options in an uncertain environment is central to optimal decision making. The anterior cingulate cortex (ACC) has been implicated in using reinforcement information to control behavior. Here we demonstrate that the ACC's critical role in reinforcement-guided behavior is neither in detecting nor in correcting errors, but in guiding voluntary choices based on the history of actions and outcomes. ACC lesions did not impair the performance of monkeys (Macaca mulatta) immediately after errors, but made them unable to sustain rewarded responses in a reinforcement-guided choice task and to integrate risk and payoff in a dynamic foraging task. These data suggest that the ACC is essential for learning the value of actions.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

References

  1. 1

    McCoy, A.N. & Platt, M.L. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat. Neurosci. 8, 1220–1227 (2005).

  2. 2

    Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999).

  3. 3

    Sugrue, L.P., Corrado, G.S. & Newsome, W.T. Matching behavior and the representation of value in the parietal cortex. Science 304, 1782–1787 (2004).

  4. 4

    Murray, E.A., Bussey, T.J. & Wise, S.P. Role of prefrontal cortex in a network for arbitrary visuomotor mapping. Exp. Brain Res. 133, 114–129 (2000).

  5. 5

    Sutton, R.S. & Barto, A.G. Reinforcement Learning (MIT Press, Cambridge, Massachusetts, 1998).

  6. 6

    Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

  7. 7

    Schultz, W. Getting formal with dopamine and reward. Neuron 36, 241–263 (2002).

  8. 8

    Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y. & Hikosaka, O. Dopamine neurons can represent context-dependent prediction error. Neuron 41, 269–280 (2004).

  9. 9

    Satoh, T., Nakai, S., Sato, T. & Kimura, M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 23, 9913–9923 (2003).

  10. 10

    Paus, T. Primate anterior cingulate cortex: where motor control, drive and cognition interface. Nat. Rev. Neurosci. 2, 417–424 (2001).

  11. 11

    Rushworth, M.F., Walton, M.E., Kennerley, S.W. & Bannerman, D.M. Action sets and decisions in the medial frontal cortex. Trends Cogn. Sci. 8, 410–417 (2004).

  12. 12

    Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S. & Cohen, J.D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001).

  13. 13

    Matsumoto, K., Suzuki, W. & Tanaka, K. Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 301, 229–232 (2003).

  14. 14

    Walton, M.E., Devlin, J.T. & Rushworth, M.F. Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 7, 1259–1265 (2004).

  15. 15

    Hadland, K.A., Rushworth, M.F., Gaffan, D. & Passingham, R.E. The anterior cingulate and reward-guided selection of actions. J. Neurophysiol. 89, 1161–1164 (2003).

  16. 16

    Morecraft, R.J. & Van Hoesen, G.W. Convergence of limbic input to the cingulate motor cortex in the rhesus monkey. Brain Res. Bull. 45, 209–232 (1998).

  17. 17

    Amiez, C., Joseph, J.P. & Procyk, E. Anterior cingulate error-related activity is modulated by predicted reward. Eur. J. Neurosci. 21, 3447–3452 (2005).

  18. 18

    Shima, K. & Tanji, J. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282, 1335–1338 (1998).

  19. 19

    Ito, S., Stuphorn, V., Brown, J.W. & Schall, J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).

  20. 20

    Ullsperger, M. & von Cramon, D.Y. Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J. Neurosci. 23, 4308–4314 (2003).

  21. 21

    Gehring, W.J., Goss, B., Coles, M.G., Meyer, D.E & Donchin, E. A neural system for error detection and compensation. Psychol. Sci. 4, 385–390 (1993).

  22. 22

    Holroyd, C.B. & Coles, M.G. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).

  23. 23

    Brown, J.W. & Braver, T.S. Learned predictions of error likelihood in the anterior cingulate cortex. Science 307, 1118–1121 (2005).

  24. 24

    Fellows, L.K. & Farah, M.J. Is anterior cingulate cortex necessary for cognitive control? Brain 128, 788–796 (2005).

  25. 25

    Swick, D. & Turken, A.U. Dissociation between conflict detection and error monitoring in the human anterior cingulate cortex. Proc. Natl Acad. Sci. USA 99, 16354–16359 (2002).

  26. 26

    Rushworth, M.F., Hadland, K.A., Gaffan, D. & Passingham, R.E. The effect of cingulate cortex lesions on task switching and working memory. J. Cogn. Neurosci. 15, 338–353 (2003).

  27. 27

    Amiez, C., Joseph, J.P. & Procyk, E. Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex, published online 5 October 2005 (doi:10.1093/cercor/bhj046).

  28. 28

    Lau, B. & Glimcher, P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555–579 (2005).

  29. 29

    Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45–58 (2004).

  30. 30

    Herrnstein, R.J. The Matching Law: Papers in Psychology and Economics (eds. Rachlin, H. & Laibson, D.I.) (Harvard Univ. Press, Cambridge, Massachusetts, 1997).

  31. 31

    Bush, G. et al. Dorsal anterior cingulate cortex: a role in reward-based decision making. Proc. Natl Acad. Sci. USA 99, 523–528 (2002).

  32. 32

    Niki, H. & Watanabe, M. Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res. 171, 213–224 (1979).

  33. 33

    Genovesio, A., Brasted, P.J., Mitz, A.R. & Wise, S.P. Prefrontal cortex activity related to abstract response strategies. Neuron 47, 307–320 (2005).

  34. 34

    Williams, S.M. & Goldman-Rakic, P.S. Widespread origin of the primate mesofrontal dopamine system. Cereb. Cortex 8, 321–345 (1998).

  35. 35

    Murray, E.A., Davidson, M., Gaffan, D., Olton, D.S. & Suomi, S. Effects of fornix transection and cingulate cortical ablation on spatial memory in rhesus monkeys. Exp. Brain Res. 74, 173–186 (1989).

  36. 36

    Morris, R., Pandya, D.N. & Petrides, M. Fiber system linking the mid-dorsolateral frontal cortex with the retrosplenial/presubicular region in the rhesus monkey. J. Comp. Neurol. 407, 183–192 (1999).

  37. 37

    Aron, A.R., Robbins, T.W. & Poldrack, R.A. Inhibition and the right inferior frontal cortex. Trends Cogn. Sci. 8, 170–177 (2004).

  38. 38

    Roberts, A.C. & Wallis, J.D. Inhibitory control and affective processing in the prefrontal cortex: neuropsychological studies in the common marmoset. Cereb. Cortex 10, 252–262 (2000).

  39. 39

    Knutson, B., Taylor, J., Kaufman, M., Peterson, R. & Glover, G. Distributed neural representation of expected value. J. Neurosci. 25, 4806–4812 (2005).

  40. 40

    Shidara, M. & Richmond, B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296, 1709–1711 (2002).

  41. 41

    Wallis, J.D. & Miller, E.K. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci. 18, 2069–2081 (2003).

  42. 42

    Roesch, M.R. & Olson, C.R. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304, 307–310 (2004).

  43. 43

    Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337–1340 (2005).

  44. 44

    Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004).

  45. 45

    Nakamura, K., Roesch, M.R. & Olson, C.R. Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. J. Neurophysiol. 93, 884–908 (2005).

  46. 46

    Aston-Jones, G. & Cohen, J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450 (2005).

  47. 47

    Yu, A.J. & Dayan, P. Uncertainty, neuromodulation, and attention. Neuron 46, 681–692 (2005).

  48. 48

    He, S.Q., Dum, R.P. & Strick, P.L. Topographic organization of corticospinal projections from the frontal lobe: motor areas on the medial surface of the hemisphere. J. Neurosci. 15, 3284–3306 (1995).

  49. 49

    Bates, J.F. & Goldman-Rakic, P.S. Prefrontal connections of medial motor areas in the rhesus monkey. J. Comp. Neurol. 336, 211–228 (1993).

  50. 50

    Paxinos, G., Huang, X.F. & Toga, A.W. The Rhesus Monkey Brain in Stereotaxic Coordinates (Academic Press, San Diego, 1999).

Download references

Acknowledgements

We are grateful to D. Gaffan for advice and encouragement. Supported by the Medical Research Council. Additional support from the Clarendon Foundation (S.W.K.), the Wellcome Trust (M.E.W.) and the Royal Society (M.J.B and M.F.S.R.).

Author information

Competing interests

The authors declare no competing financial interests.

Correspondence to Steven W Kennerley.

Supplementary information

Supplementary Fig. 1

Number of trials to update to the other response after a task-imposed response switch in Experiment 1. (PDF 57 kb)

Supplementary Fig. 2

Inter-response time performance in Experiment 1A. (PDF 54 kb)

Supplementary Fig. 3

Performance for sustaining rewarded behavior following errors that occurred at a task-imposed switch trial or all other errors. (PDF 59 kb)

Supplementary Fig. 4

Relative contributions of the high and low reward-probability responses in determining ropt for the 0.4:0.1 action-reward ratio pairing in Experiment 2. (PDF 84 kb)

Supplementary Fig. 5

The average expected reward per response for each action-reward ratio pair in Experiment 2. (PDF 64 kb)

Supplementary Fig. 6

Group response ratio plot for the action-reward ratio 0.4:0.1. (PDF 79 kb)

Supplementary Methods (PDF 189 kb)

Rights and permissions

Reprints and Permissions

About this article

Further reading

Figure 1: Diagram of the macaque brain and overview of experiments 1 and 2.
Figure 2: Performance in experiment 1A.
Figure 3: Performance for sustaining rewarded behavior following an error in experiment 1.
Figure 4: Estimates of the influence of previous reward history on current choice, in experiment 1.
Figure 5: Postoperative performance in the matching task in experiment 2.
Figure 6: Schematic of a sample choice trial in experiment 3.
Figure 7: Percentage of high reward options selected for each of the three choice pairs across three preoperative (unfilled bars) and one postoperative (hatched bars) sessions.
Figure 8: Coronal sections showing the cingulate lesion in all three monkeys that received surgery in experiments 1–3.