Associative learning of social value

Article metrics

Abstract

Our decisions are guided by information learnt from our environment. This information may come via personal experiences of reward, but also from the behaviour of social partners1,2. Social learning is widely held to be distinct from other forms of learning in its mechanism and neural implementation; it is often assumed to compete with simpler mechanisms, such as reward-based associative learning, to drive behaviour3. Recently, neural signals have been observed during social exchange reminiscent of signals seen in studies of associative learning4. Here we demonstrate that social information may be acquired using the same associative processes assumed to underlie reward-based learning. We find that key computational variables for learning in the social and reward domains are processed in a similar fashion, but in parallel neural processing streams. Two neighbouring divisions of the anterior cingulate cortex were central to learning about social and reward-based information, and for determining the extent to which each source of information guides behaviour. When making a decision, however, the information learnt using these parallel streams was combined within ventromedial prefrontal cortex. These findings suggest that human social valuation can be realized by means of the same associative processes previously established for learning other, simpler, features of the environment.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Experimental task and behavioural findings.
Figure 2: Predictions and prediction errors in social and non-social domains.
Figure 3: Agency-specific learning rates dissociate in the ACC.
Figure 4: Combination of expected value of chosen option in VMPFC.

References

  1. 1

    Fehr, E. & Fischbacher, U. The nature of human altruism. Nature 425, 785–791 (2003)

  2. 2

    Maynard Smith, J. Evolution and the Theory of Games (Cambridge Univ. Press, 1982)

  3. 3

    Delgado, M. R., Frank, R. H. & Phelps, E. A. Perceptions of moral character modulate the neural systems of reward during the trust game. Nature Neurosci. 8, 1611–1618 (2005)

  4. 4

    King-Casas, B. et al. Getting to know you: reputation and trust in a two-person economic exchange. Science 308, 78–83 (2005)

  5. 5

    Rilling, J. et al. A neural basis for social cooperation. Neuron 35, 395–405 (2002)

  6. 6

    Gallagher, H. L., Jack, A. I., Roepstorff, A. & Frith, C. D. Imaging the intentional stance in a competitive game. Neuroimage 16, 814–821 (2002)

  7. 7

    Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 1998)

  8. 8

    Behrens, T. E., Woolrich, M. W., Walton, M. E. & Rushworth, M. F. Learning the value of information in an uncertain world. Nature Neurosci. 10, 1214–1221 (2007)

  9. 9

    Courville, A. C., Daw, N. D. & Touretzky, D. S. Bayesian theories of conditioning in a changing world. Trends Cogn. Sci. 10, 294–300 (2006)

  10. 10

    Dayan, P., Kakade, S. & Montague, P. R. Learning and selective attention. Nature Neurosci. 3 (Suppl.). 1218–1223 (2000)

  11. 11

    O’Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004)

  12. 12

    Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)

  13. 13

    Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001)

  14. 14

    Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell activity signaling prediction errors of action values. Nature Neurosci. 10, 647–656 (2007)

  15. 15

    Tanaka, S. C. et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neurosci. 7, 887–893 (2004)

  16. 16

    Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006)

  17. 17

    Bayer, H. M. & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005)

  18. 18

    Haruno, M. & Kawato, M. Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. J. Neurophysiol. 95, 948–959 (2006)

  19. 19

    D’Ardenne, K., McClure, S. M., Nystrom, L. E. & Cohen, J. D. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science 319, 1264–1267 (2008)

  20. 20

    Kennerley, S. W., Walton, M. E., Behrens, T. E., Buckley, M. J. & Rushworth, M. F. Optimal decision making and the anterior cingulate cortex. Nature Neurosci. 9, 940–947 (2006)

  21. 21

    Deaner, R. O., Khera, A. V. & Platt, M. L. Monkeys pay per view: adaptive valuation of social images by rhesus macaques. Curr. Biol. 15, 543–548 (2005)

  22. 22

    Shepherd, S. V., Deaner, R. O. & Platt, M. L. Social status gates social attention in monkeys. Curr. Biol. 16, R119–R120 (2006)

  23. 23

    Rudebeck, P. H., Buckley, M. J., Walton, M. E. & Rushworth, M. F. A role for the macaque anterior cingulate gyrus in social valuation. Science 313, 1310–1312 (2006)

  24. 24

    O’Doherty, J. P. Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr. Opin. Neurobiol. 14, 769–776 (2004)

  25. 25

    Kable, J. W. & Glimcher, P. W. The neural correlates of subjective value during intertemporal choice. Nature Neurosci. 10, 1625–1633 (2007)

  26. 26

    Amodio, D. M. & Frith, C. D. Meeting of minds: the medial frontal cortex and social cognition. Nature Rev. Neurosci. 7, 268–277 (2006)

  27. 27

    Allison, T., Puce, A. & McCarthy, G. Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267–278 (2000)

  28. 28

    Castelli, F., Frith, C., Happe, F. & Frith, U. Autism, Asperger syndrome and brain mechanisms for the attribution of mental states to animated shapes. Brain 125, 1839–1849 (2002)

  29. 29

    Van Hoesen, G. W., Morecraft, R. J. & Vogt, B. A. in Neurobiology of Cingulate Cortex and Limbic Thalamus (eds Vogt, B. A. & Gabriel, M.) (Birkhäuser, 1993)

  30. 30

    Plassmann, H., O’Doherty, J. & Rangel, A. Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J. Neurosci. 27, 9984–9988 (2007)

  31. 31

    Smith, S. M. et al. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 23 (Suppl. 1). S208–S219 (2004)

  32. 32

    Woolrich, M. W., Ripley, B. D., Brady, M. & Smith, S. M. Temporal autocorrelation in univariate linear modeling of FMRI data. Neuroimage 14, 1370–1386 (2001)

Download references

Acknowledgements

Acknowledgments We would like to acknowledge funding from the UK MRC (T.E.J.B., M.F.S.R.), the Wellcome Trust (L.T.H.) and the UK EPSRC (M.W.W.). We thank S. Knight for helping with data acquisition, and K. Watkins for help with figure preparation.

Author contributions All four authors contributed to generating the hypothesis and designing the experiment. Where specific roles can be assigned, L.T.H. collected the data, T.E.J.B. and L.T.H. analysed the data, T.E.J.B. and M.W.W. built the model, and M.F.S.R. supervised the project.

Author information

Correspondence to Timothy E. J. Behrens.

Supplementary information

Supplementary Information

This file contains Supplementary Notes and Data, Supplementary Figures S1-S3, Supplementary Figure 4 with Legends and Supplementary References. (PDF 1217 kb)

PowerPoint slides

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Behrens, T., Hunt, L., Woolrich, M. et al. Associative learning of social value. Nature 456, 245–249 (2008) doi:10.1038/nature07538

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.