Access

Letters to Nature

Nature 429, 664-667 (10 June 2004) | doi:10.1038/nature02581; Received 2 December 2003; Accepted 19 April 2004

Open Innovation Challenges

naturejobs

Temporal difference models describe higher-order learning in humans

Ben Seymour1, John P. O'Doherty1, Peter Dayan2, Martin Koltzenburg3, Anthony K. Jones4, Raymond J. Dolan1, Karl J. Friston1 & Richard S. Frackowiak1,5

  1. Wellcome Department of Imaging Neuroscience, 12 Queen Square, London WC1N 3BG, UK
  2. Gatsby Computational Neuroscience Unit, Alexandra House, 17 Queen Square, London WC1N 3AR, UK
  3. Institute of Child Health, University College London, 30 Guilford St, London WC1N 1EH, UK
  4. University of Manchester Rheumatic Diseases Centre, Hope Hospital, Manchester M6 8HD, UK
  5. Fondazione Santa Lucia, 00179 Rome, Italy

Correspondence to: Ben Seymour1 Email: bseymour@fil.ion.ucl.ac.uk

Top

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning1. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning2, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

MORE ARTICLES LIKE THIS

These links to content published by NPG are automatically generated.

NEWS AND VIEWS

The rhythms of learning

Nature Neuroscience News and Views (01 Jun 2009)

Ambiguity and anxiety: when a glass half full is empty

Nature Neuroscience News and Views (01 Jul 2007)