Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Lateral habenula neurons signal errors in the prediction of reward information

Nature Neuroscience volume 14, pages 1209–1216 (2011)Cite this article

Subjects

An Erratum to this article was published on 23 November 2011

This article has been updated

Abstract

Humans and animals have the ability to predict future events, which they cultivate by continuously searching their environment for sources of predictive information. However, little is known about the neural systems that motivate this behavior. We hypothesized that information-seeking is assigned value by the same circuits that support reward-seeking, such that neural signals encoding reward prediction errors (RPEs) include analogous information prediction errors (IPEs). To test this, we recorded from neurons in the lateral habenula, a nucleus that encodes RPEs, while monkeys chose between cues that provided different chances to view information about upcoming rewards. We found that a subpopulation of lateral habenula neurons transmitted signals resembling IPEs, responding when reward information was unexpectedly cued, delivered or denied. These signals evaluated information sources reliably, even when the monkey's decisions did not. These neurons could provide a common instructive signal for reward-seeking and information-seeking behavior.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Behavioral preference to view informative reward cues.
Figure 2: Lateral habenula neurons transmit an inverted RPE signal.
Figure 3: Lateral habenula activity related to IPEs evoked by the information-predictive targets.
Figure 4: Information-related signals are strongest in a subpopulation of neurons.
Figure 5: Lateral habenula activity related to negative IPEs evoked by denial of reward information.
Figure 6: Lateral habenula activity related to positive IPEs evoked by delivery of reward information.
Figure 7: Joint coding of IPEs and cRPEs in single neurons.
Figure 8: Lateral habenula and dopamine neurons signal information probability reliably despite variable decisions.

Similar content being viewed by others

Change history

  • 22 September 2011

    In the version of this article initially published, the colored circles and horizontal bars above the x axis in the top panels of Figure 8, which indicate the median and central 90% range of each distribution, were misaligned. The error has been corrected in the HTML and PDF versions of the article.

References

  1. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    Article  CAS  Google Scholar 

  2. Rescorla, R.A. & Wagner, A.R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. in Classical Conditioning II: Current Research and Theory (eds. Black, A.H. & Prokasy, W.F.) 64–99 (Appleton Century Crofts, New York, 1972).

  3. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

    Article  CAS  Google Scholar 

  4. Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).

    Article  CAS  Google Scholar 

  5. Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647–656 (2007).

    Article  CAS  Google Scholar 

  6. Kim, H., Sul, J.H., Huh, N., Lee, D. & Jung, M.W. Role of striatum in updating values of chosen actions. J. Neurosci. 29, 14701–14712 (2009).

    Article  CAS  Google Scholar 

  7. Oyama, K., Hernadi, I., Iijima, T. & Tsutsui, K. Reward prediction error coding in dorsal striatal neurons. J. Neurosci. 30, 11447–11457 (2010).

    Article  CAS  Google Scholar 

  8. Hong, S. & Hikosaka, O. The globus pallidus sends reward-related signals to the lateral habenula. Neuron 60, 720–729 (2008).

    Article  CAS  Google Scholar 

  9. Matsumoto, M. & Hikosaka, O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447, 1111–1115 (2007).

    Article  CAS  Google Scholar 

  10. Matsumoto, M. & Hikosaka, O. Representation of negative motivational value in the primate lateral habenula. Nat. Neurosci. 12, 77–84 (2009).

    Article  CAS  Google Scholar 

  11. Holroyd, C.B. & Coles, M.G. The neural basis of human error processing: reinforcement learning, dopamine and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).

    Article  Google Scholar 

  12. O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).

    Article  CAS  Google Scholar 

  13. McClure, S.M., Berns, G.S. & Montague, P.R. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339–346 (2003).

    Article  CAS  Google Scholar 

  14. Salas, R., Baldwin, P., de Biasi, M. & Montague, P.R. BOLD responses to negative reward prediction errors in human habenula. Front. Hum. Neurosci. 4, 36 (2010).

    PubMed  PubMed Central  Google Scholar 

  15. Chew, S.H. & Ho, J.L. Hope: an empirical study of attitude toward the timing of uncertainty resolution. J. Risk Uncertain. 8, 267–288 (1994).

    Article  Google Scholar 

  16. Eliaz, K. & Schotter, A. Experimental testing of intrinsic preferences for noninstrumental information. Am. Econ. Rev. 97, 166–169 (2007).

    Article  Google Scholar 

  17. Luhmann, C.C., Chun, M.M., Yi, D.-J., Lee, D. & Wang, X.-J. Neural dissociation of delay and uncertainty in inter-temporal choice. J. Neurosci. 28, 14459–14466 (2008).

    Article  CAS  Google Scholar 

  18. Prokasy, W.F. Jr. The acquisition of observing responses in the absence of differential external reinforcement. J. Comp. Physiol. Psychol. 49, 131–134 (1956).

    Article  Google Scholar 

  19. Fantino, E. Conditioned reinforcement: choice and information. in Handbook of Operant Behavior (eds. Honig, W.K. & Staddon, J.E.R.) (Prentice Hall, Englewood Cliffs, New Jersey, 1977).

  20. Dinsmoor, J.A. Observing and conditioned reinforcement. Behav. Brain Sci. 6, 693–728 (1983).

    Article  Google Scholar 

  21. Daly, H.B. Preference for unpredictability is reversed when unpredictable nonreward is aversive: procedures, data, and theories of appetitive observing response acquisition. in Learning and Memory: The Behavioral and Biological Substrates (eds. Gormezano, I. & Wasserman, E.A.) 81–104 (L.E. Associates, 1992).

  22. Bromberg-Martin, E.S. & Hikosaka, O. Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron 63, 119–126 (2009).

    Article  CAS  Google Scholar 

  23. Ward, E. Acquisition and extinction of the observing response as a function of stimulus predictive validity. Psychon. Sci. 24, 139–141 (1971).

    Article  Google Scholar 

  24. Hayden, B.Y., Heilbronner, S.R. & Platt, M.L. Ambiguity aversion in rhesus macaques. Front. Neurosci. 4, 166 (2010).

    Article  Google Scholar 

  25. Kreps, D.M. & Porteus, E.L. Temporal resolution of uncertainty and dynamic choice theory. Econometrica 46, 185–200 (1978).

    Article  Google Scholar 

  26. Wyckoff, L.B. Jr. The role of observing responses in discrimination learning. Psychol. Rev. 59, 431–442 (1952).

    Article  Google Scholar 

  27. Hikosaka, O. The habenula: from stress evasion to value-based decision-making. Nat. Rev. Neurosci. 11, 503–513 (2010).

    Article  CAS  Google Scholar 

  28. Christoph, G.R., Leonzio, R.J. & Wilcox, K.S. Stimulation of the lateral habenula inhibits dopamine-containing neurons in the substantia nigra and ventral tegmental area of the rat. J. Neurosci. 6, 613–619 (1986).

    Article  CAS  Google Scholar 

  29. Ji, H. & Shepard, P.D. Lateral habenula stimulation inhibits rat midbrain dopamine neurons through a GABA(A) receptor–mediated mechanism. J. Neurosci. 27, 6923–6930 (2007).

    Article  CAS  Google Scholar 

  30. Shumake, J., Ilango, A., Scheich, H., Wetzel, W. & Ohl, F.W. Differential neuromodulation of acquisition and retrieval of avoidance learning by the lateral habenula and ventral tegmental area. J. Neurosci. 30, 5876–5883 (2010).

    Article  CAS  Google Scholar 

  31. Friedman, A. et al. Electrical stimulation of the lateral habenula produces an inhibitory effect on sucrose self-administration. Neuropharmacology 60, 381–387 (2010).

    Article  Google Scholar 

  32. Doya, K. Metalearning and neuromodulation. Neural Netw. 15, 495–506 (2002).

    Article  Google Scholar 

  33. Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).

    Article  CAS  Google Scholar 

  34. Sutherland, R.J. & Nakajima, S. Self-stimulation of the habenular complex in the rat. J. Comp. Physiol. Psychol. 95, 781–791 (1981).

    Article  CAS  Google Scholar 

  35. Lecourtier, L., Defrancesco, A. & Moghaddam, B. Differential tonic influence of lateral habenula on prefrontal cortex and nucleus accumbens dopamine release. Eur. J. Neurosci. 27, 1755–1762 (2008).

    Article  Google Scholar 

  36. Ikemoto, S. Brain reward circuitry beyond the mesolimbic dopamine system: a neurobiological theory. Neurosci. Biobehav. Rev. 35, 129–150 (2010).

    Article  CAS  Google Scholar 

  37. Matsumoto, M. & Hikosaka, O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841 (2009).

    Article  CAS  Google Scholar 

  38. Joshua, M., Adler, A. & Bergman, H. The dynamics of dopamine in control of motor behavior. Curr. Opin. Neurobiol. 19, 615–620 (2009).

    Article  CAS  Google Scholar 

  39. Bromberg-Martin, E.S., Matsumoto, M. & Hikosaka, O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68, 815–834 (2010).

    Article  CAS  Google Scholar 

  40. Caplin, A. & Leahy, J. Psychological expected utility theory and anticipatory feelings. Q. J. Econ. 116, 55–79 (2001).

    Article  Google Scholar 

  41. Butko, N.J. & Movellan, J.R. Infomax control of eye movements. IEEE Trans. Auto. Ment. Dev. 2, 91–107 (2010).

    Article  Google Scholar 

  42. Wyckoff, L.B. Jr. Toward a quantitative theory of secondary reinforcement. Psychol. Rev. 66, 68–78 (1959).

    Article  CAS  Google Scholar 

  43. Beierholm, U.R. & Dayan, P. Pavlovian-instrumental interaction in 'observing behavior'. PLOS Comput. Biol. 6, 9 (2010).

    Article  Google Scholar 

  44. Bromberg-Martin, E.S., Matsumoto, M., Nakahara, H. & Hikosaka, O. Multiple timescales of memory in lateral habenula and dopamine neurons. Neuron 67, 499–510 (2010).

    Article  CAS  Google Scholar 

  45. Nakamura, K., Matsumoto, M. & Hikosaka, O. Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus. J. Neurosci. 28, 5331–5343 (2008).

    Article  CAS  Google Scholar 

  46. Ranade, S.P. & Mainen, Z.F. Transient firing of dorsal raphe neurons encodes diverse and specific sensory, motor, and reward events. J. Neurophysiol. 102, 3026–3037 (2009).

    Article  Google Scholar 

  47. Redgrave, P. & Gurney, K. The short-latency dopamine signal: a role in discovering novel actions? Nat. Rev. Neurosci. 7, 967–975 (2006).

    Article  CAS  Google Scholar 

  48. Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057–1063 (2006).

    Article  CAS  Google Scholar 

  49. Roesch, M.R., Calu, D.J. & Schoenbaum, G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 10, 1615–1624 (2007).

    Article  CAS  Google Scholar 

  50. Houk, J.C., Adams, J.L. & Barto, A.G. A model of how the basal ganglia generate and use neural signals that predict reinforcement. in Models of Information Processing in the Basal Ganglia (eds. Houk, J.C., Davis, J.L. & Beiser, D.G.) 249–274 (MIT Press, Cambridge, Massachusetts, 1995).

Download references

Acknowledgements

We thank M. Matsumoto, S. Hong, I. Monosov, M. Yasuda, S. Yamamoto, Y. Tachibana, H. Kim and D. Lee for valuable discussions. This work was supported by the intramural research program at the National Eye Institute.

Author information

Authors and Affiliations

  1. Laboratory of Sensorimotor Research, National Eye Institute, US National Institutes of Health, Bethesda, Maryland, USA

    Ethan S Bromberg-Martin & Okihide Hikosaka

Authors
  1. Ethan S Bromberg-Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Okihide Hikosaka
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

E.S.B.-M. designed and performed the experiments and analyzed the data. O.H. supported these processes. E.S.B.-M. and O.H. discussed the results and wrote the manuscript.

Corresponding author

Correspondence to Ethan S Bromberg-Martin.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–11 (PDF 1901 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bromberg-Martin, E., Hikosaka, O. Lateral habenula neurons signal errors in the prediction of reward information. Nat Neurosci 14, 1209–1216 (2011). https://doi.org/10.1038/nn.2902

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2902

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing