Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Rational regulation of learning dynamics by pupil-linked arousal systems

Nature Neuroscience volume 15, pages 1040–1046 (2012)Cite this article

Subjects

This article has been updated

Abstract

The ability to make inferences about the current state of a dynamic process requires ongoing assessments of the stability and reliability of data generated by that process. We found that these assessments, as defined by a normative model, were reflected in nonluminance-mediated changes in pupil diameter of human subjects performing a predictive-inference task. Brief changes in pupil diameter reflected assessed instabilities in a process that generated noisy data. Baseline pupil diameter reflected the reliability with which recent data indicate the current state of the data-generating process and individual differences in expectations about the rate of instabilities. Together these pupil metrics predicted the influence of new data on subsequent inferences. Moreover, a task- and luminance-independent manipulation of pupil diameter predictably altered the influence of new data. Thus, pupil-linked arousal systems can help to regulate the influence of incoming data on existing beliefs in a dynamic environment.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Predictive-inference task sequence and pupillometry.
Figure 2: Task performance.
Figure 3: Reduced Bayesian model.
Figure 4: Relationship between pupil change and change-point probability.
Figure 5: Relationship between pupil diameter and relative uncertainty.
Figure 6: Individual differences in learning rate, hazard rate and pupil diameter.
Figure 7: Pupil metrics predict learning rate.
Figure 8: Effects of the pupil manipulation.

Similar content being viewed by others

Change history

  • 17 January 2017

    In the version of this article initially published, equation (7) was wrong. The error has been corrected in the HTML and PDF versions of the article.

References

  1. Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).

    Article  CAS  PubMed  Google Scholar 

  2. Nassar, M.R., Wilson, R.C., Heasly, B. & Gold, J.I. An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J. Neurosci. 30, 12366–12378 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Yu, A.J. & Dayan, P. Uncertainty, neuromodulation, and attention. Neuron 46, 681–692 (2005).

    Article  CAS  PubMed  Google Scholar 

  4. Nieuwenhuis, S., De Geus, E.J. & Aston-Jones, G. The anatomical and functional relationship between the P3 and autonomic components of the orienting response. Psychophysiology 2, 162–175 (2011).

    Article  Google Scholar 

  5. Aston-Jones, G. & Cohen, J.D. An integrative theory of locus coeruleus–norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450 (2005).

    Article  CAS  PubMed  Google Scholar 

  6. Jepma, M. & Nieuwenhuis, S. Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory. J. Cogn. Neurosci. 23, 1587–1596 (2011).

    Article  PubMed  Google Scholar 

  7. Gilzenrat, M.S., Nieuwenhuis, S., Jepma, M. & Cohen, J.D. Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn. Affect. Behav. Neurosci. 10, 252–269 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  8. Krugman, H.E. Some applications of pupil measurement. J. Mark. Res. 1, 15–19 (1964).

    Article  Google Scholar 

  9. Granholm, E. & Steinhauer, S.R. Pupillometric measures of cognitive and emotional processes. Int. J. Psychophysiol. 52, 1–6 (2004).

    Article  PubMed  Google Scholar 

  10. Schmidt, H.S. & Fortin, L.D. Electronic pupillography in disorders of arousal. in Sleeping and Waking Disorders: Indication and Technique (ed. Guilleminault, C.) 127–143 (Addison-Wesley, 1982).

  11. Kahneman, D. & Beatty, J. Pupil diameter and load on memory. Science 154, 1583–1585 (1966).

    Article  CAS  PubMed  Google Scholar 

  12. Richer, F. & Beatty, J. Contrasting effects of response uncertainty on the task-evoked pupillary response and reaction time. Psychophysiology 24, 258–262 (1987).

    Article  CAS  PubMed  Google Scholar 

  13. Hakerem, G., Sutton, S. & Zubin, J. Pupillary reactions to light in schizophrenic patients and normals. Ann. NY Acad. Sci. 105, 820–831 (1964).

    Article  CAS  PubMed  Google Scholar 

  14. Einhäuser, W., Stout, J., Koch, C. & Carter, O. Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proc. Natl. Acad. Sci. USA 105, 1704–1709 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  15. Van Olst, E.H., Heemstra, M.L. & Ten Kortenaar, T. Stimulus significance and the orienting reaction. in The Orienting Reflex in Humans: International Conference Proceedings (eds. Kimmel, H., Olst, E.H. & van Orlebeke, J.F.) 521–547 (Erlbaum, 1979).

  16. Sutton, R.S. & Barto, A.G. Reinforcement Learning: An Introduction (MIT Press, Cambridge, Massachusetts, 1998).

  17. Adams, R.P. & MacKay, D.J.C. Bayesian online changepoint detection. Preprint at <http://arxiv.org/abs/0710.3742v1> (2007).

  18. Fearnhead, P. & Liu, Z. On-line inference for multiple changepoint problems. J. R. Stat. Soc. Series B Stat. Methodol. 69, 589–605 (2007).

    Article  Google Scholar 

  19. Wilson, R.C., Nassar, M.R. & Gold, J.I. Bayesian online learning of the hazard rate in change-point problems. Neural Comput. 22, 2452–2476 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Yerkes, R.M. & Dodson, J.D. The relation of strength of stimulus to rapidity of habit-formation. J. Comp. Neurol. Psychol. 18, 459–482 (1908).

    Article  Google Scholar 

  21. Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).

    Article  CAS  PubMed  Google Scholar 

  22. Raisig, S., Welke, T., Hagendorf, H. & van der Meer, E. I spy with my little eye: detection of temporal violations in event sequences and the pupillary response. Int. J. Psychophysiol. 76, 1–8 (2010).

    Article  PubMed  Google Scholar 

  23. Friedman, D., Hakerem, G., Sutton, S. & Fleiss, J.L. Effect of stimulus uncertainty on the pupillary dilation response and the vertex evoked potential. Electroencephalogr. Clin. Neurophysiol. 34, 475–484 (1973).

    Article  CAS  PubMed  Google Scholar 

  24. Preuschoff, K., 't Hart, B.M. & Einhäuser, W. Pupil dilation signals surprise: evidence for noradrenaline's role in decision making. Front. Neurosci. 5, 115 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  25. Aston-Jones, G., Ennis, M., Pieribone, V.A., Nickell, W.T. & Shipley, M.T. The brain nucleus locus coeruleus: restricted afferent control of a broad efferent network. Science 234, 734–737 (1986).

    Article  CAS  PubMed  Google Scholar 

  26. Sara, S.J., Vankov, A. & Hervé, A. Locus coeruleus-evoked responses in behaving rats: a clue to the role of noradrenaline in memory. Brain Res. Bull. 35, 457–465 (1994).

    Article  CAS  PubMed  Google Scholar 

  27. Aston-Jones, G., Rajkowski, J. & Kubiak, P. Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience 80, 697–715 (1997).

    Article  CAS  PubMed  Google Scholar 

  28. Tully, K. & Bolshakov, V.Y. Emotional enhancement of memory: how norepinephrine enables synaptic plasticity. Mol. Brain 3, 15 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Harley, C.W. A role for norepinephrine in arousal, emotion and learning? Limbic modulation by norepinephrine and the Kety hypothesis. Prog. Neuropsychopharmacol. Biol. Psychiatry 11, 419–458 (1987).

    Article  CAS  PubMed  Google Scholar 

  30. Corbetta, M., Patel, G. & Shulman, G.L. The reorienting system of the human brain: from environment to theory of mind. Neuron 58, 306–324 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Bouret, S. & Sara, S.J. Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 28, 574–582 (2005).

    Article  CAS  PubMed  Google Scholar 

  32. Critchley, H.D., Mathias, C.J. & Dolan, R.J. Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron 29, 537–545 (2001).

    Article  CAS  PubMed  Google Scholar 

  33. Critchley, H.D. Neural mechanisms of autonomic, affective, and cognitive integration. J. Comp. Neurol. 493, 154–166 (2005).

    Article  PubMed  Google Scholar 

  34. Krugel, L.K., Biele, G., Mohr, P.N., Li, S.C. & Heekeren, H.R. Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions. Proc. Natl. Acad. Sci. USA 106, 17951–17956 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647–656 (2007).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank J. Cohen, S. du Lac, L. Ding, Y. Li and J. Nassar for helpful comments. This work was supported by the US National Institutes of Health (R01 EY015260, F31 MH093099 and T90 DA22763), the McKnight Endowment Fund for Neuroscience, the Burroughs-Wellcome Fund and the Sloan Foundation.

Author information

Authors and Affiliations

  1. Department of Neuroscience, University of Pennsylvania, Philadelphia, Pennsylvania, USA

    Matthew R Nassar, Katherine M Rumsey, Kinjan Parikh, Benjamin Heasly & Joshua I Gold

  2. Department of Psychology, Princeton University, Princeton, New Jersey, USA

    Robert C Wilson

Authors
  1. Matthew R Nassar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Katherine M Rumsey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert C Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kinjan Parikh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Benjamin Heasly
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joshua I Gold
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.R.N., J.I.G. and B.H. designed the experiment and tasks. M.R.N., K.M.R. and K.P. collected and analyzed data. M.R.N. and R.C.W. developed and applied the reduced Bayesian model. M.R.N. and J.I.G. wrote the manuscript.

Corresponding author

Correspondence to Joshua I Gold.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nassar, M., Rumsey, K., Wilson, R. et al. Rational regulation of learning dynamics by pupil-linked arousal systems. Nat Neurosci 15, 1040–1046 (2012). https://doi.org/10.1038/nn.3130

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3130

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing