Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

The computational cost of active information sampling before decision-making under uncertainty

Nature Human Behaviour volume 5, pages 935–946 (2021)Cite this article

Subjects

A Publisher Correction to this article was published on 13 July 2021

This article has been updated

Abstract

Humans often seek information to minimize the pervasive effect of uncertainty on decisions. Current theories explain how much knowledge people should gather before a decision, based on the cost–benefit structure of the problem at hand. Here, we demonstrate that this framework omits a crucial agent-related factor: the cognitive effort expended while collecting information. Using an active sampling model, we unveil a speed–efficiency trade-off whereby more informative samples take longer to find. Crucially, under sufficient time pressure, humans can break this trade-off, sampling both faster and more efficiently. Computational modelling demonstrates the existence of a cost of cognitive effort which, when incorporated into theoretical models, provides a better account of people’s behaviour and also relates to self-reported fatigue accumulated during active sampling. Thus, the way people seek knowledge to guide their decisions is shaped not only by task-related costs and benefits, but also crucially by the quantifiable computational costs incurred.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Active information-gathering task, with large and complex sampling space.
Fig. 2: A passive counterpart of the game shows that participants are sensitive to reward and uncertainty.
Fig. 3: Economic and time constraints influence the extent, speed and efficiency of active information sampling.
Fig. 4: Choosing how quickly, efficiently and extensively to sample.
Fig. 5: A model comparison robustly shows, across five experiments, that a model with a quadratic cognitive effort cost best explains active information-sampling behaviour.
Fig. 6: The cost of efficiency is associated with fatigue development.
Fig. 7: A breakable speed–efficiency trade-off governs active information sampling.

Similar content being viewed by others

Data availability

Anonymized participant data have been deposited on the Open Science Framework platform and can be found at https://osf.io/25wkh/.

Code availability

Code for replicating the main results reported in the manuscript has been deposited on the Open Science Framework platform and can be found at https://osf.io/25wkh/. Additional analysis codes are available on request from the corresponding authors.

Change history

References

  1. Clark, L., Robbins, T. W., Ersche, K. D. & Sahakian, B. J.Reflection impulsivity in current and former substance users. Biol. Psychiatry 60, 515–522 (2006).

    Article  CAS  PubMed  Google Scholar 

  2. Battaglia, P. W. & Schrater, P. R. Humans trade off viewing time and movement duration to improve visuomotor accuracy in a fast reaching task. J. Neurosci. 27, 6984–6994 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Gottlieb, J., Oudeyer, P. Y., Lopes, M. & Baranes, A. Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends Cogn. Sci. 17, 585–593 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Averbeck, B. B. Theory of choice in bandit, information sampling and foraging tasks. PLoS Comput. Biol. 11, e1004164 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  5. Juni, M. Z., Gureckis, T. M. & Maloney, L. T. ‘Information sampling behavior with explicit sampling costs’: correction to Juni, Gureckis, and Maloney (2015). Decision 3, 168 (2016).

    Article  Google Scholar 

  6. Hauser, T. U., Moutoussis, M., Dayan, P. & Dolan, R. J.Increased decision thresholds trigger extended information gathering across the compulsivity spectrum. Transl. Psychiatry 7, 1296 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  7. Savage, L. J. The Foundations of Statistics (Courier Corporation, 1972).

  8. Tversky, A. & Kahneman, D. Advances in prospect theory: cumulative representation of uncertainty. J. Risk Uncertain. 5, 297–323 (1992).

    Article  Google Scholar 

  9. Von Neumann, J., Morgenstern, O. & Kuhn, H. W. Theory of Games and Economic Behavior, Commemorative Edition (Princeton Univ. Press, 2007).

  10. Johnson, J. G. & Busemeyer, J. R. Decision making under risk and uncertainty. Wiley Interdiscip. Rev. Cogn. Sci. 1, 736–749 (2010).

    Article  PubMed  Google Scholar 

  11. Kahneman, D. & Tversky, A. in Handbook of the Fundamentals of Financial Decision Making Vol. 4 99–127 (World Scientific, 2012).

  12. Chamberlain, S. R. et al. A neuropsychological comparison of obsessive–compulsive disorder and trichotillomania. Neuropsychologia 45, 654–662 (2007).

    Article  PubMed  Google Scholar 

  13. Furl, N. & Averbeck, B. B. Parietal cortex and insula relate to evidence seeking relevant to reward-related decisions. J. Neurosci. 31, 17572–17582 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Djamshidian, A. et al. Decision making, impulsivity, and addictions: do Parkinson’s disease patients jump to conclusions? Mov. Disord. 27, 1137–1145 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  15. Hauser, T. U. et al. Increased decision thresholds enhance information gathering performance in juvenile obsessive–compulsive disorder (OCD). PLoS Comput. Biol. 13, e1005440 (2017).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  16. Hauser, T. U., Moutoussis, M., Purg, N., Dayan, P. & Dolan, R. J. Beta-blocker propranolol modulates decision urgency during sequential information gathering. J. Neurosci. 38, 7170–7178 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Jones, P. R. et al. Efficient visual information sampling develops late in childhood. J. Exp. Psychol. Gen. 148, 1138–1152 (2019).

    Article  PubMed  Google Scholar 

  18. Hertwig, R., Barron, G., Weber, E. U. & Erev, I. Decisions from experience and the effect of rare events in risky choice. Psychol. Sci. 15, 534–539 (2004).

    Article  PubMed  Google Scholar 

  19. Gottlieb, J. & Oudeyer, P. Y. Towards a neuroscience of active sampling and curiosity. Nat. Rev. Neurosci. 19, 758–770 (2018).

    Article  CAS  PubMed  Google Scholar 

  20. Bossaerts, P., Yadav, N. & Murawski, C. Uncertainty and computational complexity. Phil. Trans. R. Soc. B 374, 20180138 (2019).

    Article  PubMed  Google Scholar 

  21. Kurzban, R., Duckworth, A., Kable, J. W. & Myers, J. An opportunity cost model of subjective effort and task performance. Behav. Brain Sci. 36, 661–679 (2013).

    Article  PubMed  Google Scholar 

  22. Boureau, Y.-L., Sokol-Hessner, P. & Daw, N. D.Deciding how to decide: self-control and meta-decision making. Trends Cogn. Sci. 19, 700–710 (2015).

    Article  PubMed  Google Scholar 

  23. Shenhav, A., Botvinick, M. M. & Cohen, J. D.The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Westbrook, A. & Braver, T. S. Cognitive effort: a neuroeconomic approach. Cogn. Affect. Behav. Neurosci. 15, 395–415 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  25. Horan, M., Daddaoua, N. & Gottlieb, J. Parietal neurons encode information sampling based on decision uncertainty. Nat. Neurosci. 22, 1327–1335 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Phillips, L. D. & Edwards, W.Conservatism in a simple probability inference task. J. Exp. Psychol. 72, 346–354 (1966).

    Article  CAS  PubMed  Google Scholar 

  27. Tversky, A. & Edwards, W.Information versus reward in binary choices. J. Exp. Psychol. 71, 680–683 (1966).

    Article  CAS  PubMed  Google Scholar 

  28. Dixon, M. L. & Christoff, K. The decision to engage cognitive control is driven by expected reward-value: neural and behavioral evidence. PLoS ONE 7, e51637 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Shenhav, A., Botvinick, M. M. & Cohen, J. D.The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Manohar, S. G. et al. Reward pays the cost of noise reduction in motor and cognitive control. Curr. Biol. 25, 1707–1716 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Acerbi, L. & Ma, W. J.Practical Bayesian optimization for model fitting with Bayesian adaptive direct search. Adv. Neural Inf. Process. Syst. 30, 1836–1846 (2017).

    Google Scholar 

  32. Bogacz, R., Wagenmakers, E.-J., Forstmann, B. U. & Nieuwenhuis, S. The neural basis of the speed–accuracy tradeoff. Trends Neurosci. 33, 10–16 (2010).

    Article  CAS  PubMed  Google Scholar 

  33. Carpenter, P. A., Just, M. A. & Shell, P. What one intelligence test measures: a theoretical account of the processing in the Raven progressive matrices test. Psychol. Rev. 97, 404–431 (1990).

    Article  CAS  PubMed  Google Scholar 

  34. Ang, Y. S., Lockwood, P., Apps, M. A., Muhammed, K. & Husain, M. Distinct subtypes of apathy revealed by the apathy motivation index. PLoS ONE 12, e0169938 (2017).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  35. Patton, J. H., Stanford, M. S. & Barratt, E. S. Factor structure of Barratt impulsiveness scale. J. Clin. Psychol. 51, 768–774 (1995).

    Article  CAS  PubMed  Google Scholar 

  36. Charnov, E. L. et al. Optimal Foraging, the Marginal Value Theorem (Academic Press, 1976).

  37. Kolling, N., Behrens, T. E. J., Mars, R. B. & Rushworth, M. F. S. Neural mechanisms of foraging. Science 336, 95–98 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Shenhav, A. et al. Toward a rational and mechanistic account of mental effort. Annu. Rev. Neurosci. 40, 99–124 (2017).

    Article  CAS  PubMed  Google Scholar 

  39. Payne, J. W., Payne, J. W., Bettman, J. R. & Johnson, E. J. The Adaptive Decision Maker (Cambridge Univ. Press, 1993).

  40. Stark, D. P. & Choplin, J. M. A license to deceive: enforcing contractual myths despite consumer psychological realities academic article. NY Univ. J. Law Bus. 5, 617–744 (2009).

    Google Scholar 

  41. Otto, A. R. & Daw, N. D.The opportunity cost of time modulates cognitive effort. Neuropsychologia 123, 92–105 (2019).

    Article  PubMed  Google Scholar 

  42. Hasenstaub, A., Otte, S., Callaway, E. & Sejnowski, T. J. Metabolic cost as a unifying principle governing neuronal biophysics. Proc. Natl Acad. Sci. USA 107, 12329–12334 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Kitzbichler, M. G., Henson, R. N. A., Smith, M. L., Nathan, P. J. & Bullmore, E. T. Cognitive effort drives workspace configuration of human brain functional networks. J. Neurosci. 31, 8259–8270 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Bullmore, E. & Sporns, O. The economy of brain network organization. Nat. Rev. Neurosci. 13, 336–349 (2012).

    Article  CAS  PubMed  Google Scholar 

  45. Kostal, L. & Kobayashi, R.Optimal decoding and information transmission in Hodgkin–Huxley neurons under metabolic cost constraints. Biosystems 136, 3–10 (2015).

    Article  PubMed  Google Scholar 

  46. Hartmanis, J. & Stearns, R. E. On the computational complexity of algorithms. Trans. Am. Math. Soc. 117, 285–306 (1965).

    Article  Google Scholar 

  47. Arora, S. & Barak, B. Computational Complexity: a Modern Approach (Cambridge Univ. Press, 2009).

  48. Jones, M. & Sugden, R. Positive confirmation bias in the acquisition of information. Theory Decis. 50, 59–99 (2001).

    Article  Google Scholar 

  49. Huys, Q. J. M. et al. Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLOS Comput. Biol. 8, e1002410 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Körding, K. P. & Wolpert, D. M.Bayesian decision theory in sensorimotor control. Trends Cogn. Sci. 10, 319–326 (2006).

    Article  PubMed  Google Scholar 

  51. Husain, M. & Roiser, J. P. Neuroscience of apathy and anhedonia: a transdiagnostic approach. Nat. Rev. Neurosci. 19, 470–484 (2018).

    Article  CAS  PubMed  Google Scholar 

  52. Fleming, S. M. & Lau, H. C. How to measure metacognition. Front. Hum. Neurosci. 8, 443 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  53. Vaghi, M. M. et al. Compulsivity reveals a novel dissociation between action and confidence. Neuron 96, 348–354.e4 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Silvetti, M., Vassena, E., Abrahamse, E. & Verguts, T. Dorsal anterior cingulate–brainstem ensemble as a reinforcement meta-learner. PLoS Comput. Biol. 14, e1006370 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  55. Bonnelle, V., Manohar, S., Behrens, T. & Husain, M. Individual differences in premotor brain systems underlie behavioral apathy. Cereb. Cortex 26, 807–819 (2016).

    PubMed  Google Scholar 

  56. Lockwood, P. L. et al. Prosocial apathy for helping others when effort is required. Nat. Hum. Behav. 1, 0131 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  57. Brainard, D. H. The psychophysics toolbox. Spat. Vis. 10, 433–436 (1997).

    Article  CAS  PubMed  Google Scholar 

  58. Kleiner, M. et al. What’s new in Psychtoolbox-3? Perception 36, 1–16 (2007).

    Google Scholar 

  59. Chong, T. T. et al. Neurocomputational mechanisms underlying subjective valuation of effort costs. PLoS Biol. 15, e1002598 (2017).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  60. Le Heron, C. et al. Dysfunctional effort-based decision-making underlies apathy in genetic cerebral small vessel disease. Brain 141, 3193–3210 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  61. Le Heron, C. et al. Distinct effects of apathy and dopamine on effort-based decision-making in Parkinson’s disease. Brain 141, 1455–1469 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  62. Beck, A. T., Ward, C. H., Mendelson, M., Mock, J. & Erbaugh, J. An inventory for measuring depression. Arch. Gen. Psychiatry 4, 561–571 (1961).

    Article  CAS  PubMed  Google Scholar 

  63. Buhr, K. & Dugas, M.The Intolerance of Uncertainty Scale: psychometric properties of the English version. Behav. Res. Ther. 40, 931–945 (2002).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank L. Acerbi for help designing an optimal search algorithm (infoMax) and M. Serrano Bonilla for help with figure design. We also thank all of the members of the Cognitive Neurology Research Group for assistance as experimental confederates. This work was supported by the Wellcome Trust (206330/Z/17/Z). P.P. and M.H. were funded by the Wellcome Trust (206330/Z/17/Z). B.A. was funded by a Rhodes Scholarship. S.G.M. was funded by an MRC Clinician Scientist Fellowship (MR/P00878/X). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Department of Experimental Psychology, University of Oxford, Oxford, UK

    Pierre Petitet, Sanjay G. Manohar & Masud Husain

  2. Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK

    Bahaaeddin Attaallah, Sanjay G. Manohar & Masud Husain

Authors
  1. Pierre Petitet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bahaaeddin Attaallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sanjay G. Manohar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Masud Husain
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

P.P., B.A. and M.H. designed the study. P.P. and B.A. collected the data. P.P., B.A. and S.G.M. analysed the data. P.P. and M.H. wrote the paper.

Corresponding authors

Correspondence to Pierre Petitet or Masud Husain.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Human Behaviour thanks Jacqueline Gottlieb and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Demographic and questionnaire measures.

The table shows group means with standard deviation in parenthesis. F: Female; M: Male; YOE: years of education (since kindergarten); BDI: Beck Depression Inventory-II; AMI: Apathy Motivation Index; BIS-11: Barratt Impulsiveness Scale-11; IUS: Intolerance of Uncertainty Scale.

Extended Data Fig. 2 Experimental design.

All experiments included four types of trial (24 repetitions per trial type; 96 trials in total). The cost-benefit and temporal structure of these trial types is depicted here. R0: initial credit; ηs: sampling cost; ηe: error cost; \({{\rm{t}}}_{\max }\): time allocated to the sampling phase.

Extended Data Fig. 3 A visual illustration of efficient versus inefficient sampling.

A map of the posterior belief (top row) and of the expected change in expected error (EE) (bottom row) is plotted for the two example trials plotted in Fig. 1c (panel a: High α trial; panel b: Low α trial). The posterior belief map represents (in blue) all pixels that may be the centre of the hidden circle given the information on the screen (grey: search space; purple dots: observations inside the hidden circle; white dots: observations outside the hidden circle). The smaller this area, the lower the overall uncertainty (EE). For each sample, the associated map represents the change in EE expected to occur for every possible candidate sampling location. An efficient sampler systematically samples where the expected ΔEE is the lowest. a. During efficient sampling, the area covered by the posterior belief decays because the participant samples where the expected ΔEE is the lowest. b. By contrast, when sampling is inefficient, the area covered by the posterior belief decays less sharply because the participant does not sample where the expected ΔEE is the lowest.

Extended Data Fig. 4 On average, the expected error decreases exponentially over successive samples.

Grey lines represent, for each participant (n = 127 across all five experiments), the average Expected Error as a function of sample index. The black line shows the average across all participants ( ± s.d.).

Extended Data Fig. 5 Free parameters included in our models of speed, efficiency and extent of active information sampling.

Models were defined using the equation reported in the Methods. Tick: parameter included in the model and estimated through the fitting procedure; Cross: parameter not included in the model (effectively set to zero). *: Winning model (Model comparison in Fig. 5, Extended Data Fig. 7& Supplementary Fig. 2).

Extended Data Fig. 6 Economic and time constraints influence the extent, speed and efficiency of active information sampling.

a. Individuals sampled more with large initial reward reserves (Exp. 2, n = 20), when acquiring data was cheap (Exp. 3, n = 20), when final error was strongly penalised (Exp. 4, n = 20), and when they had time to search (Exp. 5, n = 19). Hand icon adapted from Freepik: macrovector. b-c. The initial credit and severity of error discounting did not influence how quickly or efficiently participants resolved localisation uncertainty (Exps. 2 & 4). Players sampled slower but more efficiently when acquiring data was more expensive (Exp. 3). By contrast, they speeded up but maintained a high level of efficiency (indexed by information extraction rate) under tighter time pressure (Exp. 5). Points and error bars show group mean ± s.e.m. Lines show condition averages for each participant. Positive/negative main effect of the experimental variable manipulated is shown in red/blue. Full statistical details are reported in Supplementary Tables 3-5.

Extended Data Fig. 7 Model comparison from Experiments 2-5.

The same models as in Fig. 5 are compared here. Model 1 and 2 included no cognitive effort cost (Model 1: rational specification; Model 2: nonlinear utility function; in grey) and were used as a reference. The other 12 models included a cognitive effort cost specified in terms of either speed (Models 3, 6, 9, 12), efficiency (Models 4, 7, 10, 13) or both (Models 5, 8, 11, 14). They differed with respect to the shape of the cost function (green: linear; orange: quadratic; blue: exponential; pink: sigmoid). Across all four experiments, participants’ choices of sampling speed, efficiency and extent were best explained by a model penalising both speed and efficiency with a quadratic cost function (Model 8, lowest Bayesian Information Criterion, BIC).

Supplementary information

Supplementary Information

Supplementary Methods, Supplementary Results, Supplementary Figs. 1–9 and Supplementary Tables 1–15.

Reporting Summary

Peer Review File

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Petitet, P., Attaallah, B., Manohar, S.G. et al. The computational cost of active information sampling before decision-making under uncertainty. Nat Hum Behav 5, 935–946 (2021). https://doi.org/10.1038/s41562-021-01116-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-021-01116-6

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing