Abstract
Humans often seek information to minimize the pervasive effect of uncertainty on decisions. Current theories explain how much knowledge people should gather before a decision, based on the cost–benefit structure of the problem at hand. Here, we demonstrate that this framework omits a crucial agent-related factor: the cognitive effort expended while collecting information. Using an active sampling model, we unveil a speed–efficiency trade-off whereby more informative samples take longer to find. Crucially, under sufficient time pressure, humans can break this trade-off, sampling both faster and more efficiently. Computational modelling demonstrates the existence of a cost of cognitive effort which, when incorporated into theoretical models, provides a better account of people’s behaviour and also relates to self-reported fatigue accumulated during active sampling. Thus, the way people seek knowledge to guide their decisions is shaped not only by task-related costs and benefits, but also crucially by the quantifiable computational costs incurred.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
Anonymized participant data have been deposited on the Open Science Framework platform and can be found at https://osf.io/25wkh/.
Code availability
Code for replicating the main results reported in the manuscript has been deposited on the Open Science Framework platform and can be found at https://osf.io/25wkh/. Additional analysis codes are available on request from the corresponding authors.
Change history
13 July 2021
A Correction to this paper has been published: https://doi.org/10.1038/s41562-021-01165-x
References
Clark, L., Robbins, T. W., Ersche, K. D. & Sahakian, B. J.Reflection impulsivity in current and former substance users. Biol. Psychiatry 60, 515–522 (2006).
Battaglia, P. W. & Schrater, P. R. Humans trade off viewing time and movement duration to improve visuomotor accuracy in a fast reaching task. J. Neurosci. 27, 6984–6994 (2007).
Gottlieb, J., Oudeyer, P. Y., Lopes, M. & Baranes, A. Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends Cogn. Sci. 17, 585–593 (2013).
Averbeck, B. B. Theory of choice in bandit, information sampling and foraging tasks. PLoS Comput. Biol. 11, e1004164 (2015).
Juni, M. Z., Gureckis, T. M. & Maloney, L. T. ‘Information sampling behavior with explicit sampling costs’: correction to Juni, Gureckis, and Maloney (2015). Decision 3, 168 (2016).
Hauser, T. U., Moutoussis, M., Dayan, P. & Dolan, R. J.Increased decision thresholds trigger extended information gathering across the compulsivity spectrum. Transl. Psychiatry 7, 1296 (2017).
Savage, L. J. The Foundations of Statistics (Courier Corporation, 1972).
Tversky, A. & Kahneman, D. Advances in prospect theory: cumulative representation of uncertainty. J. Risk Uncertain. 5, 297–323 (1992).
Von Neumann, J., Morgenstern, O. & Kuhn, H. W. Theory of Games and Economic Behavior, Commemorative Edition (Princeton Univ. Press, 2007).
Johnson, J. G. & Busemeyer, J. R. Decision making under risk and uncertainty. Wiley Interdiscip. Rev. Cogn. Sci. 1, 736–749 (2010).
Kahneman, D. & Tversky, A. in Handbook of the Fundamentals of Financial Decision Making Vol. 4 99–127 (World Scientific, 2012).
Chamberlain, S. R. et al. A neuropsychological comparison of obsessive–compulsive disorder and trichotillomania. Neuropsychologia 45, 654–662 (2007).
Furl, N. & Averbeck, B. B. Parietal cortex and insula relate to evidence seeking relevant to reward-related decisions. J. Neurosci. 31, 17572–17582 (2011).
Djamshidian, A. et al. Decision making, impulsivity, and addictions: do Parkinson’s disease patients jump to conclusions? Mov. Disord. 27, 1137–1145 (2012).
Hauser, T. U. et al. Increased decision thresholds enhance information gathering performance in juvenile obsessive–compulsive disorder (OCD). PLoS Comput. Biol. 13, e1005440 (2017).
Hauser, T. U., Moutoussis, M., Purg, N., Dayan, P. & Dolan, R. J. Beta-blocker propranolol modulates decision urgency during sequential information gathering. J. Neurosci. 38, 7170–7178 (2018).
Jones, P. R. et al. Efficient visual information sampling develops late in childhood. J. Exp. Psychol. Gen. 148, 1138–1152 (2019).
Hertwig, R., Barron, G., Weber, E. U. & Erev, I. Decisions from experience and the effect of rare events in risky choice. Psychol. Sci. 15, 534–539 (2004).
Gottlieb, J. & Oudeyer, P. Y. Towards a neuroscience of active sampling and curiosity. Nat. Rev. Neurosci. 19, 758–770 (2018).
Bossaerts, P., Yadav, N. & Murawski, C. Uncertainty and computational complexity. Phil. Trans. R. Soc. B 374, 20180138 (2019).
Kurzban, R., Duckworth, A., Kable, J. W. & Myers, J. An opportunity cost model of subjective effort and task performance. Behav. Brain Sci. 36, 661–679 (2013).
Boureau, Y.-L., Sokol-Hessner, P. & Daw, N. D.Deciding how to decide: self-control and meta-decision making. Trends Cogn. Sci. 19, 700–710 (2015).
Shenhav, A., Botvinick, M. M. & Cohen, J. D.The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).
Westbrook, A. & Braver, T. S. Cognitive effort: a neuroeconomic approach. Cogn. Affect. Behav. Neurosci. 15, 395–415 (2015).
Horan, M., Daddaoua, N. & Gottlieb, J. Parietal neurons encode information sampling based on decision uncertainty. Nat. Neurosci. 22, 1327–1335 (2019).
Phillips, L. D. & Edwards, W.Conservatism in a simple probability inference task. J. Exp. Psychol. 72, 346–354 (1966).
Tversky, A. & Edwards, W.Information versus reward in binary choices. J. Exp. Psychol. 71, 680–683 (1966).
Dixon, M. L. & Christoff, K. The decision to engage cognitive control is driven by expected reward-value: neural and behavioral evidence. PLoS ONE 7, e51637 (2012).
Shenhav, A., Botvinick, M. M. & Cohen, J. D.The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).
Manohar, S. G. et al. Reward pays the cost of noise reduction in motor and cognitive control. Curr. Biol. 25, 1707–1716 (2015).
Acerbi, L. & Ma, W. J.Practical Bayesian optimization for model fitting with Bayesian adaptive direct search. Adv. Neural Inf. Process. Syst. 30, 1836–1846 (2017).
Bogacz, R., Wagenmakers, E.-J., Forstmann, B. U. & Nieuwenhuis, S. The neural basis of the speed–accuracy tradeoff. Trends Neurosci. 33, 10–16 (2010).
Carpenter, P. A., Just, M. A. & Shell, P. What one intelligence test measures: a theoretical account of the processing in the Raven progressive matrices test. Psychol. Rev. 97, 404–431 (1990).
Ang, Y. S., Lockwood, P., Apps, M. A., Muhammed, K. & Husain, M. Distinct subtypes of apathy revealed by the apathy motivation index. PLoS ONE 12, e0169938 (2017).
Patton, J. H., Stanford, M. S. & Barratt, E. S. Factor structure of Barratt impulsiveness scale. J. Clin. Psychol. 51, 768–774 (1995).
Charnov, E. L. et al. Optimal Foraging, the Marginal Value Theorem (Academic Press, 1976).
Kolling, N., Behrens, T. E. J., Mars, R. B. & Rushworth, M. F. S. Neural mechanisms of foraging. Science 336, 95–98 (2012).
Shenhav, A. et al. Toward a rational and mechanistic account of mental effort. Annu. Rev. Neurosci. 40, 99–124 (2017).
Payne, J. W., Payne, J. W., Bettman, J. R. & Johnson, E. J. The Adaptive Decision Maker (Cambridge Univ. Press, 1993).
Stark, D. P. & Choplin, J. M. A license to deceive: enforcing contractual myths despite consumer psychological realities academic article. NY Univ. J. Law Bus. 5, 617–744 (2009).
Otto, A. R. & Daw, N. D.The opportunity cost of time modulates cognitive effort. Neuropsychologia 123, 92–105 (2019).
Hasenstaub, A., Otte, S., Callaway, E. & Sejnowski, T. J. Metabolic cost as a unifying principle governing neuronal biophysics. Proc. Natl Acad. Sci. USA 107, 12329–12334 (2010).
Kitzbichler, M. G., Henson, R. N. A., Smith, M. L., Nathan, P. J. & Bullmore, E. T. Cognitive effort drives workspace configuration of human brain functional networks. J. Neurosci. 31, 8259–8270 (2011).
Bullmore, E. & Sporns, O. The economy of brain network organization. Nat. Rev. Neurosci. 13, 336–349 (2012).
Kostal, L. & Kobayashi, R.Optimal decoding and information transmission in Hodgkin–Huxley neurons under metabolic cost constraints. Biosystems 136, 3–10 (2015).
Hartmanis, J. & Stearns, R. E. On the computational complexity of algorithms. Trans. Am. Math. Soc. 117, 285–306 (1965).
Arora, S. & Barak, B. Computational Complexity: a Modern Approach (Cambridge Univ. Press, 2009).
Jones, M. & Sugden, R. Positive confirmation bias in the acquisition of information. Theory Decis. 50, 59–99 (2001).
Huys, Q. J. M. et al. Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLOS Comput. Biol. 8, e1002410 (2012).
Körding, K. P. & Wolpert, D. M.Bayesian decision theory in sensorimotor control. Trends Cogn. Sci. 10, 319–326 (2006).
Husain, M. & Roiser, J. P. Neuroscience of apathy and anhedonia: a transdiagnostic approach. Nat. Rev. Neurosci. 19, 470–484 (2018).
Fleming, S. M. & Lau, H. C. How to measure metacognition. Front. Hum. Neurosci. 8, 443 (2014).
Vaghi, M. M. et al. Compulsivity reveals a novel dissociation between action and confidence. Neuron 96, 348–354.e4 (2017).
Silvetti, M., Vassena, E., Abrahamse, E. & Verguts, T. Dorsal anterior cingulate–brainstem ensemble as a reinforcement meta-learner. PLoS Comput. Biol. 14, e1006370 (2018).
Bonnelle, V., Manohar, S., Behrens, T. & Husain, M. Individual differences in premotor brain systems underlie behavioral apathy. Cereb. Cortex 26, 807–819 (2016).
Lockwood, P. L. et al. Prosocial apathy for helping others when effort is required. Nat. Hum. Behav. 1, 0131 (2017).
Brainard, D. H. The psychophysics toolbox. Spat. Vis. 10, 433–436 (1997).
Kleiner, M. et al. What’s new in Psychtoolbox-3? Perception 36, 1–16 (2007).
Chong, T. T. et al. Neurocomputational mechanisms underlying subjective valuation of effort costs. PLoS Biol. 15, e1002598 (2017).
Le Heron, C. et al. Dysfunctional effort-based decision-making underlies apathy in genetic cerebral small vessel disease. Brain 141, 3193–3210 (2018).
Le Heron, C. et al. Distinct effects of apathy and dopamine on effort-based decision-making in Parkinson’s disease. Brain 141, 1455–1469 (2018).
Beck, A. T., Ward, C. H., Mendelson, M., Mock, J. & Erbaugh, J. An inventory for measuring depression. Arch. Gen. Psychiatry 4, 561–571 (1961).
Buhr, K. & Dugas, M.The Intolerance of Uncertainty Scale: psychometric properties of the English version. Behav. Res. Ther. 40, 931–945 (2002).
Acknowledgements
We thank L. Acerbi for help designing an optimal search algorithm (infoMax) and M. Serrano Bonilla for help with figure design. We also thank all of the members of the Cognitive Neurology Research Group for assistance as experimental confederates. This work was supported by the Wellcome Trust (206330/Z/17/Z). P.P. and M.H. were funded by the Wellcome Trust (206330/Z/17/Z). B.A. was funded by a Rhodes Scholarship. S.G.M. was funded by an MRC Clinician Scientist Fellowship (MR/P00878/X). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Author information
Authors and Affiliations
Contributions
P.P., B.A. and M.H. designed the study. P.P. and B.A. collected the data. P.P., B.A. and S.G.M. analysed the data. P.P. and M.H. wrote the paper.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature Human Behaviour thanks Jacqueline Gottlieb and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Demographic and questionnaire measures.
The table shows group means with standard deviation in parenthesis. F: Female; M: Male; YOE: years of education (since kindergarten); BDI: Beck Depression Inventory-II; AMI: Apathy Motivation Index; BIS-11: Barratt Impulsiveness Scale-11; IUS: Intolerance of Uncertainty Scale.
Extended Data Fig. 2 Experimental design.
All experiments included four types of trial (24 repetitions per trial type; 96 trials in total). The cost-benefit and temporal structure of these trial types is depicted here. R0: initial credit; ηs: sampling cost; ηe: error cost; \({{\rm{t}}}_{\max }\): time allocated to the sampling phase.
Extended Data Fig. 3 A visual illustration of efficient versus inefficient sampling.
A map of the posterior belief (top row) and of the expected change in expected error (EE) (bottom row) is plotted for the two example trials plotted in Fig. 1c (panel a: High α trial; panel b: Low α trial). The posterior belief map represents (in blue) all pixels that may be the centre of the hidden circle given the information on the screen (grey: search space; purple dots: observations inside the hidden circle; white dots: observations outside the hidden circle). The smaller this area, the lower the overall uncertainty (EE). For each sample, the associated map represents the change in EE expected to occur for every possible candidate sampling location. An efficient sampler systematically samples where the expected ΔEE is the lowest. a. During efficient sampling, the area covered by the posterior belief decays because the participant samples where the expected ΔEE is the lowest. b. By contrast, when sampling is inefficient, the area covered by the posterior belief decays less sharply because the participant does not sample where the expected ΔEE is the lowest.
Extended Data Fig. 4 On average, the expected error decreases exponentially over successive samples.
Grey lines represent, for each participant (n = 127 across all five experiments), the average Expected Error as a function of sample index. The black line shows the average across all participants ( ± s.d.).
Extended Data Fig. 5 Free parameters included in our models of speed, efficiency and extent of active information sampling.
Models were defined using the equation reported in the Methods. Tick: parameter included in the model and estimated through the fitting procedure; Cross: parameter not included in the model (effectively set to zero). *: Winning model (Model comparison in Fig. 5, Extended Data Fig. 7& Supplementary Fig. 2).
Extended Data Fig. 6 Economic and time constraints influence the extent, speed and efficiency of active information sampling.
a. Individuals sampled more with large initial reward reserves (Exp. 2, n = 20), when acquiring data was cheap (Exp. 3, n = 20), when final error was strongly penalised (Exp. 4, n = 20), and when they had time to search (Exp. 5, n = 19). Hand icon adapted from Freepik: macrovector. b-c. The initial credit and severity of error discounting did not influence how quickly or efficiently participants resolved localisation uncertainty (Exps. 2 & 4). Players sampled slower but more efficiently when acquiring data was more expensive (Exp. 3). By contrast, they speeded up but maintained a high level of efficiency (indexed by information extraction rate) under tighter time pressure (Exp. 5). Points and error bars show group mean ± s.e.m. Lines show condition averages for each participant. Positive/negative main effect of the experimental variable manipulated is shown in red/blue. Full statistical details are reported in Supplementary Tables 3-5.
Extended Data Fig. 7 Model comparison from Experiments 2-5.
The same models as in Fig. 5 are compared here. Model 1 and 2 included no cognitive effort cost (Model 1: rational specification; Model 2: nonlinear utility function; in grey) and were used as a reference. The other 12 models included a cognitive effort cost specified in terms of either speed (Models 3, 6, 9, 12), efficiency (Models 4, 7, 10, 13) or both (Models 5, 8, 11, 14). They differed with respect to the shape of the cost function (green: linear; orange: quadratic; blue: exponential; pink: sigmoid). Across all four experiments, participants’ choices of sampling speed, efficiency and extent were best explained by a model penalising both speed and efficiency with a quadratic cost function (Model 8, lowest Bayesian Information Criterion, BIC).
Supplementary information
Supplementary Information
Supplementary Methods, Supplementary Results, Supplementary Figs. 1–9 and Supplementary Tables 1–15.
Rights and permissions
About this article
Cite this article
Petitet, P., Attaallah, B., Manohar, S.G. et al. The computational cost of active information sampling before decision-making under uncertainty. Nat Hum Behav 5, 935–946 (2021). https://doi.org/10.1038/s41562-021-01116-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41562-021-01116-6
This article is cited by
-
Preferences for seeking effort or reward information bias the willingness to work
Scientific Reports (2022)