Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Confidence reflects a noisy decision reliability estimate

Nature Human Behaviour volume 7, pages 142–154 (2023)Cite this article

Subjects

Abstract

Decisions vary in difficulty. Humans know this and typically report more confidence in easy than in difficult decisions. However, confidence reports do not perfectly track decision accuracy, but also reflect response biases and difficulty misjudgements. To isolate the quality of confidence reports, we developed a model of the decision-making process underlying choice-confidence data. In this model, confidence reflects a subject’s estimate of the reliability of their decision. The quality of this estimate is limited by the subject’s uncertainty about the uncertainty of the variable that informs their decision (‘meta-uncertainty’). This model provides an accurate account of choice-confidence data across a broad range of perceptual and cognitive tasks, investigated in six previous studies. We find meta-uncertainty varies across subjects, is stable over time, generalizes across some domains and can be manipulated experimentally. The model offers a parsimonious explanation for the computational processes that underlie and constrain the sense of confidence.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: CASANDRE, a two-stage process model of decision confidence, accounts for the relation between confidence reports and choice consistency.
Fig. 2: Impact of the different model components on primary choice behaviour and confidence reports.
Fig. 3: Comparison of different model architectures.
Fig. 4: Model recovery analysis.
Fig. 5: Evaluating meta-uncertainty as a psychological construct.
Fig. 6: Meta-uncertainty depends on task structure.

Similar content being viewed by others

Data availability

This study generated no new data. The data used in this study are available from the Confidence Database (available at: https://osf.io/s46pr/).

Code availability

The code supporting the findings of this study and a software package implementing the CASANDRE model is publicly available (https://github.com/gorislab/CASANDRE.git).

References

  1. Meyniel, F., Sigman, M. & Mainen, Z. F. Confidence as Bayesian probability: from neural origins to behavior. Neuron 88, 78–92 (2015).

    Article  CAS  Google Scholar 

  2. Drugowitsch, J., Mendonça, A. G., Mainen, Z. F. & Pouget, A. Learning optimal decisions with confidence. Proc. Natl Acad. Sci. USA 116, 24872–24880 (2019).

    Article  CAS  Google Scholar 

  3. Purcell, B. A. & Kiani, R. Hierarchical decision processes that operate over distinct timescales underlie choice and changes in strategy. Proc. Natl Acad. Sci. USA 113, E4531–E4540 (2016).

    Article  CAS  Google Scholar 

  4. Bahador, B. et al. Optimally interacting minds. Science 329, 1081–1085 (2010).

    Article  Google Scholar 

  5. Peirce, C. S. & Jastrow, J. On small differences in sensation. Memoirs of the National Academy of Sciences, 3, 75–83 (1884).

  6. Ratcliff, R. A theory of memory retrieval. Psychol. Rev. 85, 59–108 (1978).

    Article  Google Scholar 

  7. Vickers, D. Decision processes in visual perception. (Academic, 1979).

    Google Scholar 

  8. de Gardelle, V., Le Corre, F. & Mamassian, P. Confidence as a common currency between vision and audition. PLoS ONE 11, e0147901 (2016).

    Article  Google Scholar 

  9. Fleming, S. M., Weil, R. S., Nagy, Z. Dolan, R. J. & Rees, G. Relating introspective accuracy to individual differences in brain structure. Science 329, 1541–1543 (2010).

  10. Rouault, M., Seow, T., Gillan, C. M. & Fleming, S. M. Psychiatric symptom dimensions are associated with dissociable shifts in metacognition but not task performance. Biol. Psychiatry 84, 443–451 (2018).

    Article  Google Scholar 

  11. Kuhn, D. in Children’s Reasoning and the Mind (eds Mitchell, P. & Riggs, K. J.) 301–326 (Psychology Press, 2000).

  12. Nelson, T. O. A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychol. Bull. 95, 109–133 (1984).

    Article  CAS  Google Scholar 

  13. Fleming, S. M. & Lau, H. C. How to measure metacognition. Front. Hum. Neurosci. 8, 443 (2014).

    Article  Google Scholar 

  14. Mamassian, P. Visual confidence. Annu. Rev. Vis. Sci. 2, 459–481 (2016).

    Article  Google Scholar 

  15. Guggenmos, M. Measuring metacognitive performance: type 1 performance dependence and test-retest reliability. Neurosci. Conscious. 2021, niab040 (2021).

    Article  Google Scholar 

  16. Festinger, L. Studies in decision: I. Decision-time, relative frequency of judgment and subjective confidence as related to physical stimulus difference. J. Exp. Psychol. 32, 291–306 (1943).

    Article  Google Scholar 

  17. Hosseini, J. & Ferrell, W. R. Detectability of correctness: a measure of knowing that one knows. Instructional Sci. 11, 113–127 (1982).

    Article  Google Scholar 

  18. Critchfield, T. S. Signal-detection properties of verbal self-reports. J. Exp. Anal. Behav. 60, 495–514 (1993).

    Article  CAS  Google Scholar 

  19. Galvin, S. J., Podd, J. V., Drga, V. & Whitmore, J. Type 2 tasks in the theory of signal detectability: discrimination between correct and incorrect decisions. Psychon. Bull. Rev. 10, 843–876 (2003).

    Article  Google Scholar 

  20. Maniscalco, B. & Lau, H. A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings. Conscious. Cogn. 21, 422–430 (2012).

    Article  Google Scholar 

  21. Fleming, S. M. & Daw, N. D. Self-evaluation of decision-making: a general Bayesian framework for metacognitive computation. Psychol. Rev. 124, 91–114 (2017).

    Article  Google Scholar 

  22. Adler, W. T. & Ma, W. J. Comparing Bayesian and non-Bayesian accounts of human confidence reports. PLoS Comput. Biol. 14, e1006572 (2018).

    Article  Google Scholar 

  23. Bang, J. W., Shekhar, M. & Rahnev, D. Sensory noise increases metacognitive efficiency. J. Exp. Psychol. Gen. 148, 437–452 (2019).

    Article  Google Scholar 

  24. Khalvati, K., Kiani, R. & Rao, R. P. N. Bayesian inference with incomplete knowledge explains perceptual confidence and its deviations from accuracy. Nat. Commun. 12, 5704 (2021).

    Article  CAS  Google Scholar 

  25. Shekhar, M. & Rahnev, D. The nature of metacognitive inefficiency in perceptual decision making. Psychol. Rev. 128, 45–70 (2021).

    Article  Google Scholar 

  26. Mamassian, P. & de Gardelle, V. Modeling perceptual confidence and the confidence forced-choice paradigm. Psychol. Rev. https://doi.org/10.1037/rev0000312 (2021).

  27. Caziot, B. & Mamassian, P. Perceptual confidence judgments reflect self-consistency. J. Vis. 21, 8 (2021).

    Article  Google Scholar 

  28. Pouget, A., Drugowitsch, J. & Kepecs, A. Confidence and certainty: distinct probabilistic quantities for different goals. Nat. Neurosci. 19, 366–374 (2016).

    Article  CAS  Google Scholar 

  29. Green, D. M. & Swets, J. A. Signal Detection Theory and Psychophysics, Vol. 1 (Wiley, 1966).

  30. Koriat, A. The self-consistency model of subjective confidence. Psychol. Rev. 119, 80–113 (2012).

    Article  Google Scholar 

  31. Rahnev, D. et al. The confidence database. Nat. Hum. Behav. 4, 317–325 (2020).

    Article  Google Scholar 

  32. Navajas, J. et al. The idiosyncratic nature of confidence. Nat. Hum. Behav. 1, 810–818 (2017).

    Article  Google Scholar 

  33. Denison, R. N., Adler, W. T., Carrasco, M. & Ma, W. J. Humans incorporate attention-dependent uncertainty into perceptual decisions and confidence. Proc. Natl Acad. Sci. USA 115, 11090–11095 (2018).

    Article  CAS  Google Scholar 

  34. Rausch, M., Zehetleitner, M., Steinhauser, M. & Maier, M. E. Cognitive modelling reveals distinct electrophysiological markers of decision confidence and error monitoring. NeuroImage 218, 116963 (2020).

    Article  Google Scholar 

  35. Balakrishnan, J. D. & Ratcliff, R. Testing models of decision making using confidence ratings in classification. J. Exp. Psychol. Hum. Percept. Perform. 22, 615–633 (1996).

    Article  CAS  Google Scholar 

  36. Ferrell, W. R. A model for realism of confidence judgments: implications for underconfidence in sensory discrimination. Percept. Psychophys. 57, 246–254 (1995).

    Article  CAS  Google Scholar 

  37. Kepecs, A., Uchida, N., Zariwala, H. A. & Zachary, Z. F. Neural correlates, computation and behavioural impact of decision confidence. Nature 455, 227–231 (2008).

    Article  CAS  Google Scholar 

  38. Treisman, M. & Faulkner, A. The setting and maintenance of criteria representing levels of confidence. J. Exp. Psychol. Hum. Percept. Perform. 10, 119–139 (1984).

    Article  Google Scholar 

  39. Wallsten, T. S. & González-Vallejo, C. Statement verification: a stochastic model of judgment and response. Psychol. Rev. 101, 490–504 (1994).

    Article  Google Scholar 

  40. Jaynes, E. T. Information theory and statistical mechanics. Phys. Rev. 106, 620–630 (1957).

    Article  Google Scholar 

  41. Mahajan, S. Street-Fighting Mathematics: The Art of Educated Guessing and Opportunistic Problem Solving (MIT Press, 2010).

  42. Locke, S. M., Gaffin-Cahn, E., Hosseinizaveh, N., Mamassian, P. & Landy, M. S. Priors and payoffs in confidence judgments. Atten. Percept. Psychophys. 82, 3158–3175 (2020).

    Article  Google Scholar 

  43. Mihali, A., Broeker, M. & Horga, G. Insightful inference compensates for distorted perception. Preprint at bioRxiv https://doi.org/10.1101/2021.11.13.468497 (2021).

  44. Fetsch, C. R., Kiani, R., Newsome, W. T. & Shadlen, M. N. Effects of cortical microstimulation on confidence in a perceptual decision. Neuron 83, 797–804 (2014).

    Article  CAS  Google Scholar 

  45. Fetsch, C. R. et al. Focal optogenetic suppression in macaque area MT biases direction discrimination and decision confidence, but only transiently. eLife 7(July), e36523 (2018).

    Article  Google Scholar 

  46. Xue, K., Shekhar, M. & Rahnev, D. Examining the robustness of the relationship between metacognitive efficiency and metacognitive bias. Conscious. Cogn. 95, 103196 (2021).

    Article  Google Scholar 

  47. McCurdy, L. Y. et al. Anatomical coupling between distinct metacognitive systems for memory and visual perception. J. Neurosci. 33, 1897–1906 (2013).

    Article  CAS  Google Scholar 

  48. Baird, B., Cieslak, M., Smallwood, J., Grafton, S. T. & Schooler, J. W. Regional white matter variation associated with domain-specific metacognitive accuracy. J. Cogn. Neurosci. 27, 440–452 (2015).

    Article  Google Scholar 

  49. Lee, A. L. F., Ruby, E., Giles, N. & Lau, H. Cross-domain association in metacognitive efficiency depends on first-order task types. Front. Psychol. 9, 2464 (2018).

  50. Shields, W. E., Smith, J. D., Guttmannova, K. & Washburn, D. A. Confidence judgments by humans and rhesus monkeys. J. Gen. Psychol. 132, 165–186 (2005).

    Google Scholar 

  51. Locke, S. M., Landy, M. S. & Mamassian, P. Suprathreshold perceptual decisions constrain models of confidence. PLoS Comput. Biol. 18, e1010318 (2022).

    Article  CAS  Google Scholar 

  52. Rahnev, D. et al. Consensus goals in the field of visual metacognition. Perspect. Psychol. Sci. https://doi.org/10.1177/17456916221075615 (2022).

  53. Ko, Y. & Lau, H. A detection theoretic explanation of blindsight suggests a link between conscious perception and metacognition. Philos. Trans. R. Soc. Lond. B Biol. Sci. 367, 1401–1411 (2012).

    Article  Google Scholar 

  54. Komura, Y., Nikkuni, A., Hirashima, N., Uetake, T. & Miyamoto, A. Responses of pulvinar neurons reflect a subject’s confidence in visual categorization. Nat. Neurosci. 16, 749–755 (2013).

    Article  CAS  Google Scholar 

  55. Massoni, S., Gajdos, T. & Vergnaud, J.-C. Confidence measurement in the light of signal detection theory. Front. Psychol. 5, 1455 (2014).

  56. Zylberberg, A., Barttfeld, P. & Sigman, M. The construction of confidence in a perceptual decision. Front. Integr. Neurosci. 6, 79 (2012).

  57. Maniscalco, B., Peters, M. A. K. & Lau, H. Heuristic use of perceptual evidence leads to dissociation between performance and metacognitive sensitivity. Atten. Percept. Psychophys. 78, 923–937 (2016).

    Article  Google Scholar 

  58. Peters, M. A. K. et al. Perceptual confidence neglects decision-incongruent evidence in the brain. Nat. Hum. Behav. 1, 1–8 (2017).

    Article  Google Scholar 

  59. Fetsch, C. R., Kiani, R. & Shadlen, M. N. Predicting the accuracy of a decision: a neural mechanism of confidence. Cold Spring Harb. Symp. Quant. Biol. 79, 185–197 (2014).

  60. Murphy, P. R., Robertson, I. H., Harty, S. & O’Connell, R. G. Neural evidence accumulation persists after choice to inform metacognitive judgments. eLife 4, e11946 (2015).

    Article  Google Scholar 

  61. Maniscalco, B. & Lau, H. The signal processing architecture underlying subjective reports of sensory awareness. Neurosci. Conscious. 2016, niw002 (2016).

  62. Lak, A. et al. Orbitofrontal cortex is required for optimal waiting based on decision confidence. Neuron 84, 190–201 (2014).

    Article  CAS  Google Scholar 

  63. Kiani, R. & Shadlen, M. N. Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324, 759–764 (2009).

    Article  CAS  Google Scholar 

  64. Sanders, J. I., Hangya, B. & Kepecs, A. Signatures of a statistical computation in the human sense of confidence. Neuron 90, 499–506 (2016).

    Article  CAS  Google Scholar 

  65. Hangya, B., Sanders, J. I. & Kepecs, A. A mathematical framework for statistical decision confidence. Neural Comput. 28, 1840–1858 (2016).

    Article  Google Scholar 

  66. Adler, W. T. & Ma, W. J. Limitations of proposed signatures of Bayesian confidence. Neural Comput. 30, 3327–3354 (2018).

    Article  Google Scholar 

  67. Li, H.-H. & Ma, W. J. Confidence reports in decision-making with multiple alternatives violate the Bayesian confidence hypothesis. Nat. Commun. 11, 2004 (2020).

    Article  CAS  Google Scholar 

  68. Geurts, L. S., Cooke, J. R. H., van Bergen, R. S. & Jehee, J. F. M. Subjective confidence reflects representation of Bayesian probability in cortex. Nat. Hum. Behav. 6, 294–305 (2022).

  69. De Martino, B., Fleming, S. M., Garrett, N. & Dolan, R. Confidence in value-based choice. Nat. Neurosci. 16, 105–110 (2013).

    Article  Google Scholar 

  70. Ernst, M. O. & Banks, M. S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429 (2002).

    Article  CAS  Google Scholar 

  71. Fetsch, C. R., Pouget, A., DeAngelis, G. C. & Angelaki, D. E. Neural correlates of reliability-based cue weighting during multisensory integration. Nat. Neurosci. 15, 146 (2012).

    Article  CAS  Google Scholar 

  72. Qamar, A. T. et al. Trial-to-trial, uncertainty-based adjustment of decision boundaries in visual categorization. Proc. Natl Acad. Sci. USA 110, 20332–20337 (2013).

    Article  CAS  Google Scholar 

  73. Ma, W. J., Beck, J. M., Latham, P. E. & Pouget, A. Bayesian inference with probabilistic population codes. Nat. Neurosci. 9, 1432 (2006).

    Article  CAS  Google Scholar 

  74. Orban, G., Berkes, P., Fiser, J. & Lengyel, M. Neural variability and sampling-based probabilistic representations in the visual cortex. Neuron 92, 530–543 (2016).

    Article  CAS  Google Scholar 

  75. van Bergen, R. S., Ji Ma, W., Pratte, M. S. & Jehee, J. F. M. Sensory uncertainty decoded from visual cortex predicts behavior. Nat. Neurosci. 18, 1728–1730 (2015).

    Article  Google Scholar 

  76. Henaff, O. J., Boundy-Singer, Z. M., Meding, K., Ziemba, C. M. & Goris, R. L. T. Representation of visual uncertainty through neural gain variability. Nat. Commun. 11, 2513 (2020).

    Article  CAS  Google Scholar 

  77. Walker, E. Y., Cotton, R. J., Ma, W. J. & Tolias, A. S. A neural basis of probabilistic computation in visual cortex. Nat. Neurosci. 23, 122–129 (2020).

    Article  CAS  Google Scholar 

  78. Festa, D., Aschner, A., Davila, A., Kohn, A. & Coen-Cagli, R. Neuronal variability reflects probabilistic inference tuned to natural image statistics. Nat. Commun. 12, 3635 (2021).

    Article  CAS  Google Scholar 

  79. Allen, M. et al. Unexpected arousal modulates the influence of sensory noise on confidence. eLife 5, e18103 (2016).

    Article  Google Scholar 

  80. Maniscalco, B., McCurdy, L. Y., Odegaard, B. & Lau, H. Limited cognitive resources explain a trade-off between perceptual and metacognitive vigilance. J. Neurosci. 37, 1213–1224 (2017).

    Article  CAS  Google Scholar 

  81. Rahnev, D., Koizumi, A., McCurdy, L. Y., D’Esposito, M. & Lau, H. Confidence leak in perceptual decision making. Psychol. Sci. 26, 1664–1680 (2015).

    Article  Google Scholar 

  82. Fleming, S. M. Action-specific disruption of perceptual confidence. Psychol. Sci. 26, 89–98 (2015).

    Article  Google Scholar 

  83. Palmer, J., Huk, A. C. & Shadlen, M. N. The effect of stimulus strength on the speed and accuracy of a perceptual decision. J. Vis. 5, 1 (2005).

    Article  Google Scholar 

  84. Hanks, T. D., Mazurek, M. E., Kiani, R., Hopp, E. & Shadlen, M. N. Elapsed decision time affects the weighting of prior probability in a perceptual decision task. J. Neurosci. 31, 6339–6352 (2011).

    Article  CAS  Google Scholar 

  85. Kiani, R., Corthell, L. & Shadlen, M. N. Choice certainty is informed by both evidence and decision time. Neuron 84, 1329–1342 (2014).

    Article  CAS  Google Scholar 

  86. Zylberberg, A., Fetsch, C. R. & Shadlen, M. N. The influence of evidence volatility on choice, reaction time and confidence in a perceptual decision. eLife 5, e17688 (2016).

    Article  Google Scholar 

  87. Geisler, W. S. in The Visual Neurosciences, Vol. 10 (eds Chalupa, L. & Werne, J.) 825–837 (MIT Press, 2003).

  88. Weiss, Y., Simoncelli, E. P. & Adelson, E. H. Motion illusions as optimal percepts. Nat. Neurosci. 5, 598 (2002).

    Article  CAS  Google Scholar 

  89. Persaud, N., McLeod, P. & Cowey, A. Post-decision wagering objectively measures awareness. Nat. Neurosci. 10, 257–261 (2007).

    Article  CAS  Google Scholar 

  90. Dienes, Z. & Seth, A. Gambling on the unconscious: a comparison of wagering and confidence ratings as measures of awareness in an artificial grammar task. Conscious. Cogn. 19, 674–681 (2010).

    Article  Google Scholar 

  91. Murad, Z., Sefton, M. & Starmer, C. How do risk attitudes affect measured confidence? J. Risk Uncertain. 52, 21–46 (2016).

    Article  Google Scholar 

  92. Wichmann, F. A. & Hill, N. J. The psychometric function: I. Fitting, sampling, and goodness of fit. Percept. Psychophys. 63, 1293–1313 (2001).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank the creators and contributors to the Confidence Database and J. Navajas for making their data available. This work was supported by the US National Science Foundation (Graduate Research Fellowship to Z.M.B.-S.), the US National Institutes of Health (grant nos. T32 EY021462 and K99 EY032102 to C.M.Z., and EY032999 to R.L.T.G.), and the Whitehall Foundation (grant no. UTA19-000535 to R.L.T.G.). The funders had no role in the study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Author notes

  1. These authors contributed equally: Zoe M. Boundy-Singer, Corey M. Ziemba.

Authors and Affiliations

  1. Center for Perceptual Systems, The University of Texas at Austin, Austin, TX, USA

    Zoe M. Boundy-Singer, Corey M. Ziemba & Robbe L. T. Goris

Authors
  1. Zoe M. Boundy-Singer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Corey M. Ziemba
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robbe L. T. Goris
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Z.M.B.-S., C.M.Z. and R.L.T.G. conceived the study, developed the theory, performed the simulations, analysed the data, and wrote the paper.

Corresponding author

Correspondence to Robbe L. T. Goris.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks Dobromir Rahnev, Rachel Denison, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boundy-Singer, Z.M., Ziemba, C.M. & Goris, R.L.T. Confidence reflects a noisy decision reliability estimate. Nat Hum Behav 7, 142–154 (2023). https://doi.org/10.1038/s41562-022-01464-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-022-01464-x

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing