Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Representation of retrieval confidence by single neurons in the human medial temporal lobe

Abstract

Memory-based decisions are often accompanied by an assessment of choice certainty, but the mechanisms of such confidence judgments remain unknown. We studied the response of 1,065 individual neurons in the human hippocampus and amygdala while neurosurgical patients made memory retrieval decisions together with a confidence judgment. Combining behavioral, neuronal and computational analysis, we identified a population of memory-selective (MS) neurons whose activity signaled stimulus familiarity and confidence, as assessed by subjective report. In contrast, the activity of visually selective (VS) neurons was not sensitive to memory strength. The groups further differed in response latency, tuning and extracellular waveforms. The information provided by MS neurons was sufficient for a race model to decide stimulus familiarity and retrieval confidence. Together, our results indicate a trial-by-trial relationship between a specific group of neurons and declared memory strength in humans. We suggest that VS and MS neurons are a substrate for declarative memories.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Figure 1: The recognition memory task and behavioral results.
Figure 2: MS neurons.
Figure 3: The response of MS neurons is modulated by subjective confidence.
Figure 4: VS neurons.
Figure 5: The ability of VS neurons to differentiate visual stimuli is not influenced by confidence judgment or novelty of the stimulus.
Figure 6: MS and VS neurons signal at different times and only MS neurons are sensitive to confidence.
Figure 7: Quantification of population-level information difference resulting from confidence.
Figure 8: Computational model to decide the familiarity and confidence of a stimulus.

Similar content being viewed by others

References

  1. Kahana, M.J. Foundations of Human Memory (Oxford University Press, New York, 2012).

  2. Petrusic, W.M. & Baranski, J.V. Judging confidence influences decision processing in comparative judgments. Psychon. Bull. Rev. 10, 177–183 (2003).

    Article  PubMed  Google Scholar 

  3. Kiani, R. & Shadlen, M.N. Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324, 759–764 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Smith, J.D., Shields, W.E. & Washburn, D.A. The comparative psychology of uncertainty monitoring and metacognition. Behav. Brain Sci. 26, 317–339, discussion 340–373 (2003).

    PubMed  Google Scholar 

  5. Kepecs, A. & Mainen, Z.F. A computational framework for the study of confidence in humans and animals. Philos. Trans. R. Soc. Lond. B Biol. Sci. 367, 1322–1337 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Pouget, A., Dayan, P. & Zemel, R.S. Inference and computation with population codes. Annu. Rev. Neurosci. 26, 381–410 (2003).

    Article  CAS  PubMed  Google Scholar 

  7. Kepecs, A., Uchida, N., Zariwala, H.A. & Mainen, Z.F. Neural correlates, computation and behavioural impact of decision confidence. Nature 455, 227–231 (2008).

    Article  CAS  PubMed  Google Scholar 

  8. Squire, L.R., Stark, C.E. & Clark, R.E. The medial temporal lobe. Annu. Rev. Neurosci. 27, 279–306 (2004).

    Article  CAS  PubMed  Google Scholar 

  9. Kreiman, G., Koch, C. & Fried, I. Category-specific visual responses of single neurons in the human medial temporal lobe. Nat. Neurosci. 3, 946–953 (2000).

    Article  CAS  PubMed  Google Scholar 

  10. Viskontas, I.V., Quiroga, R.Q. & Fried, I. Human medial temporal lobe neurons respond preferentially to personally relevant images. Proc. Natl. Acad. Sci. USA 106, 21329–21334 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Logothetis, N.K. & Sheinberg, D.L. Visual object recognition. Annu. Rev. Neurosci. 19, 577–621 (1996).

    Article  CAS  PubMed  Google Scholar 

  12. Rolls, E.T. Functions of the primate temporal lobe cortical visual areas in invariant visual object and face recognition. Neuron 27, 205–218 (2000).

    Article  CAS  PubMed  Google Scholar 

  13. Rutishauser, U., Mamelak, A.N. & Schuman, E.M. Single-trial learning of novel stimuli by individual neurons of the human hippocampus-amygdala complex. Neuron 49, 805–813 (2006).

    Article  CAS  PubMed  Google Scholar 

  14. Rutishauser, U., Schuman, E.M. & Mamelak, A.N. Activity of human hippocampal and amygdala neurons during retrieval of declarative memories. Proc. Natl. Acad. Sci. USA 105, 329–334 (2008).

    Article  PubMed  Google Scholar 

  15. Xiang, J.Z. & Brown, M.W. Differential neuronal encoding of novelty, familiarity and recency in regions of the anterior temporal lobe. Neuropharmacology 37, 657–676 (1998).

    Article  CAS  PubMed  Google Scholar 

  16. Wilson, F.A. & Rolls, E.T. The effects of stimulus novelty and familiarity on neuronal activity in the amygdala of monkeys performing recognition memory tasks. Exp. Brain Res. 93, 367–382 (1993).

    Article  CAS  PubMed  Google Scholar 

  17. Rutishauser, U., Ross, I.B., Mamelak, A.N. & Schuman, E.M. Human memory strength is predicted by theta-frequency phase-locking of single neurons. Nature 464, 903–907 (2010).

    Article  CAS  PubMed  Google Scholar 

  18. Green, D. & Swets, J. Signal Detection Theory and Psychophysics (Wiley, 1966).

  19. Manns, J.R., Hopkins, R.O., Reed, J.M., Kitchener, E.G. & Squire, L.R. Recognition memory and the human hippocampus. Neuron 37, 171–180 (2003).

    Article  CAS  PubMed  Google Scholar 

  20. Rutishauser, U., Schuman, E.M. & Mamelak, A.N. Online detection and sorting of extracellularly recorded action potentials in human medial temporal lobe recordings, in vivo. J. Neurosci. Methods 154, 204–224 (2006).

    Article  PubMed  Google Scholar 

  21. Viskontas, I.V., Knowlton, B.J., Steinmetz, P.N. & Fried, I. Differences in mnemonic processing by neurons in the human hippocampus and parahippocampal regions. J. Cogn. Neurosci. 18, 1654–1662 (2006).

    Article  PubMed  Google Scholar 

  22. Britten, K.H., Newsome, W.T., Shadlen, M.N., Celebrini, S. & Movshon, J.A. A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis. Neurosci. 13, 87–100 (1996).

    Article  CAS  PubMed  Google Scholar 

  23. Hentschke, H. & Stuttgen, M.C. Computation of measures of effect size for neuroscience data sets. Eur. J. Neurosci. 34, 1887–1894 (2011).

    Article  PubMed  Google Scholar 

  24. Bogacz, R., Brown, E., Moehlis, J., Holmes, P. & Cohen, J.D. The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychol. Rev. 113, 700–765 (2006).

    Article  PubMed  Google Scholar 

  25. Vickers, D. Decision Processes in Visual Perception (Academic Press, New York, 1979).

  26. Fried, I., MacDonald, K.A. & Wilson, C.L. Single neuron activity in human hippocampus and amygdala during recognition of faces and objects. Neuron 18, 753–765 (1997).

    Article  CAS  PubMed  Google Scholar 

  27. Wixted, J.T. et al. Sparse and distributed coding of episodic memory in neurons of the human hippocampus. Proc. Natl. Acad. Sci. USA 111, 9621–9626 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Marr, D. Simple memory: a theory for archicortex. Philos. Trans. R. Soc. Lond. B Biol. Sci. 262, 23–81 (1971).

    Article  CAS  PubMed  Google Scholar 

  29. Macmillan, N.A. & Creelman, C.D. Detection Theory (Lawrence Associates, Mahwah, New Jersey, 2005).

  30. Zhang, Y. et al. Object decoding with attention in inferior temporal cortex. Proc. Natl. Acad. Sci. USA 108, 8850–8855 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Wixted, J.T. Dual-process theory and signal-detection theory of recognition memory. Psychol. Rev. 114, 152–176 (2007).

    Article  PubMed  Google Scholar 

  32. Clark, S.E. & Gronlund, S.D. Global matching models of recognition memory: How the models match the data. Psychon. Bull. Rev. 3, 37–60 (1996).

    Article  CAS  PubMed  Google Scholar 

  33. Gold, J.I. & Shadlen, M.N. The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–574 (2007).

    Article  CAS  PubMed  Google Scholar 

  34. Ratcliff, R. A theory of memory retrieval. Psychol. Rev. 85, 59–108 (1978).

    Article  Google Scholar 

  35. Hanks, T.D. et al. Distinct relationships of parietal and prefrontal cortices to evidence accumulation. Nature 520, 220–223 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Viskontas, I.V., Ekstrom, A.D., Wilson, C.L. & Fried, I. Characterizing interneuron and pyramidal cells in the human medial temporal lobe in vivo using extracellular recordings. Hippocampus 17, 49–57 (2007).

    Article  PubMed  Google Scholar 

  37. Mitchell, J.F., Sundberg, K.A. & Reynolds, J.H. Differential attention-dependent response modulation across cell classes in macaque visual area V4. Neuron 55, 131–141 (2007).

    Article  CAS  PubMed  Google Scholar 

  38. Peyrache, A. et al. Spatiotemporal dynamics of neocortical excitation and inhibition during human sleep. Proc. Natl. Acad. Sci. USA 109, 1731–1736 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  39. Vigneswaran, G., Kraskov, A. & Lemon, R.N. Large identified pyramidal cells in macaque motor and premotor cortex exhibit “thin spikes”: implications for cell type classification. J. Neurosci. 31, 14235–14242 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Robbins, A.A., Fox, S.E., Holmes, G.L., Scott, R.C. & Barry, J.M. Short duration waveforms recorded extracellularly from freely moving rats are representative of axonal activity. Front. Neural Circ. 7, 181 (2013).

    Google Scholar 

  41. Stuart, G., Schiller, J. & Sakmann, B. Action potential initiation and propagation in rat neocortical pyramidal neurons. J. Physiol. (Lond.) 505, 617–632 (1997).

    Article  CAS  Google Scholar 

  42. Hamann, S. The human amygdala and Memory. in The Human Amydala (eds. Whalen, P.J. & Phelps, E.A.) 177–203 (The Guilford Press, New York, 2009).

  43. Weierich, M.R., Wright, C.I., Negreira, A., Dickerson, B.C. & Barrett, L.F. Novelty as a dimension in the affective brain. Neuroimage 49, 2871–2878 (2010).

    Article  PubMed  Google Scholar 

  44. Metcalfe, J. Metamemory. in Learning and Memory: a Comprehensive Reference (ed. Roediger, H.L.) 349–362 (Elsevier, Oxford, 2008).

  45. Metcalfe, J. Evolution of metacognition. in Handbook of Metamemory and Memory (eds. Dunlovsky, J. & Bjork, R.) 29–46 (Psychology Press, New York, 2008).

  46. Hampton, R.R. Rhesus monkeys know when they remember. Proc. Natl. Acad. Sci. USA 98, 5359–5362 (2001).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Perry, C.J. & Barron, A.B. Honey bees selectively avoid difficult choices. Proc. Natl. Acad. Sci. USA 110, 19155–19159 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Foote, A.L. & Crystal, J.D. Metacognition in the rat. Curr. Biol. 17, 551–555 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Middlebrooks, P.G. & Sommer, M.A. Metacognition in monkeys during an oculomotor task. J. Exp. Psychol. Learn. Mem. Cogn. 37, 325–337 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  50. Fortin, N.J., Wright, S.P. & Eichenbaum, H. Recollection-like memory retrieval in rats is dependent on the hippocampus. Nature 431, 188–191 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Brainard, D.H. The Psychophysics Toolbox. Spat. Vis. 10, 433–436 (1997).

    Article  CAS  PubMed  Google Scholar 

  52. Nelson, T.O. A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychol. Bull. 95, 109–133 (1984).

    Article  CAS  PubMed  Google Scholar 

  53. Ratcliff, R., Gronlund, S.D. & Sheu, C.F. Testing global memory models using ROC curves. Psychol. Rev. 99, 518–535 (1992).

    Article  CAS  PubMed  Google Scholar 

  54. Pouzat, C., Mazor, O. & Laurent, G. Using noise signature to optimize spike-sorting and to assess neuronal classification quality. J. Neurosci. Methods 122, 43–57 (2002).

    Article  PubMed  Google Scholar 

  55. Harris, K.D., Henze, D.A., Csicsvari, J., Hirase, H. & Buzsaki, G. Accuracy of tetrode spike separation as determined by simultaneous intracellular and extracellular measurements. J. Neurophysiol. 84, 401–414 (2000).

    Article  CAS  PubMed  Google Scholar 

  56. Schmitzer-Torbert, N., Jackson, J., Henze, D., Harris, K. & Redish, A.D. Quantitative measures of cluster quality for use in extracellular recordings. Neuroscience 131, 1–11 (2005).

    Article  CAS  PubMed  Google Scholar 

  57. Diba, K. & Buzsaki, G. Hippocampal network dynamics constrain the time lag between pyramidal cells across modified environments. J. Neurosci. 28, 13448–13456 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  58. Olejnik, S. & Algina, J. Generalized eta and omega squared statistics: measures of effect size for some common research designs. Psychol. Methods 8, 434–447 (2003).

    Article  PubMed  Google Scholar 

  59. Meyers, E.M. The neural decoding toolbox. Front. Neuroinform. 7, 8 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  60. Quian Quiroga, R. & Panzeri, S. Extracting information from neuronal populations: information theory and decoding approaches. Nat. Rev. Neurosci. 10, 173–185 (2009).

    Article  CAS  PubMed  Google Scholar 

  61. Rutishauser, U. et al. Single-neuron correlates of atypical face processing in autism. Neuron 80, 887–899 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  62. Holt, G.R., Softky, W.R., Koch, C. & Douglas, R.J. Comparison of discharge variability in vitro and in vivo in cat visual cortex neurons. J. Neurophysiol. 75, 1806–1814 (1996).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank J. Kaminski, R. Adolphs, C. Anastassiou, U. Maoz, J. Wertheimer and W. Einhaeuser for discussion, Z. Fu for spike sorting, C. Heller for performing some of the surgeries, the staff of the Epilepsy Monitoring Units at Huntington Memorial Hospital and Cedars-Sinai for invaluable assistance, particularly J. Schmidt. We thank K. Birch and H. Babu for assistance with patient care and surgery, and L. Philpott and M.-T. Le for neuropsychological testing. This work was supported by the Cedars-Sinai Medical Center Department of Neurosurgery (to U.R.), National Institute of Mental Health Conte Center at Caltech (P50 MH094258), and the Gustavus and Louise Pfeiffer Research Foundation (to U.R.).

Author information

Authors and Affiliations

Authors

Contributions

U.R. and A.N.M. designed the experiments. U.R. and O.T. performed experiments. U.R., M.K. and S.Y. performed analysis. A.N.M. and I.B.R. performed surgery. J.M.C. provided patient care. U.R. and A.N.M. wrote the paper. All of the authors discussed the results at all stages of the project.

Corresponding author

Correspondence to Ueli Rutishauser.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Spike sorting and recording quality assessment.

(a-e) Metrics quantifying individual clusters. (a) Histogram of proportion of inter-spike intervals (ISIs) which are shorter than 3ms. The large majority of clusters had less than 0.5% of such short ISIs. (b) Histogram of mean firing rates. (c) Histogram of the SNR of the mean waveform peak of each unit. (d) Histogram of the SNR of the entire waveform of all units. (e) Histogram of CV2 values of all units. (f) Pairwise distance between all possible pairs of units on all wires where more than 1 cluster was isolated. Distances are expressed in units of s.d. after normalizing the data such that the distribution of waveforms around their mean is equal to 1. (g) Isolation distance of all units for which this metric was defined (n = 746, median 35.0). There was no significant difference in isolation distance between VS and MS cells (p=0.59, two-sample Kolmogorov-Smirnov test). (h) Absence of correlation between isolation distance and response strength, as quantified by ω2 for visual category regressor, for all VS cells. R2 = 0.009, p = 0.37. (i) Absence of correlation between isolation distance and response strength, as quantified by ω2 for novel/familiar regressor, for all MS cells. R2 = 0.03, p = 0.25. (j) Histogram of how many units were identified on each active wire (only wires with at least one unit identified are counted). The average yield per wire with at least one unit was 2.4 (range 1-7).

Supplementary Figure 2 Bootstrap statistics for number of cells selected.

Estimate of chance values (null distribution) of number of cells observed compared to the actual number of cells found. The red line indicates the actual number of cells observed. The null distribution (blue) was estimated by re-running the identical selection procedure after first randomly permuting the order of the labels assigned to each trial. This permutation procedure destroys to association between the spiking response and trial identity, but keeps everything else intact (number of behavioral choices, number of times a stimulus was seen). The p-value is equal to the number of chance observations (blue) which are larger than that observed (red). In cases where no chance values exceeded those observed, we set p-values to 1/B with B the number of bootstrap runs (B = 1000). (a) Significance of the number of MS and VS cells we identified for behavioral group 1 (top, p = 0.001 and p = 0.001, respectively) and group 2 (bottom, p = 0.001 and p = 0.001, respectively). (b) Significance of the number of cells that qualified as both MS and VS cells in behavioral group 1 (top, p = 0.001) and group 2 (bottom, p = 0.001). While rare, the number of cells observed was well above chance.

Supplementary Figure 3 Confidence encoding by NS and MS neurons, shown separately for different brain areas and hemispheres.

Confidence encoding by NS and MS neurons in the hippocampus (HF, panels a-c), amygdala (AMY, d-f), left side (panels g-i), and right side (j-l). The result shown in Fig 3 held when considering MS neurons separately in the hippocampus (d-f; n = 34, p = 0.00015, p = 0.0014, and p = 0.024 for all, NS, and FS neurons, respectively), amygdala (a-c; n = 31, p = 0.0024, p = 0.024, p = 0.017 for all, NS, and FS neurons, respectively), left side (g-i, n = 26, p = 0.0032, p = 0.0033, and p = 0.056 for all, NS, and FS neurons, respectively), and right side (j-l, n = 39, p = 0.000069, p = 0.0015, and p = 0.011 for all, NS, and FS neurons, respectively). (m-o) Bootstrap estimate of the null distribution and significance of difference between AUC for high and low confidence trials. Observed values (red line) are identical to those shown in Fig 3c-e. The null distribution was estimated by randomly scrambling the trials between high and low confidence. We ran 1000 runs, for each of which the average difference in AUC across all cells was calculated in the same manner as in Fig 3. (p) Further example neuronal ROCs for MS neurons, shown for high-confidence trials only, compare to Fig 2. All errorbars are ±s.e. across neurons. All p-values are one-tailed paired t-tests comparing the AUC of high and low confidence trials.

Supplementary Figure 4 Example cell that qualifies as both a VS and MS cell.

(a) Raster of all trials, grouped into familiar (top) and novel (bottom) trials. Color indicates visual category. (b) PSTH of familiar (top) and novel (bottom) trials. (c) Mean response as a function of visual category (color) and familiarity and novelty. This cell increased its firing rate significantly only for familiar landscapes (pairwise tests novel vs familiar for each category, not corrected for multiple comparisons). Note that these cells were selected independently as MS and VS cells. The contrasts shown in this panel were not used to select cells.

Supplementary Figure 5 Comparison of ability of VS cells to distinguish between visual categories as a function of confidence and familiarity.

(a,b) Show identical analysis to Fig 5a,b, but only including neurons VS neurons with baseline firing rate >1Hz (n = 78). There was no significant difference (p = 0.91 and p = 0.43 using paired sign-test and p = 0.75 and p = 0.48 using bootstrap test for confidence and familiarity, respectively).

Supplementary Figure 6 Population effect size estimation using a 2-way model with interaction term.

No interaction between the two main factors category and familiarity was found. (a-c) shows effect size for both main factors (category, familiarity) as well as their interaction for all neurons (a), only MS neurons (b) and only VS neurons (c). Dashed horizontal lines indicate the 99% confidence intervals of the null distribution (200 bootstrap runs each) for each factor (color). Note that the interaction term (red) never becomes significantly positive.

Supplementary Figure 7 Properties of extracellular waveforms (EWs).

(a) Normalized EWs of a random subset of all recorded neurons (50 waveforms are shown). (b) Histogram of the trough-to-peak time d of all units (n = 1065). The distribution was significantly bimodal (Hartigan’s dip test, p<1e-5). (c) Scatter plot of firing rate vs trough-to-peak time. Notice how, at all firing rates, there appear at least two clusters with different trough-to-peak time. (d-f) Comparison of waveforms between MS neurons and VS neurons. (d) Histogram of trough-to-peak time for MS units only (top) and VS units only (bottom). Only the distribution for VS neurons was significantly bimodal (Hartigan’s dip test, p = 0.004 vs p = 0.34 for MS neurons). (e) Waveforms of all MS (left) and VS (right) units. Colors mark short (red) and long (blue) waveforms. (f) Quantification of proportion of short and long waveforms for MS and VS neurons, respectively. The proportion of short and long waveforms was significantly different only for MS neurons (χ2 comparison of proportions, p = 2.2e-5 vs p = 0.12 for MS and VS neurons, respectively).

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–7 and Supplementary Tables 1–4 (PDF 587 kb)

Supplementary Methods Checklist (PDF 434 kb)

Source data

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rutishauser, U., Ye, S., Koroma, M. et al. Representation of retrieval confidence by single neurons in the human medial temporal lobe. Nat Neurosci 18, 1041–1050 (2015). https://doi.org/10.1038/nn.4041

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.4041

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing