Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

The medial temporal lobe supports the quality of visual short-term memory representation

Nature Human Behaviour volume 7, pages 627–641 (2023)Cite this article

Subjects

Abstract

The quality of short-term memory (STM) underlies our ability to recall the exact details of a recent event, yet how the human brain enables this core cognitive function remains poorly understood. Here we use multiple experimental approaches to test the hypothesis that the quality of STM, such as its precision or fidelity, relies on the medial temporal lobe (MTL), a region commonly associated with the ability to distinguish similar information remembered in long-term memory. First, with intracranial recordings, we find that delay-period MTL activity retains item-specific STM content that is predictive of subsequent recall precision. Second, STM recall precision is associated with an increase in the strength of intrinsic MTL-to-neocortical functional connections during a brief retention interval. Finally, perturbing the MTL through electrical stimulation or surgical removal can selectively reduce STM precision. Collectively, these findings provide converging evidence that the MTL is critically involved in the quality of STM representation.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Visual STM task and participants’ task performance.
Fig. 2: Delay-period MTL activity retains item-specific STM content to support subsequent recall precision.
Fig. 3: Directed functional connectivity between MTL and non-MTL cortical recording sites is associated with STM recall precision.
Fig. 4: Direct electrical stimulation of the MTL reduces STM precision.
Fig. 5: MTL lesion reduces STM precision.

Similar content being viewed by others

Data availability

Extracted raw iEEG and behavioural data used in this study are available via NIH server at https://research.ninds.nih.gov/zaghloul-lab/downloads.

Code availability

Custom code that supports the findings of this study is available from W.X. upon request.

References

  1. Jeneson, A. & Squire, L. R. Working memory, long-term memory, and medial temporal lobe function. Learn. Mem. 19, 15–25 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  2. Scoville, W. B. & Milner, B. Loss of recent memory after bilateral hippocampal lesions. J. Neurol. Neurosurg. Psychiatry 20, 11–21 (1957).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Milner, B., Corkin, S. & Teuber, H. L. Further analysis of the hippocampal amnesic syndrome: 14-year follow-up study of H.M. Neuropsychologia 6, 215–234 (1968).

    Article  Google Scholar 

  4. Atkinson, R. C. & Shiffrin, R. M. Human memory: a proposed system and its control processes. Psychol. Learn. Motiv. 2, 89–195 (1968).

    Article  Google Scholar 

  5. Nairne, J. S. Remembering over the short-term: the case against the standard model. Annu. Rev. Psychol. 53, 53–81 (2002).

    Article  PubMed  Google Scholar 

  6. Ma, W. J., Husain, M. & Bays, P. M. Changing concepts of working memory. Nat. Neurosci. 17, 347–356 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Zhang, W. & Luck, S. J. Discrete fixed-resolution representations in visual working memory. Nature 453, 233–235 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Bays, P. M. & Husain, M. Dynamic shifts of limited working memory resources in human vision. Science 321, 851–854 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Marr, D. Simple memory: a theory for archicortex. Philos. Trans. R. Soc. Lond. B 262, 23–81 (1971).

    Article  CAS  Google Scholar 

  10. Rolls, E. T. A quantitative theory of the functions of the hippocampal CA3 network in memory. Front. Cell. Neurosci. 7, 98 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Cappiello, M., Xie, W., David, A., Bikson, M. & Zhang, W. Transcranial direct current stimulation modulates pattern separation. Neuroreport 27, 826–832 (2016).

    Article  PubMed  Google Scholar 

  12. Bakker, A., Kirwan, C. B., Miller, M. & Stark, C. E. L. Pattern separation in the human hippocampal CA3 and dentate gyrus. Science 319, 1640–1642 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Aimone, J. B., Deng, W. & Gage, F. H. Resolving new memories: a critical look at the dentate gyrus, adult neurogenesis, and pattern separation. Neuron 70, 589–596 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Yonelinas, A. P. The hippocampus supports high-resolution binding in the service of perception, working memory and long-term memory. Behav. Brain Res. 254, 34–44 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  15. Goodrich, R. I. & Yonelinas, A. P. The medial temporal lobe supports sensing-based visual working memory. Neuropsychologia 89, 485–494 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  16. Borders, A. A., Ranganath, C. & Yonelinas, A. P. The hippocampus supports high-precision binding in visual working memory. Hippocampus https://doi.org/10.1002/hipo.23401 (2021).

  17. Xie, W., Park, H.-B., Zaghloul, K. A. & Zhang, W. Correlated individual differences in the estimated precision of working memory and long-term memory: commentary on the study by Biderman, Luria, Teodorescu, Hajaj, and Goshen-Gottstein (2019). Psychol. Sci. 31, 345–348 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  18. Boran, E., Hilfiker, P., Stieglitz, L., Sarnthein, J. & Klaver, P. Persistent neuronal firing in the medial temporal lobe supports performance and workload of visual working memory in humans. NeuroImage https://doi.org/10.1016/j.neuroimage.2022.119123 (2022).

  19. Kamiński, J. et al. Persistently active neurons in human medial frontal and medial temporal lobe support working memory. Nat. Neurosci. 20, 590–601 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Kornblith, S., Quian Quiroga, R., Koch, C., Fried, I. & Mormann, F. Persistent single-neuron activity during working memory in the human medial temporal lobe. Curr. Biol. 27, 1026–1032 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Libby, L. A., Hannula, D. E. & Ranganath, C. Medial temporal lobe coding of item and spatial information during relational binding in working memory. J. Neurosci. 34, 14233–14242 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Boran, E. et al. Persistent hippocampal neural firing and hippocampal-cortical coupling predict verbal working memory load. Sci. Adv. 5, eaav3687 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  23. Barense, M. D., Gaffan, D. & Graham, K. S. The human medial temporal lobe processes online representations of complex objects. Neuropsychologia 45, 2963–2974 (2007).

    Article  PubMed  Google Scholar 

  24. Xie, W. & Zaghloul, K. A. Visual and semantic contributions to visual short-term memory. Trends Cogn. Sci. 25, 270–271 (2021).

    Article  PubMed  Google Scholar 

  25. Sreenivasan, K. K. & D’Esposito, M. The what, where and how of delay activity. Nat. Rev. Neurosci. 20, 466–481 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Jeneson, A., Wixted, J. T., Hopkins, R. O. & Squire, L. R. Visual working memory capacity and the medial temporal lobe. J. Neurosci. 32, 3584–3589 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Goodrich, R. I., Baer, T. L., Quent, J. A. & Yonelinas, A. P. Visual working memory impairments for single items following medial temporal lobe damage. Neuropsychologia 134, 107227 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  28. Warren, D. E., Duff, M. C., Tranel, D. & Cohen, N. J. Medial temporal lobe damage impairs representation of simple stimuli. Front. Hum. Neurosci. 4, 35 (2010).

    PubMed  PubMed Central  Google Scholar 

  29. Warren, D. E., Duff, M. C., Cohen, N. J. & Tranel, D. Hippocampus contributes to the maintenance but not the quality of visual information over time. Learn. Mem. 22, 6–10 (2014).

    Article  PubMed  Google Scholar 

  30. Shrager, Y., Levy, D. A., Hopkins, R. O. & Squire, L. R. Working memory and the organization of brain systems. J. Neurosci. 28, 4818–4822 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Baddeley, A., Allen, R. & Vargha-Khadem, F. Is the hippocampus necessary for visual and verbal binding in working memory? Neuropsychologia 48, 1089–1095 (2010).

    Article  PubMed  Google Scholar 

  32. Christophel, T. B., Klink, P. C., Spitzer, B., Roelfsema, P. R. & Haynes, J.-D. The distributed nature of working memory. Trends Cogn. Sci. 21, 111–124 (2017).

    Article  PubMed  Google Scholar 

  33. Eriksson, J., Vogel, E. K., Lansner, A., Bergström, F. & Nyberg, L. Neurocognitive architecture of working memory. Neuron 88, 33–46 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Jonides, J. et al. The mind and brain of short-term memory. Annu. Rev. Psychol. 59, 193–224 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  35. Postle, B. R. How Does the brain keep information “in mind”? Curr. Dir. Psychol. Sci. 25, 151–156 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  36. Xie, W. & Zhang, W. Dissociations of the number and precision of visual short-term memory representations in change detection. Mem. Cogn. 45, 1423–1437 (2017).

    Article  Google Scholar 

  37. deBettencourt, M. T., Keene, P. A., Awh, E. & Vogel, E. K. Real-time triggering reveals concurrent lapses of attention and working memory. Nat. Hum. Behav. 3, 808–816 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  38. Schurgin, M. W., Wixted, J. T. & Brady, T. F. Psychophysical scaling reveals a unified theory of visual memory strength. Nat. Hum. Behav. 4, 1156–1172 (2020).

    Article  PubMed  Google Scholar 

  39. Ester, E. F., Anderson, D. E., Serences, J. T. & Awh, E. A neural measure of precision in visual working memory. J. Cogn. Neurosci. 25, 754–761 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  40. Kriegeskorte, N. & Wei, X. X. Neural tuning and representational geometry. Nat. Rev. Neurosci. 22, 703–718 (2021).

    Article  CAS  PubMed  Google Scholar 

  41. Manning, J. R., Jacobs, J., Fried, I. & Kahana, M. J. Broadband shifts in local field potential power spectra are correlated with single-neuron spiking in humans. J. Neurosci. 29, 13613–13620 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Rose, N. S. et al. Reactivation of latent working memories with transcranial magnetic stimulation. Science 354, 1136–1139 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Ester, E. F., Sprague, T. C. & Serences, J. T. Parietal and frontal cortex encode stimulus-specific mnemonic representations during visual working memory. Neuron 87, 893–905 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Lundqvist, M., Herman, P. & Miller, E. K. Working memory: delay activity, yes! Persistent activity? Maybe not. J. Neurosci. 38, 7013–7019 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Panichello, M. F. & Buschman, T. J. Shared mechanisms underlie the control of working memory and attention. Nature 592, 601–605 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Kriegeskorte, N., Formisano, E., Sorger, B. & Goebel, R. Individual faces elicit distinct response patterns in human anterior temporal cortex. Proc. Natl Acad. Sci. USA 104, 20600–20605 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Gluth, S. & Meiran, N. Leave-one-trial-out, LOTO, a general approach to link single-trial parameters of cognitive models to neural data. eLife 8, 1–39 (2019).

    Article  Google Scholar 

  48. Wimmer, K., Nykamp, D. Q., Constantinidis, C. & Compte, A. Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nat. Neurosci. 17, 431–439 (2014).

    Article  CAS  PubMed  Google Scholar 

  49. Galeano Weber, E. M., Hahn, T., Hilger, K. & Fiebach, C. J. Distributed patterns of occipito-parietal functional connectivity predict the precision of visual working memory. NeuroImage 146, 404–418 (2017).

    Article  PubMed  Google Scholar 

  50. Bays, P. M. Noise in neural populations accounts for errors in working memory. J. Neurosci. 34, 3632–3645 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Chapeton, J. I., Haque, R., Wittig, J. H., Inati, S. K. & Zaghloul, K. A. Large-scale communication in the human brain is rhythmically modulated through alpha coherence. Curr. Biol. 29, 2801–2811 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Nikolić, D., Mureşan, R. C., Feng, W. & Singer, W. Scaled correlation analysis: a better way to compute a cross-correlogram. Eur. J. Neurosci. 35, 742–762 (2012).

    Article  PubMed  Google Scholar 

  53. Bastos, A. M. et al. Visual areas exert feedforward and feedback influences through distinct frequency channels. Neuron 85, 390–401 (2015).

    Article  CAS  PubMed  Google Scholar 

  54. Michalareas, G. et al. Alpha-beta and gamma rhythms subserve feedback and feedforward influences among human visual cortical areas. Neuron 89, 384–397 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Vezoli, J. et al. Brain rhythms define distinct interaction networks with differential dependence on anatomy. Neuron 109, 3862–3878.e5 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Barnett, L. & Seth, A. K. The MVGC multivariate Granger causality toolbox: a new approach to Granger-causal inference. J. Neurosci. Methods 223, 50–68 (2014).

    Article  PubMed  Google Scholar 

  57. Bastos, A. M., Vezoli, J. & Fries, P. Communication through coherence with inter-areal delays. Curr. Opin. Neurobiol. 31, 173–180 (2015).

    Article  CAS  PubMed  Google Scholar 

  58. Xie, W. & Zhang, W. Negative emotion enhances mnemonic precision and subjective feelings of remembering in visual long-term memory. Cognition 166, 73–83 (2017).

    Article  PubMed  Google Scholar 

  59. Xie, W., Bainbridge, W. A., Inati, S. K., Baker, C. I. & Zaghloul, K. A. Memorability of words in arbitrary verbal associations modulates memory retrieval in the anterior temporal lobe. Nat. Hum. Behav. 4, 937–948 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  60. Gold, J. M. et al. Reduced capacity but spared precision and maintenance of working memory representations in schizophrenia. Arch. Gen. Psychiatry 67, 570–577 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  61. Xie, W., Berry, A., Lustig, C., Deldin, P. & Zhang, W. Poor sleep quality and compromised visual working memory capacity. J. Int. Neuropsychol. Soc. 25, 583–594 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  62. Hardman, K. O., Vergauwe, E. & Ricker, T. J. Categorical working memory representations are used in delayed estimation of continuous colors. J. Exp. Psychol. Hum. Percept. Perform. 43, 30–54 (2017).

    Article  PubMed  Google Scholar 

  63. Xie, W. & Zhang, W. Negative emotion boosts quality of visual working memory representation. Emotion 16, 760–774 (2016).

    Article  PubMed  Google Scholar 

  64. Kruschke, J. Doing Bayesian Data Analysis. A Tutorial with R, JAGS, and Stan (Academic Press, 2015).

  65. Xie, W., Cappiello, M., Meng, M., Rosenthal, R. & Zhang, W. ADRA2B deletion variant and enhanced cognitive processing of emotional information: A meta-analytical review. Neurosci. Biobehav. Rev. 92, 402–416 (2018).

    Article  CAS  PubMed  Google Scholar 

  66. Rosenthal, R. & Rubin, D. B. r equivalent: a simple effect size indicator. Psychol. Methods 8, 492–496 (2003).

    Article  PubMed  Google Scholar 

  67. Liu, J. et al. Stable maintenance of multiple representational formats in human visual short-term memory. Proc. Natl Acad. Sci. USA 117, 32329–32339 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  68. Axmacher, N., Schmitz, D. P., Wagner, T., Elger, C. E. & Fell, J. Interactions between medial temporal lobe, prefrontal cortex, and inferior temporal regions during visual working memory: a combined intracranial EEG and functional magnetic resonance imaging study. J. Neurosci. 28, 7304–7312 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  69. Nadel, L. & Hardt, O. Update on memory systems and processes. Neuropsychopharmacology 36, 251–273 (2011).

    Article  PubMed  Google Scholar 

  70. Thyer, W. et al. Storage in visual working memory recruits a content-independent pointer system. Psychol. Sci. https://osf.io/uhbx5/ (2022).

  71. Xie, W. et al. Schizotypy is associated with reduced mnemonic precision in visual working memory. Schizophr. Res. 193, 91–97 (2018).

    Article  PubMed  Google Scholar 

  72. Bonnen, T., Yamins, D. L. K. & Wagner, A. D. When the ventral visual stream is not enough: a deep learning account of medial temporal lobe involvement in perception. Neuron 109, 2755–2766.e6 (2021).

    Article  CAS  PubMed  Google Scholar 

  73. Bird, C. M. & Burgess, N. The hippocampus and memory: insights from spatial processing. Nat. Rev. Neurosci. 9, 182–194 (2008).

    Article  CAS  PubMed  Google Scholar 

  74. van den Berg, R., Shin, H., Chou, W.-C., George, R. & Ma, W. J. Variability in encoding precision accounts for visual short-term memory limitations. Proc. Natl Acad. Sci. USA 109, 8780–8785 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  75. Zhao, Y., Kuai, S., Zanto, T. P. & Ku, Y. Neural correlates underlying the precision of visual working memory. Neuroscience 425, 301–311 (2020).

    Article  CAS  PubMed  Google Scholar 

  76. Galeano Weber, E. M., Peters, B., Hahn, T., Bledowski, C. & Fiebach, C. J. Superior intraparietal sulcus controls the variability of visual working memory precision. J. Neurosci. 36, 5623–5635 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  77. Sutterer, D., Rosca, C. G. & Woodman, G. F. Does motor noise contaminate estimates of the precision of visual working memory? Vis. Cogn. 30, 195–201 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  78. Johnson, E. L. et al. Dynamic frontotemporal systems process space and time in working memory. PLoS Biol. 16, 1–22 (2018).

    Article  Google Scholar 

  79. Chu, C. J. et al. Emergence of stable functional networks in long-term human electroencephalography. J. Neurosci. 32, 2703–2713 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  80. Chapeton, J. I., Inati, S. K. & Zaghloul, K. A. Stable functional networks exhibit consistent timing in the human brain. Brain 140, 628–640 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  81. Dimakopoulos, V., Mégevand, P., Stieglitz, L. H., Imbach, L. & Sarnthein, J. Information flows from hippocampus to auditory cortex during replay of verbal working memory items. eLife 11, 1–19 (2022).

    Article  Google Scholar 

  82. Trotta, M. S. et al. Surface based electrode localization and standardized regions of interest for intracranial EEG. Hum. Brain Mapp. 39, 709–721 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  83. Yassa, M. A. & Stark, C. E. L. Pattern separation in the hippocampus. Trends Neurosci. 34, 515–525 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  84. Mormann, F. et al. Latency and selectivity of single neurons indicate hierarchical processing in the human medial temporal lobe. J. Neurosci. 28, 8865–8872 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  85. Xie, W., Lu Sing, J. L., Martinez-Flores, A. & Zhang, W. Induced negative arousal modulates the speed of visual working memory consolidation. Emotion 22, 179–197 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  86. El-Kalliny, M. M. et al. Changing temporal context in human temporal lobe promotes memory of distinct episodes. Nat. Commun. 10, 1–10 (2019).

    Article  CAS  Google Scholar 

  87. Bays, P. M. Evaluating and excluding swap errors in analogue tests of working memory. Sci. Rep. 6, 5 (2016).

    Article  Google Scholar 

  88. Xie, W., Wittig, J. H. & Zaghloul, K. A. in Intracranial EEG: A Guide for Cognitive Neuroscientists (ed. Axmacher, N.) 1–10 (Springer Nature, 2023).

  89. Zhou, X., Li, M., Zhou, H., Li, L. & Cui, J. Item-wise interindividual brain–behavior correlation in task neuroimaging analysis. Front. Neurosci. 12, 1–17 (2018).

    Article  Google Scholar 

  90. Ito, S. et al. Extending transfer entropy improves identification of effective connectivity in a spiking cortical network model. PLoS ONE 6, e27431 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank all the participants who selflessly volunteered their time for this study. We also thank V. Sreekumar, D. Youssef, M. Baumhauer and A. Phan for their technical support and comments. This work was made possible by the Intramural Research Program of the National Institute of Neurological Disorders and Stroke (ZIA-NS003144, K.A.Z.) and a grant from the National Institute of Mental Health (R01MH117132, W.Z.). W.X. is a recipient of the National Institute of Neurological Disorders and Stroke Competitive Postdoctoral Fellowship Award and the NIH Pathway to Independence Award (K99NS126492, W.X.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Surgical Neurology Branch, National Institute of Neurological Disorders and Stroke, National Institutes of Health, Bethesda, MD, USA

    Weizhen Xie, Julio I. Chapeton, Srijan Bhasin, Christopher Zawora, John H. Wittig Jr., Sara K. Inati & Kareem A. Zaghloul

  2. Department of Psychology, University of California, Riverside, CA, USA

    Weiwei Zhang

Authors
  1. Weizhen Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Julio I. Chapeton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Srijan Bhasin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christopher Zawora
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John H. Wittig Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sara K. Inati
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Weiwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kareem A. Zaghloul
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

W.X. conceptualized the study, collected and analysed the data, and wrote the paper, with suggestions from W.Z. and K.A.Z. Both S.B. and C.Z. contributed to data collection and data management. S.K.I. oversaw iEEG data acquisition, provided clinical assessment of iEEG waveforms and monitored the brain stimulation experiment. J.H.W. provided technical support for the brain stimulation experiment. J.I.C. provided codes and support for the analysis of functional connectivity. K.A.Z. performed all the surgical procedures and supervised the whole study. W.Z. provided additional support in the conceptualization and interpretation of the findings. All authors provided critical comments.

Corresponding authors

Correspondence to Weizhen Xie or Kareem A. Zaghloul.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks Gui Xue, Johannes Sarnthein and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 A model to measure STM precision.

Based on a mixture model7, participants’ recall errors (black) can be considered a combination of retained STM information with some variability (imprecision, red, estimated as SD and denoted as s.d. in the main text) and empirically failed memory responses uniformly distributed in the feature space (green). The latter is inversely related to the probability of recall success (Pm). Here, we use this mixture model as a measurement model to quantify the overall recall variability74, after factoring our unsuccessful recall responses that could be attributed to attentional lapses37, mis-binding87, or random guessing36,58, etc. It’s worth noting that as we randomly select the study colors and rotate the color wheel with a random degree across trials, these unsuccessful recall responses empirically would be distributed uniformly across the color space7,36,58.

Extended Data Fig. 2 Additional analyses of neural similarity across trials and in relation to recall behavior.

(a) A standardization procedure was used to obtain a proper effect size estimate for the correlation strength between stimulus-predicted item similarity pattern and the observed neural similarity pattern across trials at each time window of the task. At each time point of the task within each subject, the observed Spearman rank-order correlation values of these two variables are contrasted against a null distribution created by shuffling the trial labels of the cued item across 5,000 iterations. The standardized correlation strength is then calculated based on the mean and standard deviation of this shuffled distribution. This approach normalized the intrinsic noise in the data at each time window of the task, facilitating group-level comparison. (b) Item-specific information during STM retention in the MTL can be primarily attributed to high-frequency activity at 70–150 Hz based on the stimulus–neural association analysis outlined in Fig. 2. Based on the power values at different canonical frequency bands, we calculated the standardized correlation between the trial-by-trial stimulus similarity pattern and the trial-by-trial neural similarity pattern to capture information about the cued item in the multivariate neural data. Across participants, we find that MTL signals are significantly associated with the cued item during the STM delay period primarily in the 70–150 Hz high-frequency band (cluster-based correction, pcorrected < 0.05, outlined in an orange rectangle), with little evidence showing item-specific information in other lower frequency bands. (c) Time-varying analysis of the neural separation score and participants’ absolute recall errors suggests that the neural separation score during the STM delay period, especially during the first 500-ms following stimulus offset, can well predict later recall precision. This suggests that our findings in Fig. 2e can be generalized across adjacent timepoints during the STM retention interval. Error bars represent the s.e.m. across participants. Significant timepoints after cluster-based correction for multiple comparisons at the 0.05 level are marked in yellow.

Extended Data Fig. 3 Stimulus–neural association is robust to different analytical procedures in the current study.

As the stimulus on each trial would elicit a pattern of neural activity, the neural similarity across trials can be accessed either at the trial level or at the item level averaged across trials (the left panel in the figure). It is known that this analytical choice would lead to different magnitudes of the stimulus–neural correlation, as the correlation metric tends to scale with the number of data points included in its calculation89. It is therefore improper to directly interpret the magnitude of the raw correlation values or their Fisher’s transformation values (the middle panel in the figure) as effect size estimates. By standardizing these correlation estimates at each time point against data with shuffled trial labels (see Extended Data Fig. 2a for an example), we can obtain a proper effect size estimate of the stimulus–neural association strength given the across-trial variability within each participant at each time point (the right panel in the figure). This approach also does not scale or alter the distribution of the data, providing better estimates of standardized scores as compared with other procedures (for example, standardization across time points). Critically, regardless of the method chosen for the analysis (as also seen in Extended Data Fig. 4), the multivariate neural signals recorded from the MTL are clearly correlated with the cued item in the STM task, within the 500 ms following the offset of the task stimuli (that is, ~ 500–1,000 ms following stimulus onset). Noticeably, neural separation scores in the MTL for the cued color relative to other colors within this period across different adjacent time points are also robustly correlated with participants’ subsequent recall performance (Extended Data Fig. 2c). Combined, these results suggest that our findings during MTL recordings are robust to the different analytical procedures or parameter choices. Shaded areas in the current figure represent the s.e.m. across participants. Significant timepoints after cluster-based correction for multiple comparisons at the 0.05 level are marked in red (cued versus shuffled items) and green (cued versus both shuffled and uncued items) depending on the contrast performed.

Extended Data Fig. 4 Inverted encoding modeling (IEM) also reveals item-specific information represented in MTL high-frequency (70–150 Hz) activity during STM retention.

(a) The IEM method assumes that multivariate activity in a region (B) reflects a weighted summation of ideal stimulus information channels (C) of recorded neural populations (top right)43. The weights (W) of these information channels can be learned from brain activity in a training dataset (top left) and then applied to an independent hold-out test dataset to reconstruct stimulus information channels (C’). The resultant vector length of the normalized channel response function can be interpreted as a measure of STM precision, when the probability of recall success is high (bottom right)39. (b) Based on this approach, we have replicated observations from the more assumption-free representational similarity analysis approach. That is, information related to the cued item emerges following stimulus offset and lasts for about 500 ms during the retention interval. In contrast, information related to the uncued item is minimal and is significantly weaker as compared with that related to the cued item during STM retention interval. a.u. = arbitrary unit. Significant timepoints after cluster-based correction for multiple comparisons at the 0.05 level are marked in yellow.

Extended Data Fig. 5 Temporal generalization analysis of neural similarity based on high-frequency activity in the MTL and the connected cortical electrodes.

(a) To obtain a summary measure of information specific to the cued item, we calculated the within-item neural similarity (Swithin) for the same color across trials (excluding self-correlation) and contrasted it with the between-item neural similarity (Sbetween) estimated from the same number of randomly selected trials with a different cued color (over 5000 iterations). (b) We evaluated Swithin and Sbetween across time based on the high-frequency (70–150 Hz) activity in the MTL. The difference between Swithin and Sbetween may thus provide a summary measure of item-specific information. (c) Across participants, we find that delay-period MTL activity retains item-specific STM content that can distinguish the same color from other colors (P < 0.05 after cluster-based correction for multiple comparisons, outlined in black). This observation is consistent with the stimulus–neural association when stimulus similarity and neural similarity are directly compared. (d) To test whether neocortical information is related to MTL content in a time-sensitive manner, for each participant, we examined the neural separation profile for the cued relative to other colors in the MTL (that is, diagonal in c) and identified the local maximal value during STM retention to approximate the time point when the MTL contains the most information about the cued item during the delay period. On average this local maximal value occurs around 1057 ± 124 ms following stimulus onset. We then extracted −500 ms to 500 ms data around this time in the neocortical sites that connected with the MTL identified based on the methods outlined in Fig. 3. The shaded areas in the figure at the lower panel represent the s.e.m. across participants. (e) By examining the neural separation score, Swithin - Sbetween, in the connected neocortical sites around the MTL’s local maximal information time point, we find that neocortical representations related to the cued item are temporally dependent on that related to the MTL. (f) We calculate an asymmetry index to reveal this temporal relationship. Specifically, the asymmetric index is calculated as the difference in neural separation scores (Swithin – Sbetween) between the upper right quadrant and the lower left quadrant of the results in Extended Data Fig. 5e. This measure captures the extent to which item-specific STM information in the connected cortical electrodes changes within half a second after the recorded MTL signals reach the local peak in retaining item-specific STM content during the delay period. The unit of this measure is kept on the same scale as the cosine similarity measures used for the neural separation score (Swithin – Sbetween). In the 13 participants that we have identified meaningful MTL-neocortical connections with, we find that there is a significant asymmetry index for the cortical sites receiving information from the MTL (one-sample t-test: t(12) = 2.42, P = 0.032, Cohen’s d = 0.67, 95% CI: [0.05, 1.26]). In contrast, cortical sites sending information to the MTL do not show this effect (one-sample t-test: t(12) = −1.82, P = 0.093, Cohen’s d = −0.51, 95% CI: [−1.08, 0.08]). There is a significant interaction effect of connectivity direction (MTL-to-neocortex versus neocortex-to-MTL) and time period (before versus after the MTL’s local maximal information time point) based on a repeated measures analysis of variance (ANOVA) on the neocortical neural separation scores (F(1,12) = 9.73, P = 0.0090, partial η² = 0.45, 95% CI: [0.04, 0.67]), suggesting the temporal specificity of this observation. The grey dots and their connected lines represent data from an individual participant. The bar plots and error bars capture the mean and s.e.m. across participants, respectively.

Extended Data Fig. 6 MTL and non-MTL recordings and item-specific information revealed by stimulus–neural association analysis.

(a) To minimize the involvement of recording neocortical sites that account for only a small amount of variance in the retained STM content, we performed a feature selection procedure based on an omnibus F test. Specifically, for each participant, we identified the top 50% of the stimulus-sensitive cortical electrodes based on a one-way ANOVA on the delay-period high-frequency (70–150 Hz) activity across 9 colors. These non-MTL cortical electrodes (excluding electrodes in the hippocampus-amygdala complex, perirhinal/entorhinal cortices, and parahippocampus areas) are marked in light blue in a reconstructed brain surface in the MNI brain. The omnibus nature of this analysis does not impose any assumptions regarding specific stimulus tuning or the spatial distribution of these relevant cortical recording sites, and hence can help detect STM content retained in distributed cortical areas25,32,33 (also see Discussion). We then performed a similar stimulus–neural association analysis outlined in Fig. 2 based on these non-MTL signals. We find that these neocortical recording sites retain significant STM information about the cued item throughout the delay period. (b) MTL electrodes in the hippocampus and perirhinal/entorhinal cortices across participants are marked in orange on a reconstructed brain surface in the MNI brain. Stimulus–neural association findings from the MTL electrodes are reproduced from Fig. 2. (c) We directly compared the item-specific information in the MTL and non-MTL recording sites. STM information contained within the activity of the distributed non-MTL neocortical areas is sustained throughout the delay period, especially after the offset of item-specific information in the MTL during the mid-delay period. While these results suggest that STM information can sustain throughout the delay period in a distributed cortical network, these findings do not imply that STM depends on sustained neural activation25. Significant timepoints after cluster-based correction for multiple comparisons at the 0.05 level are marked in yellow.

Extended Data Fig. 7 Analyses of directed functional connectivity.

(a) To estimate spontaneous functional connections, we selected 20 random segments of 30-second recording data without any locking to individual task events while ensuring that participants were awake and behaving. We calculated the windowed-scaled cross-correlation estimates52 for signals from every electrode pair during these random segments, and then averaged the resulting z-scored cross-correlation estimates. The shaded areas represent the s.e.m. across different random segments. Based on this average estimate, we quantified the preferred time delay, τwmax, and its associated maximum value, Wmax, of coupling, as well as the coincidence index that captures how narrow the dominant peak is. We used a thresholding heuristic based on the distributions of Wmax and the coincidence index to separate electrode pairs into strongly and weakly connected populations (see Fig. 3b for an example and see Methods for details). (b) To examine how effective MTL-neocortical connections dynamically change throughout the STM task, we calculated the maximal coupling strength, Wmax, for each identified pair at the preferred time lag, τwmax, during the STM task using a 1,000-ms moving time window with 80% overlap. We then investigated how these directed functional connections are related to STM recall precision (Fig. 3d & 3f).

Extended Data Fig. 8 Individual best-fit parameters of different models for each participant under MTL stimulation.

(a, b) Participants’ data in each stimulation condition (grey bins) were separately fitted (solid lines) with a mixture model7 and the variable precision model74 based on maximum likelihood estimation (MLE). In the mixture model, STM precision is inversely quantified as the recall variability (SD, denoted as s.d. in the main text) after factoring out the proportion of failed memory responses uniformly distributed across the feature space (Extended Data Fig. 1). The likelihood of these failed memory responses equals one minus the probability of recall success (Pm). The variable precision model estimates STM precision using Fisher’s Information (J; higher J = better precision) assuming no STM capacity limit74. We also directly calculated the absolute recall error (AbsError) as a model-free estimate of the recall performance38,45. At the individual level, all participants show reduced STM precision for all three precision estimates (SD, AbsError, and J) under MTL stimulation. Complementary to the statistical results presented in the main text, we also evaluate the statistical significance of these measures by comparing the observed data with surrogated data that shuffle the stimulation condition labels within each participant 5,000 times (note: P values smaller than 0.001 will be marked as P < 0.001). At the group level, we find that SD, AbsError, and J are significantly affected by MTL stimulation (d, e, f), yet MTL stimulation does not significantly affect Pm (c).

Extended Data Fig. 9 MTL stimulation, as compared with non-MTL stimulation at posterior brain areas, is more likely to modulate STM precision.

(a) Electrical stimulation sites aggregated from 5 MTL stimulation cases (orange) and 4 non-MTL stimulation cases (light blue) in the MNI brain. Each case contributes to a pair of stimulation sites either in the MTL or in non-MTL posterior cortical areas (for example, the parietal cortex). (b) Stimulation consistently leads to an increase in STM recall variability in the 5 MTL cases (paired-sample t-test between stimulation on versus off: t(4) = 9.00, P = 0.00084, Cohen’s d = 4.03, 95% CI: [1.22, 6.84]). (c) In contrast, non-MTL stimulation under the same protocol during the same STM task shows little effect on STM recall variability or the likelihood of recall success. A mixed-effect ANOVA on individual best-fit model parameters shows that there is a significant stimulation by site interaction effect (within-subject factor: stimulation on versus off × between-subject factor: stimulation site: F(1, 7) = 25.15, P = 0.0015, partial η² = 0.78, 95% CI: [0.25, 0.88]), although the main effects of stimulation conditions (F(1, 7) = 4.65, P = 0.068, partial η² = 0.40, 95% CI: [0, 0.68]) and stimulation sites (F(1, 7) = 0.06, P = 0.81, partial η² = 0.009, 95% CI: [0, 0.31]) are not statistically significant. This interaction effect is also supported by results from Bayesian hierarchical modeling analysis (BF10 = 5.05). Collectively, these results suggest that electrical stimulation over the MTL has a more direct impact on STM precision.

Extended Data Fig. 10 Participants’ performance in the STM color recall task before and after brain surgery in the non-MTL group, along with individual best model fits for participants’ task performance in different lesion groups.

(a) Participants in the non-MTL lesion group have lesions in the frontal lobe, the insular, the temporal pole, or the inferior temporal lobe regions (also see Supplementary Table S3). All these non-MTL lesion participants have intact MTL structures before and after surgical treatment as verified by post-surgical MRI. (b, c) When comparing pre- and post-surgery task performance in these participants, we obtain little evidence that non-MTL lesions are sufficient to modulate participants’ STM recall performance or task performance in the perceptual/motor control task. Each grey dot in (b, c) represents data from an individual participant. The blue and red lines in represent the mixture model fits of the aggregated data for pre- and post-surgery performance, respectively. The bar plots and error bars capture the mean and s.e.m. across participants, respectively. (d) In the 8 MTL lesion cases, each participant (depicted as grey dots and connected lines) shows a reduction in STM precision (that is, an increase in STM recall variability, SD, denoted as s.d. in the main text) following the brain surgery (paired-sample t-test between pre- and post-surgery performance: t(7) = 4.34, P = 0.0034, Cohen’s d = 1.53, 95% CI: [0.46, 2.56]). In contrast, MTL lesions do not significantly affect the likelihood of recall success (paired-sample t-test: t(7) = −0.86, P = 0.42, Cohen’s d = −0.30, 95% CI: [−1.01, 0.42]) or the overall performance in the perceptual/motor control task (paired-sample t-test for absolute reproduction errors: t(7) = 1.58, P = 0.16, Cohen’s d = 0.56, 95% CI: [−0.21, 1.29]). Another 8 non-MTL cases do not show a significant effect on measures of STM recall performance (paired-sample t-test between pre- and post-surgery performance for recall variability: t(7) = −0.94, P = 0.38, Cohen’s d = −0.33, 95% CI: [−1.04,0.39]; for the likelihood of recall success: t(7) = −0.74, P = 0.48, Cohen’s d = −0.26, 95% CI: [−0.96, 0.45]) or the overall performance in the perceptual/motor control task (paired-sample t-test for absolute reproduction errors: t(7) = 1.02, P = 0.34, Cohen’s d = 0.36, 95% CI: [−0.37, 1.07]). Critically, the effect of surgery on STM precision is greater in the MTL relative to non-MTL cases, as supported by a significant interaction effect between testing time points (pre-surgery versus post-surgery) and lesion groups (MTL versus non-MTL) in a mixed-effect ANOVA on individual best-fit parameters (F(1, 14) = 14.71, P = 0.0020, partial η² = 0.51, 95% CI: [0.11, 0.71]). (e) These findings are consistent with the results from permutation tests by shuffling participants’ lesion status labels over 5,000 iterations (P < 0.001) and Bayesian hierarchical modeling (lesion × pre- and post-test interaction effect: BF10 = 797.14).

Supplementary information

Supplementary Information

Supplementary Methods and Tables 1–3.

Reporting Summary

Peer Review File

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, W., Chapeton, J.I., Bhasin, S. et al. The medial temporal lobe supports the quality of visual short-term memory representation. Nat Hum Behav 7, 627–641 (2023). https://doi.org/10.1038/s41562-023-01529-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-023-01529-5

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing