Decoding reveals the contents of visual working memory in early visual areas

Article metrics

Abstract

Visual working memory provides an essential link between perception and higher cognitive functions, allowing for the active maintenance of information about stimuli no longer in view1,2. Research suggests that sustained activity in higher-order prefrontal, parietal, inferotemporal and lateral occipital areas supports visual maintenance3,4,5,6,7,8,9,10,11, and may account for the limited capacity of working memory to hold up to 3–4 items9,10,11. Because higher-order areas lack the visual selectivity of early sensory areas, it has remained unclear how observers can remember specific visual features, such as the precise orientation of a grating, with minimal decay in performance over delays of many seconds12. One proposal is that sensory areas serve to maintain fine-tuned feature information13, but early visual areas show little to no sustained activity over prolonged delays14,15,16. Here we show that orientations held in working memory can be decoded from activity patterns in the human visual cortex, even when overall levels of activity are low. Using functional magnetic resonance imaging and pattern classification methods, we found that activity patterns in visual areas V1–V4 could predict which of two oriented gratings was held in memory with mean accuracy levels upwards of 80%, even in participants whose activity fell to baseline levels after a prolonged delay. These orientation-selective activity patterns were sustained throughout the delay period, evident in individual visual areas, and similar to the responses evoked by unattended, task-irrelevant gratings. Our results demonstrate that early visual areas can retain specific information about visual features held in working memory, over periods of many seconds when no physical stimulus is present.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Design of working memory experiment and resulting time course of fMRI activity.
Figure 2: Orientation decoding results for areas V1–V4.
Figure 3: Time-resolved decoding of individual fMRI time points.

References

  1. 1

    Baddeley, A. Working memory: looking back and looking forward. Nature Rev. Neurosci. 4, 829–839 (2003)

  2. 2

    Luck, S. J. & Vogel, E. K. The capacity of visual working memory for features and conjunctions. Nature 390, 279–281 (1997)

  3. 3

    Fuster, J. M. & Alexander, G. E. Neuron activity related to short-term memory. Science 173, 652–654 (1971)

  4. 4

    Miyashita, Y. & Chang, H. S. Neuronal correlate of pictorial short-term memory in the primate temporal cortex. Nature 331, 68–70 (1988)

  5. 5

    Miller, E. K., Erickson, C. A. & Desimone, R. Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J. Neurosci. 16, 5154–5167 (1996)

  6. 6

    Courtney, S. M., Ungerleider, L. G., Keil, K. & Haxby, J. V. Transient and sustained activity in a distributed neural system for human working memory. Nature 386, 608–611 (1997)

  7. 7

    Pessoa, L., Gutierrez, E., Bandettini, P. & Ungerleider, L. Neural correlates of visual working memory: fMRI amplitude predicts task performance. Neuron 35, 975–987 (2002)

  8. 8

    Curtis, C. E. & D'Esposito, M. Persistent activity in the prefrontal cortex during working memory. Trends Cogn. Sci. 7, 415–423 (2003)

  9. 9

    Todd, J. J. & Marois, R. Capacity limit of visual short-term memory in human posterior parietal cortex. Nature 428, 751–754 (2004)

  10. 10

    Vogel, E. K. & Machizawa, M. G. Neural activity predicts individual differences in visual working memory capacity. Nature 428, 748–751 (2004)

  11. 11

    Xu, Y. & Chun, M. M. Dissociable neural mechanisms supporting visual short-term memory for objects. Nature 440, 91–95 (2006)

  12. 12

    Magnussen, S. & Greenlee, M. W. The psychophysics of perceptual memory. Psychol. Res. 62, 81–92 (1999)

  13. 13

    Pasternak, T. & Greenlee, M. W. Working memory in primate sensory systems. Nature Rev. Neurosci. 6, 97–107 (2005)

  14. 14

    Offen, S., Schluppeck, D. & Heeger, D. J. The role of early visual cortex in visual short-term memory and visual attention. Vision Res. 10.1016/j.visres.2007.12.022 (in the press)

  15. 15

    Bisley, J. W., Zaksas, D., Droll, J. A. & Pasternak, T. Activity of neurons in cortical area MT during a memory for motion task. J. Neurophysiol. 91, 286–300 (2004)

  16. 16

    Zaksas, D. & Pasternak, T. Directional signals in the prefrontal cortex and in area MT during a working memory for visual motion task. J. Neurosci. 26, 11726–11742 (2006)

  17. 17

    Kamitani, Y. & Tong, F. Decoding the visual and subjective contents of the human brain. Nature Neurosci. 8, 679–685 (2005)

  18. 18

    Kastner, S. & Ungerleider, L. G. Mechanisms of visual attention in the human cortex. Annu. Rev. Neurosci. 23, 315–341 (2000)

  19. 19

    Sasaki, Y. et al. The radial bias: a different slant on visual orientation sensitivity in human and nonhuman primates. Neuron 51, 661–670 (2006)

  20. 20

    Tong, F. Primary visual cortex and visual awareness. Nature Rev. Neurosci. 4, 219–229 (2003)

  21. 21

    Kosslyn, S. M., Ganis, G. & Thompson, W. L. Neural foundations of imagery. Nature Rev. Neurosci. 2, 635–642 (2001)

  22. 22

    Roelfsema, P. R. Elemental operations in vision. Trends Cogn. Sci. 9, 226–233 (2005)

  23. 23

    Super, H., Spekreijse, H. & Lamme, V. A. A neural correlate of working memory in the monkey primary visual cortex. Science 293, 120–124 (2001)

  24. 24

    Ress, D., Backus, B. T. & Heeger, D. J. Activity in primary visual cortex predicts performance in a visual detection task. Nature Neurosci. 3, 940–945 (2000)

  25. 25

    Geng, J. J., Ruff, C. C. & Driver, J. Saccades to a remembered location elicit spatially specific activation in the human retinotopic visual cortex. J. Cogn. Neurosci. 21, 230–245 (2009)

  26. 26

    Miller, E. K., Li, L. & Desimone, R. Activity of neurons in anterior inferior temporal cortex during a short-term memory task. J. Neurosci. 13, 1460–1478 (1993)

  27. 27

    Averbeck, B. B., Latham, P. E. & Pouget, A. Neural correlations, population coding and computation. Nature Rev. Neurosci. 7, 358–366 (2006)

  28. 28

    Logothetis, N. K. et al. Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–157 (2001)

  29. 29

    Lee, H., Simpson, G. V., Logothetis, N. K. & Rainer, G. Phase locking of single neuron activity to theta oscillations during working memory in monkey extrastriate visual cortex. Neuron 45, 147–156 (2005)

  30. 30

    Wang, X. J. Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci. 24, 455–463 (2001)

Download references

Acknowledgements

We thank D. Brady and B. Wolfe for technical support, and J. Gore and the Vanderbilt University Institute of Imaging Science for MRI support. This work was supported by a grant from the National Eye Institute, National Institutes of Health to F.T. and a postgraduate fellowship from the Natural Sciences and Engineering Research Council of Canada to S.A.H.

Author Contributions F.T. devised and designed the experiments, S.A.H. and F.T. programmed the experiments, S.A.H. conducted the experiments and carried out the analyses with assistance from F.T., F.T. and S.A.H. wrote the paper together.

Author information

Correspondence to Frank Tong.

Supplementary information

Supplementary Information

This file contains Supplementary Methods, Supplementary Data, Supplementary References and Supplementary Figures 1-7 with Legends (PDF 2982 kb)

PowerPoint slides

Rights and permissions

Reprints and Permissions

About this article

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.