Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Letter
  • Published:

Visual working memory directly alters perception

Nature Human Behaviour volume 3, pages 827–836 (2019)Cite this article

Subjects

Abstract

Visual working memory (VWM), the ability to temporarily maintain and manipulate information, underlies a variety of critical high-level behaviours from directing attention1,2,3,4 to making complex decisions5. Here we show that its impact extends to even the most basic levels of perceptual processing, directly interacting with and even distorting the physical appearance of visual features. This interference results from and can be predicted by the recruitment of posterior perceptual cortices to maintain information in VWM6,7,8,9, which causes an overlap with the neuronal populations supporting perceptual processing10,11,12,13,14,15. Across three sets of experiments, we demonstrated bidirectional interference between VWM and low-level perception. Specifically, for both maintained colours and orientations, presenting a distractor created bias in VWM representation depending on the similarity between incoming and maintained information, consistent with the known tuning curves for these features. Moreover, holding an item in mind directly altered the appearance of new stimuli, demonstrated by changes in psychophysical discrimination thresholds. Thus, as a consequence of sharing the early visual cortices, what you see and what you are holding in mind are intertwined at even the most fundamental stages of processing.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Bidirectional interference between VWM and selective attention.
Fig. 2: Ongoing perception alters VWM representation.
Fig. 3: Testing the impact from VWM to perception in experiment 3.
Fig. 4: VWM changes the appearance of colour and orientation, reflected by the changes in discrimination thresholds.

Similar content being viewed by others

Data availability

The data that support the findings of this study are available in the Open Science Framework at https://osf.io/j7nv2/.

Code availability

The custom code developed for this study is available in the Open Science Framework at https://osf.io/j7nv2/.

References

  1. Downing, P. E. & Dodds, C. M. Competition in visual working memory for control of search. Vis. Cogn. 11, 689–703 (2004).

    Article  Google Scholar 

  2. Olivers, C. N. L., Meijer, F. & Theeuwes, J. Feature-based memory-driven attentional capture: visual working memory content affects visual attention. J. Exp. Psychol. Hum. Percept. Perform. 32, 1243–1265 (2006).

    Article  Google Scholar 

  3. Soto, D., Hodsoll, J., Rotshtein, P. & Humphreys, G. W. Automatic guidance of attention from working memory. Trends Cogn. Sci. 12, 342–348 (2008).

    Article  Google Scholar 

  4. Kiyonaga, A. & Egner, T. Center-surround inhibition in working memory. Curr. Biol. 26, 1–5 (2016).

    Article  Google Scholar 

  5. Baddeley, A. Working Memory, Thought, and Action. (OUP Oxford, 2007).

  6. Harrison, S. A. & Tong, F. Decoding reveals the contents of visual working memory in early visual areas. Nature 458, 632–635 (2009).

    Article  CAS  Google Scholar 

  7. Serences, J., Ester, E., Vogel, E. K. & Awh, E. Stimulus-specific delay activity in human primary visual cortex. Psychol. Sci. 20, 207–214 (2009).

    Article  Google Scholar 

  8. Emrich, S. M., Riggall, A. C., Larocque, J. J. & Postle, B. R. Distributed patterns of activity in sensory cortex feflect the memory. J. Neurosci. 33, 6516–6523 (2013).

    Article  CAS  Google Scholar 

  9. Riggall, A. C. & Postle, B. R. The relationship between working memory storage and elevated activity as measured with functional magnetic resonance imaging. J. Neurosci. 32, 12990–12998 (2012).

    Article  CAS  Google Scholar 

  10. Pasternak, T. & Greenlee, M. W. Working memory in primate sensory systems. Nat. Rev. Neurosci. 6, 97–107 (2005).

    Article  CAS  Google Scholar 

  11. D’Esposito, M. & Postle, B. R. The cognitive neuroscience of working memory. Annu. Rev. Psychol. 66, 115–142 (2015).

    Article  Google Scholar 

  12. Postle, B. R. Working memory as an emergent property of the mind and brain. Neuroscience 139, 23–38 (2006).

    Article  CAS  Google Scholar 

  13. Postle, B. R. The cognitive neuroscience of visual short-term memory. Curr. Opin. Behav. Sci. 1, 40–46 (2015).

    Article  Google Scholar 

  14. Christophel, T. B., Hebart, M. N. & Haynes, J.-D. Decoding the contents of visual short-term memory from human visual and parietal cortex. J. Neurosci. 32, 12983–12989 (2012).

    Article  CAS  Google Scholar 

  15. Lee, S. H., Kravitz, D. J. & Baker, C. I. Goal-dependent dissociation of visual and prefrontal cortices during working memory. Nat. Neurosci. 16, 997–999 (2013).

    Article  CAS  Google Scholar 

  16. Awh, E., Vogel, E. K. & Oh, S. H. Interactions between attention and working memory. Neuroscience 139, 201–208 (2006).

    Article  CAS  Google Scholar 

  17. Chun, M. M., Golomb, J. D. & Turk-Browne, N. B. A taxonomy of external and internal attention. Annu. Rev. Psychol. 62, 73–101 (2010).

    Article  Google Scholar 

  18. Lavie, N. & De Fockert, J. The role of working memory in attentional capture. Psychon. Bull. Rev. 12, 669–674 (2005).

    Article  Google Scholar 

  19. Cowan, N. An embedded-processes model of working memory. in Models of Working Memory: Mechanisms of Active Maintenance and Executive Control (ed. Miyake, A. & Shah, P.) 62–102 (Cambridge University Press, 1999).

  20. Corbetta, M. & Shulman, G. L. Control of goal-directed and stimulus-driven attention in the brain. Nat. Rev. Neurosci. 3, 201–215 (2002).

    Article  CAS  Google Scholar 

  21. Lepsien, J. & Nobre, A. C. Cognitive control of attention in the human brain: insights from orienting attention to mental representations. Brain. Res. 1105, 20–31 (2006).

    Article  CAS  Google Scholar 

  22. Courtney, S. M., Ungerleider, L. G., Keil, K. & Haxby, J. V. Transient and sustained activity in a distributed neural system for human working memory. Nature 386, 608–611 (1997).

    Article  CAS  Google Scholar 

  23. Fuster, J. The prefrontal cortex—an update: time is of the essence. Neuron 30, 319–333 (2001).

    Article  CAS  Google Scholar 

  24. Goldman-Rakic Cellular basis of working memory review. Neuron 14, 477–485 (1995).

    Article  CAS  Google Scholar 

  25. Miller, E. K., Erickson, C. & Desimone, R. Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J. Neurosci. 16, 5154–5167 (1996).

    Article  CAS  Google Scholar 

  26. Pessoa, L. & Ungerleider, L. G. Top-Down Mechanisms for Working Memory and Attentional Processes. in The Cognitive Neurosciences (ed. Gazzaniga, M. S.) 919–930 (MIT Press, 2004).

  27. Oliveri, M. et al. Parieto-frontal interactions in visual-object and visual-spatial working memory: evidence from transcranial magnetic stimulation. Cereb. Cortex 11, 606–618 (2001).

    Article  CAS  Google Scholar 

  28. Rademaker, R. L., Van De Ven, V. G., Tong, F. & Sack, A. T. The impact of early visual cortex transcranial magnetic stimulation on visual working memory precision and guess rate. PLoS One 12, e0175230 (2017).

    Article  Google Scholar 

  29. Rose, N. S. et al. Reactivation of latent working memories with transcranial magnetic stimulation. Science 354, 1136–1139 (2016).

    Article  CAS  Google Scholar 

  30. Zokaei, N., Manohar, S., Husain, M. & Feredoes, E. Causal dvidence for a privileged working memory state in early visual cortex. J. Neurosci. 34, 158–162 (2014).

    Article  CAS  Google Scholar 

  31. Xu, Y. Sensory cortex is nonessential in working memory storage. Trends Cogn. Sci. 22, 192–193 (2018).

    Article  Google Scholar 

  32. Xu, Y. Reevaluating the sensory account of visual working memory storage. Trends Cogn. Sci. 21, 794–815 (2017).

    Article  Google Scholar 

  33. Bettencourt, K. C. & Xu, Y. Decoding the content of visual short-term memory under distraction in occipital and parietal areas. Nat. Neurosci. 19, 150–157 (2016).

    Article  CAS  Google Scholar 

  34. Awh, E. & Jonides, J. Overlapping mechansims of attention and spatial working memory. Trends Cogn. Sci. 5, 119–126 (2001).

    Article  CAS  Google Scholar 

  35. Magnussen, S. & Greenlee, M. W. Retention and disruption of motion information in visual short-term memory. J. Exp. Psychol. Learn. Mem. Cogn. 18, 151–156 (1992).

    Article  CAS  Google Scholar 

  36. Wildegger, T., Myers, N. E., Humphreys, G. & Nobre, A. C. Supraliminal but not subliminal distracters bias working memory recall. J. Exp. Psychol. Hum. Percept. Perform. 41, 826–839 (2015).

    Article  Google Scholar 

  37. Nemes, V. A., Parry, N. R. A., Whitaker, D. & McKeefry, D. J. The retention and disruption of color information in human short-term visual memory. J. Vis. 12, 26 (2012).

    Article  Google Scholar 

  38. Rademaker, R. L., Bloem, I. M., De Weerd, P. & Sack, A. T. The impact of interference on short-term memory for visual orientation. J. Exp. Psychol. Hum. Percept. Perform. 41, 1650–1665 (2015).

    Article  Google Scholar 

  39. Lorenc, E. S., Sreenivasan, K. K., Nee, D. E., Vandenbroucke, A. R. E. & D’Esposito, M. Flexible coding of visual working memory representations during distraction. J. Neurosci. 38, 5267–5276 (2018).

    Article  CAS  Google Scholar 

  40. Mansfield, R. J. W. & Ronner, S. F. Orientation anisotropy in monkey visual cortex. Brain Res. 149, 229–234 (1978).

    Article  CAS  Google Scholar 

  41. Ringach, D. L., Shapley, R. M. & Hawken, M. J. Orientation selectivity in macaque V1: diversity and laminar dependence. J. Neurosci. 22, 5639–5651 (2002).

    Article  CAS  Google Scholar 

  42. Shapley, R., Hawken, M. & Ringach, D. L. Dynamics of orientation selectivity in the primary visual cortex and the importance of cortical inhibition. Neuron 38, 689–699 (2003).

    Article  CAS  Google Scholar 

  43. Bohon, K. S., Hermann, K. L., Hansen, T. & Conway, B. R. Representation of perceptual color space in macaque posterior inferior temporal cortex (the V4 complex). eNeuro 3, ENEURO.0039–16.2016 (2016).

    Article  Google Scholar 

  44. Geng, J. J., Di Quattro, N. E. & Helm, J. Distractor probability changes the shape of the attentional template. J. Exp. Psychol. Hum. Percept. Perform. 43, 1993–2007 (2017).

    Article  Google Scholar 

  45. Zhang, W. & Luck, S. J. Discrete fixed-resolution representations in visual working memory. Nature 453, 233–235 (2008).

    Article  CAS  Google Scholar 

  46. Bays, P. M. & Husain, M. Dynamic shifts of limited working memory resources in human vision. Science 321, 851–854 (2008).

    Article  CAS  Google Scholar 

  47. Joseph, J. S., Chun, M. M. & Nakayama, K. Attentional requirements in a ‘preattentive’ feature search task. Nature 387, 805–807 (1997).

    Article  CAS  Google Scholar 

  48. Horton, J. C. & Hoyt, W. F. The representation of the visual field in human striate cortex. Arch. Opthalmology 109, 816–824 (1991).

    Article  CAS  Google Scholar 

  49. Engel, S. A., Glover, G. H. & Wandell, B. A. Retinotopic organization in human visual cortex and the spatial precision of functional MRI. Cereb. Cortex 7, 181–192 (1997).

    Article  CAS  Google Scholar 

  50. McAdams, C. J. & Maunsell, J. H. Effects of attention on orientation-tuning functions of single neurons in macaque cortical area V4. J. Neurosci. 19, 431–441 (1999).

    Article  CAS  Google Scholar 

  51. Müller, N. G. & Kleinschmidt, A. Dynamic interaction of object- and space-based attention in retinotopic visual areas. J. Neurosci. 23, 9812–9816 (2003).

    Article  Google Scholar 

  52. Carrasco, M., Ling, S. & Read, S. Attention alters appearance. Nat. Neurosci. 7, 308–313 (2004).

    Article  CAS  Google Scholar 

  53. Fuller, S. & Carrasco, M. Exogenous attention and color perception: performance and appearance of saturation and hue. Vis. Res. 46, 4032–4047 (2006).

    Article  Google Scholar 

  54. Liu, T., Fuller, S. & Carrasco, M. Attention alters the appearance of motion coherence. Psychon. Bull. Rev. 13, 1091–1096 (2006).

    Article  Google Scholar 

  55. Prins, N. & Kingdom, F. A. A. Applying the model-comparison approach to test specific research hypotheses in psychophysical research using the Palamedes toolbox. Front. Psychol. 9, 1250 (2018).

    Article  Google Scholar 

  56. Scimeca, J. M., Kiyonaga, A. & D’Esposito, M. Reaffirming the sensory recruitment account of working memory. Trends Cogn. Sci. 22, 190–192 (2018).

    Article  Google Scholar 

  57. Albers, A. M. et al. Supplemental information shared representations for working memory and mental imagery in early visual cortex. Curr. Biol. 23, 1427–1431 (2013).

    Article  CAS  Google Scholar 

  58. Farah, M. J. Psychophysical evidence for a shared representational medium for mental images and percepts. J. Exp. Psychol. Gen. 114, 91–103 (1985).

    Article  CAS  Google Scholar 

  59. Christophel, T. B., Klink, P. C., Spitzer, B., Roelfsema, P. R. & Haynes, J. D. The distributed nature of working memory. Trends Cogn. Sci. 21, 111–124 (2017).

    Article  Google Scholar 

  60. Sreenivasan, K. K., Curtis, C. E. & D’Esposito, M. Revisiting the role of persistent neural activity during working memory. Trends Cogn. Sci. 18, 82–89 (2014).

    Article  Google Scholar 

  61. Miller, E. K. & Cohen, J. D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).

    Article  CAS  Google Scholar 

  62. Woolgar, A., Hampshire, A., Thompson, R. & Duncan, J. Adaptive coding of task-relevant information in human frontoparietal cortex. J. Neurosci. 31, 14592–14599 (2011).

    Article  CAS  Google Scholar 

  63. Mongillo, G., Barak, O. & Tsodyks, M. Synaptic theory of working memory. Science 319, 1543–1546 (2008).

    Article  CAS  Google Scholar 

  64. Gazzaley, A. & Nobre, A. C. Top-down modulation: bridging selective attention and working memory. Trends Cogn. Sci. 16, 129–135 (2012).

    Article  Google Scholar 

  65. Dowd, E. W. & Mitroff, S. R. Attentional guidance by working memory overrides salience cues in visual search. J. Exp. Psychol. Hum. Percept. Perform. 39, 1786–1796 (2013).

    Article  Google Scholar 

  66. Kiyonaga, A. & Egner, T. Working memory as internal attention: toward an integrative account of internal and external selection processes. Psychon. Bull. Rev. 20, 228–242 (2013).

    Article  Google Scholar 

  67. Kohn, A. Visual adaptation: physiology, mechanisms, and functional benefits. J. Neurophysiol. 10461, 3155–3164 (2007).

    Article  Google Scholar 

  68. Montagna, B., Pestilli, F. & Carrasco, M. Attention trades off spatial acuity. Vis. Res. 49, 735–745 (2009).

    Article  Google Scholar 

  69. Gureckis, T. M. et al. psiTurk: an open-source framework for conducting replicable behavioral experiments online. Behav. Res. Methods 48, 829–842 (2016).

    Article  Google Scholar 

  70. Kiyonaga, A. & Egner, T. The working memory stroop effect: when internal representations clash with external stimuli. Psychol. Sci. 25, 1619–1629 (2014).

    Article  Google Scholar 

  71. Crump, M. J. C., McDonnell, J. V. & Gureckis, T. M. Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS One 8, e57410 (2013).

    Article  CAS  Google Scholar 

  72. Brady, T. F. & Alvarez, G. A. Hierarchical encoding in visual working memory: ensemble statistics bias memory for individual items. Psychol. Sci. 22, 384–392 (2011).

    Article  Google Scholar 

  73. Brady, T. F. & Alvarez, G. A. No evidence for a fixed object limit in working memory: spatial ensemble representations inflate estimates of working memory capacity for complex objects. J. Exp. Psychol. Learn. Mem. Cogn. 41, 921–929 (2015).

    Article  Google Scholar 

  74. Lew, T. F. & Vul, E. Ensemble clustering in visual working memory biases location memories and reduces the Weber noise of relative positions. J. Vis. 15, 10 (2015).

    Article  Google Scholar 

  75. Shin, H. & Ma, W. J. Crowdsourced single-trial probes of visual working memory for irrelevant features. J. Vis. 16, 10 (2016).

    Article  Google Scholar 

  76. Woodman, G. F. & Luck, S. J. Do the contents of visual working memory automatically influence attentional selection during visual search? J. Exp. Psychol. Hum. Percept. Perform. 33, 363 (2007).

    Article  Google Scholar 

  77. Lakens, D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front. Psychol. 4, 863 (2013).

    Article  Google Scholar 

Download references

Acknowledgements

We thank the colleagues in the psychology department of George Washington University for their helpful discussion. Especially, we thank S. Shomstein, S. Mitroff and M. Behrmann for their insightful comments on this manuscript. This research project was supported by start-up funds granted to D.J.K. by GWU. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Department of Psychology, George Washington University, Washington, DC, USA

    Chunyue Teng & Dwight J. Kravitz

Authors
  1. Chunyue Teng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dwight J. Kravitz
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

C.T. and D.J.K. designed the experiments. C.T. collected the data and performed analysis. C.T. and D.J.K. wrote the manuscript.

Corresponding author

Correspondence to Chunyue Teng.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information: Primary Handling Editor: Marike Schiffer.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–5 and Supplementary Tables 1–5.

Reporting Summary

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Teng, C., Kravitz, D.J. Visual working memory directly alters perception. Nat Hum Behav 3, 827–836 (2019). https://doi.org/10.1038/s41562-019-0640-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-019-0640-4

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing