Visual working memory (VWM), the ability to temporarily maintain and manipulate information, underlies a variety of critical high-level behaviours from directing attention1,2,3,4 to making complex decisions5. Here we show that its impact extends to even the most basic levels of perceptual processing, directly interacting with and even distorting the physical appearance of visual features. This interference results from and can be predicted by the recruitment of posterior perceptual cortices to maintain information in VWM6,7,8,9, which causes an overlap with the neuronal populations supporting perceptual processing10,11,12,13,14,15. Across three sets of experiments, we demonstrated bidirectional interference between VWM and low-level perception. Specifically, for both maintained colours and orientations, presenting a distractor created bias in VWM representation depending on the similarity between incoming and maintained information, consistent with the known tuning curves for these features. Moreover, holding an item in mind directly altered the appearance of new stimuli, demonstrated by changes in psychophysical discrimination thresholds. Thus, as a consequence of sharing the early visual cortices, what you see and what you are holding in mind are intertwined at even the most fundamental stages of processing.
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
The data that support the findings of this study are available in the Open Science Framework at https://osf.io/j7nv2/.
The custom code developed for this study is available in the Open Science Framework at https://osf.io/j7nv2/.
Downing, P. E. & Dodds, C. M. Competition in visual working memory for control of search. Vis. Cogn. 11, 689–703 (2004).
Olivers, C. N. L., Meijer, F. & Theeuwes, J. Feature-based memory-driven attentional capture: visual working memory content affects visual attention. J. Exp. Psychol. Hum. Percept. Perform. 32, 1243–1265 (2006).
Soto, D., Hodsoll, J., Rotshtein, P. & Humphreys, G. W. Automatic guidance of attention from working memory. Trends Cogn. Sci. 12, 342–348 (2008).
Kiyonaga, A. & Egner, T. Center-surround inhibition in working memory. Curr. Biol. 26, 1–5 (2016).
Baddeley, A. Working Memory, Thought, and Action. (OUP Oxford, 2007).
Harrison, S. A. & Tong, F. Decoding reveals the contents of visual working memory in early visual areas. Nature 458, 632–635 (2009).
Serences, J., Ester, E., Vogel, E. K. & Awh, E. Stimulus-specific delay activity in human primary visual cortex. Psychol. Sci. 20, 207–214 (2009).
Emrich, S. M., Riggall, A. C., Larocque, J. J. & Postle, B. R. Distributed patterns of activity in sensory cortex feflect the memory. J. Neurosci. 33, 6516–6523 (2013).
Riggall, A. C. & Postle, B. R. The relationship between working memory storage and elevated activity as measured with functional magnetic resonance imaging. J. Neurosci. 32, 12990–12998 (2012).
Pasternak, T. & Greenlee, M. W. Working memory in primate sensory systems. Nat. Rev. Neurosci. 6, 97–107 (2005).
D’Esposito, M. & Postle, B. R. The cognitive neuroscience of working memory. Annu. Rev. Psychol. 66, 115–142 (2015).
Postle, B. R. Working memory as an emergent property of the mind and brain. Neuroscience 139, 23–38 (2006).
Postle, B. R. The cognitive neuroscience of visual short-term memory. Curr. Opin. Behav. Sci. 1, 40–46 (2015).
Christophel, T. B., Hebart, M. N. & Haynes, J.-D. Decoding the contents of visual short-term memory from human visual and parietal cortex. J. Neurosci. 32, 12983–12989 (2012).
Lee, S. H., Kravitz, D. J. & Baker, C. I. Goal-dependent dissociation of visual and prefrontal cortices during working memory. Nat. Neurosci. 16, 997–999 (2013).
Awh, E., Vogel, E. K. & Oh, S. H. Interactions between attention and working memory. Neuroscience 139, 201–208 (2006).
Chun, M. M., Golomb, J. D. & Turk-Browne, N. B. A taxonomy of external and internal attention. Annu. Rev. Psychol. 62, 73–101 (2010).
Lavie, N. & De Fockert, J. The role of working memory in attentional capture. Psychon. Bull. Rev. 12, 669–674 (2005).
Cowan, N. An embedded-processes model of working memory. in Models of Working Memory: Mechanisms of Active Maintenance and Executive Control (ed. Miyake, A. & Shah, P.) 62–102 (Cambridge University Press, 1999).
Corbetta, M. & Shulman, G. L. Control of goal-directed and stimulus-driven attention in the brain. Nat. Rev. Neurosci. 3, 201–215 (2002).
Lepsien, J. & Nobre, A. C. Cognitive control of attention in the human brain: insights from orienting attention to mental representations. Brain. Res. 1105, 20–31 (2006).
Courtney, S. M., Ungerleider, L. G., Keil, K. & Haxby, J. V. Transient and sustained activity in a distributed neural system for human working memory. Nature 386, 608–611 (1997).
Fuster, J. The prefrontal cortex—an update: time is of the essence. Neuron 30, 319–333 (2001).
Goldman-Rakic Cellular basis of working memory review. Neuron 14, 477–485 (1995).
Miller, E. K., Erickson, C. & Desimone, R. Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J. Neurosci. 16, 5154–5167 (1996).
Pessoa, L. & Ungerleider, L. G. Top-Down Mechanisms for Working Memory and Attentional Processes. in The Cognitive Neurosciences (ed. Gazzaniga, M. S.) 919–930 (MIT Press, 2004).
Oliveri, M. et al. Parieto-frontal interactions in visual-object and visual-spatial working memory: evidence from transcranial magnetic stimulation. Cereb. Cortex 11, 606–618 (2001).
Rademaker, R. L., Van De Ven, V. G., Tong, F. & Sack, A. T. The impact of early visual cortex transcranial magnetic stimulation on visual working memory precision and guess rate. PLoS One 12, e0175230 (2017).
Rose, N. S. et al. Reactivation of latent working memories with transcranial magnetic stimulation. Science 354, 1136–1139 (2016).
Zokaei, N., Manohar, S., Husain, M. & Feredoes, E. Causal dvidence for a privileged working memory state in early visual cortex. J. Neurosci. 34, 158–162 (2014).
Xu, Y. Sensory cortex is nonessential in working memory storage. Trends Cogn. Sci. 22, 192–193 (2018).
Xu, Y. Reevaluating the sensory account of visual working memory storage. Trends Cogn. Sci. 21, 794–815 (2017).
Bettencourt, K. C. & Xu, Y. Decoding the content of visual short-term memory under distraction in occipital and parietal areas. Nat. Neurosci. 19, 150–157 (2016).
Awh, E. & Jonides, J. Overlapping mechansims of attention and spatial working memory. Trends Cogn. Sci. 5, 119–126 (2001).
Magnussen, S. & Greenlee, M. W. Retention and disruption of motion information in visual short-term memory. J. Exp. Psychol. Learn. Mem. Cogn. 18, 151–156 (1992).
Wildegger, T., Myers, N. E., Humphreys, G. & Nobre, A. C. Supraliminal but not subliminal distracters bias working memory recall. J. Exp. Psychol. Hum. Percept. Perform. 41, 826–839 (2015).
Nemes, V. A., Parry, N. R. A., Whitaker, D. & McKeefry, D. J. The retention and disruption of color information in human short-term visual memory. J. Vis. 12, 26 (2012).
Rademaker, R. L., Bloem, I. M., De Weerd, P. & Sack, A. T. The impact of interference on short-term memory for visual orientation. J. Exp. Psychol. Hum. Percept. Perform. 41, 1650–1665 (2015).
Lorenc, E. S., Sreenivasan, K. K., Nee, D. E., Vandenbroucke, A. R. E. & D’Esposito, M. Flexible coding of visual working memory representations during distraction. J. Neurosci. 38, 5267–5276 (2018).
Mansfield, R. J. W. & Ronner, S. F. Orientation anisotropy in monkey visual cortex. Brain Res. 149, 229–234 (1978).
Ringach, D. L., Shapley, R. M. & Hawken, M. J. Orientation selectivity in macaque V1: diversity and laminar dependence. J. Neurosci. 22, 5639–5651 (2002).
Shapley, R., Hawken, M. & Ringach, D. L. Dynamics of orientation selectivity in the primary visual cortex and the importance of cortical inhibition. Neuron 38, 689–699 (2003).
Bohon, K. S., Hermann, K. L., Hansen, T. & Conway, B. R. Representation of perceptual color space in macaque posterior inferior temporal cortex (the V4 complex). eNeuro 3, ENEURO.0039–16.2016 (2016).
Geng, J. J., Di Quattro, N. E. & Helm, J. Distractor probability changes the shape of the attentional template. J. Exp. Psychol. Hum. Percept. Perform. 43, 1993–2007 (2017).
Zhang, W. & Luck, S. J. Discrete fixed-resolution representations in visual working memory. Nature 453, 233–235 (2008).
Bays, P. M. & Husain, M. Dynamic shifts of limited working memory resources in human vision. Science 321, 851–854 (2008).
Joseph, J. S., Chun, M. M. & Nakayama, K. Attentional requirements in a ‘preattentive’ feature search task. Nature 387, 805–807 (1997).
Horton, J. C. & Hoyt, W. F. The representation of the visual field in human striate cortex. Arch. Opthalmology 109, 816–824 (1991).
Engel, S. A., Glover, G. H. & Wandell, B. A. Retinotopic organization in human visual cortex and the spatial precision of functional MRI. Cereb. Cortex 7, 181–192 (1997).
McAdams, C. J. & Maunsell, J. H. Effects of attention on orientation-tuning functions of single neurons in macaque cortical area V4. J. Neurosci. 19, 431–441 (1999).
Müller, N. G. & Kleinschmidt, A. Dynamic interaction of object- and space-based attention in retinotopic visual areas. J. Neurosci. 23, 9812–9816 (2003).
Carrasco, M., Ling, S. & Read, S. Attention alters appearance. Nat. Neurosci. 7, 308–313 (2004).
Fuller, S. & Carrasco, M. Exogenous attention and color perception: performance and appearance of saturation and hue. Vis. Res. 46, 4032–4047 (2006).
Liu, T., Fuller, S. & Carrasco, M. Attention alters the appearance of motion coherence. Psychon. Bull. Rev. 13, 1091–1096 (2006).
Prins, N. & Kingdom, F. A. A. Applying the model-comparison approach to test specific research hypotheses in psychophysical research using the Palamedes toolbox. Front. Psychol. 9, 1250 (2018).
Scimeca, J. M., Kiyonaga, A. & D’Esposito, M. Reaffirming the sensory recruitment account of working memory. Trends Cogn. Sci. 22, 190–192 (2018).
Albers, A. M. et al. Supplemental information shared representations for working memory and mental imagery in early visual cortex. Curr. Biol. 23, 1427–1431 (2013).
Farah, M. J. Psychophysical evidence for a shared representational medium for mental images and percepts. J. Exp. Psychol. Gen. 114, 91–103 (1985).
Christophel, T. B., Klink, P. C., Spitzer, B., Roelfsema, P. R. & Haynes, J. D. The distributed nature of working memory. Trends Cogn. Sci. 21, 111–124 (2017).
Sreenivasan, K. K., Curtis, C. E. & D’Esposito, M. Revisiting the role of persistent neural activity during working memory. Trends Cogn. Sci. 18, 82–89 (2014).
Miller, E. K. & Cohen, J. D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).
Woolgar, A., Hampshire, A., Thompson, R. & Duncan, J. Adaptive coding of task-relevant information in human frontoparietal cortex. J. Neurosci. 31, 14592–14599 (2011).
Mongillo, G., Barak, O. & Tsodyks, M. Synaptic theory of working memory. Science 319, 1543–1546 (2008).
Gazzaley, A. & Nobre, A. C. Top-down modulation: bridging selective attention and working memory. Trends Cogn. Sci. 16, 129–135 (2012).
Dowd, E. W. & Mitroff, S. R. Attentional guidance by working memory overrides salience cues in visual search. J. Exp. Psychol. Hum. Percept. Perform. 39, 1786–1796 (2013).
Kiyonaga, A. & Egner, T. Working memory as internal attention: toward an integrative account of internal and external selection processes. Psychon. Bull. Rev. 20, 228–242 (2013).
Kohn, A. Visual adaptation: physiology, mechanisms, and functional benefits. J. Neurophysiol. 10461, 3155–3164 (2007).
Montagna, B., Pestilli, F. & Carrasco, M. Attention trades off spatial acuity. Vis. Res. 49, 735–745 (2009).
Gureckis, T. M. et al. psiTurk: an open-source framework for conducting replicable behavioral experiments online. Behav. Res. Methods 48, 829–842 (2016).
Kiyonaga, A. & Egner, T. The working memory stroop effect: when internal representations clash with external stimuli. Psychol. Sci. 25, 1619–1629 (2014).
Crump, M. J. C., McDonnell, J. V. & Gureckis, T. M. Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS One 8, e57410 (2013).
Brady, T. F. & Alvarez, G. A. Hierarchical encoding in visual working memory: ensemble statistics bias memory for individual items. Psychol. Sci. 22, 384–392 (2011).
Brady, T. F. & Alvarez, G. A. No evidence for a fixed object limit in working memory: spatial ensemble representations inflate estimates of working memory capacity for complex objects. J. Exp. Psychol. Learn. Mem. Cogn. 41, 921–929 (2015).
Lew, T. F. & Vul, E. Ensemble clustering in visual working memory biases location memories and reduces the Weber noise of relative positions. J. Vis. 15, 10 (2015).
Shin, H. & Ma, W. J. Crowdsourced single-trial probes of visual working memory for irrelevant features. J. Vis. 16, 10 (2016).
Woodman, G. F. & Luck, S. J. Do the contents of visual working memory automatically influence attentional selection during visual search? J. Exp. Psychol. Hum. Percept. Perform. 33, 363 (2007).
Lakens, D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front. Psychol. 4, 863 (2013).
We thank the colleagues in the psychology department of George Washington University for their helpful discussion. Especially, we thank S. Shomstein, S. Mitroff and M. Behrmann for their insightful comments on this manuscript. This research project was supported by start-up funds granted to D.J.K. by GWU. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
The authors declare no competing interests.
Peer review information: Primary Handling Editor: Marike Schiffer.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Teng, C., Kravitz, D.J. Visual working memory directly alters perception. Nat Hum Behav 3, 827–836 (2019). https://doi.org/10.1038/s41562-019-0640-4
Visual Cognition (2020)
Source information is inherently linked to working memory representation for auditory but not for visual stimuli