Abstract
Vision is dynamic, handling a continuously changing stream of input, yet most models of visual attention are static. Here, we develop a dynamic normalization model of visual temporal attention and constrain it with new psychophysical human data. We manipulated temporal attention—the prioritization of visual information at specific points in time—to a sequence of two stimuli separated by a variable time interval. Voluntary temporal attention improved perceptual sensitivity only over a specific interval range. To explain these data, we modelled voluntary and involuntary attentional gain dynamics. Voluntary gain enhancement took the form of a limited resource over short time intervals, which recovered over time. Taken together, our theoretical and experimental results formalize and generalize the idea of limited attentional resources across space at a single moment to limited resources across time at a single location.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
All behavioural data are publicly available on the Open Science Framework (OSF) (https://osf.io/dkx7n).
Code availability
All custom code for the model is publicly available on OSF (https://osf.io/dkx7n). Code for the behavioural experiments is available on GitHub (https://github.com/racheldenison/temporal-attention).
References
Carandini, M. & Heeger, D. J. Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13, 51–62 (2012).
Heeger, D. J. Normalization of cell responses in cat striate cortex. Vis. Neurosci. 9, 181–197 (1992).
Bonin, V., Mante, V. & Carandini, M. The suppressive field of neurons in lateral geniculate nucleus. J. Neurosci. 25, 10844–10856 (2005).
Busse, L., Wade, A. R. & Carandini, M. Representation of concurrent stimuli by population activity in visual cortex. Neuron 64, 931–942 (2009).
Carandini, M. & Heeger, D. J. Summation and division by neurons in primate visual cortex. Science 264, 1333–1336 (1994).
Ni, A. M. & Maunsell, J. H. R. Spatially tuned normalization explains attention modulation variance within neurons. J. Neurophysiol. 118, 1903–1913 (2017).
Li, H.-H., Carrasco, M. & Heeger, D. J. Deconstructing interocular suppression: attention and divisive normalization. PLoS Comput. Biol. 11, e1004510 (2015).
Li, H.-H., Rankin, J., Rinzel, J., Carrasco, M. & Heeger, D. J. Attention model of binocular rivalry. Proc. Natl Acad. Sci. USA 114, E6192–E6201 (2017).
Ling, S. & Blake, R. Normalization regulates competition for visual awareness. Neuron 75, 531–540 (2012).
Louie, K., LoFaro, T., Webb, R. & Glimcher, P. W. Dynamic divisive normalization predicts time-varying value coding in decision-related circuits. J. Neurosci. 34, 16046–16057 (2014).
Ohshiro, T., Angelaki, D. E. & Deangelis, G. C. A normalization model of multisensory integration. Nat. Neurosci. 14, 775–782 (2011).
Reynolds, J. H. & Heeger, D. J. The normalization model of attention. Neuron 61, 168–185 (2009).
Boynton, G. M. A framework for describing the effects of attention on visual responses. Vis. Res. 49, 1129–1143 (2009).
Lee, J. & Maunsell, J. H. R. A normalization model of attentional modulation of single unit responses. PLoS ONE 4, e4651 (2009).
Maunsell, J. H. R. Neuronal mechanisms of visual attention. Annu. Rev. Vis. Sci. 1, 373–391 (2015).
Schwedhelm, P., Krishna, B. S. & Treue, S. An extended normalization model of attention accounts for feature-based attentional enhancement of both response and coherence gain. PLoS Comput. Biol. 12, e1005225 (2016).
Smith, P. L., Sewell, D. K. & Lilburn, S. D. From shunting inhibition to dynamic normalization: attentional selection and decision-making in brief visual displays. Vis. Res. 116, 219–240 (2015).
Ni, A. M. & Maunsell, J. H. R. Neuronal effects of spatial and feature attention differ due to normalization. J. Neurosci. 39, 5493–5505 (2019).
Beuth, F. & Hamker, F. H. A mechanistic cortical microcircuit of attention for amplification, normalization and suppression. Vis. Res. 116, 241–257 (2015).
Herrmann, K., Heeger, D. J. & Carrasco, M. Feature-based attention enhances performance by increasing response gain. Vis. Res. 74, 10–20 (2012).
Herrmann, K., Montaser-Kouhsari, L., Carrasco, M. & Heeger, D. J. When size matters: attention affects performance by contrast or response gain. Nat. Neurosci. 13, 1554–1559 (2010).
Zhang, X., Japee, S., Safiullah, Z., Mlynaryk, N. & Ungerleider, L. G. A normalization framework for emotional attention. PLoS Biol. 14, e1002578 (2016).
Carandini, M., Heeger, D. J. & Movshon, J. A. Linearity and normalization in simple cells of the macaque primary visual cortex. J. Neurosci. 17, 8621–8644 (1997).
Reynaud, A., Masson, G. S. & Chavane, F. Dynamics of local input normalization result from balanced short- and long-range intracortical interactions in area V1. J. Neurosci. 32, 12558–12569 (2012).
Sit, Y. F., Chen, Y., Geisler, W. S., Miikkulainen, R. & Seidemann, E. Complex dynamics of V1 population responses explained by a simple gain-control model. Neuron 64, 943–956 (2009).
Zhou, J., Benson, N. C., Kay, K. N. & Winawer, J. Compressive temporal summation in human visual cortex. J. Neurosci. 38, 691–709 (2018).
Heeger, D. J. & Zemlianova, K. O. A recurrent circuit implements normalization, simulating the dynamics of V1 activity. Proc. Natl Acad. Sci. U. S. A. 117, 22494–22505 (2020).
Wainwright, M. J., Schwartz, O. & Simoncelli, E. P. in Statistical Theories of the Brain (eds Rao, R. P. et al.) 1–22 (MIT Press, 2002).
Westrick, Z. M., Heeger, D. J. & Landy, M. S. Pattern adaptation and normalization reweighting. J. Neurosci. 36, 9805–9816 (2016).
Wilson, H. R. & Humanski, R. Spatial frequency adaptation and contrast gain control. Vis. Res. 33, 1133–1149 (1993).
Wissig, S. C. & Kohn, A. The influence of surround suppression on adaptation effects in primary visual cortex. J. Neurophysiol. 107, 3370–3384 (2012).
Kaliukhovich, D. A. & Vogels, R. Divisive normalization predicts adaptation-induced response changes in macaque inferior temporal cortex. J. Neurosci. 36, 6116–6128 (2016).
Smith, P. L. & Sewell, D. K. A competitive interaction theory of attentional selection and decision making in brief, multielement displays. Psychol. Rev. 120, 589–627 (2013).
Smith, P. L. & Ratcliff, R. An integrated theory of attention and decision making in visual signal detection. Psychol. Rev. 116, 283–317 (2009).
Carrasco, M. Visual attention: the past 25 years. Vis. Res. 51, 1484–1525 (2011).
Carrasco, M., Ling, S. & Read, S. Attention alters appearance. Nat. Neurosci. 7, 308–313 (2004).
Liu, T., Stevens, S. T. & Carrasco, M. Comparing the time course and efficacy of spatial and feature-based attention. Vis. Res. 47, 108–113 (2007).
Müller, H. J. & Rabbitt, P. M. Reflexive and voluntary orienting of visual attention: time course of activation and resistance to interruption. J. Exp. Psychol. Hum. Percept. Perform. 15, 315–330 (1989).
Nobre, A. C. & van Ede, F. Anticipated moments: temporal structure in attention. Nat. Rev. Neurosci. 19, 34–48 (2018).
Correa, A., Lupiáñez, J. & Tudela, P. Attentional preparation based on temporal expectancy modulates processing at the perceptual level. Psychon. Bull. Rev. 12, 328–334 (2005).
Denison, R. N., Heeger, D. J. & Carrasco, M. Attention flexibly trades off across points in time. Psychon. Bull. Rev. 24, 1142–1151 (2017).
Fernández, A., Denison, R. N. & Carrasco, M. Temporal attention improves perception similarly at foveal and parafoveal locations. J. Vis. 19, 12 (2019).
Rohenkohl, G., Gould, I. C., Pessoa, J. & Nobre, A. C. Combining spatial and temporal expectations to improve visual perception. J. Vis. 14, 8 (2014).
Samaha, J., Bauer, P., Cimaroli, S. & Postle, B. R. Top-down control of the phase of alpha-band oscillations as a mechanism for temporal prediction. Proc. Natl Acad. Sci. USA 112, 8439–8444 (2015).
Anderson, B. & Sheinberg, D. L. Effects of temporal context and temporal expectancy on neural activity in inferior temporal cortex. Neuropsychologia 46, 947–957 (2008).
Correa, A., Lupiáñez, J., Madrid, E. & Tudela, P. Temporal attention enhances early visual processing: a review and new evidence from event-related potentials. Brain Res. 1076, 116–128 (2006).
Coull, J. T. & Nobre, A. C. Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. J. Neurosci. 18, 7426–7435 (1998).
Miniussi, C., Wilding, E. L., Coull, J. T. & Nobre, A. C. Orienting attention in time. Modulation of brain potentials. Brain 122, 1507–1518 (1999).
Denison, R. N., Yuval-Greenberg, S. & Carrasco, M. Directing voluntary temporal attention increases fixational stability. J. Neurosci. 39, 353–363 (2019).
Breitmeyer, B. & Ogmen, H. Visual Masking (Oxford Univ. Press, 2006).
Kahneman, D. Method, findings, and theory in studies of visual masking. Psychol. Bull. 70, 404–425 (1968).
Dux, P. E. & Marois, R. The attentional blink: a review of data and theory. Atten. Percept. Psychophys. 71, 1683–1700 (2009).
Raymond, J. E., Shapiro, K. L. & Arnell, K. M. Temporary suppression of visual processing in an RSVP task: an attentional blink? J. Exp. Psychol. Hum. Percept. Perform. 18, 849–860 (1992).
Chun, M. M. & Potter, M. C. A two-stage model for multiple target detection in rapid serial visual presentation. J. Exp. Psychol. Hum. Percept. Perform. 21, 109–127 (1995).
Wyble, B., Potter, M. C., Bowman, H. & Nieuwenstein, M. Attentional episodes in visual perception. J. Exp. Psychol. Gen. 140, 488–505 (2011).
Potter, M. C., Chun, M. M., Banks, B. S. & Muckenhoupt, M. Two attentional deficits in serial target search: the visual attentional blink and an amodal task-switch deficit. J. Exp. Psychol. Learn. Mem. Cogn. 24, 979–992 (1998).
Auksztulewicz, R., Myers, N. E., Schnupp, J. W. & Nobre, A. C. Rhythmic temporal expectation boosts neural activity by increasing neural gain. J. Neurosci. 39, 9806–9817 (2019).
Cravo, A. M., Rohenkohl, G., Wyart, V. & Nobre, A. C. Temporal expectation enhances contrast sensitivity by phase entrainment of low-frequency oscillations in visual cortex. J. Neurosci. 33, 4002–4010 (2013).
Rohenkohl, G., Cravo, A. M., Wyart, V. & Nobre, A. C. Temporal expectation improves the quality of sensory information. J. Neurosci. 32, 8424–8428 (2012).
Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995).
Giordano, A. M., McElree, B. & Carrasco, M. On the automaticity and flexibility of covert attention: a speed–accuracy trade-off analysis. J. Vis. 9, 30.1–10 (2009).
Luck, S. J., Hillyard, S. A., Mouloua, M. & Hawkins, H. L. Mechanisms of visual–spatial attention: resource allocation or uncertainty reduction? J. Exp. Psychol. Hum. Percept. Perform. 22, 725–737 (1996).
Pestilli, F. & Carrasco, M. Attention enhances contrast sensitivity at cued and impairs it at uncued locations. Vis. Res. 45, 1867–1875 (2005).
Gold, J. I. & Shadlen, M. N. The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–574 (2007).
Ratcliff, R., Smith, P. L., Brown, S. D. & McKoon, G. Diffusion decision model: current issues and history. Trends Cogn. Sci. 20, 260–281 (2016).
Cheal, M., Lyon, D. R. & Hubbard, D. C. Does attention have different effects on line orientation and line arrangement discrimination? Q. J. Exp. Psychol. A, Hum. Exp. Psychol. 43, 825–857 (1991).
Hein, E., Rolke, B. & Ulrich, R. Visual attention and temporal discrimination: differential effects of automatic and voluntary cueing. Vis. Cogn. 13, 29–50 (2006).
Ling, S. & Carrasco, M. Sustained and transient covert attention enhance the signal via different contrast response functions. Vis. Res. 46, 1210–1220 (2006).
Nakayama, K. & Mackeben, M. Sustained and transient components of focal visual attention. Vis. Res. 29, 1631–1647 (1989).
Remington, R. W., Johnston, J. C. & Yantis, S. Involuntary attentional capture by abrupt onsets. Percept. Psychophys. 51, 279–290 (1992).
Ma, W. J., Husain, M. & Bays, P. M. Changing concepts of working memory. Nat. Neurosci. 17, 347–356 (2014).
Reeves, A. & Sperling, G. Attention gating in short-term visual memory. Psychological Rev. 93, 180–206 (1986).
Sperling, G. & Weichselgartner, E. Episodic theory of the dynamics of spatial attention. Psychol. Rev. 102, 503–532 (1995).
Reeves, A. Attention as a unitary concept. Vision 4, 48 (2020).
Bundesen, C. A theory of visual attention. Psychol. Rev. 97, 523–547 (1990).
Bundesen, C., Habekost, T. & Kyllingsbæk, S. A neural theory of visual attention: bridging cognition and neurophysiology. Psychol. Rev. 112, 291–328 (2005).
Bundesen, C., Vangkilde, S. & Petersen, A. Recent developments in a computational theory of visual attention (TVA). Vis. Res. 116, 210–218 (2015).
Jones, M. R. Time Will Tell: A Theory of Dynamic Attending (Oxford Univ. Press, 2019).
Large, E. W. & Jones, M. R. The dynamics of attending: how people track time-varying events. Psychol. Rev. 106, 119–159 (1999).
Vangkilde, S., Coull, J. T. & Bundesen, C. Great expectations: temporal expectation modulates perceptual processing speed. J. Exp. Psychol. Hum. Percept. Perform. 38, 1183–1191 (2012).
Vangkilde, S., Petersen, A. & Bundesen, C. Temporal expectancy in the context of a theory of visual attention. Philos. Trans. R. Soc. B 368, 20130054 (2013).
Anton-Erxleben, K. & Carrasco, M. Attentional enhancement of spatial resolution: linking behavioural and neurophysiological evidence. Nat. Rev. Neurosci. 14, 188–200 (2013).
Carrasco, M. & Barbot, A. How attention affects spatial resolution. Cold Spring Harb. Symp. Quant. Biol. 79, 149–160 (2015).
Lawrence, M. A. & Klein, R. M. Isolating exogenous and endogenous modes of temporal attention. J. Exp. Psychol. Gen. 142, 560–572 (2013).
McCormick, C. R., Redden, R. S., Lawrence, M. A. & Klein, R. M. The independence of endogenous and exogenous temporal attention. Atten. Percept. Psychophys. 80, 1885–1891 (2018).
Moon, J., Choe, S., Lee, S. & Kwon, O. S. Temporal dynamics of visual attention allocation. Sci. Rep. 9, 3664 (2019).
Nieuwenstein, M., Van der Burg, E., Theeuwes, J., Wyble, B. & Potter, M. Temporal constraints on conscious vision: on the ubiquitous nature of the attentional blink. J. Vis. 9, 18.11–14 (2009).
Wyart, V., de Gardelle, V., Scholl, J. & Summerfield, C. Rhythmic fluctuations in evidence accumulation during decision making in the human brain. Neuron 76, 847–858 (2012).
Hilkenmeier, F. & Scharlau, I. Rapid allocation of temporal attention in the attentional blink paradigm. Eur. J. Cogn. Psychol. 22, 1222–1234 (2010).
Martens, S. & Johnson, A. Timing attention: cuing target onset interval attenuates the attentional blink. Mem. Cogn. 33, 234–240 (2005).
Visser, T. A. W., Tang, M. F., Badcock, D. R. & Enns, J. T. Temporal cues and the attentional blink: a further examination of the role of expectancy in sequential object perception. Atten. Percept. Psychophys. 76, 2212–2220 (2014).
Di Lollo, V., Kawahara, J.-I., Shahab Ghorashi, S. M. & Enns, J. T. The attentional blink: resource depletion or temporary loss of control? Psychol. Res. 69, 191–200 (2005).
Shapiro, K. L., Hanslmayr, S., Enns, J. T. & Lleras, A. Alpha, beta: the rhythm of the attentional blink. Psychon. Bull. Rev. 34, 1472–1478 (2017).
Nieuwenhuis, S., Gilzenrat, M. S., Holmes, B. D. & Cohen, J. D. The role of the locus coeruleus in mediating the attentional blink: a neurocomputational theory. J. Exp. Psychol. Gen. 134, 291–307 (2005).
Denison, R. N., Parker, J. A. & Carrasco, M. Modeling pupil responses to rapid sequential events. Behav. Res. Methods 52, 1991–2007 (2020).
Carrasco, M. in The Oxford Handbook of Attention (eds Kastner S. & Nobre A. C.) 183–230 (Oxford Univ. Press, 2014).
DeValois, R. L. & DeValois, K. K. Spatial Vision (Oxford Univ. Press, 1990).
Brainard, D. H. The Psychophysics Toolbox. Spat. Vis. 10, 433–436 (1997).
Kleiner, M., Brainard, D. H. & Pelli, D. G. What’s new in Psychtoolbox-3? Perception 36, ECVP Abstract Supplement (2007).
Pelli, D. G. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat. Vis. 10, 437–442 (1997).
Breitmeyer, B. G. & Ogmen, H. Recent models and findings in visual backward masking: a comparison, review, and update. Percept. Psychophys. 62, 1572–1595 (2000).
Acerbi, L. & Ma, W. J. Practical Bayesian optimization for model fitting with Bayesian adaptive direct search. Proc. Adv. Neural Inform. Process. Syst. 30 (2017).
Burnham, K. P. & Anderson, D. R. Model Selection and Multi-Model Inference: A Practical Information-Theoretic Approach (Springer, 2002).
Acknowledgements
This research was supported by National Institutes of Health National Eye Institute R01 EY019693 to M.C. and D.J.H., R01 EY027401 to M.C., F32 EY025533 to R.N.D. and T32 EY007136 to NYU. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. The authors thank H.-H. Li for consultation on the model and Carrasco Lab members, especially V. Peña for assistance with data collection and A. Fernández and M. Jigo for their comments on the manuscript.
Author information
Authors and Affiliations
Contributions
R.N.D., M.C. and D.J.H. conceived the project, designed the experiment and interpreted the behavioural results and model findings. R.N.D. and D.J.H. conceived the model. R.N.D. implemented the model, conducted the experiment and analysed the data. R.N.D. wrote and all three authors edited the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature Human Behaviour thanks Elkan Akyürek, Benjamin Morillon and Valentin Wyart and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Individual observer data.
Behavioural data for individual observers (data points) at each SOA (separate plots). Valid vs. invalid performance for T1 (purple) and T2 (green). For visualization, individual data across SOAs and cuing conditions were normalized separately for each target by adding a constant to equate the individual target mean with the group mean. This adjusts for individual differences in overall performance for a given target without changing the differences among cueing and SOA conditions, facilitating visualization of the pattern of data across these factors. (a) Perceptual sensitivity (d’). Data points lying above the unity line have a temporal cueing effect: higher d’ for valid than invalid trials. The improvement of d’ with temporal attention specifically for intermediate SOAs was consistent across individual observers. (b) Reaction time (RT). Data points lying below the unity line have a temporal cueing effect: faster RT for valid than invalid trials. Reaction time improvements were consistent across observers.
Extended Data Fig. 2 Behavioural statistics.
Repeated measures ANOVA table for behavioural data. SOA = stimulus onset asynchrony, dfn = degrees of freedom in the numerator, dfd = degrees of freedom in the denominator.
Supplementary information
Supplementary information
Supplementary Results, Supplementary Figs. 1–4 and Supplementary Tables 1–4.
Rights and permissions
About this article
Cite this article
Denison, R.N., Carrasco, M. & Heeger, D.J. A dynamic normalization model of temporal attention. Nat Hum Behav 5, 1674–1685 (2021). https://doi.org/10.1038/s41562-021-01129-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41562-021-01129-1
This article is cited by
-
The effect of attention shifting on Chinese children’s word reading in primary school
Psicologia: Reflexão e Crítica (2024)
-
Visual temporal attention from perception to computation
Nature Reviews Psychology (2024)
-
Task demand mediates the interaction of spatial and temporal attention
Scientific Reports (2024)
-
When temporal attention interacts with expectation
Scientific Reports (2024)
-
Behavioral gain following isolation of attention
Scientific Reports (2021)