Computational modelling of visual attention

Key Points

  • We review recent work on computational models of focal visual attention, with emphasis on the bottom-up, saliency- or image-based control of attentional deployment. We highlight five important trends that have emerged from the computational literature:

  • First, the perceptual saliency of stimuli critically depends on surrounding context; that is, the same object may or may not appear salient depending on the nature and arrangement of other objects in the scene. Computationally, this means that contextual influences, such as non-classical surround interactions, must be included in models.

  • Second, a unique 'saliency map' topographically encoding for stimulus conspicuity over the visual scene has proved to be an efficient and plausible bottom-up control strategy. Many successful models are based on such architecture, and electrophysiological as well as psychophysical studies have recently supported the idea that saliency is explicitly encoded in the brain.

  • Third, inhibition-of-return (IOR), the process by which the currently attended location is transiently inhibited, is a critical element of attentional deployment. Without IOR, attention would endlessly be attracted towards the most salient stimulus. IOR thus implements a memory of recently visited locations, and allows attention to thoroughly scan our visual environment.

  • Fourth, attention and eye movements tightly interplay, posing computational challenges with respect to the coordinate system used to control attention. Understanding the interaction between overt and covert attention is particularly important for models concerned with visual search.

  • Last, scene understanding and object recognition strongly constrain the selection of attended locations. Although several models have approached, in an information-theoretical sense, the problem of optimally deploying attention to analyse a scene, biologically plausible implementations of such a computational strategy remain to be developed.

Abstract

Five important trends have emerged from recent work on computational models of focal visual attention that emphasize the bottom-up, image-based control of attentional deployment. First, the perceptual saliency of stimuli critically depends on the surrounding context. Second, a unique 'saliency map' that topographically encodes for stimulus conspicuity over the visual scene has proved to be an efficient and plausible bottom-up control strategy. Third, inhibition of return, the process by which the currently attended location is prevented from being attended again, is a crucial element of attentional deployment. Fourth, attention and eye movements tightly interplay, posing computational challenges with respect to the coordinate system used to control attention. And last, scene understanding and object recognition strongly constrain the selection of attended locations. Insights from these five key areas provide a framework for a computational and neurobiological understanding of visual attention.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Flow diagram of a typical model for the control of bottom-up attention.
Figure 2: Recording saliency.
Figure 3: Combined model of attentional selection and object recognition.

References

  1. 1

    James, W. The Principles of Psychology (Harvard Univ. Press, Cambridge, Massachusetts, 1980/1981).

    Google Scholar 

  2. 2

    Treisman, A. M. & Gelade, G. A feature-integration theory of attention. Cogn. Psychol. 12, 97–136 (1980).An influential theory of attention and visual search.

    CAS  PubMed  Google Scholar 

  3. 3

    Bergen, J. R. & Julesz, B. Parallel versus serial processing in rapid pattern discrimination. Nature 303, 696–698 (1983).

    CAS  PubMed  Google Scholar 

  4. 4

    Treisman, A. Features and objects: The fourteenth Bartlett memorial lecture. Q. J. Exp. Psychol. A 40, 201–237 (1988).

    CAS  PubMed  Google Scholar 

  5. 5

    Nakayama, K. & Mackeben, M. Sustained and transient components of focal visual attention. Vision Res. 29, 1631–1647 (1989).

    CAS  PubMed  Google Scholar 

  6. 6

    Braun, J. & Sagi, D. Vision outside the focus of attention . Percept. Psychophys. 48, 45– 58 (1990).

    CAS  PubMed  Google Scholar 

  7. 7

    Hikosaka, O., Miyauchi, S. & Shimojo, S. Orienting a spatial attention — its reflexive, compensatory, and voluntary mechanisms. Brain Res. Cogn. Brain Res. 5, 1–9 ( 1996).

    CAS  PubMed  Google Scholar 

  8. 8

    Braun, J. & Julesz, B. Withdrawing attention at little or no cost: detection and discrimination tasks. Percept. Psychophys. 60, 1–23 (1998 ).

    CAS  PubMed  Google Scholar 

  9. 9

    Braun, J., Itti, L., Lee, D. K., Zenger, B. & Koch, C. in Visual Attention and Neural Circuits (eds Braun, J., Koch, C. & Davis, J.) (MIT, Cambridge, Massachusetts, in the press).

  10. 10

    Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193– 222 (1995).A complete review on attention.

    CAS  PubMed  Google Scholar 

  11. 11

    Crick, F. & Koch, C. Constraints on cortical and thalamic projections: the no-strong-loops hypothesis. Nature 391, 245–250 (1998).

    CAS  PubMed  Google Scholar 

  12. 12

    Hummel, J. E. & Biederman, I. Dynamic binding in a neural network for shape recognition. Psychol. Rev. 99, 480–517 (1992).

    CAS  PubMed  Google Scholar 

  13. 13

    Reynolds, J. H. & Desimone, R. The role of neural mechanisms of attention in solving the binding problem. Neuron 24, 19–29 ( 1999).

    CAS  PubMed  Google Scholar 

  14. 14

    Weichselgartner, E. & Sperling, G. Dynamics of automatic and controlled visual attention. Science 238, 778–780 (1987).

    CAS  PubMed  Google Scholar 

  15. 15

    Miller, E. K. The prefrontal cortex and cognitive control. Nature Rev. Neurosci. 1, 59–65 (2000 ).

    CAS  Google Scholar 

  16. 16

    Hopfinger, J. B., Buonocore, M. H. & Mangun, G. R. The neural mechanisms of top-down attentional control . Nature Neurosci. 3, 284– 291 (2000).

    CAS  PubMed  Google Scholar 

  17. 17

    Corbetta, M., Kincade, J. M., Ollinger, J. M., McAvoy, M. P. & Shulman, G. L. Voluntary orienting is dissociated from target detection in human posterior parietal cortex. Nature Neurosci. 3, 292–297 ( 2000); erratum 3, 521 ( 2000).

    CAS  PubMed  Google Scholar 

  18. 18

    Ungerleider, L. G. & Mishkin, M. in Analysis of Visual Behavior (eds Ingle, D. J., Goodale, M. A. & Mansfield, R. J. W.) 549–586 (MIT, Cambridge, Massachusetts, 1982).

    Google Scholar 

  19. 19

    Koch, C. & Ullman, S. Shifts in selective visual attention: towards the underlying neural circuitry. Hum. Neurobiol. 4, 219–227 (1985). One of the first explicit computational models of bottom-up attention, at the origin of the idea of a 'saliency map'.

    CAS  PubMed  Google Scholar 

  20. 20

    Didday, R. L. & Arbib, M. A. Eye movements and visual perception: A “two visual system” model. Int. J. Man–Machine Studies 7, 547–569 ( 1975).

    Google Scholar 

  21. 21

    Suder, K. & Worgotter, F. The control of low-level information flow in the visual system. Rev. Neurosci. 11, 127–146 (2000).

    CAS  PubMed  Google Scholar 

  22. 22

    Pasupathy, A. & Connor, C. E. Responses to contour features in macaque area v4. J. Neurophysiol. 82, 2490–2502 (1999).

    CAS  PubMed  Google Scholar 

  23. 23

    Braun, J. Shape-from-shading is independent of visual attention and may be a 'texton' . Spat. Vis. 7, 311–322 (1993).

    CAS  PubMed  Google Scholar 

  24. 24

    Sun, J. & Perona, P. Early computation of shape and reflectance in the visual system. Nature 379, 165– 168 (1996).

    CAS  PubMed  Google Scholar 

  25. 25

    Logothetis, N. K., Pauls, J. & Poggio, T. Shape representation in the inferior temporal cortex of monkeys. Curr. Biol. 5, 552– 563 (1995).

    CAS  PubMed  Google Scholar 

  26. 26

    Bar, M. & Biederman, I. Localizing the cortical region mediating visual awareness of object identity. Proc. Natl Acad. Sci. USA 96, 1790–1793 ( 1999).

    CAS  PubMed  Google Scholar 

  27. 27

    Vogels, R., Biederman, I., Bar, M. & Lorincz, A. Inferior temporal neurons show greater sensitivity to nonaccidental than metric differences . J. Cogn. Neurosci. (in the press).

  28. 28

    Kreiman, G., Koch, C. & Fried, I. Category-specific visual responses of single neurons in the human medial temporal lobe. Nature Neurosci. 3, 946– 953 (2000).

    CAS  PubMed  PubMed Central  Google Scholar 

  29. 29

    He, Z. J. & Nakayama, K. Perceiving textures: beyond filtering . Vision Res. 34, 151–162 (1994).

    CAS  PubMed  Google Scholar 

  30. 30

    Treue, S. & Maunsell, J. H. Attentional modulation of visual motion processing in cortical areas MT and MST. Nature 382, 539–541 (1996).

    CAS  PubMed  Google Scholar 

  31. 31

    Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. (Lond.) 160, 106–154 ( 1962).

    CAS  Google Scholar 

  32. 32

    DeSchepper, B. & Treisman, A. Visual memory for novel shapes: implicit coding without attention. J. Exp. Psychol. Learn. Mem. Cogn. 22, 27–47 (1996).

    CAS  PubMed  Google Scholar 

  33. 33

    Lee, D. K., Itti, L., Koch, C. & Braun, J. Attention activates winner-take-all competition among visual filters. Nature Neurosci. 2, 375–381 ( 1999).A detailed neural model is used to quantitatively predict attentional modulation of psychophysical pattern discrimination performance in terms of intensified competition between visual neurons.

    CAS  PubMed  Google Scholar 

  34. 34

    Yeshurun, Y. & Carrasco, M. Attention improves or impairs visual performance by enhancing spatial resolution. Nature 396, 72–75 (1998).

    CAS  PubMed  Google Scholar 

  35. 35

    Mack, A., Tang, B., Tuma, R., Kahn, S. & Rock, I. Perceptual organization and attention. Cogn. Psychol. 24, 475–501 (1992).

    CAS  PubMed  Google Scholar 

  36. 36

    Moore, C. M. & Egeth, H. Perception without attention: evidence of grouping under conditions of inattention. J. Exp. Psychol. Hum. Percept. Perform. 23, 339–352 (1997).

    CAS  PubMed  Google Scholar 

  37. 37

    Motter, B. C. Neural correlates of attentive selection for color or luminance in extrastriate area V4. J. Neurosci. 14, 2178– 2189 (1994).

    CAS  PubMed  Google Scholar 

  38. 38

    Treue, S. & Trujillo, J. C. M. Feature-based attention influences motion processing gain in macaque visual cortex. Nature 399, 575–579 (1999). Investigates two types of feedback attentional modulation: spatial-based, and non-spatial but feature-based.

    CAS  PubMed  Google Scholar 

  39. 39

    Barcelo, F., Suwazono, S. & Knight, R. T. Prefrontal modulation of visual processing in humans . Nature Neurosci. 3, 399– 403 (2000).

    CAS  PubMed  Google Scholar 

  40. 40

    Moran, J. & Desimone, R. Selective attention gates visual processing in the extrastriate cortex. Science 229, 782–784 (1985).

    CAS  PubMed  Google Scholar 

  41. 41

    Niebur, E., Koch, C. & Rosin, C. An oscillation-based model for the neuronal basis of attention. Vision Res. 33, 2789–2802 (1993).

    CAS  PubMed  Google Scholar 

  42. 42

    Chawla, D., Rees, G. & Friston, K. J. The physiological basis of attentional modulation in extrastriate visual areas. Nature Neurosci. 2, 671–676 (1999).

    CAS  PubMed  Google Scholar 

  43. 43

    Reynolds, J. H., Pasternak, T. & Desimone, R. Attention increases sensitivity of V4 neurons. Neuron 26, 703–714 ( 2000).

    CAS  PubMed  Google Scholar 

  44. 44

    Dosher, B. A. & Lu, Z. L. Mechanisms of perceptual attention in precuing of location. Vision Res. 40, 1269–1292 (2000).

    CAS  PubMed  Google Scholar 

  45. 45

    Itti, L., Koch, C. & Braun, J. Revisiting spatial vision: towards a unifying model. J. Opt. Soc. Am. A 17, 1899–1917 ( 2000).

    CAS  Google Scholar 

  46. 46

    Carrasco, M., Penpeci-Talgar, C. & Eckstein, M. Spatial covert attention increases contrast sensitivity across the CSF: support for signal enhancement. Vision Res. 40, 1203–1215 (2000).

    CAS  PubMed  PubMed Central  Google Scholar 

  47. 47

    Deco, G. & Zihl, J. A neurodynamical model of visual attention: Feedback enhancement of spatial resolution in a hierarchical system. J. Comp. Neurosci. (in the press).

  48. 48

    Daugman, J. G. Spatial visual channels in the Fourier plane. Vision Res. 24, 891–910 (1984).

    CAS  PubMed  Google Scholar 

  49. 49

    Palmer, L. A., Jones, J. P. & Stepnoski, R. A. in The Neural Basis of Visual Function (ed. Leventhal, A. G.) 246–265 (CRC, Boca Raton, Florida, 1991).

    Google Scholar 

  50. 50

    Zetzsche, C. et al. Investigation of a sensorimotor system for saccadic scene analysis: an integrated approach. Proc. 5th Int. Conf. Simulation Adaptive Behav. 5, 120–126 (1998).

    Google Scholar 

  51. 51

    Reinagel, P. & Zador, A. M. Natural scene statistics at the centre of gaze. Network Comp. Neural Syst. 10, 341–350 (1999).

    CAS  Google Scholar 

  52. 52

    Barth, E., Zetzsche, C. & Rentschler, I. Intrinsic two-dimensional features as textons. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 15, 1723– 1732 (1998).

    CAS  PubMed  Google Scholar 

  53. 53

    Nothdurft, H. Salience from feature contrast: additivity across dimensions. Vision Res. 40, 1183–1201 (2000).Psychophysical study of how orientation, motion, luminance and colour contrast cues combine to yield the saliency of visual stimuli.

    CAS  PubMed  Google Scholar 

  54. 54

    Wolfe, J. M. Visual search in continuous, naturalistic stimuli. Vision Res. 34, 1187–1195 ( 1994).

    CAS  PubMed  Google Scholar 

  55. 55

    Braun, J. Vision and attention: the role of training. Nature 393, 424–425 (1998).

    CAS  PubMed  Google Scholar 

  56. 56

    Ahissar, M. & Hochstein, S. The spread of attention and learning in feature search: effects of target distribution and task difficulty. Vision Res. 40, 1349–1364 (2000).

    CAS  PubMed  Google Scholar 

  57. 57

    Sigman, M. & Gilbert, C. D. Learning to find a shape. Nature Neurosci. 3, 264–269 (2000).

    CAS  PubMed  Google Scholar 

  58. 58

    Itti, L. & Koch, C. Feature combination strategies for saliency-based visual attention systems. J. Electronic Imaging (in the press).

  59. 59

    Wolfe, J. in Attention (ed. Pashler, H.) 13–74 (University College London, London, 1996).

    Google Scholar 

  60. 60

    Carandini, M. & Heeger, D. J. Summation and division by neurons in primate visual cortex. Science 264, 1333 –1336 (1994).

    CAS  PubMed  Google Scholar 

  61. 61

    Nothdurft, H. C. Texture discrimination by cells in the cat lateral geniculate nucleus. Exp. Brain Res. 82, 48–66 (1990).

    CAS  PubMed  Google Scholar 

  62. 62

    Allman, J., Miezin, F. & McGuinness, E. Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local–global comparisons in visual neurons. Annu. Rev. Neurosci. 8, 407–430 (1985).One of the first reports that activity of a visual neuron can be modulated by the presence of distant stimuli, far outside the neuron's receptive field.

    CAS  Google Scholar 

  63. 63

    Cannon, M. W. & Fullenkamp, S. C. Spatial interactions in apparent contrast: inhibitory effects among grating patterns of different spatial frequencies, spatial positions and orientations. Vision Res. 31, 1985–1998 (1991).

    CAS  PubMed  Google Scholar 

  64. 64

    Sillito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J. & Davis, J. Visual cortical mechanisms detecting focal orientation discontinuities. Nature 378, 492–496 (1995).

    CAS  PubMed  Google Scholar 

  65. 65

    Levitt, J. B. & Lund, J. S. Contrast dependence of contextual effects in primate visual cortex. Nature 387, 73–76 (1997).

    CAS  PubMed  Google Scholar 

  66. 66

    Gilbert, C. D. & Wiesel, T. N. Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex . J. Neurosci. 9, 2432– 2442 (1989).

    CAS  PubMed  Google Scholar 

  67. 67

    Gilbert, C., Ito, M., Kapadia, M. & Westheimer, G. Interactions between attention, context and learning in primary visual cortex. Vision Res. 40, 1217–1226 (2000).

    CAS  PubMed  Google Scholar 

  68. 68

    Ben-Av, M. B., Sagi, D. & Braun, J. Visual attention and perceptual grouping. Percept. Psychophys. 52, 277–294 ( 1992).

    CAS  PubMed  Google Scholar 

  69. 69

    Grossberg, S. & Raizada, R. D. Contrast-sensitive perceptual grouping and object-based attention in the laminar circuits of primary visual cortex. Vision Res. 40, 1413– 1432 (2000).

    CAS  PubMed  Google Scholar 

  70. 70

    Vinje, W. E. & Gallant, J. L. Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 1273–1276 (2000).

    CAS  PubMed  Google Scholar 

  71. 71

    Itti, L., Koch, C. & Niebur, E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Patt. Anal. Mach. Intell. 20, 1254–1259 (1998).

    Google Scholar 

  72. 72

    Tsotsos, J. K. et al. Modeling visual-attention via selective tuning. Artif. Intell. 78, 507–545 (1995).

    Google Scholar 

  73. 73

    Milanese, R., Gil, S. & Pun, T. Attentive mechanisms for dynamic and static scene analysis. Opt. Eng. 34, 2428–2434 ( 1995).

    Google Scholar 

  74. 74

    Itti, L. & Koch, C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res. 40, 1489–1506 (2000).

    CAS  PubMed  Google Scholar 

  75. 75

    Toet, A., Bijl, P., Kooi, F. L. & Valeton, J. M. A High-Resolution Image Dataset for Testing Search and Detection Models (TNO-TM-98–A020) (TNO Human Factors Research Institute, Soesterberg, The Netherlands, 1998).

    Google Scholar 

  76. 76

    Hamker, F. H. in Proc. 5th Neural Comp. Psychol. Workshop (NCPW'98) (eds von Heinke, D., Humphreys, G. W. & Olson, A.) 252–261 (Springer Verlag, London, 1999).

    Google Scholar 

  77. 77

    Laberge, D. & Buchsbaum, M. S. Positron emission tomographic measurements of pulvinar activity during an attention task. J. Neurosci. 10, 613–619 ( 1990).

    CAS  PubMed  Google Scholar 

  78. 78

    Robinson, D. L. & Petersen, S. E. The pulvinar and visual salience. Trends Neurosci. 15, 127–132 (1992).

    CAS  PubMed  Google Scholar 

  79. 79

    Kustov, A. A. & Robinson, D. L. Shared neural control of attentional shifts and eye movements. Nature 384, 74 –77 (1996).

    CAS  PubMed  Google Scholar 

  80. 80

    Gottlieb, J. P., Kusunoki, M. & Goldberg, M. E. The representation of visual salience in monkey parietal cortex. Nature 391, 481– 484 (1998).Electrophysiological experiments in the awake monkey indicating that some neurons explicitly encode for saliency in the posterior parietal cortex.

    CAS  PubMed  Google Scholar 

  81. 81

    Colby, C. L. & Goldberg, M. E. Space and attention in parietal cortex. Annu. Rev. Neurosci. 22, 319– 349 (1999).

    CAS  PubMed  Google Scholar 

  82. 82

    Thompson, K. G. & Schall, J. D. Antecedents and correlates of visual detection and awareness in macaque prefrontal cortex . Vision Res. 40, 1523– 1538 (2000).

    CAS  PubMed  Google Scholar 

  83. 83

    Andersen, R. A., Bracewell, R. M., Barash, S., Gnadt, J. W. & Fogassi, L. Eye position effects on visual, memory, and saccade-related activity in areas lip and 7a of macaque. J. Neurosci. 10, 1176–1196 (1990).

    CAS  PubMed  Google Scholar 

  84. 84

    Pouget, A. & Sejnowski, T. J. Spatial transformations in the parietal cortex using basis functions. J. Cogn. Neurosci. 9, 222–237 (1997).

    CAS  PubMed  Google Scholar 

  85. 85

    Blaser, E., Sperling, G. & Lu, Z. L. Measuring the amplification of attention. Proc. Natl Acad. Sci. USA 96, 11681– 11686 (1999).

    CAS  PubMed  Google Scholar 

  86. 86

    Brefczynski, J. A. & DeYoe, E. A. A physiological correlate of the 'spotlight' of visual attention. Nature Neurosci. 2, 370–374 ( 1999).

    CAS  PubMed  Google Scholar 

  87. 87

    Amari, S. & Arbib, M. A. in Systems Neuroscience (ed. Metzler, J.) 119–165 (Academic, New York, 1977).

    Google Scholar 

  88. 88

    Posner, M. I. & Cohen, Y. in Attention and Performance Vol. X (eds Bouma, H. & Bouwhuis, D.) 531– 556 (Erlbaum, Hillsdale, New Jersey, 1984).

    Google Scholar 

  89. 89

    Kwak, H. W. & Egeth, H. Consequences of allocating attention to locations and to other attributes. Percept. Psychophys. 51, 455–464 (1992).

    CAS  PubMed  Google Scholar 

  90. 90

    Klein, R. M. Inhibition of return. Trends Cogn. Sci. 4, 138–147 (2000).A complete review of inhibition of return.

    CAS  PubMed  Google Scholar 

  91. 91

    Shimojo, S., Tanaka, Y. & Watanabe, K. Stimulus-driven facilitation and inhibition of visual information processing in environmental and retinotopic representations of space. Brain Res. Cogn. Brain Res. 5, 11 –21 (1996).

    CAS  PubMed  Google Scholar 

  92. 92

    Kingstone, A. & Pratt, J. Inhibition of return is composed of attentional and oculomotor processes. Percept. Psychophys. 61, 1046–1054 (1999).

    CAS  PubMed  Google Scholar 

  93. 93

    Taylor, T. L. & Klein, R. M. Visual and motor effects in inhibition of return. J. Exp. Psychol. Hum. Percept. Perform. 26, 1639–1656 (2000).

    CAS  PubMed  Google Scholar 

  94. 94

    Tipper, S. P., Driver, J. & Weaver, B. Object-centred inhibition of return of visual attention . Q. J. Exp. Psychol. A 43, 289– 298 (1991).

    CAS  PubMed  Google Scholar 

  95. 95

    Gibson, B. S. & Egeth, H. Inhibition of return to object-based and environment-based locations. Percept. Psychophys. 55, 323–339 (1994).

    CAS  PubMed  Google Scholar 

  96. 96

    Ro, T. & Rafal, R. D. Components of reflexive visual orienting to moving objects. Percept. Psychophys. 61 , 826–836 (1999).

    CAS  PubMed  Google Scholar 

  97. 97

    Becker, L. & Egeth, H. Mixed reference frames for dynamic inhibition of return. J. Exp. Psychol. Hum. Percept. Perform. 26, 1167–1177 (2000).

    CAS  PubMed  Google Scholar 

  98. 98

    Horowitz, T. S. & Wolfe, J. M. Visual search has no memory. Nature 394, 575– 577 (1998).

    CAS  PubMed  Google Scholar 

  99. 99

    Mozer, M & Sitton, S. in Attention (ed. Pashler, H.) 341–393 (University College London, London, 1996)

    Google Scholar 

  100. 100

    Guigon, E., Grandguillaume, P., Otto, I., Boutkhil, L. & Burnod, Y. Neural network models of cortical functions based on the computational properties of the cerebral cortex. J. Physiol. (Paris) 88, 291–308 (1994).

    CAS  Google Scholar 

  101. 101

    Schill, K., Umkehrer, E., Beinlich, S., Krieger, G. & Zetzsche, C. Scene analysis with saccadic eye movements: top-down and bottom-up modeling. J. Electronic Imaging (in the press).

  102. 102

    Rybak, I. A., Gusakova, V. I., Golovan, A. V., Podladchikova, L. N. & Shevtsova, N. A. A model of attention-guided visual perception and recognition. Vision Res. 38, 2387–2400 ( 1998).

    CAS  PubMed  Google Scholar 

  103. 103

    Deco, G. & Schurmann, B. A hierarchical neural system with attentional top-down enhancement of the spatial resolution for object recognition . Vision Res. 40, 2845– 2859 (2000).

    CAS  PubMed  Google Scholar 

  104. 104

    Stark, L. W. & Choi, Y. S. in Visual Attention and Cognition (eds Zangemeister, W. H., Stiehl, H. S. & Freska, C.) 3– 69 (Elsevier Science B. V., Amsterdam, 1996).

    Google Scholar 

  105. 105

    Stark, L. W. et al. Representation of human vision in the brain: how does human perception recognize images? J. Electronic Imaging (in the press).

  106. 106

    Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nature Neurosci. 2 , 1019–1025 (1999).

    CAS  PubMed  Google Scholar 

  107. 107

    Riesenhuber, M. & Poggio, T. Models of object recognition. Nature Neurosci. S3, 1199– 1204 (2000).

    Google Scholar 

  108. 108

    O'Craven, K. M., Downing, P. E. & Kanwisher, N. fmri evidence for objects as the units of attentional selection. Nature 401, 584– 587 (1999).

    CAS  PubMed  Google Scholar 

  109. 109

    Roelfsema, P. R., Lamme, V. A. & Spekreijse, H. Object-based attention in the primary visual cortex of the macaque monkey. Nature 395, 376– 381 (1998).

    CAS  PubMed  Google Scholar 

  110. 110

    Abrams, R. A. & Law, M. B. Object-based visual attention with endogenous orienting. Percept. Psychophys. 62, 818–833 (2000).

    CAS  PubMed  Google Scholar 

  111. 111

    Webster, M. J. & Ungerleider, L. G. in The Attentive Brain (ed. Parasuraman, R.) 19–34 (MIT, Cambridge, Massachusetts, 1998).

    Google Scholar 

  112. 112

    Shepherd, M., Findlay, J. M. & Hockey, R. J. The relationship between eye movements and spatial attention. Q. J. Exp. Psychol. 38, 475– 491 (1986).

    CAS  Google Scholar 

  113. 113

    Sheliga, B. M., Riggio, L. & Rizzolatti, G. Orienting of attention and eye movements. Exp. Brain Res. 98, 507–522 (1994).

    CAS  PubMed  Google Scholar 

  114. 114

    Hoffman, J. E. & Subramaniam, B. The role of visual attention in saccadic eye movements. Percept. Psychophys. 57, 787–795 ( 1995).

    CAS  PubMed  Google Scholar 

  115. 115

    Kowler, E., Anderson, E., Dosher, B. & Blaser, E. The role of attention in the programming of saccades. Vision Res. 35, 1897–1916 (1995).

    CAS  PubMed  Google Scholar 

  116. 116

    Schall, J. D., Hanes, D. P. & Taylor, T. L. Neural control of behavior: countermanding eye movements . Psychol. Res. 63, 299– 307 (2000).

    CAS  PubMed  Google Scholar 

  117. 117

    Corbetta, M. Frontoparietal cortical networks for directing attention and the eye to visual locations: identical, independent, or overlapping neural systems? Proc. Natl Acad. Sci. USA 95, 831– 838 (1998).

    CAS  PubMed  Google Scholar 

  118. 118

    Nobre, A. C., Gitelman, D. R., Dias, E. C. & Mesulam, M. M. Covert visual spatial orienting and saccades: overlapping neural systems. Neuroimage 11, 210–216 ( 2000).

    CAS  PubMed  Google Scholar 

  119. 119

    Dominey, P. F. & Arbib, M. A. A cortico-subcortical model for generation of spatially accurate sequential saccades. Cereb. Cortex 2, 153–175 (1992).

    CAS  PubMed  Google Scholar 

  120. 120

    Motter, B. C. & Belky, E. J. The guidance of eye movements during active visual search. Vision Res. 38, 1805–1815 (1998).

    CAS  PubMed  Google Scholar 

  121. 121

    Gilchrist, I. D., Heywood, C. A. & Findlay, J. M. Saccade selection in visual search: evidence for spatial frequency specific between-item interactions. Vision Res. 39, 1373–1383 ( 1999).

    CAS  PubMed  Google Scholar 

  122. 122

    Wolfe, J. M. & Gancarz, G. in Basic and Clinical Applications of Vision Science (ed. Lakshminarayanan, V.) 189– 192 (Kluwer Academic, Dordrecht, The Netherlands, 1996 ).

    Google Scholar 

Download references

Acknowledgements

The research carried out in the laboratories of the authors on visual attention is supported by the National Science Foundation, the National Institute of Mental Health and the Office of Naval Research. We thank Alex Pouget for excellent comments and suggestions.

Author information

Affiliations

Authors

Related links

Related links

FURTHER INFORMATION

Laurent Itti's lab

Christof Koch's lab

Supplementary material for Figure 2

Glossary

CENTRE–SURROUND MECHANISMS

These involve neurons that respond to image differences between a small central region and a broader concentric antagonistic surround region.

DORSAL STREAM

Visual brain areas involved in the localization of objects and mostly found in the posterior/ superior part of the brain.

VENTRAL STREAM

Visual brain areas involved in the identification of objects and mostly found in the posterior/ inferior part of the brain.

OVERT ATTENTION

Expression of attention involving eye movements.

COVERT ATTENTION

Expression of attention without eye movements, typically thought of as a virtual 'spotlight'.

INTENSITY CONTRAST

Spatial difference (for example, detected by centre–surround mechanisms) in light intensity (luminance) in a visual scene.

COLOUR OPPONENCY

Spatial difference in colours, computed in the brain using red/green and blue/yellow centre–surround mechanisms.

NEURONAL TUNING

Property of visual neurons to only respond to certain classes of stimuli (for example, vertically orientated bars).

PSYCHOPHYSICAL THRESHOLDS

Smallest difference between two visual stimuli (for example, vertical versus tilted bar) than can reliably (that is, with a given probability of error) be reported by an observer.

HYPERCOLUMN

A patch of cortex including neurons responding to all orientations and many spatial scales, all for a single location in the visual field.

DIFFERENCE-OF-GAUSSIANS

A filter obtained by taking the difference between a narrow Gaussian distribution (the excitatory centre) and a broader Gaussian distribution with the same mean (the inhibitory surround).

GABOR WAVELET

Product of a sinusoidal grating and a two-dimensional Gaussian function.

IMPULSE RESPONSE FUNCTION

The response of a filter to a single pulse (Dirac) stimulus.

SACCADIC EYE MOVEMENT

Very rapid, ballistic eye movement (with speeds up to 800 degrees per second).

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Itti, L., Koch, C. Computational modelling of visual attention. Nat Rev Neurosci 2, 194–203 (2001). https://doi.org/10.1038/35058500

Download citation

Further reading

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing