Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Spatiotemporal mechanisms for detecting and identifying image features in human vision

Abstract

Our visual system constantly selects salient features in the environment, so that only those features are attended and targeted by further processing efforts to identify them. Models of feature detection hypothesize that salient features are localized based on contrast energy (local variance in intensity) in the visual stimulus. This hypothesis, however, has not been tested directly. We used psychophysical reverse correlation to study how humans detect and identify basic image features (bars and short line segments). Subjects detected a briefly flashed 'target bar' that was embedded in 'noise bars' that randomly changed in intensity over space and time. By studying how the intensity of the noise bars affected performance, we were able to dissociate two processing stages: an early 'detection' stage, whereby only locations of high-contrast energy in the image are selected, followed (after 100 ms) by an 'identification' stage, whereby image intensity at selected locations is used to determine the identity (whether bright or dark) of the target.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Stimuli and reverse-correlation technique.
Figure 2: Mean and variance kernels for two subjects.
Figure 3: Experiment 2.
Figure 4: Mean and variance images for false alarm trials only, from experiment 1, bright bar target.

References

  1. James, W. Principles of Psychology (Henry Holt, New York, 1890).

    Google Scholar 

  2. Kastner, S. & Ungerleider, L.G. Mechanisms of visual attention in the human cortex. Annu. Rev. Neurosci. 23, 315–341 (2000).

    Article  CAS  Google Scholar 

  3. Marr, D. Vision (Freeman, New York, 1982).

    Google Scholar 

  4. Ahumada, A.J. Jr. Perceptual classification images from Vernier acuity masked by noise. Perception 26, 18 (1996).

    Google Scholar 

  5. Ringach, D.L. Tuning of orientation detectors in human vision. Vision Res. 38, 963–972 (1998).

    Article  CAS  Google Scholar 

  6. Eckstein, M.P. & Ahumada, A.J. Jr. Classification images: a tool to analyze visual strategies. J. Vision 2, 1 (2002).

  7. Sagi, D. & Julesz, B. “Where” and “what” in vision. Science 228, 1217–1219 (1985).

    Article  CAS  Google Scholar 

  8. Tolhurst, D.J. & Dealy, R.S. The detection and identification of lines and edges. Vision Res. 15, 1367–1372 (1975).

    Article  CAS  Google Scholar 

  9. Thomas, J.P., Gille, J. & Barker, R.A. Simultaneous detection and identification: theory and data. J. Opt. Soc. Am. A 73, 751–758 (1982).

    Article  Google Scholar 

  10. Posner, M.I. & Cohen, Y. in Attention and Performance X: Control of Language Processes (eds. Bouma, H. & Bowhuis, D.G.) 531–556 (Erlbaum, Hillsdale, NJ, 1984).

    Google Scholar 

  11. Nakayama, K. & Mackeben, M. Sustained and transient components of focal visual attention. Vision Res. 29, 1631–1647 (1989).

    Article  CAS  Google Scholar 

  12. Ziebell, O. & Nothdurft, H.-C. Cueing and pop-out. Vision Res. 39, 2113–2125 (1999).

    Article  CAS  Google Scholar 

  13. Nothdurft, H.-C. Attention shifts to salient targets. Vision Res. (in press).

  14. Morrone, M.C. & Burr, D.C. Feature detection in human vision: a phase-dependent energy model. Proc. R. Soc. Lond. B Biol. Sci. 235, 221–245 (1988).

    Article  CAS  Google Scholar 

  15. Itti, L. & Koch, C. A saliency-based mechanism for overt and covert shifts of visual attention. Vision Res. 40, 1489–1506 (2000).

    Article  CAS  Google Scholar 

  16. Itti, L. & Koch, C. Computational modeling of visual attention. Nat. Rev. Neurosci. 2, 194–203 (2001).

    Article  CAS  Google Scholar 

  17. Li, Z. A saliency map in primary visual cortex. Trends Cogn. Sci. 6, 9–16 (2002).

    Article  Google Scholar 

  18. Constantinidis, C. & Steinmetz, M.A. Neuronal responses in area 7a to multiple-stimulus displays: I. Neurons encode the location of the salient stimulus. Cereb. Cortex 11, 581–591 (2001).

    Article  CAS  Google Scholar 

  19. Supèr, H., Spekreijse, H. & Lamme, A.V. Two distinct modes of sensory processing observed in monkey primary visual cortex (V1). Nat. Neurosci. 4, 304–310 (2001).

    Article  Google Scholar 

  20. Spitzer, H., Desimone, R. & Moran, J. Increased attention enhances both behavioral and neuronal performance. Science 240, 338–340 (1988).

    Article  CAS  Google Scholar 

  21. Roelfsema, P.R., Lamme, V.A.F. & Spekreijse, H. Object-based attention in the primary visual cortex of the macaque monkey. Nature 395, 376–381 (1998).

    Article  CAS  Google Scholar 

  22. Martínez, A. et al. Putting spatial attention on the map: timing and localization of stimulus selection processes in striate and extrastriate visual areas. Vision Res. 41, 1437–1457 (2001).

    Article  Google Scholar 

  23. Gottlieb, J.P., Kusunoki, M. & Goldberg, M.E. The representation of visual salience in monkey parietal cortex. Nature 391, 481–484 (1998).

    Article  CAS  Google Scholar 

  24. Piotrowski, L.N. & Campbell, F.W. A demonstration of the visual importance and flexibility of spatial-frequency amplitude and phase. Perception 11, 337–346 (1982).

    Article  CAS  Google Scholar 

  25. Kustov, A.A. & Robinson, D.L. Shared neural control of attentional shifts and eye movements. Nature 384, 74–77 (1996).

    Article  CAS  Google Scholar 

  26. Corbetta, M. et al. A common network of functional areas for attention and eye movements. Neuron 21, 761–773 (1998).

    Article  CAS  Google Scholar 

  27. Parkhurst, D., Law, K. & Niebur, E. Modeling the role of salience in the allocation of overt visual attention. Vision Res. 42, 107–123 (2002).

    Article  Google Scholar 

  28. Green, D.M. & Swets, J.A. Signal Detection Theory and Psychophysics (Wiley, New York, 1966).

    Google Scholar 

  29. Abbey, C.K. & Eckstein, M.P. Classification image analysis: estimation and statistical inference for two-alternative forced-choice experiments. J. Vision 2, 66–78 (2002).

    Article  Google Scholar 

  30. Efron, B. & Tibshirani, R. An Introduction to the Bootstrap (Chapman & Hall, New York, 1993).

    Book  Google Scholar 

  31. Adelson, E.H. & Bergen, J.R. . Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A 2, 284–299 (1985).

    Article  CAS  Google Scholar 

  32. Pollen, D.A. & Ronner, S.F. Visual cortical neurons as localized spatial frequency filters. IEEE Trans. Sys. Cybern. 13, 907–916 (1983).

    Article  Google Scholar 

  33. Pelli, D.G. Uncertainty explains many aspects of visual contrast detection and discrimination. J. Opt. Soc. Am. A 2, 1508–1530 (1985).

    Article  CAS  Google Scholar 

  34. Ahumada, A.J. Jr. Detection of tones masked by noise: a comparison of human subjects with digital-computer-simulated energy detectors of varying bandwidths. Thesis (Technical Report No. 29), UCLA (1967).

Download references

Acknowledgements

We thank H. Barlow, C. Blakemore, B. Cumming, A. Parker, D. Ringach and B. Zenger-Landolt for useful discussions. P.N. was supported by the McDonnell-Pew Program in Cognitive Neuroscience (while at Oxford) and the Wellcome Trust (while at Stanford). D.J.H. was supported by a National Eye Institute grant.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Peter Neri.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Neri, P., Heeger, D. Spatiotemporal mechanisms for detecting and identifying image features in human vision. Nat Neurosci 5, 812–816 (2002). https://doi.org/10.1038/nn886

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn886

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing