Neural mechanisms of rapid natural scene categorization in human visual cortex

Abstract

The visual system has an extraordinary capability to extract categorical information from complex natural scenes. For example, subjects are able to rapidly detect the presence of object categories such as animals or vehicles in new scenes that are presented very briefly1,2. This is even true when subjects do not pay attention to the scenes and simultaneously perform an unrelated attentionally demanding task3, a stark contrast to the capacity limitations predicted by most theories of visual attention4,5. Here we show a neural basis for rapid natural scene categorization in the visual cortex, using functional magnetic resonance imaging and an object categorization task in which subjects detected the presence of people or cars in briefly presented natural scenes. The multi-voxel pattern of neural activity in the object-selective cortex evoked by the natural scenes contained information about the presence of the target category, even when the scenes were task-irrelevant and presented outside the focus of spatial attention. These findings indicate that the rapid detection of categorical information in natural scenes is mediated by a category-specific biasing mechanism in object-selective cortex that operates in parallel across the visual field, and biases information processing in favour of objects belonging to the target object category.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Schematic overview of trial layout.
Figure 2: Schematic overview of analysis approach.
Figure 3: Results of multi-voxel pattern analysis.

References

  1. 1

    Potter, M. C. Meaning in visual search. Science 187, 965–966 (1975)

    CAS  Article  Google Scholar 

  2. 2

    Thorpe, S., Fize, D. & Marlot, C. Speed of processing in the human visual system. Nature 381, 520–522 (1996)

    CAS  Article  Google Scholar 

  3. 3

    Li, F. F., VanRullen, R., Koch, C. & Perona, P. Rapid natural scene categorization in the near absence of attention. Proc. Natl Acad. Sci. USA 99, 9596–9601 (2002)

    CAS  Article  Google Scholar 

  4. 4

    Broadbent, D. E. Perception and Communication (Oxford Univ. Press, 1958)

    Google Scholar 

  5. 5

    Treisman, A. M. & Gelade, G. A feature-integration theory of attention. Cognit. Psychol. 12, 97–136 (1980)

    CAS  Article  Google Scholar 

  6. 6

    Fei-Fei, L., VanRullen, R., Koch, C. & Perona, P. Why does natural scene categorization require little attention? Exploring attentional requirements for natural and synthetic stimuli. Vis. Cogn. 12, 893–924 (2005)

    Article  Google Scholar 

  7. 7

    Rousselet, G. A., Fabre-Thorpe, M. & Thorpe, S. J. Parallel processing in high-level categorization of natural images. Nature Neurosci. 5, 629–630 (2002)

    CAS  Article  Google Scholar 

  8. 8

    Wojciulik, E., Kanwisher, N. & Driver, J. Covert visual attention modulates face-specific activity in the human fusiform gyrus: fMRI study. J. Neurophysiol. 79, 1574–1578 (1998)

    CAS  Article  Google Scholar 

  9. 9

    Cox, D. D. & Savoy, R. L. Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19, 261–270 (2003)

    Article  Google Scholar 

  10. 10

    Haxby, J. V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001)

    CAS  Article  Google Scholar 

  11. 11

    Malach, R. et al. Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc. Natl Acad. Sci. USA 92, 8135–8139 (1995)

    CAS  Article  Google Scholar 

  12. 12

    Kriegeskorte, N., Goebel, R. & Bandettini, P. Information-based functional brain mapping. Proc. Natl Acad. Sci. USA 103, 3863–3868 (2006)

    CAS  Article  Google Scholar 

  13. 13

    Maunsell, J. H. & Treue, S. Feature-based attention in visual cortex. Trends Neurosci. 29, 317–322 (2006)

    CAS  Article  Google Scholar 

  14. 14

    Motter, B. C. Neural correlates of attentive selection for color or luminance in extrastriate area V4. J. Neurosci. 14, 2178–2189 (1994)

    CAS  Article  Google Scholar 

  15. 15

    Saenz, M., Buracas, G. T. & Boynton, G. M. Global effects of feature-based attention in human visual cortex. Nature Neurosci. 5, 631–632 (2002)

    CAS  Article  Google Scholar 

  16. 16

    Chelazzi, L., Miller, E. K., Duncan, J. & Desimone, R. A neural basis for visual search in inferior temporal cortex. Nature 363, 345–347 (1993)

    CAS  Article  Google Scholar 

  17. 17

    Duncan, J. EPS mid-career award 2004: brain mechanisms of attention. Q. J. Exp. Psychol. (Colchester) 59, 2–27 (2006)

    Article  Google Scholar 

  18. 18

    Duncan, J. & Humphreys, G. W. Visual search and stimulus similarity. Psychol. Rev. 96, 433–458 (1989)

    CAS  Article  Google Scholar 

  19. 19

    O’Regan, J. K., Rensink, R. A. & Clark, J. J. Change-blindness as a result of ‘mudsplashes’. Nature 398, 34 (1999)

    Article  Google Scholar 

  20. 20

    Rensink, R. A., O’Regan, J. K. & Clark, J. J. To see or not to see: the need for attention to perceive changes in scenes. Psychol. Sci. 8, 368–373 (1997)

    Article  Google Scholar 

  21. 21

    Simons, D. J. & Rensink, R. A. Change blindness: past, present, and future. Trends Cogn. Sci. 9, 16–20 (2005)

    Article  Google Scholar 

  22. 22

    O’Regan, J. K. Solving the “real” mysteries of visual perception: the world as an outside memory. Can. J. Psychol. 46, 461–488 (1992)

    Article  Google Scholar 

  23. 23

    Russell, B. C., Torralba, A., Murphy, K. P. & Freeman, W. T. LabelMe: a database and web-based tool for image annotation. Int. J. Comput. Vis. 77, 157–173 (2008)

    Article  Google Scholar 

  24. 24

    Walther, D. B., Caddigan, E., Fei-Fei, L. & Beck, D. M. Natural scene categories revealed in distributed patterns of activity in the human brain. J. Neurosci. (in the press)

  25. 25

    Cox, R. W. & Hyde, J. S. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 29, 162–173 (1996)

    CAS  Article  Google Scholar 

Download references

Acknowledgements

We thank P. Downing, U. Hasson, S. McMains, K. Norman, T. Stein and A. Treisman for suggestions on earlier versions of the manuscript, members of the Kastner laboratory for help with retinotopic mapping, and N. Oosterhof for help with data analysis. This work was supported by grants from the National Institutes of Health (2RO1 MH64043, 1RO1 EY017699, 2P50 MH–62196) to S.K., and a Microsoft Research New Faculty Fellowship and the F. Moss gift to L.F.-F.

Author Contributions M.V.P., in collaboration with L.F.-F. and S.K., conceived the experiment. M.V.P. acquired and analysed the data. M.V.P., L.F.-F. and S.K. interpreted the results and wrote the paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Marius V. Peelen.

Supplementary information

Supplementary Information

This file contains Supplementary Figures 1-2 with Legends, Supplementary Table 1, Supplementary Methods, a Supplementary Discussion and Supplementary References. (PDF 1690 kb)

PowerPoint slides

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Peelen, M., Fei-Fei, L. & Kastner, S. Neural mechanisms of rapid natural scene categorization in human visual cortex. Nature 460, 94–97 (2009). https://doi.org/10.1038/nature08103

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.