Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Saccade-confounded image statistics explain visual crowding

Abstract

Processing of shape information in human peripheral visual fields is impeded beyond what can be expected by poor spatial resolution. Visual crowding, the inability to identify objects in clutter, has been shown to be the primary factor limiting shape perception in peripheral vision. Despite the well-documented effects of crowding, its underlying causes remain poorly understood. Given that spatial attention both facilitates learning of image statistics and directs saccadic eye movements, we propose that the acquisition of image statistics in peripheral visual fields is confounded by eye-movement artifacts. Specifically, the image statistics acquired under a peripherally deployed spotlight of attention are systematically biased by saccade-induced image displacements. These erroneously represented image statistics lead to inappropriate contextual interactions in the periphery and cause crowding.

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

All prices are NET prices.

Figure 1: Characteristics of crowding in peripheral vision.
Figure 2: The interaction of spatial attention and saccades.
Figure 3: Spatial consequences of isotropic lateral interaction zone in V1.
Figure 4: Pair-wise image statistics.
Figure 5: Zones of inappropriate integration.

References

  1. Levi, D.M. Crowding—an essential bottleneck for object recognition: a mini-review. Vision Res. 48, 635–654 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  2. Bouma, H. Interaction effects in parafoveal letter recognition. Nature 226, 177–178 (1970).

    CAS  Article  PubMed  Google Scholar 

  3. Levi, D.M., Hariharan, S. & Klein, S. Suppressive and facilitatory spatial interactions in peripheral vision: peripheral crowding is neither size invariant nor simple contrast masking. J. Vis. 2, 167–177 (2002).

    PubMed  Google Scholar 

  4. Pelli, D.G., Palomares, M.C. & Majaj, N.J. Crowding is unlike ordinary masking: distinguishing feature integration from detection. J. Vis. 4, 1136–1169 (2004).

    PubMed  Google Scholar 

  5. Nandy, A.S. & Tjan, B.S. The nature of letter crowding as revealed by first- and second-order classification images. J. Vis. 7, 1–26 (2007).

    Article  PubMed  Google Scholar 

  6. Greenwood, J.A., Bex, P. & Dakin, S.C. Positional averaging explains crowding with letter-like stimuli. Proc. Natl. Acad. Sci. USA 106, 13130–13135 (2009).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  7. He, S., Cavanagh, P. & Intriligator, J. Attentional resolution and the locus of visual awareness. Nature 383, 334–337 (1996).

    CAS  Article  PubMed  Google Scholar 

  8. Petrov, Y., Popple, A.V. & McKee, S.P. Crowding and surround suppression: not to be confused. J. Vis. 7, 1–9 (2007).

    PubMed  Google Scholar 

  9. Toet, A. & Levi, D.M. The two-dimensional shape of spatial interaction zones in the parafovea. Vision Res. 32, 1349–1357 (1992).

    CAS  Article  PubMed  Google Scholar 

  10. Pelli, D.G. Crowding: a cortical constraint on object recognition. Curr. Opin. Neurobiol. 18, 445–451 (2008).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  11. Motter, B.C. & Simoni, D.A. The roles of cortical image separation and size in active visual search performance. J. Vis. 7, 1–15 (2007).

    Article  PubMed  Google Scholar 

  12. Petrov, Y. & Popple, A.V. Crowding is directed to the fovea and preserves only feature contrast. J. Vis. 7, 1–9 (2007).

    PubMed  Google Scholar 

  13. van den Berg, R. & Roerdink, J.B.T.M. & Cornelissen, F.W. A neurophysiologically plausible population code model for feature integration explains visual crowding. PLoS Comput. Biol. 6, e1000646 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  14. Geisler, W.S. Visual perception and the statistical properties of natural scenes. Annu. Rev. Psychol. 59, 167–192 (2008).

    Article  PubMed  Google Scholar 

  15. Karklin, Y. & Lewicki, M.S. Emergence of complex cell properties by learning to generalize in natural scenes. Nature 457, 83–86 (2009).

    CAS  Article  PubMed  Google Scholar 

  16. Olshausen, B.A. & Field, D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).

    CAS  Article  PubMed  Google Scholar 

  17. Kapadia, M.K., Ito, M., Gilbert, C.D. & Westheimer, G. Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in V1 of alert monkeys. Neuron 15, 843–856 (1995).

    CAS  Article  PubMed  Google Scholar 

  18. Stettler, D.D., Das, A., Bennett, J. & Gilbert, C.D. Lateral connectivity and contextual interactions in macaque primary visual cortex. Neuron 36, 739–750 (2002).

    CAS  Article  PubMed  Google Scholar 

  19. Sigman, M., Cecchi, G.A., Gilbert, C.D. & Magnasco, M.O. On a common circle: natural scenes and Gestalt rules. Proc. Natl. Acad. Sci. USA 98, 1935–1940 (2001).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  20. Geisler, W.S., Perry, J.S., Super, B.J. & Gallogly, D.P. Edge co-occurrence in natural images predicts contour grouping performance. Vision Res. 41, 711–724 (2001).

    CAS  Article  PubMed  Google Scholar 

  21. Ito, M. & Gilbert, C.D. Attention modulates contextual influences in the primary visual cortex of alert monkeys. Neuron 22, 593–604 (1999).

    CAS  Article  PubMed  Google Scholar 

  22. Gilbert, C., Ito, M., Kapadia, M.K. & Westheimer, G. Interactions between attention, context and learning in primary visual cortex. Vision Res. 40, 1217–1226 (2000).

    CAS  Article  PubMed  Google Scholar 

  23. Deubel, H. & Schneider, W.X. Saccade target selection and object recognition: evidence for a common attentional mechanism. Vision Res. 36, 1827–1837 (1996).

    CAS  Article  PubMed  Google Scholar 

  24. Motter, B.C. Crowding and object integration within the receptive field of V4 neurons. J. Vis. 2, 274 (2002).

    Article  Google Scholar 

  25. Ben-Shahar, O. & Zucker, S. Geometrical computations explain projection patterns of long-range horizontal connections in visual cortex. Neural Comput. 16, 445–476 (2004).

    Article  PubMed  Google Scholar 

  26. Bosking, W.H., Zhang, Y., Schofield, B. & Fitzpatrick, D. Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. J. Neurosci. 17, 2112–2127 (1997).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  27. Greenwood, J.A., Bex, P.J. & Dakin, S.C. Crowding changes appearance. Curr. Biol. 20, 496–501 (2010).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  28. Parkes, L., Lund, J., Angelucci, A., Solomon, J.A. & Morgan, M. Compulsory averaging of crowded orientation signals in human vision. Nat. Neurosci. 4, 739–744 (2001).

    CAS  Article  PubMed  Google Scholar 

  29. Levi, D.M. & Carney, T. Crowding in peripheral vision: why bigger is better. Curr. Biol. 19, 1988–1993 (2009).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  30. Freeman, J. & Simoncelli, E.P. Metamers of the ventral stream. Nat. Neurosci. 14, 1195–1201 (2011).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  31. Bahill, A.T., Adler, D. & Stark, L. Most naturally occurring human saccades have magnitudes of 15 degrees or less. Invest. Ophthalmol. Vis. Sci. 14, 68–69 (1975).

    Google Scholar 

  32. Levi, D.M., Song, S. & Pelli, D.G. Amblyopic reading is crowded. J. Vis. 7, 1–17 (2007).

    Article  PubMed  Google Scholar 

  33. Bruce, N.D.B. & Tsotsos, J.K. A statistical basis for visual field anisotropies. Neurocomputing 69, 1301–1304 (2006).

    Article  Google Scholar 

  34. White, J.M. & Bedell, H.E. The oculomotor reference in humans with bilateral macular disease. Invest. Ophthalmol. Vis. Sci. 31, 1149–1161 (1990).

    CAS  PubMed  Google Scholar 

  35. Neri, P. & Levi, D.M. Spatial resolution for feature binding is impaired in peripheral and amblyopic vision. J. Neurophysiol. 96, 142–153 (2006).

    Article  PubMed  Google Scholar 

  36. Itti, L. & Koch, C. Computational modeling of visual attention. Nat. Rev. Neurosci. 2, 194–203 (2001).

    CAS  Article  PubMed  Google Scholar 

  37. Diamond, M.R., Ross, J. & Morrone, M.C. Extraretinal control of saccadic suppression. J. Neurosci. 20, 3449–3455 (2000).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  38. Campbell, F.W. & Wurtz, R.H. Saccadic omission: why we do not see a grey-out during a saccadic eye movement. Vision Res. 18, 1297–1303 (1978).

    CAS  Article  PubMed  Google Scholar 

  39. Wurtz, R.H. Neuronal mechanisms of visual stability. Vision Res. 48, 2070–2089 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  40. García-Pérez, M.A. & Peli, E. Intrasaccadic perception. J. Neurosci. 21, 7313–7322 (2001).

    Article  PubMed  PubMed Central  Google Scholar 

  41. De Pisapia, N., Kaunitz, L. & Melcher, D. Backward masking and unmasking across saccadic eye movements. Curr. Biol. 20, 613–617 (2010).

    CAS  Article  PubMed  Google Scholar 

  42. Watson, T.L. & Krekelberg, B. The relationship between saccadic suppression and perceptual stability. Curr. Biol. 19, 1040–1043 (2009).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  43. Castet, E., Jeanjean, S. & Masson, G.S. Motion perception of saccade-induced retinal translation. Proc. Natl. Acad. Sci. USA 99, 15159–15163 (2002).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  44. Gattass, R., Gross, C.G. & Sandell, J.H. Visual topography of V2 in the macaque. J. Comp. Neurol. 201, 519–539 (1981).

    CAS  Article  PubMed  Google Scholar 

  45. Piñon, M.C., Gattass, R. & Sousa, A.P. Area V4 in Cebus monkey: extent and visuotopic organization. Cereb. Cortex 8, 685–701 (1998).

    Article  PubMed  Google Scholar 

  46. Motter, B.C. Central V4 receptive fields are scaled by the V1 cortical magnification and correspond to a constant-sized sampling of the V1 surface. J. Neurosci. 29, 5749–5757 (2009).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  47. Whitney, D. & Levi, D.M. Visual crowding: a fundamental limit on conscious perception and object recognition. Trends Cogn. Sci. 15, 160–168 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  48. Larsson, J. & Heeger, D.J. Two retinotopic visual areas in human lateral occipital cortex. J. Neurosci. 26, 13128–13142 (2006).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  49. Wilson, J.R. & Sherman, S.M. Receptive-field characteristics of neurons in cat striate cortex: changes with visual field eccentricity. J. Neurophysiol. 39, 512–533 (1976).

    CAS  Article  PubMed  Google Scholar 

  50. Lebedev, S., Gelder, P.V. & Tsui, W.H. Square-root relations between main saccadic parameters. Invest. Ophthalmol. Vis. Sci. 37, 2750–2758 (1996).

    CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank I. Biederman, A. Disney, J. Hirsch, M. Jadi, R. Millin and J. Reynolds for helpful comments on the manuscript. This research was supported by US National Institutes of Health grant EY017707 (B.S.T.).

Author information

Authors and Affiliations

Authors

Contributions

A.S.N. and B.S.T. developed the theory, designed the experiments and collected the data. A.S.N. analyzed the data and ran the model simulations. A.S.N. and B.S.T. wrote the manuscript.

Corresponding authors

Correspondence to Anirvan S Nandy or Bosco S Tjan.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–5, Supplementary Table 1 and Supplementary Note (PDF 6792 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Nandy, A., Tjan, B. Saccade-confounded image statistics explain visual crowding. Nat Neurosci 15, 463–469 (2012). https://doi.org/10.1038/nn.3021

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3021

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing