Natural image and receptive field statistics predict saccade sizes

Abstract

Humans and other primates sample the visual environment using saccadic eye movements that shift a high-resolution fovea toward regions of interest to create a clear perception of a scene across fixations. Many mammals, however, like mice, lack a fovea, which raises the question of why they make saccades. Here we describe and test the hypothesis that saccades are matched to natural scene statistics and to the receptive field sizes and adaptive properties of neural populations. Specifically, we determined the minimum amplitude of saccades in natural scenes necessary to provide uncorrelated inputs to model neural populations. This analysis predicts the distributions of observed saccade sizes during passive viewing for nonhuman primates, cats, and mice. Furthermore, disrupting the development of receptive field properties by monocular deprivation changed saccade sizes consistent with this hypothesis. Therefore, natural-scene statistics and the neural representation of natural images appear to be critical factors guiding saccadic eye movements.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Saccade size and frequency increase with increasing image size.
Fig. 2: Saccades increase the average response of cells with adaptive changes in gain.
Fig. 3: Measuring decorrelation distance in natural images.
Fig. 4: Decorrelation distance and RF size predict saccade size.
Fig. 5: Selectivity and tolerance decrease and increase saccade sizes necessary for decorrelation, respectively.
Fig. 6: Decreasing acuity with MD increases saccade sizes.

References

  1. 1.

    Curcio, C. A., Sloan, K. R., Kalina, R. E. & Hendrickson, A. E. Human photoreceptor topography. J. Comp. Neurol. 292, 497–523 (1990).

  2. 2.

    Otero-Millan, J., Macknik, S. L., Langston, R. E. & Martinez-Conde, S. An oculomotor continuum from exploration to fixation. Proc. Natl. Acad. Sci. 110, 6175–6180 (2013).

  3. 3.

    Najemnik, J. & Geisler, W. S. Optimal eye movement strategies in visual search. Nature 434, 387–391 (2005).

  4. 4.

    Dorr, M., Martinetz, T., Gegenfurtner, K. R. & Barth, E. Variability of eye movements when viewing dynamic natural scenes. J. Vis. 10, 28 (2010).

  5. 5.

    Koch, C. & Ullman, S. Shifts in selective visual attention: towards the underlying neural circuitry. Hum. Neurobiol. 4, 219–227 (1985).

  6. 6.

    Yarbus, A. L. Eye Movements and Vision. (Springer, Boston, MA, USA, 1967).

  7. 7.

    Hayhoe, M. & Ballard, D. Eye movements in natural behavior. Trends. Cogn. Sci. 9, 188–194 (2005).

  8. 8.

    Dräger, U. C. & Olsen, J. F. Ganglion cell distribution in the retina of the mouse. Invest. Ophthalmol. Vis. Sci. 20, 285–293 (1981).

  9. 9.

    Jeon, C. J., Strettoi, E. & Masland, R. H. The major cell populations of the mouse retina. J. Neurosci. 18, 8936–8946 (1998).

  10. 10.

    Walls, G. L. The evolutionary history of eye movements. Vision Res. 2, 69–80 (1962).

  11. 11.

    Land, M. F. Motion and vision: why animals move their eyes. J. Comp. Physiol. A. 185, 341–352 (1999).

  12. 12.

    Sakatani, T. & Isa, T. Quantitative analysis of spontaneous saccade-like rapid eye movements in C57BL/6 mice. Neurosci. Res. 58, 324–331 (2007).

  13. 13.

    Wang, L., Liu, M., Segraves, M. A. & Cang, J. Visual experience is required for the development of eye movement maps in the mouse superior colliculus. J. Neurosci. 35, 12281–12286 (2015).

  14. 14.

    Wagor, E., Mangini, N. J. & Pearlman, A. L. Retinotopic organization of striate and extrastriate visual cortex in the mouse. J. Comp. Neurol. 193, 187–202 (1980).

  15. 15.

    Vinje, W. E. & Gallant, J. L. Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 1273–1276 (2000).

  16. 16.

    Gawne, T. J. & Martin, J. M. Responses of primate visual cortical neurons to stimuli presented by flash, saccade, blink, and external darkening. J. Neurophysiol. 88, 2178–2186 (2002).

  17. 17.

    Reppas, J. B., Usrey, W. M. & Reid, R. C. Saccadic eye movements modulate visual responses in the lateral geniculate nucleus. Neuron 35, 961–974 (2002).

  18. 18.

    Stroud, A. C., Ledue, E. E. & Crowder, N. A. Orientation specificity of contrast adaptation in mouse primary visual cortex. J. Neurophysiol. 108, 1381–1391 (2012).

  19. 19.

    Müller, J. R., Metha, A. B., Krauskopf, J. & Lennie, P. Rapid adaptation in visual cortex to the structure of images. Science 285, 1405–1408 (1999).

  20. 20.

    Carandini, M. & Heeger, D. J. Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13, 51–62 (2011).

  21. 21.

    Tolhurst, D. J., Movshon, J. A. & Dean, A. F. The statistical reliability of signals in single neurons in cat and monkey visual cortex. Vision Res. 23, 775–785 (1983).

  22. 22.

    Niell, C. M. & Stryker, M. P. Highly selective receptive fields in mouse visual cortex. J. Neurosci. 28, 7520–7536 (2008).

  23. 23.

    Van Essen, D. C., Newsome, W. T. & Maunsell, J. H. R. The visual field representation in striate cortex of the macaque monkey: asymmetries, anisotropies, and individual variability. Vision Res. 24, 429–448 (1984).

  24. 24.

    Frazor, R. A. & Geisler, W. S. Local luminance and contrast in natural images. Vision Res. 46, 1585–1598 (2006).

  25. 25.

    Kaas, J. H. in Comparative Neurology of the Telencephalon (Ebbesson, S. O. E., ed.) 483–502 (Springer, Boston, MA, USA, 1980).

  26. 26.

    Adelson, E. H. & Bergen, J. R. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A. 2, 284–299 (1985).

  27. 27.

    Mante, V., Frazor, R. A., Bonin, V., Geisler, W. S. & Carandini, M. Independence of luminance and contrast in natural scenes and in the early visual system. Nat. Neurosci. 8, 1690–1697 (2005).

  28. 28.

    Ruderman, D. L. & Bialek, W. Statistics of natural images: scaling in the woods. Phys. Rev. Lett. 73, 814–817 (1994).

  29. 29.

    Cecchi, G. A., Rao, A. R., Xiao, Y. & Kaplan, E. Statistics of natural scenes and cortical color processing. J. Vis. 10, 21 (2010).

  30. 30.

    Mitchell, J. F., Reynolds, J. H. & Miller, C. T. Active vision in marmosets: a model system for visual neuroscience. J. Neurosci. 34, 1183–1194 (2014).

  31. 31.

    Gilchrist, I. D., Brown, V. & Findlay, J. M. Saccades without eye movements. Nature 390, 130–131 (1997).

  32. 32.

    Rust, N. C. & DiCarlo, J. J. Balanced increases in selectivity and tolerance produce constant sparseness along the ventral visual stream. J. Neurosci. 32, 10170–10182 (2012).

  33. 33.

    Geisler, W. S., Perry, J. S., Super, B. J. & Gallogly, D. P. Edge co-occurrence in natural images predicts contour grouping performance. Vision Res. 41, 711–724 (2001).

  34. 34.

    Girshick, A. R., Landy, M. S. & Simoncelli, E. P. Cardinal rules: visual orientation perception reflects knowledge of environmental statistics. Nat. Neurosci. 14, 926–932 (2011).

  35. 35.

    Prusky, G. T. & Douglas, R. M. Developmental plasticity of mouse visual acuity. Eur. J. Neurosci. 17, 167–173 (2003).

  36. 36.

    Gordon, J. A. & Stryker, M. P. Experience-dependent plasticity of binocular responses in the primary visual cortex of the mouse. J. Neurosci. 16, 3274–3286 (1996).

  37. 37.

    Stephany, C. E. et al. Plasticity of binocularity and visual acuity are differentially limited by nogo receptor. J. Neurosci. 34, 11631–11640 (2014).

  38. 38.

    Tabata, H., Shimizu, N., Wada, Y., Miura, K. & Kawano, K. Initiation of the optokinetic response (OKR) in mice. J. Vis. 10, 1–17 (2010).

  39. 39.

    Kang, E. et al. Visual acuity development and plasticity in the absence of sensory experience. J. Neurosci. 33, 17789–17796 (2013).

  40. 40.

    Wässle, H., Grünert, U. & Röhrenbeck, J. & Boycott, B.B. Cortical magnification factor and the ganglion cell density of the primate retina. Nature 341, 643–646 (1989).

  41. 41.

    Goodchild, A. K., Ghosh, K. K. & Martin, P. R. Comparison of photoreceptor spatial density and ganglion cell morphology in the retina of human, macaque monkey, cat, and the marmoset Callithrix jacchus. J. Comp. Neurol. 366, 55–75 (1996).

  42. 42.

    Sterratt, D. C., Lyngholm, D., Willshaw, D. J. & Thompson, I. D. Standard anatomical and visual space for the mouse retina: computational reconstruction and transformation of flattened retinae with the Retistruct package. PLoS Comput. Biol. 9, e1002921 (2013).

  43. 43.

    Salinas, K. J., Figueroa Velez, D. X., Zeitoun, J. H., Kim, H. & Gandhi, S. P. Contralateral bias of high spatial frequency tuning and cardinal direction selectivity in mouse visual cortex. J. Neurosci. 37, 10125–10138 (2017).

  44. 44.

    Segal, I. Y. et al. Decorrelation of retinal response to natural scenes by fixational eye movements. Proc.Natl. Acad. Sci. 112, 3110–3115 (2015).

  45. 45.

    Kuang, X., Poletti, M., Victor, J. D. & Rucci, M. Temporal encoding of spatial information during active visual fixation. Curr. Biol. 22, 510–514 (2012).

  46. 46.

    Martinez-Conde, S., Macknik, S. L., Troncoso, X. G. & Dyar, T. A. Microsaccades counteract visual fading during fixation. Neuron 49, 929 (2006).

  47. 47.

    Groner, M. T., Groner, R. & von Mühlenen, A. The effect of spatial frequency content on parameters of eye movements. Psychol. Res. 72, 601–608 (2008).

  48. 48.

    Seiple, W., Rosen, R. B. & Garcia, P. M. Abnormal fixation in individuals with age-related macular degeneration when viewing an image of a face. Optom. Vis. Sci. 90, 45–56 (2013).

  49. 49.

    Kwon, M., Nandy, A. S. & Tjan, B. S. Rapid and persistent adaptability of human oculomotor control in response to simulated central vision loss. Curr. Biol. 23, 1663–1669 (2013).

  50. 50.

    Barlow, H. Redundancy reduction revisited. Network 12, 241–253 (2001).

  51. 51.

    Scholl, B., Pattadkal, J. J., Dilly, G. A., Priebe, N. J. & Zemelman, B. V. Local integration accounts for weak selectivity of mouse neocortical parvalbumin interneurons. Neuron 87, 424–436 (2015).

  52. 52.

    Mitchell, J. F., Priebe, N. J. & Miller, C. T. Motion dependence of smooth pursuit eye movements in the marmoset. J. Neurophysiol. 113, 3954–3960 (2015).

  53. 53.

    Olmos, A. & Kingdom, F. A. A. A biologically inspired algorithm for the recovery of shading and reflectance images. Perception 33, 1463–1473 (2004).

  54. 54.

    Brainard, D. H. The psychophysics toolbox. Spat. Vis. 10, 433–436 (1997).

  55. 55.

    Samonds, J. M., Choi, V. & Priebe, N. J. Binocular alignment in mice during stereoscopic discrimination of depth. J. Vis. 16, 1329 (2016).

  56. 56.

    Dombeck, D. A., Khabbaz, A. N., Collman, F., Adelman, T. L. & Tank, D. W. Imaging large-scale neural activity with cellular resolution in awake, mobile mice. Neuron 56, 43–57 (2007).

  57. 57.

    Zhou, X. et al. Biometric measurement of the mouse eye using optical coherence tomography with focal plane advancement. Vision Res. 48, 1137–1143 (2008).

  58. 58.

    Park, Hn et al. Assessment of axial length measurements in mouse eyes. Optom. Vis. Sci. 89, 296–303 (2012).

  59. 59.

    Chaplin, T. A., Yu, H. H. & Rosa, M. G. P. Representation of the visual field in the primary visual area of the marmoset monkey: magnification factors, point-image size, and proportionality to retinal ganglion cell density. J. Comp. Neurol. 521, 1001–1019 (2013).

  60. 60.

    Albus, K. A quantitative study of the projection area of the central and the paracentral visual field in area 17 of the cat. II. The spatial organization of the orientation domain.Exp. Brain Res. 24, 181–202 (1975).

  61. 61.

    Wilson, J. R. & Sherman, S. M. Receptive-field characteristics of neurons in cat striate cortex: changes with visual field eccentricity. J. Neurophysiol. 39, 512–533 (1976).

  62. 62.

    Tusa, R. J., Palmer, L. A. & Rosenquist, A. C. The retinotopic organization of area 17 (striate cortex) in the cat. J. Comp. Neurol. 177, 213–235 (1978).

  63. 63.

    Roth, M. M. et al. Thalamic nuclei convey diverse contextual information to layer 1 of visual cortex. Nat. Neurosci. 19, 299–307 (2016).

  64. 64.

    Cavanaugh, J. R., Bair, W. & Movshon, J. A. Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. J. Neurophysiol. 88, 2530–2546 (2002).

  65. 65.

    Barlow, H. B., Blakemore, C. & Pettigrew, J. D. The neural mechanism of binocular depth discrimination. J. Physiol. (Lond.) 193, 327–342 (1967).

  66. 66.

    Campbell, F. W., Cooper, G. F. & Enroth-Cugell, C. The spatial selectivity of the visual cells of the cat. J. Physiol. (Lond.) 203, 223–235 (1969).

  67. 67.

    De Valois, R. L., Albrecht, D. G. & Thorell, L. G. Spatial frequency selectivity of cells in macaque visual cortex. Vision Res. 22, 545–559 (1982).

  68. 68.

    Yu, H. H. et al. Spatial and temporal frequency tuning in striate cortex: functional uniformity and specializations related to receptive field eccentricity. Eur. J. Neurosci. 31, 1043–1062 (2010).

  69. 69.

    De Valois, R. L., Morgan, H. & Snodderly, D. M. Psychophysical studies of monkey vision. 3. Spatial luminance contrast sensitivity tests of macaque and human observers. Vision Res. 14, 75–81 (1974).

  70. 70.

    Blake, R. & Antoinetti, D. N. Abnormal visual resolution in the Siamese cat. Science 194, 109–110 (1976).

  71. 71.

    Prusky, G. T., West, P. W. R. & Douglas, R. M. Behavioral assessment of visual acuity in mice and rats. Vision Res. 40, 2201–2209 (2000).

  72. 72.

    Nummela, S. U. et al. Psychophysical measurement of marmoset acuity and myopia. Dev. Neurobiol. 77, 300–313 (2017).

  73. 73.

    Skottun, B. C. et al. Classifying simple and complex cells on the basis of response modulation. Vision Res. 31, 1079–1086 (1991).

  74. 74.

    Liu, B. H. et al. Visual receptive field structure of cortical inhibitory neurons revealed by two-photon imaging guided recording. J. Neurosci. 29, 10520–10532 (2009).

  75. 75.

    Self, M. W. et al. Orientation-tuned surround suppression in mouse visual cortex. J. Neurosci. 34, 9290–9304 (2014).

  76. 76.

    Van den Bergh, G., Zhang, B., Arckens, L. & Chino, Y. M. Receptive-field properties of V1 and V2 neurons in mice and macaque monkeys. J. Comp. Neurol. 518, 2051–2070 (2010).

  77. 77.

    Liu, L. et al. Spatial structure of neuronal receptive field in awake monkey secondary visual cortex (V2). Proc. Natl. Acad. Sci. USA 113, 1913–1918 (2016).

  78. 78.

    Lee, D. & Malpeli, J. G. Effects of saccades on the activity of neurons in the cat lateral geniculate nucleus. J. Neurophysiol. 79, 922–936 (1998).

  79. 79.

    Körding, K. P., Kayser, C., Betsch, B. Y. & König, P. Non-contact eye-tracking on cats. J. Neurosci. Methods 110, 103–111 (2001).

  80. 80.

    Scholl, B., Pattadkal, J. J. & Priebe, N. J. Binocular disparity selectivity weakened after monocular deprivation in mouse V1. J. Neurosci. 37, 6517–6526 (2017).

Download references

Acknowledgements

We thank A. Laudano, V. Choi, and D. Greer for technical assistance, as well as M. Smith, A. Kohn, and A. Huk for helpful discussions. We are grateful to H.-H. Yu and M. Rosa for sharing marmoset RF data. We thank S. Lisberger and M. Hayhoe for helpful comments on earlier versions of this article. This work was supported by National Health Institutes Grants U01NS094330, EY019288, and EY024662, and by a Human Frontier Science Program Grant.

Author information

J.M.S., W.S.G., and N.J.P. conceived and designed the studies. J.M.S. performed experiments. J.M.S., W.S.G., and N.J.P. analyzed data. J.M.S. wrote the manuscript with contributions from W.S.G. and N.J.P.

Correspondence to Jason M. Samonds.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Integrated supplementary information

Supplementary Figure 1 Experimental setup.

(a), Natural images were displayed to mice running on a floating Styrofoam ball, while we tracked eye position with infrared cameras and running speed with an optical mouse. (b), Saccades were detected when velocity exceeded an initial threshold (red dashed line), and the beginning (green circle) and end (red circle) of saccades were determined as when the velocity fell below a second threshold in opposite directions (green dashed line).

Supplementary Figure 2 Examples of decorrelation distances vs. RF size estimated for individual images using V1 complex cells.

These are representative examples from a sample of n = 392 natural images. The measured distances are plotted along with a fit based on equation 17 in Online Methods.

Supplementary Figure 3

Entire raw probability distribution of decorrelation distances for n = 392 images for all RF sizes from Fig. 4a.

Supplementary Figure 4 Receptive field size and cortical magnification vs. eccentricity.

(a), RF size (square root of area) versus eccentricity replotted from previous studies (see legend) for macaques (n = 248 RF sizes23), marmosets (n = 597 RF sizes58), and cats (n = 643 RF sizes for individual data points59; mean and standard error for n = 178 RF sizes grouped by eccentricity60). All fits are based on the same function used in Van Essen et al.23 (b), Cortical magnification versus eccentricity replotted from previous studies (see legend). The large plot only shows the fits and the first 10° of eccentricity with linear axes to emphasize the severity of the fall-off. The inset shows all of the data (n = 82 cortical magnification estimates for cats5961). The fit for cat data was based on the same function used in Chaplin et al.58 (see Online Methods for details).

Supplementary Figure 5 More examples of changes in predicted saccade sizes as a result of RF selectivity changing.

(a), Predicted median saccade sizes are smaller for RFs with a suppressive surround compared to RFs that only have divisive normalization within the classical RF. *Extrapolated surround suppression prediction for the mouse. (b), Predicted median saccade sizes are larger for RFs that detect cardinal orientations compared to RFs that detect oblique orientations. For (a and b), vertical and horizontal lines in the circles represent 95% confidence intervals of the median (n = 392 images).

Supplementary Figure 6 The effect of decorrelation threshold on predicted saccade sizes.

(a), Median decorrelation distances decrease for increasing decorrelation thresholds (n = 392 images). (b), Predicted median saccade sizes correspondingly decrease with increasing decorrelation thresholds. Decreasing intensity in colour corresponds to different thresholds defined by the decreasing intensity of black in (a). Vertical lines in the circles represent 95% confidence intervals of the median (n = 392 images). Dashed versus solid lines represents different sets of data (see Online Methods for details).

Supplementary Figure 7 MD consistently shifts the OKR spatial frequency curve to lower spatial frequencies for the deprived eye.

(a), Normalized (by the peak) OKR gain versus spatial frequency data points for all four MD mice. b, OKR gain versus spatial frequency was similar between left and right eyes for three control mice and similar to the non-deprived eye of MD mice. For (a and b), data points are medians and error bars are 95% confidence intervals of the median (n = 20 repeats for all mice).

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–7

Reporting Summary

Supplementary Video 1

Example of mouse eye tracking and saccade detection. Circles represent detected saccades.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Further reading