Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Emergence of complex cell properties by learning to generalize in natural scenes

Nature volume 457, pages 83–86 (2009)Cite this article

Abstract

A fundamental function of the visual system is to encode the building blocks of natural scenes—edges, textures and shapes—that subserve visual tasks such as object recognition and scene understanding. Essential to this process is the formation of abstract representations that generalize from specific instances of visual input. A common view holds that neurons in the early visual system signal conjunctions of image features1,2, but how these produce invariant representations is poorly understood. Here we propose that to generalize over similar images, higher-level visual neurons encode statistical variations that characterize local image regions. We present a model in which neural activity encodes the probability distribution most consistent with a given image. Trained on natural images, the model generalizes by learning a compact set of dictionary elements for image distributions typically encountered in natural scenes. Model neurons show a diverse range of properties observed in cortical cells. These results provide a new functional explanation for nonlinear effects in complex cells3,4,5,6 and offer insight into coding strategies in primary visual cortex (V1) and higher visual areas.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Statistical patterns distinguish local regions of natural scenes.
Figure 2: Distribution coding model.
Figure 3: Model neurons exhibit properties of cortical visual neurons.
Figure 4: Generalization across natural variability.

References

  1. Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. (Lond.) 160, 106–154 (1962)

    Article  CAS  Google Scholar 

  2. Movshon, J. A., Thompson, I. D. & Tolhurst, D. J. Spatial summation in the receptive fields of simple cells in the cat's striate cortex. J. Physiol. (Lond.) 283, 53–77 (1978)

    Article  CAS  Google Scholar 

  3. Bonds, A. B. Role of inhibition in the specification of orientation selectivity of cells in the cat striate cortex. Vis. Neurosci. 2, 41–55 (1989)

    Article  CAS  Google Scholar 

  4. Jones, H. E., Wang, W. & Sillito, A. M. Spatial organization and magnitude of orientation contrast interactions in primate V1. J. Neurophysiol. 88, 2796–2808 (2002)

    Article  CAS  Google Scholar 

  5. Cavanaugh, J. R., Bair, W. & Movshon, J. A. Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. J. Neurophysiol. 88, 2530–2546 (2002)

    Article  Google Scholar 

  6. Chen, X., Han, F., Poo, M. & Dan, Y. Excitatory and suppressive receptive field subunits in awake monkey primary visual cortex (V1). Proc. Natl Acad. Sci. USA 104, 19120–19125 (2007)

    Article  ADS  CAS  Google Scholar 

  7. Chichilnisky, E. J. A simple white noise analysis of neuronal light responses. Network: Comp. Neural Syst. 12, 199–213 (2001)

    Article  CAS  Google Scholar 

  8. Carandini, M., Heeger, D. J. & Movshon, J. A. Linearity and normalization in simple cells of the macaque primary visual cortex. J. Neurosci. 17, 8621–8644 (1997)

    Article  CAS  Google Scholar 

  9. Movshon, J. A., Thompson, I. D. & Tolhurst, D. J. Receptive field organization of complex cells in the cat’s striate cortex. J. Physiol. (Lond.) 283, 79–99 (1978)

    Article  CAS  Google Scholar 

  10. Adelson, E. H. & Bergen, J. R. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A 2, 284–299 (1985)

    Article  ADS  CAS  Google Scholar 

  11. Kobatake, E. & Tanaka, K. Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. J. Neurophysiol. 71, 856–867 (1994)

    Article  CAS  Google Scholar 

  12. Gallant, J. L., Connor, C. E., Rakshit, S., Lewis, J. W. & Van Essen, D. C. Neural responses to polar, hyperbolic, and Cartesian gratings in area V4 of the macaque monkey. J. Neurophysiol. 76, 2718–2739 (1996)

    Article  CAS  Google Scholar 

  13. Connor, C. E., Brincat, S. L. & Pasupathy, A. Transformation of shape information in the ventral pathway. Curr. Opin. Neurobiol. 17, 140–147 (2007)

    Article  CAS  Google Scholar 

  14. Hegdé, J. & Van Essen, D. C. Selectivity for complex shapes in primate visual area V2. J. Neurosci. 20, RC61:1–6. (2000)

  15. Pasupathy, A. & Connor, C. E. Shape representation in area V4: position-specific tuning for boundary conformation. J. Neurophysiol. 86, 2505–2519 (2001)

    Article  CAS  Google Scholar 

  16. Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)

    Article  ADS  CAS  Google Scholar 

  17. Bell, A. J. & Sejnowski, T. J. The “independent components” of natural scenes are edge filters. Vision Res. 37, 3327–3338 (1997)

    Article  CAS  Google Scholar 

  18. Jones, J. P. & Palmer, L. A. The two-dimensional spatial structure of simple receptive fields in cat striate cortex. J. Neurophysiol. 58, 1187–1211 (1987)

    Article  CAS  Google Scholar 

  19. van Hateren, J. H. & van der Schaaf, A. Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. R. Soc. Lond. B 265, 359–366 (1998)

    Article  CAS  Google Scholar 

  20. Heeger, D. J. Normalization of cell responses in cat striate cortex. Vis. Neurosci. 9, 181–197 (1992)

    Article  CAS  Google Scholar 

  21. Heeger, D. J., Simoncelli, E. P. & Movshon, J. A. Computational models of cortical visual processing. Proc. Natl Acad. Sci. USA 93, 623–627 (1996)

    Article  ADS  CAS  Google Scholar 

  22. Rust, N. C., Schwartz, O., Movshon, J. A. & Simoncelli, E. P. Spatiotemporal elements of macaque V1 receptive fields. Neuron 46, 945–956 (2005)

    Article  CAS  Google Scholar 

  23. Cadieu, C. et al. A model of V4 shape selectivity and invariance. J. Neurophysiol. 98, 1733–1750 (2007)

    Article  Google Scholar 

  24. Schwartz, O. & Simoncelli, E. P. Natural signal statistics and sensory gain control. Nature Neurosci. 4, 819–825 (2001)

    Article  CAS  Google Scholar 

  25. Hyvärinen, A. & Hoyer, P. A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Res. 41, 2413–2423 (2001)

    Article  Google Scholar 

  26. Berkes, P. & Wiskott, L. Slow feature analysis yields a rich repertoire of complex cell properties. J. Vis. 5, 579–602 (2005)

    Article  Google Scholar 

  27. Hurri, J. & Hyvärinen, A. Simple-cell-like receptive fields maximize temporal coherence in natural video. Neural Comput. 15, 663–691 (2003)

    Article  Google Scholar 

  28. Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nature Neurosci. 2, 1019–1025 (1999)

    Article  CAS  Google Scholar 

  29. Karklin, Y. & Lewicki, M. S. A hierarchical Bayesian model for learning non-linear statistical regularities in non-stationary natural signals. Neural Comput. 17, 397–423 (2005)

    Article  Google Scholar 

  30. van Hateren, J. H. Processing of natural time series of intensities by the visual system of the blowfly. Vision Res. 37, 3407–3416 (1997)

    Article  CAS  Google Scholar 

  31. Olshausen, B. A. & Field, D. J. Sparse coding of sensory inputs. Curr. Opin. Neurobiol. 14, 481–487 (2004)

    Article  CAS  Google Scholar 

Download references

Acknowledgements

This work was supported by the Department of Energy through the Computational Science Graduate Fellowship (to Y.K.), the National Science Foundation Grant under grant numbers 0413152 and 0705677 (to M.S.L.) and the Office of Naval Research under the Multidisciplinary University Research Initiative N000140710747.

Author Contributions Y.K. and M.S.L. developed the model, analysed the results and wrote the paper; Y.K. ran the simulations.

Author information

Author notes

  1. Yan Karklin & Michael S. Lewicki

    Present address: Present address: Center for Neural Science, New York University, New York, New York, USA (Y.K.); Electrical Engineering and Computer Science Department, Case Western University, Cleveland, Ohio, USA and Wissenschaftskolleg (Institute for Advanced Study) zu Berlin, Germany (M.S.L.).,

Authors and Affiliations

  1. Computer Science Department & Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA 15213, USA,

    Yan Karklin & Michael S. Lewicki

Authors
  1. Yan Karklin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael S. Lewicki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yan Karklin or Michael S. Lewicki.

Supplementary information

Supplementary Information

This file contains Supplementary Discussions 1-5, Supplementary Figures S1-S3 with Legends, Supplementary Methods and Supplementary References. (PDF 275 kb)

PowerPoint slides

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Karklin, Y., Lewicki, M. Emergence of complex cell properties by learning to generalize in natural scenes. Nature 457, 83–86 (2009). https://doi.org/10.1038/nature07481

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature07481

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing