Letter | Published:

Emergence of complex cell properties by learning to generalize in natural scenes

Nature volume 457, pages 8386 (01 January 2009) | Download Citation

Abstract

A fundamental function of the visual system is to encode the building blocks of natural scenes—edges, textures and shapes—that subserve visual tasks such as object recognition and scene understanding. Essential to this process is the formation of abstract representations that generalize from specific instances of visual input. A common view holds that neurons in the early visual system signal conjunctions of image features1,2, but how these produce invariant representations is poorly understood. Here we propose that to generalize over similar images, higher-level visual neurons encode statistical variations that characterize local image regions. We present a model in which neural activity encodes the probability distribution most consistent with a given image. Trained on natural images, the model generalizes by learning a compact set of dictionary elements for image distributions typically encountered in natural scenes. Model neurons show a diverse range of properties observed in cortical cells. These results provide a new functional explanation for nonlinear effects in complex cells3,4,5,6 and offer insight into coding strategies in primary visual cortex (V1) and higher visual areas.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    & Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. (Lond.) 160, 106–154 (1962)

  2. 2.

    , & Spatial summation in the receptive fields of simple cells in the cat's striate cortex. J. Physiol. (Lond.) 283, 53–77 (1978)

  3. 3.

    Role of inhibition in the specification of orientation selectivity of cells in the cat striate cortex. Vis. Neurosci. 2, 41–55 (1989)

  4. 4.

    , & Spatial organization and magnitude of orientation contrast interactions in primate V1. J. Neurophysiol. 88, 2796–2808 (2002)

  5. 5.

    , & Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. J. Neurophysiol. 88, 2530–2546 (2002)

  6. 6.

    , , & Excitatory and suppressive receptive field subunits in awake monkey primary visual cortex (V1). Proc. Natl Acad. Sci. USA 104, 19120–19125 (2007)

  7. 7.

    A simple white noise analysis of neuronal light responses. Network: Comp. Neural Syst. 12, 199–213 (2001)

  8. 8.

    , & Linearity and normalization in simple cells of the macaque primary visual cortex. J. Neurosci. 17, 8621–8644 (1997)

  9. 9.

    , & Receptive field organization of complex cells in the cat’s striate cortex. J. Physiol. (Lond.) 283, 79–99 (1978)

  10. 10.

    & Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A 2, 284–299 (1985)

  11. 11.

    & Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. J. Neurophysiol. 71, 856–867 (1994)

  12. 12.

    , , , & Neural responses to polar, hyperbolic, and Cartesian gratings in area V4 of the macaque monkey. J. Neurophysiol. 76, 2718–2739 (1996)

  13. 13.

    , & Transformation of shape information in the ventral pathway. Curr. Opin. Neurobiol. 17, 140–147 (2007)

  14. 14.

    & Selectivity for complex shapes in primate visual area V2. J. Neurosci. 20, RC61:1–6. (2000)

  15. 15.

    & Shape representation in area V4: position-specific tuning for boundary conformation. J. Neurophysiol. 86, 2505–2519 (2001)

  16. 16.

    & Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)

  17. 17.

    & The “independent components” of natural scenes are edge filters. Vision Res. 37, 3327–3338 (1997)

  18. 18.

    & The two-dimensional spatial structure of simple receptive fields in cat striate cortex. J. Neurophysiol. 58, 1187–1211 (1987)

  19. 19.

    & Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. R. Soc. Lond. B 265, 359–366 (1998)

  20. 20.

    Normalization of cell responses in cat striate cortex. Vis. Neurosci. 9, 181–197 (1992)

  21. 21.

    , & Computational models of cortical visual processing. Proc. Natl Acad. Sci. USA 93, 623–627 (1996)

  22. 22.

    , , & Spatiotemporal elements of macaque V1 receptive fields. Neuron 46, 945–956 (2005)

  23. 23.

    et al. A model of V4 shape selectivity and invariance. J. Neurophysiol. 98, 1733–1750 (2007)

  24. 24.

    & Natural signal statistics and sensory gain control. Nature Neurosci. 4, 819–825 (2001)

  25. 25.

    & A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Res. 41, 2413–2423 (2001)

  26. 26.

    & Slow feature analysis yields a rich repertoire of complex cell properties. J. Vis. 5, 579–602 (2005)

  27. 27.

    & Simple-cell-like receptive fields maximize temporal coherence in natural video. Neural Comput. 15, 663–691 (2003)

  28. 28.

    & Hierarchical models of object recognition in cortex. Nature Neurosci. 2, 1019–1025 (1999)

  29. 29.

    & A hierarchical Bayesian model for learning non-linear statistical regularities in non-stationary natural signals. Neural Comput. 17, 397–423 (2005)

  30. 30.

    Processing of natural time series of intensities by the visual system of the blowfly. Vision Res. 37, 3407–3416 (1997)

  31. 31.

    & Sparse coding of sensory inputs. Curr. Opin. Neurobiol. 14, 481–487 (2004)

Download references

Acknowledgements

This work was supported by the Department of Energy through the Computational Science Graduate Fellowship (to Y.K.), the National Science Foundation Grant under grant numbers 0413152 and 0705677 (to M.S.L.) and the Office of Naval Research under the Multidisciplinary University Research Initiative N000140710747.

Author Contributions Y.K. and M.S.L. developed the model, analysed the results and wrote the paper; Y.K. ran the simulations.

Author information

Author notes

    • Yan Karklin
    •  & Michael S. Lewicki

    Present address: Center for Neural Science, New York University, New York, New York, USA (Y.K.); Electrical Engineering and Computer Science Department, Case Western University, Cleveland, Ohio, USA and Wissenschaftskolleg (Institute for Advanced Study) zu Berlin, Germany (M.S.L.).

Affiliations

  1. Computer Science Department & Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA 15213, USA

    • Yan Karklin
    •  & Michael S. Lewicki

Authors

  1. Search for Yan Karklin in:

  2. Search for Michael S. Lewicki in:

Corresponding authors

Correspondence to Yan Karklin or Michael S. Lewicki.

Supplementary information

PDF files

  1. 1.

    Supplementary Information

    This file contains Supplementary Discussions 1-5, Supplementary Figures S1-S3 with Legends, Supplementary Methods and Supplementary References.

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nature07481

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.