Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Emergence of simple-cell receptive field properties by learning a sparse code for natural images

Abstract

THE receptive fields of simple cells in mammalian primary visual cortex can be characterized as being spatially localized, oriented1–4 and bandpass (selective to structure at different spatial scales), comparable to the basis functions of wavelet transforms5,6. One approach to understanding such response properties of visual neurons has been to consider their relationship to the statistical structure of natural images in terms of efficient coding7–12. Along these lines, a number of studies have attempted to train unsupervised learning algorithms on natural images in the hope of developing receptive fields with similar properties13–18, but none has succeeded in producing a full set that spans the image space and contains all three of the above properties. Here we investigate the proposal8,12 that a coding strategy that maximizes sparseness is sufficient to account for these properties. We show that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex. The resulting sparse image code provides a more efficient representation for later stages of processing because it possesses a higher degree of statistical independence among its outputs.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1

    Hubel, D. H. & Wiesel, T. N. J. Physiol., Lond. 195, 215–244 (1968).

    CAS  Article  Google Scholar 

  2. 2

    De Valois, R. L., Albrecht, D. G. & Thorell, L. G. Vision Res. 22, 545–559 (1982).

    CAS  Article  Google Scholar 

  3. 3

    Jones, J. P. & Palmer, L. A. J. Neurophysiol. 58, 1233–1258 (1987).

    CAS  Article  Google Scholar 

  4. 4

    Parker, A. J. & Hawken, M. J. J. opt. Soc. Am. A 5, 598–605 (1988).

    ADS  CAS  Article  Google Scholar 

  5. 5

    Daugman, J. G. Computational Neuroscience (ed. Schwartz, E.) 403–423 (MIT Press, Cambridge, MA, 1990).

    Google Scholar 

  6. 6

    Field, D. J. in Wavelets, Fractals, and Fourier Transforms (eds Farge, M., Hunt, J. & Vascillicos, C.) 151–193 (Oxford Univ. Press, 1993).

    Google Scholar 

  7. 7

    Srinivasan, M. V., Laughlin, S. B. & Dubs, A. Proc. R. Soc. Lond. B216, 427–459 (1982).

    ADS  CAS  PubMed  Google Scholar 

  8. 8

    Field, D. J. J. opt. Soc. Am. A4, 2379–2394 (1987).

    ADS  CAS  Article  Google Scholar 

  9. 9

    Atick, J. J. Network 3, 213–251 (1992).

    Article  Google Scholar 

  10. 10

    van Hateren, J. H. Nature 360, 68–70 (1992).

    ADS  CAS  Article  Google Scholar 

  11. 11

    Ruderman, D. L. Network 5, 517–548 (1994).

    Article  Google Scholar 

  12. 12

    Field, D. J. Neur. Comput. 6, 559–601 (1994).

    Article  Google Scholar 

  13. 13

    Barrow, H. G. in IEEE First Int. Conf. on Neural Networks Vol. 4, (eds Caudill, M. & Butler, C.) 115–121 (Institute of Electrical and Electronics Engineers, 1994).

    Google Scholar 

  14. 14

    Sanger, T. D. in Advances in Neural Information Processing Systems Vol. I (ed. Touretzky, D.) 11–19 (Morgan-Kaufmann, 1989).

    Google Scholar 

  15. 15

    Hancock, P. J. B., Baddeley, R. J. & Smith, L. S. Network 3, 61–72 (1992).

    Article  Google Scholar 

  16. 16

    Law, C. C. & Cooper, L. N. Proc. natn. Acad. Sci. U.S.A. 91, 7797–7801 (1994).

    ADS  CAS  Article  Google Scholar 

  17. 17

    Fyfe, C. & Baddeley, R. Network 6, 333–344 (1995).

    Article  Google Scholar 

  18. 18

    Schmidhuber, J., Eldracher, M. & Foltin, B. Neur. Comput. 8 773–786 (1996).

    Article  Google Scholar 

  19. 19

    Barlow, H. B. Neur. Comput. 1, 295–311 (1989).

    Article  Google Scholar 

  20. 20

    Linsker, R. Computer 105–117 (March, 1988).

  21. 21

    Olshausen, B. A. & Field, D. J. Network 7, 333–339 (1996).

    CAS  Article  Google Scholar 

  22. 22

    Daugman, J. G. IEEE Trans. biomed. Engng. 36, 107–114 (1989).

    CAS  Article  Google Scholar 

  23. 23

    Harpur, G. F. & Prager, R. W. Network 7, 277–284 (1996).

    CAS  Article  Google Scholar 

  24. 24

    Foldiak, P. Biol. Cybernet. 64, 165–170 (1990).

    CAS  Article  Google Scholar 

  25. 25

    Zemel, R. S. thesis, Univ. Toronto (1993).

  26. 26

    Intrator, N. Neur. Comput. 4, 98–107 (1992).

    Article  Google Scholar 

  27. 27

    Bell, A. J. & Sejnowski, T. J. Neur. Comput. 7, 1129–1159 (1995).

    CAS  Article  Google Scholar 

  28. 28

    Saund, E. Neur. Comput. 7, 51–71 (1995).

    Article  Google Scholar 

  29. 29

    Hinton, G. E., Dayan, P., Frey, B. J. & Neal, R. M. Science 268, 1158–1161 (1995).

    ADS  CAS  Article  Google Scholar 

  30. 30

    Lu, Z. L., Chubb, C. & Sperling, G. Technical Report MBS 96-15 (Institute for Mathematical Behavioral Sciences, University of California at Irvine, 1996).

Download references

Author information

Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Olshausen, B., Field, D. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996). https://doi.org/10.1038/381607a0

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing