Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Visual features of intermediate complexity and their use in classification

Nature Neuroscience volume 5, pages 682–687 (2002)Cite this article

Abstract

The human visual system analyzes shapes and objects in a series of stages in which stimulus features of increasing complexity are extracted and analyzed. The first stages use simple local features, and the image is subsequently represented in terms of larger and more complex features. These include features of intermediate complexity and partial object views. The nature and use of these higher-order representations remains an open question in the study of visual processing by the primate cortex. Here we show that intermediate complexity (IC) features are optimal for the basic visual task of classification. Moderately complex features are more informative for classification than very simple or very complex ones, and so they emerge naturally by the simple coding principle of information maximization with respect to a class of images. Our findings suggest a specific role for IC features in visual processing and a principle for their extraction.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Intermediate complexity visual features were chosen by maximizing delivered information with respect to a class of objects.
Figure 2: Superiority of intermediate fragments.
Figure 3: Approximating novel faces by fragments.
Figure 4: Face and car detection examples showing broad generalization.
Figure 5: Detection response (equation 3) decreases with degree of image scrambling.

Similar content being viewed by others

References

  1. Barlow, H.B. & Foldiak, P. in The Computing Neuron (eds. Durbin, R., Miall, C. and Mitchison, G.) 54–72 (Addison-Wesley, Reading, Massachusetts, 1989).

    Google Scholar 

  2. Atick, J.J. & Redlich, N.A. What does the retina know about natural scenes? Neural Comput. 4, 196–210 (1992).

    Article  Google Scholar 

  3. Bell, A.J. & Sejnowski, T.J. The 'independent components' of natural scenes are edge filters. Vision Res. 37, 3327–3338 (1997).

    Article  CAS  Google Scholar 

  4. Field, D.J. What is the goal of sensory coding? Neural Comput. 6, 559–601 (1994).

    Article  Google Scholar 

  5. Olshausen, B. & Field, D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).

    Article  CAS  Google Scholar 

  6. Vinje, W.E. & Gallant, J.L. Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 1273–1276 (2000).

    Article  CAS  Google Scholar 

  7. Rolls, E.T. Neural organization of higher visual functions. Curr. Opin. Neurobiol. 1, 275–278 (1991).

    Article  Google Scholar 

  8. Fujita, I., Tanaka, K., Ito, M. & Cheng, K. Columns for visual features of objects in monkey inferotemporal cortex. Nature 360, 343–346 (1992).

    Article  CAS  Google Scholar 

  9. Gallant, J.L., Braun, J. & Van Essen, D.C. Selectivity for polar, hyperbolic, and cartesian gratings in macaque visual cortex. Science 259, 100–103 (1993).

    Article  CAS  Google Scholar 

  10. Tanaka, K. Neuronal mechanisms of object recognition. Science 262, 685–688 (1993).

    Article  CAS  Google Scholar 

  11. Logothetis, N.K., Pauls, J., Bulthoff, H.H. & Poggio, T. Shape representation in the inferior temporal cortex of monkeys. Curr. Biol. 5, 552–563 (1995).

    Article  CAS  Google Scholar 

  12. Wiskott, L., Fellous, J.M., Krüger, N., & von der Malsburg, C. Face recognition by elastic bunch graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 19, 775–779 (1997).

    Article  Google Scholar 

  13. Turk, M. & Pentland, A. Eigenfaces for recognition. J. Cogn. Neurosci. 3, 71–86 (1990).

    Article  Google Scholar 

  14. Belhumeur, P.N., Hespanha, J.P. & Kriegman, D.J. Eigenfaces versus fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19, 711–720 (1997).

    Article  Google Scholar 

  15. Ballard, D.H. & Brown, C.M. Computer Vision (Prentice-Hall, Inglewood Cliffs, New Jersey, 1982).

    Google Scholar 

  16. DeValois, R.L., Albrecht, D.G. & Thorell, L.G. Spatial frequency selectivity of cells in the macaque visual cortex. Vision Res. 22, 545–559 (1982).

    Article  CAS  Google Scholar 

  17. Brunelli, R. & Poggio, T. Face recognition: features versus templates. IEEE Trans. Pattern Anal. Mach. Intell. 15, 1042–1052 (1993).

    Article  Google Scholar 

  18. Lee, D.D. & Seung, H.S. Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999).

    Article  CAS  Google Scholar 

  19. Rosch, E., Mervis, C.B., Gray, W.D., Johnson, D.M. & Boyes-Braem, P. Basic objects in natural categories. Cognit. Psychol. 8, 382–439 (1976).

    Article  Google Scholar 

  20. Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019–1025 (1999).

    Article  CAS  Google Scholar 

  21. Yang, M.H., Kriegman, D.J. & Ahuja, N. Detecting faces in images: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 24, 34–58 (2002).

    Article  CAS  Google Scholar 

  22. Mel, B.W. SEEMORE: combining color, shape, and texture histogramming in a neurally inspired approach to visual object recognition. Neural Comput. 9, 777–804 (1997).

    Article  CAS  Google Scholar 

  23. Ullman, S. & Soloviev, S. Computation of pattern invariance in brain-like structures. Neural Net. 12, 1021–1036 (1999).

    Article  CAS  Google Scholar 

  24. Vogels, R. Effects of image scrambling on inferior temporal cortical responses. Neuroreport 10, 1811–1816 (1999).

    Article  CAS  Google Scholar 

  25. Grill-Spector, K., Kushnier, T., Hendler, T., Edelman, S., Itzchak, Y. & Malach, R. A sequence of object-processing stages revealed by fMRI in human occipital lobe. Hum. Brain Mapp. 6, 316–328 (1998).

    Article  CAS  Google Scholar 

  26. Logothetis, N.K. & Sheinberg, D.L. Visual object recognition. Annu. Rev. Neurosci. 19, 577–621 (1996).

    Article  CAS  Google Scholar 

  27. Bhat, D. & Nayar, K.S., Ordinal measures for image correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 20, 415–423 (1998).

    Article  Google Scholar 

  28. Friedman, N., Geiger, D. & Goldszmidt, M. Bayesian network classifiers. Mach. Learn. 29, 131–163 (1997).

    Article  Google Scholar 

Download references

Acknowledgements

We thank J. Golberger, M. Bar and N. Rubin for helpful discussions. Supported by Grant 99-28 CN-QUA.05 from the James S. McDonnell Foundation and by the Moross Laboratory at the Weizmann Institute. Face images for testing were in part from the to Carnegie Mellon University (CMU) face images database http://www.cs.cmu.edu/~har/faces.html#upright.

Author information

Authors and Affiliations

  1. Department of Computer Science and Applied Mathematics, The Weizmann Institute of Science, PO Box 26, Rehovot, 76100, Israel

    Shimon Ullman, Michel Vidal-Naquet & Erez Sali

Authors
  1. Shimon Ullman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michel Vidal-Naquet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erez Sali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shimon Ullman.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ullman, S., Vidal-Naquet, M. & Sali, E. Visual features of intermediate complexity and their use in classification. Nat Neurosci 5, 682–687 (2002). https://doi.org/10.1038/nn870

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn870

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing