Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Letter
  • Published:

Learning the parts of objects by non-negative matrix factorization


Is perception of the whole based on perception of its parts? There is psychological1 and physiological2,3 evidence for parts-based representations in the brain, and certain computational theories of object recognition rely on such representations4,5. But little is known about how brains or computers might learn the parts of objects. Here we demonstrate an algorithm for non-negative matrix factorization that is able to learn parts of faces and semantic features of text. This is in contrast to other methods, such as principal components analysis and vector quantization, that learn holistic, not parts-based, representations. Non-negative matrix factorization is distinguished from the other methods by its use of non-negativity constraints. These constraints lead to a parts-based representation because they allow only additive, not subtractive, combinations. When non-negative matrix factorization is implemented as a neural network, parts-based representations emerge by virtue of two properties: the firing rates of neurons are never negative and synaptic strengths do not change sign.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Non-negative matrix factorization (NMF) learns a parts-based representation of faces, whereas vector quantization (VQ) and principal components analysis (PCA) learn holistic representations.
Figure 2: Iterative algorithm for non-negative matrix factorization.
Figure 3: Probabilistic hidden variables model underlying non-negative matrix factorization.
Figure 4: Non-negative matrix factorization (NMF) discovers semantic features of m = 30,991 articles from the Grolier encyclopedia.

Similar content being viewed by others


  1. Palmer,S. E. Hierarchical structure in perceptual representation. Cogn. Psychol. 9, 441–474 ( 1977).

    Article  Google Scholar 

  2. Wachsmuth,E., Oram,M. W. & Perrett, D. I. Recognition of objects and their component parts: responses of single units in the temporal cortex of the macaque. Cereb. Cortex 4, 509–522 (1994).

    Article  CAS  PubMed  Google Scholar 

  3. Logothetis,N. K. & Sheinberg,D. L. Visual object recognition. Annu. Rev. Neurosci. 19, 577 –621 (1996).

    Article  CAS  PubMed  Google Scholar 

  4. Biederman,I. Recognition-by-components: a theory of human image understanding. Psychol. Rev. 94, 115–147 (1987).

    Article  PubMed  Google Scholar 

  5. Ullman,S. High-Level Vision: Object Recognition and Visual Cognition (MIT Press, Cambridge, MA, 1996).

    Book  MATH  Google Scholar 

  6. Turk,M. & Pentland,A. Eigenfaces for recognition. J. Cogn. Neurosci. 3, 71–86 (1991).

    Article  CAS  PubMed  Google Scholar 

  7. Field,D. J. What is the goal of sensory coding? Neural Comput. 6, 559–601 (1994).

    Article  Google Scholar 

  8. Foldiak,P. & Young,M. Sparse coding in the primate cortex. The Handbook of Brain Theory and Neural Networks 895 –898 (MIT Press, Cambridge, MA, 1995 ).

    Google Scholar 

  9. Olshausen,B. A. & Field,D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 ( 1996).

    Article  ADS  CAS  PubMed  Google Scholar 

  10. Lee,D. D. & Seung,H. S. Unsupervised learning by convex and conic coding. Adv. Neural Info. Proc. Syst. 9, 515–521 (1997).

    Google Scholar 

  11. Paatero,P. Least squares formulation of robust non-negative factor analysis. Chemometr. Intell. Lab. 37, 23–35 (1997).

    Article  CAS  Google Scholar 

  12. Nakayama,K. & Shimojo,S. Experiencing and perceiving visual surfaces. Science 257, 1357– 1363 (1992).

    Article  ADS  CAS  PubMed  Google Scholar 

  13. Hinton,G. E., Dayan,P., Frey,B. J. & Neal,R. M. The “wake-sleep” algorithm for unsupervised neural networks. Science 268, 1158–1161 (1995).

    Article  ADS  CAS  PubMed  Google Scholar 

  14. Salton,G. & McGill,M. J. Introduction to Modern Information Retrieval (McGraw-Hill, New York, 1983).

    MATH  Google Scholar 

  15. Landauer,T. K. & Dumais,S. T. The latent semantic analysis theory of knowledge. Psychol. Rev. 104, 211–240 (1997).

    Article  Google Scholar 

  16. Jutten,C. & Herault,J. Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture. Signal Proc. 24, 1–10 ( 1991).

    Article  MATH  Google Scholar 

  17. Bell,A. J. & Sejnowski,T. J. An information maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129–1159 ( 1995).

    Article  CAS  PubMed  Google Scholar 

  18. Bartlett,M. S., Lades,H. M. & Sejnowski, T. J. Independent component representations for face recognition. Proc. SPIE 3299, 528–539 (1998).

    Article  ADS  Google Scholar 

  19. Shepp,L. A. & Vardi,Y. Maximum likelihood reconstruction for emission tomography. IEEE Trans. Med. Imaging. 2, 113–122 (1982).

    Article  Google Scholar 

  20. Richardson,W. H. Bayesian-based iterative method of image restoration. J. Opt. Soc. Am. 62, 55–59 ( 1972).

    Article  ADS  Google Scholar 

  21. Lucy,L. B. An iterative technique for the rectification of observed distributions. Astron. J. 74, 745–754 ( 1974).

    Article  ADS  Google Scholar 

  22. Dempster,A. P., Laired,N. M. & Rubin, D. B. Maximum likelihood from incomplete data via the EM algorithm. J. Royal Stat. Soc. 39, 1– 38 (1977).

    MathSciNet  MATH  Google Scholar 

  23. Saul,L. & Pereira,F. Proceedings of the Second Conference on Empirical Methods n Natural Language Processing (eds Cardie, C. & Weischedel, R.) 81–89 (Morgan Kaufmann, San Francisco, 1997).

    Google Scholar 

Download references


We acknowledge the support of Bell Laboratories and MIT. C. Papageorgiou and T. Poggio provided us with the database of faces, and R. Sproat with the Grolier encyclopedia corpus. We thank L. Saul for convincing us of the advantages of EM-type algorithms. We have benefited from discussions with B. Anderson, K. Clarkson, R. Freund, L. Kaufman, E. Rietman, S. Roweis, N. Rubin, J. Tenenbaum, N. Tishby, M. Tsodyks, T. Tyson and M. Wright.

Author information

Authors and Affiliations


Rights and permissions

Reprints and permissions

About this article

Cite this article

Lee, D., Seung, H. Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999).

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI:

This article is cited by


By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing