Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Sparse coding with memristor networks

Abstract

Sparse representation of information provides a powerful means to perform feature extraction on high-dimensional data and is of broad interest for applications in signal processing, computer vision, object recognition and neurobiology. Sparse coding is also believed to be a key mechanism by which biological neural systems can efficiently process a large amount of complex sensory data while consuming very little power. Here, we report the experimental implementation of sparse coding algorithms in a bio-inspired approach using a 32 × 32 crossbar array of analog memristors. This network enables efficient implementation of pattern matching and lateral neuron inhibition and allows input data to be sparsely encoded using neuron activities and stored dictionary elements. Different dictionary sets can be trained and stored in the same system, depending on the nature of the input signals. Using the sparse coding algorithm, we also perform natural image processing based on a learned dictionary.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Memristor crossbar array-based computing hardware system.
Figure 2: Experimental demonstration of sparse coding using memristor network.
Figure 3: Sparse coding of different inputs using a more overcomplete dictionary.
Figure 4: Natural image processing using the memristor crossbar.

Similar content being viewed by others

References

  1. Chua, L. O. Memristor—the missing circuit element. IEEE Trans. Circuit Theory 18, 507–519 (1971).

    Article  Google Scholar 

  2. Strukov, D. B., Snider, G. S., Stewart, D. R. & Williams, R. S. The missing memristor found. Nature 453, 80–83 (2008).

    Article  CAS  Google Scholar 

  3. Waser, R. & Aono, M. Nanoionics-based resistive switching memories. Nat. Mater. 6, 833–840 (2007).

    Article  CAS  Google Scholar 

  4. Yang, Y., Chang, T. & Lu, W. in Memristors and Memristive Systems 195–221 (Springer, 2014).

  5. Kim, K.-H. et al. A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. Nano Lett. 12, 389–395 (2012).

    Article  CAS  Google Scholar 

  6. Xia, Q. et al. Memristor–CMOS hybrid integrated circuits for reconfigurable logic. Nano Lett. 9, 3640–3645 (2009).

    Article  CAS  Google Scholar 

  7. Pershin, Y. V. & Di Ventra, M. Practical approach to programmable analog circuits with memristors. IEEE Trans. Circuits Syst. I Regul. Pap. 57, 1857–1864 (2010).

    Article  Google Scholar 

  8. Jo, S. H. et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297–1301 (2010).

    Article  CAS  Google Scholar 

  9. Pershin, Y. V. & Di Ventra, M. Experimental demonstration of associative memory with memristive neural networks. Neural Networks 23, 881–886 (2010).

    Article  Google Scholar 

  10. Du, C., Ma, W., Chang, T., Sheridan, P. & Lu, W. D. Biorealistic implementation of synaptic functions with oxide memristors through internal ionic dynamics. Adv. Funct. Mater. 25, 4290–4299 (2015).

    Article  CAS  Google Scholar 

  11. Kuzum, D., Jeyasingh, R. G. D., Lee, B. & Wong, H.-S. P. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett. 12, 2179–2186 (2012).

    Article  CAS  Google Scholar 

  12. Ohno, T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat. Mater. 10, 591–595 (2011).

    Article  CAS  Google Scholar 

  13. Yang, J. J., Strukov, D. B. & Stewart, D. R. Memristive devices for computing. Nat. Nanotech. 8, 13–24 (2013).

    Article  CAS  Google Scholar 

  14. Sheridan, P. M., Du, C. & Lu, W. D. Feature extraction using memristor networks. IEEE Trans. Neural Networks Learn. Syst. 27, 2327–2336 (2016).

    Article  Google Scholar 

  15. Legenstein, R. Computer science: nanoscale connections for brain-like circuits. Nature 521, 37–38 (2015).

    Article  CAS  Google Scholar 

  16. Alibart, F., Zamanidoost, E. & Strukov, D. B. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun. 4, 2072 (2013).

    Article  Google Scholar 

  17. Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal–oxide memristors. Nature 521, 61–64 (2015).

    Article  CAS  Google Scholar 

  18. Burr, G. W. et al. in 2014 IEEE International Electron Devices Meeting 29.5.1–29.5.4 (IEEE, 2014).

  19. Guo, X. et al. Modeling and experimental demonstration of a Hopfield network analog-to-digital converter with hybrid CMOS/memristor circuits. Front. Neurosci. 9, 488 (2015).

    Article  CAS  Google Scholar 

  20. Agarwal, S. et al. Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding. Front. Neurosci. 9, 484 (2016).

    Article  Google Scholar 

  21. Kadetotad, D. et al. in Proceedings of the Biomedical Circuits and Systems Conference (BioCAS) 536–539 (IEEE, 2014).

  22. Földiák, P. & Young, M. P. Sparse coding in the primate cortex. Handb. Brain Theory Neural Netw. 1, 1064–1068 (1995).

    Google Scholar 

  23. Vinje, W. E. Sparse coding and decorrelation in primary visual cortex during natural vision. Science. 287, 1273–1276 (2000).

    Article  CAS  Google Scholar 

  24. Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).

    Article  CAS  Google Scholar 

  25. Wright, J. et al. Sparse representation for computer vision and pattern recognition. Proc. IEEE 98, 1031–1044 (2010).

    Article  Google Scholar 

  26. Lee, H., Battle, A., Raina, R. & Ng, A. Y. in Proceedings of the 19th International Conference on Neural Information Processing Systems 801–808 (MIT Press, 2006).

  27. Olshausen, B. A. & Field, D. J. Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Res. 37, 3311–3325 (1997).

    Article  CAS  Google Scholar 

  28. Lee, D. D. & Seung, H. S. Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999).

    Article  CAS  Google Scholar 

  29. Chang, T. et al. Synaptic behaviors and modeling of a metal oxide memristive device. Appl. Phys. A 102, 857–863 (2011).

    Article  CAS  Google Scholar 

  30. Rozell, C. J., Johnson, D. H., Baraniuk, R. G. & Olshausen, B. A. Sparse coding via thresholding and local competition in neural circuits. Neural Comput. 20, 2526–2563 (2008).

    Article  Google Scholar 

  31. Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. 160, 106–154 (1962).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The authors thank G. Kenyon, P. Knag, T. Chen, Z. Zhang, Y. Jeong and M. Zidan for discussions and help. This work was support by the Defense Advanced Research Projects Agency (DARPA) through award no. HR0011-13-2-0015, by the Air Force Office of Scientific Research (AFOSR) through MURI grant FA9550-12-1-0038 and by the National Science Foundation (NSF) through grant CCF-1617315.

Author information

Authors and Affiliations

Authors

Contributions

P.M.S. and W.D.L. conceived and directed the project. P.M.S., F.C., W.M., Z.Z. and W.D.L analysed the experimental data. P.M.S. and F.C. constructed the circuitry and performed the network measurements. C.D. and W.M. prepared the memristor arrays. P.M.S., F.C. and W.D.L. constructed the research frame. All authors discussed the results and implications and commented on the manuscript at all stages.

Corresponding author

Correspondence to Wei D. Lu.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary information

Supplementary information (PDF 2689 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sheridan, P., Cai, F., Du, C. et al. Sparse coding with memristor networks. Nature Nanotech 12, 784–789 (2017). https://doi.org/10.1038/nnano.2017.83

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nnano.2017.83

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing