Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Sparse coding with memristor networks

Abstract

Sparse representation of information provides a powerful means to perform feature extraction on high-dimensional data and is of broad interest for applications in signal processing, computer vision, object recognition and neurobiology. Sparse coding is also believed to be a key mechanism by which biological neural systems can efficiently process a large amount of complex sensory data while consuming very little power. Here, we report the experimental implementation of sparse coding algorithms in a bio-inspired approach using a 32 × 32 crossbar array of analog memristors. This network enables efficient implementation of pattern matching and lateral neuron inhibition and allows input data to be sparsely encoded using neuron activities and stored dictionary elements. Different dictionary sets can be trained and stored in the same system, depending on the nature of the input signals. Using the sparse coding algorithm, we also perform natural image processing based on a learned dictionary.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Memristor crossbar array-based computing hardware system.
Figure 2: Experimental demonstration of sparse coding using memristor network.
Figure 3: Sparse coding of different inputs using a more overcomplete dictionary.
Figure 4: Natural image processing using the memristor crossbar.

References

  1. 1

    Chua, L. O. Memristor—the missing circuit element. IEEE Trans. Circuit Theory 18, 507–519 (1971).

    Article  Google Scholar 

  2. 2

    Strukov, D. B., Snider, G. S., Stewart, D. R. & Williams, R. S. The missing memristor found. Nature 453, 80–83 (2008).

    CAS  Article  Google Scholar 

  3. 3

    Waser, R. & Aono, M. Nanoionics-based resistive switching memories. Nat. Mater. 6, 833–840 (2007).

    CAS  Article  Google Scholar 

  4. 4

    Yang, Y., Chang, T. & Lu, W. in Memristors and Memristive Systems 195–221 (Springer, 2014).

  5. 5

    Kim, K.-H. et al. A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. Nano Lett. 12, 389–395 (2012).

    CAS  Article  Google Scholar 

  6. 6

    Xia, Q. et al. Memristor–CMOS hybrid integrated circuits for reconfigurable logic. Nano Lett. 9, 3640–3645 (2009).

    CAS  Article  Google Scholar 

  7. 7

    Pershin, Y. V. & Di Ventra, M. Practical approach to programmable analog circuits with memristors. IEEE Trans. Circuits Syst. I Regul. Pap. 57, 1857–1864 (2010).

    Article  Google Scholar 

  8. 8

    Jo, S. H. et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297–1301 (2010).

    CAS  Article  Google Scholar 

  9. 9

    Pershin, Y. V. & Di Ventra, M. Experimental demonstration of associative memory with memristive neural networks. Neural Networks 23, 881–886 (2010).

    Article  Google Scholar 

  10. 10

    Du, C., Ma, W., Chang, T., Sheridan, P. & Lu, W. D. Biorealistic implementation of synaptic functions with oxide memristors through internal ionic dynamics. Adv. Funct. Mater. 25, 4290–4299 (2015).

    CAS  Article  Google Scholar 

  11. 11

    Kuzum, D., Jeyasingh, R. G. D., Lee, B. & Wong, H.-S. P. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett. 12, 2179–2186 (2012).

    CAS  Article  Google Scholar 

  12. 12

    Ohno, T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat. Mater. 10, 591–595 (2011).

    CAS  Article  Google Scholar 

  13. 13

    Yang, J. J., Strukov, D. B. & Stewart, D. R. Memristive devices for computing. Nat. Nanotech. 8, 13–24 (2013).

    CAS  Article  Google Scholar 

  14. 14

    Sheridan, P. M., Du, C. & Lu, W. D. Feature extraction using memristor networks. IEEE Trans. Neural Networks Learn. Syst. 27, 2327–2336 (2016).

    Article  Google Scholar 

  15. 15

    Legenstein, R. Computer science: nanoscale connections for brain-like circuits. Nature 521, 37–38 (2015).

    CAS  Article  Google Scholar 

  16. 16

    Alibart, F., Zamanidoost, E. & Strukov, D. B. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun. 4, 2072 (2013).

    Article  Google Scholar 

  17. 17

    Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal–oxide memristors. Nature 521, 61–64 (2015).

    CAS  Article  Google Scholar 

  18. 18

    Burr, G. W. et al. in 2014 IEEE International Electron Devices Meeting 29.5.1–29.5.4 (IEEE, 2014).

  19. 19

    Guo, X. et al. Modeling and experimental demonstration of a Hopfield network analog-to-digital converter with hybrid CMOS/memristor circuits. Front. Neurosci. 9, 488 (2015).

    CAS  Article  Google Scholar 

  20. 20

    Agarwal, S. et al. Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding. Front. Neurosci. 9, 484 (2016).

    Article  Google Scholar 

  21. 21

    Kadetotad, D. et al. in Proceedings of the Biomedical Circuits and Systems Conference (BioCAS) 536–539 (IEEE, 2014).

  22. 22

    Földiák, P. & Young, M. P. Sparse coding in the primate cortex. Handb. Brain Theory Neural Netw. 1, 1064–1068 (1995).

    Google Scholar 

  23. 23

    Vinje, W. E. Sparse coding and decorrelation in primary visual cortex during natural vision. Science. 287, 1273–1276 (2000).

    CAS  Article  Google Scholar 

  24. 24

    Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).

    CAS  Article  Google Scholar 

  25. 25

    Wright, J. et al. Sparse representation for computer vision and pattern recognition. Proc. IEEE 98, 1031–1044 (2010).

    Article  Google Scholar 

  26. 26

    Lee, H., Battle, A., Raina, R. & Ng, A. Y. in Proceedings of the 19th International Conference on Neural Information Processing Systems 801–808 (MIT Press, 2006).

  27. 27

    Olshausen, B. A. & Field, D. J. Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Res. 37, 3311–3325 (1997).

    CAS  Article  Google Scholar 

  28. 28

    Lee, D. D. & Seung, H. S. Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999).

    CAS  Article  Google Scholar 

  29. 29

    Chang, T. et al. Synaptic behaviors and modeling of a metal oxide memristive device. Appl. Phys. A 102, 857–863 (2011).

    CAS  Article  Google Scholar 

  30. 30

    Rozell, C. J., Johnson, D. H., Baraniuk, R. G. & Olshausen, B. A. Sparse coding via thresholding and local competition in neural circuits. Neural Comput. 20, 2526–2563 (2008).

    Article  Google Scholar 

  31. 31

    Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. 160, 106–154 (1962).

    CAS  Article  Google Scholar 

Download references

Acknowledgements

The authors thank G. Kenyon, P. Knag, T. Chen, Z. Zhang, Y. Jeong and M. Zidan for discussions and help. This work was support by the Defense Advanced Research Projects Agency (DARPA) through award no. HR0011-13-2-0015, by the Air Force Office of Scientific Research (AFOSR) through MURI grant FA9550-12-1-0038 and by the National Science Foundation (NSF) through grant CCF-1617315.

Author information

Affiliations

Authors

Contributions

P.M.S. and W.D.L. conceived and directed the project. P.M.S., F.C., W.M., Z.Z. and W.D.L analysed the experimental data. P.M.S. and F.C. constructed the circuitry and performed the network measurements. C.D. and W.M. prepared the memristor arrays. P.M.S., F.C. and W.D.L. constructed the research frame. All authors discussed the results and implications and commented on the manuscript at all stages.

Corresponding author

Correspondence to Wei D. Lu.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary information

Supplementary information (PDF 2689 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sheridan, P., Cai, F., Du, C. et al. Sparse coding with memristor networks. Nature Nanotech 12, 784–789 (2017). https://doi.org/10.1038/nnano.2017.83

Download citation

Further reading

Search

Quick links

Find nanotechnology articles, nanomaterial data and patents all in one place. Visit Nano by Nature Research