Article | Published:

Sparse coding with memristor networks

Nature Nanotechnology volume 12, pages 784789 (2017) | Download Citation

Abstract

Sparse representation of information provides a powerful means to perform feature extraction on high-dimensional data and is of broad interest for applications in signal processing, computer vision, object recognition and neurobiology. Sparse coding is also believed to be a key mechanism by which biological neural systems can efficiently process a large amount of complex sensory data while consuming very little power. Here, we report the experimental implementation of sparse coding algorithms in a bio-inspired approach using a 32 × 32 crossbar array of analog memristors. This network enables efficient implementation of pattern matching and lateral neuron inhibition and allows input data to be sparsely encoded using neuron activities and stored dictionary elements. Different dictionary sets can be trained and stored in the same system, depending on the nature of the input signals. Using the sparse coding algorithm, we also perform natural image processing based on a learned dictionary.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    Memristor—the missing circuit element. IEEE Trans. Circuit Theory 18, 507–519 (1971).

  2. 2.

    , , & The missing memristor found. Nature 453, 80–83 (2008).

  3. 3.

    & Nanoionics-based resistive switching memories. Nat. Mater. 6, 833–840 (2007).

  4. 4.

    , & in Memristors and Memristive Systems 195–221 (Springer, 2014).

  5. 5.

    et al. A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. Nano Lett. 12, 389–395 (2012).

  6. 6.

    et al. Memristor–CMOS hybrid integrated circuits for reconfigurable logic. Nano Lett. 9, 3640–3645 (2009).

  7. 7.

    & Practical approach to programmable analog circuits with memristors. IEEE Trans. Circuits Syst. I Regul. Pap. 57, 1857–1864 (2010).

  8. 8.

    et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297–1301 (2010).

  9. 9.

    & Experimental demonstration of associative memory with memristive neural networks. Neural Networks 23, 881–886 (2010).

  10. 10.

    , , , & Biorealistic implementation of synaptic functions with oxide memristors through internal ionic dynamics. Adv. Funct. Mater. 25, 4290–4299 (2015).

  11. 11.

    , , & Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett. 12, 2179–2186 (2012).

  12. 12.

    et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat. Mater. 10, 591–595 (2011).

  13. 13.

    , & Memristive devices for computing. Nat. Nanotech. 8, 13–24 (2013).

  14. 14.

    , & Feature extraction using memristor networks. IEEE Trans. Neural Networks Learn. Syst. 27, 2327–2336 (2016).

  15. 15.

    Computer science: nanoscale connections for brain-like circuits. Nature 521, 37–38 (2015).

  16. 16.

    , & Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun. 4, 2072 (2013).

  17. 17.

    et al. Training and operation of an integrated neuromorphic network based on metal–oxide memristors. Nature 521, 61–64 (2015).

  18. 18.

    et al. in 2014 IEEE International Electron Devices Meeting 29.5.1–29.5.4 (IEEE, 2014).

  19. 19.

    et al. Modeling and experimental demonstration of a Hopfield network analog-to-digital converter with hybrid CMOS/memristor circuits. Front. Neurosci. 9, 488 (2015).

  20. 20.

    et al. Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding. Front. Neurosci. 9, 484 (2016).

  21. 21.

    et al. in Proceedings of the Biomedical Circuits and Systems Conference (BioCAS) 536–539 (IEEE, 2014).

  22. 22.

    & Sparse coding in the primate cortex. Handb. Brain Theory Neural Netw. 1, 1064–1068 (1995).

  23. 23.

    Sparse coding and decorrelation in primary visual cortex during natural vision. Science. 287, 1273–1276 (2000).

  24. 24.

    & Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).

  25. 25.

    et al. Sparse representation for computer vision and pattern recognition. Proc. IEEE 98, 1031–1044 (2010).

  26. 26.

    , , & in Proceedings of the 19th International Conference on Neural Information Processing Systems 801–808 (MIT Press, 2006).

  27. 27.

    & Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Res. 37, 3311–3325 (1997).

  28. 28.

    & Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999).

  29. 29.

    et al. Synaptic behaviors and modeling of a metal oxide memristive device. Appl. Phys. A 102, 857–863 (2011).

  30. 30.

    , , & Sparse coding via thresholding and local competition in neural circuits. Neural Comput. 20, 2526–2563 (2008).

  31. 31.

    & Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. 160, 106–154 (1962).

Download references

Acknowledgements

The authors thank G. Kenyon, P. Knag, T. Chen, Z. Zhang, Y. Jeong and M. Zidan for discussions and help. This work was support by the Defense Advanced Research Projects Agency (DARPA) through award no. HR0011-13-2-0015, by the Air Force Office of Scientific Research (AFOSR) through MURI grant FA9550-12-1-0038 and by the National Science Foundation (NSF) through grant CCF-1617315.

Author information

Author notes

    • Patrick M. Sheridan
    •  & Fuxi Cai

    These authors contributed equally to this work.

Affiliations

  1. Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, Michigan 48109, USA

    • Patrick M. Sheridan
    • , Fuxi Cai
    • , Chao Du
    • , Wen Ma
    • , Zhengya Zhang
    •  & Wei D. Lu

Authors

  1. Search for Patrick M. Sheridan in:

  2. Search for Fuxi Cai in:

  3. Search for Chao Du in:

  4. Search for Wen Ma in:

  5. Search for Zhengya Zhang in:

  6. Search for Wei D. Lu in:

Contributions

P.M.S. and W.D.L. conceived and directed the project. P.M.S., F.C., W.M., Z.Z. and W.D.L analysed the experimental data. P.M.S. and F.C. constructed the circuitry and performed the network measurements. C.D. and W.M. prepared the memristor arrays. P.M.S., F.C. and W.D.L. constructed the research frame. All authors discussed the results and implications and commented on the manuscript at all stages.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Wei D. Lu.

Supplementary information

PDF files

  1. 1.

    Supplementary information

    Supplementary information

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nnano.2017.83

Further reading