Abstract
Sparse representation of information provides a powerful means to perform feature extraction on high-dimensional data and is of broad interest for applications in signal processing, computer vision, object recognition and neurobiology. Sparse coding is also believed to be a key mechanism by which biological neural systems can efficiently process a large amount of complex sensory data while consuming very little power. Here, we report the experimental implementation of sparse coding algorithms in a bio-inspired approach using a 32 × 32 crossbar array of analog memristors. This network enables efficient implementation of pattern matching and lateral neuron inhibition and allows input data to be sparsely encoded using neuron activities and stored dictionary elements. Different dictionary sets can be trained and stored in the same system, depending on the nature of the input signals. Using the sparse coding algorithm, we also perform natural image processing based on a learned dictionary.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$259.00 per year
only $21.58 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Chua, L. O. Memristor—the missing circuit element. IEEE Trans. Circuit Theory 18, 507–519 (1971).
Strukov, D. B., Snider, G. S., Stewart, D. R. & Williams, R. S. The missing memristor found. Nature 453, 80–83 (2008).
Waser, R. & Aono, M. Nanoionics-based resistive switching memories. Nat. Mater. 6, 833–840 (2007).
Yang, Y., Chang, T. & Lu, W. in Memristors and Memristive Systems 195–221 (Springer, 2014).
Kim, K.-H. et al. A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. Nano Lett. 12, 389–395 (2012).
Xia, Q. et al. Memristor–CMOS hybrid integrated circuits for reconfigurable logic. Nano Lett. 9, 3640–3645 (2009).
Pershin, Y. V. & Di Ventra, M. Practical approach to programmable analog circuits with memristors. IEEE Trans. Circuits Syst. I Regul. Pap. 57, 1857–1864 (2010).
Jo, S. H. et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297–1301 (2010).
Pershin, Y. V. & Di Ventra, M. Experimental demonstration of associative memory with memristive neural networks. Neural Networks 23, 881–886 (2010).
Du, C., Ma, W., Chang, T., Sheridan, P. & Lu, W. D. Biorealistic implementation of synaptic functions with oxide memristors through internal ionic dynamics. Adv. Funct. Mater. 25, 4290–4299 (2015).
Kuzum, D., Jeyasingh, R. G. D., Lee, B. & Wong, H.-S. P. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett. 12, 2179–2186 (2012).
Ohno, T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat. Mater. 10, 591–595 (2011).
Yang, J. J., Strukov, D. B. & Stewart, D. R. Memristive devices for computing. Nat. Nanotech. 8, 13–24 (2013).
Sheridan, P. M., Du, C. & Lu, W. D. Feature extraction using memristor networks. IEEE Trans. Neural Networks Learn. Syst. 27, 2327–2336 (2016).
Legenstein, R. Computer science: nanoscale connections for brain-like circuits. Nature 521, 37–38 (2015).
Alibart, F., Zamanidoost, E. & Strukov, D. B. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun. 4, 2072 (2013).
Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal–oxide memristors. Nature 521, 61–64 (2015).
Burr, G. W. et al. in 2014 IEEE International Electron Devices Meeting 29.5.1–29.5.4 (IEEE, 2014).
Guo, X. et al. Modeling and experimental demonstration of a Hopfield network analog-to-digital converter with hybrid CMOS/memristor circuits. Front. Neurosci. 9, 488 (2015).
Agarwal, S. et al. Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding. Front. Neurosci. 9, 484 (2016).
Kadetotad, D. et al. in Proceedings of the Biomedical Circuits and Systems Conference (BioCAS) 536–539 (IEEE, 2014).
Földiák, P. & Young, M. P. Sparse coding in the primate cortex. Handb. Brain Theory Neural Netw. 1, 1064–1068 (1995).
Vinje, W. E. Sparse coding and decorrelation in primary visual cortex during natural vision. Science. 287, 1273–1276 (2000).
Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).
Wright, J. et al. Sparse representation for computer vision and pattern recognition. Proc. IEEE 98, 1031–1044 (2010).
Lee, H., Battle, A., Raina, R. & Ng, A. Y. in Proceedings of the 19th International Conference on Neural Information Processing Systems 801–808 (MIT Press, 2006).
Olshausen, B. A. & Field, D. J. Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Res. 37, 3311–3325 (1997).
Lee, D. D. & Seung, H. S. Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999).
Chang, T. et al. Synaptic behaviors and modeling of a metal oxide memristive device. Appl. Phys. A 102, 857–863 (2011).
Rozell, C. J., Johnson, D. H., Baraniuk, R. G. & Olshausen, B. A. Sparse coding via thresholding and local competition in neural circuits. Neural Comput. 20, 2526–2563 (2008).
Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. 160, 106–154 (1962).
Acknowledgements
The authors thank G. Kenyon, P. Knag, T. Chen, Z. Zhang, Y. Jeong and M. Zidan for discussions and help. This work was support by the Defense Advanced Research Projects Agency (DARPA) through award no. HR0011-13-2-0015, by the Air Force Office of Scientific Research (AFOSR) through MURI grant FA9550-12-1-0038 and by the National Science Foundation (NSF) through grant CCF-1617315.
Author information
Authors and Affiliations
Contributions
P.M.S. and W.D.L. conceived and directed the project. P.M.S., F.C., W.M., Z.Z. and W.D.L analysed the experimental data. P.M.S. and F.C. constructed the circuitry and performed the network measurements. C.D. and W.M. prepared the memristor arrays. P.M.S., F.C. and W.D.L. constructed the research frame. All authors discussed the results and implications and commented on the manuscript at all stages.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary information
Supplementary information (PDF 2689 kb)
Rights and permissions
About this article
Cite this article
Sheridan, P., Cai, F., Du, C. et al. Sparse coding with memristor networks. Nature Nanotech 12, 784–789 (2017). https://doi.org/10.1038/nnano.2017.83
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nnano.2017.83
This article is cited by
-
Memristor-based hardware accelerators for artificial intelligence
Nature Reviews Electrical Engineering (2024)
-
On non-von Neumann flexible neuromorphic vision sensors
npj Flexible Electronics (2024)
-
Physical neural networks with self-learning capabilities
Science China Physics, Mechanics & Astronomy (2024)
-
Nanograin network memory with reconfigurable percolation paths for synaptic interactions
Light: Science & Applications (2023)
-
Echo state graph neural networks with analogue random resistive memory arrays
Nature Machine Intelligence (2023)