Sparse representation of information provides a powerful means to perform feature extraction on high-dimensional data and is of broad interest for applications in signal processing, computer vision, object recognition and neurobiology. Sparse coding is also believed to be a key mechanism by which biological neural systems can efficiently process a large amount of complex sensory data while consuming very little power. Here, we report the experimental implementation of sparse coding algorithms in a bio-inspired approach using a 32 × 32 crossbar array of analog memristors. This network enables efficient implementation of pattern matching and lateral neuron inhibition and allows input data to be sparsely encoded using neuron activities and stored dictionary elements. Different dictionary sets can be trained and stored in the same system, depending on the nature of the input signals. Using the sparse coding algorithm, we also perform natural image processing based on a learned dictionary.
Subscribe to Journal
Get full journal access for 1 year
only $4.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Chua, L. O. Memristor—the missing circuit element. IEEE Trans. Circuit Theory 18, 507–519 (1971).
Strukov, D. B., Snider, G. S., Stewart, D. R. & Williams, R. S. The missing memristor found. Nature 453, 80–83 (2008).
Waser, R. & Aono, M. Nanoionics-based resistive switching memories. Nat. Mater. 6, 833–840 (2007).
Yang, Y., Chang, T. & Lu, W. in Memristors and Memristive Systems 195–221 (Springer, 2014).
Kim, K.-H. et al. A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. Nano Lett. 12, 389–395 (2012).
Xia, Q. et al. Memristor–CMOS hybrid integrated circuits for reconfigurable logic. Nano Lett. 9, 3640–3645 (2009).
Pershin, Y. V. & Di Ventra, M. Practical approach to programmable analog circuits with memristors. IEEE Trans. Circuits Syst. I Regul. Pap. 57, 1857–1864 (2010).
Jo, S. H. et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297–1301 (2010).
Pershin, Y. V. & Di Ventra, M. Experimental demonstration of associative memory with memristive neural networks. Neural Networks 23, 881–886 (2010).
Du, C., Ma, W., Chang, T., Sheridan, P. & Lu, W. D. Biorealistic implementation of synaptic functions with oxide memristors through internal ionic dynamics. Adv. Funct. Mater. 25, 4290–4299 (2015).
Kuzum, D., Jeyasingh, R. G. D., Lee, B. & Wong, H.-S. P. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett. 12, 2179–2186 (2012).
Ohno, T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat. Mater. 10, 591–595 (2011).
Yang, J. J., Strukov, D. B. & Stewart, D. R. Memristive devices for computing. Nat. Nanotech. 8, 13–24 (2013).
Sheridan, P. M., Du, C. & Lu, W. D. Feature extraction using memristor networks. IEEE Trans. Neural Networks Learn. Syst. 27, 2327–2336 (2016).
Legenstein, R. Computer science: nanoscale connections for brain-like circuits. Nature 521, 37–38 (2015).
Alibart, F., Zamanidoost, E. & Strukov, D. B. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun. 4, 2072 (2013).
Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal–oxide memristors. Nature 521, 61–64 (2015).
Burr, G. W. et al. in 2014 IEEE International Electron Devices Meeting 29.5.1–29.5.4 (IEEE, 2014).
Guo, X. et al. Modeling and experimental demonstration of a Hopfield network analog-to-digital converter with hybrid CMOS/memristor circuits. Front. Neurosci. 9, 488 (2015).
Agarwal, S. et al. Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding. Front. Neurosci. 9, 484 (2016).
Kadetotad, D. et al. in Proceedings of the Biomedical Circuits and Systems Conference (BioCAS) 536–539 (IEEE, 2014).
Földiák, P. & Young, M. P. Sparse coding in the primate cortex. Handb. Brain Theory Neural Netw. 1, 1064–1068 (1995).
Vinje, W. E. Sparse coding and decorrelation in primary visual cortex during natural vision. Science. 287, 1273–1276 (2000).
Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).
Wright, J. et al. Sparse representation for computer vision and pattern recognition. Proc. IEEE 98, 1031–1044 (2010).
Lee, H., Battle, A., Raina, R. & Ng, A. Y. in Proceedings of the 19th International Conference on Neural Information Processing Systems 801–808 (MIT Press, 2006).
Olshausen, B. A. & Field, D. J. Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Res. 37, 3311–3325 (1997).
Lee, D. D. & Seung, H. S. Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999).
Chang, T. et al. Synaptic behaviors and modeling of a metal oxide memristive device. Appl. Phys. A 102, 857–863 (2011).
Rozell, C. J., Johnson, D. H., Baraniuk, R. G. & Olshausen, B. A. Sparse coding via thresholding and local competition in neural circuits. Neural Comput. 20, 2526–2563 (2008).
Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. 160, 106–154 (1962).
The authors thank G. Kenyon, P. Knag, T. Chen, Z. Zhang, Y. Jeong and M. Zidan for discussions and help. This work was support by the Defense Advanced Research Projects Agency (DARPA) through award no. HR0011-13-2-0015, by the Air Force Office of Scientific Research (AFOSR) through MURI grant FA9550-12-1-0038 and by the National Science Foundation (NSF) through grant CCF-1617315.
The authors declare no competing financial interests.
About this article
Cite this article
Sheridan, P., Cai, F., Du, C. et al. Sparse coding with memristor networks. Nature Nanotech 12, 784–789 (2017). https://doi.org/10.1038/nnano.2017.83
NPG Asia Materials (2021)
Scalable massively parallel computing using continuous-time data representation in nanoscale crossbar array
Nature Nanotechnology (2021)
A CMOS-integrated compute-in-memory macro based on resistive random-access memory for AI edge devices
Nature Electronics (2021)
Nature Reviews Materials (2020)
Curved neuromorphic image sensor array using a MoS2-organic heterostructure inspired by the human visual recognition system
Nature Communications (2020)