Abstract
Hyperdimensional computing is an emerging computational framework that takes inspiration from attributes of neuronal circuits including hyperdimensionality, fully distributed holographic representation and (pseudo)randomness. When employed for machine learning tasks, such as learning and classification, the framework involves manipulation and comparison of large patterns within memory. A key attribute of hyperdimensional computing is its robustness to the imperfections associated with the computational substrates on which it is implemented. It is therefore particularly amenable to emerging non-von Neumann approaches such as in-memory computing, where the physical attributes of nanoscale memristive devices are exploited to perform computation. Here, we report a complete in-memory hyperdimensional computing system in which all operations are implemented on two memristive crossbar engines together with peripheral digital complementary metal–oxide–semiconductor (CMOS) circuits. Our approach can achieve a near-optimum trade-off between design complexity and classification accuracy based on three prototypical hyperdimensional computing-related learning tasks: language classification, news classification and hand gesture recognition from electromyography signals. Experiments using 760,000 phase-change memory devices performing analog in-memory computing achieve comparable accuracies to software implementations.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
The data that support the plots within this paper and other findings of this study are available from the corresponding author upon reasonable request.
References
Kanerva, P. Sparse Distributed Memory (MIT Press, 1988).
Kanerva, P. Hyperdimensional computing: an introduction to computing in distributed representation with high-dimensional random vectors. Cogn. Comput. 1, 139–159 (2009).
Kanerva, P., Kristoferson, J. & Holst, A. Random indexing of text samples for latent semantic analysis. In Proceedings of the Annual Meeting of the Cognitive Science Society Vol. 22 (Cognitive Science Society, 2000).
Rahimi, A., Kanerva, P., Benini, L. & Rabaey, J. M. Efficient biosignal processing using hyperdimensional computing: network templates for combined learning and classification of ExG signals. Proc. IEEE 107, 123–143 (2019).
Burrello, A., Cavigelli, L., Schindler, K., Benini, L. & Rahimi, A. Laelaps: an energy-efficient seizure detection algorithm from long-term human iEEG recordings without false alarms. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE) 752–757 (IEEE, 2019).
Räsänen, O. J. & Saarinen, J. P. Sequence prediction with sparse distributed hyperdimensional coding applied to the analysis of mobile phone use patterns. IEEE Trans. Neural Netw. Learn. Syst. 27, 1878–1889 (2015).
Kleyko, D. & Osipov, E. Brain-like classifier of temporal patterns. In Proceedings of the International Conference on Computer and Information Sciences (ICCOINS) 1–6 (IEEE, 2014).
Kleyko, D., Osipov, E., Papakonstantinou, N. & Vyatkin, V. Hyperdimensional computing in industrial systems: the use-case of distributed fault isolation in a power plant. IEEE Access 6, 30766–30777 (2018).
Chang, E., Rahimi, A., Benini, L. & Wu, A. A. Hyperdimensional computing-based multimodality emotion recognition with physiological signals. In Proceedings of the IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS) 137–141 (IEEE, 2019).
Mitrokhin, A., Sutor, P., Fermüller, C. & Aloimonos, Y. Learning sensorimotor control with neuromorphic sensors: toward hyperdimensional active perception. Sci. Robot. 4, eaaw6736 (2019).
Montagna, F., Rahimi, A., Benatti, S., Rossi, D. & Benini, L. PULP-HD: accelerating brain-inspired high-dimensional computing on a parallel ultra-low power platform. In Proceedings of the 55th Annual Design Automation Conference DAC 2018, 111:1–111:6 (ACM, 2018).
Emruli, B., Gayler, R. W. & Sandin, F. Analogical mapping and inference with binary spatter codes and sparse distributed memory. In Proceedings of the International Joint Conference on Neural Networks (IJCNN) 1–8 (IEEE, 2013).
Kleyko, D., Osipov, E., Gayler, R. W., Khan, A. I. & Dyer, A. G. Imitation of honey bees’ concept learning processes using vector symbolic architectures. Biol. Inspired Cogn. Architectures 14, 57–72 (2015).
Slipchenko, S. V. & Rachkovskij, D. A. Analogical mapping using similarity of binary distributed representations. Inf. Theories Appl. 16, 269–290 (2009).
Bandaragoda, T. et al. Trajectory clustering of road traffic in urban environments using incremental machine learning in combination with hyperdimensional computing. In Proceedings of the IEEE Intelligent Transportation Systems Conference (ITSC) 1664–1670 (IEEE, 2019).
Osipov, E., Kleyko, D. & Legalov, A. Associative synthesis of finite state automata model of a controlled object with hyperdimensional computing. In Proceedings of the Annual Conference of the IEEE Industrial Electronics Society 3276–3281 (IEEE, 2017).
Kleyko, D., Frady, E. P. & Osipov, E. Integer echo state networks: hyperdimensional reservoir computing. Preprint at https://arxiv.org/pdf/1706.00280.pdf (2017).
Rahimi, A. et al. High-dimensional computing as a nanoscalable paradigm. IEEE Trans. Circuits Syst. I Regular Papers 64, 2508–2521 (2017).
Yang, J. J., Strukov, D. B. & Stewart, D. R. Memristive devices for computing. Nat. Nanotechnol. 8, 13–24 (2013).
Sebastian, A. et al. Temporal correlation detection using computational phase-change memory. Nat. Commun. 8, 1115 (2017).
Zidan, M. A., Strachan, J. P. & Lu, W. D. The future of electronics based on memristive systems. Nat. Electron. 1, 22–29 (2018).
Ielmini, D. & Wong, H.-S. P. In-memory computing with resistive switching devices. Nat. Electron. 1, 333–343 (2018).
Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R. & Eleftheriou, E. Memory devices and applications for in-memory computing. Nat. Nanotechnol. https://doi.org/10.1038/s41565-020-0655-z (2020).
Li, H. et al. Hyperdimensional computing with 3D VRRAM in-memory kernels: device-architecture co-design for energy-efficient, error-resilient language recognition. In Proceedings of the IEEE International Electron Devices Meeting (IEDM) 16.1.1–16.1.4 (IEEE, 2016).
Li, H., Wu, T. F., Mitra, S. & Wong, H. S. P. Device-architecture co-design for hyperdimensional computing with 3D vertical resistive switching random access memory (3D VRRAM). In Proceedings of the International Symposium on VLSI Technology, Systems and Application (VLSI-TSA) 1–2 (IEEE, 2017).
Wu, T. F. et al. Brain-inspired computing exploiting carbon nanotube FETs and resistive RAM: hyperdimensional computing case study. In Proceedings of the International Solid State Circuits Conference (ISSCC) 492–494 (IEEE, 2018).
Kanerva, P. Binary spatter-coding of ordered k-tuples. In Proceedings of the International Conference on Artificial Neural Networks (ICANN), Vol. 1112, 869–873 (Lecture Notes in Computer Science, Springer, 1996).
Joshi, A., Halseth, J. T. & Kanerva, P. Language geometry using random indexing. In Proceedings of the International Symposium on Quantum Interaction 265–274 (Springer, 2016).
Chua, L. Resistance switching memories are memristors. Appl. Phys. A 102, 765–783 (2011).
Wong, H.-S. P. & Salahuddin, S. Memory leads the way to better computing. Nat. Nanotechnol. 10, 191–194 (2015).
Borghetti, J. et al. ‘Memristive’ switches enable ‘stateful’ logic operations via material implication. Nature 464, 873–876 (2010).
Kvatinsky, S. et al. Magic—memristor-aided logic. IEEE Trans. Circuits Syst II Express Briefs 61, 895–899 (2014).
Shen, W. et al. Stateful logic operations in one-transistor-one-resistor resistive random access memory array. Electron Device Lett. 40, 1538–1541 (2019).
Wong, H.-S. P. et al. Phase change memory. Proc. IEEE 98, 2201–2227 (2010).
Burr, G. W. et al. Recent progress in phase-change memory technology. IEEE J. Emerging Selected Topics Circuits Syst. 6, 146–162 (2016).
Kuzum, D., Jeyasingh, R. G., Lee, B. & Wong, H.-S. P. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett. 12, 2179–2186 (2011).
Tuma, T., Pantazi, A., Le Gallo, M., Sebastian, A. & Eleftheriou, E. Stochastic phase-change neurons. Nat. Nanotechnol. 11, 693–699 (2016).
Boybat, I. et al. Neuromorphic computing with multi-memristive synapses. Nat. Commun. 9, 2514 (2018).
Sebastian, A. et al. Tutorial: brain-inspired computing using phase-change memory devices. J. Appl. Phys. 124, 111101 (2018).
Joshi, V. et al. Accurate deep neural network inference using computational phase-change memory. Nat. Commun. https://doi.org/10.1038/s41467-020-16108-9 (2020).
Hosseini, P., Sebastian, A., Papandreou, N., Wright, C. D. & Bhaskaran, H. Accumulation-based computing using phase-change memories with FET access devices. Electron Device Lett. 36, 975–977 (2015).
Le Gallo, M. et al. Mixed-precision in-memory computing. Nat. Electron. 1, 246–253 (2018).
Xiong, F., Liao, A. D., Estrada, D. & Pop, E. Low-power switching of phase-change materials with carbon nanotube electrodes. Science 332, 568–570 (2011).
Waser, R. & Aono, M. in Nanoscience and Technology: a Collection of Reviews from Nature Journals 158–165 (World Scientific, 2010).
Kent, A. D. & Worledge, D. C. A new spin on magnetic memories. Nat. Nanotechnol. 10, 187–191 (2015).
Close, G. et al. Device, circuit and system-level analysis of noise in multi-bit phase-change memory. In Proceedings of the International Electron Devices Meeting (IEDM) 29.5.1–29.5.4 (IEEE, 2010).
Breitwisch, M. et al. Novel lithography-independent pore phase change memory. In Proceedings of the Symposium on VLSI Technology 100–101 (IEEE, 2007).
Rahimi, A., Kanerva, P. & Rabaey, J. M. A robust and energy-efficient classifier using brain-inspired hyperdimensional computing. In Proceedings of the 2016 International Symposium on Low Power Electronics and Design ISLPED 2016, 64–69 (ACM, 2016).
Quasthoff, U., Richter, M. & Biemann, C. Corpus portal for search in monolingual corpora. In Proceedings of the International Conference on Language Resources and Evaluation (LREC) 1799–1802 (ELRA, 2006).
Koehn, P. Europarl: a parallel corpus for statistical machine translation. In Proceedings of the MT Summit Vol. 5, 79–86 (AAMT, 2005).
Mimaroglu, D. S. Some Text Datasets (Univ. Massachusetts, accessed 9 March 2018); https://www.cs.umb.edu/smimarog/textmining/datasets/
Rahimi, A., Benatti, S., Kanerva, P., Benini, L. & Rabaey, J. M. Hyperdimensional biosignal processing: a case study for EMG-based hand gesture recognition. In Proceedings of the 2016 IEEE International Conference on Rebooting Computing (ICRC) 1–8 (IEEE, 2016).
Chandoke, N., Chitkara, N. & Grover, A. Comparative analysis of sense amplifiers for SRAM in 65 nm CMOS technology. In Proceedings of the International Conference on Electrical, Computer and Communication Technologies (ICECCT), 1–7 (IEEE, 2015).
Acknowledgements
This work was supported in part by the European Research Council through the European Union’s Horizon 2020 Research and Innovation Programme under grant no. 682675 and in part by the European Union’s Horizon 2020 Research and Innovation Programme through the project MNEMOSENE under grant no. 780215.
Author information
Authors and Affiliations
Contributions
All authors collectively conceived the idea of in-memory hyperdimensional computing. G.K. performed the experiments and analysed the results under the supervision of M.L.G., A.R. and A.S. G.K., M.L.G., A.R. and A.S. wrote the manuscript with input from all authors.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Supplementary information
Supplementary Information
Supplementary Notes 1–6.
Supplementary Video 1
Experimental demonstration of language recognition with in-memory hyperdimensional computing implemented in the phase-change memory hardware platform.
Rights and permissions
About this article
Cite this article
Karunaratne, G., Le Gallo, M., Cherubini, G. et al. In-memory hyperdimensional computing. Nat Electron 3, 327–337 (2020). https://doi.org/10.1038/s41928-020-0410-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41928-020-0410-3
This article is cited by
-
Memristor-based hardware accelerators for artificial intelligence
Nature Reviews Electrical Engineering (2024)
-
Echo state graph neural networks with analogue random resistive memory arrays
Nature Machine Intelligence (2023)
-
A neuro-vector-symbolic architecture for solving Raven’s progressive matrices
Nature Machine Intelligence (2023)
-
AI-Empowered Methods for Smart Energy Consumption: A Review of Load Forecasting, Anomaly Detection and Demand Response
International Journal of Precision Engineering and Manufacturing-Green Technology (2023)
-
Experimentally validated memristive memory augmented neural network with efficient hashing and similarity search
Nature Communications (2022)