Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Three-dimensional memristor circuits as complex neural networks

Nature Electronics volume 3, pages 225–232 (2020)Cite this article

Subjects

Abstract

Constructing a computing circuit in three dimensions (3D) is a necessary step to enable the massive connections and efficient communications required in complex neural networks. 3D circuits based on conventional complementary metal–oxide–semiconductor transistors are, however, difficult to build because of challenges involved in growing or stacking multilayer single-crystalline silicon channels. Here we report a 3D circuit composed of eight layers of monolithically integrated memristive devices. The vertically aligned input and output electrodes in our 3D structure make it possible to directly map and implement complex neural networks. As a proof-of-concept demonstration, we programmed parallelly operated kernels into the 3D array, implemented a convolutional neural network and achieved software-comparable accuracy in recognizing handwritten digits from the Modified National Institute of Standard and Technology database. We also demonstrated the edge detection of moving objects in videos by applying groups of Prewitt filters in the 3D array to process pixels in parallel.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: The purposely designed 3D array provides functional implementation of pixel-wise parallel convolutional kernel operations in a CNN.
Fig. 2: 3D monolithic integrated memristor circuits.
Fig. 3: Principle of computing in our 3D array.
Fig. 4: Parallel convolutional kernel operations using the 3D circuits.
Fig. 5: Parallel video processing using the 3D circuits.

Data availability

The data that support the plots within this article and other findings of this study are available from the corresponding author upon reasonable request.

Code availability

The code that support the plots within this article and other findings of this study is available at https://github.com/plin83/NE_3D_CNN. The code that supports the communication between the custom-built measurement system and the integrated chip is available from the corresponding author upon reasonable request.

References

  1. Lecun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  Google Scholar 

  2. Pei, J. et al. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature 572, 106–111 (2019).

    Article  Google Scholar 

  3. Jouppi, N. P. et al. In-datacenter performance analysis of a tensor processing unit. In Proc. 44th Annual International Symposium on Computer Architecture https://doi.org/10.1145/3079856.3080246 (ACM, 2017).

  4. Sangwan, V. K. et al. Multi-terminal memtransistors from polycrystalline monolayer molybdenum disulfide. Nature 554, 500–504 (2018).

    Article  Google Scholar 

  5. Van De Burgt, Y. et al. A non-volatile organic electrochemical device as a low-voltage artificial synapse for neuromorphic computing. Nat. Mater. 16, 414–418 (2017).

    Article  Google Scholar 

  6. Strukov, D. B., Snider, G. S., Stewart, D. R. & Williams, R. S. The missing memristor found. Nature 453, 80–83 (2008).

    Article  Google Scholar 

  7. Jo, S. H. et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297–1301 (2010).

    Article  Google Scholar 

  8. Choi, S. et al. SiGe epitaxial memory for neuromorphic computing with reproducible high performance based on engineered dislocations. Nat. Mater. 17, 335–340 (2018).

    Article  Google Scholar 

  9. Li, C. et al. Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1, 52–59 (2018).

    Article  Google Scholar 

  10. Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 61–64 (2015).

    Article  Google Scholar 

  11. Indiveri, G., Linares-Barranco, B., Legenstein, R., Deligeorgis, G. & Prodromakis, T. Integration of nanoscale memristor synapses in neuromorphic computing architectures. Nanotechnology 24, 384010 (2013).

    Article  Google Scholar 

  12. Ambrogio, S. et al. Neuromorphic learning and recognition with one-transistor-one-resistor synapses and bistable metal oxide RRAM. IEEE Trans. Electron. Dev. 63, 1508–1515 (2016).

    Article  Google Scholar 

  13. Gokmen, T., Onen, M. & Haensch, W. Training deep convolutional neural networks with resistive cross-point devices. Front. Neurosci. 11, 538 (2017).

    Article  Google Scholar 

  14. Agarwal, S. et al. Achieving ideal accuracies in analog neuromorphic computing using periodic carry. In 2017 Symposium on VLSI Technology T174–T175 (IEEE, 2017).

  15. Ambrogio, S. et al. Equivalent-accuracy accelerated neural-network training using analogue memory. Nature 558, 60–67 (2018).

    Article  Google Scholar 

  16. Yao, P. et al. Face classification using electronic synapses. Nat. Commun. 8, 15199 (2017).

    Article  Google Scholar 

  17. Cai, F. et al. A fully integrated reprogrammable memristor–CMOS system for efficient multiply–accumulate operations. Nat. Electron. 2, 290–299 (2019).

    Article  Google Scholar 

  18. Glorot, X., Bordes, A. & Bengio, Y. Deep Sparse Rectifier Neural Networks. In Proc. 14th International Conference on Artificial Intelligence and Statistics (eds Gordon, G., Dunson, D. & Dudík, M.) 315–323 (PMLR, 2011).

  19. Yuste, R. From the neuron doctrine to neural networks. Nat. Rev. Neurosci. 16, 487–497 (2015).

    Article  Google Scholar 

  20. Tanaka, H. et al. Bit cost scalable technology with punch and plug process for ultra high density flash memory. In 2007 IEEE Symposium on VLSI Technology 14–15 (IEEE, 2007).

  21. Topol, A. W. et al. Three-dimensional integrated circuits. IBM J. Res. Dev. 50, 491–506 (2006).

    Article  Google Scholar 

  22. Shulaker, M. M. et al. Three-dimensional integration of nanotechnologies for computing and data storage on a single chip. Nature 547, 74–78 (2017).

    Article  Google Scholar 

  23. Adam, G. C. et al. 3-D memristor crossbars for analog and neuromorphic computing applications. IEEE Trans. Electron. Dev. 64, 312–318 (2017).

    Article  Google Scholar 

  24. Luo, Q. et al. 8-Layers 3D vertical RRAM with excellent scalability towards storage class memory applications. In 2017 IEEE International Electron Devices Meeting, IEDM 2.7.1–2.7.4 (IEEE, 2018).

  25. Li, H. et al. Four-layer 3D vertical RRAM integrated with FinFET as a versatile computing unit for brain-inspired cognitive information processing. In 2016 IEEE Symposium on VLSI Technology 1–2 (IEEE, 2016).

  26. Li, Z., Chen, P. Y., Xu, H. & Yu, S. Design of ternary neural network with 3-D vertical RRAM array. IEEE Trans. Electron. Dev. 64, 2721–2727 (2017).

    Article  Google Scholar 

  27. Silver, D. et al. Mastering the game of go with deep neural networks and tree search. Nature 529, 484–489 (2016).

    Article  Google Scholar 

  28. Strukov, D. B. & Williams, R. S. Four-dimensional address topology for circuits with stacked multilayer crossbar arrays. Proc. Natl Acad. Sci. USA 106, 20155–20158 (2009).

    Article  Google Scholar 

  29. Seok, J. Y. et al. A review of three-dimensional resistive switching cross-bar array memories from the integration and materials property points of view. Adv. Funct. Mater. 24, 5316–5339 (2014).

    Article  Google Scholar 

  30. Jiang, H. et al. Sub-10 nm Ta channel responsible for superior performance of a HfO2 memristor. Sci. Rep. 6, 28525 (2016).

    Article  Google Scholar 

  31. Chen, W. H. et al. A 65nm 1Mb nonvolatile computing-in-memory ReRAM macro with sub-16ns multiply-and-accumulate for binary DNN AI edge processors. In 2018 IEEE International Solid-State Circuits Conference, ISSCC 494–496 (IEEE, 2018).

  32. Gao, L., Chen, P. Y. & Yu, S. Demonstration of convolution kernel operation on resistive cross-point array. IEEE Electron. Dev. Lett. 37, 870–873 (2016).

    Article  Google Scholar 

Download references

Acknowledgements

This work is partially sponsored by the Air Force Research Laboratory under agreement nos FA8750-15-2-0044 and FA8750-18-2-0122, as well as by the Air Force Office of Scientific Research (AFOSR) under grant FA9550-12-1-0038. Q.X. thanks H. Weinstock of AFOSR for the initial support of this project. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the Air Force Research Laboratory or the US Government. This work was performed in part at the Center for Hierarchical Manufacturing (CHM), an NSF-sponsored Nanoscale Science and Engineering Center (NSEC) at the University of Massachusetts, Amherst, and in part at the Center for Nanoscale Systems (CNS), a member of the NSF National Nanotechnology Infrastructure Network (NNIN) at Harvard University (ECS-0335765).

Author information

Author notes

  1. Peng Lin

    Present address: Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Massachusetts, Amherst, MA, USA

    Peng Lin, Can Li, Zhongrui Wang, Yunning Li, Hao Jiang, Wenhao Song, Mingyi Rao, Ye Zhuo, Navnidhi K. Upadhyay, J. Joshua Yang & Qiangfei Xia

  2. Air Force Research Laboratory Information Directorate, Rome, New York, USA

    Mark Barnell & Qing Wu

Authors
  1. Peng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Can Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhongrui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunning Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hao Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wenhao Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mingyi Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ye Zhuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Navnidhi K. Upadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Mark Barnell
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Qing Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. J. Joshua Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Qiangfei Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Q.X., J.J.Y. and P.L. conceived the concept and designed the experiments. P.L. did the fabrication, programming, measurements, data analysis and simulation. H.J. contributed to the device optimizations. Z.W. contributed to the 3D-1T1R measurements. C.L. and P.L. did the focused ion beam SEM and took the SEM images. C.L. contributed to the simulation. H.J., M.R., Y.Z. and N.K.U. helped with the fabrication. Y.L., P.L., W.S., Z.W. and C.L. built the measurement system and firmware. Q.X., J.J.Y. and P.L. wrote the manuscript. M.B. and Q.W. and all the other authors contributed to the results analysis and commented on the manuscript.

Corresponding authors

Correspondence to J. Joshua Yang or Qiangfei Xia.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–13, Tables 1 and 2, and Notes 1–6.

Supplementary Video 1

Demonstration of video processing in the 3D array with parallel-operated Prewitt kernels.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, P., Li, C., Wang, Z. et al. Three-dimensional memristor circuits as complex neural networks. Nat Electron 3, 225–232 (2020). https://doi.org/10.1038/s41928-020-0397-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41928-020-0397-9

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing