Three-dimensional memristor circuits as complex neural networks

Abstract

Constructing a computing circuit in three dimensions (3D) is a necessary step to enable the massive connections and efficient communications required in complex neural networks. 3D circuits based on conventional complementary metal–oxide–semiconductor transistors are, however, difficult to build because of challenges involved in growing or stacking multilayer single-crystalline silicon channels. Here we report a 3D circuit composed of eight layers of monolithically integrated memristive devices. The vertically aligned input and output electrodes in our 3D structure make it possible to directly map and implement complex neural networks. As a proof-of-concept demonstration, we programmed parallelly operated kernels into the 3D array, implemented a convolutional neural network and achieved software-comparable accuracy in recognizing handwritten digits from the Modified National Institute of Standard and Technology database. We also demonstrated the edge detection of moving objects in videos by applying groups of Prewitt filters in the 3D array to process pixels in parallel.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: The purposely designed 3D array provides functional implementation of pixel-wise parallel convolutional kernel operations in a CNN.
Fig. 2: 3D monolithic integrated memristor circuits.
Fig. 3: Principle of computing in our 3D array.
Fig. 4: Parallel convolutional kernel operations using the 3D circuits.
Fig. 5: Parallel video processing using the 3D circuits.

Data availability

The data that support the plots within this article and other findings of this study are available from the corresponding author upon reasonable request.

Code availability

The code that support the plots within this article and other findings of this study is available at https://github.com/plin83/NE_3D_CNN. The code that supports the communication between the custom-built measurement system and the integrated chip is available from the corresponding author upon reasonable request.

References

  1. 1.

    Lecun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  Google Scholar 

  2. 2.

    Pei, J. et al. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature 572, 106–111 (2019).

    Article  Google Scholar 

  3. 3.

    Jouppi, N. P. et al. In-datacenter performance analysis of a tensor processing unit. In Proc. 44th Annual International Symposium on Computer Architecture https://doi.org/10.1145/3079856.3080246 (ACM, 2017).

  4. 4.

    Sangwan, V. K. et al. Multi-terminal memtransistors from polycrystalline monolayer molybdenum disulfide. Nature 554, 500–504 (2018).

    Article  Google Scholar 

  5. 5.

    Van De Burgt, Y. et al. A non-volatile organic electrochemical device as a low-voltage artificial synapse for neuromorphic computing. Nat. Mater. 16, 414–418 (2017).

    Article  Google Scholar 

  6. 6.

    Strukov, D. B., Snider, G. S., Stewart, D. R. & Williams, R. S. The missing memristor found. Nature 453, 80–83 (2008).

    Article  Google Scholar 

  7. 7.

    Jo, S. H. et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297–1301 (2010).

    Article  Google Scholar 

  8. 8.

    Choi, S. et al. SiGe epitaxial memory for neuromorphic computing with reproducible high performance based on engineered dislocations. Nat. Mater. 17, 335–340 (2018).

    Article  Google Scholar 

  9. 9.

    Li, C. et al. Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1, 52–59 (2018).

    Article  Google Scholar 

  10. 10.

    Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 61–64 (2015).

    Article  Google Scholar 

  11. 11.

    Indiveri, G., Linares-Barranco, B., Legenstein, R., Deligeorgis, G. & Prodromakis, T. Integration of nanoscale memristor synapses in neuromorphic computing architectures. Nanotechnology 24, 384010 (2013).

    Article  Google Scholar 

  12. 12.

    Ambrogio, S. et al. Neuromorphic learning and recognition with one-transistor-one-resistor synapses and bistable metal oxide RRAM. IEEE Trans. Electron. Dev. 63, 1508–1515 (2016).

    Article  Google Scholar 

  13. 13.

    Gokmen, T., Onen, M. & Haensch, W. Training deep convolutional neural networks with resistive cross-point devices. Front. Neurosci. 11, 538 (2017).

    Article  Google Scholar 

  14. 14.

    Agarwal, S. et al. Achieving ideal accuracies in analog neuromorphic computing using periodic carry. In 2017 Symposium on VLSI Technology T174–T175 (IEEE, 2017).

  15. 15.

    Ambrogio, S. et al. Equivalent-accuracy accelerated neural-network training using analogue memory. Nature 558, 60–67 (2018).

    Article  Google Scholar 

  16. 16.

    Yao, P. et al. Face classification using electronic synapses. Nat. Commun. 8, 15199 (2017).

    Article  Google Scholar 

  17. 17.

    Cai, F. et al. A fully integrated reprogrammable memristor–CMOS system for efficient multiply–accumulate operations. Nat. Electron. 2, 290–299 (2019).

    Article  Google Scholar 

  18. 18.

    Glorot, X., Bordes, A. & Bengio, Y. Deep Sparse Rectifier Neural Networks. In Proc. 14th International Conference on Artificial Intelligence and Statistics (eds Gordon, G., Dunson, D. & Dudík, M.) 315–323 (PMLR, 2011).

  19. 19.

    Yuste, R. From the neuron doctrine to neural networks. Nat. Rev. Neurosci. 16, 487–497 (2015).

    Article  Google Scholar 

  20. 20.

    Tanaka, H. et al. Bit cost scalable technology with punch and plug process for ultra high density flash memory. In 2007 IEEE Symposium on VLSI Technology 14–15 (IEEE, 2007).

  21. 21.

    Topol, A. W. et al. Three-dimensional integrated circuits. IBM J. Res. Dev. 50, 491–506 (2006).

    Article  Google Scholar 

  22. 22.

    Shulaker, M. M. et al. Three-dimensional integration of nanotechnologies for computing and data storage on a single chip. Nature 547, 74–78 (2017).

    Article  Google Scholar 

  23. 23.

    Adam, G. C. et al. 3-D memristor crossbars for analog and neuromorphic computing applications. IEEE Trans. Electron. Dev. 64, 312–318 (2017).

    Article  Google Scholar 

  24. 24.

    Luo, Q. et al. 8-Layers 3D vertical RRAM with excellent scalability towards storage class memory applications. In 2017 IEEE International Electron Devices Meeting, IEDM 2.7.1–2.7.4 (IEEE, 2018).

  25. 25.

    Li, H. et al. Four-layer 3D vertical RRAM integrated with FinFET as a versatile computing unit for brain-inspired cognitive information processing. In 2016 IEEE Symposium on VLSI Technology 1–2 (IEEE, 2016).

  26. 26.

    Li, Z., Chen, P. Y., Xu, H. & Yu, S. Design of ternary neural network with 3-D vertical RRAM array. IEEE Trans. Electron. Dev. 64, 2721–2727 (2017).

    Article  Google Scholar 

  27. 27.

    Silver, D. et al. Mastering the game of go with deep neural networks and tree search. Nature 529, 484–489 (2016).

    Article  Google Scholar 

  28. 28.

    Strukov, D. B. & Williams, R. S. Four-dimensional address topology for circuits with stacked multilayer crossbar arrays. Proc. Natl Acad. Sci. USA 106, 20155–20158 (2009).

    Article  Google Scholar 

  29. 29.

    Seok, J. Y. et al. A review of three-dimensional resistive switching cross-bar array memories from the integration and materials property points of view. Adv. Funct. Mater. 24, 5316–5339 (2014).

    Article  Google Scholar 

  30. 30.

    Jiang, H. et al. Sub-10 nm Ta channel responsible for superior performance of a HfO2 memristor. Sci. Rep. 6, 28525 (2016).

    Article  Google Scholar 

  31. 31.

    Chen, W. H. et al. A 65nm 1Mb nonvolatile computing-in-memory ReRAM macro with sub-16ns multiply-and-accumulate for binary DNN AI edge processors. In 2018 IEEE International Solid-State Circuits Conference, ISSCC 494–496 (IEEE, 2018).

  32. 32.

    Gao, L., Chen, P. Y. & Yu, S. Demonstration of convolution kernel operation on resistive cross-point array. IEEE Electron. Dev. Lett. 37, 870–873 (2016).

    Article  Google Scholar 

Download references

Acknowledgements

This work is partially sponsored by the Air Force Research Laboratory under agreement nos FA8750-15-2-0044 and FA8750-18-2-0122, as well as by the Air Force Office of Scientific Research (AFOSR) under grant FA9550-12-1-0038. Q.X. thanks H. Weinstock of AFOSR for the initial support of this project. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the Air Force Research Laboratory or the US Government. This work was performed in part at the Center for Hierarchical Manufacturing (CHM), an NSF-sponsored Nanoscale Science and Engineering Center (NSEC) at the University of Massachusetts, Amherst, and in part at the Center for Nanoscale Systems (CNS), a member of the NSF National Nanotechnology Infrastructure Network (NNIN) at Harvard University (ECS-0335765).

Author information

Affiliations

Authors

Contributions

Q.X., J.J.Y. and P.L. conceived the concept and designed the experiments. P.L. did the fabrication, programming, measurements, data analysis and simulation. H.J. contributed to the device optimizations. Z.W. contributed to the 3D-1T1R measurements. C.L. and P.L. did the focused ion beam SEM and took the SEM images. C.L. contributed to the simulation. H.J., M.R., Y.Z. and N.K.U. helped with the fabrication. Y.L., P.L., W.S., Z.W. and C.L. built the measurement system and firmware. Q.X., J.J.Y. and P.L. wrote the manuscript. M.B. and Q.W. and all the other authors contributed to the results analysis and commented on the manuscript.

Corresponding authors

Correspondence to J. Joshua Yang or Qiangfei Xia.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–13, Tables 1 and 2, and Notes 1–6.

Supplementary Video 1

Demonstration of video processing in the 3D array with parallel-operated Prewitt kernels.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lin, P., Li, C., Wang, Z. et al. Three-dimensional memristor circuits as complex neural networks. Nat Electron 3, 225–232 (2020). https://doi.org/10.1038/s41928-020-0397-9

Download citation

Further reading

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing