Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Reconfigurable heterogeneous integration using stackable chips with embedded artificial intelligence

Abstract

Artificial intelligence applications have changed the landscape of computer design, driving a search for hardware architecture that can efficiently process large amounts of data. Three-dimensional heterogeneous integration with advanced packaging technologies could be used to improve data bandwidth among sensors, memory and processors. However, such systems are limited by a lack of hardware reconfigurability and the use of conventional von Neumann architectures. Here we report stackable hetero-integrated chips that use optoelectronic device arrays for chip-to-chip communication and neuromorphic cores based on memristor crossbar arrays for highly parallel data processing. With this approach, we create a system with stackable and replaceable chips that can directly classify information from a light-based image source. We also modify this system by inserting a preprogrammed neuromorphic denoising layer that improves the classification performance in a noisy environment. Our reconfigurable three-dimensional hetero-integrated technology can be used to vertically stack a diverse range of functional layers and could provide energy-efficient sensor computing systems for edge computing applications.

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

All prices are NET prices.

Fig. 1: Integration technologies of sensor computing systems for edge computing.
Fig. 2: Components of stackable hetero-integrated neuromorphic chips.
Fig. 3: Replaceable and stackable hetero-integrated neuromorphic chips and their robust kernel operations.
Fig. 4: Application of stackable neuromorphic chips: insertion of a denoising functional layer in a noisy environment.

Data availability

The data that support the findings of this study are available from the corresponding authors upon reasonable request.

References

  1. Zhang, W. et al. Neuro-inspired computing chips. Nat. Electron. 3, 371–382 (2020).

    Article  Google Scholar 

  2. Lin, P. et al. Three-dimensional memristor circuits as complex neural networks. Nat. Electron. 3, 225–232 (2020).

    Article  Google Scholar 

  3. Chen, W.-H. et al. CMOS-integrated memristive non-volatile computing-in-memory for AI edge processors. Nat. Electron. 2, 420–428 (2019).

  4. Hills, G. et al. Modern microprocessor built from complementary carbon nanotube transistors. Nature 572, 595–602 (2019).

    Article  Google Scholar 

  5. Bishop, M. D. et al. Fabrication of carbon nanotube field-effect transistors in commercial silicon manufacturing facilities. Nat. Electron. 3, 492–501 (2020).

    Article  Google Scholar 

  6. Mukhopadhyay, S. et al. Heterogeneous integration for artificial intelligence: challenges and opportunities. IBM J. Res. Dev. 63, 4:1 (2019).

  7. Kum, H. S. et al. Heterogeneous integration of single-crystalline complex-oxide membranes. Nature 578, 75–81 (2020).

    Article  Google Scholar 

  8. Ohara, Y. et al. Chip-based hetero-integration technology for high-performance 3D stacked image sensor. In 2012 2nd IEEE CPMT Symposium Japan 1–4 (IEEE, 2012).

  9. Amir, M. F., Ko, J. H., Na, T., Kim, D. & Mukhopadhyay, S. 3-D stacked image sensor with deep neural network computation. IEEE Sens. J. 18, 4187–4199 (2018).

    Article  Google Scholar 

  10. Sabry Aly, M. M. et al. The N3XT approach to energy-efficient abundant-data computing. Proc. IEEE 107, 19–48 (2019).

    Article  Google Scholar 

  11. Bhansali, S. et al. 3D heterogeneous sensor system on a chip for defense and security applications. In Proc. SPIE 5417, Unattended/Unmanned Ground, Ocean, and Air Sensor Technologies and Applications VI 5417, 413 (2004).

    Google Scholar 

  12. Zhou, F. & Chai, Y. Near-sensor and in-sensor computing. Nat. Electron. 3, 664–671 (2020).

    Article  Google Scholar 

  13. Ng, K. W. et al. Unconventional growth mechanism for monolithic integration of III-V on silicon. ACS Nano 7, 100–107 (2013).

    Article  Google Scholar 

  14. Koma, A. Van der Waals epitaxy for highly lattice-mismatched systems. J. Cryst. Growth 201, 236–241 (1999).

    Article  Google Scholar 

  15. Liau, Z. L. & Mull, D. E. Wafer fusion: a novel technique for optoelectronic device fabrication and monolithic integration. Appl. Phys. Lett. 56, 737–739 (1990).

    Article  Google Scholar 

  16. Benwadih, M., Coppard, R., Bonrad, K., Klyszcz, A. and Vuillaume, D. High mobility flexible amorphous IGZO thin-film transistors with a low thermal budget ultra-violet pulsed light process. ACS Appl. Mater. Interfaces 8, 34513–34519 (2016).

  17. Vinet, M. et al. 3D monolithic integration: technological challenges and electrical results. Microelectron. Eng. 88, 331–335 (2011).

    Article  Google Scholar 

  18. Bao, S. et al. A review of silicon-based wafer bonding processes, an approach to realize the monolithic integration of Si-CMOS and III–V-on-Si wafers. J. Semicond. 42, 023106 (2021).

  19. Lee, S. M. et al. High performance ultrathin GaAs solar cells enabled with heterogeneously integrated dielectric periodic nanostructures. ACS Nano 9, 10356–10365 (2015).

    Article  Google Scholar 

  20. Shulaker, M. M. et al. Carbon nanotube computer. Nature 501, 526–530 (2013).

    Article  Google Scholar 

  21. Le Gallo, M. et al. Mixed-precision in-memory computing. Nat. Electron. 1, 246–253 (2018).

    Article  Google Scholar 

  22. Zhou, F. et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat. Nanotechnol. 14, 776–782 (2019).

    Article  Google Scholar 

  23. Mennel, L. et al. Ultrafast machine vision with 2D material neural network image sensors. Nature 579, 62–66 (2020).

    Article  Google Scholar 

  24. Wang, S. et al. Networking retinomorphic sensor with memristive crossbar for brain-inspired visual perception. Natl Sci. Rev. 8, nwaa172 (2021).

    Article  Google Scholar 

  25. Wang, C. et al. Scalable massively parallel computing using continuous-time data representation in nanoscale crossbar array. Nat. Nanotechnol. 16, 1079–1085 (2021).

    Article  Google Scholar 

  26. Shulaker, M. M. et al. Three-dimensional integration of nanotechnologies for computing and data storage on a single chip. Nature 547, 74–78 (2017).

    Article  Google Scholar 

  27. Choi, M. H., Koh, H. J., Yoon, E. S., Shin, K. C. and Song, K. C. Self-aligning silicon groove technology platform for the low cost optical module. In Proc. 49th Electronic Components and Technology Conference (Cat. No. 99CH36299) 1140–1144 (IEEE, 1999).

  28. Barwicz, T. et al. Integrated metamaterial interfaces for self-aligned fiber-to-chip coupling in volume manufacturing. IEEE J. Sel. Topics Quantum Electron. 25, 1–13 (2018).

    Article  Google Scholar 

  29. Yeon, H. et al. Alloying conducting channels for reliable neuromorphic computing. Nat. Nanotechnol. 15, 574–579 (2020).

    Article  Google Scholar 

  30. Ferrari, G., Gozzini, F., Molari, A. & Sampietro, M. Transimpedance amplifier for high sensitivity current measurements on nanodevices. IEEE J. Solid-State Circuits 44, 1609–1616 (2009).

    Article  Google Scholar 

  31. Gurun, G., Hasler, P. & Degertekin, F. L. Front-end receiver electronics for high-frequency monolithic CMUT-on-CMOS imaging arrays. IEEE Trans. Ultrason., Ferroelectr., Freq. Control 58, 1658–1668 (2011).

    Article  Google Scholar 

  32. Rosenstein, J. K., Wanunu, M., Merchant, C. A., Drndic, M. & Shepard, K. L. Integrated nanopore sensing platform with sub-microsecond temporal resolution. Nat. Methods 9, 487–492 (2012).

    Article  Google Scholar 

  33. Dodge, S. and Karam, L. A study and comparison of human and deep learning recognition performance under visual distortions. In 2017 26th International Conference on Computer Communication and Networks (ICCCN) 1–7 (IEEE, 2017).

  34. Brooks, T., Mildenhall, B., Xue, T., Chen, J., Sharlet, D. and Barron, J. T. Unprocessing images for learned raw denoising. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 11036–11045 (IEEE, 2019).

  35. Charte, D., Charte, F., García, S., del Jesus, M. J. & Herrera, F. A practical tutorial on autoencoders for nonlinear feature fusion: taxonomy, models, software and guidelines. Inf. Fusion 44, 78–96 (2018).

    Article  Google Scholar 

  36. Xu, L. et al. Extracting and composing robust features with denoising autoencoders. In Proc. 25th International Conference on Machine Learning 1096–1103 (IEEE, 2008).

  37. Li, C. et al. Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1, 52–59 (2018).

    Article  Google Scholar 

  38. Wang, C.-Y. et al. Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor. Sci. Adv. 6, eaba6173 (2020).

    Article  Google Scholar 

  39. Pan, C. et al. Reconfigurable logic and neuromorphic circuits based on electrically tunable two-dimensional homojunctions. Nat. Electron. 3, 383–390 (2020).

    Article  Google Scholar 

  40. Zhou, T. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photon. 15, 367–373 (2021).

    Article  Google Scholar 

Download references

Acknowledgements

J.-H.K. acknowledges financial support from the Ministry of Trade, Industry and Energy (MOTIE), South Korea, under the Fostering Global Talents for Innovative Growth Program (P0008749) by the Korea Institute for Advancement of Technology (KIAT). The team at MIT acknowledge financial support from the Korea Institute of Science and Technology (KIST) through 2E31550 and the Samsung Global Research Outreach (GRO) Program.

Author information

Authors and Affiliations

Authors

Contributions

C.C. and P.L. conceived this work and designed the experiments. J.K. directed the team. C.C., H.K., P.L., M.-K.S, H.Y. and J.K. prepared the manuscript. C.C., M.-K.S., H.Y., J.M.S., J.L. and D.L. performed the device fabrication. H.K. grew the thin films for optoelectronic devices. C.C., J.-H.K. and P.L. conducted the crossbar array measurement. C.C. designed and built the optical setup. C.C. and H.Y. performed the sensor array measurement. C.C. conducted the neural network simulations and implementations using MATLAB, Python and PyTorch. C.C., S.P. and K.R. discussed the results of the neural networks simulations. All the authors contributed to the discussion and analysis of the results.

Corresponding authors

Correspondence to Huaqiang Wu, Peng Lin or Jeehwan Kim.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Electronics thanks Rui Yang and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–12.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Choi, C., Kim, H., Kang, JH. et al. Reconfigurable heterogeneous integration using stackable chips with embedded artificial intelligence. Nat Electron (2022). https://doi.org/10.1038/s41928-022-00778-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41928-022-00778-y

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing