Artificial intelligence applications have changed the landscape of computer design, driving a search for hardware architecture that can efficiently process large amounts of data. Three-dimensional heterogeneous integration with advanced packaging technologies could be used to improve data bandwidth among sensors, memory and processors. However, such systems are limited by a lack of hardware reconfigurability and the use of conventional von Neumann architectures. Here we report stackable hetero-integrated chips that use optoelectronic device arrays for chip-to-chip communication and neuromorphic cores based on memristor crossbar arrays for highly parallel data processing. With this approach, we create a system with stackable and replaceable chips that can directly classify information from a light-based image source. We also modify this system by inserting a preprogrammed neuromorphic denoising layer that improves the classification performance in a noisy environment. Our reconfigurable three-dimensional hetero-integrated technology can be used to vertically stack a diverse range of functional layers and could provide energy-efficient sensor computing systems for edge computing applications.
This is a preview of subscription content
Subscribe to Nature+
Get immediate online access to the entire Nature family of 50+ journals
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
The data that support the findings of this study are available from the corresponding authors upon reasonable request.
Zhang, W. et al. Neuro-inspired computing chips. Nat. Electron. 3, 371–382 (2020).
Lin, P. et al. Three-dimensional memristor circuits as complex neural networks. Nat. Electron. 3, 225–232 (2020).
Chen, W.-H. et al. CMOS-integrated memristive non-volatile computing-in-memory for AI edge processors. Nat. Electron. 2, 420–428 (2019).
Hills, G. et al. Modern microprocessor built from complementary carbon nanotube transistors. Nature 572, 595–602 (2019).
Bishop, M. D. et al. Fabrication of carbon nanotube field-effect transistors in commercial silicon manufacturing facilities. Nat. Electron. 3, 492–501 (2020).
Mukhopadhyay, S. et al. Heterogeneous integration for artificial intelligence: challenges and opportunities. IBM J. Res. Dev. 63, 4:1 (2019).
Kum, H. S. et al. Heterogeneous integration of single-crystalline complex-oxide membranes. Nature 578, 75–81 (2020).
Ohara, Y. et al. Chip-based hetero-integration technology for high-performance 3D stacked image sensor. In 2012 2nd IEEE CPMT Symposium Japan 1–4 (IEEE, 2012).
Amir, M. F., Ko, J. H., Na, T., Kim, D. & Mukhopadhyay, S. 3-D stacked image sensor with deep neural network computation. IEEE Sens. J. 18, 4187–4199 (2018).
Sabry Aly, M. M. et al. The N3XT approach to energy-efficient abundant-data computing. Proc. IEEE 107, 19–48 (2019).
Bhansali, S. et al. 3D heterogeneous sensor system on a chip for defense and security applications. In Proc. SPIE 5417, Unattended/Unmanned Ground, Ocean, and Air Sensor Technologies and Applications VI 5417, 413 (2004).
Zhou, F. & Chai, Y. Near-sensor and in-sensor computing. Nat. Electron. 3, 664–671 (2020).
Ng, K. W. et al. Unconventional growth mechanism for monolithic integration of III-V on silicon. ACS Nano 7, 100–107 (2013).
Koma, A. Van der Waals epitaxy for highly lattice-mismatched systems. J. Cryst. Growth 201, 236–241 (1999).
Liau, Z. L. & Mull, D. E. Wafer fusion: a novel technique for optoelectronic device fabrication and monolithic integration. Appl. Phys. Lett. 56, 737–739 (1990).
Benwadih, M., Coppard, R., Bonrad, K., Klyszcz, A. and Vuillaume, D. High mobility flexible amorphous IGZO thin-film transistors with a low thermal budget ultra-violet pulsed light process. ACS Appl. Mater. Interfaces 8, 34513–34519 (2016).
Vinet, M. et al. 3D monolithic integration: technological challenges and electrical results. Microelectron. Eng. 88, 331–335 (2011).
Bao, S. et al. A review of silicon-based wafer bonding processes, an approach to realize the monolithic integration of Si-CMOS and III–V-on-Si wafers. J. Semicond. 42, 023106 (2021).
Lee, S. M. et al. High performance ultrathin GaAs solar cells enabled with heterogeneously integrated dielectric periodic nanostructures. ACS Nano 9, 10356–10365 (2015).
Shulaker, M. M. et al. Carbon nanotube computer. Nature 501, 526–530 (2013).
Le Gallo, M. et al. Mixed-precision in-memory computing. Nat. Electron. 1, 246–253 (2018).
Zhou, F. et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat. Nanotechnol. 14, 776–782 (2019).
Mennel, L. et al. Ultrafast machine vision with 2D material neural network image sensors. Nature 579, 62–66 (2020).
Wang, S. et al. Networking retinomorphic sensor with memristive crossbar for brain-inspired visual perception. Natl Sci. Rev. 8, nwaa172 (2021).
Wang, C. et al. Scalable massively parallel computing using continuous-time data representation in nanoscale crossbar array. Nat. Nanotechnol. 16, 1079–1085 (2021).
Shulaker, M. M. et al. Three-dimensional integration of nanotechnologies for computing and data storage on a single chip. Nature 547, 74–78 (2017).
Choi, M. H., Koh, H. J., Yoon, E. S., Shin, K. C. and Song, K. C. Self-aligning silicon groove technology platform for the low cost optical module. In Proc. 49th Electronic Components and Technology Conference (Cat. No. 99CH36299) 1140–1144 (IEEE, 1999).
Barwicz, T. et al. Integrated metamaterial interfaces for self-aligned fiber-to-chip coupling in volume manufacturing. IEEE J. Sel. Topics Quantum Electron. 25, 1–13 (2018).
Yeon, H. et al. Alloying conducting channels for reliable neuromorphic computing. Nat. Nanotechnol. 15, 574–579 (2020).
Ferrari, G., Gozzini, F., Molari, A. & Sampietro, M. Transimpedance amplifier for high sensitivity current measurements on nanodevices. IEEE J. Solid-State Circuits 44, 1609–1616 (2009).
Gurun, G., Hasler, P. & Degertekin, F. L. Front-end receiver electronics for high-frequency monolithic CMUT-on-CMOS imaging arrays. IEEE Trans. Ultrason., Ferroelectr., Freq. Control 58, 1658–1668 (2011).
Rosenstein, J. K., Wanunu, M., Merchant, C. A., Drndic, M. & Shepard, K. L. Integrated nanopore sensing platform with sub-microsecond temporal resolution. Nat. Methods 9, 487–492 (2012).
Dodge, S. and Karam, L. A study and comparison of human and deep learning recognition performance under visual distortions. In 2017 26th International Conference on Computer Communication and Networks (ICCCN) 1–7 (IEEE, 2017).
Brooks, T., Mildenhall, B., Xue, T., Chen, J., Sharlet, D. and Barron, J. T. Unprocessing images for learned raw denoising. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 11036–11045 (IEEE, 2019).
Charte, D., Charte, F., García, S., del Jesus, M. J. & Herrera, F. A practical tutorial on autoencoders for nonlinear feature fusion: taxonomy, models, software and guidelines. Inf. Fusion 44, 78–96 (2018).
Xu, L. et al. Extracting and composing robust features with denoising autoencoders. In Proc. 25th International Conference on Machine Learning 1096–1103 (IEEE, 2008).
Li, C. et al. Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1, 52–59 (2018).
Wang, C.-Y. et al. Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor. Sci. Adv. 6, eaba6173 (2020).
Pan, C. et al. Reconfigurable logic and neuromorphic circuits based on electrically tunable two-dimensional homojunctions. Nat. Electron. 3, 383–390 (2020).
Zhou, T. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photon. 15, 367–373 (2021).
J.-H.K. acknowledges financial support from the Ministry of Trade, Industry and Energy (MOTIE), South Korea, under the Fostering Global Talents for Innovative Growth Program (P0008749) by the Korea Institute for Advancement of Technology (KIAT). The team at MIT acknowledge financial support from the Korea Institute of Science and Technology (KIST) through 2E31550 and the Samsung Global Research Outreach (GRO) Program.
The authors declare no competing interests.
Peer review information
Nature Electronics thanks Rui Yang and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Choi, C., Kim, H., Kang, JH. et al. Reconfigurable heterogeneous integration using stackable chips with embedded artificial intelligence. Nat Electron (2022). https://doi.org/10.1038/s41928-022-00778-y
Nature Electronics (2022)