Artificial intelligence now advances by performing twice as many floating-point multiplications every two months, but the semiconductor industry tiles twice as many multipliers on a chip every two years. Moreover, the returns from tiling these multipliers ever more densely now diminish because signals must travel relatively farther and farther. Although travel can be shortened by stacking tiled multipliers in a three-dimensional chip, such a solution acutely reduces the available surface area for dissipating heat. Here I propose to transcend this three-dimensional thermal constraint by moving away from learning with synapses to learning with dendrites. Synaptic inputs are not weighted precisely but rather ordered meticulously along a short stretch of dendrite, termed dendrocentric learning. With the help of a computational model of a dendrite and a conceptual model of a ferroelectric device that emulates it, I illustrate how dendrocentric learning artificial intelligence—or synthetic intelligence for short—could run not with megawatts in the cloud but rather with watts on a smartphone.
This is a preview of subscription content, access via your institution
Subscribe to Nature+
Get immediate online access to Nature and 55 other Nature journal
Subscribe to Journal
Get full journal access for 1 year
only $3.90 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
The Mathematica notebook used to simulate and analyse the dendrite model is available at https://web.stanford.edu/group/brainsinsilicon/documents/Spiny_Dendrite_Model.nb.
Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020).
GPT-3. A robot wrote this article. Are you scared yet, human? The Guardian (2020); https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3.
Mehonic, A. & Kenyon, A. Brain-inspired computing needs a master plan. Nature 604, 255–260 (2022).
Dally, W., Turakhia, Y. & Han, S. Domain-specific hardware accelerators. Commun. ACM 63, 48–57 (2020).
Jouppi, N. et al. A domain-specific supercomputer for training deep neural networks. Commun. ACM 63, 67–78 (2020).
Rosenblatt, F. The perceptron—a probabilistic model for information-storage and organization in the brain. Psychol. Rev. 65, 386–408 (1958). This paper introduced the synaptocentric conception of the learning brain.
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
Schmidhuber, J. Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015).
Radford, A. et al. Language models are unsupervised multitask learners. OpenAI Blog 1, 9 (2019).
Anthony, L. F. W., Kanding, B. & Selvan, R. Carbontracker: tracking and predicting the carbon footprint of training deep learning models. Preprint at https://arxiv.org/abs/2007.03051 (2020).
Dally, W. J. et al. Hardware-enabled artificial intelligence. In 2018 IEEE Symposium on VLSI Circuits 3–6 (IEEE, 2018).
Goda, A. 3-D NAND technology achievements and future scaling perspectives. IEEE Trans. Electron Devices 67, 1373–1381 (2020).
Pekny, T. et al. A 1-Tb Density 4b/Cell 3D-NAND Flash on 176-Tier Technology with 4-Independent Planes for Read using CMOS-Under-the-Array. In 2022 IEEE International Solid-State Circuits Conference (ISSCC) 1–3 (IEEE, 2022).
Markram, H. et al. Reconstruction and simulation of neocortical microcircuitry. Cell 163, 456–492 (2015).
Park, Y. et al. 3-D stacked synapse array based on charge-trap flash memory for implementation of deep neural networks. IEEE Trans. Electron Devices 66, 420–427 (2019).
Bavandpour, M., Sahay, S., Mahmoodi, M. R. & Strukov, D. B. 3D-aCortex: an ultra-compact energy-efficient neurocomputing platform based on commercial 3D-NAND flash memories. Neuromorph. Comput. Eng. 1, 014001 (2021).
Thorpe, S., Delorme, A. & Van Rullen, R. Spike-based strategies for rapid processing. Neural Netw. 14, 715–725 (2001).
Skaggs, W. E. & McNaughton, B. L. Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science 271, 1870–1873 (1996).
Wehr, M. & Laurent, G. Odour encoding by temporal sequences of firing in oscillating neural assemblies. Nature 384, 162–166 (1996).
Vaz, A. P., Wittig, J. H., Inati, S. K. & Zaghloul, K. A. Replay of cortical spiking sequences during human memory retrieval. Science 367, 1131–1134 (2020). A specific sequence of spikes encodes memory of an episode in humans and recall involves reinstating this temporal order of activity.
Hanin, B. & Rolnick, D. Deep ReLU networks have surprisingly few activation patterns. In 33rd Conference on Neural Information Processing Systems (NeurIPS, 2019)
Cai, X., Huang, J., Bian, Y. & Church, K. Isotropy in the contextual embedding space: Clusters and manifolds. In International Conference on Learning Representations (ICLR, 2021).
Herculano-Houzel, S. Scaling of brain metabolism with a fixed energy budget per neuron: implications for neuronal activity, plasticity and evolution. PLoS ONE 6, e17514 (2011).
Sterling, P. & Laughlin, S. Principles of Neural Design (MIT, 2015).
Hemberger, M., Shein-Idelson, M., Pammer, L. & Laurent, G. Reliable sequential activation of neural assemblies by single pyramidal cells in a three-layered cortex. Neuron 104, 353–369.e5 (2019).
Ishikawa, T. & Ikegaya, Y. Locally sequential synaptic reactivation during hippocampal ripples. Sci. Adv. https://doi.org/10.1126/sciadv.aay1492 (2020). Neighbouring spines are activated serially along a dendrite, towards or away from the cell body.
Agmonsnir, H. & Segev, I. Signal delay and input synchronization in passive dendritic structures. J. Neurophysiol. 70, 2066–2085 (1993).
Iacobucci, G. & Popescu, G. NMDA receptors: linking physiological output to biophysical operation. Nat. Rev. Neurosci. 18, 236–249 (2017).
Branco, T., Clark, B. & Hausser, M. Dendritic discrimination of temporal input sequences in cortical neurons. Science 329, 1671–1675 (2010).
Matsuzaki, M. et al. Dendritic spine geometry is critical for AMPA receptor expression in hippocampal CA1 pyramidal neurons. Nat. Neurosci. 4, 1086–1092 (2001).
Kerlin, A. et al. Functional clustering of dendritic activity during decision-making. eLife https://doi.org/10.7554/eLife.46966 (2019). Task-associated calcium signals cluster within branches over approximately 10 μm, potentially supporting a large learning capacity in individual neurons.
Shoemaker, P. Neural bistability and amplification mediated by NMDA receptors: analysis of stationary equations. Neurocomputing 74, 3058–3071 (2011).
Major, G., Polsky, A., Denk, W., Schiller, J. & Tank, D. Spatiotemporally graded NMDA spike/plateau potentials in basal dendrites of neocortical pyramidal neurons. J. Neurophysiol. 99, 2584–2601 (2008).
Fino, E. et al. RuBi-glutamate: two-photon and visible-light photoactivation of neurons and dendritic spines. Front. Neural Circuits https://doi.org/10.3389/neuro.04.002.2009 (2009).
Mahowald, M. & Douglas, R. A silicon neuron. Nature 354, 515–518 (1991).
Benjamin, B. et al. Neurogrid: a mixed-analog-digital multichip system for large-scale neural simulations. Proc. IEEE 102, 699–716 (2014).
Hoffmann, M. et al. Unveiling the double-well energy landscape in a ferroelectric layer. Nature 565, 464–467 (2019).
Boescke, T., Muller, J., Brauhaus, D., Schroder, U. & Bottger, U. Ferroelectricity in hafnium oxide thin films. Appl. Phys. Lett. https://doi.org/10.1063/1.3634052 (2011).
Beyer, S. et al. FeFET: A versatile CMOS compatible device with game-changing potential. In 2020 IEEE International Memory Workshop (IMW) 1–4 (IEEE, 2020).
Dally, W. J. & Towles, B. P. Principles and Practices of Interconnection Networks (Elsevier, 2004).
Bi, G. & Poo, M. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464–10472 (1998).
Mead, C. Neuromorphic electronic systems. Proc. IEEE 78, 1629–1636 (1990). This paper introduced neuromorphic computing.
Grollier, J. et al. Neuromorphic spintronics. Nat. Electron. 3, 360–370 (2020).
Leugering, J., Nieters, P. & Pipa, G. A minimal model of neural computation with dendritic plateau potentials. Preprint at bioRxiv https://doi.org/10.1101/690792 (2022).
Beniaguev, D., Segev, I. & London, M. Single cortical neurons as deep artificial neural networks. Neuron 109, 2727–2739.e3 (2021).
Wetzstein, G. et al. Inference in artificial intelligence with deep optics and photonics. Nature 588, 39–47 (2021).
Szatmary, B. & Izhikevich, E. Spike-timing theory of working memory. PLoS Comput. Biol. https://doi.org/10.1371/journal.pcbi.1000879 (2010).
Frady, E. & Sommer, F. Robust computation with rhythmic spike patterns. Proc. Natl Acad. Sci. USA 116, 18050–18059 (2019).
Goltz, J. et al. Fast and energy-efficient neuromorphic deep learning with first-spike times. Nat. Mach. Intell. 3, 823–835 (2021).
Madhavan, A., Sherwood, T. & Strukov, D. Race logic: abusing hardware race conditions to perform useful computation. IEEE Micro 35, 48–57 (2015).
Tzimpragos, G. et al. Temporal computing with superconductors. IEEE Micro 41, 71–79 (2021).
Borgeaud, S. et al. Improving language models by retrieving from trillions of tokens. In Proceedings of the 39th International Conference on Machine Learning (eds Chaudhuri, K. et al.) 2206–2240 (ICML, 2022).
Braun, W. & Memmesheimer, R. M. High-frequency oscillations and sequence generation in two-population models of hippocampal region CA1. PLoS Comput. Biol. 18, e1009891 (2022).
Pulikkottil, V. V., Somashekar, B. P. & Bhalla, U. S. Computation, wiring, and plasticity in synaptic clusters. Curr. Opin. Neurobiol. 70, 101–112 (2021).
Mahowald, M. An Analog VLSI System for Stereoscopic Vision Vol. 265 (Springer Science & Business Media, 1994).
Mead, C. How we created neuromorphic engineering. Nat. Electron. 3, 434–435 (2020).
Henighan, T. et al. Scaling laws for autoregressive generative modeling. Preprint at https://arxiv.org/abs/2010.14701 (2020).
Hoffmann, J. et al. Training compute-optimal large language models. Preprint at https://arxiv.org/abs/2203.15556 (2022).
Diorio, C., Hasler, P., Minch, A. & Mead, C. A. A single-transistor silicon synapse. IEEE Trans. Electron Devices 43, 1972–1980 (1996). An early realization of vector-matrix multiplication inside a 2D memory chip with floating-gate transistors, precursors to the charge-trap transistors that today’s 3D memory chips use.
Boerlin, M., Machens, C. K. & Denève, S. Predictive coding of dynamical variables in balanced spiking networks. PLoS Comput. Biol. 9, e1003258 (2013).
Kulik, A. et al. Compartment-dependent colocalization of Kir3.2-containing K+ channels and GABAB receptors in hippocampal pyramidal cells. J. Neurosci. 26, 4289–4297 (2006).
Kohl, M. M. & Paulsen, O. The roles of GABAB receptors in cortical network activity. Adv. Pharmacol. 58, 205–229 (2010).
Mainen, Z. & Sejnowski, T. Influence of dendritic structure on firing pattern in model neocortical neurons. Nature 382, 363–366 (1996).
This work was supported by US Office of Naval Research (grant numbers N000141310419 and N000141512827), US National Science Foundation (grant number 2223827), Stanford Medical Center Development (Discovery Innovation Fund), Stanford Institute for Human-Centered Artificial Intelligence (HAI), C. Reynolds and GrAI Matter Labs. I thank P. Sterling for his help editing the manuscript.
K.B. is a co-founder and stockholder of Femtosense Inc. and an advisor to Radical Semiconductor.
Peer review information
Nature thanks Dilip Vasudevan and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Boahen, K. Dendrocentric learning for synthetic intelligence. Nature 612, 43–50 (2022). https://doi.org/10.1038/s41586-022-05340-6