Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Deep learning with coherent nanophotonic circuits

Nature Photonics volume 11, pages 441–446 (2017)Cite this article

Subjects

Abstract

Artificial neural networks are computational network models inspired by signal processing in the brain. These models have dramatically improved performance for many machine-learning tasks, including speech and image recognition. However, today's computing hardware is inefficient at implementing neural networks, in large part because much of it was designed for von Neumann computing schemes. Significant effort has been made towards developing electronic architectures tuned to implement artificial neural networks that exhibit improved computational speed and accuracy. Here, we propose a new architecture for a fully optical neural network that, in principle, could offer an enhancement in computational speed and power efficiency over state-of-the-art electronics for conventional inference tasks. We experimentally demonstrate the essential part of the concept using a programmable nanophotonic processor featuring a cascaded array of 56 programmable Mach–Zehnder interferometers in a silicon photonic integrated circuit and show its utility for vowel recognition.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Figure 1: General architecture of the ONN.
Figure 2: Illustration of OIU.
Figure 3: Vowel recognition.

References

  1. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  ADS  Google Scholar 

  2. Silver, D. et al. Mastering the game of go with deep neural networks and tree search. Nature 529, 484–489 (2016).

    Article  ADS  Google Scholar 

  3. Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015).

    Article  ADS  Google Scholar 

  4. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Proc. NIPS 1097–1105 (2012).

  5. Esser, S. K. et al. Convolutional networks for fast, energy efficient neuromorphic computing. Proc. Natl Acad. Sci. USA 113, 11441–11446 (2016).

    Article  Google Scholar 

  6. Mead, C. Neuromorphic electronic systems. Proc. IEEE 78, 1629–1636 (1990).

    Article  Google Scholar 

  7. Poon, C.-S. & Zhou, K. Neuromorphic silicon neurons and large-scale neural networks: challenges and opportunities. Front. Neurosci. 5, 108 (2011).

    Article  Google Scholar 

  8. Shafiee, A. et al. ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars. Proc. ISCA 43, 14–26 (2016).

    Google Scholar 

  9. Misra, J. & Saha, I. Artificial neural networks in hardware: a survey of two decades of progress. Neurocomputing 74, 239–255 (2010).

    Article  Google Scholar 

  10. Chen, Y. H., Krishna, T., Emer, J. S. & Sze, V. Eyeriss: an energy-efficient reconfigurable accelerator for deep convolutional neural networks. IEEE J. Solid-State Circuits 52, 127–138 (2017).

    Article  ADS  Google Scholar 

  11. Graves, A. et al. Hybrid computing using a neural network with dynamic external memory. Nature 538, 471–476 (2016).

    Article  ADS  Google Scholar 

  12. Tait, A. N., Nahmias, M. A., Tian, Y., Shastri, B. J. & Prucnal, P. R. in Nanophotonic Information Physics (ed. Naruse, M.) 183–222 (Springer, 2014).

  13. Tait, A. N., Nahmias, M. A., Shastri, B. J. & Prucnal, P. R. Broadcast and weight: an integrated network for scalable photonic spike processing. J. Lightw. Technol. 32, 3427–3439 (2014).

    Article  ADS  Google Scholar 

  14. Prucnal, P. R., Shastri, B. J., de Lima, T. F., Nahmias, M. A. & Tait, A. N. Recent progress in semiconductor excitable lasers for photonic spike processing. Adv. Opt. Phot. 8, 228–299 (2016).

    Article  Google Scholar 

  15. Vandoorne, K. et al. Experimental demonstration of reservoir computing on a silicon photonics chip. Nat. Commun. 5, 3541 (2014).

    Article  ADS  Google Scholar 

  16. Appeltant, L. et al. Information processing using a single dynamical node as complex system. Nat. Commun. 2, 468 (2011).

    Article  ADS  Google Scholar 

  17. Larger, L. et al. Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. Opt. Express 20, 3241–3249 (2012).

    Article  ADS  Google Scholar 

  18. Paquot, Y. et al. Optoelectronic reservoir computing. Sci. Rep. 2, 287 (2011).

    Article  Google Scholar 

  19. Vivien, L. et al. Zero-bias 40gbit/s germanium waveguide photodetector on silicon. Opt. Express 20, 1096–1101 (2012).

    Article  ADS  Google Scholar 

  20. Cardenas, J. et al. Low loss etchless silicon photonic waveguides. Opt. Express 17, 4752–4757 (2009).

    Article  ADS  Google Scholar 

  21. Yang, L., Zhang, L. & Ji, R. On-chip optical matrix-vector multiplier. In SPIE Optical Engineering + Applications, 88550F (International Society for Optics and Photonics, 2013).

    Google Scholar 

  22. Farhat, N. H., Psaltis, D., Prata, A. & Paek, E. Optical implementation of the Hopfield model. Appl. Opt. 24, 1469–1475 (1985).

    Article  ADS  Google Scholar 

  23. Harris, N. C. et al. Bosonic transport simulations in a large-scale programmable nanophotonic processor. Preprint at http://arXiv.org/abs/1507.03406 (2015).

  24. Schmidhuber, J. Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015).

    Article  Google Scholar 

  25. Lawson, C. L. & Hanson, R. J. Solving Least Squares Problems Vol. 15 (SIAM, 1995).

    Book  Google Scholar 

  26. Miller, D. A. B. Perfect optics with imperfect components. Optica 2, 747–750 (2015).

    Article  ADS  Google Scholar 

  27. Reck, M., Zeilinger, A., Bernstein, H. J. & Bertani, P. Experimental realization of any discrete unitary operator. Phys. Rev. Lett. 73, 58–61 (1994).

    Article  ADS  Google Scholar 

  28. Connelly, M. J. Semiconductor Optical Amplifiers (Springer Science & Business Media, 2007).

    Google Scholar 

  29. Selden, A. Pulse transmission through a saturable absorber. Br. J. Appl. Phys. 18, 743 (1967).

    Article  ADS  MathSciNet  Google Scholar 

  30. Bao, Q. et al. Monolayer graphene as a saturable absorber in a mode-locked laser. Nano Res. 4, 297–307 (2010).

    Article  Google Scholar 

  31. Schirmer, R. W. & Gaeta, A. L. Nonlinear mirror based on two-photon absorption. J. Opt. Soc. Am. B 14, 2865–2868 (1997).

    Article  ADS  Google Scholar 

  32. Soljačić, M., Ibanescu, M., Johnson, S. G., Fink, Y. & Joannopoulos, J. Optimal bistable switching in nonlinear photonic crystals. Phys. Rev. E 66, 055601 (2002).

    Article  ADS  Google Scholar 

  33. Xu, B. & Ming, N.-B. Experimental observations of bistability and instability in a two-dimensional nonlinear optical superlattice. Phys. Rev. Lett. 71, 3959–3962 (1993).

    Article  ADS  Google Scholar 

  34. Centeno, E. & Felbacq, D. Optical bistability infinite-size nonlinear bidimensional photonic crystals doped by a microcavity. Phys. Rev. B 62, R7683–R7686 (2000).

    Article  ADS  Google Scholar 

  35. Nozaki, K. et al. Sub-femtojoule all-optical switching using a photonic-crystal nanocavity. Nat. Photon. 4, 477–483 (2010).

    Article  ADS  Google Scholar 

  36. Ríos, C. et al. Integrated all-photonic non-volatile multilevel memory. Nat. Photon. 9, 725–732 (2015).

    Article  ADS  Google Scholar 

  37. Krizhevsky, A., Sutskever, I. & Hinton, G. E. in Imagenet Classification with Deep Convolutional Neural Networks (eds Pereira, F., Burges, C. J. C., Bottou, L. & Weinberger, K. Q.) 1097–1105 (Curran Associates, 2012).

  38. Cheng, Z., Tsang, H. K., Wang, X., Xu, K. & Xu, J.-B. In-plane optical absorption and free carrier absorption in graphene-on-silicon waveguides. IEEE J. Sel. Top. Quantum Electron. 20, 43–48 (2014).

    Article  ADS  Google Scholar 

  39. Chow, D. & Abdulla, W. H. in PRICAI 2004: Trends in Artificial Intelligence (eds Booth, R. & Zhang, M.-L.) 901–908 (Springer, 2004).

  40. Deterding, D. H. Speaker Normalisation for Automatic Speech Recognition. PhD thesis, Univ. Cambridge (1990).

  41. Hinton, G. E. & Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006).

    Article  ADS  MathSciNet  Google Scholar 

  42. Baehr-Jones, T. et al. A 25 Gb/s silicon photonics platform. Preprint at http://arXiv.org/abs/1203.0767 (2012).

  43. Harris, N. C. et al. Efficient, compact and low loss thermo-optic phase shifter in silicon. Opt. Express 22, 10487–10493 (2014).

    Article  ADS  Google Scholar 

  44. Bertsimas, D. & Nohadani, O. Robust optimization with simulated annealing. J. Global Optim. 48, 323–334 (2010).

    Article  MathSciNet  Google Scholar 

  45. Wang, Q. et al. Optically reconfigurable metasurfaces and photonic devices based on phase change materials. Nat. Photon. 10, 60–65 (2016).

    Article  ADS  Google Scholar 

  46. Tanabe, T., Notomi, M., Mitsugi, S., Shinya, A. & Kuramochi, E. Fast bistable all-optical switch and memory on a silicon photonic crystal on-chip. Opt. Lett. 30, 2575–2577 (2005).

    Article  ADS  Google Scholar 

  47. Horowitz, M. Computing's energy problem. In 2014 IEEE Int. Solid-State Circuits Conf. Digest of Technical Papers (ISSCC) 10–14 (IEEE, 2014).

  48. Arjovsky, M., Shah, A. & Bengio, Y. Unitary evolution recurrent neural networks. In Int. Conf. Machine Learning (2016).

  49. Sun, J., Timurdogan, E., Yaacobi, A., Hosseini, E. S. & Watts, M. R. Large-scale nanophotonic phased array. Nature 493, 195–199 (2013).

    Article  ADS  Google Scholar 

  50. Rechtsman, M. C. et al. Photonic Floquet topological insulators. Nature 496, 196–200 (2013).

    Article  ADS  Google Scholar 

  51. Jia, Y. et al. Caffe: convolutional architecture for fast feature embedding. In Proc. 22nd ACM Int. Conf. Multimedia (MM ’14), 675–678 (ACM, 2014).

  52. Sun, C. et al. Single-chip microprocessor that communicates directly using light. Nature 528, 534–538 (2015).

    Article  ADS  Google Scholar 

Download references

Acknowledgements

The authors thank Y. LeCun, M. Tegmark, G. Pratt, I. Chuang and V. Sze for discussions. This work was supported in part by the Army Research Office through the Institute for Soldier Nanotechnologies under contract no. W911NF-13-D0001 and in part by the National Science Foundation under grant no. CCF-1640012 and in part by the Air Force Office of Scientific Research (AFOSR) Multidisciplinary University Research Initiative (FA9550-14-1-0052) and the Air Force Research Laboratory RITA programme (FA8750-14-2-0120). M.H. acknowledges support from AFOSR STTR grants, numbers FA9550-12-C-0079 and FA9550-12-C-0038 and G. Pomrenke, of AFOSR, for his support of the OpSIS effort, through both a PECASE award (FA9550-13-1-0027) and funding for OpSIS (FA9550-10-1-0439). N.H. acknowledges support from National Science Foundation Graduate Research Fellowship grant no. 1122374.

Author information

Author notes

  1. Yichen Shen and Nicholas C. Harris: These authors contributed equally to this work.

Authors and Affiliations

  1. Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, 02139, Massachusetts, USA

    Yichen Shen, Nicholas C. Harris, Scott Skirlo, Mihika Prabhu, Dirk Englund & Marin Soljačić

  2. Elenion, 171 Madison Avenue, Suite 1100, New York, 10016, New York, USA

    Tom Baehr-Jones & Michael Hochberg

  3. Department of Mathematics, Massachusetts Institute of Technology, Cambridge, 02139, Massachusetts, USA

    Xin Sun

  4. Department of Biology, Massachusetts Institute of Technology, Cambridge, 02139, Massachusetts, USA

    Shijie Zhao

  5. Université de Sherbrooke, Administration, 2500 Boulevard de l'Université, Sherbrooke, J1K 2R1, Quebec, Canada

    Hugo Larochelle

Authors
  1. Yichen Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicholas C. Harris
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Scott Skirlo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mihika Prabhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tom Baehr-Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael Hochberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xin Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Shijie Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Hugo Larochelle
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Dirk Englund
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Marin Soljačić
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Y.S., N.C.H., S.S., X.S., S.Z., D.E. and M.S. developed the theoretical model for the optical neural network. N.H. designed the photonic chip and built the experimental set-up. N.H., Y.S. and M.P. performed the experiment. Y.S., S.S. and X.S. prepared the data and developed the code for training MZI parameters. T.B.-J. and M.H. fabricated the photonic integrated circuit. All authors contributed to writing the paper.

Corresponding authors

Correspondence to Yichen Shen or Nicholas C. Harris.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary information

Supplementary information (PDF 1410 kb)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, Y., Harris, N., Skirlo, S. et al. Deep learning with coherent nanophotonic circuits. Nature Photon 11, 441–446 (2017). https://doi.org/10.1038/nphoton.2017.93

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nphoton.2017.93

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing