Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Deep neural networks for the evaluation and design of photonic devices

Nature Reviews Materials volume 6, pages 679–700 (2021)Cite this article

Subjects

Abstract

The data-science revolution is poised to transform the way photonic systems are simulated and designed. Photonic systems are, in many ways, an ideal substrate for machine learning: the objective of much of computational electromagnetics is the capture of nonlinear relationships in high-dimensional spaces, which is the core strength of neural networks. Additionally, the mainstream availability of Maxwell solvers makes the training and evaluation of neural networks broadly accessible and tailorable to specific problems. In this Review, we show how deep neural networks, configured as discriminative networks, can learn from training sets and operate as high-speed surrogate electromagnetic solvers. We also examine how deep generative networks can learn geometric features in device distributions and even be configured to serve as robust global optimizers. Fundamental data-science concepts framed within the context of photonics are also discussed, including the network-training process, delineation of different network classes and architectures, and dimensionality reduction.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Overview of deep learning for photonics.
Fig. 2: Overview of data structures for photonics.
Fig. 3: Surrogate modelling with discriminative networks.
Fig. 4: Inverse design with discriminative networks.
Fig. 5: Methods for dimensionality reduction.
Fig. 6: Concepts for implementing generative neural networks for inverse design.
Fig. 7: Overview of variational autoencoders.
Fig. 8: Generative adversarial networks for freeform device modelling.
Fig. 9: Global topology-optimization networks.

Similar content being viewed by others

References

  1. Sun, J., Timurdogan, E., Yaacobi, A., Hosseini, E. S. & Watts, M. R. Large-scale nanophotonic phased array. Nature 493, 195–199 (2013).

    CAS  Google Scholar 

  2. Stern, B., Ji, X., Okawachi, Y., Gaeta, A. L. & Lipson, M. Battery-operated integrated frequency comb generator. Nature 562, 401–405 (2018).

    CAS  Google Scholar 

  3. Thomson, D. et al. Roadmap on silicon photonics. J. Opt. 18, 073003 (2016).

    Google Scholar 

  4. Cheng, Q., Bahadori, M., Glick, M., Rumley, S. & Bergman, K. Recent advances in optical technologies for data centers: a review. Optica 5, 1354–1370 (2018).

    CAS  Google Scholar 

  5. O’Brien, J. L., Furusawa, A. & Vuckovic, J. Photonic quantum technologies. Nat. Photonics 3, 687–695 (2009).

    Google Scholar 

  6. Tittl, A. et al. Imaging-based molecular barcoding with pixelated dielectric metasurfaces. Science 360, 1105–1109 (2018).

    CAS  Google Scholar 

  7. Baumberg, J. J., Aizpurua, J., Mikkelsen, M. H. & Smith, D. R. Extreme nanophotonics from ultrathin metallic gaps. Nat. Mater. 18, 668–678 (2019).

    CAS  Google Scholar 

  8. Urbas, A. M. et al. Roadmap on optical metamaterials. J. Opt. 18, 093005 (2016).

    Google Scholar 

  9. Yu, N. & Capasso, F. Flat optics with designer metasurfaces. Nat. Mater. 13, 139–150 (2014).

    CAS  Google Scholar 

  10. Inan, U. S. & Marshall, R. A. Numerical Electromagnetics (Cambridge Univ. Press, 2014).

  11. Jin, J.-M. The Finite Element Method in Electromagnetics (Wiley, 2014).

  12. Campbell, S. D. et al. Review of numerical optimization techniques for meta-device design [invited]. Opt. Mater. Express 9, 1842–1863 (2019).

    Google Scholar 

  13. Kim, W. J. & O’Brien, J. D. Optimization of a two-dimensional photonic-crystal waveguide branch by simulated annealing and the finite-element method. J. Opt. Soc. Am. B 21, 289–295 (2004).

    CAS  Google Scholar 

  14. Zhao, Y. et al. Broadband diffusion metasurface based on a single anisotropic element and optimized by the simulated annealing algorithm. Sci. Rep. 6, 23896 (2016).

    CAS  Google Scholar 

  15. Preble, S., Lipson, M. & Lipson, H. Two-dimensional photonic crystals designed by evolutionary algorithms. Appl. Phys. Lett. 86, 061111 (2005).

    Google Scholar 

  16. Shi, Y., Li, W., Raman, A. & Fan, S. Optimization of multilayer optical films with a memetic algorithm and mixed integer programming. ACS Photonics 5, 684–691 (2018).

    CAS  Google Scholar 

  17. Piggott, A. Y. et al. Inverse design and demonstration of a compact and broadband on-chip wavelength demultiplexer. Nat. Photonics 9, 374–377 (2015).

    CAS  Google Scholar 

  18. Lalau-Keraly, C. M., Bhargava, S., Miller, O. D. & Yablonovitch, E. Adjoint shape optimization applied to electromagnetic design. Opt. Express 21, 21693–21701 (2013).

    Google Scholar 

  19. Jensen, J. S. & Sigmund, O. Topology optimization for nano-photonics. Laser Photonics Rev. 5, 308–321 (2011).

    CAS  Google Scholar 

  20. Abbeel, P., Coates, A. & Ng, A. Y. Autonomous helicopter aerobatics through apprenticeship learning. Int. J. Robot. Res. 29, 1608–1639 (2010).

    Google Scholar 

  21. Lavecchia, A. Machine-learning approaches in drug discovery: methods and applications. Drug. Discov. Today 20, 318–331 (2015).

    Google Scholar 

  22. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012).

    Google Scholar 

  23. Wu, Y. et al. Google’s neural machine translation system: Bridging the gap between human and machine translation. Preprint at https://arxiv.org/abs/1609.08144 (2016).

  24. Abadi, M. et al. TensorFlow: Large-scale machine learning on heterogeneous systems. (2015). Software available from tensorflow.org.

  25. Paszke, A. et al. in 31st Conference on Neural Information Processing Systems (NIPS 2017) (2017).

  26. cmphy: A Python interface to the COMSOL Multiphysics API. https://github.com/fellobos/cmphy (2018).

  27. Python API overview (Lumerical Support, Vancouver, BC, Canada).

  28. MATLAB, version 7.10.0, R2020a (MathWorks Inc.)

  29. Zhang, Q., Cheng, L. & Boutaba, R. Cloud computing: state-of-the-art and research challenges. J. Internet Serv. Appl. 1, 7–18 (2010).

    CAS  Google Scholar 

  30. Nickolls, J. & Dally, W. J. The GPU computing era. IEEE Micro 30, 56–69 (2010).

    Google Scholar 

  31. Jouppi, N. P. et al. In-datacenter performance analysis of a tensor processing unit. SIGARCH Comput. Archit. News 45, 1–12 (2017).

    Google Scholar 

  32. Yao, K., Unni, R. & Zheng, Y. Intelligent nanophotonics: merging photonics and artificial intelligence at the nanoscale. Nanophotonics 8, 339–366 (2019).

    Google Scholar 

  33. So, S., Badloe, T., Noh, J., Bravo-Abad, J. & Rho, J. Deep learning enabled inverse design in nanophotonics. Nanophotonics 9, 1041–1057 (2020).

    Google Scholar 

  34. Hegde, R. S. Deep learning: a new tool for photonic nanostructure design. Nanoscale Adv. 2, 1007–1023 (2020).

    Google Scholar 

  35. Tunnermann, H. & Shirakawa, A. Deep reinforcement learning for coherent beam combining applications. Opt. Express 27, 24223–24230 (2019).

    Google Scholar 

  36. Sajedian, I., Badloe, T. & Rho, J. Optimisation of colour generation from dielectric nanostructures using reinforcement learning. Opt. Express 27, 5874–5883 (2019).

    CAS  Google Scholar 

  37. Iman Sajedian, H. L. J. R. Double-deep Q-learning to increase the efficiency of metasurface holograms. Sci. Rep. 9, 10899 (2019).

    Google Scholar 

  38. Shen, Y. et al. Deep learning with coherent nanophotonic circuits. Nat. Photonics 11, 441–446 (2017).

    CAS  Google Scholar 

  39. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    CAS  Google Scholar 

  40. Nadell, C. C., Huang, B., Malof, J. M. & Padilla, W. J. Deep learning for accelerated all-dielectric metasurface design. Opt. Express 27, 27523–27535 (2019).

    CAS  Google Scholar 

  41. Jiang, J. & Fan, J. A. Global optimization of dielectric metasurfaces using a physics-driven neural network. Nano Lett. 19, 5366–5372 (2019).

    CAS  Google Scholar 

  42. Jiang, J. & Fan, J. A. Simulator-based training of generative neural networks for the inverse design of metasurfaces. Nanophotonics 9, 1059–1069 (2019).

    Google Scholar 

  43. Ma, W., Cheng, F., Xu, Y., Wen, Q. & Liu, Y. Probabilistic representation and inverse design of metamaterials based on a deep generative model with semi-supervised learning strategy. Adv. Mater. 31, 1901111 (2019).

    Google Scholar 

  44. Liu, Z., Zhu, D., Rodrigues, S. P., Lee, K.-T. & Cai, W. Generative model for the inverse design of metasurfaces. Nano Lett. 18, 6570–6576 (2018).

    CAS  Google Scholar 

  45. So, S. & Rho, J. Designing nanophotonic structures using conditional deep convolutional generative adversarial networks. Nanophotonics 8, 1255–1261 (2019).

    Google Scholar 

  46. An, S. et al. Multifunctional metasurface design with a generative adversarial network. Preprint at https://arxiv.org/abs/1908.04851 (2019).

  47. Jiang, J. et al. Free-form diffractive metagrating design based on generative adversarial networks. ACS Nano 13, 8872–8878 (2019).

    CAS  Google Scholar 

  48. Liu, Z., Zhu, Z. & Cai, W. Topological encoding method for data-driven photonics inverse design. Opt. Express 28, 4825–4835 (2020).

    Google Scholar 

  49. Van den Oord, A. et al. Conditional image generation with pixelcnn decoders. Adv. Neural Inf. Process. Syst. 29, 4790–4798 (2016).

    Google Scholar 

  50. Krizhevsky, A., Sutskever, I. & Hinton, G. E. in Advances in Neural Information Processing Systems 1097–1105 (Association for Computing Machinery, 2012).

  51. Szegedy, C., Ioffe, S., Vanhoucke, V. & Alemi, A. A. in Thirty-First AAAI Conference on Artificial Intelligence 4278–4284 (Association for Computing Machinery, 2017).

  52. Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. Preprint at arXiv https://arxiv.org/abs/1409.1556 (2014).

  53. Szegedy, C. et al. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 1–9 (IEEE, 2015).

  54. Wiecha, P. R. & Muskens, O. L. Deep learning meets nanophotonics: A generalized accurate predictor for near fields and far fields of arbitrary 3D nanostructures. Nano Lett. 20, 329–338 (2020).

    CAS  Google Scholar 

  55. Henaff, M., Bruna, J. & LeCun, Y. Deep convolutional networks on graph-structured data. Preprint at arXiv https://arxiv.org/abs/1506.05163 (2015).

  56. Niepert, M., Ahmed, M. & Kutzkov, K. in Proceedings of the 33rd International Conference on Machine Learning, PMLR Vol. 48 2014–2023 (2016).

  57. Veličković, P. et al. Graph attention networks. Preprint at arXiv https://arxiv.org/abs/1710.10903 (2017).

  58. Bapst, V. et al. Unveiling the predictive power of static structure in glassy systems. Nat. Phys. 16, 448–454 (2020).

    CAS  Google Scholar 

  59. Duvenaud, D. K. et al. Convolutional networks on graphs for learning molecular fingerprints. Adv. Neural Inf. Process. Syst. 28, 2224–2232 (2015).

    Google Scholar 

  60. Torng, W. & Altman, R. B. Graph convolutional neural networks for predicting drug-target interactions. J. Chem. Inf. Model. 59, 4131–4149 (2019).

    CAS  Google Scholar 

  61. Li, Y., Tarlow, D., Brockschmidt, M. & Zemel, R. Gated graph sequence neural networks. Preprint at https://arxiv.org/abs/1511.05493 (2015).

  62. Ma, T., Chen, J. & Xiao, C. Constrained generation of semantically valid graphs via regularizing variational autoencoders. Adv. Neural Inf. Process. Syst. 31, 7113–7124 (2018).

    Google Scholar 

  63. Graves, A. Generating sequences with recurrent neural networks. Preprint at https://arxiv.org/abs/1308.0850 (2013).

  64. Sutskever, I., Vinyals, O. & Le, Q. V. Sequence to sequence learning with neural networks. Adv. Neural Inf. Process. Syst. 2, 3104–3112 (2014).

    Google Scholar 

  65. Luong, M.-T., Pham, H. & Manning, C. D. Effective approaches to attention-based neural machine translation. Preprint at https://arxiv.org/abs/1508.04025 (2015).

  66. Weston, J., Chopra, S. & Bordes, A. Memory networks. Preprint at https://arxiv.org/abs/1410.3916 (2014).

  67. Hughes, T. W., Williamson, I. A., Minkov, M. & Fan, S. Wave physics as an analog recurrent neural network. Sci. Adv. 5, eaay6946 (2019).

    Google Scholar 

  68. Vai, M. & Prasad, S. Automatic impedance matching with a neural network. IEEE Microw. Guided Wave Lett. 3, 353–354 (1993).

    Google Scholar 

  69. Zaabab, A. H., Zhang, Q. J. & Nakhla, M. in 1994 IEEE MTT-S International Microwave Symposium Digest (Cat. No.94CH3389-4) Vol. 1 393–396 (IEEE, 1994).

  70. Vai, M. M., Wu, S., Li, B. & Prasad, S. Reverse modeling of microwave circuits with bidirectional neural network models. IEEE Trans. Microw. Theory Tech. 46, 1492–1494 (1998).

    Google Scholar 

  71. Watson, P. M. & Gupta, K. C. Design and optimization of CPW circuits using EM-ANN models for CPW components. IEEE Trans. Microw. Theory Tech. 45, 2515–2523 (1997).

    Google Scholar 

  72. Devabhaktuni, V., Chattaraj, B., Yagoub, M. C. E. & Zhang, Q. J. in 2002 IEEE MTT-S International Microwave Symposium Digest (Cat. No.02CH37278) Vol. 2 1097–1100 (IEEE, 2002).

  73. Bandler, J. W. et al. Space mapping: the state of the art. IEEE Trans. Microw. Theory Tech. 52, 337–361 (2004).

    Google Scholar 

  74. da F. Silva, P. H., Cruz, R. M. S. & D’Assuncao, A. G. Blending PSO and ANN for optimal design of FSS filters with Koch island patch elements. IEEE Trans. Magnetics 46, 3010–3013 (2010).

    Google Scholar 

  75. da Silva, M. R., de L. Nóbrega, C., da F. Silva, P. H. & D’Assunção, A. G. Optimization of FSS with Sierpinski island fractal elements using population-based search algorithms and MLP neural network. Microw. Optical Technol. Lett. 56, 827–831 (2014).

    Google Scholar 

  76. Luna, D. R., Vasconcelos, C. F. L. & Cruz, R. M. S. in 2013 SBMO/IEEE MTT-S International Microwave Optoelectronics Conference (IMOC) 1–4 (IEEE, 2013).

  77. Zhang, Q. et al. Machine-learning designs of anisotropic digital coding metasurfaces. Adv. Theory Simul. 2, 1800132 (2019).

    Google Scholar 

  78. Liu, Y., Lu, T., Wu, K. & Jin, J.-M. in 2018 IEEE 27th Conference on Electrical Performance of Electronic Packaging and Systems (EPEPS) 261–263 (IEEE, 2018).

  79. Zhang, G., He, H. & Katabi, D. in Proceedings of the 36th International Conference on Machine Learning, PMLR Vol. 97 (eds Chaudhuri, K. & Salakhutdinov, R.) 7364–7373 (2019).

  80. Zhang, Q. J., Gupta, K. C. & Devabhaktuni, V. K. Artificial neural networks for RF and microwave design - from theory to practice. IEEE Trans. Microw. Theory Tech. 51, 1339–1350 (2003).

    Google Scholar 

  81. Rayas-Sanchez, J. E. EM-based optimization of microwave circuits using artificial neural networks: The state-of-the-art. IEEE Trans. Microw. Theory Tech. 52, 420–435 (2004).

    Google Scholar 

  82. Koziel, S., Cheng, Q. S. & Bandler, J. W. Space mapping. IEEE Microw. Mag. 9, 105–122 (2008).

    Google Scholar 

  83. Malheiros-Silveira, G. N. & Hernandez-Figueroa, H. E. Prediction of dispersion relation and PBGs in 2-D PCs by using artificial neural networks. IEEE Photonics Technol. Lett. 24, 1799–1801 (2012).

    Google Scholar 

  84. El-Mosalmy, D. D., Hameed, M. F. O., Areed, N. F. F. & Obayya, S. S. A. Novel neural network based optimization approach for photonic devices. Optical Quantum Electron. 46, 439–453 (2014).

    Google Scholar 

  85. Andrawis, R. R., Swillam, M. A., El-Gamal, M. A. & Soliman, E. A. Artificial neural network modeling of plasmonic transmission lines. Appl. Opt. 55, 2780–2790 (2016).

    Google Scholar 

  86. Chugh, S., Gulistan, A., Ghosh, S. & Rahman, B. M. A. Machine learning approach for computing optical properties of a photonic crystal fiber. Opt. Express 27, 36414–36425 (2019).

    CAS  Google Scholar 

  87. da Silva Ferreira, A., Malheiros-Silveira, G. N. & Hernández-Figueroa, H. E. Computing optical properties of photonic crystals by using multilayer perceptron and extreme learning machine. J. Lightwave Technol. 36, 4066–4073 (2018).

    Google Scholar 

  88. Asano, T. & Noda, S. Optimization of photonic crystal nanocavities based on deep learning. Opt. Express 26, 32704–32717 (2018).

    CAS  Google Scholar 

  89. Zhang, T. et al. Efficient spectrum prediction and inverse design for plasmonic waveguide systems based on artificial neural networks. Photon. Res. 7, 368–380 (2019).

    CAS  Google Scholar 

  90. da Silva Ferreira, A., da Silva Santos, C. H., Gonçalves, M. S. & Figueroa, H. E. H. Towards an integrated evolutionary strategy and artificial neural network computational tool for designing photonic coupler devices. Appl. Soft Comput. 65, 1–11 (2018).

    Google Scholar 

  91. Tahersima, M. H. et al. Deep neural network inverse design of integrated photonic power splitters. Sci. Rep. 9, 1368 (2019).

    Google Scholar 

  92. Gabr, A. M. et al. Design and optimization of optical passive elements using artificial neural networks. J. Opt. Soc. Am. B 36, 999–1007 (2019).

    CAS  Google Scholar 

  93. Hammond, A. M. & Camacho, R. M. Designing integrated photonic devices using artificial neural networks. Opt. Express 27, 29620–29638 (2019).

    CAS  Google Scholar 

  94. Gostimirovic, D. & Ye, W. N. An open-source artificial neural network model for polarization-insensitive silicon-on-insulator subwavelength grating couplers. IEEE J. Sel. Top. Quantum Electron. 25, 1–5 (2019).

    Google Scholar 

  95. Miyatake, Y., Sekine, N., Toprasertpong, K., Takagi, S. & Takenaka, M. Computational design of efficient grating couplers using artificial intelligence. Jpn. J. Appl. Phys. 59, SGGE09 (2020).

    CAS  Google Scholar 

  96. Peurifoy, J. et al. Nanophotonic particle simulation and inverse design using artificial neural networks. Sci. Adv. 4, eaar4206 (2018).

    Google Scholar 

  97. Ma, W., Cheng, F. & Liu, Y. Deep-learning-enabled on-demand design of chiral metamaterials. ACS Nano 12, 6326–6334 (2018).

    CAS  Google Scholar 

  98. Li, Y. et al. Self-learning perfect optical chirality via a deep neural network. Phys. Rev. Lett. 123, 213902 (2019).

    CAS  Google Scholar 

  99. Tao, Z. et al. Exploiting deep learning network in optical chirality tuning and manipulation of diffractive chiral metamaterials. Nanophotonics 9, 2945–2956 (2020).

    Google Scholar 

  100. Malkiel, I. et al. Plasmonic nanostructure design and characterization via deep learning. Light Sci. Appl. 7, 60 (2018).

    Google Scholar 

  101. Sajedian, I., Kim, J. & Rho, J. Finding the optical properties of plasmonic structures by image processing using a combination of convolutional neural networks and recurrent neural networks. Microsyst. Nanoeng. 5, 27 (2019).

    Google Scholar 

  102. Baxter, J. et al. Plasmonic colours predicted by deep learning. Sci. Rep. 9, 8074 (2019).

    Google Scholar 

  103. Balin, I., Garmider, V., Long, Y. & Abdulhalim, I. Training artificial neural network for optimization of nanostructured VO2-based smart window performance. Opt. Express 27, A1030–A1040 (2019).

    CAS  Google Scholar 

  104. Blanchard-Dionne, A.-P. & Martin, O. J. F. Teaching optics to a machine learning network. Opt. Lett. 45, 2922–2925 (2020).

    Google Scholar 

  105. Inampudi, S. & Mosallaei, H. Neural network based design of metagratings. Appl. Phys. Lett. 112, 241102 (2018).

    Google Scholar 

  106. An, S. et al. A deep learning approach for objective-driven all-dielectric metasurface design. ACS Photonics 6, 3196–3207 (2019).

    CAS  Google Scholar 

  107. Chen, Y., Zhu, J., Xie, Y., Feng, N. & Liu, Q. H. Smart inverse design of graphene-based photonic metamaterials by an adaptive artificial neural network. Nanoscale 11, 9749–9755 (2019).

    CAS  Google Scholar 

  108. Phan, A. D. et al. Deep learning for the inverse design of mid-infrared graphene plasmons. Crystals 10, 125 (2020).

    CAS  Google Scholar 

  109. Gao, L., Li, X., Liu, D., Wang, L. & Yu, Z. A bidirectional deep neural network for accurate silicon color design. Adv. Mater. 31, 1905467 (2019).

    CAS  Google Scholar 

  110. Liu, D., Tan, Y., Khoram, E. & Yu, Z. Training deep neural networks for the inverse design of nanophotonic structures. ACS Photonics 5, 1365–1369 (2018).

    CAS  Google Scholar 

  111. Pilozzi, L., Farrelly, F. A., Marcucci, G. & Conti, C. Machine learning inverse problem for topological photonics. Commun. Phys. 1, 57 (2018).

    Google Scholar 

  112. Long, Y., Ren, J., Li, Y. & Chen, H. Inverse design of photonic topological state via machine learning. Appl. Phys. Lett. 114, 181105 (2019).

    Google Scholar 

  113. Singh, R., Agarwal, A. M. & Anthony, B. W. Mapping the design space of photonic topological states via deep learning. Opt. Express 28, 27893–27902 (2020).

    CAS  Google Scholar 

  114. Zaabab, A. H., Zhang, Q. J. & Nakhla, M. A neural network modeling approach to circuit optimization and statistical design. IEEE Trans. Microw. Theory Tech. 43, 1349–1358 (1995).

    Google Scholar 

  115. Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980 (2014).

  116. Hegde, R. S. Accelerating optics design optimizations with deep learning. Opt. Eng. 58, 065103 (2019).

    Google Scholar 

  117. Lin, R., Zhai, Y., Xiong, C. & Li, X. Inverse design of plasmonic metasurfaces by convolutional neural network. Opt. Lett. 45, 1362–1365 (2020).

    Google Scholar 

  118. Zhang, C., Jin, J., Na, W., Zhang, Q. & Yu, M. Multivalued neural network inverse modeling and applications to microwave filters. IEEE Trans. Microw. Theory Tech. 66, 3781–3797 (2018).

    Google Scholar 

  119. So, S., Mun, J. & Rho, J. Simultaneous inverse design of materials and structures via deep learning: Demonstration of dipole resonance engineering using core–shell nanoparticles. ACS Appl. Mater. Interfaces 11, 24264–24268 (2019).

    CAS  Google Scholar 

  120. Xu, L. et al. Enhanced light–matter interactions in dielectric nanostructures via machine-learning approach. Adv. Photon. 2, 026003 (2019).

    Google Scholar 

  121. Zimek, A., Schubert, E. & Kriegel, H.-P. A survey on unsupervised outlier detection in high-dimensional numerical data. Stat. Anal. Data Min. 5, 363–387 (2012).

    Google Scholar 

  122. Marimont, R. & Shapiro, M. Nearest neighbour searches and the curse of dimensionality. IMA J. Appl. Math. 24, 59–70 (1979).

    Google Scholar 

  123. Melati, D. et al. Mapping the global design space of nanophotonic components using machine learning pattern recognition. Nat. Commun. 10, 4775 (2019).

    Google Scholar 

  124. Kramer, M. A. Nonlinear principal component analysis using autoassociative neural networks. AIChE J. 37, 233–243 (1991).

    CAS  Google Scholar 

  125. Baldi, P. Proceedings of the 2011 International Conference on Unsupervised and Transfer Learning workshop 27, 37–49 (2012).

    Google Scholar 

  126. Kiarashinejad, Y., Abdollahramezani, S. & Adibi, A. Deep learning approach based on dimensionality reduction for designing electromagnetic nanostructures. NPJ Comput. Mater. 6, 12 (2020).

    Google Scholar 

  127. Shi, X., Qiu, T., Wang, J., Zhao, X. & Qu, S. Metasurface inverse design using machine learning approaches. J. Phys. D Appl. Phys. 53, 275105 (2020).

    CAS  Google Scholar 

  128. Kiarashinejad, Y. et al. Knowledge discovery in nanophotonics using geometric deep learning. Adv. Intell. Syst. 2, 1900132 (2020).

    Google Scholar 

  129. Karras, T., Aila, T., Laine, S. & Lehtinen, J. Progressive growing of GANs for improved quality, stability, and variation. Preprint at https://arxiv.org/abs/1710.10196 (2018).

  130. Salimans, T. et al. Improved techniques for training GANS. Preprint at https://arxiv.org/abs/1606.03498 (2016).

  131. Sell, D., Yang, J., Doshay, S., Yang, R. & Fan, J. A. Large-angle, multifunctional metagratings based on freeform multimode geometries. Nano Lett. 17, 3752–3757 (2017).

    CAS  Google Scholar 

  132. Phan, T. et al. High-efficiency, large-area, topology-optimized metasurfaces. Light Sci. Appl. 8, 48 (2019).

    Google Scholar 

  133. Fan, J. A. Freeform metasurface design based on topology optimization. MRS Bull. 45, 196–201 (2020).

    Google Scholar 

  134. Hughes, T. W., Minkov, M., Williamson, I. A. & Fan, S. Adjoint method and inverse design for nonlinear nanophotonic devices. ACS Photonics 5, 4781–4787 (2018).

    CAS  Google Scholar 

  135. Maclaurin, D., Duvenaud, D. & Adams, R. P. in Proceedings of the 32nd International Conference on International Conference on Machine Learning Vol. 37 2113–2122 (JMLR, 2015).

  136. Minkov, M. et al. Inverse design of photonic crystals through automatic differentiation. ACS Photonics 7, 1729–1741 (2020).

    CAS  Google Scholar 

  137. Hughes, T. W., Williamson, I. A., Minkov, M. & Fan, S. Forward-mode differentiation of Maxwell’s equations. ACS Photonics 6, 3010–3016 (2019).

    CAS  Google Scholar 

  138. Uria, B., Murray, I. & Larochelle, H. RNADE: The real-valued neural autoregressive density-estimator. Preprint at https://arxiv.org/abs/1306.0186 (2013).

  139. Oord, A. V. D. et al. WaveNet: A generative model for raw audio. Preprint at https://arxiv.org/abs/1609.03499 (2016).

  140. Larochelle, H. & Murray, I. in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics 15, 29–37 (2011).

    Google Scholar 

  141. van den Oord, A., Kalchbrenner, N. & Kavukcuoglu, K. Pixel recurrent neural networks. Preprint at https://arxiv.org/abs/1601.06759 (2016).

  142. Kingma, D. P. & Welling, M. Auto-encoding variational bayes. Preprint at https://arxiv.org/abs/1312.6114 (2013).

  143. Goodfellow, I. et al. in Advances in Neural Information Processing Systems Vol. 27 (eds Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D. & Weinberger, K. Q.) 2672–2680 (Curran Associates, 2014).

  144. Sohn, K., Lee, H. & Yan, X. in Advances in Neural Information Processing Systems Vol. 28 (eds Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M. & Garnett, R.) 3483–3491 (Curran Associates, 2015).

  145. Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I. & Frey, B. Adversarial autoencoders. Preprint at arXiv https://arxiv.org/abs/1511.05644 (2015).

  146. Kingma, D. P., Mohamed, S., Rezende, D. J. & Welling, M. Semi-supervised learning with deep generative models. Adv. Neural Inf. Process. Syst. 27, 3581–3589 (2014).

    Google Scholar 

  147. Liu, Z., Raju, L., Zhu, D. & Cai, W. A hybrid strategy for the discovery and design of photonic structures. IEEE J. Emerg. Sel. Top. Circuits Syst. 10, 126–135 (2020).

    Google Scholar 

  148. Radford, A., Metz, L. & Chintala, S. Unsupervised representation learning with deep convolutional generative adversarial networks. Preprint at https://arxiv.org/abs/1511.06434 (2015).

  149. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. & Courville, A. C. Improved training of Wasserstein GANs. Adv. Neural Inf. Process. Syst. 30, 5767–5777 (2017).

    Google Scholar 

  150. Brock, A., Donahue, J. & Simonyan, K. Large scale GAN training for high fidelity natural image synthesis. Preprint at https://arxiv.org/abs/1809.11096 (2018).

  151. Arjovsky, M., Chintala, S. & Bottou, L. in Proceedings of the 34th International Conference on Machine Learning, Proceedings of Machine Learning Research Vol. 10 (eds Precup, D. & Teh, Y. W.) 214–223 (PMLR, 2017).

  152. Hodge, J. A., Mishra, K. V. & Zaghloul, A. I. in 2019 IEEE International Symposium on Phased Array System Technology (PAST) 1–6 (IEEE, 2019).

  153. Hodge, J. A., Vijay Mishra, K. & Zaghloul, A. I. in 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP) 1–5 (IEEE, 2019).

  154. Kudyshev, Z. A., Kildishev, A. V., Shalaev, V. M. & Boltasseva, A. in CLEO: QELS_Fundamental Science, FTh3C.2 (Optical Society of America, 2019).

  155. Kudyshev, Z. A., Kildishev, A. V., Shalaev, V. M. & Boltasseva, A. Machine-learning-assisted metasurface design for high-efficiency thermal emitter optimization. Appl. Phys. Rev. 7, 021407 (2020).

    CAS  Google Scholar 

  156. Yang, J. & Fan, J. A. Topology-optimized metasurfaces: impact of initial geometric layout. Opt. Lett. 42, 3161–3164 (2017).

    CAS  Google Scholar 

  157. Yang, J., Sell, D. & Fan, J. A. Freeform metagratings based on complex light scattering dynamics for extreme, high efficiency beam steering. Ann. Phys. 530, 1700302 (2018).

    Google Scholar 

  158. Yang, J. & Fan, J. A. Analysis of material selection on dielectric metasurface performance. Opt. Express 25, 23899–23909 (2017).

    CAS  Google Scholar 

  159. Sell, D., Yang, J., Doshay, S. & Fan, J. A. Periodic dielectric metasurfaces with high-efficiency, multiwavelength functionalities. Adv. Opt. Mater. 5, 1700645 (2017).

    Google Scholar 

  160. Sell, D. et al. Ultra-high-efficiency anomalous refraction with dielectric metasurfaces. ACS Photonics 5, 2402–2407 (2018).

    CAS  Google Scholar 

  161. Wang, E. W., Sell, D., Phan, T. & Fan, J. A. Robust design of topology-optimized metasurfaces. Opt. Mater. Express 9, 469–482 (2019).

    Google Scholar 

  162. Wen, F., Jiang, J. & Fan, J. A. Progressive-growing of generative adversarial networks for metasurface optimization. ACS Photonics 7, 2098–2104 (2019).

    Google Scholar 

  163. Vercruysse, D., Sapra, N. V., Su, L., Trivedi, R. & Vučković, J. Analytical level set fabrication constraints for inverse design. Sci. Rep. 9, 8999 (2019).

    Google Scholar 

  164. Huang, J. et al. Implementation of on-chip multi-channel focusing wavelength demultiplexer with regularized digital metamaterials. Nanophotonics 9, 159–166 (2019).

    Google Scholar 

  165. Vercruysse, D., Sapra, N. V., Su, L. & Vuckovic, J. Dispersion engineering with photonic inverse design. IEEE J. Sel. Top. Quantum Electron. 26, 1–6 (2019).

    Google Scholar 

  166. Chen, M., Jiang, J. & Fan, J. Design space reparameterization enforces hard geometric constraints in inverse-designed nanophotonic devices. ACS Photonics 7, 3141–3151 (2020).

    CAS  Google Scholar 

  167. Jiang, J. & Fan, J. A. Multiobjective and categorical global optimization of photonic structures based on ResNet generative neural networks. Nanophotonics https://doi.org/10.1515/nanoph-2020-0407 (2020).

    Article  Google Scholar 

  168. Azunre, P. et al. Guaranteed global optimization of thin-film optical systems. New J. Phys. 21, 073050 (2019).

    CAS  Google Scholar 

  169. Griebel, T. G. M. Numerical integration using sparse grids. Numer. Algorithms 18, 209 (1998).

    Google Scholar 

  170. Schneider, P.-I. et al. Benchmarking five global optimization approaches for nano-optical shape optimization and parameter reconstruction. ACS Photonics 6, 2726–2733 (2019).

    CAS  Google Scholar 

  171. Elsawy, M. M. et al. Global optimization of metasurface designs using statistical learning methods. Sci. Rep. 9, 17918 (2019).

    Google Scholar 

  172. Raissi, M., Perdikaris, P. & Karniadakis, G. E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019).

    Google Scholar 

  173. Trivedi, R., Su, L., Lu, J., Schubert, M. F. & Vuckovic, J. Data-driven acceleration of photonic simulations. Sci. Rep. 9, 19728 (2019).

    CAS  Google Scholar 

  174. Chew, W. C., Tong, M. S. & Hu, B. Integral equation methods for electromagnetic and elastic waves. Synth. Lect. Comput. Electromagnetics 3, 1–241 (2008).

    Google Scholar 

  175. Mishchenko, M. I., Travis, L. D. & Mackowski, D. W. T-matrix computations of light scattering by nonspherical particles: A review. J. Quant. Spectrosc. Radiat. Transf. 55, 535–575 (1996).

    CAS  Google Scholar 

  176. Pan, S. J. & Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010).

    Google Scholar 

  177. Qu, Y., Jing, L., Shen, Y., Qiu, M. & Soljacic, M. Migrating knowledge between physical scenarios based on artificial neural networks. ACS Photonics 6, 1168–1174 (2019).

    CAS  Google Scholar 

  178. Andrychowicz, M. et al. Learning to learn by gradient descent by gradient descent. Adv. Neural Inf. Process. Syst. 29, 3981–3989 (2016).

    Google Scholar 

  179. Finn, C., Abbeel, P. & Levine, S. in Proceedings of the 34th International Conference on Machine Learning Vol. 70, 1126–1135 (JMLR, 2017).

  180. Deng, J. et al. in Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition 248–255 (2009).

  181. Krizhevsky, A. Learning Multiple Layers of Features from Tiny Images (Univ. Toronto, 2012).

  182. Jiang, J. et al. MetaNet: a new paradigm for data sharing in photonics research. Opt. Express 28, 13670–13681 (2020).

    CAS  Google Scholar 

  183. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016).

  184. Kullback, S. Information Theory and Statistics (Courier Corporation, 1997).

  185. Lin, J. Divergence measures based on the Shannon entropy. IEEE Trans. Inf. Theory 37, 145–151 (1991).

    Google Scholar 

  186. Kudyshev, Z. A., Kildishev, A. V., Shalaev, V. & Boltasseva, A. in 2019 International Applied Computational Electromagnetics Society Symposium (ACES) 1–2 (ACES, 2019).

  187. Chen, M., Jiang, J. & Fan, J. in 2020 Conference on Lasers and Electro-Optics (CLEO): QELS_Fundamental Science FW4B–4 (Optical Society of America, 2020).

Download references

Acknowledgements

The authors thank Q. Huang, L. Gan and K. Kanada for feedback on the manuscript. This work was supported by the Office of Naval Research under award number N00014-20-1-2105, ARPA-E under award number DE-AR0001212 and the David and Lucile Packard Foundation under award number 2016-65132.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Stanford University, Stanford, CA, USA

    Jiaqi Jiang, Mingkun Chen & Jonathan A. Fan

Authors
  1. Jiaqi Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingkun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonathan A. Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors researched data for the article, discussed the content and contributed to the writing and revising of the manuscript.

Corresponding author

Correspondence to Jonathan A. Fan.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, J., Chen, M. & Fan, J.A. Deep neural networks for the evaluation and design of photonic devices. Nat Rev Mater 6, 679–700 (2021). https://doi.org/10.1038/s41578-020-00260-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41578-020-00260-1

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics