Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Enabling real-time multi-messenger astrophysics discoveries with deep learning


Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos. In this Expert Recommendation, we review the key challenges of real-time observations of gravitational wave sources and their electromagnetic and astroparticle counterparts, and make a number of recommendations to maximize their potential for scientific discovery. These recommendations refer to the design of scalable and computationally efficient machine learning algorithms; the cyber-infrastructure to numerically simulate astrophysical sources, and to process and interpret multi-messenger astrophysics data; the management of gravitational wave detections to trigger real-time alerts for electromagnetic and astroparticle follow-ups; a vision to harness future developments of machine learning and cyber-infrastructure resources to cope with the big-data requirements; and the need to build a community of experts to realize the goals of multi-messenger astrophysics.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Fig. 1: Visualization of the results of a numerical relativity simulation of two neutron stars before and after a merger.


  1. Abbott, B. P. et al. GWTC-1: a gravitational-wave transient catalog of compact binary mergers observed by LIGO and Virgo during the first and second observing runs. Phys. Rev. X 9, 031040 (2019).

  2. Arnett, W. D., Bahcall, J. N., Kirshner, R. P. & Woosley, S. E. Supernova 1987A. Annu. Rev. Astron. Astrophys. 27, 629–700 (1989).

    ADS  Article  Google Scholar 

  3. Abbott, B. P. et al. GW170817: observation of gravitational waves from a binary neutron star inspiral. Phys. Rev. Lett. 119, 161101 (2017).

    ADS  Article  Google Scholar 

  4. Abbott, B. P. et al. Estimating the contribution of dynamical ejecta in the kilonova associated with GW170817. Astrophys. J. Lett. 850, L39 (2017).

    ADS  Article  Google Scholar 

  5. IceCube Collaboration Neutrino emission from the direction of the blazar TXS 0506+056 prior to the IceCube-170922A alert. Science 361, 147–151 (2018).

    ADS  Google Scholar 

  6. Large Synoptic Survey Telescope LSST system and survey key numbers. LSST (2018).

  7. Abell, P. A. et al. LSST Science Book, version 2.0. Preprint at (2009).

  8. Robertson, B. E. et al. Galaxy formation and evolution science in the era of the Large Synoptic Survey Telescope. Nat. Rev. Phys. 1, 450–462 (2019).

    Article  Google Scholar 

  9. Owen, B. J. & Sathyaprakash, B. S. Matched filtering of gravitational waves from inspiraling compact binaries: computational cost and template placement. Phys. Rev. D 60, 022002 (1999).

    ADS  Article  Google Scholar 

  10. Harry, I., Privitera, S., Bohé, A. & Buonanno, A. Searching for gravitational waves from compact binaries with precessing spins. Phys. Rev. D 94, 024012 (2016).

    ADS  Article  Google Scholar 

  11. Huerta, E. A. et al. Complete waveform model for compact binaries on eccentric orbits. Phys. Rev. D 95, 024038 (2017).

    ADS  Article  Google Scholar 

  12. Huerta, E. A. et al. BOSS-LDG: a novel computational framework that brings together blue waters, open science grid, shifter and the LIGO data grid to accelerate gravitational wave discovery. In 2017 IEEE 13th International Conference on e-Science 335–344 (IEEE, 2017).

  13. Huerta, E. A., Haas, R., Jha, S., Neubauer, M. & Katz, D. S. Supporting high-performance and high-throughput computing for experimental science. Comput. Softw. Big Sci. 3, 5 (2019).

    Article  Google Scholar 

  14. Weitzel, D. et al. Data access for LIGO on the OSG. In Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact 24, 1-6 (PEARC, 2017).

  15. Abbott, B. P. et al. Observing gravitational-wave transient GW150914 with minimal assumptions. Phys. Rev. D 93, 122004 (2016).

    ADS  Article  Google Scholar 

  16. Jones, P. W., Osipov, A. & Rokhlin, V. Randomized approximate nearest neighbors algorithm. Proc. Natl Acad. Sci. USA 108, 15679–15686 (2011).

    ADS  Article  Google Scholar 

  17. Liang, S., Liu, Y., Wang, C. & Jian, L. Design and evaluation of a parallel k-nearest neighbor algorithm on CUDA-enabled GPU. In 2010 IEEE 2nd Symposium on Web Society 53–60 (IEEE, 2010).

  18. Andre, J. C. et al. Big data and extreme-scale computing: pathways to convergence toward a shaping strategy for a future software and data ecosystem for scientific inquiry. Int. J. High Perform. Comput. Appl. 32, 435–479 (2018).

  19. Engineering National Academies of Sciences and Medicine Future Directions for NSF Advanced Computing Infrastructure Support. U.S. Science and Engineering in 2017–2020 (The National Academies Press, 2016).

  20. Metzger, B. D. & Berger, E. What is the most promising electromagnetic counterpart of a neutron star binary merger? Astrophys. J. 746, 48 (2012).

    ADS  Article  Google Scholar 

  21. Siegel, D. M. & Metzger, B. D. Three-dimensional grmhd simulations of neutrino-cooled accretion disks from neutron star mergers. Astrophys. J. 858, 52 (2018).

    ADS  Article  Google Scholar 

  22. Abbott, B. P. et al. Prospects for observing and localizing gravitational-wave transients with advanced LIGO and advanced Virgo. Living Rev. Relativ. 21, 3 (2018).

  23. Drout M. R. et al. Light curves of the neutron star merger GW170817/SSS17a: implications for r-process nucleosynthesis. Science 358, 1570–1574 (2017).

    ADS  Article  Google Scholar 

  24. Mooley, K. P. et al. A mildly relativistic wide-angle outflow in the neutron-star merger event GW170817. Nature 554, 207–210 (2018).

    ADS  Article  Google Scholar 

  25. Andreoni, I. et al. Mary, a pipeline to aid discovery of optical transients. Publ. Astron. Soc. Aust. 34, e037 (2017).

    ADS  Article  Google Scholar 

  26. Sedaghat, N. & Mahabal, A. Effective image differencing with convolutional neural networks for real-time transient hunting. Mon. Not. R. Astron. Soc. 476, 5365–5376 (2018).

    ADS  Article  Google Scholar 

  27. Jones, D. O. et al. Measuring dark energy properties with photometrically classified Pan-STARRS supernovae. II. Cosmological parameters. Astrophys. J. 857, 51 (2018).

    ADS  Article  Google Scholar 

  28. Kessler, R. et al. Results from the supernova photometric classification challenge. Publ. Astron. Soc. Pac. 122, 1415–4131 (2010).

    Article  Google Scholar 

  29. Scolnic, D. et al. How many kilonovae can be found in past, present, and future survey data sets? Astrophys. J. Lett. 852, L3 (2018).

    ADS  Article  Google Scholar 

  30. Setzer, C. N. et al. Serendipitous discoveries of kilonovae in the LSST main survey: maximising detections of sub-threshold gravitational wave events. Mon. Not. R. Astron. Soc. 485, 4260–4273 (2019).

    ADS  Article  Google Scholar 

  31. Schutz, B. F. Determining the Hubble constant from gravitational wave observations. Nature 323, 310–311 (1986).

    ADS  Article  Google Scholar 

  32. Soares-Santos, M. et al. First measurement of the Hubble constant from a dark standard siren using the Dark Energy Survey galaxies and the LIGO/Virgo binary–black-hole merger GW170814. Astrophys. J. 876, L7 (2019).

    ADS  Article  Google Scholar 

  33. Abbott, B. P. et al. A gravitational-wave standard siren measurement of the Hubble constant. Nature 551, 85–88 (2017).

    ADS  Article  Google Scholar 

  34. Cowperthwaite, P. S. et al. The electromagnetic counterpart of the binary neutron star merger LIGO/Virgo GW170817. II. UV, optical, and near-infrared light curves and comparison to kilonova models. Astrophys. J. Lett. 848, L17 (2017).

    ADS  Article  Google Scholar 

  35. Fishbach, M. et al. A standard siren measurement of the Hubble constant from GW170817 without the electromagnetic counterpart. Astrophys. J. Lett. 871, L13 (2019).

    ADS  Article  Google Scholar 

  36. D. Sánchez, H., Huertas-Company, M., Bernardi, M., Tuccillo, D. & Fischer, J. L. Improving galaxy morphologies for SDSS with deep learning. Mon. Not. R. Astron. Soc. 476, 3661–3676 (2018).

    ADS  Article  Google Scholar 

  37. Khan, A. et al. Deep learning at scale for the construction of galaxy catalogs in the Dark Energy Survey. Phys. Lett. B 795, 248–258 (2019).

    ADS  Article  Google Scholar 

  38. Eisenstein, D. J. et al. SDSS-III: massive spectroscopic surveys of the distant Universe, the Milky Way, and extra-solar planetary systems. Astron J. 142, 72 (2011).

  39. Dark Energy Survey Collaboration et al. The Dark Energy Survey: more than dark energy — an overview. Mon. Not. R. Astron. Soc. 460, 1270–1299 (2016).

    ADS  Article  Google Scholar 

  40. Riess, A. G., Casertano, S., Yuan, W., Macri, L. M. & Scolnic, D. Large Magellanic Cloud cepheid standards provide a 1% foundation for the determination of the Hubble constant and stronger evidence for physics beyond ΛCDM. Astrophys. J. 876, 85 (2019).

    ADS  Article  Google Scholar 

  41. Aghanim, N. et al. Planck 2018 results. VI. Cosmological parameters. Preprint at (2018).

  42. Freedman, W. L. Cosmology at a crossroads. Nat. Astron. 1, 0121 (2017).

    ADS  Article  Google Scholar 

  43. Poulin, V., Smith, T. L., Karwal, T. & Kamionkowski, M. Early dark energy can resolve the Hubble tension. Phys. Rev. Lett. 122, 221301 (2019).

    ADS  Article  Google Scholar 

  44. Lecun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    ADS  Article  Google Scholar 

  45. George, D. & Huerta, E. A. Deep neural networks to enable real-time multimessenger astrophysics. Phys. Rev. D 97, 044039 (2018).

    ADS  Article  Google Scholar 

  46. George, D. & Huerta, E. A. Deep Learning for real-time gravitational wave detection and parameter estimation: results with advanced LIGO data. Phys. Lett. B 778, 64–70 (2018).

    ADS  Article  Google Scholar 

  47. Rebei, A. et al. Fusing numerical relativity and deep learning to detect higher-order multipole waveforms from eccentric binary black hole mergers. Phys. Rev. D 100, 044025 (2019)

  48. George, D., Shen, H. & Huerta, E. A. Classification and unsupervised clustering of ligo data with deep transfer learning. Phys. Rev. D 97, 101501 (2018).

    ADS  Article  Google Scholar 

  49. Shen, H., George, D., Huerta, E. A. & Zhao, Z. Denoising gravitational waves with enhanced deep recurrent denoising auto-encoders. In 2019 IEEE International Conference on Acoustics, Speech and Signal Processing 3237–3241 (IEEE, 2019).

  50. Shen, H., George, D., Huerta, E. A. & Zhao, Z. Denoising gravitational waves using deep learning with recurrent denoising autoencoders. Preprint at (2017).

  51. Wei, W. & Huerta, E. A. Gravitational wave denoising of binary black hole mergers with deep learning. Preprint at (2019).

  52. Chua, A. J. K., Galley, C. R. & Vallisneri, M. ROMAN: Reduced-order modeling with artificial neurons. Phys. Rev. Lett. 122, 211101 (2019).

  53. Dreissigacker, C., Sharma, R., Messenger, C. & Prix, R. Deep-learning continuous gravitational waves. Phys. Rev. D 100, 044009 (2019).

  54. Gabbard, H., Williams, M., Hayes, F. & Messenger, C. Matching matched filtering with deep networks for gravitational-wave astronomy. Phys. Rev. Lett. 120, 141103 (2018).

    ADS  Article  Google Scholar 

  55. Nakano, H. et al. Comparison of various methods to extract ringdown frequency from gravitational wave data. Phys. Rev. D 99, 124032 (2019)

  56. Shen, H., Huerta, E. A. & Zhao, Z. Deep learning at scale for gravitational wave parameter estimation of binary black hole mergers. Preprint at (2019).

  57. Springenberg, J. T., Klein, A., Falkner, S. & Hutter, F. Bayesian optimization with robust Bayesian neural networks. In Advances in Neural Information Processing Systems 29 (NIPS 2016) (eds Lee, D. D., Sugiyama, M., Luxburg, U. V., Guyon, I. & Garnett, R.) 4134–4142 (Curran Associates, 2016).

  58. Burrows, A., Hayes, J. & Fryxell, B. A. On the nature of core-collapse supernova explosions. Astrophys. J. 450, 830 (1995).

    ADS  Article  Google Scholar 

  59. Burrows, A., Radice, D. & Vartanyan, D. Three-dimensional supernova explosion simulations of 9-, 10-, 11-, 12-, and 13- M stars. Mon. Not. R. Astron. Soc. 485, 3153–3168 (2019).

    ADS  Article  Google Scholar 

  60. Mösta, P. et al. r-process nucleosynthesis from three-dimensional magnetorotational core-collapse supernovae. Astrophys. J. 864, 171 (2018).

    ADS  Article  Google Scholar 

  61. Radice, D., Morozova, V., Burrows, A., Vartanyan, D. & Nagakura, H. Characterizing the gravitational wave signal from core-collapse supernovae. Astrophys. J. Lett. 876, L9 (2019).

    ADS  Article  Google Scholar 

  62. Janka, H.-T., Melson, T. & Summa, A. Physics of core-collapse supernovae in three dimensions: a sneak preview. Annu. Rev. Nucl. Part. Sci. 66, 341–375 (2016).

    ADS  Article  Google Scholar 

  63. Woosley, S. & Janka, T. The physics of core-collapse supernovae. Nat. Phys. 1, 147–154 (2005).

    Article  Google Scholar 

  64. Gossan, S. E. et al. Observing gravitational waves from core-collapse supernovae in the advanced detector era. Phys. Rev. D 93, 042002 (2016).

    ADS  Article  Google Scholar 

  65. Aurisano, A. et al. A convolutional neural network neutrino event classifier. J. Instrum. 11, P09001 (2016).

    Article  Google Scholar 

  66. Choma, N. et al. Graph neural networks for icecube signal classification. In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA) 386–391 (IEEE, 2018).

  67. Hinderer, T. et al. Discerning the binary neutron star or neutron star-black hole nature of GW170817 with gravitational wave and electromagnetic measurements. Preprint at (2018).

  68. Kim, B. et al. Deep fluids: a generative network for parameterized fluid simulations. Comput. Graph Forum 38, 59-70 (2019).

  69. Ling, J., Kurzawski, A. & Templeton, J. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. J. Fluid Mech. 807, 155–166 (2016).

    ADS  MathSciNet  Article  Google Scholar 

  70. Maulik, R., San, O., Rasheed, A. & Vedula, P. Subgrid modelling for two-dimensional turbulence using neural networks. J. Fluid Mech. 858, 122–144 (2019).

    ADS  MathSciNet  Article  Google Scholar 

  71. Viganò, D. & Palenzuela, C. Fitting of extended sub-grid scale models in compressible turbulent MHD. Preprint at (2019).

  72. Xie, C., Wang, J., Li, K. & Ma, C. Artificial neural network approach to large-eddy simulation of compressible isotropic turbulence. Phys. Rev. E 99, 053113 (2019).

    ADS  Article  Google Scholar 

  73. Berg, J. & Nyström, K. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing 317, 28–41 (2018).

    Article  Google Scholar 

  74. Weinan, E., Han, J. & Jentzen, A. Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun. Math. Stat. 5, 349–380 (2017).

    MathSciNet  Article  Google Scholar 

  75. Duez, M. D. & Zlochower, Y. Numerical relativity of compact binaries in the 21st century. Rep. Prog. Phys. 82, 016902 (2019).

    ADS  Article  Google Scholar 

  76. Baiotti, L. & Rezzolla, L. Binary neutron star mergers: a review of Einstein’s richest laboratory. Rep. Prog. Phys. 80, 096901 (2017).

    ADS  MathSciNet  Article  Google Scholar 

  77. Lippuner, J. & Roberts, L. F. Skynet: a modular nuclear reaction network library. Astrophys. J. Suppl. Ser. 233, 18 (2017).

    ADS  Article  Google Scholar 

  78. Paschalidis, V., Ruiz, M. & Shapiro, S. L. Relativistic simulations of black hole–neutron star coalescence: the jet emerges. Astrophys. J. 806, L14 (2015).

    ADS  Article  Google Scholar 

  79. Ruiz, M., Lang, R. N., Paschalidis, V. & Shapiro, S. L. Binary neutron star mergers: a jet engine for short gamma-ray bursts. Astrophys. J. 824, L6 (2016).

    ADS  Article  Google Scholar 

  80. Fernández, R. et al. Long-term GRMHD simulations of neutron star merger accretion disks: implications for electromagnetic counterparts. Mon. Not. R. Astron. Soc. 482, 3373 (2019).

  81. Nouri, F. H. et al. Evolution of the magnetized, neutrino-cooled accretion disk in the aftermath of a black hole–neutron star binary merger. Phys. Rev. D 97, 083014 (2018).

    ADS  Article  Google Scholar 

  82. Radice, D. et al. Binary neutron star mergers: mass ejection, electromagnetic counterparts, and nucleosynthesis. Astrophys. J. 869, 130 (2018).

    ADS  Article  Google Scholar 

  83. Kasen, D., Badnell, N. R. & Barnes, J. Opacities and spectra of the r-process ejecta from neutron star mergers. Astrophys. J. 774, 25 (2013).

    ADS  Article  Google Scholar 

  84. Berger, M. J. & Colella, P. Local adaptive mesh refinement for shock hydrodynamics. J. Comput. Phys. 82, 64–84 (1989).

    ADS  Article  Google Scholar 

  85. Chen, T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. Neural ordinary differential equations. Advances in Neural Information Processing Systems 31, 6571-6583 (2018).

  86. Radice, D. et al. Neutrino-driven convection in core-collapse supernovae: high-resolution simulations. Astrophys. J. 820, 76 (2016).

    ADS  Article  Google Scholar 

  87. Giacomazzo, B., Zrake, J., Duffell, P., MacFadyen, A. I. & Perna, R. Producing magnetar magnetic fields in the merger of binary neutron stars. Astrophys. J. 809, 39 (2015).

    ADS  Article  Google Scholar 

  88. EuroHPC Leading the way in the European supercomputing. EuroHPC (2018).

  89. Huerta, E. A. et al. BOSS-LDG: a novel computational framework that brings together blue waters, open science grid, shifter and the LIGO data grid to accelerate gravitational wave discovery. In 2017 IEEE 13th International Conference on e-Science 335–344 (IEEE, 2017).

  90. Arcavi, I. et al. Optical follow-up of gravitational-wave events with las cumbres observatory. Astrophys. J. 848, L33 (2017).

    ADS  Article  Google Scholar 

  91. Coughlin, M. W. et al. Optimizing searches for electromagnetic counterparts of gravitational wave triggers. Mon. Not. R. Astron. Soc. 478, 692–702 (2018).

    ADS  Article  Google Scholar 

  92. California Institute of Technology NED gravitational wave follow-up service. NED (2019).

  93. Pennsylvania State University Astrophysical multimessenger observatory network. AMON (2019).

  94. Cowperthwaite, P. S. et al. Astro 2020 Science White Paper: Joint Gravitational Wave and Electromagnetic Astronomy with LIGO and LSST in the 2020’s. Preprint at (2019).

  95. Marshall, P. et al. Science-Driven Optimization of the LSST Observing Strategy. Preprint at, 10.5281/zenodo.842713 (2017).

  96. A. Kinney et al. The W. M. Keck Observatory Scientific Strategic Plan. Keck Observers' Newsletter (2016).

  97. Narayan, G. et al. Machine learning-based brokers for real-time classification of the LSST alert stream. Astrophys. J. Suppl. 236, 9 (2018).

    ADS  Article  Google Scholar 

  98. Smith, K. W. et al. Lasair: the transient alert broker for LSST:UK. Res. Notes AAS 3, 26 (2019).

    ADS  Article  Google Scholar 

  99. AEON Team Astronomical Event Observatory Network NOAO (2018).

  100. The National Academies of Sciences, Engineering, and Medicine. The decadal survey on astronomy and astrophysics (astro2020). The National Academies of Sciences, Engineering, and Medicine (2019).

  101. Katz, D. S. et al. Community organizations: changing the culture in which research software is developed and sustained. Comp. Sci. Eng. 21, 8–24 (2019).

    Article  Google Scholar 

  102. Elmer, P., Neubauer, M. & Sokoloff, M. D. Strategic plan for a Scientific Software Innovation Institute (S2I2) for high energy physics. Preprint at (2017).

  103. Albrecht, J. et al. A roadmap for HEP software and computing R&D for the 2020s. Comput. Softw. Big Sci. 3, 7 (2019).

  104. Allen, G. et al. Multi-messenger astrophysics: harnessing the data revolution. Preprint at (2018).

  105. Hotokezaka, K., Beniamini, P. & Piran, T. Neutron star mergers as sites of r-process nucleosynthesis and short gamma-ray bursts. Int. J. Mod. Phys. D 27, 1842005 (2018).

    ADS  MathSciNet  Article  Google Scholar 

  106. Löffler, F. et al. The Einstein toolkit: a community computational infrastructure for relativistic astrophysics. Class. Quantum Gravity 29, 115001 (2012).

    ADS  Article  Google Scholar 

  107. Radice, D. & Rezzolla, L. THC: a new high-order finite-difference high-resolution shock-capturing code for special-relativistic hydrodynamics. Astron. Astrophys. 547, A26 (2012).

    ADS  Article  Google Scholar 

Download references


The authors gratefully acknowledge support from NVIDIA, Argonne Leadership Computing Facility, Oak Ridge Leadership Computing Facility, and the National Science Foundation through grant NSF-1848815. Artwork in this manuscript was supported in part by the National Science Foundation through grants ACI-1238993, NSF-1550514 and TG-PHY160053.

Author information

Authors and Affiliations



E.A.H. led and coordinated the writing of this Expert Recommendation. All authors contributed to developing the ideas, and writing and reviewing this manuscript. S.R. produced the artwork in figure 1.

Corresponding author

Correspondence to E. A. Huerta.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information

Nature Reviews Physics thanks Brant Robertson, Viviana Acquaviva and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Huerta, E.A., Allen, G., Andreoni, I. et al. Enabling real-time multi-messenger astrophysics discoveries with deep learning. Nat Rev Phys 1, 600–608 (2019).

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:

Further reading


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing