Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Enabling real-time multi-messenger astrophysics discoveries with deep learning


Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos. In this Expert Recommendation, we review the key challenges of real-time observations of gravitational wave sources and their electromagnetic and astroparticle counterparts, and make a number of recommendations to maximize their potential for scientific discovery. These recommendations refer to the design of scalable and computationally efficient machine learning algorithms; the cyber-infrastructure to numerically simulate astrophysical sources, and to process and interpret multi-messenger astrophysics data; the management of gravitational wave detections to trigger real-time alerts for electromagnetic and astroparticle follow-ups; a vision to harness future developments of machine learning and cyber-infrastructure resources to cope with the big-data requirements; and the need to build a community of experts to realize the goals of multi-messenger astrophysics.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it


Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Visualization of the results of a numerical relativity simulation of two neutron stars before and after a merger.


  1. Abbott, B. P. et al. GWTC-1: a gravitational-wave transient catalog of compact binary mergers observed by LIGO and Virgo during the first and second observing runs. Phys. Rev. X 9, 031040 (2019).

  2. Arnett, W. D., Bahcall, J. N., Kirshner, R. P. & Woosley, S. E. Supernova 1987A. Annu. Rev. Astron. Astrophys. 27, 629–700 (1989).

    Article  ADS  Google Scholar 

  3. Abbott, B. P. et al. GW170817: observation of gravitational waves from a binary neutron star inspiral. Phys. Rev. Lett. 119, 161101 (2017).

    Article  ADS  Google Scholar 

  4. Abbott, B. P. et al. Estimating the contribution of dynamical ejecta in the kilonova associated with GW170817. Astrophys. J. Lett. 850, L39 (2017).

    Article  ADS  Google Scholar 

  5. IceCube Collaboration Neutrino emission from the direction of the blazar TXS 0506+056 prior to the IceCube-170922A alert. Science 361, 147–151 (2018).

    ADS  Google Scholar 

  6. Large Synoptic Survey Telescope LSST system and survey key numbers. LSST (2018).

  7. Abell, P. A. et al. LSST Science Book, version 2.0. Preprint at (2009).

  8. Robertson, B. E. et al. Galaxy formation and evolution science in the era of the Large Synoptic Survey Telescope. Nat. Rev. Phys. 1, 450–462 (2019).

    Article  Google Scholar 

  9. Owen, B. J. & Sathyaprakash, B. S. Matched filtering of gravitational waves from inspiraling compact binaries: computational cost and template placement. Phys. Rev. D 60, 022002 (1999).

    Article  ADS  Google Scholar 

  10. Harry, I., Privitera, S., Bohé, A. & Buonanno, A. Searching for gravitational waves from compact binaries with precessing spins. Phys. Rev. D 94, 024012 (2016).

    Article  ADS  Google Scholar 

  11. Huerta, E. A. et al. Complete waveform model for compact binaries on eccentric orbits. Phys. Rev. D 95, 024038 (2017).

    Article  ADS  Google Scholar 

  12. Huerta, E. A. et al. BOSS-LDG: a novel computational framework that brings together blue waters, open science grid, shifter and the LIGO data grid to accelerate gravitational wave discovery. In 2017 IEEE 13th International Conference on e-Science 335–344 (IEEE, 2017).

  13. Huerta, E. A., Haas, R., Jha, S., Neubauer, M. & Katz, D. S. Supporting high-performance and high-throughput computing for experimental science. Comput. Softw. Big Sci. 3, 5 (2019).

    Article  Google Scholar 

  14. Weitzel, D. et al. Data access for LIGO on the OSG. In Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact 24, 1-6 (PEARC, 2017).

  15. Abbott, B. P. et al. Observing gravitational-wave transient GW150914 with minimal assumptions. Phys. Rev. D 93, 122004 (2016).

    Article  ADS  Google Scholar 

  16. Jones, P. W., Osipov, A. & Rokhlin, V. Randomized approximate nearest neighbors algorithm. Proc. Natl Acad. Sci. USA 108, 15679–15686 (2011).

    Article  ADS  Google Scholar 

  17. Liang, S., Liu, Y., Wang, C. & Jian, L. Design and evaluation of a parallel k-nearest neighbor algorithm on CUDA-enabled GPU. In 2010 IEEE 2nd Symposium on Web Society 53–60 (IEEE, 2010).

  18. Andre, J. C. et al. Big data and extreme-scale computing: pathways to convergence toward a shaping strategy for a future software and data ecosystem for scientific inquiry. Int. J. High Perform. Comput. Appl. 32, 435–479 (2018).

  19. Engineering National Academies of Sciences and Medicine Future Directions for NSF Advanced Computing Infrastructure Support. U.S. Science and Engineering in 2017–2020 (The National Academies Press, 2016).

  20. Metzger, B. D. & Berger, E. What is the most promising electromagnetic counterpart of a neutron star binary merger? Astrophys. J. 746, 48 (2012).

    Article  ADS  Google Scholar 

  21. Siegel, D. M. & Metzger, B. D. Three-dimensional grmhd simulations of neutrino-cooled accretion disks from neutron star mergers. Astrophys. J. 858, 52 (2018).

    Article  ADS  Google Scholar 

  22. Abbott, B. P. et al. Prospects for observing and localizing gravitational-wave transients with advanced LIGO and advanced Virgo. Living Rev. Relativ. 21, 3 (2018).

  23. Drout M. R. et al. Light curves of the neutron star merger GW170817/SSS17a: implications for r-process nucleosynthesis. Science 358, 1570–1574 (2017).

    Article  ADS  Google Scholar 

  24. Mooley, K. P. et al. A mildly relativistic wide-angle outflow in the neutron-star merger event GW170817. Nature 554, 207–210 (2018).

    Article  ADS  Google Scholar 

  25. Andreoni, I. et al. Mary, a pipeline to aid discovery of optical transients. Publ. Astron. Soc. Aust. 34, e037 (2017).

    Article  ADS  Google Scholar 

  26. Sedaghat, N. & Mahabal, A. Effective image differencing with convolutional neural networks for real-time transient hunting. Mon. Not. R. Astron. Soc. 476, 5365–5376 (2018).

    Article  ADS  Google Scholar 

  27. Jones, D. O. et al. Measuring dark energy properties with photometrically classified Pan-STARRS supernovae. II. Cosmological parameters. Astrophys. J. 857, 51 (2018).

    Article  ADS  Google Scholar 

  28. Kessler, R. et al. Results from the supernova photometric classification challenge. Publ. Astron. Soc. Pac. 122, 1415–4131 (2010).

    Article  Google Scholar 

  29. Scolnic, D. et al. How many kilonovae can be found in past, present, and future survey data sets? Astrophys. J. Lett. 852, L3 (2018).

    Article  ADS  Google Scholar 

  30. Setzer, C. N. et al. Serendipitous discoveries of kilonovae in the LSST main survey: maximising detections of sub-threshold gravitational wave events. Mon. Not. R. Astron. Soc. 485, 4260–4273 (2019).

    Article  ADS  Google Scholar 

  31. Schutz, B. F. Determining the Hubble constant from gravitational wave observations. Nature 323, 310–311 (1986).

    Article  ADS  Google Scholar 

  32. Soares-Santos, M. et al. First measurement of the Hubble constant from a dark standard siren using the Dark Energy Survey galaxies and the LIGO/Virgo binary–black-hole merger GW170814. Astrophys. J. 876, L7 (2019).

    Article  ADS  Google Scholar 

  33. Abbott, B. P. et al. A gravitational-wave standard siren measurement of the Hubble constant. Nature 551, 85–88 (2017).

    Article  ADS  Google Scholar 

  34. Cowperthwaite, P. S. et al. The electromagnetic counterpart of the binary neutron star merger LIGO/Virgo GW170817. II. UV, optical, and near-infrared light curves and comparison to kilonova models. Astrophys. J. Lett. 848, L17 (2017).

    Article  ADS  Google Scholar 

  35. Fishbach, M. et al. A standard siren measurement of the Hubble constant from GW170817 without the electromagnetic counterpart. Astrophys. J. Lett. 871, L13 (2019).

    Article  ADS  Google Scholar 

  36. D. Sánchez, H., Huertas-Company, M., Bernardi, M., Tuccillo, D. & Fischer, J. L. Improving galaxy morphologies for SDSS with deep learning. Mon. Not. R. Astron. Soc. 476, 3661–3676 (2018).

    Article  ADS  Google Scholar 

  37. Khan, A. et al. Deep learning at scale for the construction of galaxy catalogs in the Dark Energy Survey. Phys. Lett. B 795, 248–258 (2019).

    Article  ADS  Google Scholar 

  38. Eisenstein, D. J. et al. SDSS-III: massive spectroscopic surveys of the distant Universe, the Milky Way, and extra-solar planetary systems. Astron J. 142, 72 (2011).

  39. Dark Energy Survey Collaboration et al. The Dark Energy Survey: more than dark energy — an overview. Mon. Not. R. Astron. Soc. 460, 1270–1299 (2016).

    Article  ADS  Google Scholar 

  40. Riess, A. G., Casertano, S., Yuan, W., Macri, L. M. & Scolnic, D. Large Magellanic Cloud cepheid standards provide a 1% foundation for the determination of the Hubble constant and stronger evidence for physics beyond ΛCDM. Astrophys. J. 876, 85 (2019).

    Article  ADS  Google Scholar 

  41. Aghanim, N. et al. Planck 2018 results. VI. Cosmological parameters. Preprint at (2018).

  42. Freedman, W. L. Cosmology at a crossroads. Nat. Astron. 1, 0121 (2017).

    Article  ADS  Google Scholar 

  43. Poulin, V., Smith, T. L., Karwal, T. & Kamionkowski, M. Early dark energy can resolve the Hubble tension. Phys. Rev. Lett. 122, 221301 (2019).

    Article  ADS  Google Scholar 

  44. Lecun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  ADS  Google Scholar 

  45. George, D. & Huerta, E. A. Deep neural networks to enable real-time multimessenger astrophysics. Phys. Rev. D 97, 044039 (2018).

    Article  ADS  Google Scholar 

  46. George, D. & Huerta, E. A. Deep Learning for real-time gravitational wave detection and parameter estimation: results with advanced LIGO data. Phys. Lett. B 778, 64–70 (2018).

    Article  ADS  Google Scholar 

  47. Rebei, A. et al. Fusing numerical relativity and deep learning to detect higher-order multipole waveforms from eccentric binary black hole mergers. Phys. Rev. D 100, 044025 (2019)

  48. George, D., Shen, H. & Huerta, E. A. Classification and unsupervised clustering of ligo data with deep transfer learning. Phys. Rev. D 97, 101501 (2018).

    Article  ADS  Google Scholar 

  49. Shen, H., George, D., Huerta, E. A. & Zhao, Z. Denoising gravitational waves with enhanced deep recurrent denoising auto-encoders. In 2019 IEEE International Conference on Acoustics, Speech and Signal Processing 3237–3241 (IEEE, 2019).

  50. Shen, H., George, D., Huerta, E. A. & Zhao, Z. Denoising gravitational waves using deep learning with recurrent denoising autoencoders. Preprint at (2017).

  51. Wei, W. & Huerta, E. A. Gravitational wave denoising of binary black hole mergers with deep learning. Preprint at (2019).

  52. Chua, A. J. K., Galley, C. R. & Vallisneri, M. ROMAN: Reduced-order modeling with artificial neurons. Phys. Rev. Lett. 122, 211101 (2019).

  53. Dreissigacker, C., Sharma, R., Messenger, C. & Prix, R. Deep-learning continuous gravitational waves. Phys. Rev. D 100, 044009 (2019).

  54. Gabbard, H., Williams, M., Hayes, F. & Messenger, C. Matching matched filtering with deep networks for gravitational-wave astronomy. Phys. Rev. Lett. 120, 141103 (2018).

    Article  ADS  Google Scholar 

  55. Nakano, H. et al. Comparison of various methods to extract ringdown frequency from gravitational wave data. Phys. Rev. D 99, 124032 (2019)

  56. Shen, H., Huerta, E. A. & Zhao, Z. Deep learning at scale for gravitational wave parameter estimation of binary black hole mergers. Preprint at (2019).

  57. Springenberg, J. T., Klein, A., Falkner, S. & Hutter, F. Bayesian optimization with robust Bayesian neural networks. In Advances in Neural Information Processing Systems 29 (NIPS 2016) (eds Lee, D. D., Sugiyama, M., Luxburg, U. V., Guyon, I. & Garnett, R.) 4134–4142 (Curran Associates, 2016).

  58. Burrows, A., Hayes, J. & Fryxell, B. A. On the nature of core-collapse supernova explosions. Astrophys. J. 450, 830 (1995).

    Article  ADS  Google Scholar 

  59. Burrows, A., Radice, D. & Vartanyan, D. Three-dimensional supernova explosion simulations of 9-, 10-, 11-, 12-, and 13- M stars. Mon. Not. R. Astron. Soc. 485, 3153–3168 (2019).

    Article  ADS  Google Scholar 

  60. Mösta, P. et al. r-process nucleosynthesis from three-dimensional magnetorotational core-collapse supernovae. Astrophys. J. 864, 171 (2018).

    Article  ADS  Google Scholar 

  61. Radice, D., Morozova, V., Burrows, A., Vartanyan, D. & Nagakura, H. Characterizing the gravitational wave signal from core-collapse supernovae. Astrophys. J. Lett. 876, L9 (2019).

    Article  ADS  Google Scholar 

  62. Janka, H.-T., Melson, T. & Summa, A. Physics of core-collapse supernovae in three dimensions: a sneak preview. Annu. Rev. Nucl. Part. Sci. 66, 341–375 (2016).

    Article  ADS  Google Scholar 

  63. Woosley, S. & Janka, T. The physics of core-collapse supernovae. Nat. Phys. 1, 147–154 (2005).

    Article  Google Scholar 

  64. Gossan, S. E. et al. Observing gravitational waves from core-collapse supernovae in the advanced detector era. Phys. Rev. D 93, 042002 (2016).

    Article  ADS  Google Scholar 

  65. Aurisano, A. et al. A convolutional neural network neutrino event classifier. J. Instrum. 11, P09001 (2016).

    Article  Google Scholar 

  66. Choma, N. et al. Graph neural networks for icecube signal classification. In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA) 386–391 (IEEE, 2018).

  67. Hinderer, T. et al. Discerning the binary neutron star or neutron star-black hole nature of GW170817 with gravitational wave and electromagnetic measurements. Preprint at (2018).

  68. Kim, B. et al. Deep fluids: a generative network for parameterized fluid simulations. Comput. Graph Forum 38, 59-70 (2019).

  69. Ling, J., Kurzawski, A. & Templeton, J. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. J. Fluid Mech. 807, 155–166 (2016).

    Article  ADS  MathSciNet  Google Scholar 

  70. Maulik, R., San, O., Rasheed, A. & Vedula, P. Subgrid modelling for two-dimensional turbulence using neural networks. J. Fluid Mech. 858, 122–144 (2019).

    Article  ADS  MathSciNet  Google Scholar 

  71. Viganò, D. & Palenzuela, C. Fitting of extended sub-grid scale models in compressible turbulent MHD. Preprint at (2019).

  72. Xie, C., Wang, J., Li, K. & Ma, C. Artificial neural network approach to large-eddy simulation of compressible isotropic turbulence. Phys. Rev. E 99, 053113 (2019).

    Article  ADS  Google Scholar 

  73. Berg, J. & Nyström, K. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing 317, 28–41 (2018).

    Article  Google Scholar 

  74. Weinan, E., Han, J. & Jentzen, A. Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun. Math. Stat. 5, 349–380 (2017).

    Article  MathSciNet  Google Scholar 

  75. Duez, M. D. & Zlochower, Y. Numerical relativity of compact binaries in the 21st century. Rep. Prog. Phys. 82, 016902 (2019).

    Article  ADS  Google Scholar 

  76. Baiotti, L. & Rezzolla, L. Binary neutron star mergers: a review of Einstein’s richest laboratory. Rep. Prog. Phys. 80, 096901 (2017).

    Article  ADS  MathSciNet  Google Scholar 

  77. Lippuner, J. & Roberts, L. F. Skynet: a modular nuclear reaction network library. Astrophys. J. Suppl. Ser. 233, 18 (2017).

    Article  ADS  Google Scholar 

  78. Paschalidis, V., Ruiz, M. & Shapiro, S. L. Relativistic simulations of black hole–neutron star coalescence: the jet emerges. Astrophys. J. 806, L14 (2015).

    Article  ADS  Google Scholar 

  79. Ruiz, M., Lang, R. N., Paschalidis, V. & Shapiro, S. L. Binary neutron star mergers: a jet engine for short gamma-ray bursts. Astrophys. J. 824, L6 (2016).

    Article  ADS  Google Scholar 

  80. Fernández, R. et al. Long-term GRMHD simulations of neutron star merger accretion disks: implications for electromagnetic counterparts. Mon. Not. R. Astron. Soc. 482, 3373 (2019).

  81. Nouri, F. H. et al. Evolution of the magnetized, neutrino-cooled accretion disk in the aftermath of a black hole–neutron star binary merger. Phys. Rev. D 97, 083014 (2018).

    Article  ADS  Google Scholar 

  82. Radice, D. et al. Binary neutron star mergers: mass ejection, electromagnetic counterparts, and nucleosynthesis. Astrophys. J. 869, 130 (2018).

    Article  ADS  Google Scholar 

  83. Kasen, D., Badnell, N. R. & Barnes, J. Opacities and spectra of the r-process ejecta from neutron star mergers. Astrophys. J. 774, 25 (2013).

    Article  ADS  Google Scholar 

  84. Berger, M. J. & Colella, P. Local adaptive mesh refinement for shock hydrodynamics. J. Comput. Phys. 82, 64–84 (1989).

    Article  ADS  Google Scholar 

  85. Chen, T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. Neural ordinary differential equations. Advances in Neural Information Processing Systems 31, 6571-6583 (2018).

  86. Radice, D. et al. Neutrino-driven convection in core-collapse supernovae: high-resolution simulations. Astrophys. J. 820, 76 (2016).

    Article  ADS  Google Scholar 

  87. Giacomazzo, B., Zrake, J., Duffell, P., MacFadyen, A. I. & Perna, R. Producing magnetar magnetic fields in the merger of binary neutron stars. Astrophys. J. 809, 39 (2015).

    Article  ADS  Google Scholar 

  88. EuroHPC Leading the way in the European supercomputing. EuroHPC (2018).

  89. Huerta, E. A. et al. BOSS-LDG: a novel computational framework that brings together blue waters, open science grid, shifter and the LIGO data grid to accelerate gravitational wave discovery. In 2017 IEEE 13th International Conference on e-Science 335–344 (IEEE, 2017).

  90. Arcavi, I. et al. Optical follow-up of gravitational-wave events with las cumbres observatory. Astrophys. J. 848, L33 (2017).

    Article  ADS  Google Scholar 

  91. Coughlin, M. W. et al. Optimizing searches for electromagnetic counterparts of gravitational wave triggers. Mon. Not. R. Astron. Soc. 478, 692–702 (2018).

    Article  ADS  Google Scholar 

  92. California Institute of Technology NED gravitational wave follow-up service. NED (2019).

  93. Pennsylvania State University Astrophysical multimessenger observatory network. AMON (2019).

  94. Cowperthwaite, P. S. et al. Astro 2020 Science White Paper: Joint Gravitational Wave and Electromagnetic Astronomy with LIGO and LSST in the 2020’s. Preprint at (2019).

  95. Marshall, P. et al. Science-Driven Optimization of the LSST Observing Strategy. Preprint at, 10.5281/zenodo.842713 (2017).

  96. A. Kinney et al. The W. M. Keck Observatory Scientific Strategic Plan. Keck Observers' Newsletter (2016).

  97. Narayan, G. et al. Machine learning-based brokers for real-time classification of the LSST alert stream. Astrophys. J. Suppl. 236, 9 (2018).

    Article  ADS  Google Scholar 

  98. Smith, K. W. et al. Lasair: the transient alert broker for LSST:UK. Res. Notes AAS 3, 26 (2019).

    Article  ADS  Google Scholar 

  99. AEON Team Astronomical Event Observatory Network NOAO (2018).

  100. The National Academies of Sciences, Engineering, and Medicine. The decadal survey on astronomy and astrophysics (astro2020). The National Academies of Sciences, Engineering, and Medicine (2019).

  101. Katz, D. S. et al. Community organizations: changing the culture in which research software is developed and sustained. Comp. Sci. Eng. 21, 8–24 (2019).

    Article  Google Scholar 

  102. Elmer, P., Neubauer, M. & Sokoloff, M. D. Strategic plan for a Scientific Software Innovation Institute (S2I2) for high energy physics. Preprint at (2017).

  103. Albrecht, J. et al. A roadmap for HEP software and computing R&D for the 2020s. Comput. Softw. Big Sci. 3, 7 (2019).

  104. Allen, G. et al. Multi-messenger astrophysics: harnessing the data revolution. Preprint at (2018).

  105. Hotokezaka, K., Beniamini, P. & Piran, T. Neutron star mergers as sites of r-process nucleosynthesis and short gamma-ray bursts. Int. J. Mod. Phys. D 27, 1842005 (2018).

    Article  ADS  MathSciNet  Google Scholar 

  106. Löffler, F. et al. The Einstein toolkit: a community computational infrastructure for relativistic astrophysics. Class. Quantum Gravity 29, 115001 (2012).

    Article  ADS  Google Scholar 

  107. Radice, D. & Rezzolla, L. THC: a new high-order finite-difference high-resolution shock-capturing code for special-relativistic hydrodynamics. Astron. Astrophys. 547, A26 (2012).

    Article  ADS  Google Scholar 

Download references


The authors gratefully acknowledge support from NVIDIA, Argonne Leadership Computing Facility, Oak Ridge Leadership Computing Facility, and the National Science Foundation through grant NSF-1848815. Artwork in this manuscript was supported in part by the National Science Foundation through grants ACI-1238993, NSF-1550514 and TG-PHY160053.

Author information

Authors and Affiliations



E.A.H. led and coordinated the writing of this Expert Recommendation. All authors contributed to developing the ideas, and writing and reviewing this manuscript. S.R. produced the artwork in figure 1.

Corresponding author

Correspondence to E. A. Huerta.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information

Nature Reviews Physics thanks Brant Robertson, Viviana Acquaviva and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huerta, E.A., Allen, G., Andreoni, I. et al. Enabling real-time multi-messenger astrophysics discoveries with deep learning. Nat Rev Phys 1, 600–608 (2019).

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:

This article is cited by


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing