Abstract
Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos. In this Expert Recommendation, we review the key challenges of real-time observations of gravitational wave sources and their electromagnetic and astroparticle counterparts, and make a number of recommendations to maximize their potential for scientific discovery. These recommendations refer to the design of scalable and computationally efficient machine learning algorithms; the cyber-infrastructure to numerically simulate astrophysical sources, and to process and interpret multi-messenger astrophysics data; the management of gravitational wave detections to trigger real-time alerts for electromagnetic and astroparticle follow-ups; a vision to harness future developments of machine learning and cyber-infrastructure resources to cope with the big-data requirements; and the need to build a community of experts to realize the goals of multi-messenger astrophysics.
This is a preview of subscription content, access via your institution
Relevant articles
Open Access articles citing this article.
-
FAIR principles for AI models with a practical application for accelerated high energy diffraction microscopy
Scientific Data Open Access 10 November 2022
-
A FAIR and AI-ready Higgs boson decay dataset
Scientific Data Open Access 14 February 2022
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$99.00 per year
only $8.25 per issue
Rent or buy this article
Get just this article for as long as you need it
$39.95
Prices may be subject to local taxes which are calculated during checkout
References
Abbott, B. P. et al. GWTC-1: a gravitational-wave transient catalog of compact binary mergers observed by LIGO and Virgo during the first and second observing runs. Phys. Rev. X 9, 031040 (2019).
Arnett, W. D., Bahcall, J. N., Kirshner, R. P. & Woosley, S. E. Supernova 1987A. Annu. Rev. Astron. Astrophys. 27, 629–700 (1989).
Abbott, B. P. et al. GW170817: observation of gravitational waves from a binary neutron star inspiral. Phys. Rev. Lett. 119, 161101 (2017).
Abbott, B. P. et al. Estimating the contribution of dynamical ejecta in the kilonova associated with GW170817. Astrophys. J. Lett. 850, L39 (2017).
IceCube Collaboration Neutrino emission from the direction of the blazar TXS 0506+056 prior to the IceCube-170922A alert. Science 361, 147–151 (2018).
Large Synoptic Survey Telescope LSST system and survey key numbers. LSST https://www.lsst.org/scientists/keynumbers (2018).
Abell, P. A. et al. LSST Science Book, version 2.0. Preprint at https://arxiv.org/abs/0912.0201 (2009).
Robertson, B. E. et al. Galaxy formation and evolution science in the era of the Large Synoptic Survey Telescope. Nat. Rev. Phys. 1, 450–462 (2019).
Owen, B. J. & Sathyaprakash, B. S. Matched filtering of gravitational waves from inspiraling compact binaries: computational cost and template placement. Phys. Rev. D 60, 022002 (1999).
Harry, I., Privitera, S., Bohé, A. & Buonanno, A. Searching for gravitational waves from compact binaries with precessing spins. Phys. Rev. D 94, 024012 (2016).
Huerta, E. A. et al. Complete waveform model for compact binaries on eccentric orbits. Phys. Rev. D 95, 024038 (2017).
Huerta, E. A. et al. BOSS-LDG: a novel computational framework that brings together blue waters, open science grid, shifter and the LIGO data grid to accelerate gravitational wave discovery. In 2017 IEEE 13th International Conference on e-Science 335–344 (IEEE, 2017).
Huerta, E. A., Haas, R., Jha, S., Neubauer, M. & Katz, D. S. Supporting high-performance and high-throughput computing for experimental science. Comput. Softw. Big Sci. 3, 5 (2019).
Weitzel, D. et al. Data access for LIGO on the OSG. In Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact 24, 1-6 (PEARC, 2017).
Abbott, B. P. et al. Observing gravitational-wave transient GW150914 with minimal assumptions. Phys. Rev. D 93, 122004 (2016).
Jones, P. W., Osipov, A. & Rokhlin, V. Randomized approximate nearest neighbors algorithm. Proc. Natl Acad. Sci. USA 108, 15679–15686 (2011).
Liang, S., Liu, Y., Wang, C. & Jian, L. Design and evaluation of a parallel k-nearest neighbor algorithm on CUDA-enabled GPU. In 2010 IEEE 2nd Symposium on Web Society 53–60 (IEEE, 2010).
Andre, J. C. et al. Big data and extreme-scale computing: pathways to convergence toward a shaping strategy for a future software and data ecosystem for scientific inquiry. Int. J. High Perform. Comput. Appl. 32, 435–479 (2018).
Engineering National Academies of Sciences and Medicine Future Directions for NSF Advanced Computing Infrastructure Support. U.S. Science and Engineering in 2017–2020 (The National Academies Press, 2016).
Metzger, B. D. & Berger, E. What is the most promising electromagnetic counterpart of a neutron star binary merger? Astrophys. J. 746, 48 (2012).
Siegel, D. M. & Metzger, B. D. Three-dimensional grmhd simulations of neutrino-cooled accretion disks from neutron star mergers. Astrophys. J. 858, 52 (2018).
Abbott, B. P. et al. Prospects for observing and localizing gravitational-wave transients with advanced LIGO and advanced Virgo. Living Rev. Relativ. 21, 3 (2018).
Drout M. R. et al. Light curves of the neutron star merger GW170817/SSS17a: implications for r-process nucleosynthesis. Science 358, 1570–1574 (2017).
Mooley, K. P. et al. A mildly relativistic wide-angle outflow in the neutron-star merger event GW170817. Nature 554, 207–210 (2018).
Andreoni, I. et al. Mary, a pipeline to aid discovery of optical transients. Publ. Astron. Soc. Aust. 34, e037 (2017).
Sedaghat, N. & Mahabal, A. Effective image differencing with convolutional neural networks for real-time transient hunting. Mon. Not. R. Astron. Soc. 476, 5365–5376 (2018).
Jones, D. O. et al. Measuring dark energy properties with photometrically classified Pan-STARRS supernovae. II. Cosmological parameters. Astrophys. J. 857, 51 (2018).
Kessler, R. et al. Results from the supernova photometric classification challenge. Publ. Astron. Soc. Pac. 122, 1415–4131 (2010).
Scolnic, D. et al. How many kilonovae can be found in past, present, and future survey data sets? Astrophys. J. Lett. 852, L3 (2018).
Setzer, C. N. et al. Serendipitous discoveries of kilonovae in the LSST main survey: maximising detections of sub-threshold gravitational wave events. Mon. Not. R. Astron. Soc. 485, 4260–4273 (2019).
Schutz, B. F. Determining the Hubble constant from gravitational wave observations. Nature 323, 310–311 (1986).
Soares-Santos, M. et al. First measurement of the Hubble constant from a dark standard siren using the Dark Energy Survey galaxies and the LIGO/Virgo binary–black-hole merger GW170814. Astrophys. J. 876, L7 (2019).
Abbott, B. P. et al. A gravitational-wave standard siren measurement of the Hubble constant. Nature 551, 85–88 (2017).
Cowperthwaite, P. S. et al. The electromagnetic counterpart of the binary neutron star merger LIGO/Virgo GW170817. II. UV, optical, and near-infrared light curves and comparison to kilonova models. Astrophys. J. Lett. 848, L17 (2017).
Fishbach, M. et al. A standard siren measurement of the Hubble constant from GW170817 without the electromagnetic counterpart. Astrophys. J. Lett. 871, L13 (2019).
D. Sánchez, H., Huertas-Company, M., Bernardi, M., Tuccillo, D. & Fischer, J. L. Improving galaxy morphologies for SDSS with deep learning. Mon. Not. R. Astron. Soc. 476, 3661–3676 (2018).
Khan, A. et al. Deep learning at scale for the construction of galaxy catalogs in the Dark Energy Survey. Phys. Lett. B 795, 248–258 (2019).
Eisenstein, D. J. et al. SDSS-III: massive spectroscopic surveys of the distant Universe, the Milky Way, and extra-solar planetary systems. Astron J. 142, 72 (2011).
Dark Energy Survey Collaboration et al. The Dark Energy Survey: more than dark energy — an overview. Mon. Not. R. Astron. Soc. 460, 1270–1299 (2016).
Riess, A. G., Casertano, S., Yuan, W., Macri, L. M. & Scolnic, D. Large Magellanic Cloud cepheid standards provide a 1% foundation for the determination of the Hubble constant and stronger evidence for physics beyond ΛCDM. Astrophys. J. 876, 85 (2019).
Aghanim, N. et al. Planck 2018 results. VI. Cosmological parameters. Preprint at https://arxiv.org/abs/1807.06209 (2018).
Freedman, W. L. Cosmology at a crossroads. Nat. Astron. 1, 0121 (2017).
Poulin, V., Smith, T. L., Karwal, T. & Kamionkowski, M. Early dark energy can resolve the Hubble tension. Phys. Rev. Lett. 122, 221301 (2019).
Lecun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
George, D. & Huerta, E. A. Deep neural networks to enable real-time multimessenger astrophysics. Phys. Rev. D 97, 044039 (2018).
George, D. & Huerta, E. A. Deep Learning for real-time gravitational wave detection and parameter estimation: results with advanced LIGO data. Phys. Lett. B 778, 64–70 (2018).
Rebei, A. et al. Fusing numerical relativity and deep learning to detect higher-order multipole waveforms from eccentric binary black hole mergers. Phys. Rev. D 100, 044025 (2019)
George, D., Shen, H. & Huerta, E. A. Classification and unsupervised clustering of ligo data with deep transfer learning. Phys. Rev. D 97, 101501 (2018).
Shen, H., George, D., Huerta, E. A. & Zhao, Z. Denoising gravitational waves with enhanced deep recurrent denoising auto-encoders. In 2019 IEEE International Conference on Acoustics, Speech and Signal Processing 3237–3241 (IEEE, 2019).
Shen, H., George, D., Huerta, E. A. & Zhao, Z. Denoising gravitational waves using deep learning with recurrent denoising autoencoders. Preprint at https://arxiv.org/abs/1711.09919 (2017).
Wei, W. & Huerta, E. A. Gravitational wave denoising of binary black hole mergers with deep learning. Preprint at https://arxiv.org/abs/1901.00869 (2019).
Chua, A. J. K., Galley, C. R. & Vallisneri, M. ROMAN: Reduced-order modeling with artificial neurons. Phys. Rev. Lett. 122, 211101 (2019).
Dreissigacker, C., Sharma, R., Messenger, C. & Prix, R. Deep-learning continuous gravitational waves. Phys. Rev. D 100, 044009 (2019).
Gabbard, H., Williams, M., Hayes, F. & Messenger, C. Matching matched filtering with deep networks for gravitational-wave astronomy. Phys. Rev. Lett. 120, 141103 (2018).
Nakano, H. et al. Comparison of various methods to extract ringdown frequency from gravitational wave data. Phys. Rev. D 99, 124032 (2019)
Shen, H., Huerta, E. A. & Zhao, Z. Deep learning at scale for gravitational wave parameter estimation of binary black hole mergers. Preprint at https://arxiv.org/abs/1903.01998 (2019).
Springenberg, J. T., Klein, A., Falkner, S. & Hutter, F. Bayesian optimization with robust Bayesian neural networks. In Advances in Neural Information Processing Systems 29 (NIPS 2016) (eds Lee, D. D., Sugiyama, M., Luxburg, U. V., Guyon, I. & Garnett, R.) 4134–4142 (Curran Associates, 2016).
Burrows, A., Hayes, J. & Fryxell, B. A. On the nature of core-collapse supernova explosions. Astrophys. J. 450, 830 (1995).
Burrows, A., Radice, D. & Vartanyan, D. Three-dimensional supernova explosion simulations of 9-, 10-, 11-, 12-, and 13- M ⊙ stars. Mon. Not. R. Astron. Soc. 485, 3153–3168 (2019).
Mösta, P. et al. r-process nucleosynthesis from three-dimensional magnetorotational core-collapse supernovae. Astrophys. J. 864, 171 (2018).
Radice, D., Morozova, V., Burrows, A., Vartanyan, D. & Nagakura, H. Characterizing the gravitational wave signal from core-collapse supernovae. Astrophys. J. Lett. 876, L9 (2019).
Janka, H.-T., Melson, T. & Summa, A. Physics of core-collapse supernovae in three dimensions: a sneak preview. Annu. Rev. Nucl. Part. Sci. 66, 341–375 (2016).
Woosley, S. & Janka, T. The physics of core-collapse supernovae. Nat. Phys. 1, 147–154 (2005).
Gossan, S. E. et al. Observing gravitational waves from core-collapse supernovae in the advanced detector era. Phys. Rev. D 93, 042002 (2016).
Aurisano, A. et al. A convolutional neural network neutrino event classifier. J. Instrum. 11, P09001 (2016).
Choma, N. et al. Graph neural networks for icecube signal classification. In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA) 386–391 (IEEE, 2018).
Hinderer, T. et al. Discerning the binary neutron star or neutron star-black hole nature of GW170817 with gravitational wave and electromagnetic measurements. Preprint at https://arxiv.org/abs/1808.03836 (2018).
Kim, B. et al. Deep fluids: a generative network for parameterized fluid simulations. Comput. Graph Forum 38, 59-70 (2019).
Ling, J., Kurzawski, A. & Templeton, J. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. J. Fluid Mech. 807, 155–166 (2016).
Maulik, R., San, O., Rasheed, A. & Vedula, P. Subgrid modelling for two-dimensional turbulence using neural networks. J. Fluid Mech. 858, 122–144 (2019).
Viganò, D. & Palenzuela, C. Fitting of extended sub-grid scale models in compressible turbulent MHD. Preprint at https://arxiv.org/abs/1904.04099 (2019).
Xie, C., Wang, J., Li, K. & Ma, C. Artificial neural network approach to large-eddy simulation of compressible isotropic turbulence. Phys. Rev. E 99, 053113 (2019).
Berg, J. & Nyström, K. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing 317, 28–41 (2018).
Weinan, E., Han, J. & Jentzen, A. Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun. Math. Stat. 5, 349–380 (2017).
Duez, M. D. & Zlochower, Y. Numerical relativity of compact binaries in the 21st century. Rep. Prog. Phys. 82, 016902 (2019).
Baiotti, L. & Rezzolla, L. Binary neutron star mergers: a review of Einstein’s richest laboratory. Rep. Prog. Phys. 80, 096901 (2017).
Lippuner, J. & Roberts, L. F. Skynet: a modular nuclear reaction network library. Astrophys. J. Suppl. Ser. 233, 18 (2017).
Paschalidis, V., Ruiz, M. & Shapiro, S. L. Relativistic simulations of black hole–neutron star coalescence: the jet emerges. Astrophys. J. 806, L14 (2015).
Ruiz, M., Lang, R. N., Paschalidis, V. & Shapiro, S. L. Binary neutron star mergers: a jet engine for short gamma-ray bursts. Astrophys. J. 824, L6 (2016).
Fernández, R. et al. Long-term GRMHD simulations of neutron star merger accretion disks: implications for electromagnetic counterparts. Mon. Not. R. Astron. Soc. 482, 3373 (2019).
Nouri, F. H. et al. Evolution of the magnetized, neutrino-cooled accretion disk in the aftermath of a black hole–neutron star binary merger. Phys. Rev. D 97, 083014 (2018).
Radice, D. et al. Binary neutron star mergers: mass ejection, electromagnetic counterparts, and nucleosynthesis. Astrophys. J. 869, 130 (2018).
Kasen, D., Badnell, N. R. & Barnes, J. Opacities and spectra of the r-process ejecta from neutron star mergers. Astrophys. J. 774, 25 (2013).
Berger, M. J. & Colella, P. Local adaptive mesh refinement for shock hydrodynamics. J. Comput. Phys. 82, 64–84 (1989).
Chen, T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. Neural ordinary differential equations. Advances in Neural Information Processing Systems 31, 6571-6583 (2018).
Radice, D. et al. Neutrino-driven convection in core-collapse supernovae: high-resolution simulations. Astrophys. J. 820, 76 (2016).
Giacomazzo, B., Zrake, J., Duffell, P., MacFadyen, A. I. & Perna, R. Producing magnetar magnetic fields in the merger of binary neutron stars. Astrophys. J. 809, 39 (2015).
EuroHPC Leading the way in the European supercomputing. EuroHPC https://eurohpc-ju.europa.eu/#inline-nav-1 (2018).
Huerta, E. A. et al. BOSS-LDG: a novel computational framework that brings together blue waters, open science grid, shifter and the LIGO data grid to accelerate gravitational wave discovery. In 2017 IEEE 13th International Conference on e-Science 335–344 (IEEE, 2017).
Arcavi, I. et al. Optical follow-up of gravitational-wave events with las cumbres observatory. Astrophys. J. 848, L33 (2017).
Coughlin, M. W. et al. Optimizing searches for electromagnetic counterparts of gravitational wave triggers. Mon. Not. R. Astron. Soc. 478, 692–702 (2018).
California Institute of Technology NED gravitational wave follow-up service. NED https://ned.ipac.caltech.edu/gwf/overview (2019).
Pennsylvania State University Astrophysical multimessenger observatory network. AMON https://www.amon.psu.edu/amon-system/ (2019).
Cowperthwaite, P. S. et al. Astro 2020 Science White Paper: Joint Gravitational Wave and Electromagnetic Astronomy with LIGO and LSST in the 2020’s. Preprint at https://arxiv.org/abs/1904.02718 (2019).
Marshall, P. et al. Science-Driven Optimization of the LSST Observing Strategy. Preprint at https://arxiv.org/abs/1708.04058, 10.5281/zenodo.842713 (2017).
A. Kinney et al. The W. M. Keck Observatory Scientific Strategic Plan. Keck Observers' Newsletter https://www2.keck.hawaii.edu/inst/newsletters/Vol20/scientificstrategicplan.pdf (2016).
Narayan, G. et al. Machine learning-based brokers for real-time classification of the LSST alert stream. Astrophys. J. Suppl. 236, 9 (2018).
Smith, K. W. et al. Lasair: the transient alert broker for LSST:UK. Res. Notes AAS 3, 26 (2019).
AEON Team Astronomical Event Observatory Network NOAO http://ast.noao.edu/data/aeon (2018).
The National Academies of Sciences, Engineering, and Medicine. The decadal survey on astronomy and astrophysics (astro2020). The National Academies of Sciences, Engineering, and Medicine http://sites.nationalacademies.org/DEPS/astro2020/index.htm (2019).
Katz, D. S. et al. Community organizations: changing the culture in which research software is developed and sustained. Comp. Sci. Eng. 21, 8–24 (2019).
Elmer, P., Neubauer, M. & Sokoloff, M. D. Strategic plan for a Scientific Software Innovation Institute (S2I2) for high energy physics. Preprint at https://arxiv.org/abs/1712.06592 (2017).
Albrecht, J. et al. A roadmap for HEP software and computing R&D for the 2020s. Comput. Softw. Big Sci. 3, 7 (2019).
Allen, G. et al. Multi-messenger astrophysics: harnessing the data revolution. Preprint at https://arxiv.org/abs/1807.04780 (2018).
Hotokezaka, K., Beniamini, P. & Piran, T. Neutron star mergers as sites of r-process nucleosynthesis and short gamma-ray bursts. Int. J. Mod. Phys. D 27, 1842005 (2018).
Löffler, F. et al. The Einstein toolkit: a community computational infrastructure for relativistic astrophysics. Class. Quantum Gravity 29, 115001 (2012).
Radice, D. & Rezzolla, L. THC: a new high-order finite-difference high-resolution shock-capturing code for special-relativistic hydrodynamics. Astron. Astrophys. 547, A26 (2012).
Acknowledgements
The authors gratefully acknowledge support from NVIDIA, Argonne Leadership Computing Facility, Oak Ridge Leadership Computing Facility, and the National Science Foundation through grant NSF-1848815. Artwork in this manuscript was supported in part by the National Science Foundation through grants ACI-1238993, NSF-1550514 and TG-PHY160053.
Author information
Authors and Affiliations
Contributions
E.A.H. led and coordinated the writing of this Expert Recommendation. All authors contributed to developing the ideas, and writing and reviewing this manuscript. S.R. produced the artwork in figure 1.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information
Nature Reviews Physics thanks Brant Robertson, Viviana Acquaviva and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Huerta, E.A., Allen, G., Andreoni, I. et al. Enabling real-time multi-messenger astrophysics discoveries with deep learning. Nat Rev Phys 1, 600–608 (2019). https://doi.org/10.1038/s42254-019-0097-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s42254-019-0097-4
This article is cited by
-
FAIR principles for AI models with a practical application for accelerated high energy diffraction microscopy
Scientific Data (2022)
-
A FAIR and AI-ready Higgs boson decay dataset
Scientific Data (2022)
-
Computational challenges for multimodal astrophysics
Nature Computational Science (2022)
-
Stark spectral line broadening modeling by machine learning algorithms
Neural Computing and Applications (2022)
-
Visualizing big science projects
Nature Reviews Physics (2021)
