The development of reusable artificial intelligence (AI) models for wider use and rigorous validation by the community promises to unlock new opportunities in multi-messenger astrophysics. Here we develop a workflow that connects the Data and Learning Hub for Science, a repository for publishing AI models, with the Hardware-Accelerated Learning (HAL) cluster, using funcX as a universal distributed computing service. Using this workflow, an ensemble of four openly available AI models can be run on HAL to process an entire month’s worth (August 2017) of advanced Laser Interferometer Gravitational-Wave Observatory data in just seven minutes, identifying all four binary black hole mergers previously identified in this dataset and reporting no misclassifications. This approach combines advances in AI, distributed computing and scientific data infrastructure to open new pathways to conduct reproducible, accelerated, data-driven discovery.
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Advanced LIGO data used in this manuscript are open-source and readily available at the Gravitational Wave Open Science Center40. Modelled waveforms used to train, validate and test our AI models were produced using the open-source PyCBC library47. The waveform family used was SEOBNRv348. The datasets generated and/or analysed during the current study are available from the corresponding author upon reasonable request.
All the required software to reproduce our results, encompassing AI models and post-processing scripts, are open-source and readily available at the DLHub and may be found at ref. 49. Software to produce waveforms at scale in high-performance computing platforms with PyCBC may be provided upon request.
Abbott, B. P. et al. Observation of gravitational waves from a binary black hole merger. Phys. Rev. Lett. 116, 061102 (2016).
Abbott, B. P. et al. GWTC-1: a gravitational-wave transient catalog of compact binary mergers observed by LIGO and Virgo during the first and second observing runs. Phys. Rev. X 9, 031040 (2019).
Abbott, R. et al. GWTC-2: compact binary coalescences observed by LIGO and Virgo during the first half of the third observing run. Preprint at https://arxiv.org/abs/2010.14527 (2020).
Abbott, R. et al. Population properties of compact objects from the second LIGO–Virgo gravitational-wave transient catalog. Astrophys. J. Lett. 913, L7 (2021).
Soares-Santos, M. et al. First measurement of the Hubble constant from a dark standard siren using the Dark Energy Survey galaxies and the LIGO/Virgo binary–black-hole merger GW170814. Astrophys. J. Lett. 876, L7 (2019).
Abbott, B. P. et al. A gravitational-wave standard siren measurement of the Hubble constant. Nature 551, 85–88 (2017).
Schutz, B. F. Determining the Hubble constant from gravitational wave observations. Nature 323, 310–311 (1986).
Berti, E., Yagi, K. & Yunes, N. Extreme gravity tests with gravitational waves from compact binary coalescences: (I) inspiral–merger. Gen. Relativ. Gravit. 50, 46 (2018).
Abbott, B. et al. Tests of general relativity with the binary black hole signals from the LIGO-Virgo catalog GWTC-1. Phys. Rev. D 100, 104036 (2019).
Radice, D. et al. Dynamical mass ejection from binary neutron star mergers. Mon. Not. R. Astron. Soc. 460, 3255–3271 (2016).
Metzger, B. D. Kilonovae. Living Rev. Relativ. 23, 1 (2020).
Huerta, E. A. et al. BOSS-LDG: a novel computational framework that brings together Blue Waters, Open Science Grid, Shifter and the LIGO Data Grid to accelerate gravitational wave discovery. In Proc. IEEE 13th International Conference on e-Science 335–344 (2017).
Huerta, E. A., Haas, R., Jha, S., Neubauer, M. & Katz, D. S. Supporting high-performance and high-throughput computing for experimental science. Comput. Softw. Big Sci. 3, 5 (2019).
Weitzel, D. et al. Data access for LIGO on the OSG. In Proc. PEARC17: Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact (eds Hart, D. L. & Dahan, M.) (Association for Computing Machinery, 2017).
Asch, M. et al. Big data and extreme-scale computing: pathways to convergence – toward a shaping strategy for a future software and data ecosystem for scientific inquiry. Int. J. High Perform. Comput. Appl. 32, 435–479 (2018).
Huerta, E. A. et al. Enabling real-time multi-messenger astrophysics discoveries with deep learning. Nat. Rev. Phys. 1, 600–608 (2019).
George, D. & Huerta, E. A. Deep neural networks to enable real-time multimessenger astrophysics. Phys. Rev. D 97, 044039 (2018).
George, D. & Huerta, E. A. Deep learning for real-time gravitational wave detection and parameter estimation: results with advanced LIGO data. Phys. Lett. B 778, 64–70 (2018).
Gabbard, H., Williams, M., Hayes, F. & Messenger, C. Matching matched filtering with deep networks for gravitational-wave astronomy. Phys. Rev. Lett. 120, 141103 (2018).
Lin, Y.-C. & Wu, J.-H. P. Detection of gravitational waves using Bayesian neural networks. Phys. Rev. D 103, 063034 (2021).
Wang, H., Wu, S., Cao, Z., Liu, X. & Zhu, J.-Y. Gravitational-wave signal recognition of LIGO data by deep learning. Phys. Rev. D 101, 104003 (2020).
Zevin, M. et al. Gravity Spy: integrating advanced LIGO detector characterization, machine learning, and citizen science. Class. Quantum Gravity 34, 064003 (2017).
Torres-Forné, A., Cuoco, E., Font, J. A. & Marquina, A. Application of dictionary learning to denoise LIGO’s blip noise transients. Phys. Rev. D 102, 023011 (2020).
Nakano, H. et al. Comparison of various methods to extract ringdown frequency from gravitational wave data. Phys. Rev. D 99, 124032 (2019).
Fan, X., Li, J., Li, X., Zhong, Y. & Cao, J. Applying deep neural networks to the detection and space parameter estimation of compact binary coalescence with a network of gravitational wave detectors. Sci. China Phys. Mech. Astron. 62, 969512 (2019).
Deighan, D. S., Field, S. E., Capano, C. D. & Khanna, G. Genetic-algorithm-optimized neural networks for gravitational wave classification. Neural Comput. & Applic. https://doi.org/10.1007/s00521-021-06024-4 (2020).
Miller, A. L. et al. How effective is machine learning to detect long transient gravitational waves from neutron stars in a real search? Phys. Rev. D 100, 062005 (2019).
Krastev, P. G. Real-time detection of gravitational waves from binary neutron stars using artificial neural networks. Phys. Lett. B 803, 135330 (2020).
Schäfer, M. B., Ohme, F. & Nitz, A. H. Detection of gravitational-wave signals from binary neutron star mergers using machine learning. Phys. Rev. D 102, 063015 (2020).
Khan, A., Huerta, E. & Das, A. Physics-inspired deep learning to characterize the signal manifold of quasi-circular, spinning, non-precessing binary black hole mergers. Phys. Lett. B 808, 135628 (2020).
Dreissigacker, C., Sharma, R., Messenger, C., Zhao, R. & Prix, R. Deep-learning continuous gravitational waves. Phys. Rev. D 100, 044009 (2019).
Beheshtipour, B. & Papa, M. A. Deep learning for clustering of continuous gravitational wave candidates. Phys. Rev. D 101, 064009 (2020).
Skliris, V., Norman, M. R. K. & Sutton, P. J. Real-time detection of unmodeled gravitational-wave transients using convolutional neural networks. Preprint at https://arxiv.org/abs/2009.14611 (2020).
Khan, S. & Green, R. Gravitational-wave surrogate models powered by artificial neural networks. Phys. Rev. D 103, 064015 (2021).
Chua, A. J. K., Galley, C. R. & Vallisneri, M. Reduced-order modeling with artificial neurons for gravitational-wave inference. Phys. Rev. Lett. 122, 211101 (2019).
Wei, W. & Huerta, E. A. Deep learning for gravitational wave forecasting of neutron star mergers. Phys. Lett. B 816, 136185 (2021).
Wei, W. et al. Deep learning with quantized neural networks for gravitational wave forecasting of eccentric compact binary coalescence. Preprint at https://arxiv.org/abs/2012.03963 (2020).
Cuoco, E. et al. Enhancing gravitational-wave science with machine learning. Mach. Learn. Sci. Technol. 2, 011002 (2021).
Wei, W., Khan, A., Huerta, E. A., Huang, X. & Tian, M. Deep learning ensemble for real-time gravitational wave detection of spinning binary black hole mergers. Phys. Lett. B 812, 136029 (2021).
Vallisneri, M., Kanner, J., Williams, R., Weinstein, A. & Stephens, B. The LIGO Open Science Center. J. Phys. Conf. Ser. 610, 012021 (2015).
Kindratenko, V. et al. HAL: computer system for scalable deep learning. In Proc. PEARC20: Practice and Experience in Advanced Research Computing 41–48 (Association for Computing Machinery, 2020).
Li, Z. et al. DLHub: simplifying publication, discovery, and use of machine learning models in science. J. Parallel Distrib. Comput. 147, 64–76 (2021).
Chard, R. et al. DLHub: model and data serving for science. In Proc. IEEE International Parallel and Distributed Processing Symposium 283–292 (2019).
Allen, M. G. et al. ESCAPE – addressing Open Science challenges. Preprint at https://arxiv.org/abs/2012.11534 (2020).
Chard, R. et al. FuncX: a federated function serving fabric for science. In Proc. 29th International Symposium on High-Performance Parallel and Distributed Computing 65–76, (Association for Computing Machinery, 2020).
Chard, K., Tuecke, S. & Foster, I. Efficient and secure transfer, synchronization, and sharing of big data. IEEE Cloud Comput. 1, 46–55 (2014).
Nitz, A. H. et al. PyCBC. Free and open software to study gravitational waves. https://pycbc.org/ (2021).
Pan, Y. et al. Inspiral-merger-ringdown waveforms of spinning, precessing black-hole binaries in the effective-one-body formalism. Phys. Rev. D 89, 084006 (2014).
Huerta, E. A. et al. AI-driven Gravitational Wave Detection (Data and Learning Hub for Science, 2021); https://doi.org/10.26311/ZWFW-3N68
van den Oord, A. et al. WaveNet: a generative model for raw audio. In Proc. 9th ISCA Speech Synthesis Workshop (eds Bonafont, A. & Prahallad, K.) 135–136 (2016).
We gratefully acknowledge NSF awards OAC-1931561 and OAC-1934757 (E.A.H.), OAC-1931306 (B.B.) and OAC-2004894 (I.F.). E.A.H. gratefully acknowledges the Innovative and Novel Computational Impact on Theory and Experiment project ‘Multi-Messenger Astrophysics at Extreme Scale in Summit’. This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under contract no. DE-AC05-00OR22725. This work used resources supported by the NSF’s Major Research Instrumentation program, the HAL cluster (grant no. OAC-1725729), as well as by the University of Illinois at Urbana-Champaign. DLHub is based upon work initially supported by Laboratory Directed Research and Development funding from Argonne National Laboratory, provided by the Director, Office of Science, of the DOE under contract no. DE-AC02-06CH11357. We thank NVIDIA for their continued support.
The authors declare no competing interests.
Peer review information Nature Astronomy thanks Elena Cuoco, Plamen Krastev and Linqing Wen for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Huerta, E.A., Khan, A., Huang, X. et al. Accelerated, scalable and reproducible AI-driven gravitational wave detection. Nat Astron 5, 1062–1068 (2021). https://doi.org/10.1038/s41550-021-01405-0