Machine learning at the energy and intensity frontiers of particle physics

Abstract

Our knowledge of the fundamental particles of nature and their interactions is summarized by the standard model of particle physics. Advancing our understanding in this field has required experiments that operate at ever higher energies and intensities, which produce extremely large and information-rich data samples. The use of machine-learning techniques is revolutionizing how we interpret these data samples, greatly increasing the discovery potential of present and future experiments. Here we summarize the challenges and opportunities that come with the use of machine learning at the frontiers of particle physics.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Machine learning for calorimetry at CMS.
Fig. 2: Separating signal events from background in the ATLAS experiment.
Fig. 3: Neutrino selection and isolation in MicroBooNE.
Fig. 4: Exploring NOvA’s event-selection neural network using t-SNE.

References

  1. 1.

    Glaser, D. A. Some effects of ionizing radiation on the formation of bubbles in liquids. Phys. Rev. 87, 665 (1952).

    ADS  Article  CAS  Google Scholar 

  2. 2.

    Evans, L. & Bryant, P. LHC machine. J. Instrum. 3, S08001 (2008).

    Article  Google Scholar 

  3. 3.

    Alves, A. A. Jr et al. The LHCb detector at the LHC. J. Instrum. 3, S08005 (2008).

    Google Scholar 

  4. 4.

    Aad, G. et al. The ATLAS experiment at the CERN Large Hadron Collider. J. Instrum. 3, S08003 (2008).

    Article  Google Scholar 

  5. 5.

    Chatrchyan, S. et al. The CMS experiment at the CERN LHC. J. Instrum. 3, S08004 (2008).

    Google Scholar 

  6. 6.

    Bhat, P. Multivariate analysis methods in particle physics. Annu. Rev. Nucl. Part. Sci. 61, 281–309 (2011).

    ADS  Article  CAS  Google Scholar 

  7. 7.

    Rosenblatt, F. Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms (Spartan Books, Berlin, 1961).

    Google Scholar 

  8. 8.

    Reed, R. & Marks, R. Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks (MIT Press, Cambridge, 1999).

    Google Scholar 

  9. 9.

    Breiman, L., Friedman, J. H., Olshen, R. A. & Stone, C. J. Classification and Regression Trees (Wadsworth International Group, Belmont, 1984).

    Google Scholar 

  10. 10.

    Freund, Y. & Schapire, R. E. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997).

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    The ALEPH Collaboration. Determination of |Vub| from the measurement of the inclusive charmless semileptonic branching ratio of b hadrons. Eur. Phys. J. C 6, 555–574 (1999).

    ADS  Article  Google Scholar 

  12. 12.

    OPAL Collaboration. A measurement of the production of D*± mesons on the Z0 resonance. Z. Phys. C 67, 27–44 (1995).

    Article  Google Scholar 

  13. 13.

    Chiappetta, P., Colangelo, P., De Felice, P., Nardulli, G. & Pasquariello, G. Higgs search by neural networks at LHC. Phys. Lett. B 322, 219–223 (1994).

    ADS  Article  CAS  Google Scholar 

  14. 14.

    Peterson, C., Rognvaldsson, T. & Lönnblad, L. JETNET 3.0—a versatile artificial neural network package. Comput. Phys. Commun. 81, 185–220 (1994).

    ADS  Article  Google Scholar 

  15. 15.

    Buskulic, D. et al. Measurement of the tau polarisation at the Z resonance. Z. Phys. C 59, 369–386 (1993).

    ADS  Article  CAS  Google Scholar 

  16. 16.

    Babbage, W. S. & Thompson, L. F. The use of neural networks in γ-π0 discrimination. Nucl. Instrum. Methods A 330, 482–486 (1993).

    ADS  Article  Google Scholar 

  17. 17.

    Lönnblad, L., Peterson, C. & Rognvaldsson, T. Pattern recognition in high energy physics with artificial neural networks — JETNET 2.0. Comput. Phys. Commun. 70, 167–182 (1992).

    ADS  Article  Google Scholar 

  18. 18.

    Peterson, C. & Rögnvaldsson, T. S. An introduction to artificial neural networks. In 14th CERN School of Computing (ed. Verkerk, C.) 113–170 (CERN, 1992).

  19. 19.

    Lönnblad, L., Peterson, C. & Rögnvaldsson, T. Using neural networks to identify jets. Nucl. Phys. B 349, 675–702 (1991).

    ADS  Article  Google Scholar 

  20. 20.

    Lönnblad, L., Peterson, C. & Rögnvaldsson, T. Finding gluon jets with a neural trigger. Phys. Rev. Lett. 65, 1321–1324 (1990).

    ADS  Article  PubMed  Google Scholar 

  21. 21.

    Denby, B. Neural networks and cellular automata in experimental high-energy physics. Comput. Phys. Commun. 49, 429–448 (1988).

    ADS  Article  Google Scholar 

  22. 22.

    Roe, B. P. et al. Boosted decision trees as an alternative to artificial neural networks for particle identification. Nucl. Instrum. Methods A 543, 577–584 (2005).

    ADS  Article  CAS  Google Scholar 

  23. 23.

    Aad, G. et al. Observation of a new particle in the search for the standard model Higgs boson with the ATLAS detector at the LHC. Phys. Lett. B 716, 1–29 (2012).

    ADS  Article  CAS  Google Scholar 

  24. 24.

    Chatrchyan, S. et al. Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC. Phys. Lett. B 716, 30–61 (2012).

    ADS  Article  CAS  Google Scholar 

  25. 25.

    Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016).

    ADS  Article  PubMed  CAS  Google Scholar 

  26. 26.

    Russakovsky, O. et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015).

    MathSciNet  Article  Google Scholar 

  27. 27.

    Vagata, P. & Wilfong, K. Scaling the Facebook data warehouse to 300 PB. Facebook Code https://code.fb.com/core-data/scaling-the-facebook-data-warehouse-to-300-pb/ (2014).

  28. 28.

    CMS Collaboration. Boosted decision trees in the level-1 muon endcap trigger at CMS. In 18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research 21–25 (CERN, 2017).

  29. 29.

    Aaij, R. et al. The LHCb trigger and its performance in 2011. J. Instrum. 8, P04022 (2013).

    Article  CAS  Google Scholar 

  30. 30.

    Gligorov, V. V. & Williams, M. Efficient, reliable and fast high-level triggering using a bonsai boosted decision tree. J. Instrum. 8, P02013 (2013). This paper presents a boosted decision tree that identifies data in real time at LHCb and that has been used in more than 200 journal articles so far.

    Article  Google Scholar 

  31. 31.

    Likhomanenko, T. et al. LHCb topological trigger reoptimization. J. Phys. Conf. Ser. 664, 082025 (2015).

    Article  Google Scholar 

  32. 32.

    The LHCb Collaboration. LHCb detector performance. Int. J. Mod. Phys. A 30, 1530022 (2015).

    Article  CAS  Google Scholar 

  33. 33.

    Aaij, R. et al. Search for dark photons in 13 TeV pp collisions. Phys. Rev. Lett. 120, 061801 (2018).

    ADS  Article  PubMed  CAS  Google Scholar 

  34. 34.

    Hushchyn, M. et al. GRID storage optimization in transparent and user-friendly way for LHCb datasets. J. Phys. Conf. Ser. 898, 062023 (2017).

    Article  Google Scholar 

  35. 35.

    Derkach, D. et al. LHCb trigger streams optimization. J. Phys. Conf. Ser. 898, 062026 (2017).

    Article  Google Scholar 

  36. 36.

    Borisyak, M., Ratnikov, F., Derkach, D. & Ustyuzhanin, A. Towards automation of data quality system for CERN CMS experiment. J. Phys. Conf. Ser. 898, 092041 (2017).

    Article  Google Scholar 

  37. 37.

    Kuznetsov, V. et al. Predicting dataset popularity for the CMS experiment. J. Phys. Conf. Ser. 762, 012048 (2016).

    Article  Google Scholar 

  38. 38.

    Hushchyn, M., Charpentier, P. & Ustyuzhanin, A. Disk storage management for LHCb based on data popularity estimator. J. Phys. Conf. Ser. 664, 042026 (2015).

    Article  Google Scholar 

  39. 39.

    Bonacorsi, D. et al. Monitoring data transfer latency in CMS computing operations. J. Phys. Conf. Ser. 664, 032033 (2015).

    Article  Google Scholar 

  40. 40.

    CMS Collaboration. Energy calibration and resolution of the CMS electromagnetic calorimeter in pp collisions at √s = 7 TeV. J. Instrum. 8, P09009 (2013).

    Article  CAS  Google Scholar 

  41. 41.

    The ATLAS Collaboration. Evidence for the H → b b decay with the ATLAS detector. J. High Energy Phys. 12, 24 (2017).

    Google Scholar 

  42. 42.

    CMS Collaboration. Evidence for the Decay of the Higgs Boson to Bottom Quarks. Report No. CMS-PAS-HIG-16-044, https://cds.cern.ch/record/2278170 (CERN, 2017).

  43. 43.

    Aad, G. et al. Evidence for the Higgs-boson Yukawa coupling to tau leptons with the ATLAS detector. J. High Energy Phys. 4, 117 (2015).

    ADS  Article  CAS  Google Scholar 

  44. 44.

    Adam-Bourdarios, C. et al. The Higgs boson machine learning challenge. J. Mach. Learn. Res. Worksh. Conf. Proc. 42, 19–55 (2014). This work helped to popularize particle physics in the general machine-learning community and advertised recent advances in machine learning within the particle-physics community.

    Google Scholar 

  45. 45.

    CMS Collaboration & LHCb Collaboration Observation of the rare B s 0 → μ + μ decay from the combined analysis of CMS and LHCb data. Nature 522, 68–72 (2015).

    ADS  Article  CAS  Google Scholar 

  46. 46.

    Aaij, R. et al. Measurement of the B s 0 → μ + μ branching fraction and effective lifetime and search for B 0 → μ + μ decays. Phys. Rev. Lett. 118, 191801 (2017).

    ADS  Article  CAS  Google Scholar 

  47. 47.

    Yonghui, W. et al. Google’s neural machine translation system: bridging the gap between human and machine translation. Preprint at https://arxiv.org/abs/1609.08144 (2016).

  48. 48.

    LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    ADS  Article  PubMed  CAS  Google Scholar 

  49. 49.

    Baldi, P., Sadowski, P. & Whiteson, D. Searching for exotic particles in high-energy physics with deep learning. Nat. Commun. 5, 4308 (2014). This paper introduced deep learning to high-energy physics and explains the difference between shallow networks with high-level features and deep networks that find their own high-level features.

    ADS  Article  PubMed  CAS  Google Scholar 

  50. 50.

    de Oliveira, L., Kagan, M., Mackey, L., Nachman, B. & Schwartzman, A. Jet-images — deep learning edition. J. High Energy Phys. 7, 69 (2016). This paper started investigations of deep-learning approaches to jets in quantum chromodynamics and includes a detailed discussion of CNNs and supporting exploration of network behaviour.

    Article  Google Scholar 

  51. 51.

    Racah, E. et al. Revealing fundamental physics from the Daya Bay Neutrino Experiment using deep neural networks. In 15th IEEE International Conference on Machine Learning and Applications 892–897 (IEEE, 2016).

  52. 52.

    Aurisano, A. et al. A convolutional neural network neutrino event classifier. J. Instrum. 11, P09001 (2016). The paper presents the first CNN to be used for a physics analysi s 70 and includes a detailed discussion of the method and comparison to more traditional neutrino-identification methods.

  53. 53.

    Sadowski, P, et al. Efficient antihydrogen detection in antimatter physics by deep learning. J. Phys. Commun. 1, 025001 (2017).

    Article  Google Scholar 

  54. 54.

    Renner, J. Background rejection in NEXT using deep neural networks. J. Instrum. 12, T01004 (2017).

    Article  Google Scholar 

  55. 55.

    Wielgosz, M., Skoczeń, A. & Mertik, M. Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets. Nucl. Instrum. Methods A 867, 40–50 (2017).

    ADS  Article  CAS  Google Scholar 

  56. 56.

    Edelen, A. L. et al. Neural networks for modeling and control of particle accelerators. IEEE Trans. Nucl. Sci. 63, 878–897 (2016).

    ADS  Article  Google Scholar 

  57. 57.

    LeCun, Y. et al. Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 541–551 (1989).

    Article  Google Scholar 

  58. 58.

    Gers, F. A., Schmidhuber, J. & Cummins, F. Learning to forget: continual prediction with LSTM. Neural Comput. 12, 2451–2471 (1999).

    Article  Google Scholar 

  59. 59.

    Cogan, J., Kagan, M., Strauss, E. & Schwartzman, A. Jet-images: computer vision inspired techniques for jet tagging. J. High Energy Phys. 2, 118 (2015).

    ADS  Article  Google Scholar 

  60. 60.

    Baldi, P., Bauer, K., Eng, C., Sadowski, P. & Whiteson, D. Jet substructure classification in high-energy physics with deep neural networks. Phys. Rev. D 93, 094034 (2016).

    ADS  Article  CAS  Google Scholar 

  61. 61.

    Barnard, J., Dawe, E. N., Dolan, M. J. & Rajcic, N. Parton shower uncertainties in jet substructure analyses with deep neural networks. Phys. Rev. D 95, 014018 (2017).

    ADS  Article  Google Scholar 

  62. 62.

    Komiske, P. T., Metodiev, E. M. & Schwartz, M. D. Deep learning in color: towards automated quark/gluon jet discrimination. J. High Energy Phys. 1, 110 (2017).

    ADS  Article  MATH  Google Scholar 

  63. 63.

    de Oliveira, L., Paganini, M. & Nachman, B. Learning particle physics by example: location-aware generative adversarial networks for physics synthesis. Comput. Softw. Big Sci. 1, 4 (2017).

    Article  Google Scholar 

  64. 64.

    Kasieczka, G., Plehn, T., Russell, M. & Schell, T. Deep-learning top taggers or the end of QCD? J. High Energy Phys. 5, 6 (2017).

    ADS  Article  Google Scholar 

  65. 65.

    Shimmin, C. et al. Decorrelated jet substructure tagging using adversarial neural networks. Phys. Rev. D 96, 074034 (2017).

    ADS  Article  Google Scholar 

  66. 66.

    The ATLAS Collaboration. Quark versus Gluon Jet Tagging using Jet Images with the ATLAS Detector. Report No. ATL-PHYS-PUB-2017-017, https://cds.cern.ch/record/2275641 (CERN, 2017).

  67. 67.

    CMS Collaboration. New Developments for Jet Substructure Reconstruction in CMS. Report No. CMS-DP-2017-027, https://cds.cern.ch/record/2275226 (CERN, 2017).

  68. 68.

    NOvA Collaboration. The NOvA Technical Design Report. Report No. FERMILAB-DESIGN-2007-01 (FNAL, 2007)

  69. 69.

    Szegedy, C. et al. Going deeper with convolutions. In 2015 IEEE Conference on Computer Vision and Pattern Recognition 1–9 (IEEE, 2015).

  70. 70.

    Adamson, P. et al. Constraints on oscillation parameters from v e appearance and v μ disappearance in NOvA. Phys. Rev. Lett. 118, 231801 (2017).

    ADS  Article  PubMed  CAS  Google Scholar 

  71. 71.

    Acciarri, R. et al. Design and construction of the MicroBooNE detector. J. Instrum. 12, P02017 (2017).

    Article  Google Scholar 

  72. 72.

    Adamson, P. et al. Search for active-sterile neutrino mixing using neutral-current interactions in NOvA. Phys. Rev. D 96, 072006 (2017).

    ADS  Article  Google Scholar 

  73. 73.

    Acciarri, R. et al. Convolutional neural networks applied to neutrino events in a liquid argon time projection chamber. J. Instrum. 12, P03011 (2017).

    Article  Google Scholar 

  74. 74.

    Ren, S., He, K., Girshick, R. & Sun, J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 29, 061137 (2017).

    Google Scholar 

  75. 75.

    ATLAS Collaboration. Performance of b-jet identification in the ATLAS experiment. J. Instrum. 11, P04008 (2016).

    Article  CAS  Google Scholar 

  76. 76.

    CMS Collaboration. CMS Phase 1 Heavy Flavour Identification Performance and Developments. Report No. CERN-CMS-DP-2017-013, https://cds.cern.ch/record/2263802 (CERN, 2017).

  77. 77.

    ATLAS Collaboration. Identification of Jets Containing b-Hadrons with Recurrent Neural Networks at the ATLAS Experiment. Report No. ATL-PHYS-PUB-2017-003, https://cds.cern.ch/record/2255226 (CERN, 2017).

  78. 78.

    ATLAS Collaboration. Optimisation and Performance Studies of the ATLAS b-Tagging Algorithms for the 2017-18 LHC Run. Report No. ATL-PHYS-PUB-2017-013, https://cds.cern.ch/record/2273281 (CERN, 2017).

  79. 79.

    CMS Collaboration. Heavy Flavor Identification at CMS with Deep Neural Networks. Report No. CMS-DP-2017-005, https://cds.cern.ch/record/2255736 (CERN, 2017).

  80. 80.

    Guest, D. et al. Jet flavor classification in high-energy physics with deep neural networks. Phys. Rev. D 94, 112002 (2016).

    ADS  Article  Google Scholar 

  81. 81.

    Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. Nature 323, 533–536 (1986).

    ADS  Article  MATH  Google Scholar 

  82. 82.

    ATLAS Collaboration. Electron efficiency measurements with the ATLAS detector using 2012 LHC proton–proton collision data. Eur. Phys. J. C 77, 195 (2017).

    ADS  Article  CAS  Google Scholar 

  83. 83.

    The CMS Collaboration. Performance of electron reconstruction and selection with the CMS detector in proton-proton collisions at √s = 8 TeV. J. Instrum. 10, P06005 (2015).

    Article  CAS  Google Scholar 

  84. 84.

    Sachdev, K. Muon Neutrino to Electron Neutrino Oscillation in NOvA. PhD thesis, Univ. Minnesota (2015).

    Google Scholar 

  85. 85.

    Chatrchyan, S. et al. Evidence for the 125 GeV Higgs boson decaying to a pair of τ leptons. J. High Energy Phys. 5, 104 (2014).

    ADS  Article  CAS  Google Scholar 

  86. 86.

    van der Maaten, L. Accelerating t-SNE using tree- based algorithms. J. Mach. Learn. Res. 15, 3221–3245 (2014).

    MathSciNet  MATH  Google Scholar 

  87. 87.

    Louppe, G., Cho, K., Becot, C. & Cranmer, K. QCD-aware recursive neural networks for jet physics. Preprint at https://arxiv.org/abs/1702.00748 (2017).

  88. 88.

    Goodfellow, I. J. et al. Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27, 2672–2680 (2014).

    Google Scholar 

  89. 89.

    Rezende, D.J., Mohamed, S. & Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. J. Mach. Learn. Res. Worksh. Conf. Proc. 32, 1278–1286 (2014).

    Google Scholar 

  90. 90.

    Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. Preprint at https://arxiv.org/abs/1312.6114 (2014).

  91. 91.

    Paganini, M., de Oliveira, L. & Nachman, B. Accelerating science with generative adversarial networks: an application to 3D particle showers in multilayer calorimeters. Phys. Rev. Lett. 120, 042003 (2018).

    ADS  Article  PubMed  Google Scholar 

  92. 92.

    Carminati, F. et al. Calorimetry with deep learning: particle classification, energy regression, and simulation for high-energy physics. In NIPS Deep Learning for Physical Sciences Workshop (NIPS, 2017).

  93. 93.

    Louppe, G., Kagan, M. & Cranmer, K. Learning to pivot with adversarial networks. Adv. Neural Inf. Process. Syst. 30, 981–990 (2017). This paper forms part of a collection of works that present a more nuanced loss function and, along with similar work for BDTs (see section ‘Conclusions and outlook’), could lead to a new paradigm for training machine-learning models in high-energy physics.

    Google Scholar 

  94. 94.

    Stevens, J. & Williams, M. uBoost: a boosting method for producing uniform selection efficiencies from multivariate classifiers. J. Instrum. 8, P12013 (2013).

    Article  Google Scholar 

  95. 95.

    Rogozhnikov, A., Bukva, A., Gligorov, V. V., Ustyuzhanin, A. & Williams, M. New approaches for boosting to uniformity. J. Instrum. 10, T03002 (2015).

    Article  CAS  Google Scholar 

  96. 96.

    Dery, L. M., Nachman, B., Rubbo, F. & Schwartzman, A. Weakly supervised classification in high energy physics. J. High Energy Phys. 5, 145 (2017).

    ADS  Article  MATH  Google Scholar 

  97. 97.

    Baldi, P., Cranmer, K., Faucett, T., Sadowski, P. & Whiteson, D. Parameterized neural networks for high-energy physics. Eur. Phys. J. C 76, 235 (2016).

    ADS  Article  CAS  Google Scholar 

  98. 98.

    Aaij, R. et al. Search for hidden-sector bosons in B 0K *0 μ + μ decays. Phys. Rev. Lett. 115, 161802 (2015).

    ADS  Article  PubMed  CAS  Google Scholar 

  99. 99.

    The ATLAS collaboration. Search for the \(b\bar{b}\) bb decay of the standard model Higgs boson in associated (W/Z)H production with the ATLAS detector. J. High Energy Phys. 1, 69 (2015).

    MathSciNet  MATH  Google Scholar 

  100. 100.

    Chatrchyan, S. et al. Search for the standard model Higgs boson produced in association with a W or a Z boson and decaying to bottom quarks. Phys. Rev. D 89, 012003 (2014).

    ADS  Article  CAS  Google Scholar 

  101. 101.

    CMS Collaboration. 2015 ECAL Detector Performance Plots. Report No. CMS-DP-2015-057, https://cds.cern.ch/record/2114735 (CERN, 2015).

Download references

Reviewer information

Nature thanks C. Backhouse, M. Pierini and the other anonymous reviewer(s) for their contribution to the peer review of this work.

Author information

Affiliations

Authors

Contributions

All authors contributed to writing this Review. A.R. and M.W. were the principal editors.

Corresponding authors

Correspondence to Alexander Radovic or Mike Williams.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Radovic, A., Williams, M., Rousseau, D. et al. Machine learning at the energy and intensity frontiers of particle physics. Nature 560, 41–48 (2018). https://doi.org/10.1038/s41586-018-0361-2

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing