Review Article | Published:

# Machine learning at the energy and intensity frontiers of particle physics

## Abstract

Our knowledge of the fundamental particles of nature and their interactions is summarized by the standard model of particle physics. Advancing our understanding in this field has required experiments that operate at ever higher energies and intensities, which produce extremely large and information-rich data samples. The use of machine-learning techniques is revolutionizing how we interpret these data samples, greatly increasing the discovery potential of present and future experiments. Here we summarize the challenges and opportunities that come with the use of machine learning at the frontiers of particle physics.

## Access optionsAccess options

from\$8.99

All prices are NET prices.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## References

1. 1.

Glaser, D. A. Some effects of ionizing radiation on the formation of bubbles in liquids. Phys. Rev. 87, 665 (1952).

2. 2.

Evans, L. & Bryant, P. LHC machine. J. Instrum. 3, S08001 (2008).

3. 3.

Alves, A. A. Jr et al. The LHCb detector at the LHC. J. Instrum. 3, S08005 (2008).

4. 4.

Aad, G. et al. The ATLAS experiment at the CERN Large Hadron Collider. J. Instrum. 3, S08003 (2008).

5. 5.

Chatrchyan, S. et al. The CMS experiment at the CERN LHC. J. Instrum. 3, S08004 (2008).

6. 6.

Bhat, P. Multivariate analysis methods in particle physics. Annu. Rev. Nucl. Part. Sci. 61, 281–309 (2011).

7. 7.

Rosenblatt, F. Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms (Spartan Books, Berlin, 1961).

8. 8.

Reed, R. & Marks, R. Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks (MIT Press, Cambridge, 1999).

9. 9.

Breiman, L., Friedman, J. H., Olshen, R. A. & Stone, C. J. Classification and Regression Trees (Wadsworth International Group, Belmont, 1984).

10. 10.

Freund, Y. & Schapire, R. E. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997).

11. 11.

The ALEPH Collaboration. Determination of |Vub| from the measurement of the inclusive charmless semileptonic branching ratio of b hadrons. Eur. Phys. J. C 6, 555–574 (1999).

12. 12.

OPAL Collaboration. A measurement of the production of D*± mesons on the Z0 resonance. Z. Phys. C 67, 27–44 (1995).

13. 13.

Chiappetta, P., Colangelo, P., De Felice, P., Nardulli, G. & Pasquariello, G. Higgs search by neural networks at LHC. Phys. Lett. B 322, 219–223 (1994).

14. 14.

Peterson, C., Rognvaldsson, T. & Lönnblad, L. JETNET 3.0—a versatile artificial neural network package. Comput. Phys. Commun. 81, 185–220 (1994).

15. 15.

Buskulic, D. et al. Measurement of the tau polarisation at the Z resonance. Z. Phys. C 59, 369–386 (1993).

16. 16.

Babbage, W. S. & Thompson, L. F. The use of neural networks in γ-π0 discrimination. Nucl. Instrum. Methods A 330, 482–486 (1993).

17. 17.

Lönnblad, L., Peterson, C. & Rognvaldsson, T. Pattern recognition in high energy physics with artificial neural networks — JETNET 2.0. Comput. Phys. Commun. 70, 167–182 (1992).

18. 18.

Peterson, C. & Rögnvaldsson, T. S. An introduction to artificial neural networks. In 14th CERN School of Computing (ed. Verkerk, C.) 113–170 (CERN, 1992).

19. 19.

Lönnblad, L., Peterson, C. & Rögnvaldsson, T. Using neural networks to identify jets. Nucl. Phys. B 349, 675–702 (1991).

20. 20.

Lönnblad, L., Peterson, C. & Rögnvaldsson, T. Finding gluon jets with a neural trigger. Phys. Rev. Lett. 65, 1321–1324 (1990).

21. 21.

Denby, B. Neural networks and cellular automata in experimental high-energy physics. Comput. Phys. Commun. 49, 429–448 (1988).

22. 22.

Roe, B. P. et al. Boosted decision trees as an alternative to artificial neural networks for particle identification. Nucl. Instrum. Methods A 543, 577–584 (2005).

23. 23.

Aad, G. et al. Observation of a new particle in the search for the standard model Higgs boson with the ATLAS detector at the LHC. Phys. Lett. B 716, 1–29 (2012).

24. 24.

Chatrchyan, S. et al. Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC. Phys. Lett. B 716, 30–61 (2012).

25. 25.

Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016).

26. 26.

Russakovsky, O. et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015).

27. 27.

Vagata, P. & Wilfong, K. Scaling the Facebook data warehouse to 300 PB. Facebook Code https://code.fb.com/core-data/scaling-the-facebook-data-warehouse-to-300-pb/ (2014).

28. 28.

CMS Collaboration. Boosted decision trees in the level-1 muon endcap trigger at CMS. In 18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research 21–25 (CERN, 2017).

29. 29.

Aaij, R. et al. The LHCb trigger and its performance in 2011. J. Instrum. 8, P04022 (2013).

30. 30.

Gligorov, V. V. & Williams, M. Efficient, reliable and fast high-level triggering using a bonsai boosted decision tree. J. Instrum. 8, P02013 (2013). This paper presents a boosted decision tree that identifies data in real time at LHCb and that has been used in more than 200 journal articles so far.

31. 31.

Likhomanenko, T. et al. LHCb topological trigger reoptimization. J. Phys. Conf. Ser. 664, 082025 (2015).

32. 32.

The LHCb Collaboration. LHCb detector performance. Int. J. Mod. Phys. A 30, 1530022 (2015).

33. 33.

Aaij, R. et al. Search for dark photons in 13 TeV pp collisions. Phys. Rev. Lett. 120, 061801 (2018).

34. 34.

Hushchyn, M. et al. GRID storage optimization in transparent and user-friendly way for LHCb datasets. J. Phys. Conf. Ser. 898, 062023 (2017).

35. 35.

Derkach, D. et al. LHCb trigger streams optimization. J. Phys. Conf. Ser. 898, 062026 (2017).

36. 36.

Borisyak, M., Ratnikov, F., Derkach, D. & Ustyuzhanin, A. Towards automation of data quality system for CERN CMS experiment. J. Phys. Conf. Ser. 898, 092041 (2017).

37. 37.

Kuznetsov, V. et al. Predicting dataset popularity for the CMS experiment. J. Phys. Conf. Ser. 762, 012048 (2016).

38. 38.

Hushchyn, M., Charpentier, P. & Ustyuzhanin, A. Disk storage management for LHCb based on data popularity estimator. J. Phys. Conf. Ser. 664, 042026 (2015).

39. 39.

Bonacorsi, D. et al. Monitoring data transfer latency in CMS computing operations. J. Phys. Conf. Ser. 664, 032033 (2015).

40. 40.

CMS Collaboration. Energy calibration and resolution of the CMS electromagnetic calorimeter in pp collisions at √s = 7 TeV. J. Instrum. 8, P09009 (2013).

41. 41.

The ATLAS Collaboration. Evidence for the H → b b decay with the ATLAS detector. J. High Energy Phys. 12, 24 (2017).

42. 42.

CMS Collaboration. Evidence for the Decay of the Higgs Boson to Bottom Quarks. Report No. CMS-PAS-HIG-16-044, https://cds.cern.ch/record/2278170 (CERN, 2017).

43. 43.

Aad, G. et al. Evidence for the Higgs-boson Yukawa coupling to tau leptons with the ATLAS detector. J. High Energy Phys. 4, 117 (2015).

44. 44.

Adam-Bourdarios, C. et al. The Higgs boson machine learning challenge. J. Mach. Learn. Res. Worksh. Conf. Proc. 42, 19–55 (2014). This work helped to popularize particle physics in the general machine-learning community and advertised recent advances in machine learning within the particle-physics community.

45. 45.

CMS Collaboration & LHCb Collaboration Observation of the rare B s 0 → μ + μ decay from the combined analysis of CMS and LHCb data. Nature 522, 68–72 (2015).

46. 46.

Aaij, R. et al. Measurement of the B s 0 → μ + μ branching fraction and effective lifetime and search for B 0 → μ + μ decays. Phys. Rev. Lett. 118, 191801 (2017).

47. 47.

Yonghui, W. et al. Google’s neural machine translation system: bridging the gap between human and machine translation. Preprint at https://arxiv.org/abs/1609.08144 (2016).

48. 48.

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

49. 49.

Baldi, P., Sadowski, P. & Whiteson, D. Searching for exotic particles in high-energy physics with deep learning. Nat. Commun. 5, 4308 (2014). This paper introduced deep learning to high-energy physics and explains the difference between shallow networks with high-level features and deep networks that find their own high-level features.

50. 50.

de Oliveira, L., Kagan, M., Mackey, L., Nachman, B. & Schwartzman, A. Jet-images — deep learning edition. J. High Energy Phys. 7, 69 (2016). This paper started investigations of deep-learning approaches to jets in quantum chromodynamics and includes a detailed discussion of CNNs and supporting exploration of network behaviour.

51. 51.

Racah, E. et al. Revealing fundamental physics from the Daya Bay Neutrino Experiment using deep neural networks. In 15th IEEE International Conference on Machine Learning and Applications 892–897 (IEEE, 2016).

52. 52.

Aurisano, A. et al. A convolutional neural network neutrino event classifier. J. Instrum. 11, P09001 (2016). The paper presents the first CNN to be used for a physics analysi s 70 and includes a detailed discussion of the method and comparison to more traditional neutrino-identification methods.

53. 53.

Sadowski, P, et al. Efficient antihydrogen detection in antimatter physics by deep learning. J. Phys. Commun. 1, 025001 (2017).

54. 54.

Renner, J. Background rejection in NEXT using deep neural networks. J. Instrum. 12, T01004 (2017).

55. 55.

Wielgosz, M., Skoczeń, A. & Mertik, M. Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets. Nucl. Instrum. Methods A 867, 40–50 (2017).

56. 56.

Edelen, A. L. et al. Neural networks for modeling and control of particle accelerators. IEEE Trans. Nucl. Sci. 63, 878–897 (2016).

57. 57.

LeCun, Y. et al. Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 541–551 (1989).

58. 58.

Gers, F. A., Schmidhuber, J. & Cummins, F. Learning to forget: continual prediction with LSTM. Neural Comput. 12, 2451–2471 (1999).

59. 59.

Cogan, J., Kagan, M., Strauss, E. & Schwartzman, A. Jet-images: computer vision inspired techniques for jet tagging. J. High Energy Phys. 2, 118 (2015).

60. 60.

Baldi, P., Bauer, K., Eng, C., Sadowski, P. & Whiteson, D. Jet substructure classification in high-energy physics with deep neural networks. Phys. Rev. D 93, 094034 (2016).

61. 61.

Barnard, J., Dawe, E. N., Dolan, M. J. & Rajcic, N. Parton shower uncertainties in jet substructure analyses with deep neural networks. Phys. Rev. D 95, 014018 (2017).

62. 62.

Komiske, P. T., Metodiev, E. M. & Schwartz, M. D. Deep learning in color: towards automated quark/gluon jet discrimination. J. High Energy Phys. 1, 110 (2017).

63. 63.

de Oliveira, L., Paganini, M. & Nachman, B. Learning particle physics by example: location-aware generative adversarial networks for physics synthesis. Comput. Softw. Big Sci. 1, 4 (2017).

64. 64.

Kasieczka, G., Plehn, T., Russell, M. & Schell, T. Deep-learning top taggers or the end of QCD? J. High Energy Phys. 5, 6 (2017).

65. 65.

Shimmin, C. et al. Decorrelated jet substructure tagging using adversarial neural networks. Phys. Rev. D 96, 074034 (2017).

66. 66.

The ATLAS Collaboration. Quark versus Gluon Jet Tagging using Jet Images with the ATLAS Detector. Report No. ATL-PHYS-PUB-2017-017, https://cds.cern.ch/record/2275641 (CERN, 2017).

67. 67.

CMS Collaboration. New Developments for Jet Substructure Reconstruction in CMS. Report No. CMS-DP-2017-027, https://cds.cern.ch/record/2275226 (CERN, 2017).

68. 68.

NOvA Collaboration. The NOvA Technical Design Report. Report No. FERMILAB-DESIGN-2007-01 (FNAL, 2007)

69. 69.

Szegedy, C. et al. Going deeper with convolutions. In 2015 IEEE Conference on Computer Vision and Pattern Recognition 1–9 (IEEE, 2015).

70. 70.

Adamson, P. et al. Constraints on oscillation parameters from v e appearance and v μ disappearance in NOvA. Phys. Rev. Lett. 118, 231801 (2017).

71. 71.

Acciarri, R. et al. Design and construction of the MicroBooNE detector. J. Instrum. 12, P02017 (2017).

72. 72.

Adamson, P. et al. Search for active-sterile neutrino mixing using neutral-current interactions in NOvA. Phys. Rev. D 96, 072006 (2017).

73. 73.

Acciarri, R. et al. Convolutional neural networks applied to neutrino events in a liquid argon time projection chamber. J. Instrum. 12, P03011 (2017).

74. 74.

Ren, S., He, K., Girshick, R. & Sun, J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 29, 061137 (2017).

75. 75.

ATLAS Collaboration. Performance of b-jet identification in the ATLAS experiment. J. Instrum. 11, P04008 (2016).

76. 76.

CMS Collaboration. CMS Phase 1 Heavy Flavour Identification Performance and Developments. Report No. CERN-CMS-DP-2017-013, https://cds.cern.ch/record/2263802 (CERN, 2017).

77. 77.

ATLAS Collaboration. Identification of Jets Containing b-Hadrons with Recurrent Neural Networks at the ATLAS Experiment. Report No. ATL-PHYS-PUB-2017-003, https://cds.cern.ch/record/2255226 (CERN, 2017).

78. 78.

ATLAS Collaboration. Optimisation and Performance Studies of the ATLAS b-Tagging Algorithms for the 2017-18 LHC Run. Report No. ATL-PHYS-PUB-2017-013, https://cds.cern.ch/record/2273281 (CERN, 2017).

79. 79.

CMS Collaboration. Heavy Flavor Identification at CMS with Deep Neural Networks. Report No. CMS-DP-2017-005, https://cds.cern.ch/record/2255736 (CERN, 2017).

80. 80.

Guest, D. et al. Jet flavor classification in high-energy physics with deep neural networks. Phys. Rev. D 94, 112002 (2016).

81. 81.

Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. Nature 323, 533–536 (1986).

82. 82.

ATLAS Collaboration. Electron efficiency measurements with the ATLAS detector using 2012 LHC proton–proton collision data. Eur. Phys. J. C 77, 195 (2017).

83. 83.

The CMS Collaboration. Performance of electron reconstruction and selection with the CMS detector in proton-proton collisions at √s = 8 TeV. J. Instrum. 10, P06005 (2015).

84. 84.

Sachdev, K. Muon Neutrino to Electron Neutrino Oscillation in NOvA. PhD thesis, Univ. Minnesota (2015).

85. 85.

Chatrchyan, S. et al. Evidence for the 125 GeV Higgs boson decaying to a pair of τ leptons. J. High Energy Phys. 5, 104 (2014).

86. 86.

van der Maaten, L. Accelerating t-SNE using tree- based algorithms. J. Mach. Learn. Res. 15, 3221–3245 (2014).

87. 87.

Louppe, G., Cho, K., Becot, C. & Cranmer, K. QCD-aware recursive neural networks for jet physics. Preprint at https://arxiv.org/abs/1702.00748 (2017).

88. 88.

Goodfellow, I. J. et al. Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27, 2672–2680 (2014).

89. 89.

Rezende, D.J., Mohamed, S. & Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. J. Mach. Learn. Res. Worksh. Conf. Proc. 32, 1278–1286 (2014).

90. 90.

Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. Preprint at https://arxiv.org/abs/1312.6114 (2014).

91. 91.

Paganini, M., de Oliveira, L. & Nachman, B. Accelerating science with generative adversarial networks: an application to 3D particle showers in multilayer calorimeters. Phys. Rev. Lett. 120, 042003 (2018).

92. 92.

Carminati, F. et al. Calorimetry with deep learning: particle classification, energy regression, and simulation for high-energy physics. In NIPS Deep Learning for Physical Sciences Workshop (NIPS, 2017).

93. 93.

Louppe, G., Kagan, M. & Cranmer, K. Learning to pivot with adversarial networks. Adv. Neural Inf. Process. Syst. 30, 981–990 (2017). This paper forms part of a collection of works that present a more nuanced loss function and, along with similar work for BDTs (see section ‘Conclusions and outlook’), could lead to a new paradigm for training machine-learning models in high-energy physics.

94. 94.

Stevens, J. & Williams, M. uBoost: a boosting method for producing uniform selection efficiencies from multivariate classifiers. J. Instrum. 8, P12013 (2013).

95. 95.

Rogozhnikov, A., Bukva, A., Gligorov, V. V., Ustyuzhanin, A. & Williams, M. New approaches for boosting to uniformity. J. Instrum. 10, T03002 (2015).

96. 96.

Dery, L. M., Nachman, B., Rubbo, F. & Schwartzman, A. Weakly supervised classification in high energy physics. J. High Energy Phys. 5, 145 (2017).

97. 97.

Baldi, P., Cranmer, K., Faucett, T., Sadowski, P. & Whiteson, D. Parameterized neural networks for high-energy physics. Eur. Phys. J. C 76, 235 (2016).

98. 98.

Aaij, R. et al. Search for hidden-sector bosons in B 0K *0 μ + μ decays. Phys. Rev. Lett. 115, 161802 (2015).

99. 99.

The ATLAS collaboration. Search for the $$b\bar{b}$$ bb decay of the standard model Higgs boson in associated (W/Z)H production with the ATLAS detector. J. High Energy Phys. 1, 69 (2015).

100. 100.

Chatrchyan, S. et al. Search for the standard model Higgs boson produced in association with a W or a Z boson and decaying to bottom quarks. Phys. Rev. D 89, 012003 (2014).

101. 101.

CMS Collaboration. 2015 ECAL Detector Performance Plots. Report No. CMS-DP-2015-057, https://cds.cern.ch/record/2114735 (CERN, 2015).

### Reviewer information

Nature thanks C. Backhouse, M. Pierini and the other anonymous reviewer(s) for their contribution to the peer review of this work.

## Author information

All authors contributed to writing this Review. A.R. and M.W. were the principal editors.

### Competing interests

The authors declare no competing interests.

Correspondence to Alexander Radovic or Mike Williams.

## Rights and permissions

Reprints and Permissions

• #### DOI

https://doi.org/10.1038/s41586-018-0361-2

• ### Deep Neural Network Inverse Design of Integrated Photonic Power Splitters

• , Keisuke Kojima
• , Toshiaki Koike-Akino
• , Devesh Jha
• , Bingnan Wang
• , Chungwei Lin
•  & Kieran Parsons

Scientific Reports (2019)

• ### Energy flow networks: deep sets for particle jets

• Patrick T. Komiske
• , Eric M. Metodiev
•  & Jesse Thaler

Journal of High Energy Physics (2019)