Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Autoencoders on field-programmable gate arrays for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider

Nature Machine Intelligence volume 4, pages 154–161 (2022)Cite this article

Subjects

An Author Correction to this article was published on 12 April 2022

This article has been updated

A preprint version of the article is available at arXiv.

Abstract

To study the physics of fundamental particles and their interactions, the Large Hadron Collider was constructed at CERN, where protons collide to create new particles measured by detectors. Collisions occur at a frequency of 40 MHz, and with an event size of roughly 1 MB it is impossible to read out and store the generated amount of data from the detector and therefore a multi-tiered, real-time filtering system is required. In this paper, we show how to adapt and deploy deep-learning-based autoencoders for the unsupervised detection of new physics signatures in the challenging environment of a real-time event selection system at the Large Hadron Collider. The first-stage filter, implemented on custom electronics, decides within a few microseconds whether an event should be kept or discarded. At this stage, the rate is reduced from 40 MHz to about 100 kHz. We demonstrate the deployment of an unsupervised selection algorithm on this custom electronics, running in as little as 80 ns and enhancing the signal-over-background ratio by three orders of magnitude. This work enables the practical deployment of these networks during the next data-taking campaign of the Large Hadron Collider.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Model performance at floating-point precision.
Fig. 2: Model performance at floating-point precision.
Fig. 3: Compressed model performance.

Similar content being viewed by others

Data availability

The data used in this study are openly available at Zenodo57,58,59,60,62.

Code availability

The QKeras library is available at github.com/google/qkeras, where the work presented here is using QKeras version 0.9.0. The hls4ml library with custom layers used in the paper are under AE_L1_paper branch and available at https://github.com/fastmachinelearning/hls4ml/tree/AE_L1_paper.

Change history

References

  1. LHC Machine. JINST 3, S08001 (2008).

  2. Aad, G. et al. The ATLAS Experiment at the CERN Large Hadron Collider. J. Instrum. 3, S08003 (2008).

  3. Chatrchyan, S. et al. The CMS Experiment at the CERN LHC. J. Instrum. 3, S08004 (2008).

  4. Sirunyan, A. M. et al. Performance of the CMS Level-1 trigger in proton-proton collisions at \(\sqrt{s}=\) 13 TeV. J. Instrum. 15, P10017 (2020).

    Article  Google Scholar 

  5. The Phase-2 upgrade of the CMS Level-1 trigger. CMS Technical Design Report CERN-LHCC-2020-004 CMS-TDR-021 (2020).

  6. Aad, G. et al. Operation of the ATLAS trigger system in Run 2. J. Instrum. 15, P10004 (2020).

    Article  Google Scholar 

  7. Technical Design Report for the Phase-II Upgrade of the ATLAS TDAQ System. ATLAS Technical Design Report CERN-LHCC-2017-020 ATLAS-TDR-029 (2017).

  8. Aad, G. et al. Observation of a new particle in the search for the standard model Higgs boson with the ATLAS detector at the LHC. Phys. Lett. B 716, 1 (2012).

    Article  Google Scholar 

  9. Chatrchyan, S. et al. Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC. Phys. Lett. B 716, 30 (2012).

    Article  Google Scholar 

  10. Aarrestad, T. et al. The dark machines anomaly score challenge: Benchmark data and model independent event classification for the large hadron collider. SciPost Phys. 12, 2542 (2022).

    Article  Google Scholar 

  11. Kasieczka, G. et al. The LHC olympics 2020: A community challenge for anomaly detection in high energy physics. Rep. Prog. Phys. 84, 124201 (2021).

  12. Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. Preprint at https://arxiv.org/abs/1312.6114 (2014).

  13. Rezende, D. J., Mohamed, S. & Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. Preprint at https://arxiv.org/abs/1401.4082 (2014).

  14. Heimel, T., Kasieczka, G., Plehn, T. & Thompson, J. M. QCD or What? SciPost Phys. 6, 30 (2019).

    Article  Google Scholar 

  15. Farina, M., Nakai, Y. & Shih, D. Searching for new physics with deep autoencoders. Phys. Rev. D 101, 075021 (2020).

    Article  Google Scholar 

  16. Cerri, O. et al. Variational autoencoders for new physics mining at the Large Hadron Collider. J. High Eenergy Phys. 2019, 36 (2019).

  17. Knapp, O. et al. Adversarially Learned Anomaly Detection on CMS Open Data: re-discovering the top quark. Eur. Phys. J. Plus 136, 236 (2021).

    Article  Google Scholar 

  18. Venieris, S. I., Kouris, A. & Bouganis, C.-S. Toolflows for mapping convolutional neural networks on FPGAs: A survey and future directions. Preprint at https://arxiv.org/abs/1803.05900 (2018).

  19. Guo, K., Zeng, S., Yu, J., Wang, Y. & Yang, H. A survey of FPGA-based neural network inference accelerators. https://arxiv.org/abs/1712.08934 (2019).

  20. Shawahna, A., Sait, S. M. & El-Maleh, A. FPGA-based accelerators of deep learning networks for learning and classification: a review. IEEE Access 7, 7823 (2019).

    Article  Google Scholar 

  21. Umuroglu, Y. et al. FINN: A framework for fast, scalable binarized neural network inference. In Proc. 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays 65 (ACM, 2017).

  22. Blott, M. et al. FINN-R: An end-to-end deep-learning framework for fast exploration of quantized neural networks. Preprint at https://arxiv.org/abs/1809.04570 (2018).

  23. Summers, S. et al. Fast inference of boosted decision trees in FPGAs for particle physics. J. Instrum. 15, P05026 (2020).

    Article  Google Scholar 

  24. Hong, T. M. et al. Nanosecond machine learning event classification with boosted decision trees in FPGA for high energy physics. J. Instrum. 16, P08016 (2021).

  25. Duarte, J. et al. Fast inference of deep neural networks in FPGAs for particle physics. J. Instrum. 13, P07027 (2018).

  26. Ngadiuba, J. et al. Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML. Mach. Learn. Sci. Technol. 2, 2632 (2020).

    Article  Google Scholar 

  27. Iiyama, Y. et al. Distance-weighted graph neural networks on FPGAs for real-time particle reconstruction in high energy physics. Front. Big Data 3, 598927 (2020).

    Article  Google Scholar 

  28. Aarrestad, T. et al. Fast convolutional neural networks on FPGAs with HLS4ML. Mach. Learn. Sci. Technol. 2, 045015 (2021).

    Article  Google Scholar 

  29. Heintz, A. et al. Accelerated charged particle tracking with graph neural networks on FPGAs. In 34th Conference on Neural Information Processing Systems (2020).

  30. Summers, S. et al. Fast inference of boosted decision trees in FPGAs for particle physics. J. Instrum. 15, P05026 (2020).

    Article  Google Scholar 

  31. Coelho, C. Qkeras https://github.com/google/qkeras (2019).

  32. Coelho, C. N. et al. Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors. Nat. Mach. Intell. 3, 675–686 (2021).

  33. D’Agnolo, R. T. & Wulzer, A. Learning new physics from a machine. Phys. Rev. D 99, 015014 (2019).

    Article  Google Scholar 

  34. Mikuni, V., Nachman, B. & Shih, D. Online-compatible unsupervised non-resonant anomaly detection. Preprint at https://arxiv.org/abs/2111.06417 (2021).

  35. LeCun, Y., Denker, J. S. & Solla, S. A. Optimal brain damage. In Advances in Neural Information Processing Systems (ed. Touretzky, D. S.) Vol. 2, 598 (Morgan-Kaufmann, 1990).

  36. Han, S., Mao, H. & Dally, W. J. Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. In 4th Int. Conf. Learning Representations (ed. Bengio, Y. & LeCun, Y.) (2016).

  37. Blalock, D., Ortiz, J. J. G., Frankle, J. & Guttag, J. What is the state of neural network pruning? In Proc. Machine Learning and Systems Vol. 2, 129 (2020).

  38. Moons, B., Goetschalckx, K., Berckelaer, N. V. & Verhelst, M. Minimum energy quantized neural networks. In 2017 51st Asilomar Conf. Signals, Systems, and Computers (ed. Matthews, M. B.) 1921 (2017).

  39. Courbariaux, M., Bengio, Y. & David, J.-P. BinaryConnect: Training deep neural networks with binary weights during propagations. In Adv. Neural Information Processing Systems (eds. Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M. & Garnett, R.) Vol. 28, 3123 (Curran Associates, 2015).

  40. Zhang, D., Yang, J., Ye, D. & Hua, G. LQ-nets: Learned quantization for highly accurate and compact deep neural networks. In Proc. European Conference on Computer Vision (eds. Ferrari, V., Hebert, M., Sminchisescu, C. & Weiss, Y.) (2018).

  41. Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R. & Bengio, Y. Quantized neural networks: training neural networks with low precision weights and activations. J. Mach. Learn. Res. 18, 6869–6898 (2018).

    MathSciNet  MATH  Google Scholar 

  42. Rastegari, M., Ordonez, V., Redmon, J. & Farhadi, A. XNOR-Net: ImageNet classification using binary convolutional neural networks. In 14th European Conf. Computer Vision 525 (Springer, 2016).

  43. Micikevicius, P. et al. Mixed precision training. In 6th Int. Conf. Learning Representations (2018).

  44. Zhuang, B., Shen, C., Tan, M., Liu, L. & Reid, I. Towards effective low-bitwidth convolutional neural networks. In 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition 7920 (2018).

  45. Wang, N., Choi, J., Brand, D., Chen, C.-Y. & Gopalakrishnan, K. Training deep neural networks with 8-bit floating point numbers. In Adv. Neural Information Processing Systems (eds. Bengio, S. et al.) Vol. 31, 7675 (Curran Associates, 2018).

  46. An, J. & Cho, S. Variational autoencoder based anomaly detection using reconstruction probability. Special Lecture IE 2, 1–18 (2015).

    Google Scholar 

  47. Nagel, M., van Baalen, M., Blankevoort, T. & Welling, M. Data-free quantization through weight equalization and bias correction. In 2019 IEEE/CVF International Conf. Computer Vision 1325 (2019).

  48. Meller, E., Finkelstein, A., Almog, U. & Grobman, M. Same, same but different: Recovering neural network quantization error through weight factorization. In Proc. 36th International Conf. Machine Learning (eds. Chaudhuri, K. & Salakhutdinov, R.) Vol. 97, 4486 (PMLR, 2019).

  49. Zhao, R., Hu, Y., Dotzel, J., Sa, C. D. & Zhang, Z. Improving neural network quantization without retraining using outlier channel splitting. In Proc. 36th Int. Conference on Machine Learning (eds. Chaudhuri, K. & Salakhutdinov, R.) Vol. 97, 7543 (PMLR, 2019).

  50. Banner, R., Nahshan, Y., Hoffer, E. & Soudry, D. Post-training 4-bit quantization of convolution networks for rapid-deployment. In Adv. Neural Information Processing Systems (eds. Wallach, H. et al.) Vol. 32, 7950 (Curran Associates, 2019).

  51. Pappalardo, A. brevitas https://github.com/Xilinx/brevitas (2020).

  52. Shin, S., Boo, Y. & Sung, W. Knowledge distillation for optimization of quantized deep neural networks. In 2020 IEEE Workshop on Signal Processing Systems (2020).

  53. Polino, A., Pascanu, R. & Alistarh, D. Model compression via distillation and quantization. In Int. Conf. Learning Representations (2018).

  54. Gao, M. et al. An embarrassingly simple approach for knowledge distillation. Preprint at https://arxiv.org/abs/1812.01819 (2019).

  55. Mishra, A. & Marr, D. Apprentice: using knowledge distillation techniques to improve low-precision network accuracy. In Int. Conf. Learning Representations (2018).

  56. Nguyen, T. Q. et al. Topology classification with deep learning to improve real-time event selection at the LHC. Comput. Softw. Big Sci. 3, 12 (2019).

    Article  Google Scholar 

  57. Govorkova, E. et al. Unsupervised new physics detection at 40 mhz: LQ → b τ signal benchmark dataset. Zenodo https://doi.org/10.5281/zenodo.5055454 (2021).

  58. Govorkova, E. et al. Unsupervised new physics detection at 40 mhz: A → 4 leptons signal benchmark dataset. Zenodo https://doi.org/10.5281/zenodo.5046446 (2021).

  59. Govorkova, E. et al. Unsupervised new physics detection at 40 mhz: h0 → ττ signal benchmark dataset. Zenodo https://doi.org/10.5281/zenodo.5061633 (2021).

  60. Govorkova, E. et al. Unsupervised new physics detection at 40 mhz: h+ → τν signal benchmark dataset. Zenodo https://doi.org/10.5281/zenodo.5061688 (2021).

  61. Govorkova, E. et al. LHC physics dataset for unsupervised new physics detection at 40 MHz. Preprint at https://arxiv.org/abs/2107.02157 (2021).

  62. Govorkova, E. et al. Unsupervised new physics detection at 40 mhz: training dataset. Zenodo https://doi.org/10.5281/zenodo.5046389 (2021).

  63. Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proc. 32nd International Conference on Machine Learning (eds. Bach, F. & Blei, D.) Vol. 37, 448 (PMLR, 2015).

  64. Maas, A. L., Hannun, A. Y. & Ng, A. Y. Rectifier nonlinearities improve neural network acoustic models. In ICML Workshop on Deep Learning for Audio, Speech and Language Processing (2013).

  65. Nair, V. & Hinton, G. E. Rectified linear units improve restricted boltzmann machines. In ICML (eds. Fürnkranz, J. & Joachims, T.) 807 (Omnipress, 2010).

  66. Kingma, D. P. & Ba, J. Adam: A Method for Stochastic Optimization. Preprint at https://arxiv.org/abs/1412.6980 (2014).

  67. Joyce, J. M. in International Encyclopedia of Statistical Science 720–722 (Springer, 2011); https://doi.org/10.1007/978-3-642-04898-2_327

  68. Higgins, I. et al. beta-vae: Learning basic visual concepts with a constrained variational framework (2016).

  69. Chollet, F. et al. Keras https://keras.io (2015).

  70. Xilinx. Vivado design suite user guide: High-level synthesis. https://www.xilinx.com/support/documentation/sw_manuals/xilinx2020_1/ug902-vivado-high-level-synthesis.pdf (2020).

  71. EMP Collaboration. emp-fwk homepage. https://serenity.web.cern.ch/serenity/emp-fwk/ (2019).

  72. D’Agnolo, R. T. & Wulzer, A. Learning new physics from a machine. Phys. Rev. D 99, 015014 (2019).

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement no. 772369) and the ERC-POC programme (grant no. 996696).

Author information

Authors and Affiliations

  1. European Organization for Nuclear Research (CERN), Geneva, Switzerland

    Ekaterina Govorkova, Ema Puljak, Thea Aarrestad, Thomas James, Vladimir Loncar, Maurizio Pierini, Adrian Alan Pol, Nicolò Ghielmetti, Maksymilian Graczyk & Sioni Summers

  2. Institute of Physics Belgrade, Belgrade, Serbia

    Vladimir Loncar

  3. Politecnico di Milano, Milan, Italy

    Nicolò Ghielmetti

  4. Imperial College London, London, UK

    Maksymilian Graczyk

  5. Fermi National Accelerator Laboratory, Batavia, USA

    Jennifer Ngadiuba

  6. California Institute of Technology, Pasadena, USA

    Jennifer Ngadiuba & Thong Q. Nguyen

  7. University of California San Diego, La Jolla, USA

    Javier Duarte

  8. University of Illinois at Chicago, Chicago, USA

    Zhenbin Wu

Authors
  1. Ekaterina Govorkova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ema Puljak
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thea Aarrestad
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas James
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vladimir Loncar
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Maurizio Pierini
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Adrian Alan Pol
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Nicolò Ghielmetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Maksymilian Graczyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Sioni Summers
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Jennifer Ngadiuba
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Thong Q. Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Javier Duarte
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Zhenbin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

V.L., M.P., A.A.P., N.G., M.G., S.S., J.D. and Z.W. conceived and designed the hls4ml software library. M.P., T.Q.N. and Z.W. designed and prepared the dataset format. E.G., E.P., T.A., T.J., V.L., M.P., J.N., T.Q.N. and Z.W. designed and implemented autoencoders in hls4ml. E.G., E.P., T.A., T.J., M.P. and J.D. wrote the paper.

Corresponding author

Correspondence to Ekaterina Govorkova.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks the anonymous reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Network architectures.

Network architecture for the DNN AE (top) and CNN AE (bottom) models. The corresponding VAE models are derived introducing the Gaussian sampling in the latent space, for the same encoder and decoder architectures (see text).

Extended Data Fig. 2 TPR ratios for different bit width.

TPR ratios versus model bit width for the VAE CNN (left) and DNN (right) models tested on four new physics benchmark models, using DKL as figure of merit for PTQ (top) and QAT (bottom) strategies.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Govorkova, E., Puljak, E., Aarrestad, T. et al. Autoencoders on field-programmable gate arrays for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider. Nat Mach Intell 4, 154–161 (2022). https://doi.org/10.1038/s42256-022-00441-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-022-00441-3

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics