Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

Language models for quantum simulation

Nature Computational Science volume 4, pages 11–18 (2024)Cite this article

Subjects

Abstract

A key challenge in the effort to simulate today’s quantum computing devices is the ability to learn and encode the complex correlations that occur between qubits. Emerging technologies based on language models adopted from machine learning have shown unique abilities to learn quantum states. We highlight the contributions that language models are making in the effort to build quantum computers and discuss their future role in the race to quantum advantage.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Autoregressive strategy for text and qubit sequences.
Fig. 2: RNNs for qubit sequences.
Fig. 3: Attention for text and qubit sequences.
Fig. 4: Language models enabling the scaling of quantum computing through a virtuous cycle.

Similar content being viewed by others

References

  1. Arute, F. et al. Quantum supremacy using a programmable superconducting processor. Nature 574, 505–510 (2019).

    Article  Google Scholar 

  2. Zhong, H.-S. et al. Quantum computational advantage using photons. Science 370, 1460–1463 (2020).

    Article  Google Scholar 

  3. Madsen, L. S. et al. Quantum computational advantage with a programmable photonic processor. Nature 606, 75–81 (2022).

    Article  Google Scholar 

  4. Kim, Y. et al. Evidence for the utility of quantum computing before fault tolerance. Nature 618, 500–505 (2023).

    Article  Google Scholar 

  5. Carrasquilla, J., Torlai, G., Melko, R. G. & Aolita, L. Reconstructing quantum states with generative models. Nat. Mach. Intell. 1, 155–161 (2019).

    Article  Google Scholar 

  6. Torlai, G. & Melko, R. G. Machine-learning quantum states in the nisq era. Annu. Rev. Condensed Matter Phys. 11, 325–344 (2020).

    Article  Google Scholar 

  7. Torlai, G. et al. Quantum process tomography with unsupervised learning and tensor networks. Nat. Commun. 14, 2858 (2023).

    Article  Google Scholar 

  8. Sweke, R., Kesselring, M. S., Nieuwenburg, E. P. L. & Eisert, J. Reinforcement learning decoders for fault-tolerant quantum computation. Mach. Learn. Sci. Technol. 2, 025005 (2020).

    Article  Google Scholar 

  9. Durrer, R. et al. Automated tuning of double quantum dots into specific charge states using neural networks. Phys. Rev. Appl. 13, 054019 (2020).

    Article  Google Scholar 

  10. Moon, H. et al. Machine learning enables completely automatic tuning of a quantum device faster than human experts. Nat. Commun. 11, 4161 (2020).

    Article  Google Scholar 

  11. Zwolak, J. P. & Taylor, J. M. Advances in automation of quantum dot devices control. Rev. Mod. Phys. 95, 011006 (2023).

    Article  Google Scholar 

  12. Czischek, S. et al. Miniaturizing neural networks for charge state autotuning in quantum dots. Mach. Learn. Sci. Technol. 3, 015001 (2021).

    Article  Google Scholar 

  13. Teoh, Y. H., Drygala, M., Melko, R. G. & Islam, R. Machine learning design of a trapped-ion quantum spin simulator. Quantum Sci. Technol. 5, 024001 (2020).

    Article  Google Scholar 

  14. Carrasquilla, J. Machine learning for quantum matter. Adv. Phys. X 5, 1797528 (2020).

    Google Scholar 

  15. Dawid, A. et al. Modern applications of machine learning in quantum sciences. Preprint at https://arxiv.org/abs/2204.04198 (2022).

  16. Melko, R. G., Carleo, G., Carrasquilla, J. & Cirac, J. I. Restricted Boltzmann machines in quantum physics. Nat. Phys. 15, 887–892 (2019).

    Article  Google Scholar 

  17. Torlai, G. & Melko, R. G. Learning thermodynamics with Boltzmann machines. Phys. Rev. B 94, 165134 (2016).

    Article  Google Scholar 

  18. Torlai, G. et al. Neural-network quantum state tomography. Nat. Phys. 14, 447–450 (2018).

    Article  Google Scholar 

  19. Carleo, G. & Troyer, M. Solving the quantum many-body problem with artificial neural networks. Science 355, 602–606 (2017).

    Article  MathSciNet  Google Scholar 

  20. OpenAI: GPT-4 technical report. Preprint at https://arxiv.org/abs/2303.08774 (2023).

  21. Born, M. Quantenmechanik der stoßvorgänge. Z. Phys. 38, 803–827 (1926).

    Article  Google Scholar 

  22. Wei, V., Coish, W. A., Ronagh, P. & Muschik, C. A. Neural-shadow quantum state tomography. Preprint at https://arxiv.org/abs/2305.01078 (2023).

  23. Islam, R. et al. Measuring entanglement entropy in a quantum many-body system. Nature 528, 77–83 (2015).

    Article  Google Scholar 

  24. Wu, D., Wang, L. & Zhang, P. Solving statistical mechanics using variational autoregressive networks. Phys. Rev. Lett. 122, 080602 (2019).

    Article  Google Scholar 

  25. Liu, J.-G., Mao, L., Zhang, P. & Wang, L. Solving quantum statistical mechanics with variational autoregressive networks and quantum circuits. Mach. Learn. Sci. Technol. 2, 025011 (2021).

    Article  Google Scholar 

  26. Xie, H., Zhang, L. & Wang, L. Ab-initio study of interacting fermions at finite temperature with neural canonical transformation. J. Mach. Learn. 1, 38–59 (2022).

    Article  Google Scholar 

  27. Wang, Z. & Davis, E. J. Calculating Rényi entropies with neural autoregressive quantum states. Phys. Rev. A 102, 062413 (2020).

    Article  MathSciNet  Google Scholar 

  28. Sharir, O., Levine, Y., Wies, N., Carleo, G. & Shashua, A. Deep autoregressive models for the efficient variational simulation of many-body quantum systems. Phys. Rev. Lett. 124, 020503 (2020).

    Article  Google Scholar 

  29. Barrett, T. D., Malyshev, A. & Lvovsky, A. I. Autoregressive neural-network wavefunctions for ab initio quantum chemistry. Nat. Mach. Intell. 4, 351–358 (2022).

    Article  Google Scholar 

  30. White, S. R. Density matrix formulation for quantum renormalization groups. Phys. Rev. Lett. 69, 2863–2866 (1992).

    Article  Google Scholar 

  31. Orús, R. Tensor networks for complex quantum systems. Nat. Rev. Phys. 1, 538–550 (2019).

    Article  Google Scholar 

  32. Pan, F. & Zhang, P. Simulation of quantum circuits using the big-batch tensor network method. Phys. Rev. Lett. 128, 030501 (2022).

  33. Tindall, J., Fishman, M., Stoudenmire, M. & Sels, D. Efficient tensor network simulation of IBM’s kicked Ising experiment. Preprint at https://arxiv.org/abs/2306.14887 (2023).

  34. Cramer, M. et al. Efficient quantum state tomography. Nat. Commun. 1, 149 (2010).

    Article  Google Scholar 

  35. Cross, A. W., Bishop, L. S., Sheldon, S., Nation, P. D. & Gambetta, J. M. Validating quantum computers using randomized model circuits. Phys. Rev. A 100, 032328 (2019).

    Article  Google Scholar 

  36. Levine, Y., Sharir, O., Cohen, N. & Shashua, A. Quantum entanglement in deep learning architectures. Phys. Rev. Lett. 122, 065301 (2019).

    Article  MathSciNet  Google Scholar 

  37. Deng, D.-L., Li, X. & Das Sarma, S. Quantum entanglement in neural network states. Phys. Rev. X 7, 021021 (2017).

    Google Scholar 

  38. Hibat-Allah, M., Ganahl, M., Hayward, L. E., Melko, R. G. & Carrasquilla, J. Recurrent neural network wave functions. Phys. Rev. Res. 2, 023358 (2020).

    Article  Google Scholar 

  39. Roth, C. Iterative retraining of quantum spin models using recurrent neural networks. Preprint at https://arxiv.org/abs/2003.06228 (2020).

  40. Elman, J. L. Finding structure in time. Cogn. Sci. 14, 179–211 (1990).

    Article  Google Scholar 

  41. Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9, 1735–1780 (1997).

    Article  Google Scholar 

  42. Lipton, Z. C., Berkowitz, J. & Elkan, C. A critical review of recurrent neural networks for sequence learning. Preprint at https://arxiv.org/abs/1506.00019 (2015).

  43. Morawetz, S., De Vlugt, I. J. S., Carrasquilla, J. & Melko, R. G. U(1)-symmetric recurrent neural networks for quantum state reconstruction. Phys. Rev. A 104, 012401 (2021).

    Article  MathSciNet  Google Scholar 

  44. Flurin, E., Martin, L. S., Hacohen-Gourgy, S. & Siddiqi, I. Using a recurrent neural network to reconstruct quantum dynamics of a superconducting qubit from physical observations. Phys. Rev. X 10, 011006 (2020).

    Google Scholar 

  45. Hibat-Allah, M., Melko, R. G. & Carrasquilla, J. Supplementing recurrent neural network wave functions with symmetry and annealing to improve accuracy. Preprint at https://arxiv.org/abs/2207.14314 (2022).

  46. Reh, M., Schmitt, M. & Gärttner, M. Optimizing design choices for neural quantum states. Phys. Rev. B 107, 195115 (2023).

    Article  Google Scholar 

  47. Hibat-Allah, M., Melko, R. G. & Carrasquilla, J. Investigating topological order using recurrent neural networks. Phys. Rev. B 108, 075152 (2023).

  48. Luo, D. et al. Gauge-invariant and anyonic-symmetric autoregressive neural network for quantum lattice models. Phys. Rev. Res. 5, 013216 (2023).

    Article  Google Scholar 

  49. Banchi, L., Grant, E., Rocchetto, A. & Severini, S. Modelling non-Markovian quantum processes with recurrent neural networks. New J. Phys. 20, 123030 (2018).

    Article  Google Scholar 

  50. Tang, Y., Weng, J. & Zhang, P. Neural-network solutions to stochastic reaction networks. Nat. Mach. Intell. 5, 376–385 (2023).

    Article  Google Scholar 

  51. Casert, C., Vieijra, T., Whitelam, S. & Tamblyn, I. Dynamical large deviations of two-dimensional kinetically constrained models using a neural-network state ansatz. Phys. Rev. Lett. 127, 120602 (2021).

    Article  MathSciNet  Google Scholar 

  52. Reh, M., Schmitt, M. & Gärttner, M. Time-dependent variational principle for open quantum systems with artificial neural networks. Phys. Rev. Lett. 127, 230501 (2021).

    Article  MathSciNet  Google Scholar 

  53. Lin, S.-H. & Pollmann, F. Scaling of neural-network quantum states for time evolution. Phys. Status Solidi B 259, 2100172 (2022).

    Article  Google Scholar 

  54. Donatella, K., Denis, Z., Boité, A. L. & Ciuti, C. Dynamics with autoregressive neural quantum states: application to critical quench dynamics. Phys. Rev. A 108, 022210 (2023).

  55. Baireuther, P., O’Brien, T. E., Tarasinski, B. & Beenakker, C. W. J. Machine-learning-assisted correction of correlated qubit errors in a topological code. Quantum 2, 48 (2018).

    Article  Google Scholar 

  56. Chamberland, C. & Ronagh, P. Deep neural decoders for near term fault-tolerant experiments. Quantum Sci. Technol. 3, 044002 (2018).

    Article  Google Scholar 

  57. Baireuther, P., Caio, M. D., Criger, B., Beenakker, C. W. J. & O’Brien, T. E. Neural network decoder for topological color codes with circuit level noise. New J. Phys. 21, 013003 (2019).

    Article  Google Scholar 

  58. Varsamopoulos, S., Bertels, K. & Almudever, C. G. Comparing neural network based decoders for the surface code. IEEE Trans. Comput. 69, 300–311 (2020).

    Article  MathSciNet  Google Scholar 

  59. Varbanov, B. M., Serra-Peralta, M., Byfield, D. & Terhal, B. M. Neural network decoder for near-term surface-code experiments. Preprint at https://arxiv.org/abs/2307.03280 (2023).

  60. Czischek, S., Moss, M. S., Radzihovsky, M., Merali, E. & Melko, R. G. Data-enhanced variational Monte Carlo simulations for Rydberg atom arrays. Phys. Rev. B 105, 205108 (2022).

    Article  Google Scholar 

  61. Moss, M. S. et al. Enhancing variational Monte Carlo using a programmable quantum simulator. Preprint at https://arxiv.org/abs/2308.02647 (2023).

  62. Wu, D., Rossi, R., Vicentini, F. & Carleo, G. From tensor-network quantum states to tensorial recurrent neural networks. Phys. Rev. Res. 5, L032001 (2023).

  63. Devlin, J., Chang, M., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT Vol. 1 (eds Burstein, J. et al.) 4171–4186 (Association for Computational Linguistics, 2019).

  64. Taylor, R. et al. Galactica: a large language model for science. Preprint at https://arxiv.org/abs/2211.09085 (2022).

  65. Wei, J. et al. Emergent abilities of large language models. Preprint at https://arxiv.org/abs/2206.07682 (2022).

  66. Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems 30 (eds Guyon, Y. et al.) 5998–6008 (NeurIPS, 2017).

  67. Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Commun. ACM 60, 84–90 (2017).

    Article  Google Scholar 

  68. Kaplan, J. et al. Scaling laws for neural language models. Preprint at https://arxiv.org/abs/2001.08361 (2020).

  69. Carrasquilla, J. et al. Probabilistic simulation of quantum circuits using a deep-learning architecture. Phys. Rev. A 104, 032610 (2021).

    Article  MathSciNet  Google Scholar 

  70. Cha, P. et al. Attention-based quantum tomography. Mach. Learn. Sci. Technol. 3, 01LT01 (2021).

    Article  Google Scholar 

  71. Wang, H., Weber, M., Izaac, J. & Lin, C.Y.-Y. Predicting properties of quantum systems with conditional generative models. Preprint at https://arxiv.org/abs/2211.16943 (2022).

  72. Ma, H., Sun, Z., Dong, D., Chen, C. & Rabitz, H. Tomography of quantum states from structured measurements via quantum-aware transformer. Preprint at https://arxiv.org/abs/2305.05433 (2023).

  73. Sprague, K. & Czischek, S. Variational Monte Carlo with large patched transformers. Preprint at https://arxiv.org/abs/2306.03921 (2023).

  74. Viteritti, L. L., Rende, R. & Becca, F. Transformer variational wave functions for frustrated quantum spin systems. Phys. Rev. Lett. 130, 236401 (2023).

    Article  Google Scholar 

  75. von Glehn, I., Spencer, J. S. & Pfau, D. A Self-attention ansatz for ab-initio quantum chemistry. Preprint at https://arxiv.org/abs/2211.13672 (2023).

  76. Wu, Y., Guo, C., Fan, Y., Zhou, P. & Shang, H. NNQS-transformer: an efficient and scalable neural network quantum states approach for ab initio quantum chemistry. Preprint at https://arxiv.org/abs/2306.16705 (2023).

  77. Neklyudov, K. et al. Wasserstein quantum Monte Carlo: a novel approach for solving the quantum many-body Schrödinger equation. Preprint at https://arxiv.org/abs/2307.07050 (2023).

  78. Sharir, O., Chan, G. K.-L. & Anandkumar, A. Towards neural variational Monte Carlo that scales linearly with system size. Preprint at https://arxiv.org/abs/2212.11296 (2022).

  79. Zhang, Y.-H. & Di Ventra, M. Transformer quantum state: a multipurpose model for quantum many-body problems. Phys. Rev. B 107, 075147 (2023).

    Article  Google Scholar 

  80. Gao, N. & Günnemann, S. Generalizing neural wave functions. Preprint at https://arxiv.org/abs/2302.04168 (2023).

  81. An, Z., Wu, J., Yang, M., Zhou, D. L. & Zeng, B. Unified quantum state tomography and Hamiltonian learning using transformer models: a language-translation-like approach for quantum systems. Preprint at https://arxiv.org/abs/2304.12010 (2023).

  82. Bennewitz, E. R., Hopfmueller, F., Kulchytskyy, B., Carrasquilla, J. & Ronagh, P. Neural error mitigation of near-term quantum simulations. Nat. Mach. Intell. 4, 618–624 (2022).

    Article  Google Scholar 

  83. Jastrow, R. Many-body problem with strong forces. Phys. Rev. 98, 1479–1484 (1955).

    Article  Google Scholar 

  84. Torlai, G. & Melko, R. G. Neural decoder for topological codes. Phys. Rev. Lett. 119, 030501 (2017).

    Article  MathSciNet  Google Scholar 

  85. Cao, H., Pan, F., Wang, Y. & Zhang, P. qecGPT: decoding quantum error-correcting codes with generative pre-trained transformers. Preprint at https://arxiv.org/abs/2307.09025 (2023).

  86. Bausch, J. et al. Learning to decode the surface code with a recurrent, transformer-based neural network. Preprint at https://arxiv.org/abs/2310.05900 (2023).

  87. Chiu, C. S. et al. String patterns in the doped Hubbard model. Science 365, 251–256 (2019).

    Article  MathSciNet  Google Scholar 

  88. Becca, F. & Sorella, S. Quantum Monte Carlo Approaches for Correlated Systems (Cambridge Univ. Press, 2017).

  89. Luo, D., Chen, Z., Carrasquilla, J. & Clark, B. K. Autoregressive neural network for simulating open quantum systems via a probabilistic formulation. Phys. Rev. Lett. 128, 090501 (2022).

    Article  MathSciNet  Google Scholar 

  90. Gutiérrez, I. L. & Mendl, C. B. Real time evolution with neural-network quantum states. Quantum 6, 627 (2022).

    Article  Google Scholar 

  91. Hibat-Allah, M., Inack, E. M., Wiersema, R., Melko, R. G. & Carrasquilla, J. Variational neural annealing. Nat. Mach. Intell. 3, 952–961 (2021).

  92. Vicentini, F., Rossi, R. & Carleo, G. Positive-definite parametrization of mixed quantum states with deep neural networks. Preprint at https://arxiv.org/abs/2206.13488 (2022).

Download references

Acknowledgements

We thank their many students and postdocs who have contributed to these ideas over the years. This work is supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Canadian Institute for Advanced Research (CIFAR) AI chair program. Research at Perimeter Institute is supported in part by the Government of Canada through the Department of Innovation, Science and Economic Development Canada and by the Province of Ontario through the Ministry of Economic Development, Job Creation and Trade.

Author information

Authors and Affiliations

  1. Department of Physics and Astronomy, University of Waterloo, Waterloo, Ontario, Canada

    Roger G. Melko & Juan Carrasquilla

  2. Perimter Institute for Theoretical Physics, Waterloo, Ontario, Canada

    Roger G. Melko

  3. Vector Institute, MaRS Centre, Toronto, Ontario, Canada

    Juan Carrasquilla

  4. Department of Physics, University of Toronto, Toronto, Ontario, Canada

    Juan Carrasquilla

Authors
  1. Roger G. Melko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Carrasquilla
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Both authors contributed equally to this work.

Corresponding author

Correspondence to Roger G. Melko.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Computational Science thanks the anonymous reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Jie Pan, in collaboration with the Nature Computational Science team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Melko, R.G., Carrasquilla, J. Language models for quantum simulation. Nat Comput Sci 4, 11–18 (2024). https://doi.org/10.1038/s43588-023-00578-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s43588-023-00578-0

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing