The prospect of achieving quantum advantage with quantum neural networks (QNNs) is exciting. Understanding how QNN properties (for example, the number of parameters M) affect the loss landscape is crucial to designing scalable QNN architectures. Here we rigorously analyze the overparametrization phenomenon in QNNs, defining overparametrization as the regime where the QNN has more than a critical number of parameters Mc allowing it to explore all relevant directions in state space. Our main results show that the dimension of the Lie algebra obtained from the generators of the QNN is an upper bound for Mc, and for the maximal rank that the quantum Fisher information and Hessian matrices can reach. Underparametrized QNNs have spurious local minima in the loss landscape that start disappearing when M ≥ Mc. Thus, the overparametrization onset corresponds to a computational phase transition where the QNN trainability is greatly improved. We then connect the notion of overparametrization to the QNN capacity, so that when a QNN is overparametrized, its capacity achieves its maximum possible value.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
Quantum Machine Intelligence Open Access 29 September 2023
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$99.00 per year
only $8.25 per issue
Rent or buy this article
Prices vary by article type
Prices may be subject to local taxes which are calculated during checkout
All data were generated by simulating quantum circuits with the open-source library Qibo35. The dataset utilized in this study was taken from the NTangled dataset48 and can be downloaded from GitHub at https://github.com/LSchatzki/NTangled_Datasets. Source data are available with this paper and in ref. 49.
All code to generate the figures and analyses in this study is publicly available from ref. 49.
Mohri, M., Rostamizadeh, A. & Talwalkar, A. Foundations of Machine Learning (MIT Press, 2018).
Vamathevan, J. et al. Applications of machine learning in drug discovery and development. Nat. Rev. Drug Discov. 18, 463–477 (2019).
Schmidt, J., Marques, M. R., Botti, S. & Marques, M. A. Recent advances and applications of machine learning in solid-state materials science. npj Comput. Mater. 5, 83 (2019).
Blum, A. L. & Rivest, R. L. Training a 3-node neural network is NP-complete. Neural Netw. 5, 117–127 (1992).
Neyshabur, B., Li, Z., Bhojanapalli, S., LeCun, Y., & Srebro, N. Towards understanding the role of over-parametrization in generalization of neural networks. Preprint at arXiv https://doi.org/10.48550/arXiv.1805.12076 (2018).
Zhang, C., Bengio, S., Hardt, M., Recht, B. & Vinyals, O. Understanding deep learning (still) requires rethinking generalization. Commun. ACM 64, 107–115 (2021).
Allen-Zhu, Z., Li, Y. & Song, Z. A convergence theory for deep learning via over-parameterization. In Proceedings of the 36th International Conference on Machine Learning 242–252 (PMLR, 2019).
Du, S. S., Zhai, X., Poczos, B., & Singh, A. Gradient descent provably optimizes over-parameterized neural networks. Preprint at arXiv https://doi.org/10.48550/arXiv.1810.02054 (2018).
Preskill, J. Quantum computing in the NISQ era and beyond. Quantum 2, 79 (2018).
Cerezo, M. et al. Variational quantum algorithms. Nat. Rev. Phys. 3, 625–644 (2021).
Bharti, K. et al. Noisy intermediate-scale quantum algorithms. Rev. Mod. Phys. 94, 015004 (2022).
Huang, H.-Y. et al. Power of data in quantum machine learning. Nat. Commun. 12, 2631 (2021).
Abbas, A. et al. The power of quantum neural networks. Nat. Comput. Sci. 1, 403–409 (2021).
Bittel, L. & Kliesch, M. Training variational quantum algorithms is NP-hard. Phys. Rev. Lett. 127, 120502 (2021).
Wierichs, D., Gogolin, C. & Kastoryano, M. Avoiding local minima in variational quantum eigensolvers with the natural gradient optimizer. Phys. Rev. Res. 2, 043246 (2020).
Anschuetz, E. R. Critical points in quantum generative models. Preprint at arXiv https://doi.org/10.48550/arXiv.2109.06957 (2021).
McClean, J. R., Boixo, S., Smelyanskiy, V. N., Babbush, R. & Neven, H. Barren plateaus in quantum neural network training landscapes. Nat. Commun. 9, 4812 (2018).
Cerezo, M., Sone, A., Volkoff, T., Cincio, L. & Coles, P. J. Cost function dependent barren plateaus in shallow parametrized quantum circuits. Nat. Commun. 12, 1791 (2021).
Wang, S. et al. Noise-induced barren plateaus in variational quantum algorithms. Nat. Commun. 12, 6961 (2021).
Stilck França, D. & Garcia-Patron, R. Limitations of optimization algorithms on noisy quantum devices. Nat. Phys. 17, 1221–1227 (2021).
Wiersema, R. et al. Exploring entanglement and optimization within the Hamiltonian variational ansatz. PRX Quantum 1, 020319 (2020).
Kiani, B. T., Lloyd, S. & Maity, R. Learning unitaries by gradient descent. Preprint at arXiv https://doi.org/10.48550/arXiv.2001.11897 (2020).
Funcke, L. et al. Best-approximation error for parametric quantum circuits. In 2021 IEEE International Conference on Web Services (ICWS) 693–702 (IEEE, 2021).
Lee, J., Magann, A. B., Rabitz, H. A. & Arenz, C. Progress toward favorable landscapes in quantum combinatorial optimization. Phys. Rev. A 104, 032401 (2021).
Zeier, R. & Schulte-Herbrüggen, T. Symmetry principles in quantum systems theory. J. Math. Phys. 52, 113510 (2011).
Meyer, J. J. Fisher information in noisy intermediate-scale quantum applications. Quantum 5, 539 (2021).
Moore, K. W. & Rabitz, H. Exploring constrained quantum control landscapes. J. Chem. Phys. 137, 134113 (2012).
Larocca, M., Calzetta, E. & Wisniacki, D. A. Exploiting landscape geometry to enhance quantum optimal control. Phys. Rev. A 101, 023410 (2020).
Wu, R.-B., Long, R., Dominy, J., Ho, T.-S. & Rabitz, H. Singularities of quantum control landscapes. Phys. Rev. A 86, 013405 (2012).
Rach, N., Müller, M. M., Calarco, T. & Montangero, S. Dressing the chopped-random-basis optimization: a bandwidth-limited access to the trap-free landscape. Phys. Rev. A 92, 062343 (2015).
Larocca, M., Poggi, P. M. & Wisniacki, D. A. Quantum control landscape for a two-level system near the quantum speed limit. J. Phys. A Math. Theor. 51, 385305 (2018).
Larocca, M. et al. Diagnosing barren plateaus with tools from quantum optimal control. Quantum 6, 824 (2022).
Haug, T., Bharti, K. & Kim, M. S. Capacity and quantum geometry of parametrized quantum circuits. PRX Quantum 2, 040309 (2021).
Larocca, M., Calzetta, E. & Wisniacki, D. Fourier compression: a customization method for quantum control protocols. Phys. Rev. A 102, 033108 (2020).
Efthymiou, S. et al. Qibo: a framework for quantum simulation with hardware acceleration. Quantum Sci. Technol. 7, 015018 (2021).
Peruzzo, A. et al. A variational eigenvalue solver on a photonic quantum processor. Nat. Commun. 5, 4213 (2014).
García-Martín, D., Larocca, M. & Cerezo, M. Effects of noise on the overparametrization of quantum neural networks. Preprint at arXiv https://doi.org/10.48550/arXiv.2302.05059 (2023).
Fukumizu, K. A regularity condition of the information matrix of a multilayer perceptron network. Neural Netw. 9, 871–879 (1996).
Chan, N. & Kwong, M. K. Hermitian matrix inequalities and a conjecture. Am. Math. Monthly 92, 533–541 (1985).
Glaser, S. J. et al. Training Schrödinger’s cat: quantum optimal control. Eur. Phys. J. D 69, 279 (2015).
Rembold, P. et al. Introduction to quantum optimal control for quantum sensing with nitrogen-vacancy centers in diamond. AVS Quantum Sci. 2, 024701 (2020).
Ebadi, S. et al. Quantum phases of matter on a 256-atom programmable quantum simulator. Nature 595, 227–232 (2021).
Magann, A. B., Rudinger, K. M., Grace, M. D. & Sarovar, M. Feedback-based quantum optimization. Phys. Rev. Lett. 129, 250502 (2022).
Magann, A. B. et al. From pulses to circuits and back again: a quantum optimal control perspective on variational quantum algorithms. PRX Quantum 2, 010101 (2021).
Hsieh, M., Wu, R. & Rabitz, H. Topology of the quantum control landscape for observables. J. Chem. Phys. 130, 104109 (2009).
Ho, T.-S., Dominy, J. & Rabitz, H. Landscape of unitary transformations in controlled quantum dynamics. Phys. Rev. A 79, 013422 (2009).
Riviello, G. et al. Searching for quantum optimal control fields in the presence of singular critical points. Phys. Rev. A 90, 013404 (2014).
Schatzki, L., Arrasmith, A., Coles, P. J. & Cerezo, M. Entangled datasets for quantum machine learning. Preprint at arXiv https://doi.org/10.48550/arXiv.2109.03400 (2021).
García-Martín, D. DiegoGM91/theory-of-overparametrization: v0.0.1. Zenodo https://doi.org/10.5281/zenodo.7916659 (2023).
We thank P. de Niverville, J. Nakhleh, S. Efthymiou, L. Schatzki, M. Farinati and Z. Szabo for useful conversations. N.J. and D.G.-M. were supported by the US DOE through a quantum computing program sponsored by the Los Alamos National Laboratory (LANL) Information Science & Technology Institute. D.G.-M. acknowledges partial financial support from project QuantumCAT (ref. 001-P-001644), co-funded by the Generalitat de Catalunya and the European Union Regional Development Fund within the ERDF Operational Program of Catalunya, and from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. 951911 (AI4Media). P.J.C. and M.C. were initially supported by the Laboratory Directed Research and Development (LDRD) program of LANL under project no. 20190065DR. P.J.C. also acknowledges support from the LANL ASC Beyond Moore’s Law project. M.C. also acknowledges support from the Center for Nonlinear Studies at LANL. This work was supported by the US DOE, Office of Science, Office of Advanced Scientific Computing Research, under the Accelerated Research in Quantum Computing (ARQC) program.
The authors declare no competing interests.
Peer review information
Nature Computational Science thanks the anonymous reviewers for their contribution to the peer review of this work. Peer reviewer reports are available. Primary Handling Editor: Jie Pan, in collaboration with the Nature Computational Science team.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Sections 1–8 and Figs. 1–5.
Supplementary Section 7 and Fig. 1. This folder contains data from the VQE simulations. Each file contains an array with the eigenvalues of the quantum Fisher information or Hessian matrices.
Supplementary Section 7 and Fig. 2. This folder contains data from the VQE simulations. Each file contains an array with the eigenvalues of the quantum Fisher information matrix.
Supplementary Section 7 and Fig. 4. This folder contains data from the unitary compilation simulations. Each file contains either an array of cost function values for every optimization step, or an array with the eigenvalues of the quantum Fisher information or Hessian matrices.
Supplementary Section 7 and Fig. 5. This folder contains data from the autoencoder simulations. Each file contains either an array of cost function values for every optimization step, or an array with the eigenvalues of the quantum Fisher information matrix.
This folder contains data from the VQE simulations. Each file contains an array of cost function values for every optimization step.
This folder contains data from the VQE simulations. Each file contains an array with the eigenvalues of the quantum Fisher information or Hessian matrices.
This folder contains data from the VQE simulations in the presence of noise. Each file contains an array with the eigenvalues of the quantum Fisher information matrix or with the matrix itself.
About this article
Cite this article
Larocca, M., Ju, N., García-Martín, D. et al. Theory of overparametrization in quantum neural networks. Nat Comput Sci 3, 542–551 (2023). https://doi.org/10.1038/s43588-023-00467-6
This article is cited by
Quantum Machine Intelligence (2023)