Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

A structural optimization algorithm with stochastic forces and stresses

A preprint version of the article is available at arXiv.

Abstract

We propose an algorithm for optimizations in which the gradients contain stochastic noise. This arises, for example, in structural optimizations when computations of forces and stresses rely on methods involving Monte Carlo sampling, such as quantum Monte Carlo or neural network states, or are performed on quantum devices that have intrinsic noise. Our proposed algorithm is based on the combination of two ingredients: an update rule derived from the steepest-descent method, and a staged scheduling of the targeted statistical error and step size, with position averaging. We compare it with commonly applied algorithms, including some of the latest machine learning optimization methods, and show that the algorithm consistently performs efficiently and robustly under realistic conditions. Applying this algorithm, we achieve full-degree optimizations in solids using ab initio many-body computations, by auxiliary-field quantum Monte Carlo with plane waves and pseudopotentials. A potential metastable structure in Si is discovered using density-functional calculations with synthetic noisy forces.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Comparison of the convergence of FSSD versus line-search and ML algorithms, in a Si phase transition problem.
Fig. 2: Convergence and performance in MoS2, for the FSSD and ML optimization algorithms.
Fig. 3: Illustration of SET, and the acceleration in optimization efficiency.
Fig. 4: A direct PW-AFQMC geometry optimization with the FSSD × SET algorithm.
Fig. 5: The metastable structure we discovered in Si.

Similar content being viewed by others

Data availability

All data in this Article were generated with Quantum Espresso 5.0.1 and/or plane-wave AFQMC v26-2020.04.02 developed in our group (unpublished). Source data for Figs. 1–4 are available at https://github.com/schen24wm/geoopt-srcdata and Zenodo48 and also available with this Article. Detailed parameters of the Cmca structure (Fig. 5) we found are available in Supplementary Section 5.

Code availability

Code of FSSD × SET is available at https://github.com/schen24wm/fssd-set and Zenodo49, and available as Supplementary Software 1 with this Article.

References

  1. Frenkel, D. & Smit, B. Understanding Molecular Simulation: from Algorithms to Applications (Academic, 2002).

  2. Leach, A. Molecular Modelling: Principles and Applications 2nd edn (Prentice Hall, 2001).

  3. Hohenberg, P. & Kohn, W. Inhomogeneous electron gas. Phys. Rev. 136, B864 (1964).

    Article  MathSciNet  Google Scholar 

  4. Jones, R. O. Density functional theory: its origins, rise to prominence, and future. Rev. Mod. Phys. 87, 897 (2015).

    Article  MathSciNet  Google Scholar 

  5. Becke, A. D. Perspective: Fifty years of density-functional theory in chemical physics. J. Chem. Phys. 140, 18A301 (2014).

    Article  Google Scholar 

  6. Burke, K. Perspective on density functional theory. J. Chem. Phys. 136, 150901 (2012).

    Article  Google Scholar 

  7. Car, R. & Parrinello, M. Unified approach for molecular dynamics and density-functional theory. Phys. Rev. Lett. 55, 2471 (1985).

    Article  Google Scholar 

  8. Hedin, L. New method for calculating the one-particle Green’s function with application to the electron-gas problem. Phys. Rev. 139, A796 (1965).

    Article  Google Scholar 

  9. Georges, A., Kotliar, G., Krauth, W. & Rozenberg, M. J. Dynamical mean-field theory of strongly correlated fermion systems and the limit of infinite dimensions. Rev. Mod. Phys. 68, 13 (1996).

    Article  MathSciNet  Google Scholar 

  10. Foulkes, W. M. C., Mitas, L., Needs, R. J. & Rajagopal, G. Quantum Monte Carlo simulations of solids. Rev. Mod. Phys. 73, 33 (2001).

    Article  Google Scholar 

  11. Zhang, S. & Krakauer, H. Quantum Monte Carlo method using phase-free random walks with Slater determinants. Phys. Rev. Lett. 90, 136401 (2003).

    Article  Google Scholar 

  12. Rillo, G., Morales, M. A., Ceperley, D. M. & Pierleoni, C. Coupled electron–ion Monte Carlo simulation of hydrogen molecular crystals. J. Chem. Phys. 148, 102314 (2018).

    Article  Google Scholar 

  13. Tirelli, A., Tenti, G., Nakano, K. & Sorella, S. High-pressure hydrogen by machine learning and quantum Monte Carlo. Phys. Rev. B 106, L041105 (2022).

    Article  Google Scholar 

  14. Levine, I. N. Quantum Chemistry (Prentice Hall, 1991).

  15. Cramer, C. J. Essentials of Computational Chemistry (Wiley, 2002).

  16. Suewattana, M., Purwanto, W., Zhang, S., Krakauer, H. & Walter, E. J. Phaseless auxiliary-field quantum Monte Carlo calculations with plane waves and pseudopotentials: applications to atoms and molecules. Phys. Rev. B 75, 245123 (2007).

    Article  Google Scholar 

  17. Jia, Z.-A. et al. Quantum neural network states: a brief review of methods and applications. Adv. Quantum Technol. 2, 1800077 (2019).

    Article  Google Scholar 

  18. Carleo, G. & Troyer, M. Solving the quantum many-body problem with artificial neural networks. Science 355, 602 (2017).

    Article  MathSciNet  MATH  Google Scholar 

  19. Lanyon, B. P. et al. Towards quantum chemistry on a quantum computer. Nat. Chem. 2, 106–111 (2010).

    Article  Google Scholar 

  20. Huggins, W. J. et al. Unbiasing fermionic quantum Monte Carlo with a quantum computer. Nature 603, 416–420 (2022).

    Article  Google Scholar 

  21. Guareschi, R. & Filippi, C. Ground- and excited-state geometry optimization of small organic molecules with quantum Monte Carlo. J. Chem. Theory Comput. 9, 5513–5525 (2013).

    Article  Google Scholar 

  22. Zen, A., Zhelyazov, D. & Guidoni, L. Optimized structure and vibrational properties by error affected potential energy surfaces. J. Chem. Theory Comput. 8, 4204–4215 (2012).

    Article  Google Scholar 

  23. Barborini, M., Sorella, S. & Guidoni, L. Structural optimization by quantum Monte Carlo: investigating the low-lying excited states of ethylene. J. Chem. Theory Comput. 8, 1260–1269 (2012).

    Article  Google Scholar 

  24. Wagner, L. K. & Grossman, J. C. Quantum Monte Carlo calculations for minimum energy structures. Phys. Rev. Lett. 104, 210201 (2010).

    Article  Google Scholar 

  25. Tiihonen, J., Kent, P. R. C. & Krogel, J. T. Surrogate Hessian accelerated structural optimization for stochastic electronic structure theories. J. Chem. Phys. 156, 054104 (2022).

    Article  Google Scholar 

  26. Chiesa, S., Ceperley, D. M. & Zhang, S. Accurate, efficient, and simple forces computed with quantum Monte Carlo methods. Phys. Rev. Lett. 94, 036404 (2005).

    Article  Google Scholar 

  27. Assaraf, R. & Caffarel, M. Zero-variance zero-bias principle for observables in quantum Monte Carlo: application to forces. J. Chem. Phys. 119, 10536 (2003).

    Article  Google Scholar 

  28. Robbins, H. & Monro, S. A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951).

    Article  MathSciNet  MATH  Google Scholar 

  29. Armijo, L. Minimization of functions having Lipschitz continuous first partial derivatives. Pac. J. Math. 16, 1–3 (1966).

    Article  MathSciNet  MATH  Google Scholar 

  30. Wolfe, P. Convergence conditions for ascent methods. SIAM Rev. 11, 226–235 (1969).

    Article  MathSciNet  MATH  Google Scholar 

  31. Wolfe, P. Convergence conditions for ascent methods. II: Some corrections. SIAM Rev. 13, 185–188 (1971).

    Article  MathSciNet  MATH  Google Scholar 

  32. Bertsekas, D. P. & Tsitsiklis, J. N. Gradient convergence in gradient methods with errors. SIAM J. Optim. 10, 627–642 (2000).

    Article  MathSciNet  MATH  Google Scholar 

  33. Bertsekas, D. P., Nonlinear Programming (Athena, 2016).

  34. Debye, P. Näherungsformeln für die zylinderfunktionen für große werte des arguments und unbeschränkt veränderliche werte des index. Math. Ann. 67, 535–558 (1909).

    Article  MathSciNet  MATH  Google Scholar 

  35. Hestenes, M. R. & Stiefel, E. Methods of conjugate gradients for solving linear systems. J. Res. Natl Bur. Stand. 49, 409–436 (1952).

    Article  MathSciNet  MATH  Google Scholar 

  36. Shewchuk, J. R. An Introduction to the Conjugate Gradient Method Without the Agonizing Pain http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf (Carnegie Mellon Univ., 1994).

  37. Fletcher, R. & Reeves, C. M. Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964).

    Article  MathSciNet  MATH  Google Scholar 

  38. Polak, E. & Ribière, G. Note sur la convergence de méthodes de directions conjuguées. ESAIM Math. Model. Numer. Anal. 3, 35–43 (1969).

    MATH  Google Scholar 

  39. Schraudolph N. N. & Graepel, T. Combining conjugate direction methods with stochastic approximation of gradients. Proc. Mach. Learning Res. R4, 248–253 (2003) .

  40. Tieleman T. & Hinton, G. Lecture 6a: Neural Networks for Machine Learning (Computer Science, Univ. Toronto, 2012); https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf

  41. Zeiler, M. D. ADADELTA: an adaptive learning rate method. Preprint at https://doi.org/10.48550/arxiv.1212.5701 (2012).

  42. Kingma D. P. & Ba, J. Adam: a method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980 (2014).

  43. Polyak, B. T. & Juditsky, A. B. Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30, 838–855 (1992).

    Article  MathSciNet  MATH  Google Scholar 

  44. Polyak, B. New method of stochastic approximation type. Autom. Remote Control 51, 937–1008 (1990).

    MathSciNet  MATH  Google Scholar 

  45. Ruppert, D. Efficient Estimations from a Slowly Convergent Robbins–Monro Process Technical Report (Cornell Univ. Operations Research and Industrial Engineering, 1988).

  46. Qian, N. On the momentum term in gradient descent learning algorithms. Neural Netw. 12, 145–151 (1999).

    Article  Google Scholar 

  47. Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. Nature 323, 533–536 (1986).

    Article  MATH  Google Scholar 

  48. Chen, S. schen24wm/geoopt-srcdata: FSSDxSET source data v0.1. Zenodo https://doi.org/10.5281/zenodo.7157782 (2022).

  49. Chen, S. schen24wm/fssd-set: FSSDxSET v0.1. Zenodo https://doi.org/10.5281/zenodo.7157763 (2022).

  50. Chen, M., Yu, T.-Q. & Tuckerman, M. E. Locating landmarks on high-dimensional free energy surfaces. Proc. Natl Acad. Sci. USA 112, 3235–3240 (2015).

    Article  Google Scholar 

  51. Laio, A. & Parrinello, M. Escaping free-energy minima. Proc. Natl Acad. Sci. USA 99, 12562–12566 (2002).

    Article  Google Scholar 

  52. Martin, R. M. Electronic Structure: Basic Theory and Practical Methods (Cambridge Univ. Press, 2020).

  53. Momma, K. & Izumi, F. VESTA: a three-dimensional visualization system for electronic and structural analysis. J. Appl. Crystallogr. 41, 653–658 (2008).

    Article  Google Scholar 

Download references

Acknowledgements

We thank B. Busemeyer, M. Lindsey, F. Ma, M. A. Morales, M. Motta, A. Sengupta, S. Sorella and Y. Yang for discussions. S.C. thanks the Center for Computational Quantum Physics, Flatiron Institute, for support and hospitality. We acknowledge financial support from the following grant: US Department of Energy (DOE) grant DE-SC0001303 (S.C.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We thank William & Mary Research Computing and Flatiron Institute Scientific Computing Center for computational resources and technical support. The Flatiron Institute is a division of the Simons Foundation.

Author information

Authors and Affiliations

Authors

Contributions

S.C. and S.Z. conceived the project. S.C. executed the research with support and supervision from S.Z. S.C. and S.Z. wrote the manuscript. S.Z. obtained funding.

Corresponding author

Correspondence to Siyuan Chen.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Computational Science thanks the anonymous reviewers for their contribution to the peer review of this work. Primary Handling Editor: Jie Pan, in collaboration with the Nature Computational Science team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–9 and Discussion.

Supplementary Software 1

Scripts and an example for the FSSD-SET algorithm.

Source data

Source Data Fig. 1

Statistical source data.

Source Data Fig. 2

Statistical source data.

Source Data Fig. 3

Statistical source data.

Source Data Fig. 4

Statistical source data.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Zhang, S. A structural optimization algorithm with stochastic forces and stresses. Nat Comput Sci 2, 736–744 (2022). https://doi.org/10.1038/s43588-022-00350-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s43588-022-00350-w

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing