Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Overcoming the barrier of orbital-free density functional theory for molecular systems using deep learning

Nature Computational Science volume 4, pages 210–223 (2024)Cite this article

Subjects

A preprint version of the article is available at arXiv.

Abstract

Orbital-free density functional theory (OFDFT) is a quantum chemistry formulation that has a lower cost scaling than the prevailing Kohn–Sham DFT, which is increasingly desired for contemporary molecular research. However, its accuracy is limited by the kinetic energy density functional, which is notoriously hard to approximate for non-periodic molecular systems. Here we propose M-OFDFT, an OFDFT approach capable of solving molecular systems using a deep learning functional model. We build the essential non-locality into the model, which is made affordable by the concise density representation as expansion coefficients under an atomic basis. With techniques to address unconventional learning challenges therein, M-OFDFT achieves a comparable accuracy to Kohn–Sham DFT on a wide range of molecules untouched by OFDFT before. More attractively, M-OFDFT extrapolates well to molecules much larger than those seen in training, which unleashes the appealing scaling of OFDFT for studying large molecules including proteins, representing an advancement of the accuracy–efficiency trade-off frontier in quantum chemistry.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Overview of M-OFDFT.
Fig. 2: Results of M-OFDFT compared with classical OFDFT on molecular systems.
Fig. 3: Extrapolation performance of M-OFDFT compared with other deep learning methods.
Fig. 4: Empirical time cost of M-OFDFT compared with KSDFT on molecules (n = 808) at various scales.

Similar content being viewed by others

Data availability

All molecular structures used in this study can be freely accessed from public sources: ethanol structures are from the MD17 dataset36 at http://www.sgdml.org/#datasets, QM939 molecular structures are from https://doi.org/10.6084/m9.figshare.978904, QMugs45 molecular structures are from https://doi.org/10.3929/ethz-b-000482129, and structures of chignolin, the BBL-H142W system (PDB ID 2WXC) and the protein B system (PDB ID 1PRB) are from ref. 47 at https://www.deshawresearch.com/downloads/download_trajectory_science2011.cgi. Example evaluation data for reproducing the analyses in this work are available at https://doi.org/10.6084/m9.figshare.c.6877432 (ref. 79). Source data are provided with this paper.

Code availability

The code for implementing the proposed methodology is available at https://doi.org/10.5281/zenodo.10616893 (ref. 80). Trained neural network model checkpoints are available at https://doi.org/10.6084/m9.figshare.c.6877432 (ref. 79).

References

  1. Seminario, J. M. Recent Developments and Applications of Modern Density Functional Theory (Elsevier, 1996).

  2. Jain, A. et al. The Materials Project: a materials genome approach to accelerating materials innovation. APL Mater. 1, 011002 (2013).

    Article  Google Scholar 

  3. Kohn, W. & Sham, Lu. Jeu Self-consistent equations including exchange and correlation effects. Phys. Rev. 140, A1133 (1965).

    Article  MathSciNet  Google Scholar 

  4. Thomas, L. H. The calculation of atomic fields. Math. Proc. Camb. Phil. Soc. 23, 542–548 (1927).

  5. Fermi, E. Eine statistische methode zur bestimmung einiger eigenschaften des atoms und ihre anwendung auf die theorie des periodischen systems der elemente. Z. Phys. 48, 73–79 (1928).

    Article  Google Scholar 

  6. Wang, Y. A. & Carter, E. A. Orbital-free kinetic-energy density functional theory. Theor. Methods Condens. Phase Chem. 5, 117–184 (2000).

  7. Karasiev, V. V., Chakraborty, D. & Trickey, S. B. In Many-electron Approaches in Physics, Chemistry and Mathematics: A Multidisciplinary View (eds Bach, V. & Delle Site, L.) 113–134 (Springer, 2014).

  8. Hodges, C. H. Quantum corrections to the Thomas–Fermi approximation—the Kirzhnits method. Can. J. Phys. 51, 1428–1437 (1973).

    Article  Google Scholar 

  9. Brack, M., Jennings, B. K. & Chu, Y. H. On the extended Thomas–Fermi approximation to the kinetic energy density. Phys. Lett. B 65, 1–4 (1976).

    Article  Google Scholar 

  10. Wang, L.-W. & Teter, M. P. Kinetic-energy functional of the electron density. Phys. Rev. B 45, 13196 (1992).

  11. Wang, Y. A., Govind, N. & Carter, E. A. Orbital-free kinetic-energy density functionals with a density-dependent kernel. Phys. Rev. B 60, 16350 (1999).

  12. Huang, C. & Carter, E. A. Nonlocal orbital-free kinetic energy density functional for semiconductors. Phys. Rev. B 81, 045206 (2010).

    Article  Google Scholar 

  13. Hung, L. & Carter, E. A. Accurate simulations of metals at the mesoscale: explicit treatment of 1 million atoms with quantum mechanics. Chem. Phys. Lett. 475, 163–170 (2009).

    Article  Google Scholar 

  14. Witt, W. C., Beatriz, G., Dieterich, J. M. & Carter, E. A. Orbital-free density functional theory for materials research. J. Mater. Res. 33, 777–795 (2018).

    Article  Google Scholar 

  15. García-Aldea, D. & Alvarellos, J. E. Kinetic energy density study of some representative semilocal kinetic energy functionals. J. Chem. Phys. 127, 144109 (2007).

  16. Xia, J., Huang, C., Shin, I. & Carter, E. A. Can orbital-free density functional theory simulate molecules? J. Chem. Phys. 136, 084102 (2012).

    Article  Google Scholar 

  17. Teale, A. M. et al. DFT exchange: sharing perspectives on the workhorse of quantum chemistry and materials science. Phys. Chem. Chem. Phys. 24, 28700–28781 (2022).

    Article  Google Scholar 

  18. Snyder, J. C., Rupp, M., Hansen, K., Müller, Klaus-Robert & Burke, K. Finding density functionals with machine learning. Phys. Rev. Lett. 108, 253002 (2012).

    Article  Google Scholar 

  19. Li, L. et al. Pure density functional for strong correlation and the thermodynamic limit from machine learning. Phys. Rev. B 94, 245129 (2016).

    Article  Google Scholar 

  20. Brockherde, F. et al. Bypassing the Kohn–Sham equations with machine learning. Nat. Commun. 8, 872 (2017).

    Article  Google Scholar 

  21. Seino, J., Kageyama, R., Fujinami, M., Ikabata, Y. & Nakai, H. Semi-local machine-learned kinetic energy density functional with third-order gradients of electron density. J. Chem. Phys. 148, 241705 (2018).

  22. Imoto, F., Imada, M. & Oshiyama, A. Order-N orbital-free density-functional calculations with machine learning of functional derivatives for semiconductors and metals. Phys. Rev. Res. 3, 033198 (2021).

    Article  Google Scholar 

  23. Yao, K. & Parkhill, J. Kinetic energy of hydrocarbons as a function of electron density and convolutional neural networks. J. Chem. Theory Comput. 12, 1139–1147 (2016).

    Article  Google Scholar 

  24. Meyer, R., Weichselbaum, M. & Hauser, A. W. Machine learning approaches toward orbital-free density functional theory: simultaneous training on the kinetic energy density functional and its functional derivative. J. Chem. Theory Comput. 16, 5685–5694 (2020).

    Article  Google Scholar 

  25. Remme, R., Kaczun, T., Scheurer, M., Dreuw, A. & Hamprecht, F. A. KineticNet: Ddeep learning a transferable kinetic energy functional for orbital-free density functional theory. J. Chem. Phys. 159, 144113 (2023).

    Article  Google Scholar 

  26. García-González, P., Alvarellos, J. E. & Chacón, E. Nonlocal kinetic-energy-density functionals. Phys. Rev. B 53, 9509–9512 (1996).

    Article  Google Scholar 

  27. Mi, W., Genova, A. & Pavanello, M. Nonlocal kinetic energy functionals by functional integration. J. Chem. Phys. 148, 184107 (2018).

    Article  Google Scholar 

  28. Kato, T. On the eigenfunctions of many-particle systems in quantum mechanics. Commun. Pure Appl. Math. 10, 151–177 (1957).

    Article  MathSciNet  Google Scholar 

  29. Ying, C. et al. Do transformers really perform badly for graph representation? Adv. Neural Inf. Process. Syst. 34, 28877–28888 (2021).

    Google Scholar 

  30. Shi, Y. et al. Benchmarking Graphormer on large-scale molecular modeling datasets. Preprint at https://arxiv.org/abs/2203.04810 (2022).

  31. Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 6000–6010 (2017).

  32. Smith, J. S., Isayev, O. & Roitberg, A. E. ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost. Chem. Sci. 8, 3192–3203 (2017).

    Article  Google Scholar 

  33. Schütt, K. T., Sauceda, H. E., Kindermans, P.-J., Tkatchenko, A. & Müller, K.-R. SchNet—a deep learning architecture for molecules and materials. J. Chem. Phys. 148, 241722 (2018).

  34. Thölke, P. & De Fabritiis, G. Equivariant transformers for neural network based molecular potentials. In International Conference on Learning Representations (2021).

  35. Liao, Y.-L. & Smidt, T. Equiformer: equivariant graph attention transformer for 3D atomistic graphs. In Eleventh International Conference on Learning Representations (2023); https://openreview.net/forum?id=KwmPfARgOTD

  36. Chmiela, S. et al. Machine learning of accurate energy-conserving molecular force fields. Sci. Adv. 3, e1603015 (2017).

    Article  Google Scholar 

  37. Chmiela, S., Sauceda, H. E., Poltavsky, I., Müller, K.-R. & Tkatchenko, A. sGDML: constructing accurate and data efficient molecular force fields using machine learning. Comput. Phys. Commun. 240, 38–45 (2019).

  38. Ruddigkeit, L., Van Deursen, R., Blum, L. C. & Reymond, J-L. Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. J. Chem. Inf. Model. 52, 2864–2875 (2012).

  39. Ramakrishnan, R., Dral, P. O., Rupp, M. & von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1, 140022 (2014).

    Article  Google Scholar 

  40. Constantin, L. A., Fabiano, E., Laricchia, S. & Della Sala, F. Semiclassical neutral atom as a reference system in density functional theory. Phys. Rev. Lett. 106, 186406 (2011).

    Article  Google Scholar 

  41. Karasiev, V. V. & Trickey, S. B. Issues and challenges in orbital-free density functional calculations. Comput. Phys. Commun. 183, 2519–2527 (2012).

    Article  Google Scholar 

  42. von Weizsäcker, C. F. Zur theorie der kernmassen. Z. Phys. 96, 431–458 (1935).

    Article  Google Scholar 

  43. Hirshfeld, F. L. Bonded-atom fragments for describing molecular charge densities. Theor. Chim. Acta 44, 129–138 (1977).

    Article  Google Scholar 

  44. Sun, Q. et al. PySCF: the Python-based simulations of chemistry framework. Wiley Interdiscip. Rev. Comput. Mol. Sci. 8, e1340 (2018).

    Article  Google Scholar 

  45. Isert, C., Atz, K., Jiménez-Luna, José. & Schneider, G. QMugs, quantum mechanical properties of drug-like molecules. Sci. Data 9, 273 (2022).

    Article  Google Scholar 

  46. Du, Y., Li, S., Tenenbaum, J. & Mordatch, I. Learning iterative reasoning through energy minimization. In International Conference on Machine Learning 5570–5582 (PMLR, 2022).

  47. Lindorff-Larsen, K., Piana, S., Dror, R. O. & Shaw, D. E. How fast-folding proteins fold. Science 334, 517–520 (2011).

    Article  Google Scholar 

  48. Neuweiler, H. et al. The folding mechanism of BBL: plasticity of transition-state structure observed within an ultrafast folding protein family. J. Mol. Biol. 390, 1060–1073 (2009).

    Article  Google Scholar 

  49. Wang, T., Zhu, Y. & Gai, F. Folding of a three-helix bundle at the folding speed limit. J. Phys. Chem. B 108, 3694–3697 (2004).

    Article  Google Scholar 

  50. Rayson, M. J. & Briddon, P. R. Rapid iterative method for electronic-structure eigenproblems using localised basis functions. Comput. Phys. Commun. 178, 128–134 (2008).

    Article  MathSciNet  Google Scholar 

  51. VandeVondele, J. et al. QUICKSTEP: fast and accurate density functional calculations using a mixed Gaussian and plane waves approach. Comput. Phys. Commun. 167, 103–128 (2005).

    Article  Google Scholar 

  52. Govind, N., Wang, Y. A., Da Silva, A. J. R. & Carter, E. A. Accurate ab initio energetics of extended systems via explicit correlation embedded in a density functional environment. Chem. Phys. Lett. 295, 129–134 (1998).

    Article  Google Scholar 

  53. Zhao, W. X. et al. A survey of large language models. Preprint at https://arxiv.org/abs/2303.18223 (2023).

  54. Zheng, S. et al. Towards predicting equilibrium distributions for molecular systems with deep learning. Preprint at https://arxiv.org/abs/2306.05445 (2023).

  55. Krondorfer, J. K., Binder, C. W. & Hauser, A. W. Symmetry- and gradient-enhanced Gaussian process regression for the active learning of potential energy surfaces in porous materials. J. Chem. Phys. 159, 014115 (2023).

    Article  Google Scholar 

  56. Chen, Y., Zhang, L., Wang, H. & Weinan, E. DeePKS: a comprehensive data-driven approach toward chemically accurate density functional theory. J. Chem. Theory Comput. 17, 170–181 (2021).

  57. Kirkpatrick, J. et al. Pushing the frontiers of density functionals by solving the fractional electron problem. Science 374, 1385–1389 (2021).

    Article  Google Scholar 

  58. Li, L. et al. Kohn–Sham equations as regularizer: building prior knowledge into machine-learned physics. Phys. Rev. Lett. 126, 036401 (2021).

    Article  Google Scholar 

  59. Dunlap, B. I. Robust and variational fitting. Phys. Chem. Chem. Phys. 2, 2113–2116 (2000).

    Article  Google Scholar 

  60. Bardo, R. D. & Ruedenberg, K. Even-tempered atomic orbitals. VI. Optimal orbital exponents and optimal contractions of Gaussian primitives for hydrogen, carbon, and oxygen in molecules. J. Chem. Phys. 60, 918–931 (1974).

    Article  Google Scholar 

  61. Han, J. et al. Deep potential: a general representation of a many-body potential energy surface. Commun. Comput. Phys. 23, 629–639 (2018).

  62. Li, H. et al. Deep-learning density functional theory Hamiltonian for efficient ab initio electronic-structure calculation. Nat. Comput. Sci. 2, 367–377 (2022).

    Article  Google Scholar 

  63. Li, F., Fujiwara, K., Okura, F. & Matsushita, Y. A closer look at rotation-invariant deep point cloud analysis. In Proc. IEEE/CVF International Conference on Computer Vision 16218–16227 (IEEE, 2021).

  64. Puny, O. et al. Frame averaging for invariant and equivariant network design. In International Conference on Learning Representations (2022); https://openreview.net/forum?id=zIUyj55nXR (2022).

  65. Schütt, K., Unke, O. & Gastegger, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning 9377–9388 (PMLR, 2021).

  66. Batzner, S. et al. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun. 13, 2453 (2022).

    Article  Google Scholar 

  67. Fazlyab, M., Robey, A., Hassani, H., Morari, M. & Pappas, G. Efficient and accurate estimation of Lipschitz constants for deep neural networks. Adv. Neural Inf. Process. Syst. 32, 11427–11438 (2019).

  68. Amari, S. Natural gradient works efficiently in learning. Neural Comput. 10, 251–276 (1998).

  69. Yoshikawa, N. & Sumita, M. Automatic differentiation for the direct minimization approach to the Hartree–Fock method. J. Phys. Chem. A 126, 8487–8493 (2022).

    Article  Google Scholar 

  70. Hoffmann, R. An extended Hückel theory. I. Hydrocarbons. J. Chem. Phys. 39, 1397–1412 (1963).

    Article  Google Scholar 

  71. Wang, J. et al. Machine learning of coarse-grained molecular dynamics force fields. ACS Cent. Sci. 5, 755–767 (2019).

    Article  Google Scholar 

  72. McGibbon, R. T. et al. MDTraj: a modern open library for the analysis of molecular dynamics trajectories. Biophys. J. 109, 1528–1532 (2015).

    Article  Google Scholar 

  73. Eastman, P. et al. OpenMM 7: rapid development of high performance algorithms for molecular dynamics. PLoS Comput. Biol. 13, e1005659 (2017).

    Article  Google Scholar 

  74. Case, D. A. et al. Amber 10 Technical Report (Univ. California, 2008).

  75. Ho, G. S., Lignères, V. L. & Carter, E. A. Introducing PROFESS: a new program for orbital-free density functional theory calculations. Comput. Phys. Commun. 179, 839–854 (2008).

    Article  MathSciNet  Google Scholar 

  76. Enkovaara, J. et al. Electronic structure calculations with GPAW: a real-space implementation of the projector augmented-wave method. J. Phys. Condens. Matter 22, 253202 (2010).

    Article  Google Scholar 

  77. Mi, W. et al. ATLAS: a real-space finite-difference implementation of orbital-free density functional theory. Comput. Phys. Commun. 200, 87–95 (2016).

    Article  Google Scholar 

  78. Shao, X., Jiang, K., Mi, W., Genova, A. & Pavanello, M. DFTpy: an efficient and object-oriented platform for orbital-free DFT simulations. Wiley Interdiscip. Rev. Comput. Mol. Sci. 11, e1482 (2021).

    Article  Google Scholar 

  79. Zhang, H. et al. Overcoming the barrier of orbital-free density functional theory in molecular systems using deep learning. figshare https://doi.org/10.6084/m9.figshare.c.6877432 (2024).

  80. Zhang, H. et al. Overcoming the barrier of orbital-free density functional theory in molecular systems using deep learning. Zenodo https://doi.org/10.5281/zenodo.10616893 (2024).

Download references

Acknowledgements

We thank P. G. Giorgi, W. C. Witt, S. Ehlert, Z. Wang, L. Cheng, J. Hermann and Z. Liu for insightful discussions and constructive feedback; X. He and Y. Min for suggestions on protein preprocessing; H. Yang for help with trying other OFDFT software; Y. Shi for suggestions and feedback on model design and optimization; and J. Bai for helping with figure design. We received no specific funding for this work. H.Z., S.L. and J.Y. did this work during an internship at Microsoft Research AI4Science.

Author information

Author notes

  1. These authors contributed equally: He Zhang, Siyuan Liu.

Authors and Affiliations

  1. National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, China

    He Zhang & Nanning Zheng

  2. Microsoft Research AI4Science, Beijing, China

    He Zhang, Siyuan Liu, Jiacheng You, Chang Liu, Shuxin Zheng, Ziheng Lu, Tong Wang & Bin Shao

Authors
  1. He Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siyuan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiacheng You
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shuxin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ziheng Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Nanning Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Bin Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

C.L. led the research under the support from N.Z. and B.S. C.L. is the lead contact. C.L., S.Z. and B.S. conceived the project. S.L., C.L., H.Z. and J.Y. deduced and designed data generation methods, enhancement modules, training pipeline and density optimization. H.Z., S.Z. and J.Y. designed and implemented the deep learning model. H.Z. and S.L. conducted the experiments. Z.L. and T.W. contributed to the experiment design and evaluation protocol. C.L., H.Z., S.L. and S.Z. wrote the paper with input from all authors.

Corresponding authors

Correspondence to Chang Liu, Shuxin Zheng or Bin Shao.

Ethics declarations

Competing interests

C.L., S.Z. and B.S. have filed a patent on M-OFDFT (application number: PCT/CN2023/112628). The other authors declare no competing interests.

Peer review

Peer review information

Nature Computational Science thanks Andreas Hauser and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available. Primary Handling Editor: Kaitlin McCardle, in collaboration with the Nature Computational Science team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Text, Figs. 1–19 and Tables 1–9.

Peer Review File

Source data

Source Data Fig. 2

Statistical source data.

Source Data Fig. 3

Statistical source data.

Source Data Fig. 4

Statistical source data.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Liu, S., You, J. et al. Overcoming the barrier of orbital-free density functional theory for molecular systems using deep learning. Nat Comput Sci 4, 210–223 (2024). https://doi.org/10.1038/s43588-024-00605-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s43588-024-00605-8

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics