Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Efficient sampling of high-dimensional free energy landscapes using adaptive reinforced dynamics

Nature Computational Science volume 2, pages 20–29 (2022)Cite this article

Subjects

A preprint version of the article is available at arXiv.

Abstract

Enhanced sampling methods such as metadynamics and umbrella sampling have become essential tools for exploring the configuration space of molecules and materials. At the same time, they have long faced a number of issues such as the inefficiency when dealing with a large number of collective variables (CVs) or systems with high free energy barriers. Here we show that, with clustering and adaptive tuning techniques, the reinforced dynamics (RiD) scheme can be used to efficiently explore the configuration space and free energy landscapes with a large number of CVs or systems with high free energy barriers. We illustrate this by studying various representative and challenging examples. First we demonstrate the efficiency of adaptive RiD compared with other methods and construct the nine-dimensional (9D) free energy landscape of a peptoid trimer, which has energy barriers of more than 8 kcal mol−1. We then study the folding of the protein chignolin using 18 CVs. In this case, both the folding and unfolding rates are observed to be 4.30 μs−1. Finally, we propose a protein structure refinement protocol based on RiD. This protocol allows us to efficiently employ more than 100 CVs for exploring the landscape of protein structures and it gives rise to an overall improvement of 14.6 units over the initial global distance test–high accuracy (GDT-HA) score.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: The workflow of adaptive RiD.
Fig. 2: The accuracy and efficiency of adaptive RiD.
Fig. 3: Free energy curves of the peptoid trimer (s1pe)3.
Fig. 4: Folding and unfolding of the protein chignolin.
Fig. 5: Protein structure refinement of three targets: R0974s1, R0986s1 and R1002-D2.
Fig. 6: Detailed analysis of the target R1002-D2.

Similar content being viewed by others

Data availability

The initial files of all examples for running adaptive RiD are available from Zenodo63. Our peptoid models come from Weiser’s work32. Our chignolin model is obtained from the Protein Data Bank (PDB 5AWL). The MD trajectories of chignolin from Anton can be obtained from ref. 36. The targets in CASP13 can be obtained from the official CASP list (https://predictioncenter.org/casp13/targetlist.cgi). The Markov state models of R0974s1, R0986s1 and R1002-D2 can be obtained from the work of Heo and colleagues47. The input PLUMED2 files are available via Plumed Nest under plumID:21.034. Source data are provided with this paper.

Code availability

Python implementations of our codes are available at GitHub (https://github.com/dongdawn/rid) and Zenodo64.

References

  1. Laio, A. & Parrinello, M. Escaping free-energy minima. Proc. Natl Acad. Sci. USA 99, 12562–12566 (2002).

    Article  Google Scholar 

  2. Barducci, A., Bussi, G. & Parrinello, M. Well-tempered metadynamics: a smoothly converging and tunable free-energy method. Phys. Rev. Lett. 100, 020603 (2008).

    Article  Google Scholar 

  3. Torrie, G. M. & Valleau, J. P. Nonphysical sampling distributions in Monte Carlo free-energy estimation: umbrella sampling. J. Comput. Phys. 23, 187–199 (1977).

    Article  Google Scholar 

  4. Rosso, L., Mináry, P., Zhu, Z. & Tuckerman, M. E. On the use of the adiabatic molecular dynamics technique in the calculation of free energy profiles. J. Chem. Phys. 116, 4389–4402 (2002).

    Article  Google Scholar 

  5. Maragliano, L. & Vanden-Eijnden, E. A temperature accelerated method for sampling free energy and determining reaction pathways in rare events simulations. Chem. Phys. Lett. 426, 168–175 (2006).

    Article  Google Scholar 

  6. Abrams, J. B. & Tuckerman, M. E. Efficient and direct generation of multidimensional free energy surfaces via adiabatic dynamics without coordinate transformations. J. Phys. Chem. B 112, 15742–15757 (2008).

    Article  Google Scholar 

  7. Abrams, C. F. & Vanden-Eijnden, E. Large-scale conformational sampling of proteins using temperature-accelerated molecular dynamics. Proc. Natl Acad. Sci. USA 107, 4961–4966 (2010).

    Article  Google Scholar 

  8. Maragliano, L. & Vanden-Eijnden, E. Single-sweep methods for free energy calculations. J. Chem. Phys. 128, 184110 (2008).

    Article  Google Scholar 

  9. Piana, S. & Laio, A. A bias-exchange approach to protein folding. J. Phys. Chem. B 111, 4553–4559 (2007).

    Article  Google Scholar 

  10. Pfaendtner, J. & Bonomi, M. Efficient sampling of high-dimensional free-energy landscapes with parallel bias metadynamics. J. Chem. Theory Comput. 11, 5062–5067 (2015).

    Article  Google Scholar 

  11. Arushi, P., Fu, C. D., Massimiliano, B. & Jim, P. Biasing smarter, not harder, by partitioning collective variables into families in parallel bias metadynamics. J. Chem. Theory Comput. 14, 4985–4990 (2018).

    Article  Google Scholar 

  12. Stecher, T., Bernstein, N. & Csányi, G. Free energy surface reconstruction from umbrella samples using Gaussian process regression. J. Chem. Theory Comput. 10, 4079–4097 (2014).

    Article  Google Scholar 

  13. Mones, L., Bernstein, N. & Csányi, G. Exploration, sampling and reconstruction of free energy surfaces with Gaussian process regression. J. Chem. Theory Comput. 12, 5100–5110 (2016).

    Article  Google Scholar 

  14. Schneider, E., Dai, L., Topper, R. Q., Drechsel-Grau, C. & Tuckerman, M. E. Stochastic neural network approach for learning high-dimensional free energy surfaces. Phys. Rev. Lett. 119, 150601 (2017).

    Article  MathSciNet  Google Scholar 

  15. Zhang, L., Wang, H. & E, W. Reinforced dynamics for enhanced sampling in large atomic and molecular systems. J. Chem. Phys. 148, 124113 (2018).

    Article  Google Scholar 

  16. Sidky, H. & Whitmer, J. K. Learning free energy landscapes using artificial neural networks. J. Chem. Phys. 148, 104111 (2018).

    Article  Google Scholar 

  17. Guo, A. Z. et al. Adaptive enhanced sampling by force-biasing using neural networks. J. Chem. Phys. 148, 134108 (2018).

    Article  Google Scholar 

  18. Sultan, M. M., Wayment-Steele, H. K. & Pande, V. S. Transferable neural networks for enhanced sampling of protein dynamics. J. Chem. Theory Comput. 14, 1887–1894 (2018).

    Article  Google Scholar 

  19. Bonati, L., Zhang, Y.-Y. & Parrinello, M. Neural networks-based variationally enhanced sampling. Proc. Natl Acad. Sci. USA 116, 17641–17647 (2019).

    Article  MathSciNet  MATH  Google Scholar 

  20. Sevgen, E., Guo, A., Sidky, H., Whitmer, J. K. & de Pablo, J. J. Combined force-frequency sampling for simulation of systems having rugged free energy landscapes. J. Chem. Theory Comput. 16, 1448–1455 (2020).

    Article  Google Scholar 

  21. Valsson, O. & Parrinello, M. Variational approach to enhanced sampling and free energy calculations. Phys. Rev. Lett. 113, 090601 (2014).

    Article  Google Scholar 

  22. Shaffer, P., Valsson, O. & Parrinello, M. Enhanced, targeted sampling of high-dimensional free-energy landscapes using variationally enhanced sampling, with an application to chignolin. Proc. Natl Acad. Sci. USA 113, 1150–1155 (2016).

    Article  Google Scholar 

  23. Cendagorta, J. R., Tolpin, J., Schneider, E., Topper, R. Q. & Tuckerman, M. E. Comparison of the performance of machine learning models in representing high-dimensional free energy surfaces and generating observables. J. Phys. Chem. B 124, 3647–3660 (2020).

    Article  Google Scholar 

  24. Sugita, Y. & Okamoto, Y. Replica-exchange molecular dynamics method for protein folding. Chem. Phys. Lett. 314, 141–151 (1999).

    Article  Google Scholar 

  25. Ducheyne, P. Comprehensive Biomaterials Vol. 1 (Elsevier, 2015).

  26. Sun, J. & Zuckermann, R. N. Peptoid polymers: a highly designable bioinspired material. ACS Nano 7, 4715–4732 (2013).

    Article  Google Scholar 

  27. Mojsoska, B., Zuckermann, R. N. & Jenssen, H. Structure–activity relationship study of novel peptoids that mimic the structure of antimicrobial peptides. Antimicrob. Agents Chemother. 59, 4112–4120 (2015).

    Article  Google Scholar 

  28. Li, N. et al. Blockade of CD28 by a synthetical peptoid inhibits T-cell proliferation and attenuates graft-versus-host disease. Cell. Mol. Immunol. 7, 133–142 (2010).

    Article  Google Scholar 

  29. Luo, Y. et al. Aβ42-binding peptoids as amyloid aggregation inhibitors and detection ligands. ACS Chem. Neurosci. 4, 952–962 (2013).

    Article  Google Scholar 

  30. Mirijanian, D. T., Mannige, R. V., Zuckermann, R. N. & Whitelam, S. Development and use of an atomistic CHARMM-based forcefield for peptoid simulation. J. Comput. Chem. 35, 360–370 (2014).

    Article  Google Scholar 

  31. Mukherjee, S., Zhou, G., Michel, C. & Voelz, V. A. Insights into peptoid helix folding cooperativity from an improved backbone potential. J. Phys. Chem. B 119, 15407–15417 (2015).

    Article  Google Scholar 

  32. Weiser, L. J. & Santiso, E. E. Molecular modeling studies of peptoid polymers. AIMS Mater. Sci. 4, 1029–1051 (2017).

    Article  Google Scholar 

  33. Weiser, L. J. & Santiso, E. E. A CGenFF-based force field for simulations of peptoids with both cis and trans peptide bonds. J. Comput. Chem. 40, 1946–1956 (2019).

    Article  Google Scholar 

  34. Gorske, B. C., Stringer, J. R., Bastian, B. L., Fowler, S. A. & Blackwell, H. E. New strategies for the design of folded peptoids revealed by a survey of noncovalent interactions in model systems. J. Am. Chem. Soc. 131, 16555–16567 (2009).

    Article  Google Scholar 

  35. Honda, S. et al. Crystal structure of a ten-amino acid protein. J. Am. Chem. Soc. 130, 15327–15331 (2008).

    Article  Google Scholar 

  36. Lindorff-Larsen, K., Piana, S., Dror, R. O. & Shaw, D. E. How fast-folding proteins fold. Science 334, 517–520 (2011).

    Article  Google Scholar 

  37. Kührová, P., De Simone, A., Otyepka, M. & Best, R. B. Force-field dependence of chignolin folding and misfolding: comparison with experiment and redesign. Biophys. J. 102, 1897–1906 (2012).

    Article  Google Scholar 

  38. Zhang, T., Nguyen, P. H., Nasica-Labouze, J., Mu, Y. & Derreumaux, P. Folding atomistic proteins in explicit solvent using simulated tempering. J. Phys. Chem. B 119, 6941–6951 (2015).

    Article  Google Scholar 

  39. Miao, Y., Feixas, F., Eun, C. & McCammon, J. A. Accelerated molecular dynamics simulations of protein folding. J. Comput. Chem. 36, 1536–1549 (2015).

    Article  Google Scholar 

  40. Jumper, J. et al. Highly accurate protein structure prediction with Alphafold. Nature 596, 583–589 (2021).

    Article  Google Scholar 

  41. Zemla, A. LGA: a method for finding 3D similarities in protein structures. Nucleic Acids Res. 31, 3370–3374 (2003).

    Article  Google Scholar 

  42. Raval, A., Piana, S., Eastwood, M. P., Dror, R. O. & Shaw, D. E. Refinement of protein structure homology models via long, all-atom molecular dynamics simulations. Proteins 80, 2071–2079 (2012).

    Article  Google Scholar 

  43. Feig, M. & Mirjalili, V. Protein structure refinement via molecular-dynamics simulations: what works and what does not? Proteins 84, 282–292 (2016).

    Article  Google Scholar 

  44. Heo, L. & Feig, M. Experimental accuracy in protein structure refinement via molecular dynamics simulations. Proc. Natl Acad. Sci. USA 115, 13276–13281 (2018).

    Article  Google Scholar 

  45. Park, H., Ovchinnikov, S., Kim, D. E., DiMaio, F. & Baker, D. Protein homology model refinement by large-scale energy optimization. Proc. Natl Acad. Sci. USA 115, 3054–3059 (2018).

    Article  Google Scholar 

  46. Park, H. et al. High-accuracy refinement using Rosetta in CASP13. Proteins 87, 1276–1282 (2019).

    Article  Google Scholar 

  47. Heo, L., Arbour, C. F. & Feig, M. Driven to near-experimental accuracy by refinement via molecular dynamics simulations. Proteins 87, 1263–1275 (2019).

    Article  Google Scholar 

  48. Zhang, J. & Zhang, Y. A novel side-chain orientation dependent potential derived from random-walk reference state for protein fold selection and structure prediction. PLoS ONE 5, e15386 (2010).

    Article  Google Scholar 

  49. Jing, X. & Xu, J. Improved protein model quality assessment by integrating sequential and pairwise features using deep learning. Bioinformatics 36, 5361–5367 (2020).

    Article  Google Scholar 

  50. Abraham, M. J. et al. GROMACS: high performance molecular simulations through multi-level parallelism from laptops to supercomputers. SoftwareX 1, 19–25 (2015).

    Article  Google Scholar 

  51. Tribello, G. A., Bonomi, M., Branduardi, D., Camilloni, C. & Bussi, G. PLUMED 2: new feathers for an old bird. Comput. Phys. Commun. 185, 604–613 (2014).

    Article  Google Scholar 

  52. Bussi, G., Donadio, D. & Parrinello, M. Canonical sampling through velocity rescaling. J. Chem. Phys. 126, 014101 (2007).

    Article  Google Scholar 

  53. Parrinello, M. & Rahman, A. Polymorphic transitions in single crystals: a new molecular dynamics method. J. Appl. Phys. 52, 7182 (1981).

    Article  Google Scholar 

  54. Darden, T., York, D. & Pedersen, L. Particle mesh Ewald: an Nlog(N) method for Ewald sums in large systems. J. Chem. Phys. 98, 10089–10092 (1993).

    Article  Google Scholar 

  55. Hess, B., Bekker, H., Berendsen, H. & Fraaije, J. LINCS: a linear constraint solver for molecular simulations. J. Comput. Chem. 18, 1463–1472 (1997).

    Article  Google Scholar 

  56. Jorgensen, W. L., Chandrasekhar, J., Madura, J. D., Impey, R. W. & Klein, M. L. Comparison of simple potential functions for simulating liquid water. J. Chem. Phys. 79, 926–935 (1983).

    Article  Google Scholar 

  57. Abadi, M. et al. Tensorflow: a system for large-scale machine learning. In Proc. 12th USENIX Conference on Operating Systems Design and Implementation OSDI’16 265–283 (USENIX Association, 2016).

  58. Kingma, D. & Ba, J. Adam: a method for stochastic optimization. In Proc. 3rd International Conference on Learning Representations (ICLR, 2015).

  59. Patriksson, A. & van der Spoel, D. A temperature predictor for parallel tempering simulations. Phys. Chem. Chem. Phys. 10, 2073–2077 (2008).

    Article  Google Scholar 

  60. Piana, S., Lindorff-Larsen, K. & Shaw, D. E. How robust are protein folding simulations with respect to force field parameterization? Biophys. J. 100, L47–L49 (2011).

    Article  Google Scholar 

  61. Paissoni, C. & Camilloni, C. How to determine accurate conformational ensembles by metadynamics metainference: a chignolin study case. Front. Mol. Biosci. 8, 694130 (2021).

    Article  Google Scholar 

  62. Huang, J. et al. CHARMM36m: an improved force field for folded and intrinsically disordered proteins. Nat. Methods 14, 71–73 (2017).

    Article  Google Scholar 

  63. Wang, D. & Wang, Y. Initial files of examples for running adaptive RiD. Zenodo https://doi.org/10.5281/zenodo.5674402 (2021).

  64. Wang, D. & Wang, Y. Codes of adaptive reinforced dynamics. Zenodo https://doi.org/10.5281/zenodo.5674474 (2021).

Download references

Acknowledgements

The work of D.W., L.Z. and W.E is supported in part by a gift from iFlytek to Princeton University. The work of H.W. is supported by the National Science Foundation of China under grant no.11871110 and Beijing Academy of Artificial Intelligence (BAAI). The work of L.Z. is also supported by the DOE Center of Chemistry in Solutions and at Interfaces (CSI) through award no. DE-SC0019394.

Author information

Authors and Affiliations

  1. Program in Applied and Computational Mathematics, Princeton University, Princeton, NJ, USA

    Dongdong Wang & Linfeng Zhang

  2. DP Technology, Beijing, People’s Republic of China

    Dongdong Wang, Yanze Wang, Junhan Chang & Linfeng Zhang

  3. College of Chemistry and Molecular Engineering, Peking University, Beijing, People’s Republic of China

    Yanze Wang & Junhan Chang

  4. Laboratory of Computational Physics, Institute of Applied Physics and Computational Mathematics, Beijing, People’s Republic of China

    Han Wang

  5. School of Mathematical Sciences, Peking University, Beijing, People’s Republic of China

    Weinan E.

  6. Department of Mathematics and Program in Applied and Computational Mathematics, Princeton University, Princeton, NJ, USA

    Weinan E.

  7. Beijing Institute of Big Data Research, Beijing, People’s Republic of China

    Weinan E.

Authors
  1. Dongdong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanze Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junhan Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Linfeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Han Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Weinan E.
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

D.W., L.Z., H.W. and W.E. conceptualized the research. D.W., Y.W. and J.C. conducted the research and performed data analysis. D.W. L.Z., H.W. and W.E. drafted the manuscript. All authors commented on and revised the manuscript.

Corresponding authors

Correspondence to Linfeng Zhang or Han Wang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review information

Nature Computational Science thanks Vojtech Spiwok, Max Bonomi and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Handling editor: Jie Pan, in collaboration with the Nature Computational Science team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–8 and Tables 1–5.

Peer Review File

Source data

Source Data Fig. 2

Statistical source data.

Source Data Fig. 3

Statistical source data.

Source Data Fig. 4

Statistical source data.

Source Data Fig. 5

Statistical source data.

Source Data Fig. 6

Statistical source data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., Wang, Y., Chang, J. et al. Efficient sampling of high-dimensional free energy landscapes using adaptive reinforced dynamics. Nat Comput Sci 2, 20–29 (2022). https://doi.org/10.1038/s43588-021-00173-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s43588-021-00173-1

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing