Improved fragment sampling for ab initio protein structure prediction using deep neural networks

Abstract

A typical approach to predicting unknown native structures of proteins is to assemble the amino acid residues (fragments) extracted from known structures. The quality of these extracted fragments, which are used to build protein-specific fragment libraries, can determine the success or failure of sampling near-native conformations. Here we show how a high-quality fragment library can be built using deep contextual learning techniques. Our algorithm, called DeepFragLib, employs bidirectional long short-term-memory recurrent neural networks with knowledge distillation for initial fragment classification, followed by an aggregated residual transformation network with cyclically dilated convolution for detecting near-native fragments. DeepFragLib improves the position-averaged proportion of near-native fragments by 12.2% over existing methods and, consequently, produces better near-native structures for 72.0% of the free-modelling domain targets tested when integrated with Rosetta. DeepFragLib is fully parallelized and available for use in conjunction with structure prediction programs.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: The overall flowchart of DeepFragLib.
Fig. 2: Quality assessment of fragment libraries.
Fig. 3: Quality assessment in secondary structure classes.
Fig. 4: Evaluation of the top1 models sampled by Rosetta simulations.
Fig. 5: Case study of the distribution of TM-Scores.

Data availability

The full template fragment database HR956 used for fragment library construction and all four CASP datasets used for the quality evaluation of fragment libraries are available on Code Ocean (https://doi.org/10.24433/CO.3579011.v1)49.

Code availability

All source codes and models of DeepFragLib are publicly available through a Code Ocean compute capsule (https://doi.org/10.24433/CO.3579011.v1)49 and on GitHub (https://github.com/ElwynWang/DeepFragLib). We have also provided an online server for DeepFragLib at http://structpred.life.tsinghua.edu.cn/DeepFragLib.html.

References

  1. 1.

    Bradley, P., Misura, K. M. S. & Baker, D. Toward high-resolution de novo structure prediction for small proteins. Science 309, 1868–1871 (2005).

  2. 2.

    Dill, K. A. & MacCallum, J. L. The protein-folding problem 50 years on. Science 338, 1042–1046 (2012).

  3. 3.

    Rigden, D. J. From Protein Structure To Function With Bioinformatics Ch. 1. (Springer, 2017).

  4. 4.

    Soding, J. Big-data approaches to protein structure prediction. Science 355, 248–249 (2017).

  5. 5.

    Kim, D. E., Blum, B., Bradley, P. & Baker, D. Sampling bottlenecks in de novo protein structure prediction. J. Mol. Biol. 393, 249–260 (2009).

  6. 6.

    Jothi, A. Principles, challenges and advances in ab initio protein structure prediction. Protein Peptide Lett. 19, 1194–1204 (2012).

  7. 7.

    Wang, T., Yang, Y., Zhou, Y. & Gong, H. LRFragLib: an effective algorithm to identify fragments for de novo protein structure prediction. Bioinformatics 33, 677–684 (2017).

  8. 8.

    Baeten, L. et al. Reconstruction of protein backbones from the BriX collection of canonical protein fragments. PLoS Comput. Biol. 4, e1000083 (2008).

  9. 9.

    Xu, J. Distance-based protein folding powered by deep learning. Preprint at https://arxiv.org/abs/1811.03481 (2018).

  10. 10.

    Evans, R. et al. De novo structure prediction with deep-learning based scoring. In Thirteenth Critical Assessment of Techniques for Protein Structure Prediction Abstracts (Iberostar Paraiso, 2018).

  11. 11.

    Hanson, J., Paliwal, K., Litfin, T., Yang, Y. & Zhou, Y. Accurate prediction of protein contact maps by coupling residual two-dimensional bidirectional long short-term memory with convolutional neural networks. Bioinformatics 34, 4039–4045 (2018).

  12. 12.

    Simons, K. T., Kooperberg, C., Huang, E. & Baker, D. Assembly of protein tertiary structures from fragments with similar local sequences using simulated annealing and Bayesian scoring functions. J. Mol. Biol. 268, 209–225 (1997).

  13. 13.

    Xu, D. & Zhang, Y. Ab initio protein structure assembly using continuous structure fragments and optimized knowledge-based force field. Proteins 80, 1715–1735 (2012).

  14. 14.

    Yang, J. et al. The I-TASSER Suite: protein structure and function prediction. Nat. Methods 12, 7–8 (2015).

  15. 15.

    Rohl, C. A., Strauss, C. E., Misura, K. M. & Baker, D. Protein structure prediction using Rosetta. Methods Enzymol. 383, 66–93 (2004).

  16. 16.

    Kim, D. E., Chivian, D. & Baker, D. Protein structure prediction and analysis using the Robetta server. Nucleic Acids Res. 32, W526–W531 (2004).

  17. 17.

    Gront, D., Kulp, D. W., Vernon, R. M., Strauss, C. E. & Baker, D. Generalized fragment picking in Rosetta: design, protocols and applications. PloS ONE 6, e23294 (2011).

  18. 18.

    Kalev, I. & Habeck, M. HHfrag: HMM-based fragment detection using HHpred. Bioinformatics 27, 3110–3116 (2011).

  19. 19.

    Trevizani, R., Custodio, F. L., Dos Santos, K. B. & Dardenne, L. E. Critical features of fragment libraries for protein structure prediction. PloS ONE 12, e0170131 (2017).

  20. 20.

    Bhattacharya, D., Adhikari, B., Li, J. & Cheng, J. FRAGSION: ultra-fast protein fragment library generation by IOHMM sampling. Bioinformatics 32, 2059–2061 (2016).

  21. 21.

    de Oliveira, S. H. P. & Deane, C. M. Combining co-evolution and secondary structure prediction to improve fragment library generation. Bioinformatics 34, 2219–2227 (2018).

  22. 22.

    Wang, S., Sun, S., Li, Z., Zhang, R. & Xu, J. Accurate de novo prediction of protein contact map by ultra-deep learning model. PLoS Comput. Biol. 13, e1005324 (2017).

  23. 23.

    Wang, S., Li, Z., Yu, Y. & Xu, J. Folding membrane proteins by deep transfer learning. Cell Syst. 5, 202–211 e203 (2017).

  24. 24.

    Paliwal, K., Hanson, J., Litfin, T., Zhou, Y. & Yang, Y. Improving prediction of protein secondary structure, backbone angles, solvent accessibility and contact numbers by using predicted contact maps and an ensemble of recurrent and residual convolutional neural networks. Bioinformatics 35, 2403–2410 (2018).

  25. 25.

    Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9, 1735–1780 (1997).

  26. 26.

    Xie, S., Girshick, R., Dollár, P., Tu, Z. & He, K. Aggregated residual transformations for deep neural networks. In Proc.Conf. Computer Vision and Pattern Recognition 5987–5995 (IEEE, 2017).

  27. 27.

    Schuster, M. & Paliwal, K. K. Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45, 2673–2681 (1997).

  28. 28.

    Heffernan, R., Yang, Y., Paliwal, K. & Zhou, Y. Capturing non-local interactions by long short-term memory bidirectional recurrent neural networks for improving prediction of protein secondary structure, backbone angles, contact numbers and solvent accessibility. Bioinformatics 33, 2842–2849 (2017).

  29. 29.

    He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. Preprint at https://arxiv.org/abs/1512.03385 (2015).

  30. 30.

    Hinton, G., Vinyals, O. & Dean, J. Distilling the knowledge in a neural network. Preprint at https://arxiv.org/abs/1503.02531 (2015).

  31. 31.

    Yu, F. & Koltun, V. Multi-scale context aggregation by dilated convolutions. Preprint at https://arxiv.org/abs/1511.07122 (2015).

  32. 32.

    Altschul, S. F. et al. Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Res. 25, 3389–3402 (1997).

  33. 33.

    Zhang, Y. & Skolnick, J. Scoring function for automated assessment of protein structure template quality. Proteins 57, 702–710 (2004).

  34. 34.

    Wang, G. & Dunbrack, R. L. Jr. PISCES: a protein sequence culling server. Bioinformatics 19, 1589–1591 (2003).

  35. 35.

    Kabsch, W. & Sander, C. Dictionary of protein secondary structure—pattern-recognition of hydrogen-bonded and geometrical features. Biopolymers 22, 2577–2637 (1983).

  36. 36.

    Berman, H. M. et al. The protein data bank. Nucleic Acids Res. 28, 235–242 (2000).

  37. 37.

    Fox, N. K., Brenner, S. E. & Chandonia, J. M. SCOPe: Structural classification of proteins—extended, integrating SCOP and ASTRAL data and classification of new structures. Nucleic Acids Res. 42, D304–D309 (2014).

  38. 38.

    Jones, D. T. Protein secondary structure prediction based on position-specific scoring matrices. J. Mol. Biol. 292, 195–202 (1999).

  39. 39.

    Hubner, I. A., Deeds, E. J. & Shakhnovich, E. I. Understanding ensemble protein folding at atomic detail. Proc. Natl Acad. Sci. USA 103, 17747–17752 (2006).

  40. 40.

    Carugo, O. & Pongor, S. A normalized root-mean-square distance for comparing protein three-dimensional structures. Protein Sci. 10, 1470–1473 (2001).

  41. 41.

    Henikoff, S. & Henikoff, J. G. Amino acid substitution matrices from protein blocks. Proc. Natl Acad. Sci. USA 89, 10915–10919 (1992).

  42. 42.

    Kidera, A., Konishi, Y., Oka, M., Ooi, T. & Scheraga, H. A. Statistical analysis of the physical properties of the 20 naturally occurring amino acids. J. Protein Chem. 4, 23–55 (1985).

  43. 43.

    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014).

  44. 44.

    Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980 (2014).

  45. 45.

    Keskar, N. S. & Socher, R. Improving generalization performance by switching from Adam to SGD. Preprint at https://arxiv.org/abs/1712.07628 (2017).

  46. 46.

    Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proc. 32nd Int. Conf. Machine Learning. Vol. 37 (JMLR, 2015).

  47. 47.

    Abadi, M. et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems (TensorFlow, 2015); http://download.tensorflow.org/paper/whitepaper2015.pdf

  48. 48.

    Zhang, Y. & Skolnick, J. TM-align: a protein structure alignment algorithm based on the TM-score. Nucleic Acids Res. 33, 2302–2309 (2005).

  49. 49.

    Tong, W. et al. Improved fragment sampling for ab initio protein structure prediction using deep neural networks (Code Ocean, 2019); https://doi.org/10.24433/CO.3579011.v1

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (grant numbers 31670723, 81861138009, 91746119 and 31621092) and from the Beijing Advanced Innovation Center for Structural Biology to H.G., as well as by the Australian Research Council (grant number DP180102060) and the National Health and Medical Research Council of Australia (grant number 1121629) to Y.Z.

Author information

Affiliations

Authors

Contributions

T.W. contributed to methodology, experimental design, software, formal analysis, the server and writing of the original draft. Y.Q. contributed to formal analysis and the server. W.D. contributed to the server. W.M. was involved in methodology. Y.Z. was involved in experimental design and writing. H.G. contributed to experimental design and was responsible for supervision, writing and funding acquisition. All authors reviewed the final manuscript.

Corresponding authors

Correspondence to Yaoqi Zhou or Haipeng Gong.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Methods (details of our in-house residue-residue contact refinement models), Supplementary Figs. S1–S22, Supplementary Tables S1–S11, and Supplementary references

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, T., Qiao, Y., Ding, W. et al. Improved fragment sampling for ab initio protein structure prediction using deep neural networks. Nat Mach Intell 1, 347–355 (2019). https://doi.org/10.1038/s42256-019-0075-7

Download citation

Further reading