Protein structure prediction can be used to determine the three-dimensional shape of a protein from its amino acid sequence1. This problem is of fundamental importance as the structure of a protein largely determines its function2; however, protein structures can be difficult to determine experimentally. Considerable progress has recently been made by leveraging genetic information. It is possible to infer which amino acid residues are in contact by analysing covariation in homologous sequences, which aids in the prediction of protein structures3. Here we show that we can train a neural network to make accurate predictions of the distances between pairs of residues, which convey more information about the structure than contact predictions. Using this information, we construct a potential of mean force4 that can accurately describe the shape of a protein. We find that the resulting potential can be optimized by a simple gradient descent algorithm to generate structures without complex sampling procedures. The resulting system, named AlphaFold, achieves high accuracy, even for sequences with fewer homologous sequences. In the recent Critical Assessment of Protein Structure Prediction5 (CASP13)—a blind assessment of the state of the field—AlphaFold created high-accuracy structures (with template modelling (TM) scores6 of 0.7 or higher) for 24 out of 43 free modelling domains, whereas the next best method, which used sampling and contact information, achieved such accuracy for only 14 out of 43 domains. AlphaFold represents a considerable advance in protein-structure prediction. We expect this increased accuracy to enable insights into the function and malfunction of proteins, especially in cases for which no structures for homologous proteins have been experimentally determined7.
Subscribe to Journal
Get full journal access for 1 year
We are sorry, but there is no personal subscription option available for your country.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Our training, validation and test data splits (CATH domain codes) are available from https://github.com/deepmind/deepmind-research/tree/master/alphafold_casp13. The following versions of public datasets were used in this study: PDB 2018-03-15; CATH 2018-03-16; Uniclust30 2017-10; and PSI-BLAST nr dataset (as of 15 December 2017).
Source code for the distogram, reference distogram and torsion prediction neural networks, together with the neural network weights and input data for the CASP13 targets are available for research and non-commercial use at https://github.com/deepmind/deepmind-research/tree/master/alphafold_casp13. We make use of several open-source libraries to conduct our experiments, particularly HHblits36, PSI-BLAST37 and the machine-learning framework TensorFlow (https://github.com/tensorflow/tensorflow) along with the TensorFlow library Sonnet (https://github.com/deepmind/sonnet), which provides implementations of individual model components50. We also used Rosetta9 under license.
Dill, K. A., Ozkan, S. B., Shell, M. S. & Weikl, T. R. The protein folding problem. Annu. Rev. Biophys. 37, 289–316 (2008).
Dill, K. A. & MacCallum, J. L. The protein-folding problem, 50 years on. Science 338, 1042–1046 (2012).
Schaarschmidt, J., Monastyrskyy, B., Kryshtafovych, A. & Bonvin, A. M. J. J. Assessment of contact predictions in CASP12: co-evolution and deep learning coming of age. Proteins 86, 51–66 (2018).
Kirkwood, J. Statistical mechanics of fluid mixtures. J. Chem. Phys. 3, 300–313 (1935).
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K. & Moult, J. Critical assessment of methods of protein structure prediction (CASP)—Round XIII. Proteins 87, 1011–1020 (2019).
Zhang, Y. & Skolnick, J. Scoring function for automated assessment of protein structure template quality. Proteins 57, 702–710 (2004).
Zhang, Y. Protein structure prediction: when is it useful? Curr. Opin. Struct. Biol. 19, 145–155 (2009).
Senior, A. W. et al. Protein structure prediction using multiple deep neural networks in the 13th Critical Assessment of Protein Structure Prediction (CASP13). Proteins 87, 1141–1148 (2019).
Das, R. & Baker, D. Macromolecular modeling with Rosetta. Annu. Rev. Biochem. 77, 363–382 (2008).
Jones, D. T. Predicting novel protein folds by using FRAGFOLD. Proteins 45, 127–132 (2001).
Zhang, C., Mortuza, S. M., He, B., Wang, Y. & Zhang, Y. Template-based and free modeling of I-TASSER and QUARK pipelines using predicted contact maps in CASP12. Proteins 86, 136–151 (2018).
Kirkpatrick, S., Gelatt, C. D. Jr & Vecchi, M. P. Optimization by simulated annealing. Science 220, 671–680 (1983).
Berman, H. M. et al. The Protein Data Bank. Nucleic Acids Res. 28, 235–242 (2000).
Altschuh, D., Lesk, A. M., Bloomer, A. C. & Klug, A. Correlation of co-ordinated amino acid substitutions with function in viruses related to tobacco mosaic virus. J. Mol. Biol. 193, 693–707 (1987).
Ovchinnikov, S., Kamisetty, H. & Baker, D. Robust and accurate prediction of residue–residue interactions across protein interfaces using evolutionary information. eLife 3, e02030 (2014).
Seemayer, S., Gruber, M. & Söding, J. CCMpred—fast and precise prediction of protein residue–residue contacts from correlated mutations. Bioinformatics 30, 3128–3130 (2014).
Morcos, F. et al. Direct-coupling analysis of residue coevolution captures native contacts across many protein families. Proc. Natl Acad. Sci. USA 108, E1293–E1301 (2011).
Jones, D. T., Buchan, D. W., Cozzetto, D. & Pontil, M. PSICOV: precise structural contact prediction using sparse inverse covariance estimation on large multiple sequence alignments. Bioinformatics 28, 184–190 (2012).
Skwark, M. J., Raimondi, D., Michel, M. & Elofsson, A. Improved contact predictions using the recognition of protein like contact patterns. PLOS Comput. Biol. 10, e1003889 (2014).
Jones, D. T., Singh, T., Kosciolek, T. & Tetchner, S. MetaPSICOV: combining coevolution methods for accurate prediction of contacts and long range hydrogen bonding in proteins. Bioinformatics 31, 999–1006 (2015).
Wang, S., Sun, S., Li, Z., Zhang, R. & Xu, J. Accurate de novo prediction of protein contact map by ultra-deep learning model. PLOS Comput. Biol. 13, e1005324 (2017).
Jones, D. T. & Kandathil, S. M. High precision in protein contact prediction using fully convolutional neural networks and minimal sequence features. Bioinformatics 34, 3308–3315 (2018).
Ovchinnikov, S. et al. Improved de novo structure prediction in CASP11 by incorporating coevolution information into Rosetta. Proteins 84, 67–75 (2016).
Aszódi, A. & Taylor, W. R. Estimating polypeptide α-carbon distances from multiple sequence alignments. J. Math. Chem. 17, 167–184 (1995).
Zhao, F. & Xu, J. A position-specific distance-dependent statistical potential for protein structure and functional study. Structure 20, 1118–1126 (2012).
Xu, J. & Wang, S. Analysis of distance-based protein structure prediction by deep learning in CASP13. Proteins 87, 1069–1081 (2019).
Aszódi, A., Gradwell, M. J. & Taylor, W. R. Global fold determination from a small number of distance restraints. J. Mol. Biol. 251, 308–326 (1995).
Kandathil, S. M., Greener, J. G. & Jones, D. T. Prediction of interresidue contacts with DeepMetaPSICOV in CASP13. Proteins 87, 1092–1099 (2019).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (2016).
Simons, K. T., Kooperberg, C., Huang, E. & Baker, D. Assembly of protein tertiary structures from fragments with similar local sequences using simulated annealing and Bayesian scoring functions. J. Mol. Biol. 268, 209–225 (1997).
Liu, D. C. & Nocedal, J. On the limited memory BFGS method for large scale optimization. Math. Program. 45, 503–528 (1989).
Li, Y., Zhang, C., Bell, E. W., Yu, D.-J. & Zhang, Y. Ensembling multiple raw coevolutionary features with deep residual neural networks for contact-map prediction in CASP13. Proteins 87, 1082–1091 (2019).
Konagurthu, A. S., Lesk, A. M. & Allison, L. Minimum message length inference of secondary structure from protein coordinate data. Bioinformatics 28, i97–i105 (2012).
Dawson, N. L. et al. CATH: an expanded resource to predict protein function through structure and sequence. Nucleic Acids Res. 45, D289–D295 (2017).
Mirdita, M. et al. Uniclust databases of clustered and deeply annotated protein sequences and alignments. Nucleic Acids Res. 45, D170–D176 (2017).
Remmert, M., Biegert, A., Hauser, A. & Söding, J. HHblits: lightning-fast iterative protein sequence searching by HMM–HMM alignment. Nat. Methods 9, 173–175 (2012).
Altschul, S. F. et al. Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Res. 25, 3389–3402 (1997).
Yu, F. & Koltun, V. Multi-scale context aggregation by dilated convolutions. Preprint at arXiv https://arxiv.org/abs/1511.07122 (2015).
Oord, A. d. et al. Wavenet: a generative model for raw audio. Preprint at arXiv https://arxiv.org/abs/1609.03499 (2016).
Clevert, D.-A., Unterthiner, T. & Hochreiter, S. Fast and accurate deep network learning by exponential linear units (ELUs). Preprint at arXiv https://arxiv.org/abs/1511.07289 (2015).
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014).
Kabsch, W. & Sander, C. Dictionary of protein secondary structure: pattern recognition of hydrogen-bonded and geometrical features. Biopolymers 22, 2577–2637 (1983).
Yang, Y. et al. Sixty-five years of the long march in protein secondary structure prediction: the final stretch? Briefings Bioinf. 19, 482–494 (2018).
Zemla, A., Venclovas, C., Moult, J. & Fidelis, K. Processing and analysis of CASP3 protein structure predictions. Proteins 37, 22–29 (1999).
Mariani, V., Biasini, M., Barbato, A. & Schwede, T. lDDT: a local superposition-free score for comparing protein structures and models using distance difference tests. Bioinformatics 29, 2722–2728 (2013).
Abriata, L. A., Tamo, G. E. & Dal Peraro, M. A further leap of improvement in tertiary structure prediction in CASP13 prompts new routes for future assessments. Proteins 87, 1100–1112 (2019).
Kayikci, M. et al. Visualization and analysis of non-covalent contacts using the Protein Contacts Atlas. Nat. Struct. Mol. Biol. 25, 185–194 (2018).
Croll, T. I. et al. Evaluation of template-based modeling in CASP13. Proteins 87, 1113–1127 (2019).
Sundararajan, M., Taly, A. & Yan, Q. Axiomatic attribution for deep networks. In Proc. 34th International Conference on Machine Learning Vol. 70, 3319–3328 (2017).
Abadi, M. et al. Tensorflow: a system for large-scale machine learning. In Proc. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) 265–283 (2016).
Söding, J., Biegert, A. & Lupas, A. N. The HHpred interactive server for protein homology detection and structure prediction. Nucleic Acids Res. 33, W244–W248 (2005).
Cong, Q. et al. An automatic method for CASP9 free modeling structure prediction assessment. Bioinformatics 27, 3371–3378 (2011).
Zhang, Y. & Skolnick, J. TM-align: a protein structure alignment algorithm based on the TM-score. Nucleic Acids Res. 33, 2302–2309 (2005).
Tovchigrechko, A., Wells, C. A. & Vakser, I. A. Docking of protein models. Protein Sci. 11, 1888–1896 (2002).
Audet, M. et al. Crystal structure of misoprostol bound to the labor inducer prostaglandin E2 receptor. Nat. Chem. Biol. 15, 11–17 (2019).
We thank C. Meyer for assistance in preparing the paper; B. Coppin, O. Vinyals, M. Barwinski, R. Sun, C. Elkin, P. Dolan, M. Lai and Y. Li for their contributions and support; O. Ronneberger for reading the paper; the rest of the DeepMind team for their support; the CASP13 organisers and the experimentalists whose structures enabled the assessment.
A.W.S., J.K., T.G., J.J., L.S., R.E., H.P., C.Q., K.S., A.Ž. and A.B. have filed provisional patent applications relating to machine learning for predicting protein structures. The remaining authors declare no competing interests.
Peer review information Nature thanks Mohammed AlQuraishi and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data figures and tables
a, The overall folding system. Feature extraction stages (constructing the MSA using sequence database search and computing MSA-based features) are shown in yellow; the structure-prediction neural network in green; potential construction in red; and structure realization in blue. b, The layers used in one block of the deep residual convolutional network. The dilated convolution is applied to activations of reduced dimension. The output of the block is added to the representation from the previous layer. The bypass connections of the residual network enable gradients to pass back through the network undiminished, permitting the training of very deep networks.
a, Precisions (as shown in Fig. 1c) for long-range contact prediction in CASP13 for the most probable L, L/2 or L/5 contacts, where L is the length of the domain. The distance distributions used by AlphaFold (AF) in CASP13, thresholded to contact predictions, are compared with submissions by the two best-ranked contact prediction methods in CASP13: 498 (RaptorX-Contact26) and 032 (TripletRes32), on ‘all groups’ targets, with updated domain definitions for T0953s2. b, c, True distances (b) and modes of the predicted distogram (c) for CASP13 target T0990. CASP divides this chain into three domains as shown (D3 is inserted in D2) for which there are 39, 36 and 42 HHblits alignments, respectively (from the CASP website).
a, lDDT12 versus distogram lDDT12 (see Methods, ‘Accuracy’). The distogram accuracy predicts the lDDT of the realized structure well (particularly for medium- and long-range residue pairs, as well as the TM score as shown in Fig. 4a) for both CASP13 (n = 500: 5 decoys for domains excluding T0999) and test (n = 377) datasets. Data are shown with Pearson’s correlation coefficients. b, DLDDT12 against the effective number of sequences in the MSA (Neff) normalized by sequence length (n = 377). The number of effective sequences correlates with this measure of distogram accuracy (r = 0.634). c, Structure accuracy measures, computed on the test set (n = 377), for gradient descent optimization of different forms of the potential. Top, removing terms in the potential, and showing the effect of following optimization with Rosetta relax. ‘P’ shows the significance of the potential giving different results from ‘Full’, for a two-tailed paired data t-test. ‘Bins’ shows the number of bins fitted by the spline before extrapolation and the number in the full distribution. In CASP13, splines were fitted to the first 51 of 64 bins. Bottom, reducing the resolution of the distogram distributions. The original 64-bin distogram predictions are repeatedly downsampled by a factor of 2 by summing adjacent bins, in each case with constant extrapolation beyond 18 Å (the last quarter of the bins). The two-level potential in the final row, which was designed to compare with contact predictions, is constructed by summing the probability mass below 8 Å and between 8 and 14 Å, with constant extrapolation beyond 14 Å. The TM scores in this table are plotted in Fig. 4b.
Extended Data Fig. 4 TM score versus per-target computation time computed as an average over the test set.
Structure realization requires a modest computation budget, which can be parallelized over multiple machines. Full optimization with noisy restarts (orange) is compared with initialization from sampled torsions (blue). Computation is measured as the product of the number of (CPU-based) machines and time elapsed and can be largely parallelized. Longer targets take longer to optimize. Figure 2e shows how the TM score increases with the number of repeats of gradient descent. n = 377.
a, The TM score for each of the five AlphaFold CASP13 submissions are shown. Simulated annealing with fragment assembly entries are shown in blue. Gradient-descent entries are shown in yellow. Gradient descent was only used for targets T0975 and later, so to the left of the black line we also show the results for a single ‘back-fill’ run of gradient descent for each earlier target using the deployed system. T0999 (1,589 residues) was manually segmented based on HHpred51 homology matching. b, Average TM scores of the AlphaFold CASP13 submissions (n = 104 domains), comparing the first model submitted, the best-of-five model (submission with highest GDT_TS), a single run of full-chain gradient descent (a CASP13 run for T0975 and later, back-fill for earlier targets) and a single CASP13 run of fragment assembly with domain segmentation (using a gradient descent submission for T0999). c, The formula-standardized (z) scores of the assessors for GDT TS + QCS52, best-of-five for CASP FM (n = 31) and FM/TBM (n = 12) domains comparing AlphaFold with the closest competitor (group 322), coloured by domain category. AlphaFold performs better (P = 0.0032, one-tailed paired statistic t-test).
Often protein function can be inferred by finding homologous proteins of known function. Here we show that the FM predictions of AlphaFold give greater accuracy in a structure-based search for homologous domains in the CATH database. For each of the FM or TBM/FM domains, the top-one submission and ground truth are compared to all 30,744 CATH S40 non-redundant domains with TM-align53. For the 36 domains for which there is a good ground-truth match (score > 0.5), we show the percentage of decoys for which a domain with the same CATH code (CATH in red, CA in green; CAT results are close to CATH results) as the top ground-truth match is in the top-k matches with score > 0.5. Curves are shown for AlphaFold and the next-best group (322). AlphaFold predictions determine the matching fold more accurately. Determination of the matching CATH domain can provide insights into the function of a new protein.
Protein–protein interaction is an important domain for understanding protein function that has hitherto largely been limited to template-based models because of the need for high-accuracy predictions, although there has been moderate success54 in docking with predicted structures up to 6 Å r.m.s.d. This figure shows that the predictions by AlphaFold improve accuracy in the interface regions of chains in hetero-dimer structures and are probably better candidates for docking, although docking did not form part of the AlphaFold system and all submissions were for isolated chains rather than complexes. For the five all-groups heterodimer CASP13 targets, the full-atom r.m.s.d. values of the interface residues (residues with a ground-truth inter-chain heavy-atom distance <10 Å) are computed for the chain submissions of all groups (green), relative to the target complex. Results >8 Å are not shown. AlphaFold (blue) achieves consistently high accuracy interface regions and, for 4 out of 5 targets, predicts interfaces below <5 Å for both chains.
T1011 (PDB 6M9T) is the EP3 receptor bound to misoprostol-FA55. a, The native structure showing the ligand in a pocket. b, c, Submission 5 (78.0 GDT TS) by AlphaFold (b), made without knowledge of the ligand, shows a pocket more similar to the true pocket than that of the best other submission (322, model 3, 68.7 GDT TS) (c). Both submissions are aligned to the native protein using the same subset of residues from the helices close to the ligand pocket and visualized with the interior pocket together with the native ligand position.
The contact probability map of T0986s2, and the summed absolute value of the Integrated Gradient, ∑c|SI,Ji,j,c|, of the input two-dimensional features with respect to the expected distance between five different pairs of residues (I,J): (1) a helix self-contact, (2) a long-range strand–strand contact, (3) a medium-range strand–strand contact, (4) a non-contact and (5) a very long-range strand–strand contact. Each pair is shown as two red dots on the diagrams. Darker colours indicate a higher attribution weight.
For T0986s2 (TM score 0.8), the top 10 input pairs, including self-pairs, with the highest attribution weight for each of the five output pairs shown in Extended Data Fig. 9, are shown as lines (or spheres for self-pairs) coloured by sensitivity, lighter green colours indicate more sensitive, and the output pair is shown as a blue line.
About this article
Cite this article
Senior, A.W., Evans, R., Jumper, J. et al. Improved protein structure prediction using potentials from deep learning. Nature 577, 706–710 (2020). https://doi.org/10.1038/s41586-019-1923-7
Single-sequence and profile-based prediction of RNA solvent accessibility using dilated convolutional neural network
Informatics in Medicine Unlocked (2021)
BMC Bioinformatics (2021)
Current Opinion in Structural Biology (2021)