Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Geometric deep learning on molecular representations

Nature Machine Intelligence volume 3, pages 1023–1032 (2021)Cite this article

Subjects

Abstract

Geometric deep learning (GDL) is based on neural network architectures that incorporate and process symmetry information. GDL bears promise for molecular modelling applications that rely on molecular representations with different symmetry properties and levels of abstraction. This Review provides a structured and harmonized overview of molecular GDL, highlighting its applications in drug discovery, chemical synthesis prediction and quantum chemistry. It contains an introduction to the principles of GDL, as well as relevant molecular representations, such as molecular graphs, grids, surfaces and strings, and their respective properties. The current challenges for GDL in the molecular sciences are discussed, and a forecast of future opportunities is attempted.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Exemplary molecular representations for a selected molecule.
Fig. 2: Deep learning on molecular graphs.
Fig. 3: Chemical language modelling.

Similar content being viewed by others

References

  1. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  Google Scholar 

  2. Schmidhuber, J. Deep learning in neural networks: an overview. Neur. Netw. 61, 85–117 (2015).

    Article  Google Scholar 

  3. Gawehn, E., Hiss, J. A. & Schneider, G. Deep learning in drug discovery. Mol. Inform. 35, 3–14 (2016).

    Article  Google Scholar 

  4. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. In International Conference on Machine Learning Vol. 34, 1263–1272 (2017).

  5. Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021).

    Article  Google Scholar 

  6. Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems Vol. 30, 5998–6008 (2017).

  7. Hinton, G. et al. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Proces. Mag. 29, 82–97 (2012).

    Article  Google Scholar 

  8. Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems Vol. 25, 1097–1105 (2012).

  9. Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A. & Vandergheynst, P. Geometric deep learning: going beyond euclidean data. IEEE Signal Process. Mag. 34, 18–42 (2017).

    Article  Google Scholar 

  10. Todeschini, R. & Consonni, V. Molecular Descriptors for Chemoinformatics Vols I–II (John Wiley & Sons, 2009).

  11. Townshend, R. J. et al. Geometric deep learning of RNA structure. Science 373, 1047–1051 (2021).

    Article  Google Scholar 

  12. Gainza, P. et al. Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning. Nat. Methods 17, 184–192 (2020).

    Article  Google Scholar 

  13. Segler, M. H., Kogej, T., Tyrchan, C. & Waller, M. P. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS Centr. Sci. 4, 120–131 (2018).

    Article  Google Scholar 

  14. Bronstein, M. M., Bruna, J., Cohen, T. & Veličković, P. Geometric deep learning: grids, groups, graphs, geodesics, and gauges. Preprint at https://arxiv.org/abs/2104.13478 (2021).

  15. Mumford, D., Fogarty, J. & Kirwan, F. Geometric Invariant Theory Vol. 34 (Springer Science & Business Media, 1994).

  16. Cohen, T. S. & Welling, M. Group equivariant convolutional networks. In International Conference on Machine Learning Vol. 33, 2990–2999 (2016).

  17. Kondor, R. & Trivedi, S. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International Conference on Machine Learning Vol. 35, 2747–2755 (2018).

  18. Moriguchi, I., Hirono, S., Liu, Q., Nakagome, I. & Matsushita, Y. Simple method of calculating octanol/water partition coefficient. Chem. Pharmaceut. Bull. 40, 127–130 (1992).

    Article  Google Scholar 

  19. Cybenko, G. Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2, 303–314 (1989).

    Article  MathSciNet  MATH  Google Scholar 

  20. Hoffmann, R. & Laszlo, P. Representation in chemistry. Angew. Chem. Int. Ed. Engl. 30, 1–16 (1991).

    Article  Google Scholar 

  21. Kipf, T. N. & Welling, M. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations Vol 5. (2017).

  22. Duvenaud, D. et al. Convolutional networks on graphs for learning molecular fingerprints. In International Conference on Neural Information Processing Systems, Vol. 28, 2224–2232 (2015).

  23. Monti, F. et al. Geometric deep learning on graphs and manifolds using mixture model CNNs. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 5115–5124 (2017).

  24. Battaglia, P. et al. Interaction networks for learning about objects, relations and physics. In Advances in Neural Information Processing Systems Vol. 29, 4502–4510 (2016).

  25. Battaglia, P. W. et al. Relational inductive biases, deep learning, and graph networks. Preprint at https://arxiv.org/abs/1806.01261 (2018).

  26. Zhou, J. et al. Graph neural networks: a review of methods and applications. AI Open 1, 57–81 (2020).

    Article  Google Scholar 

  27. Schütt, K. T., Arbabzadah, F., Chmiela, S., Müller, K. R. & Tkatchenko, A. Quantum-chemical insights from deep tensor neural networks. Nat. Commun. 8, 13890 (2017).

    Article  Google Scholar 

  28. Klicpera, J., Groß, J. & Günnemann, S. Directional message passing for molecular graphs. In International Conference on Learning Representations Vol. 8 (2020).

  29. Feinberg, E. N. et al. PotentialNet for molecular property prediction. ACS Central Science 4, 1520–1530 (2018).

    Article  Google Scholar 

  30. Torng, W. & Altman, R. B. Graph convolutional neural networks for predicting drug-target interactions. J. Chem. Inf. Model. 59, 4131–4149 (2019).

    Article  Google Scholar 

  31. Stokes, J. M. et al. A deep learning approach to antibiotic discovery. Cell 180, 688–702 (2020).

    Article  Google Scholar 

  32. Somnath, V. R., Bunne, C., Coley, C. W., Krause, A. & Barzilay, R. Learning graph models for retrosynthesis prediction. In Advances in Neural Information Processing Systems Vol. 34 (2021).

  33. Coley, C. W. et al. A graph-convolutional neural network model for the prediction of chemical reactivity. Chem. Sci. 10, 370–377 (2019).

    Article  Google Scholar 

  34. Li, J., Cai, D. & He, X. Learning graph-level representation for drug discovery. Preprint at https://arxiv.org/abs/1709.03741 (2017).

  35. Liu, K. et al. Chemi-Net: a molecular graph convolutional network for accurate drug property prediction. Int. J. Mol. Sci. 20, 3389 (2019).

    Article  Google Scholar 

  36. Unke, O. T. & Meuwly, M. PhysNet: a neural network for predicting energies, forces, dipole moments, and partial charges. J. Chem. Theory Comput. 15, 3678–3693 (2019).

    Article  Google Scholar 

  37. Schütt, K. T., Unke, O. T. & Gastegger, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. Preprint at https://arxiv.org/abs/2102.03150 (2021).

  38. Schütt, K. T., Sauceda, H. E., Kindermans, P.-J., Tkatchenko, A. & Müller, K.-R. SchNet–a deep learning architecture for molecules and materials. J. Chem. Phys. 148, 241722 (2018).

    Article  Google Scholar 

  39. Schütt, K., Gastegger, M., Tkatchenko, A., Müller, K.-R. & Maurer, R. J. Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions. Nat. Commun. 10, 5024 (2019).

    Article  Google Scholar 

  40. Bogojeski, M., Vogt-Maranto, L., Tuckerman, M. E., Müller, K.-R. & Burke, K. Quantum chemical accuracy from density functional approximations via machine learning. Nat. Commun. 11, 5223 (2020).

    Article  Google Scholar 

  41. Yang, K. et al. Analyzing learned molecular representations for property prediction. J. Chem. Inf. Model. 59, 3370–3388 (2019).

    Article  Google Scholar 

  42. Axelrod, S. & Gomez-Bombarelli, R. Molecular machine learning with conformer ensembles. Preprint at https://arxiv.org/abs/2012.08452 (2020).

  43. Jiménez-Luna, J., Grisoni, F. & Schneider, G. Drug discovery with explainable artificial intelligence. Nat. Mach. Intell. 2, 573–584 (2020).

    Article  Google Scholar 

  44. Jiménez-Luna, J., Skalic, M., Weskamp, N. & Schneider, G. Coloring molecules with explainable artificial intelligence for preclinical relevance assessment. J. Chem. Inf. Model. 61, 1083–1094 (2021).

    Article  Google Scholar 

  45. Schnake, T. et al. Xai for graphs: explaining graph neural network predictions by identifying relevant walks. Preprint at https://arxiv.org/abs/2006.03589 (2020).

  46. Sun, M., Xing, J., Wang, H., Chen, B. & Zhou, J. MoCL: data-driven molecular fingerprint via knowledge-aware contrastive learning from molecular graph. In Proc. 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining 3585–3594 (Association for Computing Machinery, 2021); https://doi.org/10.1145/3447548.3467186

  47. Coley, C. W., Barzilay, R., Green, W. H., Jaakkola, T. S. & Jensen, K. F. Convolutional embedding of attributed molecular graphs for physical property prediction. J. Chem. Inf. Model. 57, 1757–1772 (2017).

    Article  Google Scholar 

  48. Li, Y., Vinyals, O., Dyer, C., Pascanu, R. & Battaglia, P. Learning deep generative models of graphs. Preprint at https://arxiv.org/abs/1803.03324 (2018).

  49. Simonovsky, M. & Komodakis, N. GraphVAE: towards generation of small graphs using variational autoencoders. In International Conference on Artificial Neural Networks Vol. 27, 412–422 (2018).

  50. De Cao, N. & Kipf, T. MolGAN: An implicit generative model for small molecular graphs. Preprint at https://arxiv.org/abs/1805.11973 (2018).

  51. Zhou, Z., Kearnes, S., Li, L., Zare, R. N. & Riley, P. Optimization of molecules via deep reinforcement learning. Sci. Rep. 9, 10752 (2019).

    Article  Google Scholar 

  52. Jin, W., Barzilay, R. & Jaakkola, T. Junction tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning Vol. 35, 2323–2332 (2018).

  53. You, J., Liu, B., Ying, Z., Pande, V. & Leskovec, J. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in Neural Information Processing Systems Vol. 31, 6410–6421 (2018).

  54. Jin, W., Barzilay, R. & Jaakkola, T. Multi-objective molecule generation using interpretable substructures. In International Conference on Machine Learning Vol. 37, 4849–4859 (2020).

  55. Lei, T., Jin, W., Barzilay, R. & Jaakkola, T. Deriving neural architectures from sequence and graph kernels. In International Conference on Machine Learning Vol. 34, 2024-2033 (2017).

  56. Xu, K., Hu, W., Leskovec, J. & Jegelka, S. How powerful are graph neural networks? Preprint at https://arxiv.org/abs/1810.00826 (2018).

  57. Chen, Z., Chen, L., Villar, S. & Bruna, J. Can graph neural networks count substructures? In Advances in Neural Information Processing Systems Vol. 33, 10383–10395 (2020).

  58. Bouritsas, G., Frasca, F., Zafeiriou, S. & Bronstein, M. M. Improving graph neural network expressivity via subgraph isomorphism counting. Preprint at https://arxiv.org/abs/2006.09252 (2020).

  59. Bodnar, C. et al. Weisfeiler and Lehman go topological: message passing simplicial networks. In International Conference on Learning Representations: Workshop on Geometrical and Topological Representation Learning (2021).

  60. Thomas, N. et al. Tensor field networks: rotation-and translation-equivariant neural networks for 3D point clouds. Preprint at https://arxiv.org/abs/1802.08219 (2018).

  61. Satorras, V. G., Hoogeboom, E. & Welling, M. E(n) equivariant graph neural networks. Preprint at https://arxiv.org/abs/2102.09844 (2021).

  62. Anderson, B., Hy, T. S. & Kondor, R. Cormorant: covariant molecular neural networks. In Advances in Neural Information Processing Systems Vol. 32, 14537–14546 (2019).

  63. Miller, B. K., Geiger, M., Smidt, T. E. & Noé, F. Relevance of rotationally equivariant convolutions for predicting molecular properties. Preprint at https://arxiv.org/abs/2008.08461 (2020).

  64. Fuchs, F., Worrall, D., Fischer, V. & Welling, M. SE(3)-transformers: 3D roto-translation equivariant attention networks. In Advances in Neural Information Processing Systems Vol. 33 (2020).

  65. Unke, O. T. et al. SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects. Preprint at https://arxiv.org/abs/2105.00304 (2021).

  66. Batzner, S. et al. SE(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Preprint at https://arxiv.org/abs/2101.03164 (2021).

  67. Unke, O. T. et al. SE(3)-equivariant prediction of molecular wavefunctions and electronic densities. Preprint at https://arxiv.org/abs/2106.02347 (2021).

  68. Hermann, J., Schätzle, Z. & Noé, F. Deep-neural-network solution of the electronic Schrödinger equation. Nat. Chem. 12, 891–897 (2020).

    Article  Google Scholar 

  69. Pfau, D., Spencer, J. S., Matthews, A. G. & Foulkes, W. M. C. Ab initio solution of the many-electron Schrödinger equation with deep neural networks. Phys. Rev. Res. 2, 033429 (2020).

    Article  Google Scholar 

  70. Choo, K., Mezzacapo, A. & Carleo, G. Fermionic neural-network states for ab-initio electronic structure. Nat. Commun. 11, 2368 (2020).

    Article  Google Scholar 

  71. Rajan, K., Zielesny, A. & Steinbeck, C. Decimer: towards deep learning for chemical image recognition. J. Cheminform. 12, 65 (2020).

    Article  Google Scholar 

  72. Cramer, R. D., Patterson, D. E. & Bunce, J. D. Comparative molecular field analysis (comfa). 1. effect of shape on binding of steroids to carrier proteins. J. Am. Chem. Soc. 110, 5959–5967 (1988).

    Article  Google Scholar 

  73. Klebe, G. in 3D QSAR in Drug Design (eds. Kubinyi, H. et al.) 87–104 (Springer, 1998).

  74. Jiménez, J., Skalic, M., Martinez-Rosell, G. & De Fabritiis, G. KDEEP: protein–ligand absolute binding affinity prediction via 3d-convolutional neural networks. J. Chem. Inf. Model. 58, 287–296 (2018).

    Article  Google Scholar 

  75. Ragoza, M., Hochuli, J., Idrobo, E., Sunseri, J. & Koes, D. R. Protein–ligand scoring with convolutional neural networks. J. Chem. Inf. Model. 57, 942–957 (2017).

    Article  Google Scholar 

  76. Jiménez, J., Doerr, S., Martinez-Rosell, G., Rose, A. S. & De Fabritiis, G. DeepSite: protein-binding site predictor using 3d-convolutional neural networks. Bioinformatics 33, 3036–3042 (2017).

    Article  Google Scholar 

  77. Ahmed, E. et al. A survey on deep learning advances on different 3d data representations. Preprint at https://arxiv.org/abs/1808.01462 (2018).

  78. Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A. & Battaglia, P. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations Vol. 8 (2020).

  79. Liu, Q. et al. OctSurf: efficient hierarchical voxel-based molecular surface representation for protein-ligand affinity prediction. J. Molec. Graph. Model. 105, 107865 (2021).

    Article  Google Scholar 

  80. Mylonas, S. K., Axenopoulos, A. & Daras, P. DeepSurf: a surface-based deep learning approach for the prediction of ligand binding sites on proteins. Preprint at https://arxiv.org/abs/2002.05643 (2020).

  81. Wiswesser, W. J. Historic development of chemical notations. J. Chem. Inf. Comput. Sci. 25, 258–263 (1985).

    Article  Google Scholar 

  82. Wiswesser, W. J. The Wiswesser line formula notation. Chem. Eng. News Arch. 30, 3523–3526 (1952).

    Article  Google Scholar 

  83. Heller, S., McNaught, A., Stein, S., Tchekhovskoi, D. & Pletnev, I. InChI: the worldwide chemical structure identifier standard. J. Cheminform. 5, 7 (2013).

    Article  Google Scholar 

  84. Weininger, D. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. J. Chem. Inf. Comput. Sci. 28, 31–36 (1988).

    Article  Google Scholar 

  85. Öztürk, H., Özgür, A., Schwaller, P., Laino, T. & Ozkirimli, E. Exploring chemical space using natural language processing methodologies for drug discovery. Drug Discov. Today 25, 689–705 (2020).

    Article  Google Scholar 

  86. Cadeddu, A., Wylie, E. K., Jurczak, J., Wampler-Doty, M. & Grzybowski, B. A. Organic chemistry as a language and the implications of chemical linguistics for structural and retrosynthetic analyses. Angew. Chem. Int. Ed. 53, 8108–8112 (2014).

    Article  Google Scholar 

  87. Schwaller, P., Gaudin, T., Lanyi, D., Bekas, C. & Laino, T. Found in translation: predicting outcomes of complex organic chemistry reactions using neural sequence-to-sequence models. Chem. Sci. 9, 6091–6098 (2018).

    Article  Google Scholar 

  88. O’Boyle, N. M. Towards a universal SMILES representation: a standard method to generate canonical SMILES based on the InChI. J. Cheminform. 4, 22 (2012).

    Article  Google Scholar 

  89. Weininger, D., Weininger, A. & Weininger, J. L. SMILES. 2. Algorithm for generation of unique SMILES notation. J. Chem. Inf. Comput. Sci. 29, 97–101 (1989).

    Article  MATH  Google Scholar 

  90. Gómez-Bombarelli, R. et al. Automatic chemical design using a data-driven continuous representation of molecules. ACS Centr. Sci. 4, 268–276 (2018).

    Article  Google Scholar 

  91. O’Boyle, N. & Dalke, A. DeepSmiles: An adaptation of smiles for use in machine-learning of chemical structures. Preprint at https://doi.org/10.26434/chemrxiv.7097960.v1 (2018).

  92. Krenn, M., Häse, F., Nigam, A., Friederich, P. & Aspuru-Guzik, A. Self-referencing embedded strings (SELFIES): a 100% robust molecular string representation. Mach. Learn. 1, 045024 (2020).

    Google Scholar 

  93. Skinnider, M. A., Stacey, R. G., Wishart, D. S. & Foster, L. J. Chemical language models enable navigation in sparsely populated chemical space. Nat. Mach. Intell. 3, 759–770 (2021).

  94. Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning Internal Representations by Error Propagation (eds. Rumelhart, D. E. & McClelland, J. L.) 318–362 (MIT Press, 1985).

  95. Hochreiter, S. The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncert. Fuzz. Knowl. Based Syst. 6, 107–116 (1998).

    Article  MATH  Google Scholar 

  96. Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neur. Comput. 9, 1735–1780 (1997).

    Article  Google Scholar 

  97. Chung, J., Gulcehre, C., Cho, K. & Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. Preprint at https://arxiv.org/abs/1412.3555 (2014).

  98. Yuan, W. et al. Chemical space mimicry for drug discovery. J. Chem. Inf. Model. 57, 875–882 (2017).

    Article  Google Scholar 

  99. Gupta, A. et al. Generative recurrent networks for de novo drug design. Mol. Inform. 37, 1700111 (2018).

    Article  Google Scholar 

  100. Merk, D., Friedrich, L., Grisoni, F. & Schneider, G. De novo design of bioactive small molecules by artificial intelligence. Mol. Inform. 37, 1700153 (2018).

    Article  Google Scholar 

  101. Olivecrona, M., Blaschke, T., Engkvist, O. & Chen, H. Molecular de-novo design through deep reinforcement learning. J. Cheminform. 9, 48 (2017).

    Article  Google Scholar 

  102. Popova, M., Isayev, O. & Tropsha, A. Deep reinforcement learning for de novo drug design. Sci. Adv. 4, eaap7885 (2018).

    Article  Google Scholar 

  103. Grisoni, F. et al. Combining generative artificial intelligence and on-chip synthesis for de novo drug design. Sci. Adv. 7, eabg3338 (2021).

    Article  Google Scholar 

  104. Arús-Pous, J. et al. Randomized SMILES strings improve the quality of molecular generative models. J. Cheminform. 11, 71 (2019).

    Article  Google Scholar 

  105. Bjerrum, E. J. & Threlfall, R. Molecular generation with recurrent neural networks (RNNs). Preprint at https://arxiv.org/abs/1705.04612 (2017).

  106. Grisoni, F., Moret, M., Lingwood, R. & Schneider, G. Bidirectional molecule generation with recurrent neural networks. J. Chem. Inf. Model. 60, 1175–1183 (2020).

    Article  Google Scholar 

  107. Nagarajan, D. et al. Computational antimicrobial peptide design and evaluation against multidrug-resistant clinical isolates of bacteria. J. Biol. Chem. 293, 3492–3509 (2018).

    Article  Google Scholar 

  108. Grisoni, F. et al. Designing anticancer peptides by constructive machine learning. ChemMedChem 13, 1300–1302 (2018).

    Article  Google Scholar 

  109. Das, P. et al. Accelerated antimicrobial discovery via deep generative models and molecular dynamics simulations. Nat. Biomed. Eng. 5, 613–623 (2021).

    Article  Google Scholar 

  110. Zheng, S., Li, Y., Chen, S., Xu, J. & Yang, Y. Predicting drug–protein interaction using quasi-visual question answering system. Nat. Mach. Intell. 2, 134–140 (2020).

    Article  Google Scholar 

  111. Wang, X. et al. Optimizing pharmacokinetic property prediction based on integrated datasets and a deep learning approach. J. Chem. Inf. Model. 60, 4603–4613 (2020).

    Article  Google Scholar 

  112. Senior, A. W. et al. Protein structure prediction using multiple deep neural networks in the 13th Critical Assessment of Protein Structure Prediction (CASP13). Proteins 87, 1141–1148 (2019).

    Article  Google Scholar 

  113. Tsai, S.-T., Kuo, E.-J. & Tiwary, P. Learning molecular dynamics with simple language model built upon long short-term memory neural network. Nat. Commun. 11, 5155 (2020).

    Article  Google Scholar 

  114. Gomez-Bombarelli, R. et al. Automatic chemical design using a data-driven continuous representation of molecules. ACS Centr. Sci. 4, 268–276 (2018).

    Article  Google Scholar 

  115. Lin, X., Quan, Z., Wang, Z.-J., Huang, H. & Zeng, X. A novel molecular representation with BiGRU neural networks for learning atom. Brief. Bioinform. 21, 2099–2111 (2019).

    Article  Google Scholar 

  116. Preuer, K., Renz, P., Unterthiner, T., Hochreiter, S. & Klambauer, G. Fréchet ChemNet distance: a metric for generative models for molecules in drug discovery. J. Chem. Inf. Model. 58, 1736–1741 (2018).

    Article  Google Scholar 

  117. Schwaller, P. et al. Molecular transformer: a model for uncertainty-calibrated chemical reaction prediction. ACS Centr. Sci. 5, 1572–1583 (2019).

    Article  Google Scholar 

  118. Schwaller, P. et al. Predicting retrosynthetic pathways using transformer-based models and a hyper-graph exploration strategy. Chemical Science 11, 3316–3325 (2020).

    Article  Google Scholar 

  119. Pesciullesi, G., Schwaller, P., Laino, T. & Reymond, J.-L. Transfer learning enables the molecular transformer to predict regio-and stereoselective reactions on carbohydrates. Nat. Commun. 11, 4874 (2020).

    Article  Google Scholar 

  120. Kreutter, D., Schwaller, P. & Reymond, J.-L. Predicting enzymatic reactions with a molecular transformer. Chem. Sci. 12, 8648–8659 (2021).

    Article  Google Scholar 

  121. Schwaller, P., Vaucher, A. C., Laino, T. & Reymond, J.-L. Prediction of chemical reaction yields using deep learning. Mach. Learn. 2, 015016 (2021).

    Google Scholar 

  122. Schwaller, P. et al. Mapping the space of chemical reactions using attention-based neural networks. Nat. Mach. Intell. 3, 144–152 (2021).

  123. Morris, P., St. Clair, R., Hahn, W. E. & Barenholtz, E. Predicting binding from screening assays with transformer network embeddings. J. Chem. Inf. Model. 60, 4191–4199 (2020).

    Article  Google Scholar 

  124. Rong, Y. et al. Self-supervised graph transformer on large-scale molecular data. In Advances in Neural Information Processing Systems Vol. 33, 12559–12571 (2020).

  125. Grechishnikova, D. Transformer neural network for protein-specific de novo drug generation as a machine translation problem. Sci. Rep. 11, 321 (2021).

    Article  Google Scholar 

  126. Baek, M. et al. Accurate prediction of protein structures and interactions using a three-track neural network. Science 373, 871–876 (2021).

    Article  Google Scholar 

  127. Méndez-Lucio, O., Baillif, B., Clevert, D.-A., Rouquié, D. & Wichard, J. De novo generation of hit-like molecules from gene expression signatures using artificial intelligence. Nat. Commun. 11, 10 (2020).

    Article  Google Scholar 

  128. Griffiths, R.-R. & Hernández-Lobato, J. M. Constrained Bayesian optimization for automatic chemical design using variational autoencoders. Chem. Sci. 11, 577–586 (2020).

    Article  Google Scholar 

  129. Hirohara, M., Saito, Y., Koda, Y., Sato, K. & Sakakibara, Y. Convolutional neural network based on SMILES representation of compounds for detecting chemical motif. BMC Bioinform. 19, 83–94 (2018).

    Article  Google Scholar 

  130. Kimber, T. B., Engelke, S., Tetko, I. V., Bruno, E. & Godin, G. Synergy effect between convolutional neural networks and the multiplicity of SMILES for improvement of molecular prediction. Preprint at https://arxiv.org/abs/1812.04439 (2018).

  131. Zheng, S., Yan, X., Yang, Y. & Xu, J. Identifying structure–property relationships through smiles syntax analysis with self-attention mechanism. J. Chem. Inf. Model. 59, 914–923 (2019).

    Article  Google Scholar 

  132. ElAbd, H. et al. Amino acid encoding for deep learning applications. BMC Bioinform. 21, 12 (2020).

    Article  Google Scholar 

  133. Satorras, V. G., Hoogeboom, E., Fuchs, F. B., Posner, I. & Welling, M. E(n) equivariant normalizing flows for molecule generation in 3d. In Advances in Neural Information Processing Systems Vol. 33 (2021).

  134. Gebauer, N. W., Gastegger, M., Hessmann, S. S., Müller, K.-R. & Schütt, K. T. Inverse design of 3d molecular structures with conditional generative neural networks. Preprint at https://arxiv.org/abs/2109.04824 (2021).

  135. Fujita, T. & Winkler, D. A. Understanding the roles of the “two QSARs”. J. Chem. Inf. Model. 56, 269–274 (2016).

    Article  Google Scholar 

  136. Hu, W. et al. Open Graph Benchmark: datasets for machine learning on graphs. In Advances in Neural Information Processing Systems Vol. 33, 22118–22133 (2020).

  137. Wu, Z. et al. MoleculeNet: a benchmark for molecular machine learning. Chem. Sci. 9, 513–530 (2018).

    Article  Google Scholar 

  138. Polykovskiy, D. et al. Molecular sets (MOSES): a benchmarking platform for molecular generation models. Front. Pharmacol. 11, 565644 (2020).

    Article  Google Scholar 

  139. Brown, N., Fiscato, M., Segler, M. H. & Vaucher, A. C. GuacaMol: benchmarking models for de novo molecular design. J. Chem. Inf. Model. 59, 1096–1108 (2019).

    Article  Google Scholar 

  140. von Lilienfeld, O. A., Müller, K.-R. & Tkatchenko, A. Exploring chemical compound space with quantum-based machine learning. Nat. Rev. Chem. 12, 945–951 (2020).

    Google Scholar 

  141. Unke, O. T. et al. Machine learning force fields. Chem. Rev. 121, 10142–10186 (2021).

    Article  Google Scholar 

  142. Isert, C., Atz, K., Jiménez-Luna, J. & Schneider, G. QMugs: quantum mechanical properties of drug-like molecules. Preprint at https://arxiv.org/abs/2107.00367 (2021).

  143. Ramakrishnan, R., Dral, P. O., Rupp, M. & Von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1, 140022 (2014).

    Article  Google Scholar 

  144. Jin, W., Coley, C., Barzilay, R. & Jaakkola, T. Predicting organic reaction outcomes with weisfeiler-lehman network. In Advances in Neural Information Processing Systems Vol. 30, 2607–2616 (2017).

  145. LeCun, Y. et al. in The Handbook of Brain Theory and Neural Networks (ed. Arbib, M. A.) 255–258 (MIT Press, 1995).

  146. Sutton, R. S. & Barto, A. G.Reinforcement Learning: An Introduction (MIT Press, 2018).

  147. Pan, S. J. & Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2009).

    Article  Google Scholar 

  148. Nguyen, L. A., He, H. & Pham-Huy, C. Chiral drugs: an overview. Int. J. Biomed. Sci. 2, 85 (2006).

    Google Scholar 

  149. Valsecchi, C., Grisoni, F., Motta, S., Bonati, L. & Ballabio, D. Nura: a curated dataset of nuclear receptor modulators. Toxicol. Appl. Pharmacol. 407, 115244 (2020).

    Article  Google Scholar 

  150. Bemis, G. W. & Murcko, M. A. The properties of known drugs. 1. Molecular frameworks. J. Med. Chem. 39, 2887–2893 (1996).

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the Swiss National Science Foundation (SNSF, grant no. 205321_182176) and the ETH RETHINK initiative.

Author information

Author notes

  1. These authors contributed equally: Kenneth Atz, Francesca Grisoni.

Authors and Affiliations

  1. Department of Chemistry and Applied Biosciences, RETHINK, ETH Zurich, Zurich, Switzerland

    Kenneth Atz, Francesca Grisoni & Gisbert Schneider

  2. Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands

    Francesca Grisoni

  3. ETH Singapore SEC Ltd, Singapore, Singapore

    Gisbert Schneider

Authors
  1. Kenneth Atz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesca Grisoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gisbert Schneider
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Francesca Grisoni or Gisbert Schneider.

Ethics declarations

Competing interests

G.S. declares a potential financial conflict of interest as co-founder of inSili.com LLC, Zurich, and in his role as scientific consultant to the pharmaceutical industry.

Peer review information

Nature Machine Intelligence thanks Jonathan Hirst and Oliver Wieder for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Atz, K., Grisoni, F. & Schneider, G. Geometric deep learning on molecular representations. Nat Mach Intell 3, 1023–1032 (2021). https://doi.org/10.1038/s42256-021-00418-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-021-00418-8

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics