Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Primer
  • Published:

Graph neural networks

Nature Reviews Methods Primers volume 4, Article number: 17 (2024) Cite this article

Subjects

Abstract

Graphs are flexible mathematical objects that can represent many entities and knowledge from different domains, including in the life sciences. Graph neural networks (GNNs) are mathematical models that can learn functions over graphs and are a leading approach for building predictive models on graph-structured data. This combination has enabled GNNs to advance the state of the art in many disciplines, from discovering new antibiotics and identifying drug-repurposing candidates to modelling physical systems and generating new molecules. This Primer provides a practical and accessible introduction to GNNs, describing their properties and applications to the life and physical sciences. Emphasis is placed on the practical implications of key theoretical limitations, new ideas to solve these challenges and important considerations when using GNNs on a new task.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Molecular property prediction example: given a molecule, a GNN predicts its ability to inhibit HIV replication.
Fig. 2: Molecule similarity and overfitting.
Fig. 3: Important data symmetries for GNNs.
Fig. 4: GNNs for knowledge graphs and molecular property prediction.
Fig. 5: Examples of GNNs for generative modelling.

Similar content being viewed by others

Code availability

Example code can be found at https://github.com/HannesStark/GNN-primer/blob/main/GNN-primer_HIV_classification.ipynb.

References

  1. Gori, M., Monfardini, G. & Scarselli, F. A new model for learning in graph domains. In Proceedings 2005 IEEE International Joint Conference Neural Networks 729–734 (IEEE, 2005).

  2. Merkwirth, C. & Lengauer, T. Automatic generation of complementary descriptors with molecular graph networks. J. Chem. Inf. Model. 45, 1159–1168 (2005).

    Article  CAS  PubMed  Google Scholar 

  3. Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M. & Monfardini, G. The graph neural network model. IEEE Trans. Neural Netw. 20, 61–80 (2008). Although the genealogy of the development is multifaced, this is often considered as the first instance of GNNs.

    Article  PubMed  Google Scholar 

  4. Bronstein, M. M., Bruna, J., Cohen, T. & Veličković, P. Geometric deep learning: grids, groups, graphs, geodesics, and gauges. Preprint at https://doi.org/10.48550/arXiv.2104.13478 (2021). Book with a very comprehensive introduction to the theoretical aspects behind GNNs and other geometric deep learning architectures.

  5. Jegelka, S. Theory of graph neural networks: representation and learning. Preprint at https://doi.org/10.48550/arXiv.2204.07697 (2022).

  6. Morgan, H. L. The generation of a unique machine description for chemical structures-a technique developed at chemical abstracts service. J. Chem. Doc. 5, 107–113 (1965).

    Article  CAS  Google Scholar 

  7. Chandak, P., Huang, K. & Zitnik, M. Building a knowledge graph to enable precision medicine. Sci. Data 10, 67 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  8. Fey, M. & Lenssen, J. E. Fast graph representation learning with PyTorch Geometric. Preprint at https://doi.org/10.48550/arXiv.1903.02428 (2019). PyTorch Geometric is the most widely used library to develop GNNs.

  9. Wang, M. et al. Deep Graph Library: a graph-centric, highly-performant package for graph neural networks. Preprint at https://doi.org/10.48550/arXiv.1909.01315 (2019).

  10. Yang, K. et al. Analyzing learned molecular representations for property prediction. J. Chem. Inf. Model. 59, 3370–3388 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Geiger, M. & Smidt, T. e3nn: Euclidean neural networks. Preprint at https://doi.org/10.48550/arXiv.2207.09453 (2022).

  12. Hu, W. et al. Open Graph Benchmark: datasets for machine learning on graphs. Adv. Neural Inf. Process. Syst. 22118–22133 (NeurIPS Proceedings, 2020). OGB is the most widely used benchmark for GNNs with a wide variety of datasets, each with its own leaderboard.

  13. Dummit, D. S. & Foote, R. M. Abstract algebra 7th edn (Wiley, 2004).

  14. Xu, K., Hu, W., Leskovec, J. & Jegelka, S. How powerful are Graph Neural Networks? In International Conference on Learning Representations (ICLR, 2019). To our knowledge, this work, concurrently with [Mor+19], was the first to propose and use the analogy of GNNs to WL isomorphism test to study their expressivity.

  15. Morris, C. et al. Weisfeiler and Leman go neural: higher-order graph neural networks. Proc. AAAI Conf. Artif. Intell. 33, 4602–4609 (2019).

    Google Scholar 

  16. Vignac, C., Loukas, A. & Frossard, P. Building powerful and equivariant graph neural networks with structural message-passing. Adv. Neural Inf. Process. Syst. 33, 14143–14155 (2020).

    Google Scholar 

  17. Abboud, R., Ceylan, I.I., Grohe, M. & Lukasiewicz, T. The surprising power of graph neural networks with random node initialization. In 30th International Joint Conferences on Artificial Intelligence 2112–2118 (International Joint Conferences on Artificial Intelligence Organization, 2021).

  18. Sato, R., Yamada, M. & Kashima, H. Random features strengthen graph neural networks. In Proceedings of the 2021 SIAM International Conference on Data Mining 333–341 (Society for Industrial and Applied Mathematics, 2021).

  19. Dwivedi, V. P. et al. Benchmarking graph neural networks. J. Mach. Learn. Res. 24, 1–48 (2023).

    MathSciNet  Google Scholar 

  20. Beaini, D. et al. Directional graph networks. In Proceedings of the 38th International Conference on Machine Learning 748–758 (PMLR, 2021).

  21. Lim, D. et al. Sign and basis invariant networks for spectral graph representation learning. In International Conference on Learning Representations (ICLR, 2023).

  22. Keriven, N. & Vaiter, S. What functions can Graph Neural Networks compute on random graphs? The role of Positional Encoding. Preprint at https://doi.org/10.48550/arXiv.2305.14814 (2023).

  23. Zhang, B., Luo, S., Wang, L. & He, D. Rethinking the expressive power of GNNs via graph biconnectivity. In International Conference on Learning Representations (ICLR, 2023).

  24. Di Giovanni, F. et al. How does over-squashing affect the power of GNNs? Preprint at https://doi.org/10.48550/arXiv.2306.03589 (2023).

  25. Razin, N., Verbin, T. & Cohen, N. On the ability of graph neural networks to model interactions between vertices. In 37th Conference on Neural Information Processing Systems (NeurIPS, 2023).

  26. Bouritsas, G., Frasca, F., Zafeiriou, S. & Bronstein, M. M. Improving graph neural network expressivity via subgraph isomorphism counting. IEEE Trans. Pattern Anal. Mach. Intell. 45, 657–668 (2023).

    Article  PubMed  Google Scholar 

  27. Sun, Z., Deng, Z.-H., Nie, J.-Y. & Tang, J. RotatE: knowledge graph embedding by relational rotation in complex space. Preprint at https://doi.org/10.48550/arXiv.1902.10197 (2019).

  28. Abboud, R., Ceylan, I., Lukasiewicz, T. & Salvatori, T. BoxE: a box embedding model for knowledge base completion. Adv. Neural Inf. Process. Syst. 33, 9649–9661 (2020).

    Google Scholar 

  29. Pavlović, A. & Sallinger, E. ExpressivE: a spatio-functional embedding for knowledge graph completion. In International Conference on Learning Representations (ICLR, 2023).

  30. Veličković, P. et al. Graph attention networks. In International Conference on Learning Representations (ICLR, 2017). Graph attention networks are the first application of the idea of attention to graphs, and they are one of the most widely used architectures to date.

  31. Corso, G., Cavalleri, L., Beaini, D., Liò, P. & Veličković, P. Principal neighbourhood aggregation for graph nets. Adv. Neural Inf. Process. Syst. 33, 13260–13271 (2020).

    Google Scholar 

  32. Gasteiger, J., Weißenberger, S. & Günnemann, S. Diffusion improves graph learning. Adv. Neural Inf. Process. Syst. 32, 13366–13378 (2019).

    Google Scholar 

  33. Gutteridge, B., Dong, X., Bronstein, M. & Di Giovanni, F. DRew: dynamically rewired message passing with delay. In International Conference on Machine Learning (eds Krause, A. et. al.) 12252–12267 (ICML, 2023).

  34. Rampášek, L. et al. Recipe for a general, powerful, scalable graph transformer. Adv. Neural Inf. Process. Syst. 35, 14501–14515 (2022).

    Google Scholar 

  35. Dwivedi, V. P. et al. Long range graph benchmark. Adv. Neural Inf. Process. Syst. 35, 22326–22340 (2022).

    Google Scholar 

  36. Dwivedi, V. P. & Bresson, X. A generalization of transformer networks to graphs. Preprint at https://doi.org/10.48550/arXiv.2012.09699 (2020).

  37. Kreuzer, D., Beaini, D., Hamilton, W., Létorneau, V. & Tossou, P. Rethinking graph transformers with spectral attention. Adv. Neural Inf. Process. Syst. 34, 21618–21629 (2021).

    Google Scholar 

  38. Bodnar, C. et al. Weisfeiler and Lehman go topological: message passing simplicial networks. In Proceedings of the 38th International Conference on Machine Learning (eds Meila, M. & Zhang, T.) 1026–1037 (PMLR, 2021).

  39. Bodnar, C. et al. Weisfeiler and Lehman go cellular: cw networks. Adv. Neural Inf. Process. Syst. 34, 2625–2640 (2021).

    Google Scholar 

  40. Chamberlain, B. et al. Grand: graph neural diffusion. In Proceedings of the 38th International Conference on Machine Learning (eds Meila, M. & Zhang, T.) 1407–1418 (PMLR, 2021).

  41. Chamberlain, B. et al. Beltrami flow and neural diffusion on graphs. Adv. Neural Inf. Process. Syst. 34, 1594–1609 (2021).

    Google Scholar 

  42. Di Giovanni, F., Rowbottom, J., Chamberlain, B. P., Markovich, T. & Bronstein, M. M. Graph neural networks as gradient flows. Preprint at https://doi.org/10.48550/arXiv.2206.10991 (2022).

  43. Rusch, T. K., Chamberlain, B., Rowbottom, J., Mishra, S. & Bronstein, M. Graph-coupled oscillator networks. In Proceedings of the 39th International Conference on Machine Learning (eds Chaudhuri, K. et al.) 18888–18909 (PMLR, 2022).

  44. Schütt, K. et al. SchNet: a continuous-filter convolutional neural network for modeling quantum interactions. In NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems (eds von Luxburg, U. et al.) 992–1002 (Curran Associates Inc., 2017). SchNet is one of the earliest and most prominent examples of SE(3)-invariant GNNs.

  45. Satorras, V. G., Hoogeboom, E. & Welling, M. E(n) equivariant graph neural networks. In Proceedings of the 38th International Conference on Machine Learning (eds Meila, M. & Zhang, T.) 9323–9332 (PMLR, 2021).

  46. Dym, N. & Maron, H. On the universality of rotation equivariant point cloud networks. In International Conference on Learning Representations (ICLR, 2021).

  47. Thomas, N. et al. Tensor field networks: rotation- and translation-equivariant neural networks for 3D point clouds. Preprint at https://doi.org/10.48550/arXiv.1802.08219 (2018).

  48. Jing, B., Eismann, S., Suriana, P., Townshend, R. J. & Dror, R. Learning from protein structure with geometric vector perceptrons. In International Conference on Learning Representations (ICLR, 2021).

  49. Gasteiger, J., Groß, J. & Günnemann, S. Directional message passing for molecular graphs. In Adv. Neural Inf. Process. Syst. (NeurIPS, 2020).

  50. Gasteiger, J., Becker, F. & Günnemann, S. GemNet: universal directional graph neural networks for molecules. Adv. Neural Inf. Process. Syst. 34, 6790–6802 (2021).

    Google Scholar 

  51. Baldassarre, F. & Azizpour, H. Explainability techniques for graph convolutional networks. Preprint at https://doi.org/10.48550/arXiv.1905.13686 (2019).

  52. Schlichtkrull, M. S., De Cao, N. & Titov, I. Interpreting graph neural networks for NLP with differentiable edge masking. In International Conference on Learning Representations (ICLR, 2021).

  53. Ying, Z., Bourgeois, D., You, J., Zitnik, M. & Leskovec, J. GNNExplainer: generating explanations for graph neural networks. Adv. Neural Inf. Process. Syst. 32, 9240–9251 (2019).

    PubMed  PubMed Central  Google Scholar 

  54. Huang, Q., Yamada, M., Tian, Y., Singh, D. & Chang, Y. GraphLIME: local interpretable model explanations for graph neural networks. IEEE Trans. Knowl. Data Eng. 35, 6968–6962 (2023).

    Google Scholar 

  55. Yuan, H., Tang, J., Hu, X. & Ji, S. XGNN: towards model-level explanations of graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 430–438 (2020).

  56. Yuan, H., Yu, H., Gui, S. & Ji, S. Explainability in graph neural networks: a taxonomic survey. IEEE Trans. Pattern Anal. Mach. Intell. 45, 5782–5799 (2022).

    Google Scholar 

  57. Kakkad, J., Jannu, J., Sharma, K., Aggarwal, C. & Medya, S. A survey on explainability of graph neural networks. Preprint at https://doi.org/10.48550/arXiv.2306.01958 (2023).

  58. Hirschfeld, L., Swanson, K., Yang, K., Barzilay, R. & Coley, C. W. Uncertainty quantification using neural networks for molecular property prediction. J. Chem. Inf. Model. 60, 3770–3780 (2020).

    Article  CAS  PubMed  Google Scholar 

  59. Hsu, H. H.-H., Shen, Y., Tomani, C. & Cremers, D. What makes graph neural networks miscalibrated? In Adv. Neural Inf. Process. Syst. (NeurIPS, 2022).

  60. Stadler, M., Charpentier, B., Geisler, S., Zügner, D. & Günnemann, S. Graph posterior network: Bayesian predictive uncertainty for node classification. Adv. Neural Inf. Process. Syst. 34, 18033–18048 (2021).

    Google Scholar 

  61. Wang, X., Liu, H., Shi, C. & Yang, C. Be confident! towards trustworthy graph neural networks via confidence calibration. Adv. Neural Inf. Process. Syst. 34, 23768–23779 (2021).

    Google Scholar 

  62. Huang, K., Jin, Y., Candes, E. & Leskovec, J. Uncertainty quantification over graph with conformalized graph neural networks. Preprint at https://doi.org/10.48550/arXiv.2305.14535 (2023).

  63. Batzner, S. et al. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun. 13, 2453 (2022).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  64. Schlichtkrull, M. S. et al. Modeling relational data with graph convolutional networks. In The Semantic Web. ESWC 2018. Lecture Notes in Computer Science (eds Gangemi, A. et al.) 593–607 (Springer, Cham, 2018).

  65. Sun, Q. et al. SUGAR: subgraph neural network with reinforcement pooling and self-supervised mutual information mechanism. In WWW ’21: Proceedings of the Web Conference 2021 (eds Leskovec, J. et al.) 2081–2091 (Association for Computing Machinery, 2021).

  66. Sharma, K. et al. A survey of graph neural networks for social recommender systems. Preprint at https://doi.org/10.48550/arXiv.2212.04481 (2022).

  67. Stokes, J. M. et al. A deep learning approach to antibiotic discovery. Cell 180, 688–702.e13 (2020). Discovery of a novel antibiotic, halicin, via GNNs, one of the most prominent examples of the application of GNNs to scientific discovery.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  68. Feinberg, E. N., Joshi, E., Pande, V. S. & Cheng, A. C. Improvement in ADMET prediction with multitask deep featurization. J. Med. Chem. 63, 8835–8848 (2020).

    Article  CAS  PubMed  Google Scholar 

  69. Peng, Y. et al. Enhanced graph isomorphism network for molecular ADMET properties prediction. IEEE Access 8, 168344–168360 (2020).

    Article  Google Scholar 

  70. Murphy, M. et al. Efficiently predicting high resolution mass spectra with graph neural networks. In Proceedings of the 40th International Conference on Machine Learning (eds Krause, A. et al.) 25549–25562 (PMLR, 2023).

  71. Bevilacqua, B. et al. Equivariant subgraph aggregation networks. In International Conference on Learning Representations (ICLR, 2022).

  72. Guo, M. et al. Hierarchical grammar-induced geometry for data-efficient molecular property prediction. In Proceedings of the 40th International Conference on Machine Learning (eds Krause, A. et al.) 12055–12076 (PMLR, 2023).

  73. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning (eds Precup, D. & Teh, Y. W.) 1263–1272 (PMLR, 2017). To our knowledge, this paper is the first to formalize the idea of message passing as presented in this Primer and proposes applications of GNNs to quantum chemistry, which remains one of the scientific fields in which GNNs have seen most applications.

  74. Axelrod, S. & Gómez-Bombarelli, R. GEOM, energy-annotated molecular conformations for property prediction and molecular generation. Sci. Data 9, 185 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  75. Hermann, J., Schätzle, Z. & Noé, F. Deep-neural-network solution of the electronic Schrödinger equation. Nat. Chem. 12, 891–897 (2020).

    Article  CAS  PubMed  Google Scholar 

  76. Gao, N. & Günnemann, S. Generalizing neural wave functions. In International Conference on Machine Learning 10708–10726 (ICML, 2023).

  77. Kingma, D. P. & Welling, M. Auto-encoding variational bayes. In International Conferece on Learning Representations (ICLR, 2014).

  78. Goodfellow, I. et al. Generative adversarial networks. Commun. ACM 63, 139–144 (2020).

    Article  Google Scholar 

  79. Mitton, J., Senn, H. M., Wynne, K. & Murray-Smith, R. A graph VAE and graph transformer approach to generating molecular graphs. Preprint at https://doi.org/10.48550/arXiv.2104.04345 (2021).

  80. Jin, W., Barzilay, R. & Jaakkola, T. Junction tree variational autoencoder for molecular graph generation. In Proceedings of the 35th International Conference on Machine Learning (eds Dy, J. & Krause, A.) 2323–2332 (PMLR, 2018).

  81. Jin, W., Barzilay, R. & Jaakkola, T. Hierarchical generation of molecular graphs using structural motifs. In Proceedings of the 37th International Conference on Machine Learning (eds Daumé, H. & Singh, A.) 4839–4848 (PMLR, 2020).

  82. Vignac, C. & Frossard, P. Top-N: equivariant set and graph generation without exchangeability. In International Conference on Learning Representations (ICLR, 2022).

  83. Jo, J., Lee, S. & Hwang, S. J. Score-based generative modeling of graphs via the system of stochastic differential equations. In Proceedings of the 39th International Conference on Machine Learning (eds Chaudhuri, K. et al.) 10362–10383 (PMLR, 2022).

  84. Vignac, C. et al. DiGress: discrete denoising diffusion for graph generation. In International Conference on Learning Representations (ICLR, 2023).

  85. Dauparas, J. et al. Robust deep learning–based protein sequence design using ProteinMPNN. Science 378, 49–56 (2022).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  86. Moon, S., Zhung, W., Yang, S., Lim, J. & Kim, W. Y. PIGNet: a physicsinformed deep learning model toward generalized drug–target interaction predictions. Chem. Sci. 13, 3661–3673 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  87. Xu, M. et al. GeoDiff: a geometric diffusion model for molecular conformation generation. In International Conference on Learning Representations (ICLR, 2022).

  88. Jing, B., Corso, G., Chang, J., Barzilay, R. & Jaakkola, T. S. Torsional diffusion for molecular conformer generation. In Adv. Neural Inf. Process. Syst. (eds Sanmi, K. et al.) (NeurIPS, 2022).

  89. Ingraham, J., Riesselman, A., Sander, C. & Marks, D. Learning protein structure with a differentiable simulator. In International Conference on Learning Representations (ICLR, 2019).

  90. Jing, B. et al. EigenFold: generative protein structure prediction with diffusion models. Preprint at https://doi.org/10.48550/arXiv.2304.02198 (2023).

  91. Corso, G., Stärk, H., Jing, B., Barzilay, R. & Jaakkola, T. S. DiffDock: diffusion steps, twists, and turns for molecular docking. In International Conference on Learning Representations (ICLR, 2023).

  92. Ingraham, J. et al. Illuminating protein space with a programmable generative model. Nature 623, 1070–1078 (2023).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  93. Watson, J. L. et al. De novo design of protein structure and function with RFdiffusion. Nature 620, 1089–1100 (2023).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  94. Fu, X., Xie, T., Rebello, N. J., Olsen, B. D. & Jaakkola, T. Simulate time-integrated coarse-grained molecular dynamics with geometric machine learning. Preprint at https://doi.org/10.48550/arXiv.2204.10348 (2022).

  95. Wang, W. et al. Generative coarse-graining of molecular conformations. In International Conference on Machine Learning 23213–23236 (ICML, 2022).

  96. Yang, S. & Gomez-Bombarelli, R. Chemically transferable generative backmapping of coarse-grained proteins. In Proceedings of the 40th International Conference on Machine Learning (eds Krause, A. et al.) 39277–39298 (PMLR, 2023).

  97. Huang, K. et al. Therapeutics data commons: machine learning datasets and tasks for drug discovery and development. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021 (NeurIPS, 2021).

  98. Bajpai, A. K. et al. Systematic comparison of the protein-protein interaction databases from a user’s perspective. J. Biomed. Inform. 103, 103380 (2020).

    Article  PubMed  Google Scholar 

  99. Tripp, A., Bacallado, S., Singh, S. & Hernández-Lobato, J. M. Tanimoto random features for scalable molecular machine learning. In Adv. Neural Inf. Process. Syst. (NeurIPS, 2023).

  100. Stärk, H. et al. 3D Infomax improves GNNs for molecular property prediction. In Proceedings of the 39th International Conference on Machine Learning (eds Chaudhuri, K. et al.) 20479–20502 (PMLR, 2022).

  101. Thakoor, S. et al. Large-scale representation learning on graphs via bootstrapping. In International Conference on Learning Representations (ICLR, 2022).

  102. Devlin, J., Chang, M., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (eds Burstein, J. et al.) 4171–4186 (Association for Computational Linguistics, 2019).

  103. Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020).

    Google Scholar 

  104. Lin, Z. et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 379, 1123–1130 (2023).

    Article  ADS  MathSciNet  CAS  PubMed  Google Scholar 

  105. Dosovitskiy, A. et al. An image is worth 16x16 words: transformers for image recognition at scale. In International Conference on Learning Representations (ICLR, 2021).

  106. Misra, I. & van der Maaten, L. Self-supervised learning of pretext-invariant representations. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 6707–6717 (IEEE, 2020).

  107. He, K., Fan, H., Wu, Y., Xie, S. & Girshick, R. Momentum Contrast for unsupervised visual representation learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 9726–9735 (IEEE, 2020).

  108. Liu, Y. et al. Graph self-supervised learning: a survey. IEEE Trans. Knowl. Data Eng. 35, 5879–5900 (2023).

    Google Scholar 

  109. Rusch, T. K., Bronstein, M. M. & Mishra, S. A survey on oversmoothing in graph neural networks. Preprint at https://doi.org/10.48550/arXiv.2303.10993 (2023).

  110. Xu, K. et al. Representation learning on graphs with jumping knowledge networks. In Proceedings of the 35th International Conference on Machine Learning (eds Dy, J. & Krause, A.) 5453–5462 (PMLR, 2018).

  111. Di Giovanni, F., Rowbottom, J., Chamberlain, B. P., Markovich, T. & Bronstein, M. M. Understanding convolution on graphs via energies. In Transact. Mach. Learn. Res. 2835–8856 (2023).

  112. Rusch, T. K., Chamberlain, B. P., Mahoney, M. W., Bronstein, M. M. & Mishra, S. Gradient gating for deep multi-rate learning on graphs. In International Conference on Learning Representations (ICLR, 2023).

  113. Alon, U. & Yahav, E. On the bottleneck of graph neural networks and its practical implications. In International Conference on Learning Representations (ICLR, 2021).

  114. Topping, J., Di Giovanni, F., Chamberlain, B. P., Dong, X. & Bronstein, M. M. Understanding over-squashing and bottlenecks on graphs via curvature. In International Conference on Learning Representations (ICLR, 2022).

  115. Dimitrov, R., Zhao, Z., Abboud, R. & Ceylan, I. I. PlanE: representation learning over planar graphs. Preprint at https://doi.org/10.48550/arXiv.2307.01180 (2023).

  116. Hosseinzadeh, M. M., Cannataro, M., Guzzi, P. H. & Dondi, R. Temporal networks in biology and medicine: a survey on models, algorithms, and tools. Netw. Model. Anal. Health Inform. Bioinform. 12, 10 (2023).

    Article  PubMed  Google Scholar 

  117. Kipf, T. N. & Welling, M. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR, 2017). Graph convolutional network was the architecture that set off the recent years of development of GNNs.

Download references

Acknowledgements

The authors thank R. Wu, S. Yang, D. Lim, A. Corso and M.-M. Troadec for their help in reviewing the manuscript before submission. The authors also thank B. Jing, F. Di Giovanni, J. Yim, C. Vignac and F. Faltings for useful discussions. This work was supported by the NSF Expeditions grant (award 1918839), the Machine Learning for Pharmaceutical Discovery and Synthesis (MLPDS) consortium, the DTRA Discovery of Medical Countermeasures Against New and Emerging (DOMANE) threats program, the DARPA Accelerated Molecular Discovery program, the NSF AI Institute CCF-2112665 and the NSF Award 2134795.

Author information

Author notes

  1. These authors contributed equally: Gabriele Corso, Hannes Stark.

Authors and Affiliations

  1. CSAIL, MIT, Cambridge, MA, US

    Gabriele Corso, Hannes Stark, Stefanie Jegelka, Tommi Jaakkola & Regina Barzilay

  2. School of CIT, TU Munich, Munich, Germany

    Stefanie Jegelka

Authors
  1. Gabriele Corso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hannes Stark
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefanie Jegelka
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tommi Jaakkola
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Regina Barzilay
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Introduction (R.B., G.C., H.S., S.J. and T.J.); Experimentation (R.B., G.C., H.S., S.J. and T.J.); Results (R.B., G.C., H.S., S.J. and T.J.); Applications (R.B., G.C., H.S., S.J. and T.J.); Reproducibility and data deposition (R.B., G.C., H.S. and S.J.); Limitations and optimizations (R.B., G.C., H.S., S.J. and T.J.); Outlook (R.B., G.C., H.S., S.J. and T.J.); overview of the Primer (all authors).

Corresponding authors

Correspondence to Gabriele Corso, Hannes Stark or Regina Barzilay.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Reviews Methods Primers thanks Jiliang Tang; Siddhartha Mishra, who co-reviewed with Konstantin Rusch; and Rex Ying, who co-reviewed with Tinglin Huang, for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Related links

ChEMBL: https://www.ebi.ac.uk/chembl/

Chemprop: https://github.com/chemprop/chemprop

Deep Graph Library: https://www.dgl.ai/

e3nn: https://e3nn.org/

PDBBind: http://www.pdbbind.org.cn/

Protein Data Bank: https://www.rcsb.org/

PyTorch Geometric: https://pytorch-geometric.readthedocs.io/en/latest/

Glossary

Big O notation

Notation used in complexity theory to indicate how the worst-case runtime of an algorithm increases as the size of the input increases.

Composition pattern

A simple example composition pattern is if molecule A binds to protein B and protein B is involved in the mechanism of disease C, then A is a potential candidate for C.

Deep learning

Subset of machine learning that uses artificial neural network models with multiple layers learning to automatically extract features and complex patterns from data.

Embeddings

Arrays of numbers produced by a deep learning model abstractly capture a model’s understanding of an object.

Features

Information about the object under analysis that is passed as inputs to the model.

Knowledge graph completion

Task in which missing information in a knowledge graph is predicted based on existing relationships and patterns within the graph.

Message-passing layer

Fundamental component of graph neural networks that iteratively aggregates and updates the features from neighbouring nodes, enabling the propagation of information throughout the graph structure.

Planar graphs

A planar graph is one that can be drawn on a 2D page without edges crossing each other.

ReLU

The rectified linear unit (ReLU) is the most common type of non-linear function used in neural networks and has the simple form ReLU(x) = max(0,x).

Representations

Arrays of numbers that capture attributes of an object.

ROC-AUC

(Area under the curve of the receiver operator characteristic). A measure of the precision of a binary classifier that is informative in settings with unbalanced classes.

Scaffold

Core substructures within molecular graphs shared by multiple compounds that often have similar properties.

Transductive task

Setting that involves making predictions at inference time on a partially labelled graph, for a subset of the nodes within the graph. Models trained in a transductive setting do not generalize to other graphs.

Uncertainty

Uncertainty refers to the lack of confidence or precision in a model’s prediction. Taking this ambiguity into account is often important in real-world applications of machine learning models.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Corso, G., Stark, H., Jegelka, S. et al. Graph neural networks. Nat Rev Methods Primers 4, 17 (2024). https://doi.org/10.1038/s43586-024-00294-7

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s43586-024-00294-7

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics