Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Comparing 2D and 3D representations for face-based genetic syndrome diagnosis

European Journal of Human Genetics volume 31, pages 1010–1016 (2023)Cite this article

Subjects

Abstract

Human genetic syndromes are often challenging to diagnose clinically. Facial phenotype is a key diagnostic indicator for hundreds of genetic syndromes and computer-assisted facial phenotyping is a promising approach to assist diagnosis. Most previous approaches to automated face-based syndrome diagnosis have analyzed different datasets of either 2D images or surface mesh-based 3D facial representations, making direct comparisons of performance challenging. In this work, we developed a set of subject-matched 2D and 3D facial representations, which we then analyzed with the aim of comparing the performance of 2D and 3D image-based approaches to computer-assisted syndrome diagnosis. This work represents the most comprehensive subject-matched analyses to date on this topic. In our analyses of 1907 subject faces representing 43 different genetic syndromes, 3D surface-based syndrome classification models significantly outperformed 2D image-based models trained and evaluated on the same subject faces. These results suggest that the clinical adoption of 3D facial scanning technology and continued collection of syndromic 3D facial scan data may substantially improve face-based syndrome diagnosis.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Frontal renderings of the same example subject surface scan captured using different distances between the camera and the subject face.
Fig. 2: Subject demographic histogram.
Fig. 3

Similar content being viewed by others

Data availability

The 3D facial scan data used in this work is available through application to the Face Base consortium (www.facebase.org).

Notes

  1. www.3dmd.com

  2. www.facebase.org

References

  1. Thevenot J, Lopez M, Hadid A. A survey on computer vision for assistive medical diagnosis from faces. IEEE J Biomed Health Inform. 2018:22:1497–511.

  2. Hennekam R, Allanson J, Krantz I. Gorlin’s Syndromes of the Head and Neck, Oxford University Press, 2010.

  3. Hart T, Hart P. Genetic studies of craniofacial anomalies: clinical implications and applications. Orthod Craniofacial Res. 2009;12:212–20.

    Article  CAS  Google Scholar 

  4. Matthews H, de Jong G, Maal T, Claes P. Static and motion facial analysis for craniofacial assessment and diagnosing diseases. Annu Rev Biomed Data Sci. 2022;5:19–42.

    Article  PubMed  Google Scholar 

  5. MacEachern SJ, Forkert ND. Machine learning for precision medicine. Genome. 2021;64:416–25.

    Article  PubMed  Google Scholar 

  6. Alsafar MCER, Werghi N, Al-Muhairi H, Alsafar H. Using facial images for the diagnosis of genetic syndromes: a survey. in International Conference on Communications, Signal Processing, and their Applications, 2015.

  7. Gurovich Y, Hanani Y, Bar O, Nadav G, Fleischer N, Gelbman D, et al. Identifying facial phenotypes of genetic disorders using deep learning. Nat Med. 2019;25:60–4.

    Article  CAS  PubMed  Google Scholar 

  8. Hallgrímsson B, Aponte JD, Katz DC, Bannister JJ, Riccardi SL, Mahasuwan N, et al. Automated syndrome diagnosis by three-dimensional facial imaging. Genet Med. 2020;22:1–12.

    Article  Google Scholar 

  9. Bannister J, Wilms M, Aponte D, Katz D, Klein OD, Bernier FP, et al. A deep invertible 3D facial shape model for interpretable genetic syndrome diagnosis. IEEE J Biomed Health Inform. 2022;26:3229–39.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Wilamowska K, Shapiro L, Heike C. Classification of 3D face shape in 22q11.2 deletion syndrome. in IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2009.

  11. Anas I, Bamgbose B, Nuhu S. A comparison between 2D and 3D methods of quantifying facial morphology. Heliyon. 2019;5:e01880.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Zogheib T, Jacobs R, Bornstein M, Agbaje J, Anumendem D, Klazen Y, et al. Comparison of 3D scanning versus 2D photography for the identification of facial soft-tissue landmarks. Open Dent J. 2018;12:61–71.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Bilge AS, Sankur B, Taha M. Comparative evaluation of 3D vs. 2D modality for automatic detection of facial action units. Pattern Recognit. 2012;45:767–82.

    Article  Google Scholar 

  14. Chang K, Bowyer K, Flynn P, Chen X. Multi-biometrics using facial appearance, shape and temperature. in Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004.

  15. Zhou S, Xiao S. 3D face recognition: a survey. Hum-centric Comput Inform Sci. 2018;8. https://hcis-journal.springeropen.com/articles/10.1186/s13673-018-0157-2.

  16. “Face ID Security,” 2017. [Online]. Available: www.apple.com/business-docs/FaceID_Security_Guide.pdf.

  17. Hsieh T-C, Aviram B-H, Moosa S, Ehmke N, Gripp KW, Pantel JT, et al. GestaltMatcher facilitates rare disease matching using facial phenotype descriptors. Nat Genet. 2022;54:349–57.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Bannister JJ, Crites SR, Aponte JD, Katz DC, Wilms M, Klein OD, et al. Fully automatic landmarking of syndromic 3D facial surface scans using 2D images. Sensors. 2020;20:3171.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Amberg B, Romdhani S, Vetter T. Optimal Step Nonrigid ICP Algorithms for Surface Registration. in IEEE Conference on Computer Vision and Pattern Recognition, 2007.

  20. Serengil SI, Ozpinar A. “LightFace: a hybrid deep face recognition framework,” in 2020 Innovations in Intelligent Systems and Applications Conference (ASYU), 2020.

  21. Deng J, Guo J, Xue N, Zafeiriou S. ArcFace: Additive Angular Margin Loss for Deep Face Recognition. in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

  22. Farkas LG, Katic MJ, Forrest CR. International anthropometric study of facial morphology in various ethnic groups/races. J Craniofacial Surg. 2005;16:615–46.

    Article  Google Scholar 

  23. Berretti S, Del Bimbo A, Pala P. Distinguishing facial features for ethnicity-based 3D face recognition. ACM Trans Intell Syst Technol (TIST). 2012;3:1–20.

    Article  Google Scholar 

Download references

Funding

This research was funded by the National Institutes of Health (U01-DE024440), the Canada Research Chairs program, as well as by the River Fund at Calgary Foundation.

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, University of Calgary, Calgary, AB, Canada

    Jordan J. Bannister

  2. Department of Pediatrics, Department of Community Health Sciences, Alberta Children’s Hospital Research Institute, University of Calgary, Calgary, AB, Canada

    Matthias Wilms

  3. Department of Cell Biology and Anatomy, University of Calgary, Calgary, AB, Canada

    J. David Aponte, David C. Katz & Benedikt Hallgrímsson

  4. Program in Craniofacial Biology, Department of Orofacial Sciences, University of California, San Francisco, CA, USA

    Ophir D. Klein

  5. Department of Medical Genetics and the Alberta Children’s Hospital Research Institute, University of Calgary, Calgary, AB, Canada

    Francois P. Bernier

  6. Department of Pediatrics, University of Colorado School of Medicine, Aurora, CO, USA

    Richard A. Spritz

  7. Department of Radiology, Alberta Children’s Hospital Research Institute, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada

    Nils D. Forkert

Authors
  1. Jordan J. Bannister
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthias Wilms
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. David Aponte
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David C. Katz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ophir D. Klein
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Francois P. Bernier
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Richard A. Spritz
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Benedikt Hallgrímsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Nils D. Forkert
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

(1) Research Project: A. Conception, B. Organization, C. Execution; (2) Statistical Analysis: A. Design, B. Execution, C. Review and Critique; (3) Manuscript: A. Writing of the First Draft, B. Review and Critique. JJB: 1A, 1B, 1C, 2A, 2B, 3A. DA: 1B, 1C, 2C, 3B. DK: 1B, 1C, 2C, 3B. ODK: 1A, 1B, 3B. FPJB: 1A, 1B, 3B. RAS: 1A, 1B, 3B. BH: 1A, 1B, 2A, 2C, 3B. NDF: 1A, 1B, 2A, 2C, 3B.

Corresponding author

Correspondence to Jordan J. Bannister.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethics approval

This study was conducted in accordance with the Declaration of Helsinki. Ethics approval was granted by the Conjoint Health Research Ethics Board (CHREB# REB14-0340_REN4) at the University of Calgary. Informed consent informed was obtained from all subjects. Consent for the publication of facial images was obtained for the example subject shown in Figs. 1 and 3.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bannister, J.J., Wilms, M., Aponte, J.D. et al. Comparing 2D and 3D representations for face-based genetic syndrome diagnosis. Eur J Hum Genet 31, 1010–1016 (2023). https://doi.org/10.1038/s41431-023-01308-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41431-023-01308-w

This article is cited by

Search

Advanced search

Quick links