Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Interpretable deep-learning models to help achieve the Sustainable Development Goals

Nature Machine Intelligence volume 3, page 926 (2021)Cite this article

Subjects

To the Editor — Artificial intelligence (AI) algorithms have the potential to support the Sustainable Development Goals (SDGs) of the United Nations (UN)1. For instance, Jean et al.2 propose a pioneering method to identify and track regions of poverty using satellite images via convolutional neural networks (CNNs), a study that has important implications for SDG 1 (on ending poverty in all its forms everywhere). However, one of the main limitations of current AI technology is the lack of interpretability of these models, which gives rise to concerns about trust and ethical use. In particular, decisions made by black-box AI models may not be accepted by governments or the general public, an issue that is being addressed by the ethical guidelines for ‘trustworthy AI’ by the European Commission3. We argue that there is also an excessive trust in the models, which results in decisions that cannot be explained or confirmed. An undesirable result is that officials are removed from the accountability they need to have. One well-known example of a problematic lack of interpretability is the medical application Stream developed by DeepMind, which was aimed at kidney-disease prediction4. As the deep-learning version of Stream was unable to provide interpretability of the decision-making process, it was not approved to be used in practice. Later on, the application got approval when a simple decision-tree model (which did not provide the same high accuracy as the deep-learning model) was used instead, as it provided high interpretability of the decision-making process.

Recent advances in the context of interpretable AI5 have brought mathematical techniques to provide interpretability properties for the training and test data sets, model parameters, output and so on. Bringing transparency to black-box AI systems with such methods is very important in the context of ethical AI applications. We emphasize the difference between methods that provide mere explanations regarding AI results, and methods that yield complete interpretability of those results, as discussed by Rudin6. The ideal scenario is when a deep-learning model is interpretable from its inception7. However, in many applications, particularly dealing with deep learning, the models are already trained on extensive databases, and it is costly (or impossible) to reformulate them in an interpretable framework, while maintaining high accuracy. We argue that interpretability needs to be a requirement, not an option, and consequently the models that do not adhere to this principle are unacceptable. We advocate the use of a framework, such as that developed by Cranmer et al.8, to add interpretability to already-trained deep-learning models. This method relies on a deep-learning model with a separable internal structure and an inductive bias relevant to the nature of the data. Then, symbolic expressions are fit to the functions composing the model by means of a genetic algorithm that stochastically combines algebraic formulas. The power of this method lies in the possibility of analytical expressions to perform the same predictions initially carried out by the black-box deep-learning model. It is important to clarify that the resemblance of the predictions between the new and the original models will of course depend on the quality of the fit, but the work by Cranmer et al.8 is very encouraging in the sense that their symbolic-based model exhibits better generalization capabilities than the original black-box model. An important aspect of this process is to promote sparse models through regularization terms in the loss, which effectively favors the principle of Occam’s Razor.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Vinuesa, R. et al. Nat. Commun. 11, 233 (2020).

    Article  Google Scholar 

  2. Jean, N. et al. Science 353, 790–794 (2016).

    Article  Google Scholar 

  3. European Commission. https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai (2019).

  4. King, D. https://deepmind.com/blog/article/streams-and-ai (2017).

  5. Fan, F., Xiong, J., Li, M. & Wang, G. IEEE Trans. Radiat. Plasma Med. Sci. (2021).

  6. Rudin, C. Nat. Mach. Intell. 1, 206–215 (2019).

    Article  Google Scholar 

  7. Jiang, C. et al. Phys. Fluids 33, 055133 (2021).

    Article  Google Scholar 

  8. Cranmer, M. et al. NeurIPS Proceedings 33, 17429–17442 (2020).

    Google Scholar 

Download references

Acknowledgements

The authors acknowledge V. Dignum for her insightful comments on this manuscript. R.V. acknowledges the financial support from the Swedish Research Council (VR).

Author information

Authors and Affiliations

  1. FLOW, Engineering Mechanics, KTH Royal Institute of Technology, Stockholm, Sweden

    Ricardo Vinuesa

  2. AI Sustainability Center, Stockholm, Sweden

    Ricardo Vinuesa

  3. Smart Cities, School of Creative Technology, Saxion University of Applied Sciences, Enschede, The Netherlands

    Beril Sirmacek

Authors
  1. Ricardo Vinuesa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Beril Sirmacek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ricardo Vinuesa.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vinuesa, R., Sirmacek, B. Interpretable deep-learning models to help achieve the Sustainable Development Goals. Nat Mach Intell 3, 926 (2021). https://doi.org/10.1038/s42256-021-00414-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-021-00414-y

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing