Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Accelerating evidence-informed decision-making for the Sustainable Development Goals using machine learning

Nature Machine Intelligence volume 2, pages 559–565 (2020)Cite this article

Subjects

Abstract

The United Nations Sustainable Development Goal 2 (SDG 2) is to achieve zero hunger by 2030. We have designed Persephone, a machine learning model, to support a diverse volunteer network of 77 researchers from 23 countries engaged in creating interdisciplinary evidence syntheses in support of SDG 2. Such evidence syntheses, whatever the specific topic, assess original studies to determine the effectiveness of interventions. By gathering and summarizing current evidence and providing objective recommendations they can be valuable aids to decision-makers. However, they are time-consuming; estimates range from 18 months to three years to produce a single review. Persephone analysed 500,000 unstructured text summaries from prominent sources of agricultural research, determining with 90% accuracy the subset of studies that would eventually be selected by expert researchers. We demonstrate that machine learning models can be invaluable in placing evidence into the hands of policymakers.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Evidence synthesis and machine learning analytics.
Fig. 2: Agricultural interventions ontology.
Fig. 3: Persephone.
Fig. 4: MLMs reduce screening time for researchers.
Fig. 5: Comparison of human versus machine-selected studies.

References

  1. Ioannidis, J. P. A. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses: mass production of systematic reviews and meta-analyses. Milbank Q. 94, 485–514 (2016).

    Article  Google Scholar 

  2. Masaki, T., Custer, S., Eskenazi, A., Stern, A. & Latourell, R. Decoding Data Use: How Do Leaders Use Data and Use it to Accelerate Development (AidData, 2017).

  3. Cairney, P. & Oliver, K. How should academics engage in policymaking to achieve impact? Polit. Stud. Rev. 18, 228–244 (2020).

    Article  Google Scholar 

  4. Bornmann, L. & Mutz, R. Growth rates of modern science: a bibliometric analysis based on the number of publications and cited references. J. Assoc. Inf. Sci. Technol. 66, 2215–2222 (2015).

    Article  Google Scholar 

  5. Head, B. W. Reconsidering evidence-based policy: key issues and challenges. Policy Soc. 29, 77–94 (2010).

    Article  Google Scholar 

  6. Littell, J. H. Conceptual and practical classification of research reviews and other evidence synthesis products. Campbell Syst. Rev. 14, 1–21 (2018).

    Article  Google Scholar 

  7. Gurevitch, J., Koricheva, J., Nakagawa, S. & Stewart, G. Meta-analysis and the science of research synthesis. Nature 555, 175–182 (2018).

    Article  Google Scholar 

  8. Parker, T. H. et al. Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31, 711–719 (2016).

    Article  Google Scholar 

  9. Haddaway, N. R. & Westgate, M. J. Predicting the time needed for environmental systematic reviews and systematic maps. Conserv. Biol. 33, 434–443 (2019).

    Article  Google Scholar 

  10. Chalmers, I. et al. How to increase value and reduce waste when research priorities are set. Lancet 383, 156–165 (2014).

    Article  Google Scholar 

  11. Lau, J. Editorial: systematic review automation thematic series. Syst. Rev. 8, 70 (2019).

    Article  Google Scholar 

  12. Çano, E. & Morisio, M. Hybrid recommender systems: a systematic literature review. Intell. Data Anal. 21, 1487–1524 (2017).

    Article  Google Scholar 

  13. Howard, B. E. et al. SWIFT-Review: a text-mining workbench for systematic review. Syst. Rev. 5, 87 (2016).

    Article  Google Scholar 

  14. Marshall, I. J. & Wallace, B. C. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst. Rev. 8, 163 (2019).

    Article  Google Scholar 

  15. Espey, J. Using evidence & data to drive action on the SDGs. SDSN http://unsdsn.org/news/2018/06/28/using-evidence-data-to-drive-action-on-the-sdgs/ (2018).

  16. Moher, D., Liberati, A., Tetzlaff, J. & Altman, D. G. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 339, b2535 (2009).

    Google Scholar 

  17. Caliskan, A., Bryson, J. J. & Narayanan, A. Semantics derived automatically from language corpora contain human-like biases. Science 356, 183–186 (2017).

    Article  Google Scholar 

  18. Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C. & Brigham, T. J. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J. Clin. Epidemiol. 68, 617–626 (2015).

    Article  Google Scholar 

  19. Fagan, J. C. An evidence-based review of academic web search engines, 2014–2016: implications for librarians’ practice and research agenda. Inf. Technol. Libr. 36, 7–47 (2017).

    Google Scholar 

  20. Davidson, B. Storytelling and evidence-based policy: lessons from the grey literature. Palgrave Commun. 3, 17093 (2017).

    Article  Google Scholar 

  21. McAuley, L., Pham, B., Tugwell, P. & Moher, D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet 356, 1228–1231 (2000).

    Article  Google Scholar 

  22. Mikolov, T., Chen, K., Corrado, G. & Dean, J. Efficient estimation of word representations in vector space. https://arxiv.org/abs/1301.3781 (2013).

  23. Yano, T. & Kang, M. Taking advantage of Wikipedia in natural language processing. http://www.cs.cmu.edu/~taey/pub/wiki.pdf (2008).

  24. Sharma, Y., Agrawal, G., Jain, P. & Kumar, T. Vector representation of words for sentiment analysis using GloVe. In 2017 Int. Conf. Intelligent Communication and Computational Techniques 279–284 (ICCT, 2017).

  25. Hearst, M. A. Automatic acquisition of hyponyms from large text corpora. In Proc. 14th Conf. Computational Linguistics Vol. 2 539–545 (Association for Computational Linguistics, 1992).

  26. Pavlidis, P., Wapinski, I. & Noble, W. S. Support vector machine classification on the web. Bioinformatics 20, 586–587 (2004).

    Article  Google Scholar 

  27. Veena, G., Gupta, D., Daniel, A. N. & Roshny, S. A learning method for coreference resolution using semantic role labeling features. In 2017 Int. Conf. Advances in Computing, Communications and Informatics 67–72 (ICACCI, 2017).

  28. Paulavets, M. E., Porciello, J., Kiryllau, Y. I. & Einarson, S. A taxonomy creation for agriculture using classical machine learning algorithms. Big Data Adv. Anal. 5, 45–50 (2019).

    Google Scholar 

  29. Lewis, D. P., Jebara, T. & Noble, W. S. Support vector machine learning from heterogeneous data: an empirical analysis using protein sequence and structure. Bioinformatics 22, 2753–2760 (2006).

    Article  Google Scholar 

  30. Bojanowski, P., Grave, E., Joulin, A. & Mikolov, T. Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017).

    Article  Google Scholar 

  31. Joshi, M., Agarwal, R. C. & Kumar, V. Predicting rare classes: can boosting make any weak learner strong? In Proc. Eighth ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining 297–306 (ACM, 2002).

  32. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. https://arxiv.org/abs/1810.04805 (2018).

  33. Willett, J., Baldwin, T., Martinez, D. & Webb, A. Classification of study region in environmental science abstracts. In Proc. Australasian Language Technology Association Workshop 118–122 (ALTA, 2012).

  34. Kaushik, N. & Chatterjee, N. Automatic relationship extraction from agricultural text for ontology construction. Inf. Process. Agric. 5, 60–73 (2018).

    Google Scholar 

  35. Beltagy, I., Lo, K. & Cohan, A. SciBERT: a pretrained language model for scientific text. https://arxiv.org/abs/1903.10676 (2019).

  36. Acevedo, M. et al. A scoping review of adoption of climate-resilient crops by small-scale producers in low- and middle-income countries. Nat. Plants https://doi.org/10.1038/s41477-020-00783-z (2020).

  37. Baltenwick, I. et al. A scoping review of feed interventions and livelihoods of small-scale livestock keepers. Nat. Plants https://doi.org/10.1038/s41477-020-00786-w (2020).

  38. Stathers, T. et al. A scoping review of interventions for crop postharvest loss reduction in sub-Saharan Africa and South Asia. Nat. Sustain. https://doi.org/10.1038/s41893-020-00622-1 (2020).

  39. Liverpool-Tasie, L. S. O. et al. A scoping review of market links between value chain actors and small-scale producers in developing regions. Nat. Sustain. https://doi.org/10.1038/s41893-020-00621-2 (2020).

  40. Ricciardi, V. et al. A scoping review of research funding for small-scale farmers in water scarce regions. Nat. Sustain. https://doi.org/10.1038/s41893-020-00623-0 (2020).

  41. Piñeiro, V. et al. A scoping review on incentives for adoption of sustainable agricultural practices and their outcomes. Nat. Sustain. https://doi.org/10.1038/s41893-020-00617-y (2020).

  42. Bizikova, L. et al. A scoping review of the contributions of farmers’ organizations to smallholder agriculture. Nat. Food https://doi.org/10.1038/s43016-020-00164-x (2020).

  43. Maïga, W. H. E. et al. A systematic review of youth skills training programmes in agriculture in low- and middle-income countries. Nat. Food https://doi.org/10.1038/s43016-020-00172-x (2020).

  44. Webb, P. & Kennedy, E. Impacts of agriculture on nutrition: nature of the evidence and research gaps. Food Nutr. Bull. 35, 126–132 (2014).

    Article  Google Scholar 

  45. Yuan, Y. & Hunt, R. H. Systematic reviews: the good, the bad, and the ugly. Am. J. Gastroenterol. 104, 1086–1092 (2009).

    Article  Google Scholar 

  46. Haddaway, N. R. et al. A framework for stakeholder engagement during systematic reviews and maps in environmental management. Environ. Evid. 6, 11 (2017).

    Article  Google Scholar 

  47. Arnott, D. Cognitive biases and decision support systems development: a design science approach. Inf. Syst. J. 16, 55–78 (2006).

    Article  Google Scholar 

  48. Minas, R. K. & Crosby, M. E. In Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience (eds Schmorrow, D. D. & Fidopiastis, C. M.) 242–252 (Springer, 2016).

  49. Gil, Y., Greaves, M., Hendler, J. & Hirsh, H. Amplify scientific discovery with artificial intelligence. Science 346, 171–172 (2014).

    Article  Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge the funding of this research was provided by the Federal Ministry of Economic Cooperation (BMZ Germany) and the Bill and Melinda Gates OPP1210352 for the project Ceres2030: Sustainable Solutions to End Hunger. Thank you to Wences Almazan for the artful reproductions of Persephone, and a very heartfelt thank you to all of collaborators and partners who participated in Ceres2030.

Author information

Authors and Affiliations

  1. Department of Global Development, Cornell University, Ithaca, NY, USA

    Jaron Porciello & Stefan Einarson

  2. EPAM Systems, Minsk, Belarus

    Maryia Ivanina

  3. Belarusian State University, Minsk, Belarus

    Maryia Ivanina

  4. Department of Computer and Information Sciences, Cornell University, Ithaca, NY, USA

    Maidul Islam & Haym Hirsh

Authors
  1. Jaron Porciello
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maryia Ivanina
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maidul Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefan Einarson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haym Hirsh
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.P. designed the approach, managed the project and contributed to some of the programming. M.Ivanina performed most of the programming. M.Islam performed most of the automation and retrieval for grey literature. S.E. assisted with automation and technical infrastructure. H.H. consulted on machine-learning models. J.P. wrote the paper with contributions from M.Ivanina.

Corresponding author

Correspondence to Jaron Porciello.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplemental Figures 1–4 and Supplemental Tables 1–4

Source data

Source Data Fig. 2

Statistical source data

Source Data Fig. 4

Statistical source data

Source Data Fig. 5

Statistical source data

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Porciello, J., Ivanina, M., Islam, M. et al. Accelerating evidence-informed decision-making for the Sustainable Development Goals using machine learning. Nat Mach Intell 2, 559–565 (2020). https://doi.org/10.1038/s42256-020-00235-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-020-00235-5

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing