Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Analysis
  • Published:

The incoherence of sustainability literacy assessed with the Sulitest

Nature Sustainability volume 4, pages 555–560 (2021)Cite this article

Subjects

Abstract

Improving sustainability knowledge has long been central to international efforts to achieve sustainable development. In response to these efforts, which are formalized in Target 4.7 of the Sustainable Development Goals, a global group of scholars and practitioners, in cooperation with the United Nations, designed and fielded the ‘Sulitest’, a survey tool that assesses sustainability knowledge in a variety of contexts. The Sulitest has been taken by over 160,000 individuals across 63 countries. Despite its substantial use, there is little systematic analysis of the data or the test itself. We analyse the Sulitest using both exploratory and confirmatory factor analysis, common techniques for identifying latent components within observed data. The Sulitest was designed to measure knowledge within four themes; however, this architecture is not supported by the data. Analysis suggests there is no coherent structure of sustainability knowledge. We urge caution to policymakers and educators when using the Sulitest as a diagnostic tool for assessing sustainability knowledge.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Scree plot produced by EFA.
Fig. 2: Expected versus actual factor structure as determined by EFA.

Similar content being viewed by others

Data availability

The data that support the findings of this study were provided to the researchers by Sulitest with the condition that the exact question wording and response data not be made publicly available. However, data are available on reasonable request from Sulitest (www.sulitest.org). Aggregate data and further information, including coding, on the analysis, within the bounds of the memorandum of understanding, are available from the authors on request.

References

  1. The Sulitest Story (Sulitest, accessed 1 March 2019); https://www.sulitest.org/en/the-sulitest-initiative.html

  2. Décamps, A., Barbat, G., Carteron, J. C., Hands, V. & Parkes, C. Sulitest: a collaborative initiative to support and assess sustainability literacy in higher education. Int. J. Manag. Educ. 15, 138–152 (2017).

    Article  Google Scholar 

  3. Sauvé, L. Environmental education and sustainable development: a further appraisal. Can. J. Environ. Educ. 1, 7–34 (1996).

    Google Scholar 

  4. Brundtland, G. H. The Brundtland report: ‘Our common future’. Med. Confl. Surviv. 4, 17–25 (1987).

  5. Abbasi, D. R. Americans and Climate Change: Closing the Gap between Science and Action (Yale School of Forestry & Environmental Studies, 2006).

  6. Weiss, M. & Barth, M. Global research landscape of sustainability curricula implementation in higher education. Int. J. Sustain. High. Educ. 20, 570–589 (2019).

    Article  Google Scholar 

  7. Leal Filho, W. (ed.) Implementing Sustainability in the Curriculum of Universities: Approaches, Methods and Projects (Springer, 2018).

  8. Scheunpflug, A. & Asbrand, B. Global education and education for sustainability. Environ. Educ. Res. 12, 33–46 (2006).

  9. Jerman, P. L., Coull, B. C., Elzerman, A. W. & Schmidt, M. G. in Sustainability on Campus: Stories and Strategies for Change (eds Barlett, P. F. & Chase, G. W.) 243–258 (MIT Press, 2004).

  10. Knutsson, B. Segmented prizing: biopolitical differentiation in education for sustainable development. Comp. J. Comp. Int. Educ. https://doi.org/10.1080/03057925.2019.1629276 (2019).

  11. What is Education for Sustainable Development? (UNESCO, 2019); https://en.unesco.org/themes/education-sustainable-development/what-is-esd

  12. Gallway, S. Capturing transformative change in education: the challenge of tracking progress towards SDG Target 4.7. Policy Pract. A Dev. Educ. Rev. 23, 124–138 (2016).

  13. Kamau, M., Chasek, P. & O’Connor, D. C. Transforming Mulitlateral Diplomacy: The Inside Story of the Sustainable Development Goals (Routledge, 2018).

  14. Mastrángelo, M. E. et al. Key knowledge gaps to achieve global sustainability goals. Nat. Sustain. 2, 1115–1121 (2019).

    Article  Google Scholar 

  15. The Future We Want (United Nations, 2012); https://www.un.org/ga/search/view_doc.asp?symbol=A/RES/66/288

  16. Raising & Mapping Awareness of the Global Goals (Higher Education Sustainability Initiative, 2020); https://www.sulitest.org/files/source/hlpf2020_report.pdf

  17. Horvath, N., Stewart, M. & Shea, M. Toward instruments of assessing sustainability knowledge: assessment development, process, and results from a pilot survey at the University of Maryland. J. Sustain. Educ. 5, 5 (2013).

    Google Scholar 

  18. Zwickle, A., Koontz, T., Slagle, K. & Bruskotte, J. Assessing sustainability knowledge of a student population. Int. J. Sustain. High. Educ. 15, 375–389 (2014).

    Article  Google Scholar 

  19. Akeel, U., Bell, S. & Mitchell, J. E. Assessing the sustainability literacy of the Nigerian engineering community. J. Clean. Prod. 212, 666–676 (2019).

  20. Zwickle, A. & Jones, K. in Handbook of Sustainability and Social Science Research (eds Leal Filho, W. et al.) 435–451 (Springer, 2018); https://doi.org/10.1007/978-3-319-67122-2_25

  21. Mapping Awareness of the Global Goals (Higher Education Sustainability Initiative, 2017); https://www.sulitest.org/files/source/hlpf2017report.pdf

  22. The Sulitest Organization (Sulitest, accessed 11 January 2020); https://www.sulitest.org/en/the-organization.html

  23. Worthington, R. L. & Whittaker, T. A. Scale development research: a content analysis and recommendations for best practices. Couns. Psychol. 34, 806–838 (2006).

    Article  Google Scholar 

  24. Nunnally, J. C. Psychometric Theory (McGraw-Hill, 1978).

  25. Knekta, E., Runyon, C. & Eddy, S. One size doesn’t fit all: using factor analysis to gather validity evidence when using surveys in your research. CBE Life Sci. Educ. 18, 1–17 (2019).

    Article  Google Scholar 

  26. Wiley, E. W., Shavelson, R. J. & Kurpius, A. A. On the factorial structure of the SAT and implications for next-generation college readiness assessments. Educ. Psychol. Meas. 74, 859–874 (2014).

  27. Revelle, W. psych: procedures for psychological, psychometric, and personality research. R package version 1.0–95 (2020).

  28. Hooper, D., Coughlan, J. & Mullen, M. R. Structural equation modelling: guidelines for determining model fit. Electron. J. Bus. Res. Methods 6, 53–60 (2008).

  29. Kuehl, C., Sparks, A., Hodges, H., Smith, E. R. A. N. & Maynard, K. Measuring sustainability literacy - developing and assessing a measure of sustainability literacy on a UC Campus. In 2016 Annual Meeting of the American Political Science Association (American Political Science Association, 2016).

  30. Redclift, M. Sustainable development (1987-2005): an oxymoron comes of age. Sustain. Dev. 13, 212–227 (2005).

  31. Hopwood, B., Mellor, M. & O’Brien, G. Sustainable development: mapping different approaches. Sustain. Dev. 13, 38–52 (2005).

  32. Cogut, G., Webster, N. J., Marans, R. W. & Callewaert, J. Links between sustainability-related awareness and behavior: the moderating role of engagement. Int. J. Sustain. High. Educ. 20, 1240–1257 (2019).

    Article  Google Scholar 

  33. Kollmuss, A. & Agyeman, J. Mind the Gap: why do people act environmentally and what are the barriers to pro-environmental behavior? Environ. Educ. Res. 8, 239–260 (2010).

  34. Heeren, A. J. et al. Is sustainability knowledge half the battle?: an examination of sustainability knowledge, attitudes, norms, and efficacy to understand sustainable behaviours. Int. J. Sustain. High. Educ. 17, 613–632 (2016).

    Article  Google Scholar 

  35. MacCallum, R. C., Zhang, S., Preacher, K. J. & Rucker, D. D. On the practice of dichotomization of quantitative variables. Psychol. Methods 7, 19–40 (2002).

    Article  Google Scholar 

  36. Debelak, R. & Tran, U. S. Principal component analysis of smoothed tetrachoric correlation matrices as a measure of dimensionality. Educ. Psychol. Meas. 73, 63–77 (2013).

    Article  Google Scholar 

Download references

Acknowledgements

We thank A. Décamps and the Sulitest Organization for making the underlying data accessible.

Author information

Authors and Affiliations

  1. Department of Political Science and Institute for the Study of Environment, Sustainability & Energy, Northern Illinois University, Dekalb, IL, USA

    Colin Kuehl

  2. Political Science and Policy Studies, Elon University, Elon, NC, USA

    Aaron C. Sparks

  3. Bren School of Environmental Science and Management, University of California, Santa Barbara, CA, USA

    Heather Hodges

  4. Department of Political Science, University of California, Santa Barbara, CA, USA

    Eric R. A. N. Smith

Authors
  1. Colin Kuehl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aaron C. Sparks
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heather Hodges
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eric R. A. N. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to manuscript preparation. C.K. coordinated data collection and preparation, coordinated tasks and, in conjunction with A.C.S. and E.R.A.N.S., performed the data analysis and interpretation. H.H. helped in task coordination and aided in theoretical development.

Corresponding author

Correspondence to Colin Kuehl.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Sustainability thanks Valeria Schwanitz and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Fig. 1 and Tables 1 and 2.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kuehl, C., Sparks, A.C., Hodges, H. et al. The incoherence of sustainability literacy assessed with the Sulitest. Nat Sustain 4, 555–560 (2021). https://doi.org/10.1038/s41893-021-00687-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41893-021-00687-6

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing