Improving sustainability knowledge has long been central to international efforts to achieve sustainable development. In response to these efforts, which are formalized in Target 4.7 of the Sustainable Development Goals, a global group of scholars and practitioners, in cooperation with the United Nations, designed and fielded the ‘Sulitest’, a survey tool that assesses sustainability knowledge in a variety of contexts. The Sulitest has been taken by over 160,000 individuals across 63 countries. Despite its substantial use, there is little systematic analysis of the data or the test itself. We analyse the Sulitest using both exploratory and confirmatory factor analysis, common techniques for identifying latent components within observed data. The Sulitest was designed to measure knowledge within four themes; however, this architecture is not supported by the data. Analysis suggests there is no coherent structure of sustainability knowledge. We urge caution to policymakers and educators when using the Sulitest as a diagnostic tool for assessing sustainability knowledge.
This is a preview of subscription content
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
The data that support the findings of this study were provided to the researchers by Sulitest with the condition that the exact question wording and response data not be made publicly available. However, data are available on reasonable request from Sulitest (www.sulitest.org). Aggregate data and further information, including coding, on the analysis, within the bounds of the memorandum of understanding, are available from the authors on request.
The Sulitest Story (Sulitest, accessed 1 March 2019); https://www.sulitest.org/en/the-sulitest-initiative.html
Décamps, A., Barbat, G., Carteron, J. C., Hands, V. & Parkes, C. Sulitest: a collaborative initiative to support and assess sustainability literacy in higher education. Int. J. Manag. Educ. 15, 138–152 (2017).
Sauvé, L. Environmental education and sustainable development: a further appraisal. Can. J. Environ. Educ. 1, 7–34 (1996).
Brundtland, G. H. The Brundtland report: ‘Our common future’. Med. Confl. Surviv. 4, 17–25 (1987).
Abbasi, D. R. Americans and Climate Change: Closing the Gap between Science and Action (Yale School of Forestry & Environmental Studies, 2006).
Weiss, M. & Barth, M. Global research landscape of sustainability curricula implementation in higher education. Int. J. Sustain. High. Educ. 20, 570–589 (2019).
Leal Filho, W. (ed.) Implementing Sustainability in the Curriculum of Universities: Approaches, Methods and Projects (Springer, 2018).
Scheunpflug, A. & Asbrand, B. Global education and education for sustainability. Environ. Educ. Res. 12, 33–46 (2006).
Jerman, P. L., Coull, B. C., Elzerman, A. W. & Schmidt, M. G. in Sustainability on Campus: Stories and Strategies for Change (eds Barlett, P. F. & Chase, G. W.) 243–258 (MIT Press, 2004).
Knutsson, B. Segmented prizing: biopolitical differentiation in education for sustainable development. Comp. J. Comp. Int. Educ. https://doi.org/10.1080/03057925.2019.1629276 (2019).
What is Education for Sustainable Development? (UNESCO, 2019); https://en.unesco.org/themes/education-sustainable-development/what-is-esd
Gallway, S. Capturing transformative change in education: the challenge of tracking progress towards SDG Target 4.7. Policy Pract. A Dev. Educ. Rev. 23, 124–138 (2016).
Kamau, M., Chasek, P. & O’Connor, D. C. Transforming Mulitlateral Diplomacy: The Inside Story of the Sustainable Development Goals (Routledge, 2018).
Mastrángelo, M. E. et al. Key knowledge gaps to achieve global sustainability goals. Nat. Sustain. 2, 1115–1121 (2019).
The Future We Want (United Nations, 2012); https://www.un.org/ga/search/view_doc.asp?symbol=A/RES/66/288
Raising & Mapping Awareness of the Global Goals (Higher Education Sustainability Initiative, 2020); https://www.sulitest.org/files/source/hlpf2020_report.pdf
Horvath, N., Stewart, M. & Shea, M. Toward instruments of assessing sustainability knowledge: assessment development, process, and results from a pilot survey at the University of Maryland. J. Sustain. Educ. 5, 5 (2013).
Zwickle, A., Koontz, T., Slagle, K. & Bruskotte, J. Assessing sustainability knowledge of a student population. Int. J. Sustain. High. Educ. 15, 375–389 (2014).
Akeel, U., Bell, S. & Mitchell, J. E. Assessing the sustainability literacy of the Nigerian engineering community. J. Clean. Prod. 212, 666–676 (2019).
Zwickle, A. & Jones, K. in Handbook of Sustainability and Social Science Research (eds Leal Filho, W. et al.) 435–451 (Springer, 2018); https://doi.org/10.1007/978-3-319-67122-2_25
Mapping Awareness of the Global Goals (Higher Education Sustainability Initiative, 2017); https://www.sulitest.org/files/source/hlpf2017report.pdf
The Sulitest Organization (Sulitest, accessed 11 January 2020); https://www.sulitest.org/en/the-organization.html
Worthington, R. L. & Whittaker, T. A. Scale development research: a content analysis and recommendations for best practices. Couns. Psychol. 34, 806–838 (2006).
Nunnally, J. C. Psychometric Theory (McGraw-Hill, 1978).
Knekta, E., Runyon, C. & Eddy, S. One size doesn’t fit all: using factor analysis to gather validity evidence when using surveys in your research. CBE Life Sci. Educ. 18, 1–17 (2019).
Wiley, E. W., Shavelson, R. J. & Kurpius, A. A. On the factorial structure of the SAT and implications for next-generation college readiness assessments. Educ. Psychol. Meas. 74, 859–874 (2014).
Revelle, W. psych: procedures for psychological, psychometric, and personality research. R package version 1.0–95 (2020).
Hooper, D., Coughlan, J. & Mullen, M. R. Structural equation modelling: guidelines for determining model fit. Electron. J. Bus. Res. Methods 6, 53–60 (2008).
Kuehl, C., Sparks, A., Hodges, H., Smith, E. R. A. N. & Maynard, K. Measuring sustainability literacy - developing and assessing a measure of sustainability literacy on a UC Campus. In 2016 Annual Meeting of the American Political Science Association (American Political Science Association, 2016).
Redclift, M. Sustainable development (1987-2005): an oxymoron comes of age. Sustain. Dev. 13, 212–227 (2005).
Hopwood, B., Mellor, M. & O’Brien, G. Sustainable development: mapping different approaches. Sustain. Dev. 13, 38–52 (2005).
Cogut, G., Webster, N. J., Marans, R. W. & Callewaert, J. Links between sustainability-related awareness and behavior: the moderating role of engagement. Int. J. Sustain. High. Educ. 20, 1240–1257 (2019).
Kollmuss, A. & Agyeman, J. Mind the Gap: why do people act environmentally and what are the barriers to pro-environmental behavior? Environ. Educ. Res. 8, 239–260 (2010).
Heeren, A. J. et al. Is sustainability knowledge half the battle?: an examination of sustainability knowledge, attitudes, norms, and efficacy to understand sustainable behaviours. Int. J. Sustain. High. Educ. 17, 613–632 (2016).
MacCallum, R. C., Zhang, S., Preacher, K. J. & Rucker, D. D. On the practice of dichotomization of quantitative variables. Psychol. Methods 7, 19–40 (2002).
Debelak, R. & Tran, U. S. Principal component analysis of smoothed tetrachoric correlation matrices as a measure of dimensionality. Educ. Psychol. Meas. 73, 63–77 (2013).
We thank A. Décamps and the Sulitest Organization for making the underlying data accessible.
The authors declare no competing interests.
Peer review information Nature Sustainability thanks Valeria Schwanitz and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kuehl, C., Sparks, A.C., Hodges, H. et al. The incoherence of sustainability literacy assessed with the Sulitest. Nat Sustain 4, 555–560 (2021). https://doi.org/10.1038/s41893-021-00687-6