Explaining topic prevalence in answers to open-ended survey questions about climate change


Citizens’ opinions are crucial for action on climate change, but are, owing to the complexity of the issue, diverse and potentially unformed1. We contribute to the understanding of public views on climate change and to knowledge needed by decision-makers by using a new approach to analyse answers to the open survey question ‘what comes to mind when you hear the words ‘climate change’?’. We apply automated text analysis, specifically structural topic modelling2, which induces distinct topics based on the relative frequencies of the words used in 2,115 responses. From these data, originating from the new, nationally representative Norwegian Citizen Panel, four distinct topics emerge: Weather/Ice, Future/Impact, Money/Consumption and Attribution. We find that Norwegians emphasize societal aspects of climate change more than do respondents in previous US and UK studies3,4,5,6. Furthermore, variables that explain variation in closed questions, such as gender and education, yield different and surprising results when employed to explain variation in what respondents emphasize. Finally, the sharp distinction between scepticism and acceptance of conventional climate science, often seen in previous studies, blurs in many textual responses as scepticism frequently turns into ambivalence.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Figure 1: Effects of climate concern and age on topic prevalence.
Figure 2: Effect of gender and education on topic prevalence.


  1. 1

    Hulme, M. Why We Disagree about Climate Change: Understanding Controversy, Inaction and Opportunity (Cambridge Univ. Press, 2009).

  2. 2

    Roberts, M. E. et al. Structural topic models for open-ended survey responses. Am. J. Political Sci. 58, 1064–1082 (2014).

  3. 3

    Leiserowitz, A. Climate change risk perception and policy preferences: the role of affect, imagery, and values. Climatic Change 77, 45–72 (2006).

  4. 4

    Leiserowitz, A. A. American risk perceptions: Is climate change dangerous? Risk Anal. 25, 1433–1442 (2005).

  5. 5

    Smith, N. & Leiserowitz, A. The rise of global warming skepticism: Exploring affective image associations in the United States over time. Risk Anal. 32, 1021–1032 (2012).

  6. 6

    Lorenzoni, I., Leiserowitz, A., de Franca Doria, M., Poortinga, W. & Pidgeon, N. F. Cross-national comparisons of image associations with “global warming” and “climate change” among laypeople in the United States of America and Great Britain. J. Risk Res. 9, 265–281 (2006).

  7. 7

    Austgulen, M. H. & Stø, E. Norsk skepsis og usikkerhet om klimaendringer. Tidsskr. Samfunnsforskning 54, 124–150 (2013).

  8. 8

    Dunlap, R. E. & McCright, A. M. A widening gap: Republican and Democratic views on climate change. Environ. Sci. Policy Sustainable Dev. 50, 26–35 (2008).

  9. 9

    Engels, A., Hüther, O., Schäfer, M. & Held, H. Public climate-change skepticism, energy preferences and political participation. Glob. Environ. Change 23, 1018–1027 (2013).

  10. 10

    Scruggs, L. & Benegal, S. Declining public concern about climate change: Can we blame the great recession? Glob. Environ. Change 22, 505–515 (2012).

  11. 11

    Tingley, D. & Tomz, M. Conditional cooperation and climate change. Comp. Polit. Stud. 47, 344–368 (2014).

  12. 12

    McCright, A. M. The effects of gender on climate change knowledge and concern in the American public. Population Environ. 32, 66–87 (2010).

  13. 13

    Watson, R. T. & Albritton, D. L. Climate Change 2001: Synthesis Report (IPCC, Cambridge Univ. Press, 2001).

  14. 14

    Fløttum, K, Gjesdal, A. M., Gjerstad, Ø., Koteyko, N. & Salway, A. Representations of the future in English language blogs on climate change. Glob. Environ. Change 29, 213–222 (2014).

  15. 15

    Morton, T. A., Rabinovich, A., Marshall, D. & Bretschneider, P. The future that may (or may not) come: How framing changes responses to uncertainty in climate change communications. Glob. Environ. Change 21, 103–109 (2011).

  16. 16

    Stoneman, P., Sturgis, P. & Allum, N. Exploring public discourses about emerging technologies through statistical clustering of open-ended survey questions. Public Underst. Sci. 22, 850–868 (2013).

  17. 17

    Shwom, R., Bidwell, D., Dan, A. & Dietz, T. Understanding US public support for domestic climate change policies. Glob. Environ. Change 20, 472–482 (2010).

  18. 18

    Sunstein, C. R. On the divergent American reactions to terrorism and climate change. Columbia Law Rev. 107, 503–557 (2007).

  19. 19

    Lyons, J. Semantics Vol. 1, 2 (Cambridge Univ. Press, 1977).

  20. 20

    Kerbrat-Orecchioni, C. L’énonciation: de la Subjectivité dans le Langage (Armand Colin, 2009).

  21. 21

    Roberts, M. E., Stewart, B. M. & Tingley, D. stm: R Package for Structural Topic Models (2014); http://structuraltopicmodel.com

  22. 22

    Roberts, M. E. et al. Online appendix to structural topic models for open-ended survey responses. Am. J. Political Sci. 58, 1064–1082 (2014).

  23. 23

    Bischof, J. & Airoldi, E. M. Proc. 29th International Conference on Machine Learning (ICML-12) 201–208 (2012).

  24. 24

    Ivarsflaten, E. et al. Endre Tvinnereim and Jacob Aars. Norsk Medborgerpanel [Norwegian Citizen Panel] Round 1: Data file (Univ. Bergen, 2014).

  25. 25

    Høgestøl, A. & Skjervheim, Ø. Norwegian Citizen Panel 2013: First Wave Methodology Report (Univ. Bergen, 2014).

  26. 26

    Tourangeau, R. & Bradburn, N. Handbook of Survey Research (Emerald, 2000).

  27. 27

    Weijters, B., Geuens, M. & Schillewaert, N. The proximity effect: The role of inter-item distance on reverse-item bias. Int. J. Res. Mark. 26, 2–12 (2009).

  28. 28

    Tourangeau, R., Rasinski, K. A., Bradburn, N. & D’Andrade, R. Belief accessibility and context effects in attitude measurement. J. Exp. Soc. Psychol. 25, 401–421 (1989).

  29. 29

    Leviston, Z., Walker, I. & Morwinski, S. Your opinion on climate change might not be as common as you think. Nature Clim. Change 3, 334–337 (2012).

Download references


The Research Council of Norway provided financial support for part of the work under the Lingclim project (grant no. 220654). We are very grateful to B. M. Stewart for support on technical issues related to our structural topic models. We also thank participants at the Norwegian Citizen Panel conference, Bergen, 6–7 November 2014.

Author information

E.T. conceived the study and performed the quantitative analysis. E.T. and K.F. performed the qualitative analysis and structured and wrote the text.

Correspondence to Endre Tvinnereim.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tvinnereim, E., Fløttum, K. Explaining topic prevalence in answers to open-ended survey questions about climate change. Nature Clim Change 5, 744–747 (2015). https://doi.org/10.1038/nclimate2663

Download citation

Further reading