Measuring systems thinking

Systems thinking has been promoted as a way to improve human–environmental interactions, but analytical approaches to measure degrees of systems thinking remain elusive. If more complex thinking does improve sustainable decision-making, new methods to validate this prevalent hypothesis must be developed.

US Supreme Court Justice Potter Stewart famously said “I know it when I see it” when defining what should, and should not, be considered pornography1. In his statement, Justice Stewart was communicating a frustration with trying to assess whether observable objects belong to one category, or another, under conditions when criteria for inclusion do not clearly exist. Assessing the degree of ‘systems thinking’ in environmental decision-making is very similar to the difficulty and ambiguity that Justice Stewart expressed in his simple statement. Even though the degree of systems thinking is thought to be both obvious and essential for solving complex social and environmental problems, criteria for what constitutes evidence of systems thinking is highly underdeveloped. In this issue of Nature Sustainability Levy and colleagues2 try to address what it means to be a systems thinker by combining motifs found in decision-maker mental models and provide novel empirical ways to measure the elusive construct. Such methods will allow researchers to test hypotheses related to complexity of mental model structure and determine whether they do actually correlate with improved decision-making and environmental outcomes.

General systems theory, the interdisciplinary study of systems, both natural and manmade, has been around for more than 75 years3. The idea is somewhat straightforward: that the world is composed of various systems, with inter-related but independent parts, which interact and produce complex outcomes. Understanding these various systems requires that we make models, either informal mental models or formalized computer models4, that draw conceptual boundaries around a system and identify how a system’s structure relates to its behaviour and function5. If our conceptual models of a system sufficiently align with reality, then we can better predict its behaviour, consider possible outcomes and therefore improve our decision-making about complex social or environmental problems. But do people think in terms of systems?

In a study by Sternman and Sweeney, 212 students from MIT were provided with a description of the relationship between greenhouse gas emissions, atmospheric concentrations and global mean temperature and asked to predict the emissions trajectory required to stabilize atmospheric CO2 (ref. 6). Although knowledge in climatology or calculus were not needed to determine the correct answer, 84% drew patterns that violated system principles of accumulation, an indication that their mental models were misaligned with the somewhat straightforward relationships that account for climate change in a ‘simple’ system. The implication being that policy preferences for the majority of study participants, based on these flawed mental models, do not reflect real world system dynamics of the problem. But what about morecomplex and ‘wicked’ problems for which there is no correct answer and we don’t necessarily know the key relationships that lead to outcomes like improved sustainability or well-being? How do we know if people are systems thinkers or not?

Levy and colleagues try to address this problem and move the field forward in three major ways. First, they were able to identify microstructures in stakeholder mental models that reflect how decision-makers internalize external reality in their minds, and the degree to which these internal representations are more or less complex. While previous studies have promoted to some degree the use of general network metrics7 the authors use more nuanced network theory approaches to delineate simple and complex causal patterns of reasoning. However, it is not just the new measurements alone, they further cluster decision-makers into three categories of systems thinking based on inclusion of these motifs in a hierarchy. Such measurements provide an ordinal ranking and establish criteria for lower and higher order reasoning as they relate to the environmental systems that individuals interact with.

Second, the authors find that increased education and experience modestly correlate with higher levels of systems thinking. Although, as the authors mention, there is evidence to the contrary8 and local ‘experts’ independent of academic training can and often do exhibit highly complex mental models9, the directionality is an indication that thinking about the complexity of a system can be nurtured both informally, with increasing interaction with the environment via experiences, and formally through academic training, is promising for promoting systems thinking among stakeholders involved in the management of natural resource systems.

Third, the authors evaluate correlations between specific measurements that indicate higher levels of systems thinking, providing new ways to parse the construct in the future and establish a benchmark for specific empirical metrics of general systems thinking. This is perhaps the biggest contribution because it provides a way in which different degrees of complexity of mental models may or may not correlate with quality of decision-making or a range of other independent (for example, what types of experiences, interventions or training may lead to more complex reasoning patterns?) or dependent (for example, what are the implications for more complex reasoning patterns in managing fisheries, forests or farms more sustainably?) variables. Indeed, these metrics provide a way for researchers in the future to truly evaluate the trade-offs in simpler to more-complex thinking, and test the dominant hypothesis that systems thinkers make better decisions and lead to better and more-sustainable outcomes.

Notwithstanding, one major question that the study does not address is whether refined, rarefied and simplified knowledge compared to more-complex constructed and accumulated knowledge may be more or less efficient in understanding human decision-making in general10. When does an informal or formal model of a system become overly complex for the problem at hand? Perhaps counter to Occam’s razor, or the law of parsimony11, Levy and colleagues argue that if mental models are more complex structurally, they are assumed to lead to better outcomes but the evidence to support this is scant.

In summary, Levy and colleagues provide a novel approach to empirically test assumptions about the value of systems thinking to improve sustainable decision-making and help define what evidence of systems thinking is. But if more-complex thinking does improve sustainable decision-making more studies are needed to validate this predominant assumption to understand when and under what conditions systems thinking can be fostered and, further, whether systems thinkers make better and more-sustainable decisions.


  1. 1.

    Jacobellis v. Ohio 378 US 184 (US Supreme Court, 1964).

  2. 2.

    Levy, M. A., Lubell, M. N. & McRoberts, N. (2018).

  3. 3.

    von Bertalanffy, L. Modern Theories of Development: an Introduction to Theoretical Biology (Oxford Univ. Press, New York, 1933).

  4. 4.

    Meadows, D. H. Thinking in Systems: a Primer (Chelsea Green, White River Junction, 2008).

    Google Scholar 

  5. 5.

    Hmelo-Silver, C. E. & Pfeffer, M. G. Cognitive Sci. 28, 127–38 (2004).

    Article  Google Scholar 

  6. 6.

    Sterman, J. D. & Sweeney, L. B. Climatic Change 80, 213–238 (2007).

    Article  CAS  Google Scholar 

  7. 7.

    Özesmi, U. & Özesmi, S. Ecol. Model. 176, 43–64 (2004).

    Article  Google Scholar 

  8. 8.

    Stier, A. et al. Conserv. Lett. 10, 67–76 (2017).

    Article  Google Scholar 

  9. 9.

    Gray, S., McFall, A., Hilsberg, J. & Arlinghaus, R. J. Outdoor Recreat. Tourism 12, 1–13 (2015).

    Article  Google Scholar 

  10. 10.

    Kahneman, D. Thinking, Fast and Slow (Farrar, Straus and Giroux, New York, 2011).

  11. 11.

    Gauch, H. G. Scientific Method in Practice (Cambridge Univ. Press, Cambridge, 2003).

Download references

Author information



Corresponding author

Correspondence to Steven Gray.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gray, S. Measuring systems thinking. Nat Sustain 1, 388–389 (2018).

Download citation

Further reading


Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing