Introduction

Governance for urban health is defined by the need for a systems approach to tackle the complex causes of ill-health, requiring multi-stakeholder collaboration across sectors, including the translation of knowledge from research to practice (Kickbusch and Gleicher, 2012; Gatzweiler et al., 2018). Scientific knowledge and evidence are usually conceived outside of policy systems and then brokered or disseminated into the policy process, with varying degrees of success (Cairney and Oliver, 2017). Explanations for failed translation include epistemological, disciplinary, political, and professional differences, and interactions across these categories (Smith and Joyce, 2012) and a range of ‘barriers’ (see Innvaer et al., 2002; Oliver et al., 2014a) identified under the assumption that evidence-based policy models are accurate (Cairney and Oliver, 2017). The Lalonde Report (1974) and WHO Healthy Cities Movement (Hancock and Duhl, 1986) advocated for public health professionals to influence urban planning policy and decision-making to improve the social determinants of health. Indicators, and specifically urban health indicators (UHIs), are one growing form of evidence that public health practitioners use to inform urban policy-makers, often assuming a rational and linear process from evidence to action (Pineo et al., 2018a, 2019; Decoville, 2018; Sébastien, Bauler and Lehtonen, 2014; Innes and Booher, 2000; Pastille Consortium, 2002). Although scholars and practitioners previously conceived of policy and decision-making as a positivist ‘rational’ approach that involved ‘objective data, logical deductive analysis and systematic comparison of alternatives’ (Innes and Booher, 2010, p. 17), this conceptualisation has been rejected by scholars who recognise the complex and contested nature of policy processes, and the use of evidence within these processes (Cairney, 2012; Innes and Booher, 2010; Cairney and Oliver, 2017). It is therefore unclear how UHIs may inform or influence the complex urban planning policy and decision-making context.

Research on indicators has tended to focus on their development and validation, rather than their application in policy processes (Decoville, 2018; Sébastien, Bauler and Lehtonen, 2014; Wong, 2006; Innes and Booher, 2010; Pastille Consortium, 2002). The Pastille Consortium argued that policy and decision-makers do not use sustainability indicators; but rather indicators ‘function inside the governance process’ (2002, p. 90). Studies highlight sustainability indicators’ roles in supporting: legitimation of actors’ policy agendas; cross-departmental and transdisciplinary relationship building; and the construction and presentation of new knowledge to inform policy, bridging diverse institutional norms and values (Pastille Consortium, 2002; Sébastien, Bauler and Lehtonen, 2014). Nevertheless, studies on the use of UHI tools have identified their direct use in urban planning policy (e.g., Bhatia, 2014; Farhang et al., 2008), for example where indicators that highlighted air pollution exposures were used to formulate policy detailing mechanical ventilation requirements for new housing (Bhatia, 2014).

Despite uncertainty regarding the value of indicators to policy-makers, the production of UHI tools continues apace and our previous work has investigated this trend through a two-part systematic review, including a census of UHI tools and a narrative synthesis of studies on their policy application (Pineo et al., 2018a, 2019). We defined UHI tools broadly, encompassing ‘a collection of summary measures about the physical urban environment’s contribution to human health and wellbeing’ and liveability, quality of life and walkability/physical activity topics (Pineo et al., 2017, p. 2). Our systematic review of UHI tool characteristics identified 145 tools globally, comprising 8006 indicators, from which we developed a taxonomy. A growing number of UHI tools report data at neighbourhood scale or lower and display data via digital interactive maps (Pineo et al., 2018a).

Our narrative synthesis of ten studies on the use of UHI tools by built environment policy-makers found that participatory development of UHIs and reporting data at neighbourhood scale resulted in increased influence on health-promoting urban policy and decision-making compared to UHI tools that were not participatory and did not report this spatial scale of data (Pineo et al., 2019). Such indicators supported health-in-all-policies (HiAP) and whole-of-society approaches and helped identify health or spatial inequalities. Most of the papers in the narrative synthesis adopted case study designs (7/10) and with the exception of two studies, they were all written by individuals involved in developing the UHI tool being investigated (ibid). We developed a theory of change of how UHI tools inform policy that demonstrates the complex factors that can facilitate or inhibit the role of indicators in governance processes. The review showed that developing and applying UHI tools for policy-makers consumption is rarely a rational and linear process.

The synthesis distinguished between ‘participatory’ UHI tools that emerged from a process of co-production with healthy built environment stakeholders and ‘expert-led’ tools that had minimal external participation. A participatory process of indicator development was encouraged and adopted by Bhatia (2014) and Farhang et al. (2008) regarding the San Francisco Indicators Project and other case studies, including Hunt and Lewin (2000), Van Assche et al. (2010), Corburn and Cohen (2012), and Corburn et al. (2014), covering six of the UHI tools. The other four UHI tools were categorised as ‘expert-led’ and they involved minimal community or stakeholder involvement in their development (as reported by Lowe et al., 2015; Lerman, 2011; Shepherd and McMahon, 2009). The key benefits or impacts of developing and applying UHI tools included: informing policy development, creating awareness and knowledge of urban health issues, facilitating collaboration across stakeholders, supporting monitoring of urban environment health impacts, and providing evidence about health or spatial inequalities (Pineo et al., 2019).

Our systematic review also investigated how UHI tools represent and help to support policy-makers with the complexity of urban health systems and policy-making itself. Urban health is a complex system (Rydin et al., 2012; Northridge, Sclar and Biswas, 2003; Galea and Vlahov, 2005) and indicators are claimed to either mask complexity (Rothenberg et al., 2015; Decoville, 2018) or simplify it for policy-makers (Holden, 2001; Saisana and Tarantola, 2002). We synthesised strategies for developing and applying UHI tools to support policy-makers with seven characteristics of complexity; however, such strategies were rarely explained in detail by indicator producers, with notable exceptions from Corburn and Cohen (2012) and Van Assche et al. (2010).

This study addresses a knowledge gap by providing a rich account of the specific mechanisms through which our case study UHI tools support urban planning policy and decision-makers to promote health in complex governance contexts in San Francisco, Melbourne and Sydney. Our research objectives were as follows: (1) explore indicator producers and users’ views (or mental models) of the use and value of UHI tools for health promotion in the complex policy process, and in relation to the complexity of urban health, and (2) develop and test in a participatory manner a causal loop diagram of users and producers’ mental models. We combine collaborative rationality (Innes and Booher, 2010) with system thinking (Sterman, 2000) to conduct a thematic analysis and produce a qualitative systems model of indicator producers and users’ views. We contextualise our findings in relation to theory and the wider literature of evidence-based policy, indicator use and urban governance.

Diverse assumptions and values underlying public health and urban planning

Our conceptual approach to investigating the use and value of UHI tools recognises the diverse assumptions and values underlying public health and urban planning research and practice. First, these professions have different epistemological perspectives, characterised crudely by positivism and social constructivism, affecting how ‘evidence’ may be perceived and used by practitioners and academics. Public health researchers may prioritise positivist epidemiological evidence to inform policy (see Banta, 2003), yet practitioners consider a range of other factors and forms of evidence (Oliver, Lorenc and Innvaer, 2014; Oliver et al., 2014; Smith and Joyce, 2012). Modernist planning theory was built on rationality and the premise that ‘knowledge can be harnessed through planning to achieve positive change’ (Rydin, 2007, p. 53). Critics of this approach argued that knowledge is socially constructed and that a rational policy process conflicts with the values of urban planning, such as advocating for disadvantaged communities (Healey, 1997). Planning theory thus shifted to a communicative approach in which wide stakeholder engagement was underpinned with recognition that multiple and conflicting knowledge claims must be addressed in the planning process (Rydin, 2007). Allmendinger (2002) positions communicative planning approaches in the broader post-positivist planning theory landscape and recognises that other theoretical positions co-exist.

Second, the overarching purposes and underlying values of public health and urban planning are not necessarily aligned. Public health’s goal is ‘to reduce suffering and improve health’ (Lee and Zarowsky, 2015, p. 6). The purpose of urban planning is broadly to manage change in urban environments, with varied objectives in different localities and among different planners, including economic development, quality of life, efficiency and sustainable development (Bolan, 2017; Barton, 2017). Campbell and Marshall (2002) found a lack of consensus and reluctance to discuss values normatively among practicing planners. Lee and Zarowsky (2015) noted that public health professionals have some shared values, including: evidence-based decisions, justice, equity and participatory practice. These diverse assumptions mean that the concept of ‘evidence’ and its value in policy-making may carry different meanings and public health and planning practitioners may not have shared goals.

We adopt systems thinking and collaborative rationality in recognition of the diverse assumptions and values underpinning UHI tool producers and users’ professions. Systems thinking refocuses attention from detailed analysis of constituent parts to examination of the behaviour of the whole system of policy and decision-making, and can be done in a participatory manner with diverse stakeholders (Sterman, 2000; Meadows and Wright, 2008). Systems thinking is about examining problems from different viewpoints and ‘expand[ing] the boundaries of our mental models’ (Sterman, 2000, p. 511). Mental models are a ‘psychological core of understanding’ (Johnson-Laird, 1983, p. 81) or ‘stories about how dynamic systems work’ (Luna-Reyes and Andersen, 2003, p. 281). Using qualitative system dynamics methods (principally the development of causal loop diagrams from interview data and participatory modelling), we seek to understand characteristics of complex systems, including causal connections across a large number of factors and feedback relations—key aspects of thinking in systems. A systems approach in this research allows for a departure from the traditional focus on the barriers and facilitators of evidence use to a more holistic understanding of where improvements or interventions are possible. Furthermore, systems thinking moves beyond the rational model of policy and decision-making, building on Simon’s (1976) bounded rationality principle.

Innes and Booher (2003, 2010) combined components of Habermas’s (1984, 1987) communicative rationality and emancipatory knowledge with negotiation theory, complexity theory and their own planning practice knowledge to form their DIAD (diversity, interdependence and authentic dialogue) theory of collaborative rationality. This descriptive and normative theory is ‘a principle of operation that has become more and more relevant in the field of planning’ (Cociña et al., 2019, p. 131). Collaborative rationality recognises the failure of traditional linear models of policy-making and implementation, which rely on technical expertise. ‘Lay knowledge’ and the inclusion of diverse knowledge claims in collaborative policy-making processes are important when dealing with complex problems (Innes and Booher, 2000, p. 5). In essence, collaborative rationality means that multiple stakeholders deliberate on shared problems through in-person dialogue with the aim of reaching consensus. Collaboratively rational processes result in ‘new knowledge and unanticipated policies and practices’ alongside systemic ‘changes in the values, goals, shared understandings, and the underlying attitudes of the participants’ (Innes and Booher, 2010, p. 34). Innes and Booher argue that these systemic changes will help to reduce challenges associated with complexity by creating new relationships, capabilities and attitudes, which are dispersed and linked across networks and diverse actors. Thus, collaborative rationality provides a helpful lens through which to explore the complexity of promoting health through urban planning, via UHI tools.

Methods

The methods are described in summary format below with further detail in the supplementary material.

Case studies and interviews

We adopted a multiple case study design focusing on interview participants in cities with significant experience using UHI tools, based on Pineo et al. (2018a, 2019). The San Francisco Indicators Project (SFIP) and Community Indicators Victoria (CIV), are exemplifying cases of ‘participatory-led’ and ‘expert-led’ approaches in Pineo et al. (2019) and provide data aggregation at neighbourhood scale, allowing for investigation of the value of these characteristics. The long history of both projects (SFIP was published in 2007 and CIV in 2006) suggests that indicator producers and users would be familiar with their potential value and use. Both tools have a significant secondary literature (Bhatia, 2014; Bhatia and Corburn, 2011; Farhang et al., 2008; Davern et al., 2011, 2017; Lowe et al., 2015). During early interviews, participants did not view these tools as discrete phenomena, but instead they spoke about many indicator projects, some of which grew out of SFIP and CIV or were created by their producers (Supplementary material, section 1.5). Therefore, we widened our focus to UHI tool producers and users who had relevant experience of UHI tools in San Francisco, Victoria and New South Wales. All participants had knowledge of either SFIP or CIV. Table 1 describes interview and workshop (described below) participants’ characteristics regarding profession, organisation and location.

Table 1 Interview and workshop participant characteristics.
Full size table

Interviews were semi-structured, audio-recorded and transcribed. Interviews took place in San Francisco, USA (April 2016), and Melbourne and Sydney, Australia (March 2018). All participants were informed and consented. Eight out of 22 (36.4%) participants opted for anonymity, therefore the results have been fully anonymised. The interview guides were informed by our systematic review (Pineo et al., 2018a, 2019) and conceptual approach described above. Interview questions were different for indicator producers and users (Supplementary material, section 1.1). Indicator producers were asked more about the process of developing a UHI tool and their perception of how it is used. Indicator users were asked to discuss their familiarity with UHI tools and to describe a project in which they applied a UHI tool. Both groups were asked about how indicators may help urban planners deal with the complexity of urban health.

Data analysis and modelling

We analysed interview data using thematic analysis (Braun and Clarke, 2006; Nowell et al., 2017), adopting a hybrid inductive and deductive coding approach (Fereday and Muir-Cochrane, 2006), analysing data at latent and semantic levels. The semantic level involves coding data based on what is explicitly written or said, and then interpreting patterns and broader implications. For example, we coded data in an inductively derived category called ‘perceptions of UHI tools’ that included codes such as ‘data quality and availability’ and ‘number of indicators’. Latent level coding goes beyond the surface level looking for ‘underlying ideas, assumptions and conceptualisations—and ideologies’ thereby interpreting meaning when developing codes and themes from the data (Braun and Clarke, 2006, p. 84). We derived a codebook with five categories (knowledge, social context, governance, professional and complexity) and definitions from the theory of change in Pineo et al. (2019) and our conceptual approach. In most cases we inductively derived codes from the data to fit within these pre-determined categories to ensure we addressed our research questions from the perspective of our conceptual framework, however inductive categories were also developed (as described). Coding at the latent level produced codes such as ‘legitimising community priorities’ and ‘reluctance to engage with health, fear blockage’ (note: this code was a shorthand description for indicator producers’ perception that planners did not engage with health professionals due to fear that the purpose of such engagement would be to obstruct new development to avoid potential negative health impacts). Transcripts were coded in NVivo qualitative data analysis software (QSR International Pty Ltd., version 11.4.3, 2017).

Theme development involved iteratively reviewing the coded data, developing mind maps (in Nvivo) and mapping codes across emerging themes in an Excel spreadsheet. This process was undertaken in several stages following Braun and Clarke (2006) and Nowell et al. (2017). In the supplementary material (section 1.2) we describe how we developed candidate themes and defined the final themes. Our hybrid process of deductive/inductive coding and theme development involved the researchers selecting and presenting moments in the data that we interpreted as meaningful for our conceptual framework. We therefore recognise our position in influencing what is presented in the results.

We conceptualised our causal loop diagram (CLD) using the results of our systematic review (Supplementary material, section 1.3). We developed the CLD using interview data following a modified causal mapping process based on categorisation of themes in coding trees and identification of causal relations (Eker and Zimmermann, 2016), and transformation of text into words-and-arrow diagrams (Kim and Andersen, 2012). We recorded whether cause and effect relations were shared across the two interview groups (producers and users). Divergent thinking was represented through colour coding in detailed sections of the final model. We went through an iterative process of combining overlapping variables and re-drawing sections of the model to increase clarity (Sterman, 2000).

The preliminary high-level model and a selection of detailed model sections were tested and refined through a participatory workshop in San Francisco in January 2019. Financial constraints prohibited conducting workshops in Australia. The half-day workshop in San Francisco aimed to elicit feedback on: (1) the overall usefulness and credibility of the model as a representation of the participants’ mental models; (2) the plausibility of the variables, interconnections and feedback relations; and (3) opportunities for interventions to improve policy implementation, community involvement and UHI tool use. The workshop agenda and process for analysing workshop data are described in the supplementary material (section 1.4). We reviewed suggested changes to the preliminary CLD in relation to the interview data. We only made changes where interview data supported these adjustments (in other words, where the mental models of interview and workshop participants were overlapping). We took this approach for two reasons: (1) workshop participants’ views were not representative of the wider group and the CLD sought to describe shared mental models and (2) we sought to avoid adding causal links in order to maintain the simplicity of the overall high-level model. Changes that were supported across data from workshop and interview participants were adopted.

Results

We present each of the five themes (and sub-themes) that emerged from the thematic analysis in Table 2 and relate these to the high-level CLD (Fig. 2). Each of these themes is described in a sub-section of the results with illustrative interview quotes and reference to the CLD in Fig. 2 and more detailed CLDs. The supplementary material contains: additional detailed sections of the CLD (section 1.6); a glossary of CLD variable names (section 1.7), and additional feedback from the modelling workshop (section 1.8). Box 1 contains an introduction to reading CLDs.

Table 2 Theme names and a brief description following the thematic analysis.
Full size table

Overall story of indicator production and use

Before describing the five themes individually, we introduce the main causal loop diagram and the overall story of indicator production and use in the interview settings. Figure 2 shows the interconnections and feedback relations involved with UHI tool production and use described by study participants. Beginning in the bottom left, indicator producers created UHI tools to fill a perceived knowledge gap. In some cases this is effective (loop B1), however it also increases the overall number of indicators available, which creates confusion and reduces the ‘Perceived relevance and authority of UHI tools’ thus reducing their perceived value and use (loop R1). In essence, the growing number of UHI tools may not lead to increased indicator use among policy-makers because they may undermine the perceived relevance and authority of such tools. Inter-sectoral relationships were created through the development and application of UHI tools and this increased advocacy for urban health and health-promoting policy, which increased the ‘Perceived value of UHI tools’ (loops R4 and R5). Competition among policy objectives meant that health-promoting policy was not always implemented, however this policy resistance could be disrupted through UHI tools and community involvement in UHI tools (loops R3, R4, R7, R8 and B3). These dynamics are further described in the remainder of this section.

A key context point relates to the social issues that indicator producers and users described in their cities (specifically San Francisco, Melbourne and Sydney) as contributing to the motivation for UHI tool development (loop B1). Participants described significant population growth coupled with housing affordability crises, resulting in ‘winners and losers’and ‘disenfranchised’ communities. The changing urban context resulting from an influx of high-income residents and the widening of wealth inequalities was described by one participant as: ‘the city is starting to lose its soul’. Understanding and addressing inequities through urban planning was frequently mentioned as a reason for producing and using UHI tools. Although there were many similarities across the interview settings, two topics can be differentiated. In San Francisco, participants described a strong level of community opposition to new development or regeneration. In Australia, participants spoke frequently about tensions and power dynamics between state and local government.

Box 1—How to read a causal loop diagram

Causal loop diagrams are composed of variables, which are linked by arrows, or ‘causal links’ representing cause and effect relations (Sterman, 2000, p. 138). Causal links represent the direction of change between variables, noted through polarity symbols ( + /–) next to the arrow. Sterman (2000) emphasises that causal link polarity represents the structure of a system, not the behaviour of individual variables for two reasons. First, variables are not connected in isolation, but they are linked to multiple interconnected variables within the system. A positive link indicates that change to the independent variable results in change to the dependent variable in the same direction (either increasing or decreasing from where it would have otherwise been). A negative link, in turn, indicates that change to the independent variable will result in change in the opposite direction to the dependent variable. Each change is relative to what it would have otherwise been, if everything else had remained constant, which is unlikely. Second, CLDs do not identify and differentiate between stocks and flows within a system, defined as ‘the accumulations of resource in a system and the rates of change that alter those resources’ (ibid, p. 140), which can cause misinterpretation. Important feedback loops are labelled as either reinforcing or balancing (denoted with an arrow in the direction of the causal loop, clockwise or counter-clockwise, and the letter ‘R’ or ‘B’ and numbered in large CLDs). The previously described terminology is evident in Fig. 1. There are three variables: problem, fix and unintended consequence. A ‘problem’ causes a ‘fix’, which then reduces the ‘problem’ in the top balancing loop. However, the ‘fix’ also causes an ‘unintended consequence’ over time (the delay is marked with the double lines crossing the arrow between variables), which then increases the ‘problem’. This is part of a reinforcing feedback loop, which could be described in lay terms here as a vicious cycle. This is called a ‘fixes that fail’ systems archetype (Senge, 2006). Systems archetypes are reoccurring patterns of causal structure that ‘embody the key to learning to see structures’ (ibid; p. 93), of which Senge provides ten examples that are widely referenced in the system dynamics literature.

Fig. 1
figure 1

Causal loop diagram showing a ‘fixes that fail’ archetype (Senge, 2006).

Full size image

Advocacy through relationships

Participants’ emphasised the importance of building and maintaining relationships. Indicator producers cultivated relationships with a range of non-health stakeholders as a means of health advocacy via UHI tool development and use. These relationships built a network of advocates for health promotion across the public and private sector (Fig. 2, loops R4 and R5).

Fig. 2
figure 2

High-level CLD of UHI tool use and value, as perceived by indicator producers and users (workshop modification in red).

Full size image

The main ways in which UHI tools were used to create relationships included: collaboratively building UHI tools; meetings and correspondence between indicator producers and users about indicator results; and training sessions, conferences and events run by indicator producers (Fig. 3). These activities occurred over multiple years and facilitated ongoing communication between producers and users. In several cases, relationship building was deemed so essential that it formed part of the job description for staff involved in indicator development and management.

Fig. 3
figure 3

Detailed view of the causes and effects of inter-sectoral relationships (arrow colours denote diverse participants’ perspectives: orange for producers, green for users, blue for both and grey for the researcher).

Full size image

The benefits of building relationships were perceived to be multiple and they grew over time. Some relationships resulted in partner organisations (or individuals) becoming advocates for the messages, which the UHI tool sought to communicate. One indicator producer said that ‘engaging with industry partners is so incredibly important, because it’s not the indicator… that’s going to make the difference. It’s all the people. …because the amplification is really the critical thing.’

Building relationships with non-health actors created opportunities to share knowledge about how urban environments impact health and how this can be addressed through policy and decision-making. This included opportunities to learn about each other’s policy-related opportunities and constraints. An indicator producer explained that there were ‘inherent misunderstanding[s]… Because a lot of these city departments are worried about their infrastructure project not getting developed, or about these other consequences of what our [indicator] research might be when our research isn’t trying to advocate to not doing any infrastructure, it’s more just like what [are] the public health impacts.’

Other related benefits of relationships described by participants included: building trust and credibility; supporting funding applications; creating a receptive audience for knowledge about urban health; creating positive relationships in the contentious planning environment; and spreading ownership of UHI tools across multiple organisations. Furthermore, indicator producers recognised that UHI tools were an important tool for communication and described indicators as ‘a launching pad for engagement’ and a ‘lubricant’ to talk about research findings.

Re-framing knowledge to change norms

UHI tools and indicator producers’ advocacy and relationship-building activities re-framed knowledge about urban health issues and challenged urban planning professional norms and ways of working (Fig. 2, Loops R6 and connected to B3). These efforts affected urban planners and wider stakeholders, including the community, politicians and developers. By re-framing knowledge, indicator projects did not necessarily provide users with new knowledge, rather they provided a different way of looking at a problem, showed ‘interconnections’ or a ‘fuller picture’ and may have ‘expand[ed]’ knowledge.

The term ‘frame’ was mentioned in multiple examples in the interview data. First, participants said that concepts about the urban environment’s impact on health had been around for a long time, but were previously framed differently, such as under the concept of sustainability. UHI tools and related advocacy activities were seen as shifting, or re-framing, the agenda from sustainability to health, wellbeing and liveability, but covering familiar planning concepts such as pedestrian-friendly neighbourhoods and open space. Second, framing was also used to describe problem-solving activities through indicators, ‘to change the problem frame’. An indicator producer said that the ‘indicator fills an information gap that moves the solution forward’. They gave an example about understanding the causes of pedestrian fatalities in the city, a high priority for policy-makers. City government perceived that ‘the problem frame was…behavioural…’ but the indicator producers knew it was environmental, based on traffic design, volume and speed. They had unsuccessfully tried to communicate this information via epidemiological research ‘but that wasn’t useful evidence’. They then developed an indicator and ‘it became a manageable problem, so 5% of the streets accounted for 55% of the serious and fatal injuries. (…) almost immediately city policy shifted from focusing on residential neighbourhood traffic calming, to …arterial traffic calming.’ The idea that single indicators are powerful communication tools, which can change a problem frame to motivate action was a sentiment shared by some indicator users. They described ‘numbers’ as useful for communicating with politicians and also being used by politicians to rally effort and focus attention: ‘we need to simplify things for people, but it’s such a complex issue. And the simplification is really helpful for political action…’

Indicators also re-framed knowledge by incorporating community views, thereby broadening the scope of knowledge that informs planning. Community knowledge represented in indicators was portrayed as powerful evidence to argue for different ways of working in terms of planning policy and decision-making, challenging existing norms. One indicator user said ‘planning is thinking a lot more about health than it ever did before the [UHI tool]. In part because [the indicator producers] were very persistent, and they said, ‘You have to look at this, this is what the community is demanding.’ However, there were limits to how far interview participants thought people may expand their knowledge as a result of UHI tools. Some producers perceived users as ‘programmatic in their work’, not interested in wider issues or time-poor and not able to engage with the UHI tool.

Representing community interests and inequity

Using UHI tools to gather community perceptions and increase awareness and action about disadvantaged communities’ challenges was a core driver for many indicator producers and users (Fig. 2, Loops B1, B3 and R8). This was primarily achieved in two ways: (1) publishing indicator data at a spatial scale allowing neighbourhood comparison, and (2) involving disadvantaged communities in selecting indicators. In the context of disadvantaged communities, UHI tools were used to: challenge powerful interests (such as developers), tell difficult truths to politicians or community representatives, legitimise community concerns within the planning system and mitigate the impact of new development on these communities. Examples related to the housing affordability impact of neighbourhood improvements and the location of alcohol or gambling outlets in poor neighbourhoods.

A commonly discussed problem related to rapidly growing areas, where planners sought to attract private sector developers to build houses and infrastructure, such as supermarkets. In this scenario planners were ‘a bit beholden to’ commercial interests but tried to negotiate for more pedestrian-friendly design or relocation of alcohol outlets away from deprived communities. They were met with responses from the developers such as ‘well we won’t build your supermarket then.’ In this example and others, actors brought health and environment data together to advocate on behalf of disadvantaged communities that may not have had the power to otherwise resist undesirable or health-harming developments, further demonstrating the value of cross-sector relationships.

Disrupting problematic constraints to healthy planning

Indicator producers and users deployed UHI tools in multiple ways to disrupt a wide range of constraints inherent to the planning system and its function within urban contexts, described in three sub-themes: urban economics, land-use politics and legal framework governing urban planning (Fig. 4). However, participants frequently described setbacks, and found their work applying indicators as ‘very tiring’, ‘frustrating’ and needing ‘courage’.

Fig. 4
figure 4

Detailed view of the unintended consequences and policy resistance mechanisms caused by health-promoting policy (arrow colours denote diverse participants’ perspectives: orange for producers, green for users, blue for both and grey for the researcher).

Full size image

Urban economics

Urban and development economics affected many aspects of planning policy, decisions and implementation (Fig. 4, loops Rc, Rd, Re). UHI tools were used to challenge economic arguments against designing and building health-promoting environments. Poor understanding of economics was a key challenge raised by some indicator users, yet was rarely discussed by indicator producers. Evidence from UHIs was used to increase health-related design and planning through negotiation with developers, strengthening arguments for policies, and determining how development impact feesFootnote 1 should be allocated.

Planners described applying indicators as ‘leverage’ during negotiations with developers, yet the latter were frequently described as arguing that modifying design for health purposes would come at the expense of other policy objectives. For example, one indicator user described developers as saying ‘Oh, there’s all these things, they make it more expensive, and we’re trying to provide affordable housing.’ Indicators were seen to help counter developers’ economic arguments by providing evidence of ‘co-benefits’ (e.g., of green infrastructure supporting cooling and biodiversity). One UHI tool developed with significant community involvement was seen to be a powerful argument about the value of health within the economic and political context of planning decision-making: ‘The developer and the planning department (…) decided to mitigate and give all those residents right of return and rent control. (…) That was the result of the authority of health… the power of the value of health, the currency of health, with evidence, into the political process…’ Discussions about development economics on particular sites became part of wider economic and political issues of housing affordability.

Land-use politics

Land-use politics were viewed by some interview participants as a ‘blood sport’ controlled by specific powerful groups, inhibiting the creation of healthy urban environments. Some Australian participants spoke of a ‘a very contested, uneasy relationship’ between multiple tiers of government. Politicians were characterised as prioritising short-term gains over evidence and long-term community needs. However, politicians were also recognised as important drivers of change to support healthy urban environments. Overcoming challenges of land-use politics involved: publishing indicator evidence to directly disrupt or influence political processes, working with political decision-makers to influence change (building trusted relationships over time), and/or being ready to support politicians when opportunities arose to influence politicians and decision-makers.

Politicians could be important advocates for health promotion if the indicator producers involved them in the production of indicators (co-production). This involvement ensured that politicians felt ‘ownership’ of indicators. Where politicians or other influential indicator users felt ownership, they would drive change: ‘there has to be that kind of, let’s just say, the political engine behind the indicator, whether it’s a mayor or an empowered bureaucrat… or a citizen, (…) You have to create ownership of the user, and the best way to do that is to do it with them.’ Politicians or citizen demand related to indicators could ‘rally’ efforts, focus attention and drive cross-departmental working in a way that un-elected officials could not (see Fig. 5).

Fig. 5
figure 5

Detailed view of the causes and effects of the stability of UHI tools (arrow colours denote diverse participants’ perspectives: orange for producers, green for users, blue for both and grey for the researcher).

Full size image

Legal framework governing urban planning

The planning system was described as operating within a legal framework where some health-promoting policy and implementation options were unavailable, however indicators could disrupt such constraints (see Fig. 4, loop Re). Planning legal system challenges included: the risk of planners’ decisions being contested (resulting in costly legal fees), perceptions that adjudicators in the legal appeals system were ideologically driven and unaware or ‘hostile [toward] scientific method’ (i.e., dismissive of certain evidence claims), and mistaken perceptions that planning had significant power over health-related topics and land uses, such as stopping fast food outlets. In relation to the latter, one Australian planner said, ‘planning can’t do that’ because of how fast food outlets were classed in the planning scheme, which was ‘quite rigid’ and ‘controlled by the state government.’ However, evidence from UHI tools and related advocacy were used to influence legislation and local planning policy, as well as local development negotiations, when indicators and advocacy fit into the requirements of the planning legal system. Indicator users described indicators as powerful evidence to challenge ‘business as usual models’. Indicators were used as evidence to justify health-related policies in statutory plans. Interview participants attributed great value to indicators to move toward ‘evidence-based decisions’ but described many instances where factors other than evidence had driven planning decisions.

Avoiding failure through design

Indicator users and producers worked together to shape effective UHI tools, with recognition that indicators would only tell part of a story and would only be part of a bigger picture of health promotion activities (Fig. 2, Loops R1, R2, R9 and B1). Indicator producers’ strategies to create influential UHI tools were developed over time and improved through trial and error. Such strategies included: creating ownership among users; building relationships with indicator users and other advocates; focusing on problems that need to be solved, regardless of data availability; focusing on strategic indicators to drive policy change; presenting data in a captivating format (including maps) at the right spatial scale; building on the success of previous UHI tools where concepts have become institutionalised; and communicating with audiences in their language about their problems, avoiding terms like ‘health’ and ‘social determinants of health’ (which were seen as being beyond the remit of non-health practitioners). Indicator producers saw the powerful strategies of relationship building and advocacy as being in a mutually reinforcing relationship with the view of their UHI tool as being credible, scientifically valid and appropriately presented for their audience. Indicator users described the following shortcomings of some UHI tools: poor data quality and availability; too many indicators available, but not necessarily the right indicators; and fragility of UHI tools that could become unavailable at any time due to funding or political changes.

The concept of stable UHI tools emerged from the interviews in which participants described fragile tools (characterised by losing funding, being forgotten and not being supported by politicians) and stable tools, which had the financial and political support to enable long-term relationships and impact. Figure 5 shows that increased ‘Stability of UHI tools’ led to ‘Institutionalised UHI tools’, which increased ‘Inter-sectoral relationships’ and ‘Culture of health-promotion’. The role of politicians’ agendas, performance in the UHI tool, politicians’ fear of accountability and organisational memory are also connected to the stability of UHI tools (as previously described in ‘Land use politics’). Resources for UHI tools was crucial and linked to planners’ view that indicators were the ‘Cinderella of planning’ due to a lack of resources to monitor outcomes of the planning system.

Dealing with complexity through indicators

Participants described multiple ways in which indicators helped planning policy and decision-makers understand or address the complexity of urban health. In summary, indicators could show interconnections between urban health variables/domains, identify unintended consequences or policy resistance and allow actors to see problems in a different way. However, some participants identified risks related to indicators and complexity, such as using a single indicator devoid of contextual information to justify policies.

UHI tools were seen as a useful resource about urban health challenges, however participants were divided in opinion about whether these tools helped to show or explain interconnections among variables or domains. Furthermore, participants spoke differently about how UHI tools could help policy and decision-makers address interconnected urban health challenges. Indicator producers outlined specific UHI tool characteristics that highlighted interconnections, including: presenting a conceptual framework showing UHI tool domains and indicators, listing a number of indicators or domains to show the multiple urban environment issues that affect health, and reporting cause and effect relations among indicators. In contrast, indicator users described difficulty ‘drawing relationships’ between indicators and perceived urban health as a ‘messy web’ that UHI tools did not necessarily help them to ‘untangle.’ One indicator user wanted to see fewer indicators within specific UHI tools, with more focus on single indicators that represented interrelations among different systems.

Indicator producers and users relayed stories about the use of UHI tools to detect unintended consequences of policies or a lack of change in the system following policy implementation (i.e., policy resistance). One example was about a persistent problem with pedestrian injuries: ‘we’re doing very detailed analysis over what are the cause and effects and what are the right tools to address these things. We aren’t seeing the results that everyone wants us to see. That’s somewhere we’re really using this data and [these] indicators to really inform traffic engineering.’ Poorly devised indicators were perceived by some indicator users to drive unintended consequences when adopted through land-use policies, such as target-driven policies to control density and urban sprawl. One indicator producer explained that there were limited resources to improve health through the built environment, and therefore trade-offs between objectives were perhaps inevitable but could be monitored through indicators.

There were diverse views of indicators as either providing a ‘fuller picture’ or just being a starting point for further analysis. These views related to the risk of political attraction to simple figures as representations of complex problems. Indicators were seen by some participants as simplifying information to such an extent that important information was lost. This view highlighted a tension between the number of indicators presented in UHI tools and the ability of users to process and act on that information. One indicator user said: ‘you can’t just focus on certain things and expect everything else to fall into place. Indicators will show a particular issue from a particular angle that can be used to help inform your response to that. …but they aren’t the full picture. …it’s certainly helpful when used in the right way and not used to just support a certain contention in isolation…’ In summary, although single indicators could drive political action, this came at the risk of triggering inappropriate policy responses.

Participatory workshop feedback

The participatory workshop feedback was used to test the overall usefulness and credibility of the CLD model (as detailed in section ‘Data analysis and modelling’) Workshop participants said that the model was useful and it represented their general view of how UHI tools influence urban planning policy and decision-making. An indicator producer said that it was helpful to know more about how UHI tools influenced policy-making because this was not something their organisation understood well. One indicator user said that it was helpful to explore the policy implementation section of the model, specifically to identify possible interventions for improvement. There was substantial consensus about the interconnections and feedback relations within the model. Participants identified five additional causal relations and six missing topics; however, they were also in agreement that adding more causal links could reduce the comprehensibility of the model. Following the workshop, we introduced causal relations and variables as summarised in Table 3 (see further detail in Supplementary material, section 1.8).

Table 3 Summary of suggested changes to causal loop diagrams (CLDs) and specific modifications in the diagrams.
Full size table

A number of factors were discussed in the workshop that did not reflect interview discussions and therefore no changes were made to the model. Such factors included: the impact of power dynamics between different community groups and other urban actors and how some groups were ‘weaponising the data’ through UHI tools to achieve their aims; the additional layers of complexity that were not evidenced in the policy-making and policy implementation elements of the model; the effect of new data, media and technology on local governance; and the potential motivations for creating new UHI tools.

Discussion

Our investigation fills a knowledge gap in the literature and shows how indicators influence policy and implementation directly and indirectly through our causal loop diagram. Contrary to the typical portrayal of indicator use as part of a rational and linear policy-making process, we contend that UHI tool development and application are endogenous elements of the urban planning policy and decision-making system, in support of claims by Innes and Booher (2000), Pastille Consortium (2002), and Sébastien et al. (2014). By stating that UHI tools are endogenous in policy and decision-making processes we build on language by the Pastille Consortium (2002) who said that indicators ‘are not exogenous factors parachuted in’ to governance (p. 90). Endogenous explanations or theories use the variables within a closed CLD to account for behaviour (Sterman, 2000). Therefore we use our CLD to explain the effects of UHI tools within the interconnected processes of indicator creation and application, and policy development and implementation. The cases that we studied showed that UHI tools and indicator producers achieved impact by helping to: build relationships, drive advocacy, re-frame knowledge, increase awareness of disadvantaged communities’ interests, and disrupt problematic constraints to healthy planning, such as urban economics, land-use politics and the legal framework governing urban planning. These routes to influence were most evident in SFIP and CIV where long-term investment supported iterative indicator development and application, including direct use of indicators by policy and decision-makers. This section describes the novelty of our findings and interprets them in relation to collaborative rationality, systems thinking and wider literature. First we briefly describe the strengths and limitations of this study.

This research took place in three settings, linked by similar social and economic urban contexts and planning systems, increasing the transferability of the results to inform other UHI tool producers, users or researchers. Interview participants also had knowledge of a wide range of UHI tools, beyond SFIP and CIV. This breadth of knowledge was complemented by in-depth knowledge, gained over a decade, about the development and application of the SFIP and CIV tools. Analysis of this detailed knowledge within the study’s theoretical framework, combining collaborative rationality and systems theory, has allowed the generation of useful theory-based insights that may be applicable beyond the cases examined.

Lack of generalisability is an inherent limitation of case study research (Bryman, 2004), although this research has addressed multiple UHI tools and settings. The significant and long-term resources behind SFIP and CIV mean that they are not necessarily representative of other UHI tools with regard to their use by policy and decision-makers. Investigating the relationship between UHI tools and complexity was challenging and the direct questions in the interview may have influenced participants’ responses, although we attempted to overcome this by asking about complexity in multiple, more indirect, ways.

Embedded UHI tools and governance

We argue that embedded UHI tools, meaning ‘stable’ and ‘institutionalised’ in our study, were directly used in policy and influenced policy and governance networks. Sébastien, Bauler and Lehtonen (2014) outlined three types of indicator use and influence (instrumental, conceptual and political), describing ‘use’ as the ‘concrete act of handling the indicators in a policy context’ and ‘influence’ as the ‘impact on policy processes at any moment in the indicator production and utilisation chain’ (p. 320). We see the differentiation of ‘instrumental’ from ‘conceptual’ and ‘political’ use and influence as relating to diverse conceptualisations of knowledge and evidence use in policy-making as either: (1) rational and linear or (2) socially constructed and complex (Pineo et al., 2018b). The knowledge translation and evidence-based policy and indicator use literatures broadly reject a rational and linear model of evidence use by policy-makers. We believe that this rejection has led researchers of indicator use to claim that indicators are not instrumentally used but can have conceptual and political influence on governance processes, exemplified by Sébastien, Bauler and Lehtonen’s (2014) statement that indicators ‘failed as direct input to policy-making’ (p. 316).

We identified a diversity of perspectives, that are not mutually exclusive, about the use and influence of UHI tools and evidence in governance. Urban planners identified very specific cases where indicators were directly (i.e., instrumentally) used in policy or negotiations with developers. However, a more diffuse level of impact was evident through the relationships, advocacy, knowledge re-framing and other benefits of UHI tools outlined above. This reinforces Innes’s (1998) claim that as ‘information becomes gradually embedded’ it alters actors’ mental models and this achievement is ‘even more significant than that attributed to it by the rational instrumental model of public choice; it is also far more difficult to isolate and describe’ (p. 53) We identify all three types of Sébastien and colleagues’ (2014) indicator use and influence of UHI tools. We argue that instrumental indicator use and influence may have been possible due to the embedded nature of the UHI tools we studied. We perceive conceptual and political influence to be more impactful in promoting health through urban planning over time. All three types of indicator use and influence emerged endogenously as a result of a UHI tool’s influence on the wider system in which it operates. The value of our CLD is precisely in isolating and describing in detail how information influenced governance.

Embedded UHI tools and complexity

Embedded UHI tools (stable and institutionalised) exhibit many qualities to support actors with the complexity of urban health policy-making. First, stable UHI tools (with long-term funding and political support) allow indicator producers and users to interact and react to each other’s knowledge and needs over time, resulting in a number of reinforcing feedback loops that further embed UHI tool knowledge across multiple sectors. Second, we contend that embedded UHI tools involve processes that are comparable to Innes and Booher (2010) collaboratively rational processes and this helps to explain their value for addressing complexity. We found that SFIP and CIV met most of the three conditions of collaboratively rational processes: diversity of interests among participants, interdependence of actors (who could not independently have their needs met), and authentic dialogue that adheres to Habermas’ (1984) ideal speech conditions (Innes and Booher, 2010; p. 35). Such processes result in actors learning about the ‘reciprocal nature of their interests’ and ‘new means to achieve their interests’, among other benefits (ibid; p. 37). This allows diverse actors to link across networks and multiple sub-systems within the complex urban health system, learning about and testing policy solutions over time.

Finally, other characteristics of embedded UHI tools support indicator users to address challenges associated with the complexity of urban health and policy processes, such as: (1) allowing longitudinal assessment of how interconnected factors change over time addressing the counterintuitive and dynamic nature of the complex urban health system and (2) supporting policy-makers with the identification of unintended consequences and policy resistance as these effects emerge over time. These benefits align with the approaches for managing complex urban health systems such as: adaptive management, co-producing knowledge, participatory governance, integrated planning, and the use of monitoring systems (Corburn and Cohen, 2012; Rydin et al., 2012; Corburn, 2013, 2015; Gatzweiler et al., 2018).

Re-framing and embedding diverse knowledge types across institutions

UHI tools were used to re-frame knowledge about the social determinants of health to challenge institutional and professional norms and promote health through urban planning. Local institutions and governance arrangements were important in determining how indicators were perceived and used. Indicator producers developed an understanding of these factors over time through building trusted relationships during the development and application of UHI tools (as shown in Figs. 2 and 3). Using this new understanding, producers were able to modify their strategies to influence policy-making through UHI tool data or otherwise. Our causal model shows how the development of inter-sectoral relationships helped actors to understand and bridge across diverse institutional cultures, norms and values to promote health through urban planning.

We found that UHI tools represented and supported multi-directional sharing of diverse types of knowledge, including scientific and community knowledge through ongoing interaction among indicator producers and users. Scientific knowledge is translated to a wide range of actors through UHI tool projects. Community knowledge is also represented in UHI tool development and application that could result in urban planning policies and decisions which better respond to diverse communities’ health and place needs. The Pastille Consortium (2002) and Innes and Booher (2010) called this type of knowledge ‘experiential’ and ‘lay’ respectively and they described it as: (1) essential for addressing complex sustainability challenges and (2) potentially powerful in influencing policy-makers.

Legitimating policy positions through advocacy

Multiple actors used UHI tools as a means of legitimating policy positions. Policy legitimation is the justification of actions, decisions and behaviour within institutions to some widely agreed premises by referencing an agreed ‘norm, value or routine’ (Pastille Consortium, 2002, p. 60). The Pastille Consortium identified legitimation occurring at both policy agenda (national or international) and policy network levels (among policy actors). Similarly, Sébastien, Bauler and Lehtonen (2014) found that sustainable development indicators were ‘used politically as ammunition in the efforts of policy actors to legitimise their positions, worldviews, and visions of sustainability’ (p. 325). We found multiple examples where UHI tools were employed as part of an argument to legitimate policy positions, particularly those that went against the status quo, at policy agenda and network levels (encapsulated in the ‘Ability to challenge business as usual’ variable in Fig. 2).

At the policy agenda level, UHI tool producers successfully shifted the discourse about sustainable development, inserting a health lens where it had not previously existed at city, state or national levels. In one case this involved advocates in multiple organisations pushing the message of the UHI tool to raise awareness of the impact of urban built environment policy on health, wellbeing and liveability. This occurred over a period of at least ten years, highlighting the value of embedded UHI tools because the message of the UHI tool became more accepted by policy actors over time. As Innes and Booher (2000) said ‘[a]s the ideas become taken for granted they have their strongest impact on action’ (p. 177).

At the policy network level, the impact of inserting health into the urban planning process for specific developments was achieved through case-by-case use of the UHI tool. This involved policy actors and other stakeholders (such as community representatives) negotiating UHI tool application. There were debates about how the results of particular indicator analyses should be interpreted and translated into design requirements for new development. Community representatives relied on their perception that indicators would have an elevated status in the decision-making process in comparison to their own views, which was reinforced by the individuals (indicator producers) helping them to apply indicators. Again, the embeddedness of UHI tools supports this type of policy legitimation as the weight of indicators in the decision-making process could increase over time as knowledge about health and the built environment increases. However, not all stakeholders perceived health evidence as having the elevated status placed on it by some study participants. Thus, the legitimation of planning-related decisions on these grounds may change over time as actors reassess the role of health, wellbeing and liveability in wider sustainable development discourses.

Conclusion

Our findings have implications for indicator users and producers who seek to promote health through urban planning, and specifically through the use of UHI tools. We have explained how UHI tools and indicator producers achieved health promotion through urban planning by: building relationships, supporting advocacy, re-framing knowledge, increasing awareness of disadvantaged communities’ interests, and disrupting problematic constraints to healthy planning, such as urban economics, land-use politics and the legal framework governing urban planning. In conclusion, we highlight two challenges for research and implementation, followed by future research directions.

First, we consider the relevance of claims from the sustainability indicator literature about the relative imbalance in research about designing indicators versus exploring their role in the policy process (e.g., Pastille Consortium, 2002; Innes and Booher, 2000). Although we agree that this imbalance exists for UHI tools as well (Pineo et al., 2018a, 2019), continued research on technical urban health indicator development is warranted for four reasons: (1) indicators may influence policy outside of collaborative processes, (2) indicator users and producers may have different perceptions about the important characteristics of indicators to influence policy, (3) indicator data is likely to under-represent the needs of certain population groups (e.g., women and low-income communities) (Cicerchia, 1996; Parnell and Poyser, 2002; Buvinic and Levine, 2016; Criado-Perez, 2019) and (4) there may be unintended consequences of oversimplifying complex issues through indicators (Decoville, 2018). Second, we found that community involvement in UHI tool development and application could result in urban planning policies and decisions that genuinely respond to diverse communities’ needs. However, managing the expectations of diverse community representatives is a significant undertaking and will require diverse skills and funding than those which are usually required for the analysis of indicator data.

This research highlighted several future research directions for the study of indicators and policy. The supposition that indicators will necessarily drive policy action has been widely challenged and this concern is further validated by our systematic review (Pineo et al., 2019) and the findings reported here. This raises questions about the effectiveness of monitoring systems linked to the UN Sustainable Development Goals, a problem previously articulated by Acuto and Parnell (2016). They argue that the SDG programme of tracking progress via indicators ‘is far ahead of the science-policy capability on the ground’ and consensus is needed about who would set and monitor metrics in a manner agreeable to all parties. Furthermore, focusing attention on measurable indicators may divert attention from other important sustainability and health issues that were either not identified by indicator producers or were not feasible to measure (Pastille Consortium, 2002; Innes and Booher, 2010).

Future research could further explore how indicators shape policy debates and which factors inhibit the measurement and inclusion of locally important issues. This links to the power of different actors within health and planning governance processes. Workshop participants spoke of how some groups in San Francisco are ‘weaponising’ data to fight particular policy positions. Similarly, Sébastien, Bauler and Lehtonen (2014) spoke of indicators being used as ‘ammunition’ in political debates (p. 334). Use of indicators to legitimate policy positions has been explored in relation to the Slum Dwellers International ‘Know your city’ project that resulted in community-derived data and knowledge that is used to stake a claim in the global urban agenda (Cociña et al., 2019). Further exploration of such power dynamics and how UHI tools mediate or contribute to which actors have power within urban planning processes would help uncover areas of leverage to create healthier urban environments. Finally, our research raised questions for future research about the risk of having too few indicators and the underlying motivations of indicator producers.