Social and behavioural factors are critical to the emergence, spread and containment of human disease, and are key determinants of the course, duration and outcomes of disease outbreaks. Recent epidemics of Ebola in West Africa and coronavirus disease 2019 (COVID-19) globally have reinforced the importance of developing infectious disease models that better integrate social and behavioural dynamics and theories. Meanwhile, the growth in capacity, coordination and prioritization of social science research and of risk communication and community engagement (RCCE) practice within the current pandemic response provides an opportunity for collaboration among epidemiological modellers, social scientists and RCCE practitioners towards a mutually beneficial research and practice agenda. Here, we provide a review of the current modelling methodologies and describe the challenges and opportunities for integrating them with social science research and RCCE practice. Finally, we set out an agenda for advancing transdisciplinary collaboration for integrated disease modelling and for more robust policy and practice for reducing disease transmission.
Social and behavioural factors are critical to the emergence, spread and containment of human disease and are key determinants of the course, duration and outcomes of outbreaks. The feasibility and acceptability of adopting recommended health behaviours are intertwined with social, economic, environmental and political complexities that affect individuals, communities and societies1,2,3,4. Individuals and communities have vital roles in reducing transmission during outbreak response5 by maintaining preventive behaviours, and actively contributing to response design, implementation and monitoring. Communities and individuals interpret and implement behavioural and other policy recommendations based on factors including their understanding and perception of the disease threat6, their level of trust in governing authorities and other information sources, and their physical, financial and social capacity to voluntarily take action7. The design, availability and accessibility of health-related services are bidirectional factors that can increase or reduce the demand for services, in both emergency contexts and routine healthcare settings8,9,10. Despite its importance, individual and community action during outbreaks as well as the factors that influence it are often considered to be distinct from epidemiological trends and biomedical interventions. Such perspectives also pervade the design of disease models.
The field of disease modelling has a considerable and growing influence on vital questions related to public health policy11. Large-scale epidemics of new pathogens and global health threats12,13,14 (for example, pandemic influenza15, Zika16, Ebola17,18 and COVID-19 (ref. 19)) have underscored the importance of developing models that integrate social and behavioural dynamics and better reflect the realities of affected communities; however, such experiences also highlight substantial challenges20. Better incorporation of social and behavioural factors into disease models will probably improve their predictive accuracy and thereby inform more effective response measures and policies21,22. Despite the clear need and potential opportunity, progress towards more integrated disease modelling has been slow.
The experience of recent outbreaks has also led to a growing appreciation of, and an investment in, social science research as well as risk communication and community engagement (RCCE) interventions as part of response efforts23,24,25. Since the 2014–2016 Ebola outbreak in West Africa, there has been a concerted effort to incorporate social science research and RCCE approaches into traditional response pillars (for example, case management and surveillance)26,27 through the formation of multidisciplinary response teams28. Social science research offers vital sources of social and behavioural data, including quantitative and qualitative data. Similarly, as a central pillar of outbreak response, RCCE interventions can generate real-time reporting of community surveillance data; ongoing qualitative feedback on response measures; and spatial and temporal data for evaluating response interventions. The growth in priority and coordination of social science research and RCCE interventions offers renewed opportunities for collaboration to improve disease modelling.
A review of epidemiological and social science research from the West Africa Ebola outbreak illustrated the comparative advantages of each discipline and the key areas for improvement in terms of collaboration and research design23. The review confirmed that disciplinary approaches drove and defined research priorities—epidemiological research focused on narrow interpretations of transmission dynamics, associations and outcomes, whereas social science research focused on factors that influence the access, availability and use of services including gender, social roles, vulnerabilities, food insecurity and mobility. Although acknowledged by both epidemiology and social science disciplines, some key public health priorities around social and behavioural factors, such as the roles of funeral practices and community attitudes to disease response, were difficult to adequately quantify in models. This resulted in diverging conclusions regarding the mechanisms that drive transmission and may have undermined decision-making about how to prioritize resources to optimally avert cases. The review also found that epidemiological research drew on broad population (for example, demographic and socioeconomic) data to infer local-level insights; by contrast, social and behavioural research (for example, anthropological studies using individual-level data) often inferred general findings from very small sample sizes. This highlights the trade-offs between local and global information availability29. Such divergence in methodological approach and theory reveals biases that are inherent in research priorities and applications that arguably hinder outbreak response decision-making.
To advance a mutually beneficial research and practice agenda, social scientists, RCCE practitioners and disease modellers will require a common framework for understanding and interpreting social, behavioural and operational data and theory. This means that, for modellers, a key goal is to use complex information to develop parsimonious, integrated models that are better designed to capture the factors that underlie observed disease trends compared with models that use traditional sources of epidemiological data alone. For social scientists and RCCE practitioners, more explicit, data-informed modelling of social and behavioural processes (and the extrinsic and intrinsic drivers influencing them) will generate quantitative evidence on the roles and importance of social learning30 and community action, as well as the impacts of RCCE and other non-pharmaceutical interventions.
Modelling social and behavioural factors
Mathematical models have a long history of influencing the scientific understanding of disease epidemiology. Classical disease modelling is based on coupled ordinary differential equations and assumes homogeneous mixing of individuals within a population. This approach has been used for nearly a century, beginning with Kermack, McKendrick, and Walker in 192731. In this simple framework, individuals are in one of the susceptible–infected–recovered states. Importantly, they are assumed not to adapt any behaviour throughout an outbreak31. Instead, the dynamics of the model are fully specified at the population level by the number or proportion of individuals in each disease state. New infections occur with a probability of the density of infectious and susceptible individuals and a per-contact probability of transmission. As such, the classical well-mixed models make the very strong behavioural assumption that contact behaviour is invariant to the prevalence of an infectious disease, analogous to an ideal gas mixing indiscriminately in a chamber. Although this strong assumption is true for some diseases (for example, the common cold) or situations (for example, a boy’s boarding school32), the history of major epidemics demonstrates that microscale interactions between individuals are important. In practice, observed interactions reflect a range of social, cultural, political, economic and behavioural shifts over the course of a disease; they include both collective and individual behaviours and, most critically, they are dynamic and variable. As such, many commonly used disease modelling approaches (for example, statistical curve fitting models and population-level, compartmental susceptible–infected–recovered models) tend to lack sufficient consideration and mechanistic detail relating to how behaviours mediate or exacerbate transmission, the social nature of behaviour during epidemics, the role of community action and the contextual underpinnings that drive heterogeneity in human action33,34,35.
Systematic reviews by Funk et al.33, Verelst et al.36, Weston et al.37 and Gersovitz38 considered a broad range of existing disease modelling efforts that account for human behaviour. These reviews emphasize the value of attempts to include social and behavioural parameters in modelling. However, their findings suggest that most model assumptions reflected a limited understanding of the interplay between social and behavioural factors on disease transmission and inconsistently used existing social and behavioural theory and data. These reviews highlight a tendency of models to focus on the economic literature addressing behaviour, or theories adapted from previous modelling papers, rather than theories from psychology or sociology37. The use of operational response data is not considered. The models reviewed tended to be theoretical and/or lacked validation against known phenomena or observed data and were therefore unlikely to accurately reflect the complexity and heterogeneity of sociobehavioural drivers and dynamics33.
Nonetheless, existing modelling approaches that attempt to integrate human behaviour offer opportunities to build on methodologies and insights (Table 1). The following section reviews four such modelling approaches.
Economic epidemiologists capture behavioural influence in observed disease dynamics, and vice versa, by introducing the notion of prevalence-elastic behaviour into epidemic models22,39. The concept of prevalence-elastic behaviour is used to quantify how population-level infection rates and personal infection status influence the adoption of behavioural recommendations, such as vaccination22,40,41 and social distancing42,43. For example, as the prevalence of sexually transmitted infections grows in a community, people may increase demand for condoms, as occurred during AIDS epidemics44,45; similarly, proximity to disease can increase vaccine acceptance46. Economic epidemiologists model how rational individuals (by literal definition) would behave given some level of disease prevalence. They behave as Homo economicus would behave given the associated health risks and costs of protection (for example, isolating and missing work as a result)47. The resulting dynamics, termed rational epidemics48, capture a clearly important phenomenon: at points during an epidemic at which risk of infection is high, the cost–benefit trade-off shifts in favour of preventative behaviour adoption49. By contrast, when infection rates are low, there is less incentive for individuals to adopt health behaviours. Importantly, models that incorporate such prevalence-elastic individual behaviour mechanisms can explain subexponential growth dynamics, including epidemics that exhibit stationary disease dynamics when the susceptible population is large50.
Such principles have implications for control strategies in terms of timing and options. Explicit consideration of trade-offs that drive individual decision-making around behaviour adoption enables the evaluation of subsidies, penalties and incentives as interventions39,40. Applications of economic epidemiology have been used to investigate trends in the voluntary avoidance of air travel during epidemics of swine flu or A/H1N1 influenza among ticketed passengers51,52. They have also been used to consider the economic drivers of transmission-enhancing behaviours in emerging zoonotic disease systems, as decisions about movement in areas with high interaction with livestock or wildlife often reflect people’s consideration of their economic livelihoods53. Finally, economic epidemiology models have been used to examine potential externalities from health interventions. For example, measles vaccination can lead to positive externalities through herd immunity (that is, population-level vaccination can prevent measles infection even in those who have not received the vaccine), whereas negative externalities through the overuse of antibiotics can lead to resistance (that is, individuals not receiving antibiotics can be infected with resistant bacteria and experience treatment failure from the overuse of antibiotics in other individuals)40.
Although economic epidemiology offers modelling methods to account for decision-making, we note that human behaviour does not always conform to the canonical definition of rational decision-making whereby agents optimize individual benefit or utility. Affective (and not necessarily conscious) factors, including a range of emotions such as fear and anger, coupled with pervasive errors in appraising risks (including base-rate neglect, framing effects, endowment effects, loss aversion, availability bias and anchoring), all compounded by network conformity effects, can combine to produce behaviours that are far from canonically rational, with far-from-optimal results.
Behavioural change as network dynamics
Modelling disease spread in contact networks provides a representation of the heterogeneity and complexity of human behaviour in the form of the network, or graph, in which an epidemic can occur54,55,56,57. Top-down interventions affecting contact patterns can be modelled as changes to the network structure57, whereas bottom-up behavioural changes are often modelled by coupling an epidemic with the spread of preventative information or behaviours related to the disease33. Interventions in this context can be any piece of information (for example, a vaccination campaign or information about transmission risks) that can reduce the risks of transmission. For top-down approaches, the objective of the model is to inform the best targeting strategies, which can be optimized to find highly connected individuals58, disconnect social groups59,60 or be robust to data quality61. Bottom-up approaches tend to be more descriptive, often modelled as extensions of classic disease models, assuming random contact within populations and letting disease and information spread and interact33,62.
Generalizations exist to account for network structure63,64, correlations between physical and information networks65, the importance of timing65,66 and many versions on multilayer networks67,68,69,70,71. However, these efforts are largely theoretical as data informing the complexities of human physical and communication networks (and their overlap) are scarce and difficult to collect. Importantly, such efforts also tend to assume a static structure with fixed behaviour, knowledge and contact patterns. Using models of adaptive networks (in which connections between individuals change), one can also study behavioural change as the spread of a pathogen (that is, the dynamics on the network) and of the network structure itself (that is, the dynamics of the network)72,73,74. Often, individuals in an adaptive network might simply cut or interrupt their contacts with individuals who are infectious75,76 or reduce the rate of transmission through preventive measures62,77,78. Beyond this, individuals can also alter the structure of the network itself, replacing infectious contacts—temporarily or permanently—to protect themselves79,80,81. Such models can produce rich dynamics, including increased vaccine effectiveness82, epidemic cycles83, accelerated spread84 and unstable fluctuations in the model85. Although learning adaptive behaviour of a population on the basis of empirical data has been considered86, little is known about the social learning behind the adaptive behaviour itself87,88. It becomes important to know how to include adaptation as an endogenous behavioural change rather than as an exogenous set of rules. Social science contributions can be impactful here by providing empirically derived insights regarding behavioural change, adaptation, the limits on adaptation and the timeframes over which adaptation occurs in an outbreak scenario7.
Coupled contagion models with an extension to Agent_Zero
Coupled contagion modelling provides insights into the role of emotions, namely fear, in epidemic dynamics77. Specifically, fear is modeled as a contagion that influences behavioural decisions, which in turn impact disease transmission89. In contrast to the prevalence-elastic behaviour that is characteristic of economic epidemiology models, fear can spread independently from a local prevalence signal (through behavioural contagion), and behavioural responses to fear are not limited to reduced contact (for example, fleeing can increase contact). The essential dynamic is that high disease prevalence induces fear, which in turn produces protective behaviours, such as self-isolation. This distancing suppresses disease spread. However, with reduced incidence, the fear evaporates, and people return to their activities despite the presence of individuals who are infectious in the community. This results in susceptible individuals coming into contact with individuals who are infective, igniting a second wave. This behavioural mechanism was, in part, behind waves of the 1918–1919 pandemic, as well as waves of the present COVID-19 pandemic in many areas. The original model has recently been extended to include two interacting fears: one of disease and one of vaccine, providing yet other behavioural routes to multiple waves of disease.
Whereas fear is considered to be contagious in this model90, the cognitive underpinnings of fear acquisition, fear transmission and fear extinction were not explicitly modelled, nor were deliberative biases, heuristics or network effects. These are represented, simply and provisionally, in Epstein’s Agent_Zero91, ‘who’ is endowed with affective, deliberative and social modules. Agent_Zero91 is a new theoretical entity grounded in contemporary cognitive neuroscience that captures (albeit crudely) some of the well-documented ways in which Homo sapiens differs from Homo economicus. It is offered as a minimal, cognitively plausible, but mathematically formal alternative to the rational actor.
Although Agent_Zero has not yet been applied explicitly to disease transmission, it is well-suited to populate agent-based epidemic models with cognitively plausible individuals. The framework is particularly promising to represent collective behaviours such as fear-driven refusal of safe vaccines, or the rejection of mask-wearing and physical distancing measures, which shape disease dynamics. Agent_Zero’s modular framework provides a way to explain such behaviours and, in turn, influence them. The framework can be used to examine the underpinnings of behaviour on the basis of clearly articulated, even if numerous, assumptions. Data-based parameterization will nonetheless depend on experimental findings—absent field studies that can capture relevant information during outbreaks. New methods of sentiment analysis using social media or RCCE data are promising in this regard, as discussed further below.
Agent-based modelling for policy
Individual-level, or agent-based models (ABMs), offer the opportunity to model disease transmission across agents that are representative of the unique, underlying sociodemographic, clinical and other characteristics that make up a population being affected by an outbreak. In an ABM, each individual is explicitly represented, and there is no loss of information due to aggregating or pooling individuals into homogeneous groups. Rather, ABMs can replicate the receptiveness of a heterogeneous community to interventions, such as the use of condoms to prevent transmission of HIV, termination of pregnancies at high risk for congenital Zika syndrome or funeral practices to prevent Ebola transmission.
The technology of ABMs has matured considerably over the last 20 years to the point at which many urban-scale, national-scale and even billion-agent planetary models have been developed and applied to urgent policy problems from smallpox bioterrorism to bird flu, swine flu, Ebola, Zika and the ongoing COVID-19 pandemic92,93. Substantial investment—including through the NIH Modelling of Infectious Disease Agent Study (https://midasnetwork.us/)94—has led to ABMs providing high-profile contributions to policymaking15,95,96,97,98,99.
When informed by sufficient, individual-level data for model parameterization and fitting, and when accompanied by understanding of model limitations, ABMs can be valuable for the design of behavioural interventions. The hallmark of ABMs is that macroscopic patterns emerge ‘from the bottom up’, from direct agent interactions. Thus, they are laboratories in which we can examine the macro effects of interventions at the microscale or community scale. However, in these frameworks, individual behaviour is sometimes considered in isolation of contextual factors, for example, community-level dynamics or policy-level factors that can influence decision-making.
ABMs can incorporate structurally rich and even dynamic representations of the social or environmental context if designed and adequately parameterized to do so. Mathematically detailed depictions of social network structures or geographical data can be readily included in an ABM—structures that can be challenging to capture in other dynamic modelling approaches100. Furthermore, ABMs can include extensive representations of individual adaptation—particularly over longer time horizons—enabling the simulation of the long-term potential impact of any policy or intervention potentially shaped by many adaptive responses101. Even on shorter time scales, many of the behaviourally plausible representations of individual responses described elsewhere in this Review require deep engagement with models of adaptation (for example, endogenous changes in individual contact patterns driven by local disease prevalence). Fear is by no means the only emotion that can be modelled102—trust in the medical establishment or in government generally are other such emotions. As in all modelling, gains in insights from the use of ABMs for representing social and behavioural factors will need to be weighed against the number, type and accuracy of input assumptions.
Opportunities for advancing models
These modelling approaches integrate social and behavioural data to understand disease spread (Table 1). However, there are clear shortcomings to these approaches that prevent them, in their current forms, from addressing key policy questions, such as those related to healthcare access, availability and use; gender, economic, social, cultural and political contexts; and population mobility, as they relate to disease dynamics23.
Why do we find these gaps? There are numerous reasons. First, modellers may understandably have a limited understanding of key social dynamics103,104. Second, the lack of robust social, behavioural and operational data means that key dynamics cannot be accurately represented in models without the use of hypothetical scenarios. Third, owing to the complexities of each science, epidemiological modellers may not be familiar with how to use community-level data generated by social sciences11, nor how to interpret and apply the wide range of theories about human behaviour and broader sociocultural and systemic dynamics105.
Social science research, RCCE practice
The disciplines that comprise social science research and the operational practices and research related to RCCE are distinct but intimately related. Within the context of epidemic response, they can generate valuable data and evidence (for example, through monitoring and evaluation of RCCE programs, observational studies and the statistical results derived from data) on a range of structural, sociocultural, political, environmental and economic drivers of behaviour affecting disease transmission and engagement with response mechanisms. Social science research and analytics have a critical role in contributing to epidemiological tools, including surveillance or contact tracing, patient intake and line list creation. RCCE approaches support communities to understand new disease threats, interpret and adapt top-down directives, identify community priorities and actions, amplify and replicate community-derived protective actions, and provide essential feedback on response measures.
Efforts have accelerated in recent years to garner a consensus on the essential role of social science research and RCCE practice in understanding and containing disease transmission. This has been in response to well-documented failures within recent outbreaks when such approaches were not adequately prioritized, funded or integrated with biomedical response interventions106. For example, initially poor RCCE approaches within the context of the West Africa Ebola outbreak have been identified as having a strong influence on disease transmission107,108,109. Equally, neglect of social science research as a core area of strategic investment in epidemics delayed the establishment of long-term, comparative datasets, structural investments in data collection systems, conventions on data sharing, and the development of key inter-disciplinary partnerships and methodological innovations23.
Social science research in epidemic contexts
Social science research during epidemics contributes to an understanding of the longer-term structural and environmental factors and behaviours that impact disease transmission23,110. To better understand outbreak dynamics and to inform response strategies and interventions, social science research offers a wide scope of approaches, methodologies and data in support of specific research questions. These generate insights into core epidemic dynamics and how these factors may influence transmission. Such approaches provide context and data for understanding sociocultural trends (for example, mobility, migration, gender and the vulnerability of women111, impacts on children112 and high-risk populations113), reluctance or resistance to interventions114 (for example, trust in governments115 and institutions116), healthcare (for example, access to and availability of health-seeking behaviour117), infection and recovery (for example, infection prevention118), epidemiology and the public health response (for example, perceived feasibility of preventive measures7 and mass quarantines119). With accelerated research and development of diagnostics, vaccines and therapeutics, evidence from social sciences offers important insights for participatory practices120,121 with multiple stakeholders and optimizing implementation of clinical research in pandemic contexts122,123.
Social science research is often based on analyses undertaken using routine data (for example, quantitative, systematic household or healthcare worker surveys) to measure trends and impacts over time as well as data and evidence gleaned from qualitative methods. In recent years, social science research has generated substantial data on perceptions and behaviours related to outbreaks of Ebola7,124,125,126, Zika127, polio128 and COVID-19 (ref. 129). The role of social science research during the West Africa and Democratic Republic of the Congo Ebola outbreaks, in particular, has led to more active integration and operationalization of social and behavioural data, including epidemiological and geospatial information, as part of epidemic response130,131.
The Minimum Quality Standards and Indicators for Community Engagement (CE) define CE as a “foundational action for working with traditional, community, civil society, government, and opinion groups and leaders; and expanding collective or group roles in addressing the issues that affect their lives. CE empowers social groups and social networks, builds upon local strengths and capacities, and improves local participation, ownership, adaptation, and communication”132.
Coordinated within broader government-led epidemic responses and often implemented together with local, national and international organizations, CE supports communities to identify and address their most pressing issues. CE is characterized by community-level mobilization and organization through trusted intermediaries, community-generated action planning methodologies133, community liaisons for basic services134, social protection and redressal, and community/citizen accountability mechanisms and feedback loops135. CE practitioners aim to fully integrate community insights operationally into outbreak response pillars and within government-led response decision-making structures26 to ensure that these services are responsive to community needs and inputs132,136. When undertaken with quality and rigour, CE and participatory leadership can increase the demand for health services137, support buy-in for health responses, improve the quality and satisfaction of services, and strengthen health systems138.
There has historically been limited standardization of CE, particularly around data collection, to enable evaluation139,140. A review conducted by UNICEF identified more than 1,000 distinct CE indicators addressing a broad range of behavioural parameters (UNICEF, available on request). When CE datasets are available, they are often not formally considered for distribution or preservation141. They are often decentralized, fragmented and not representative at the population level, and are therefore difficult to compare across countries. Recently, there have been efforts to develop minimum standards for practice and measurement for CE132, and better documentation of how evidence can be used to adapt response interventions131. When prioritized, standardized and better coordinated to address these methodological and other issues, CE data on a range of behavioural, temporal and spatial parameters can, and has, been collected in real time and at scale during epidemics142,143.
Risk Communication (RC) can be generally understood as an iterative exchange of information among individuals, groups and institutions related to the assessment, characterization and management of risk144,145. It is a cross-cutting tool that aims to provide real-time information to support individual and community decision-making. RC approaches use a range of messaging platforms and methodological approaches, including communication for behavioural change, mass media, social media, health education and health promotion146.
As a well-developed area of research and practice, RC has evolved to take into consideration subjective and objective risks determined by social, cultural, economic and psychological factors144. Real-time and retrospective data and analysis related to RC and associated outcomes has been undertaken within the context of COVID-19 (ref. 147) and Ebola148, including large-scale community feedback data by monitoring rumours during the Ebola outbreak in the Democratic Republic of the Congo135,149, as well as tracking digital misinformation during the Zika outbreak150, about HIV/AIDS151 and SARS152, and on social media153,154,155,156.
An integral part of RC involves identifying and countering misinformation and disinformation, the impacts of which can be extreme and immediate157. The experience of the COVID-19 epidemic has further highlighted the pernicious nature of misinformation as a risk in itself158, and has spurred the emerging field of infodemiology. Infodemiology—the science of understanding how information spreads—will be an important source of data in an age of increased communication and messaging virality159. The large volume of information—some accurate, some not—that comes alongside an epidemic is not new160, but amplification of the information through social media makes it a much more complex phenomenon to understand and manage. Infodemics can have an impact on health directly by influencing decision-making, and indirectly by promoting stigma, racism and discrimination161.
Challenges and opportunities for a collaborative agenda for integrated modelling
This is not the first call for multidisciplinary dialogue to integrate behavioural determinants into disease modelling162,163. The COVID-19 pandemic has underscored the necessity and increased the urgency for such collaboration. Unprecedented attention and advancements in the field of disease modelling offer new opportunities to build on the existing body of work to integrate social and behavioural parameters. Concurrent growth in the capacity, coordination and prioritization of social science research and RCCE practice also provides opportunities for more meaningful dialogue. The following section outlines the key challenges and opportunities for advancing a collaborative agenda for integrated modelling.
Building a multidisciplinary community of practice and common framework for collaboration
While there is ad hoc collaboration across the disciplines, this is typically unsystematic and opportunistic in nature. There is presently no consistent set of protocols for collaborative research and model design across modelling and social science disciplines, nor a uniform approach to data collection and management. Despite calls for collaboration, there is not a widely shared appreciation of the comparative strengths of different disciplines. Disease modellers generally have a strong understanding of disease epidemiology and transmission dynamics for predicting the course of an epidemic, but less expertise on sociocultural and other non-biological drivers of differential behaviour related to disease transmission. By contrast, social scientists and RCCE practitioners generally use more holistic frameworks, theories and empirical methods to understand concerns, beliefs, assets and actions of individuals and communities that are most affected by disease (including those marginalized within communities) but may lack the tools to quantify the health impacts of social and behavioural interventions or to translate insights into meaningful model inputs. Unpacking these comparative advantages, and aligning on common goals and terms, are the first steps towards developing a community of practice.
Collaboration begins at the conceptualization of research questions about specific behavioural and epidemiological outcomes. Figure 1 presents a schematic of key actors and indicative processes of multidisciplinary participatory disease model design. Such a design process should be responsive to direct input from local communities and simultaneously focused on addressing the data-for-decision needs and priorities of response decision-makers. This includes adequately planning for and addressing ethical concerns related to data collection, transparency, validation of assumptions, unintended consequences, and accountability to communities and research participants164. To best operationalize behavioural data in real-time, the disease response architecture itself must be adjusted, for example, by consistently embedding social scientists and RCCE practitioners within epidemiological teams, or by planning for a dedicated, integrated analytics cell.
Developing computationally tractable formulations and validating model assumptions
Mathematical models are necessarily developed with simplifying assumptions. To define research questions that better incorporate social and behavioural factors, new formulations to account for such factors will be required. Modellers must strike a delicate balance, harnessing available data and theory on complex social and behavioural phenomena, while retaining important modelling properties, such as tractability and parsimony. There is ample scope to carefully evaluate modelling assumptions and identify opportunities for improvement.
In present adaptive network models, for example, behavioural reactions to the disease are often selected using an ‘Occam’s razor’ approach—incorporating intuitive mechanisms of which the mathematical form is often chosen for convenience. These models tend to make two large assumptions. First, that information is clearly localized in the network such that individuals mostly react to the state of their direct neighbours or contacts. Second, that the scale of behavioural reactions follows a linear relationship with perceived or actual epidemic risk. However, emerging evidence from research in health psychology165 shows how social sciences can help to parametrize more realistic models and relax the first assumption. Similarly, behavioural data on individual reactions to local cases of an infectious disease166 will help modellers to relax the second assumption to better understand the often-non-linear relationship between behavioural changes (at different depths of one’s social circles) and perceived risks associated with the disease. In this way, input from social sciences can lead to better adaptive network models, guided not by mathematical convenience but by a better understanding of the psychology of social and behavioural change.
To improve accuracy and draw correct conclusions, modellers can validate (and adjust) fundamental assumptions about issues such as trust, fear, response efficacy and self-efficacy with social scientists and practitioners who have context-specific expertise and direct links to communities. Core concepts, such as trust in official authorities versus local leaders, are challenging to generalize due to complex sociocultural and political dimensions in different contexts. For example, although there was a frequent assumption among modellers about how fear led to evasion of public health interventions during the West Africa Ebola outbreak, social scientists observed that fear was not unidirectional; there was fear of response authorities in some cases, but there was also fear of the consequences of the disease (for example, of death and infecting family), which motivated people to voluntarily modify their behaviours, including deep-rooted cultural practices167. Social constructs also influenced how fear manifested. For example, although families with patients with suspected Ebola were fearful of contracting Ebola, the sense of duty and obligation to care for a family member often prevailed over the underlying fear of becoming infected168.
Accessing and making full use of social, behavioural and operational data
In cases in which there is an insufficient availability of comprehensive data to quantifiably incorporate social and behavioural factors, mathematical models by their very nature depend on simplifying assumptions. Identifying appropriate parameters and incorporating associated essential data—that is, taking complicated, dynamic processes and interpreting them as something that can be incorporated into a mathematical model—is a pervasive challenge162.
Nonetheless, there is clearly room to make better use of data that are available, but of which access requires direct outreach and collaboration across the disciplinary divide. For example, one review of the modelling literature found that less than 80% (of 178 articles examined) of studies that sought to incorporate behavioural factors used any empirical data for either parameterization or validation36. Another review found that, although 50% (21 out of 42) of studies explicitly used behavioural data sources, these sources were limited to secondary data (for example, survey data cited from previous papers), demographic data and epidemiological data37. Modellers can make use of a broader set of available social and behavioural data to achieve more representative selection parameters and validation processes that result in more nuanced and accurate assumptions. Data of different types and sources can be collected as a component of structured CE activities during outbreak response and can in turn be used to model—retrospectively and in real time—the impacts of CE interventions on disease transmission. Large-scale, spatiotemporal mixed CE data (social, behavioural and operational) from Sierra Leone are presently being used to retrospectively investigate contextualized trends in behavioural change during the 2014–2015 Ebola outbreak and to quantify the impacts of CE on behavioural and epidemiological outcomes, including modelling of the relationship between CE mobilizer visits and disease transmission, mobility and behavioural change142.
Strengthening coordinated collection and consolidation of social and behavioural data
In turn, social scientists and RCCE practitioners can improve the collection of relevant social and behavioural data at sufficient scale and with quality and rigour. Social and behavioural data collection, management, storage, aggregation and sharing within the context of epidemics in real-time can be difficult to perform at scale162, but has been shown to be achievable if prioritized and funded142,169.
The RCCE Collective Service—a new coordination mechanism led by the WHO, UNICEF and IFRC—has now produced the global COVID-19 RCCE Framework and Core Indicators and a standardized question bank, the first such global, multi-country framework for social and behavioural data collection and RCCE in an outbreak response. This approach and platform represent a meaningful step forwards to enable more systematic cross-country data collection, analysis and synthesis170. Similarly, knowledge and data-sharing platforms such as the GOARN-R171, Ebola Anthropology Platform (http://www.ebola-anthropology.net) and Social Science in Humanitarian Action Platform (https://www.socialscienceinaction.org) as well as initiatives such as the Sonar-Global Network172 have also increased the visibility, accessibility and application of social science research. Similarly, the use of digital tools for data collection during the COVID-19 epidemic has offered insights into opportunities for modelling with more granular behaviour data, for example, self-reported mask compliance from large-scale social media surveys; changes in contact patterns through geocoded cell phone mobility data173,174,175; and tracking and coding sentiments, rumours, and misinformation and disinformation through digital social listening initiatives176.
At the community level, disease-specific social and behavioural risk factors can be included in case investigation forms during outbreaks. Although there are examples of these being included previously (for example, burial participation for Ebola investigations177 and occupational setting for COVID-19 investigations178), social and behavioural data elements are usually not included in standard case investigation forms178. Retrospective sociobehavioural assessments outside of the case investigation process are relied on much later into the outbreak to understand the role of social and behavioural factors on the epidemiology of the outbreak179.
Data collection during outbreak response can be best achieved through the establishment of transdisciplinary teams working together during outbreak response to collect and analyse data. During the Ebola epidemic response in the Democratic Republic of the Congo, the Social Science Analytics Cell established by UNICEF more proactively prioritized data collection and used integrated multidisciplinary outbreak analytics to develop and communicate real-time evidence to explain differential trends in outbreak dynamics, understand changes in health outcomes over time and measure broader health and community impacts of the outbreak14,180,181. These examples help to illustrate the potential of both collecting better data to inform models and developing better models to quantify the impact of social and behavioural interventions.
Drawing on the comparative advantages and examples detailed above, we offer the following recommendations to advance better integration of social science research and RCCE practice in epidemiological modelling:
Build a community of practice for networking, partnership-building and collaboration. Testing and learning together using existing models that incorporate simple behavioural mechanisms (such as those described above in “Modelling social and behavioural factors“) will enable social scientists and disease modellers to collaborate using observed data to develop models for future application and as part of epidemic preparedness. The community could be its own entity with membership or be a collaborative effort between existing entities that to date have been predominantly focused on social sciences or mathematical modelling.
Develop protocols for the integration of contextual, sociocultural and behavioural factors into modelling. Establish ethical frameworks, data protections, research protocols and collaboration processes to guide the co-creation of research questions, frameworks, methods and other innovations. Collaboratively designed research questions (Table 2) will lead to more targeted and focused model outcomes, and to more evolved modelling methodologies. Given the vast research resources available from present and past pandemics182,183, a key focus should be the testing, calibration and validation of dynamic models that integrate real-time and historical social science evidence in public health emergencies. Focus collaboration on identifying ways to parameterize quantitative and qualitative data, the theories that underpin decision-making processes, along with understanding and incorporating existing historical data on individuals, communities and populations, with respect to social norms, cultural practices, existing ‘baseline’ political economy and other factors112,184,185.
Build or expand on open-source model and code repositories that can focus on integrated disease modelling. All stakeholders would benefit from the availability of existing data, methods, themes, models and software to all stakeholders186. A model and code repository would offer a wide range of social science and modelling resources based on present standards of practice and will support more efficient innovation, collaboration, and ongoing iteration and discovery. Existing repositories (such as the CoMSES Net platform for agent-based modelling (https://www.comses.net)) can form the basis of expanded efforts to extend resources to include social science disciplines.
Develop prepositioned sets of RCCE data frameworks, including thematic priorities, measures and indicators. At the response level, incorporating social, behavioural and community-level data into outbreak investigation protocols will enable the timely collection of data—for example, including data on the adoption of transmission-reducing behaviours and the level of adherence to these, as well as the reasons for refusal or inability to adopt behaviours—that can be mined for insights regarding the potential social and behavioural drivers of an outbreak. Such data can inform the context-varying assumptions and parameterization of integrated disease models in real time during an outbreak.
Preposition data sharing agreements between researchers, governments and other response actors to facilitate rapid data movement from field-based data collection centres to centres of modelling and research, particularly at the country level whereby findings can be directly applied to response operations in near real time. Disease modellers need better access to social science and RCCE data across platforms and networks. Data, even from small-scale initiatives, can be used to develop and test integrated modelling frameworks187, even as standards for larger-scale data collection are developed. As the data gap is addressed, more information will improve model formulation and calibration to the contextual realities of disease and behavioural dynamics.
Establish and sustain transdisciplinary teams to routinely work together during an outbreak response. The institutionalization of cross-pillar coordination and the development of agreed standard operating procedures spanning the administrative, biomedical and RCCE pillars (for example, that undertaken by the Social Science Analytics Cell in the Democratic Republic of the Congo) is needed for integrated analytics become a reality in outbreak response. Working within the established policy contained within international frameworks (such as the International Health Regulations and Emergency Response Framework), the operationalization of on-the-ground collaborations between epidemiologists, social science professionals and practitioners will provide a robust foundation for our recommendations towards integrated modelling.
Facilitate training and peer exchange to develop mutual understanding and shared skill sets among epidemiologists, modellers, social scientists, RCCE practitioners and decision-makers. Create training opportunities for modellers to acquire a foundational understanding of core theoretical and methodological principles in social sciences and to engage with the complex theoretical bases of behavioural, sociocultural and social sciences research. Social scientists can be trained on the various types and forms of model inputs and frameworks that may be appropriate for investigating social and behavioural questions in epidemic contexts. This might include incorporation into existing graduate programmes, research exchanges between institutions or similar to short courses developed for increasing knowledge across disciplines188,189. It should be recognized that opportunities for collaboration for researchers in low- and middle-income countries are often inequitable, despite evidence that such access improves research outcomes190. Access to research and data, as well as the capacity, resources and opportunities to use this data, are often inequitable and/or underrecognized191. A priority focus should be placed on researchers from lower-middle-income countries working in their own contexts.
Review research undertaken during recent epidemics from the perspectives of epidemiology and social science. Understanding how the respective fields have addressed research questions within specific outbreaks will provide a targeted analysis of broader research themes prioritized by epidemiological and social science research, an improved understanding of the factors and assumptions addressed within these themes, and the identification of areas of complementarity and respective gaps between disciplines from real-world examples. Such reviews can support an understanding of how the research lens applied can result in differing assumptions, findings and recommendations, and encourage a mutual understanding of the limitations and opportunities of the respective disciplines23.
Ensure that restitution to local communities is mainstreamed into modelling practices. To close the gap between modellers, social scientists and RCCE practitioners, serious attention needs to be given to how research outcomes are returned to local communities to form a basis for action. The concept—if not always the systematic practice—of feedback loops is well established within humanitarian response, whereby community-generated data result in a response or action192. For example, social science research data can directly inform community engagement and action planning26. Although, as we have seen, modelling often uses secondary source data, integrated modelling will bring modellers closer to the communities that they model. When integrated modelling aims to better reflect the realities of communities by using community-generated data, the onus is on all researchers to ensure that those communities benefit from the outputs. A comprehensive review should be conducted to examine communication practices within the modelling community to determine how to rapidly, efficiently, thoroughly and realistically revert integrated modelling findings and discoveries to national, regional and local stakeholders.
Jointly and clearly communicate with policymakers to co-create recommendations for immediate, mid-term and long-term actions. There is often a gap between what decision-makers and policy-makers want from models and what models deliver, due to inherent model uncertainties as well as a poor communication and expectation setting193. It is essential to communicate model limitations, while identifying those that can be addressed with additional information. This should involve describing how social and behavioural theories have been included in the modelling, which data or proxies have been used and what the results have demonstrated in light of the original collaborative design process. Communication must be carefully crafted to be interpretable by scientists, policy-makers and the public to avoid misinterpretation or misuse of models and outputs194,195,196.
Advocate for increased investment in integrated disease modelling. Researchers and policy-makers must engage in joint advocacy to donors and grant-making organizations. This is best performed by developing a collaborative—and fundable—research agenda supported by a clear, evidence-based investment case. A funding gap analysis is urgently needed to determine and address the global capacity demands for multidisciplinary research, including integrated disease modelling. A consortium of champions, drawn from the Community of Practice and representative of the different participants in the integrated modelling process (Fig. 1), should undertake the analysis.
Disease modellers, social science professionals and RCCE practitioners share the objectives of reducing preventable mortality and socioeconomic burdens associated with disease transmission and disease response measures. Integrated modelling that better incorporates social and behavioural dynamics will advance predictive accuracy and thereby more effectively guide policy and response measures in service of these objectives. To advance a synergistic research and practice agenda, social scientists, RCCE practitioners and disease modellers should develop a community of practice and develop a common framework for understanding and interpreting, social, behavioural and operational data and theory. Such efforts will support mutually reinforcing calibration and validation of models and social science research, while also providing quantitative insights into the impacts of RCCE interventions. A failure to better integrate modelling can result in divergent policy recommendations, which has practical and ethical implications for disease response. While the inherent complexity of disease modelling that incorporates social and behavioural factors can limit the degree to which community realties can be included, the potential of integrated disease modelling remains to be explored.
Raude, J., McColl, K., Flamand, C. & Apostolidis, T. Understanding health behavior changes in response to outbreaks: findings from a longitudinal study of a large epidemic of mosquito-borne disease. Soc. Sci. Med. 230, 184–193 (2019).
Kapiriri, L. & Ross, A. The politics of disease epidemics: a comparative analysis of the SARS, Zika, and Ebola outbreaks. Glob. Soc. Welf. 7, 33–45 (2020).
Lewis, M. The economics of epidemics. Georget. J. Int. Aff. 2, 25–31 (2001).
Gelfand M. J. et al. The relationship between cultural tightness–looseness and COVID-19 cases and deaths: a global analysis. Lancet Planet. Health https://doi.org/10.1016/S2542-5196(20)30301-6 (2021).
Marston, C., Renedo, A. & Miles, S. Community participation is crucial in a pandemic. Lancet 395, 1676–1678 (2020).
Shultz, J. M. et al. The role of fear-related behaviors in the 2013–2016 West Africa Ebola virus disease outbreak. Curr. Psychiatry Rep. 18, 104 (2016).
Abramowitz, S. et al. The opposite of denial: social learning at the onset of the Ebola emergency in Liberia. J. Health Commun. 22, 59–65 (2017).
Lee, C., Ayers, S. L. & Kronenfeld, J. J. The association between perceived provider discrimination, healthcare utilization and health status in racial and ethnic minorities. Ethn. Dis. 19, 330–337 (2009).
Fenton, J. J., Jerant, A. F., Bertakis, K. D. & Franks, P. The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Arch. Intern. Med. 172, 405–411 (2012).
Carter, S. E. et al. Barriers and enablers to treatment-seeking behavior and causes of high-risk practices in Ebola: a case study from Sierra Leone. J. Health Commun. 22, 31–38 (2017).
Kretzschmar, M. Disease modeling for public health: added value, challenges, and institutional constraints. J. Public Health Policy 41, 39–51 (2020).
Brauer, F. Mathematical epidemiology: past, present, and future. Infect. Dis. Model 2, 113–127 (2017).
Chowell, G., Sattenspiel, L., Bansal, S. & Viboud, C. Mathematical models to characterize early epidemic growth: a review. Phys. Life Rev. 18, 66–97 (2016).
Polonsky, J. A. et al. Outbreak analytics: a developing data science for informing the response to emerging pathogens. Philos. Trans. R. Soc. Lond. B Biol. Sci. 374, 20180276 (2019).
Longini, I. M. Jr et al. Containing pandemic influenza at the source. Science 309, 1083–1087 (2005).
Zhang, Q. et al. Spread of Zika virus in the Americas. Proc. Natl Acad. Sci. USA. 114, E4334–E4343 (2017).
Chretien, J.-P., Riley, S. & George, D. B. Mathematical modeling of the West Africa Ebola epidemic. eLife 4, e09186 (2015).
Chowell, G. & Nishiura, H. Transmission dynamics and control of Ebola virus disease (EVD): a review. BMC Med. 12, 196 (2014).
Adam, D. Special report: the simulations driving the world’s response to COVID-19. Nature 580, 316–318 (2020).
Siegenfeld, A. F., Taleb, N. N. & Bar-Yam, Y. Opinion: what models can and cannot tell us about COVID-19. Proc. Natl Acad. Sci. USA 117, 16092–16095 (2020).
Manfredi, P. & D’Onofrio, A., eds. Modeling the Interplay Between Human Behavior and the Spread of Infectious Diseases (Springer-Verlag, 2013).
Philipson, T. in Handbook of Health Economics (eds Culyer, A. and Newhouse, J.) Vol. 1, Ch. 33, 1761–1799 (Elsevier, 2000).
Abramowitz, S. A., Hipgrave, D. B., Witchard, A. & Heymann, D. L. Lessons from the West Africa Ebola epidemic: a systematic review of epidemiological and social and behavioral science research priorities. J. Infect. Dis. 218, 1730–1738 (2018).
Bedford, J. et al. Application of social science in the response to Ebola, Equateur Province, Democratic Republic of the Congo/Application des sciences sociales dans la riposte a la maladie a virus Ebola, province de l’Equateur, Republique democratique du Congo. Wkly Epidemiological Rec. 94, 19–24 (2019).
Norton, A. et al. A living mapping review for COVID-19 funded research projects: six-month update [version 3; peer review: 2 approved]. Wellcome Open Res. 5, 209 (2021).
Pedi, D. et al. The development of standard operating procedures for social mobilization and community engagement in sierra leone during the West Africa Ebola outbreak of 2014-2015. J. Health Commun. 22, 39–50 (2017).
RCCE Collective Service. Operational guide for engaging communities in contact tracing World Health Organization (2021); https://apps.who.int/iris/bitstream/handle/10665/341553/WHO-2019-nCoV-Contact_tracing-Community_engagement-2021.1-eng.pdf?sequence=1
Cellules d’Analyses en Sciences Sociales (CASS). Social Science Support for COVID-19: Lessons Learned Brief 3 7 (2020); https://www.unicef.org/drcongo/media/4131/file/CASS-Brief3-recommendations.pdf
Xepapadeas, A. The spatial dimension in environmental and resource economics. Environ. Dev. Econ. 15, 747–758 (2010).
Reed, M. S. et al. What is social learning? Ecol. Soc. 15, r1 (2010).
Kermack, W. O., McKendrick, A. G. & Walker, G. T. A contribution to the mathematical theory of epidemics. Proc. R. Soc. Lond. A 115, 700–721 (1927).
Influenza in a boarding school. Brit. Med. J. 1, 587–587 (1978).
Funk, S., Salathé, M. & Jansen, V. A. A. Modelling the influence of human behaviour on the spread of infectious diseases: a review. J. R. Soc. Interface 7, 1247–1256 (2010).
Eksin, C., Paarporn, K. & Weitz, J. S. Systematic biases in disease forecasting-the role of behavior change. Epidemics 27, 96–105 (2019).
Bedford, J. et al. A new twenty-first century science for effective epidemic response. Nature 575, 130–136 (2019).
Verelst, F., Willem, L. & Beutels, P. Behavioural change models for infectious disease transmission: a systematic review (2010-2015). J. R. Soc. Interface https://doi.org/10.1098/rsif.2016.0820 (2016).
Weston, D., Hauck, K. & Amlôt, R. Infection prevention behaviour and infectious disease modelling: a review of the literature and recommendations for the future. BMC Public Health 18, 336 (2018).
Gersovitz, M. The economics of infection control. Annu. Rev. Resour. Econ. 3, 277–296 (2011).
Perrings, C. et al. Merging economics and epidemiology to improve the prediction and management of infectious disease. Ecohealth 11, 464–475 (2014).
Althouse, B. M., Bergstrom, T. C. & Bergstrom, C. T. Evolution in health and medicine Sackler colloquium: a public choice framework for controlling transmissible and evolving diseases. Proc. Natl Acad. Sci. USA 107, 1696–1701 (2010).
Ward, C. J. Influenza vaccination campaigns: is an ounce of prevention worth a pound of cure? Am. Econ. J. Appl. Econ. 6, 38–72 (2014).
Fenichel, E. P. Economic considerations for social distancing and behavioral based policies during an epidemic. J. Health Econ. 32, 440–451 (2013).
Acemoglu, D., Chernozhukov, V., Werning, I. & Whinston, M. D. Optimal Targeted Lockdowns in a Multi-Group SIR Model Working Paper 27102 (National Bureau of Economic Research, 2020); https://doi.org/10.3386/w27102
Ahituv, A., Hotz, V. J. & Philipson, T. The responsiveness of the demand for condoms to the local prevalence of AIDS. J. Hum. Resour. 31, 869–897 (1996).
Kremer, M. Integrating behavioral choice into epidemiological models of AIDS. Q. J. Econ. 111, 549–573 (1996).
Justwan, F., Baumgaertner, B., Carlisle, J. E., Carson, E. & Kizer, J. The effect of trust and proximity on vaccine propensity. PLoS ONE 14, e0220658 (2019).
Chen, F. H. Rational behavioral response and the transmission of STDs. Theor. Popul. Biol. 66, 307–316 (2004).
Geoffard, P.-Y. & Philipson, T. Rational epidemics and their public control. Int. Econ. Rev. 37, 603–624 (1996).
Fenichel, E. P. et al. Adaptive human behavior in epidemiological models. Proc. Natl Acad. Sci. USA 108, 6306 (2011).
Morin, B. R., Fenichel, E. P. & Castillo-Chavez, C. SIR dynamics with economically driven contact rates. Nat. Resour. Model. 26, 505–525 (2013).
Fenichel, E. P., Kuminoff, N. V. & Chowell, G. Skip the trip: air travelers’ behavioral responses to pandemic influenza. PLoS ONE 8, e58249 (2013).
Hung, Y. W. et al. Impact of a free care policy on the utilisation of health services during an Ebola outbreak in the Democratic Republic of Congo: an interrupted time-series analysis. BMJ Glob. Health 5, e002119 (2020).
Modeling Anthropogenic Effects in the Spread of Infectious Diseases (MASpread) Project. EcoServices: Disease Risks. Arizona State University (Accessed 17 April 2021); http://ecoservices.asu.edu/Diseaserisks/DRindex.html
Morris, M. Network Epidemiology: A Handbook for Survey Design and Data Collection (OUP, 2004).
Meyers, L. Contact network epidemiology: bond percolation applied to infectious disease prediction and control. Bull. Am. Math. Soc. 44, 63–86 (2007).
Pastor-Satorras, R., Castellano, C., Van Mieghem, P. & Vespignani, A. Epidemic processes in complex networks. Rev. Mod. Phys. 87, 925–979 (2015).
Wang, Z. et al. Statistical physics of vaccination. Phys. Rep. 664, 1–113 (2016).
Cohen, R., Havlin, S. & Ben-Avraham, D. Efficient immunization strategies for computer networks and populations. Phys. Rev. Lett. 91, 247901 (2003).
Salathé, M. & Jones, J. H. Dynamics and control of diseases in networks with community structure. PLoS Comput. Biol. 6, e1000736 (2010).
Hébert-Dufresne, L., Allard, A., Young, J.-G. & Dubé, L. J. Global efficiency of local immunization on complex networks. Sci. Rep. 3, 2171 (2013).
Rosenblatt, S. F., Smith, J. A., Gauthier, G. R. & Hébert-Dufresne, L. Immunization strategies in networks with missing data. PLoS Comput. Biol. 16, e1007897 (2020).
Funk, S., Gilad, E., Watkins, C. & Jansen, V. A. A. The spread of awareness and its impact on epidemic outbreaks. Proc. Natl Acad. Sci. USA 106, 6872–6877 (2009).
Funk, S. & Jansen, V. A. A. Interacting epidemics on overlay networks. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 81, 036118 (2010).
Hébert-Dufresne, L., Mistry, D. & Althouse, B. M. Spread of infectious disease and social awareness as parasitic contagions on clustered networks. Phys. Rev. Res. 2, 033306 (2020).
Marceau, V., Noël, P.-A., Hébert-Dufresne, L., Allard, A. & Dubé, L. J. Modeling the dynamical interaction between epidemics on overlay networks. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 84, 026105 (2011).
Fu, F., Christakis, N. A. & Fowler, J. H. Dueling biological and social contagions. Sci. Rep. 7, 43634 (2017).
Granell, C., Gómez, S. & Arenas, A. Competing spreading processes on multiplex networks: awareness and epidemics. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 90, 012808 (2014).
Fan, C.-J. et al. Effect of individual behavior on the interplay between awareness and disease spreading in multiplex networks. Phys. A 461, 523–530 (2016).
Scatà, M., Di Stefano, A., Liò, P. & La Corte, A. The impact of heterogeneity and awareness in modeling epidemic spreading on multiplex networks. Sci. Rep. 6, 37105 (2016).
Wang, W. et al. Suppressing disease spreading by using information diffusion on multiplex networks. Sci. Rep. 6, 29259 (2016).
Zheng, C., Xia, C., Guo, Q. & Dehmer, M. Interplay between SIR-based disease spreading and awareness diffusion on multiplex networks. J. Parallel Distrib. Comput. 115, 20–28 (2018).
Gross, T. & Blasius, B. Adaptive coevolutionary networks: a review. J. R. Soc. Interface 5, 259–271 (2008).
Gross, T. & Sayama, H. in Adaptive Networks: Theory, Models and Applications (eds Gross, T. & Sayama, H.) 1–8 (Springer, 2009).
Wang, Z., Andrews, M. A., Wu, Z.-X., Wang, L. & Bauch, C. T. Coupled disease–behavior dynamics on complex networks: a review. Phys. Life Rev. 15, 1–29 (2015).
Valdez, L. D., Macri, P. A. & Braunstein, L. A. Intermittent social distancing strategy for epidemic control. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 85, 036108 (2012).
Tunc, I., Shkarayev, M. S. & Shaw, L. B. Epidemics in adaptive social networks with temporary link deactivation. J. Stat. Phys. 151, 355–366 (2013).
Epstein, J. M., Parker, J., Cummings, D. & Hammond, R. A. Coupled contagion dynamics of fear and disease: mathematical and computational explorations. PLoS ONE 3, e3955 (2008).
Kiss, I. Z., Cassell, J., Recker, M. & Simon, P. L. The impact of information transmission on epidemic outbreaks. Math. Biosci. 225, 1–10 (2010).
Gross, T., D’Lima, C. J. D. & Blasius, B. Epidemic dynamics on an adaptive network. Phys. Rev. Lett. 96, 208701 (2006).
Zanette, D. H. & Risau-Gusmán, S. Infection spreading in a population with evolving contacts. J. Biol. Phys. 34, 135–148 (2008).
Marceau, V., Noël, P.-A., Hébert-Dufresne, L., Allard, A. & Dubé, L. J. Adaptive networks: coevolution of disease and topology. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 82, 036116 (2010).
Shaw, L. B. & Schwartz, I. B. Enhanced vaccine control of epidemics in adaptive networks. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 81, 046120 (2010).
Althouse, B. M. & Hébert-Dufresne, L. Epidemic cycles driven by host behaviour. J. R. Soc. Interface https://doi.org/10.1098/rsif.2014.0575 (2014).
Scarpino, S. V., Allard, A. & Hébert-Dufresne, L. The effect of a prudent adaptive behaviour on disease transmission. Nat. Phys. 12, 1042–1046 (2016).
Shaw, L. B. & Schwartz, I. B. Fluctuating epidemics on adaptive networks. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 77, 066101 (2008).
Sayama, H. et al. Modeling complex systems with adaptive networks. Comput. Math. Appl. 65, 1645–1664 (2013).
Do, A.-L., Rudolf, L. & Gross, T. Patterns of cooperation: fairness and coordination in networks of interacting agents. N. J. Phys. 12, 063023 (2010).
Van Segbroeck, S., Santos, F. C., Lenaerts, T. & Pacheco, J. M. Selection pressure transforms the nature of social dilemmas in adaptive networks. N. J. Phys. 13, 013007 (2011).
Zhan, X.-X. et al. Coupling dynamics of epidemic spreading and information diffusion on complex networks. Appl. Math. Comput. 332, 437–448 (2018).
Hatfield, E., Cacioppo, J. T. & Rapson, R. L. Emotional contagion. Curr. Dir. Psychol. Sci. 2, 96–100 (1993).
Epstein, J. M. Agent_Zero: Toward Neurocognitive Foundations for Generative Social Science (Princeton Univ. Press, 2014).
Barton, C. M. et al. Call for transparency of COVID-19 models. Science 368, 482–483 (2020).
Hammond, R., Ornstein, J. T., Purcell, R., Haslam, M. D., & Kasman, M. Modeling robustness of COVID-19 containment policies. Preprint at OSF https://doi.org/10.31219/osf.io/h5ua7 (2021).
Cooley, P. C. et al. The model repository of the models of infectious disease agent study. IEEE Trans. Inf. Technol. Biomed. 12, 513–522 (2008).
Eubank, S. et al. Modelling disease outbreaks in realistic urban social networks. Nature 429, 180–184 (2004).
Burke, D. S. et al. Individual-based computational modeling of smallpox epidemic control strategies. Acad. Emerg. Med. 13, 1142–1149 (2006).
Ferguson, N. M. et al. Strategies for mitigating an influenza pandemic. Nature 442, 448–452 (2006).
Germann, T. C., Kadau, K., Longini, I. M. Jr & Macken, C. A. Mitigation strategies for pandemic influenza in the United States. Proc. Natl Acad. Sci. USA 103, 5935–5940 (2006).
Longini, I. M. Jr et al. Containing a large bioterrorist smallpox attack: a computer simulation approach. Int. J. Infect. Dis. 11, 98–108 (2007).
Hammond, R. A. Considerations and Best Practices in Agent-Based Modeling to Inform Policy (National Academies Press, 2015).
Wallace, R et al. Assessing the Use of Agent-Based Models for Tobacco Regulation (National Academies Press, 2015).
Pedro, S. A. et al. Conditions for a second wave of COVID-19 due to interactions between disease dynamics and social processes. Front. Phys. 8, 574514 (2020).
Walters, C. E., Meslé, M. M. I. & Hall, I. M. Modelling the global spread of diseases: a review of current practice and capability. Epidemics 25, 1–8 (2018).
Li, Y., Lawley, M. A., Siscovick, D. S., Zhang, D. & Pagán, J. A. Agent-based modeling of chronic diseases: a narrative review and future research directions. Prev. Chronic Dis. 13, E69 (2016).
Weston, D., Ip, A. & Amlôt, R. Examining the application of behaviour change theories in the context of infectious disease outbreaks and emergency response: a review of reviews. BMC Public Health 20, 1483 (2020).
Ripoll, S., Gercama, I., Jones, T. & Wilkinson, A. Social Science in Epidemics: Ebola Virus Disease Lessons Learned Background Report, UNICEF, IDS & Anthrologica https://opendocs.ids.ac.uk/opendocs/handle/20.500.12413/14160 (Institute of Development Studies, 2018).
DuBois, M., Wake, C., Sturridge, S. & Bennett, C. The Ebola Response in West Africa: Exposing the Politics and Culture of International Aid (Overseas Development Institute, 2015).
Hird, T. et al. Lessons From Ebola Affected Communities: Being Prepared for Future Health Crises (Africa All Party Parliamentary Group, 2016).
WHO. Report of the Ebola Interim Assessment Panel—July 2015 (2020).
Ashworth, H. C., Dada, S., Buggy, C. & Lees, S. The importance of developing rigorous social science methods for community engagement and behavior change during outbreak response. Disaster Med. Public Health Prep. 1–6 (2020).
Wenham, C. et al. Women are most affected by pandemics—lessons from past outbreaks. Nature 583, 194–198 (2020).
Schwartz, D. A., Anoko, J. N. & Abramowitz, S. A. Pregnant in the Time of Ebola: Women and Their Children in the 2013-2015 West African Epidemic (Springer International Publishing, 2019).
Moore, M. D. Historicising ‘containment and delay’: COVID-19, the NHS and high-risk patients. Wellcome Open Res. 5, 130 (2020).
Marcis, F. L., Enria, L., Abramowitz, S., Saez, A.-M. & Faye, S. L. B. Three acts of resistance during the 2014–16 West Africa Ebola epidemic. J. Humanitarian Aff. 1, 23–31 (2019).
Parker, M., Hanson, T. M., Vandi, A., Babawo, L. S. & Allen, T. Ebola and public authority: saving loved ones in Sierra Leone. Med. Anthropol. 38, 440–454 (2019).
Vinck, P., Pham, P. N., Bindu, K. K., Bedford, J. & Nilles, E. J. Institutional trust and misinformation in the response to the 2018–19 Ebola outbreak in North Kivu, DR Congo: a population-based survey. Lancet Infect. Dis. 19, 529–536 (2019).
Ripoll, S., Gercama, I. & Jones, T. Rapid Appraisal of Key Health-Seeking Behaviours in Epidemics. SSHAP Practical Approaches brief 5, UNICEF, IDS & Anthrologica https://opendocs.ids.ac.uk/opendocs/handle/20.500.12413/15430 (Institute of Development Studies, 2020).
Bielicki, J. A. et al. Monitoring approaches for health-care workers during the COVID-19 pandemic. Lancet Infect. Dis. 20, e261–e267 (2020).
Chu, I. Y.-H., Alam, P., Larson, H. J. & Lin, L. Social consequences of mass quarantine during epidemics: a systematic review with implications for the COVID-19 response. J. Travel Med. 27, taaa192 (2020).
R&D Good Participatory Practice for COVID-19 Clinical Trials: a Toolbox (World Health Organization, 2020); https://www.who.int/publications/m/item/r-d-good-participatory-practice-for-covid-19-clinical-trials-a-toolbox
Hankins, C. Good Participatory Practice Guidelines for Trials of Emerging (and Re-emerging) Pathogens That are Likely to Cause Severe Outbreaks in the Near Future and For Which Few or No Medical Countermeasures Exist (GPP-EP) (WHO, 2016).
Sigfrid, L. et al. Addressing challenges for clinical research responses to emerging epidemics and pandemics: a scoping review. BMC Med. 18, 190 (2020).
Gobat, N. H. et al. Talking to the people that really matter about their participation in pandemic clinical research: a qualitative study in four European countries. Health Expect. 21, 387–395 (2018).
Richards, P. et al. Social pathways for ebola virus disease in rural Sierra Leone, and some implications for containment. PLoS Negl. Trop. Dis. 9, e0003567 (2015).
Jalloh, M. F. et al. National survey of Ebola-related knowledge, attitudes and practices before the outbreak peak in Sierra Leone: August 2014. BMJ Glob. Health 2, e000285 (2017).
Bedford, J. Social science and behavioral data compilation, DRC Ebola outbreak, November 2018 - February 2019. Social Science in Humanitarian Action and GOARN Research Social Science Group (2019); https://opendocs.ids.ac.uk/opendocs/bitstream/handle/20.500.12413/14144/SSHAP_data_compilation_brief_November_2018_updated.pdf
Pinchoff, J. et al. Evidence-based process for prioritizing positive behaviors for promotion: Zika prevention in Latin America and the Caribbean and applicability to future health emergency responses. Glob. Health Sci. Pr. 7, 404–417 (2019).
Guirguis, S., Obregon, R., Coleman, M., Hickler, B. & SteelFisher, G. Placing human behavior at the center of the fight to eradicate polio: lessons learned and their application to other life-saving interventions. J. Infect. Dis. 216, S331–S336 (2017).
Research Guides: Social Science Data Resources: COVID-19 https://guides.library.yale.edu/covid19impacts (Accessed 17 April 2021).
Rohan, H., Bausch, D. G. & Blanchet, K. Action not justification: how to use social science to improve outbreak response. PLoS Blogs (2018); https://collectionsblog.plos.org/action-not-justification-how-to-use-social-science-to-improve-outbreak-response/
Bardosh, K. et al. Towards People-Centred Epidemic Preparedness and Response: From Knowledge to Action (Wellcome/DFID, 2019).
UNICEF Minimum Quality Standards and Indicators for Community Engagement. Guidance Towards High Quality, Evidence-Based Community Engagement in The Development and Humanitarian Contexts. (2020); https://www.unicef.org/mena/reports/community-engagement-standards
Hennessey Lavery, S. et al. The community action model: a community-driven model designed to address disparities in health. Am. J. Public Health 95, 611–616 (2005).
Boyce, M. R. & Katz, R. Community health workers and pandemic preparedness: current and prospective roles. Front. Public Health 7, 62 (2019).
Baggio, O. Real-Time Ebola Community Feedback Mechanism (SSHAP Case Study 10, UNICEF, IDS and Anthrologica, 2020).
Collective Communication and Community Engagement in Humanitarian Action: How to Guide for Leaders and Responders (CDAC Network, 2019).
Ackerman Gulaid, L. & Kiragu, K. Lessons learnt from promising practices in community engagement for the elimination of new HIV infections in children by 2015 and keeping their mothers alive: summary of a desk review. J. Int. AIDS Soc. 15, 17390 (2012).
Gilmore, B. et al. Community engagement for COVID-19 prevention and control: a rapid evidence synthesis. BMJ Glob. Health 5, e003188 (2020).
O’Mara-Eves, A. et al. The effectiveness of community engagement in public health interventions for disadvantaged groups: a meta-analysis. BMC Public Health 15, 129 (2015).
Milton, B., Attree, P., French, B., Povall, S. L. & Popay, J. The impact of community engagement on health and social outcomes: a systematic review. 47, 316–334 (2011).
Abramowitz, S. et al. Data Sharing in Public Health Emergencies: Anthropological and Historical Perspectives on Data Sharing During the 2014-2016 Ebola Epidemic and the 2016 Yellow Fever Epidemic (Wellcome Trust, 2018); https://www.glopid-r.org/wp-content/uploads/2019/07/data-sharing-in-public-health-emergencies-yellow-fever-and-ebola.pdf
Bedson, J. et al. Community engagement in outbreak response: lessons from the 2014-2016 Ebola outbreak in Sierra Leone. BMJ Glob. Health 5, e002145 (2020).
Jalloh, M. Design and implementation of an integrated digital system for community engagement and community-based surveillance during the 2014-2016 Ebola outbreak in Sierra Leone. BMJ Global Health 5, e003936 (2020).
McComas, K. A. Defining moments in risk communication research: 1996-2005. J. Health Commun. 11, 75–91 (2006).
Glik, D. C. Risk communication for public health emergencies. Annu. Rev. Public Health 28, 33–54 (2007).
WHO General Information on Risk Communication (2015).
Tworek, H., Beacock, I. & Ojo, E. Democratic health communications during Covid-19: a RAPID response (UBC Centre for the Study of Democratic Institutions, 2020); https://democracy.arts.ubc.ca/2020/09/14/covid-19/
Winters, M. et al. Risk communication and ebola-specific knowledge and behavior during 2014-2015 outbreak, Sierra Leone. Emerg. Infect. Dis. 24, 336–344 (2018).
Novetta. Social Media Analysis of ‘Tu vois Les Retombées’ Facebook Page (Insecurity Insight, 2020); http://insecurityinsight.org/wp-content/uploads/2020/06/Social-Media-Analysis-Novetta-June-2020.pdf
Ghenai, A. & Mejova, Y. Catching Zika Fever: Application of Crowdsourcing and Machine Learning for Tracking Health Misinformation on Twitter. Preprint at arXiv https://arxiv.org/abs/1707.03778 (2017).
Taggart, T., Grewe, M. E., Conserve, D. F., Gliwa, C. & Roman Isler, M. Social media and HIV: a systematic review of uses of social media in HIV communication. J. Med. Internet Res. 17, e248 (2015).
Smith, R. D. Responding to global infectious disease outbreaks: lessons from SARS on the role of risk perception, communication and management. Soc. Sci. Med. 63, 3113–3123 (2006).
Li, C. et al. Retrospective analysis of the possibility of predicting the COVID-19 outbreak from Internet searches and social media data, China, 2020. Euro Surveill. 25, 2000199 (2020).
Lu, Y. & Zhang, L. Social media WeChat infers the development trend of COVID-19. J. Infect. 81, e82–e83 (2020).
Effenberger, M. et al. Association of the COVID-19 pandemic with Internet search volumes: a Google TrendsTM Analysis. Int. J. Infect. Dis. 95, 192–197 (2020).
Gallotti, R., Valle, F., Castaldo, N., Sacco, P. & De Domenico, M. Assessing the risks of ‘infodemics’ in response to COVID-19 epidemics. Nat. Hum. Behav. 4, 1285–1293 (2020).
Bhattacharjee, S. & Dotto, C. Case study: understanding the impact of polio vaccine disinformation in Pakistan. First Draft (20 February 2020); https://firstdraftnews.org/long-form-article/first-draft-case-study-understanding-the-impact-of-polio-vaccine-disinformation-in-pakistan/
Krause, N. M., Freiling, I., Beets, B. & Brossard, D. Fact-checking as risk communication: the multi-layered risk of misinformation in times of COVID-19. J. Risk Res. 23, 1052–1059 (2020).
Eysenbach, G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J. Med. Internet Res. 11, e11 (2009).
Eysenbach, G. Infodemiology: the epidemiology of (mis)information. Am. J. Med. 113, 763–765 (2002).
Islam, M. S. et al. COVID-19-related infodemic and its impact on public health: a global social media analysis. Am. J. Trop. Med. Hyg. 103, 1621–1629 (2020).
Funk, S. et al. Nine challenges in incorporating the dynamics of behaviour in infectious diseases models. Epidemics 10, 21–25 (2015).
Davis, P. K., O’Mahony, A., Gulden, T. R., Sieck, K. & Osoba, O. A. Priority Challenges for Social and Behavioral Research and Its Modeling (RAND, 2018).
WHO Guidance For Managing Ethical Issues In Infectious Disease Outbreaks (2016).
Bruine de Bruin, W., Parker, A. M., Galesic, M. & Vardavas, R. Reports of social circles’ and own vaccination behavior: a national longitudinal survey. Health Psychol. 38, 975–983 (2019).
Facebook. COVID-19 Interactive Map & Dashboard (Accessed 14 April 2020); https://dataforgood.facebook.com/covid-survey/?region=WORLD
Pruyt, E., Auping, W. L. & Kwakkel, J. H. Ebola in west Africa: model-based exploration of social psychological effects and interventions: Ebola in West Africa. Syst. Res. Behav. Sci. 32, 2–14 (2015).
Schmidt-Hellerau, K. et al. Homecare for sick family members while waiting for medical help during the 2014-2015 Ebola outbreak in Sierra Leone: a mixed methods study. BMJ Glob. Health 5, e002732 (2020).
Baggio, O. Case Study, Real-Time Ebola Community Feedback Mechanism (Social Science in Humanitarian Action, 2020); https://core.ac.uk/download/pdf/326024204.pdf
WHO, UNICEF and IFRC. The Collective Service (2020); https://www.who.int/teams/risk-communication/the-collective-service
WHO. COVID-19 Knowledge Hub (2020); https://extranet.who.int/goarn/COVID19Hub
Giles-Vernick, T. et al. A new social sciences network for infectious threats. Lancet Infect. Dis. 19, 461–463 (2019).
Preventive Health Survey (Facebook, 2020); https://dataforgood.fb.com/tools/preventive-health-survey/
COVID-19 Community Mobility Reports (Google, 2020).
Badr, H. S. et al. Association between mobility patterns and COVID-19 transmission in the USA: a mathematical modelling study. Lancet Infect. Dis. 20, 1247–1254 (2020).
WHO. Early AI-supported Response with Social Listening (2020); https://whoinfodemic.citibeats.com/?cat=fYJ1oBNEUQtfbExrkGvsyr
WHO. Ebola or Marburg Case Investigation and Recording Sheet (16 June 2020); https://www.who.int/publications/m/item/ebola-or-marburg-case-investigation-and-recording-sheet
CDC. Investigating a COVID-19 Case (2020); https://www.cdc.gov/coronavirus/2019-ncov/php/contact-tracing/contact-tracing-plan/investigating-covid-19-case.html
WHO. Disease Case Investigation Forms (Accessed 14 April 2021); https://www.who.int/emergencies/outbreak-toolkit/data-collection-standards/disease-case-investigation-forms
Social Science Support for COVID-19: Lessons Learned Brief 1 (Cellule D’analyse en Sciences Sociales, 2020).
Rivers, C., Pollett, S. & Viboud, C. The opportunities and challenges of an Ebola modeling research coordination group. PLoS Negl. Trop. Dis. 14, e0008158 (2020).
WHO. Global Health Observatory (Accessed 14 April 2021); https://www.who.int/data/gho
Data Portal (RCCE Collective Service: Risk Communication and Community Engagement, 2020); https://www.rcce-collective.net/data/
Richards, P. Ebola: How a People’s Science Helped End an Epidemic (Zed Books, 2016).
Social Science in Humanitarian Action Platform Social Science in Humanitarian Action, Key Considerations: Engaging Twa communities in Equateur Province (2018).
Heesterbeek, H. et al. Modeling infectious disease dynamics in the complex landscape of global health. Science 347, aaa4339 (2015).
Skrip, L., Fallah, M. P., Bedson, J., Hébert-Dufresne, L. & Althouse, B. M. Coordinated support for local action: a modeling study of strategies to facilitate behavior adoption in urban poor communities of Liberia for sustained COVID-19 suppression. Preprint at medRxiv https://doi.org/10.1101/2020.08.11.20172031 (2020).
Online Database of Training on Social Dimensions of Infectious Threats (Sonar Global, Accessed 14 April 2021); https://www.sonar-global.eu/trainings/
OpenWHO. https://openwho.org (Accessed 14 April 2021).
Gwynn, S. Access to Research in the Global South: Reviewing the Evidence (International Network for the Availability of Scientific Publications, 2019).
Urassa, M. et al. Cross-cultural research must prioritize equitable collaboration. Nat. Hum. Behav. https://doi.org/10.1038/s41562-021-01076-x (2021).
Bonino, F., Jean, I. & Knox-Clarke, P. Closing the Loop: Effective Feedback in Humanitarian Contexts (ALNAP/ODI, 2014).
Metcalf, C. J. E., Edmunds, W. J. & Lessler, J. Six challenges in modelling for public health policy. Epidemics 10, 93–96 (2015).
Cobey, S. Modeling infectious disease dynamics. Science 368, 713–714 (2020).
Ordway, D.-M. Epidemiological Models: 10 Things to Know About Coronavirus Research (Harvard Kennedy School, 2020); https://journalistsresource.org/tip-sheets/research/epidemiological-models-coronavirus/
Knight, G. M. et al. Bridging the gap between evidence and policy for infectious diseases: how models can aid public health decision-making. Int. J. Infect. Dis. 42, 17–23 (2016).
We thank M. Izzo and D. Mistry for contributing to the figure. TG-V: SoNAR-Global project has received funding from the EU Horizon 2020 research and innovation program (grant agreement no. 825671). G.C. is partially supported by NSF grants (nos 2026797 and 2034003). N.G. reports funding from the EU Horizon 2020 research and innovation programme (grant agreement no. 101003589; RECOVER), Wellcome Trust and UK Foreign, Commonwealth and Development Office (FCDO) (grant agreement no. BZR02530) and UKRI/NIHR 2019-nCoV Rapid Response Call (grant no. NIHR200907). L.H.-D. acknowledges the National Institutes of Health 1P20 GM125498-01 Centers of Biomedical Research Excellence Award. J.B. acknowledges support as part of the ‘Data Modeling Community Engagement in Health Emergencies’ project funded by the Bill & Melinda Gates Foundation. S.F. was supported by the Wellcome Trust (no. 210758/Z/18/Z).
The authors declare no competing interests.
Peer review information Nature Human Behaviour thanks Jude Bayham and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Bedson, J., Skrip, L.A., Pedi, D. et al. A review and agenda for integrated disease models including social and behavioural factors. Nat Hum Behav 5, 834–846 (2021). https://doi.org/10.1038/s41562-021-01136-2