Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

# How epidemic psychology works on Twitter: evolution of responses to the COVID-19 pandemic in the U.S.

## Abstract

Disruptions resulting from an epidemic might often appear to amount to chaos but, in reality, can be understood in a systematic way through the lens of “epidemic psychology”. According to Philip Strong, the founder of the sociological study of epidemic infectious diseases, not only is an epidemic biological; there is also the potential for three psycho-social epidemics: of fear, moralization, and action. This work empirically tests Strong’s model at scale by studying the use of language of 122M tweets related to the COVID-19 pandemic posted in the U.S. during the whole year of 2020. On Twitter, we identified three distinct phases. Each of them is characterized by different regimes of the three psycho-social epidemics. In the refusal phase, users refused to accept reality despite the increasing number of deaths in other countries. In the anger phase (started after the announcement of the first death in the country), users’ fear translated into anger about the looming feeling that things were about to change. Finally, in the acceptance phase, which began after the authorities imposed physical-distancing measures, users settled into a “new normal” for their daily activities. Overall, refusal of accepting reality gradually died off as the year went on, while acceptance increasingly took hold. During 2020, as cases surged in waves, so did anger, re-emerging cyclically at each wave. Our real-time operationalization of Strong’s model is designed in a way that makes it possible to embed epidemic psychology into real-time models (e.g., epidemiological and mobility models).

## Introduction

In our daily lives, our dominant perception is of order. But every now and then chaos threatens that order: epidemics dramatically break out, revolutions erupt, empires suddenly fall, and stock markets crash. Epidemics, in particular, present not only collective health hazards but also special challenges to mental health and public order that need to be addressed by social and behavioral sciences (Van Bavel et al., 2020). Almost 30 years ago, in the wake of the AIDS epidemic, Philip Strong, the founder of the sociological study of epidemic infectious diseases, reflected: “the human origin of epidemic psychology lies not so much in our unruly passions as in the threat of epidemic disease to our everyday assumptions” (Strong, 1990). In the recent COVID-19 pandemic (Brooks et al., 2020) (an ongoing pandemic of a coronavirus disease), it has been shown that the main source of uncertainty and anxiety has indeed come from the disruption of what Alfred Shutz called the “routines and recipes” of daily life (Schutz et al., 1973) (e.g., every simple act, from eating at work to visiting our parents, takes on new meanings).

Yet, the chaos resulting from an epidemic turns out to be more predictable than what one would initially expect. Philip Strong observed that any new health epidemic resulted into three psycho-social epidemics: of fear, moralization, and action. The epidemic of fear represents the fear of catching the disease, which comes with the suspicion against alleged disease carriers, which, in turn, may spark panic and irrational behavior. The epidemic of moralization is characterized by moral responses both to the viral epidemic itself and to the epidemic of fear, which may result in either positive reactions (e.g., cooperation) or negative ones (e.g., stigmatization). The epidemic of action accounts for the rational or irrational changes of daily habits that people make in response to the disease or as a result of the two other psycho-social epidemics. Strong was writing in the wake of the AIDS/HIV crisis, but he based his model on studies that went back to Europe’s Black Death in the 14th century. Importantly, he showed that these three psycho-social epidemics are created by language and incrementally fed through it: language transmits the fear that the infection is an existential threat to humanity and that we are all going to die; language depicts the epidemic as a verdict on human failings and as a moral judgment on minorities; and language shapes the means through which people collectively intend to, however pointless, act against the threat.

There have been numerous studies of how information propagated on social media during epidemic outbreaks occurred in the past decade, such as Zika (Fu et al., 2016; Sommariva et al., 2018; Wood, 2018), Ebola (Oyeyemi et al., 2014), and the H1N1 influenza (Chew and Eysenbach, 2010). Similarly, with the outbreak of COVID-19, people around the world have been collectively expressing their thoughts and concerns about the pandemic on social media. As such, researchers studied this epidemic from multiple angles: social media posts have been analyzed in terms of content and behavioral markers (Hou, 2020; Li et al., 2020), and of tracking the diffusion of COVID-related information (Cinelli et al., 2020) and misinformation (Ferrara, 2020; Kouzy et al., 2020; Pulido et al., 2020; Yang et al., 2020). Search queries have suggested specific information-seeking responses to the pandemic (Bento et al., 2020). Psychological responses to COVID-19 have been studied mostly though surveys (Qiu et al., 2020; Wang et al., 2020).

Hitherto, there has never been any large-scale empirical study of whether the use of language during an epidemic reflects Strong’s model. With the opportunity of having a sufficiently large-scale data at hand, we set out to test whether Strong’s model did hold on Twitter during COVID-19, and did so at the scale of an entire country, that of the United States. By running our study on Twitter, we expose the interpretation of our results to a number of limitations, notably to issues of representativeness (Li et al., 2013), self-presentation biases (Waterloo et al., 2018), and data noise (Ferrara et al., 2016; Saif et al., 2012). Indeed, recent surveys estimated that only 22% of U.S. adults use Twitter (Wojcik and Hughes, 2019), and that the characteristics of these users deviate from the general population’s: compared to the average adult in the U.S., Twitter users are much younger, are more likely to have college degrees, and are slightly more likely to identify with the Democratic Party. Despite such limitations, social media represents the response of a relevant part of the general population to global events, and it does so at a scale and a granularity that have been so far unattainable by publicly available data sources.

After operationalizing Strong’s model by using lexicons for psycholinguistic text analysis, upon the collection of 122M tweets about the epidemic from 1 February to 31 December, we conducted a quantitative analysis on the differences in language style and a thematic analysis of the actual social media posts. The temporal scope of our study does not capture the full pandemic as it is still ongoing at the time of writing but it includes the three major contagion waves in the U.S., characterizing the entire year of 2020. The first wave captured the initial diffusion of the virus in the world, and then its arrival and first peak in the U.S. The following two waves captured subsequent periods of alarming diffusion.

The three psycho-social epidemics, as theorized by Strong, evolve concurrently over time. In the particular case of our period of study, we found that this concurrent evolution resulted into three regimes or phases, which are not part of Strong’s theoretical framework and experimentally emerged. In the first phase (the refusal phase), the psycho-social epidemic of fear began. Twitter users in the U.S. refused to accept reality: they feared the uncertainty created by the disruption of what was considered to be “normal”; focused their moral concerns on others in an act of distancing oneself from others; yet, despite all this, they refused to change the normal course of action. After the announcement of the first death in the country, the second phase (the anger phase) began: the psycho-social epidemic of fear intensified while the epidemics of morality and action kicked-off abruptly. Twitter users expressed more anger than fear about the looming feeling that things were about to change; focused their moral concerns on oneself in an act of reckoning with what was happening; and suspended their daily activities. After the authorities imposed physical-distancing measures, the third phase (the acceptance phase) took over: the epidemic of fear started to fade away while the epidemics of morality and action turned into more constructive and forward-looking social processes. Twitter users expressed more sadness than anger or fear; focused their moral concerns on the collective and, in so doing, promoted pro-social behavior; and found a “new normal” in their daily activities, which consisted of their past daily activities being physically restricted to their homes and neighborhoods. The phase of acceptance dominated Twitter conversations for the rest of the year, although the anger phase re-emerged cyclically with the rise of new waves of contagion. In particular, we observed two peaks of anger: when the death toll in the U.S. reached 100,000 people (second contagion wave), and when President Trump tested positive to COVID-19 (third contagion wave).

## Dataset

From an existing collection of COVID-related tweets (Chen et al., 2020), we gathered 554,941,519 tweets posted between 1 February up to 31 December. We focused our analysis on the United States, the country where Twitter penetration is highest. To identify Twitter users living in it, we parsed the free-text location description of their user profile (e.g., “San Francisco, CA”). We did so by using a set of custom regular expressions that match variations for the expression “United States of America”, as well as the names of 333 cities and 51 states in the U.S. (and their combinations). Albeit not always accurate, matching location strings against known location names is a tested approach that yields good results for a coarse-grained localization at state or country-level (Dredze et al., 2013). Overall, we were left with 6,271,835 unique users in the U.S. who posted 122,320,155 tweets in English.

Before analyzing how our language categories unfolded over time, we experimentally tested whether the number of data points at hand was sufficient to compute our metrics. It was indeed the case as the number of active users per day varied from a minimum of 72k on 2 February to a maximum of 1.84M on 18 March, with and average of 437k. A small number of accounts tweeted a disproportionately high number of times, reaching a maximum of 15,823 tweets; those were clearly automated accounts, which were discarded by our methodology. As we shall discuss in Methods, we normalized all our aggregate temporal measures so that they were not affected by the fluctuating volume of tweets over time.

## Methods

### Coding Strong’s model

Back in the 1990s, Philip Strong was able not only to describe the psychological impact of epidemics on social order but also to model it. He observed that the early reaction to major fatal epidemics is a distinctive psycho-social form and can be modeled along three main dimensions: fear, morality, and action. During a large-scale epidemic, basic assumptions about social interaction and, more generally, about social order are disrupted, and, more specifically, they are so by: the fear of others, competing moralities, and the responses to the epidemic. Crucially, all these three elements are created, transmitted, and mediated by language: language transmits fears, elaborates on the stigmatization of minorities, and shapes the means through which people collectively respond to the epidemic (Cap, 2016; Goffman, 2009; Strong, 1990).

As opposed to existing attempts to model psychological and social aspects of epidemic crises (Khan and Huremović, 2019; McConnell, 2005), Strong’s model meets our three main choice criteria:

1. (1)

Well-grounded: It proposes a comprehensive, highly cited, and still relevant theoretical model, which is based on an extensive review of studies of past large-scale epidemics that did span centuries and, as such, were of different nature, speaking to the generalizability of the framework.

2. (2)

Focused on psycho-social aspects: Its main goal is to characterize people’s psychological and social responses to epidemics rather than describing how an epidemic unfolds over time.

3. (3)

Directly operationalizable from language use: The description of the psycho-social responses provided by Strong lends itself to operationalization, as its defining concepts have been mapped to language markers by previous literature (as Table 1 shows).

We operationalized Strong’s epidemic psychology theoretical framework in two steps. First, three authors hand-coded Strong’s seminal paper (Strong, 1990) using line-by-line coding (Gibbs, 2007) to identify keywords that characterize the three psycho-social epidemics. For each of the three psycho-social epidemics, the three authors generated independent lists of keywords that were conservatively combined by intersecting them. The words that were left out by the intersection were mostly synonyms (e.g., “catching disease” as a synonym for “contagion”), so we did not discard any important concept. According to Strong, the three psycho-social epidemics are intertwined and, as such, the concepts that define one specific psycho-social epidemic might be relevant to the remaining two as well. For example, suspicion is an element of the epidemic of fear but is tightly related to stigmatization as well, a phenomenon that Strong describes as typical of the epidemic of moralization. In our coding exercise, we adhered as much as possible to the description in Strong’s paper and obtained a strict partition of keywords across psycho-social epidemics. In the second step, the same three authors mapped each of these keywords to language categories, namely sets of words that reflect how these concepts are expressed in natural language (e.g., words expressing anger or trust). We took these categories from four existing language lexicons widely used in psychometric studies:

• Linguistic Inquiry Word Count (LIWC) (Tausczik and Pennebaker, 2010): A lexicon of words and word stems grouped into over 125 categories reflecting emotions, social processes, and basic functions, among others. The LIWC lexicon is based on the premise that the words people use to communicate can provide clues to their psychological states (Tausczik and Pennebaker, 2010). It allows written passages to be analyzed syntactically (how the words are used together to form phrases or sentences) and semantically (an analysis of the meaning of the words or phrases).

• Emolex (Mohammad and Turney, 2013): A lexicon that classifies 6k+ words and stems into the eight primary emotions of Plutchik’s psychoevolutionary theory (Plutchik, 1991).

• Moral Foundation Lexicon (Graham et al., 2009): A lexicon of 318 words and stems, which are grouped into five categories of moral foundations (Graham et al., 2013): harm, fairness, in-group, authority, and purity. Each of which is further split into expressions of virtue or vice.

• Pro-social behavior (Frimer et al., 2014): A lexicon of 146 pro-social words and stems, which have been found to be frequently used when people describe pro-social goals (Frimer et al., 2014).

The three authors grouped similar keywords together and mapped groups of keywords to one or more language categories. This grouping and mapping procedure was informed by previous studies that investigated how these keywords are expressed through language. These studies are listed in Table 1.

### Language categories over time

We considered that a tweet contained a language category c if at least one of the tweet’s words or stems belonged to that category. The tweet-category association is binary and disregards the number of matching words within the same tweet. That is mainly because, in short snippets of text (tweets are limited to 280 characters), multiple occurrences are rare and do not necessarily reflect the intensity of a category (Russell, 2013). For each language category c, we counted the number of users Uc(t) who posted at least one tweet at time t containing that category. We then obtained the fraction of users who mentioned category c by dividing Uc(t) by the total number of users U(t) who tweeted at time t:

$${f}_{c}(t)=\frac{{U}_{c}(t)}{U(t)}.$$
(1)

Computing the fraction of users rather than the fraction of tweets prevents biases introduced by exceptionally active users, thus capturing more faithfully the prevalence of different language categories in our Twitter population. This also helps discounting the impact of social bots, which tend to have anomalous levels of activity (especially retweeting (Bessi and Ferrara, 2016)).

Different categories might be verbalized with considerably different frequencies. For example, the language category “I” (first-person pronoun) from the LIWC lexicon naturally occurred much more frequently than the category “death” from the same lexicon. To enable a comparison across categories, we standardized all the fractions:

$${z}_{c}(t)=\frac{{f}_{c}(t)-{\mu }_{[0,T]}({f}_{c})}{{\sigma }_{[0,T]}({f}_{c})},$$
(2)

where μ(fc) and σ(fc) represent the mean and standard deviation of the fc(t) scores over the whole time period, from t = 0 (1 February) to t = T (16 April). These z-scores ease also the interpretation of the results as they represent the relative variation of a category’s prevalence compared to its average: they take on values higher (lower) than zero when the original value is higher (lower) than the average.

### Other behavioral markers

To assess the validity of our operationalization of Strong’s model, we compared its results with the output of alternative state-of-the-art text-mining techniques, and with real-world mobility patterns.

#### Interaction types

We compared the results obtained via word-matching with a state-of-the-art deep learning tool for Natural Language Processing designed to capture fundamental types of social interactions from conversational language (Choi et al., 2020). This tool uses Long Short-Term Memory neural networks (LSTMs) (Hochreiter and Schmidhuber, 1997) that take in input a 300-dimensional GloVe representation of words (Pennington et al., 2014) and output a series of confidence scores in the range [0, 1] that estimate the likelihood that the text expresses certain types of social interactions. The classifiers exhibited a very high classification performance, up to an Area Under the ROC Curve (AUC) of 0.98. AUC is a performance metric that measures the ability of the model to assign higher confidence scores to positive examples (i.e., text characterized by the type of interaction of interest) than to negative examples, independent of any fixed decision threshold; the expected value for random classification is 0.5, whereas an AUC of 1 indicates a perfect classification.

Out of the 10 interaction types that the tool can classify (Deri et al., 2018), three were detected frequently with likelihood > 0.5 in our Twitter data: conflict (expressions of contrast or diverging views (Tajfel et al., 1979)), social support (giving emotional or practical aid and companionship (Fiske et al., 2007)), and power (expressions that mark a person’s power over the behavior and outcomes of another (Blau, 1964)).

Given a tweet’s textual message m and an interaction type i, we used the classifier to compute the likelihood score li(m) that the message contains that interaction type. We then binarized the confidence scores using a threshold-based indicator function:

$${l}_{i}^{\theta }(m)=\left\{\begin{array}{ll}1,&\,\text{if}\,\,{l}_{i}{\it(m)}\ge {\theta }_{i}\\ 0,&\,\text{otherwise}\,\end{array}\right.$$
(3)

Following the original approach (Choi et al., 2020), we used a different threshold for each interaction type, as the distributions of their likelihood scores tend to vary considerably. We thus picked conservatively θi as the value of the 85th percentile of the distribution of the confidence scores li, thus favoring precision over recall. Last, similar to how we constructed temporal signals for the language categories, we counted the number of users Ui(t) who posted at least one tweet at time t that contains interaction type i. We then obtained the fraction of users who mentioned interaction type i by dividing Ui(t) by the total number of users U(t) who tweeted at time t:

$${f}_{i}(t)=\frac{{U}_{i}(t)}{U(t)}.$$
(4)

Last, we min−max normalized these fractions, considering the minimum and maximum values during the whole time period [0, T]:

$${f}_{i}(t)=\frac{{f}_{i}(t)-\mathop{\min }\limits_{[0,T]}({f}_{i})}{\mathop{\max }\limits_{[0,T]}({f}_{i})-\mathop{\min }\limits_{[0,T]}({f}_{i})}.$$
(5)

#### Mentions of medical entities

We used a state-of-the-art deep learning method for medical entity extraction to identify medical symptoms on Twitter in relation to COVID-19 (Scepanovic et al., 2020). When applied to tweets, the method extracts n-grams representing medical symptoms (e.g., “feeling sick”). This method is based on the Bi-LSTM sequence-tagging architecture (Huang et al., 2015) in combination with GloVe word embeddings (Pennington et al., 2014) and RoBERTa contextual embeddings (Liu et al., 2019). To optimize the entity extraction performance on noisy textual data from social media, we trained its sequence-tagging architecture on the Micromed database (Jimeno-Yepes et al., 2015), a collection of tweets manually labeled with medical entities. The hyper-parameters we used are: 256 hidden units, a batch size of 4, and a learning rate of 0.1 which we gradually halved whenever there was no performance improvement after 3 epochs. We trained for a maximum of 200 epochs or before the learning rate became too small (≤0.0001). The final model achieved an F1-score of 0.72 on Micromed. The F1-score is a performance measure that combines precision (the fraction of extracted entities that are actually medical entities) and recall (the fraction of medical entities present in the text that the method is able to retrieve). We based our implementation on Flair (Akbik et al., 2019) and Pytorch (Paszke et al., 2017), two popular deep learning libraries in Python.

For each unique medical entity e, we counted the number of users Ue(t) who posted at least one tweet at time t that mentioned that entity. We then obtained the fraction of users who mentioned medical entity e by dividing Ue(t) by the total number of users U(t) who tweeted at time t:

$${f}_{e}(t)=\frac{{U}_{e}(t)}{U(t)}.$$
(6)

Last, we min–max normalize these fractions, considering the minimum and maximum values during the whole time period [0, T]:

$${f}_{e}(t)=\frac{{f}_{e}(t)-\mathop{\min }\limits_{[0,T]}({f}_{e})}{\mathop{\max }\limits_{[0,T]}({f}_{e})-\mathop{\min }\limits_{[0,T]}({f}_{e})}.$$
(7)

#### Mobility traces

Foursquare is a local search and discovery mobile application that relies on the users’ past mobility records to recommend places user might may like. The application uses GPS geo-localization to estimate the user position and to infer the places they visited. In response to the COVID-19 crisis, Foursquare made publicly available the data gathered from a pool of 13 million U.S. users. These users were “always-on” during the period of data collection, meaning that they allowed the application to gather geo-location data at all times, even when the application was not in use. The data (published through the visitdata.orgwebsite) consists of the daily number of users vs,j visiting any venue of type j in state s, starting from 1 February to the present day (e.g., 419,256 users visited schools in Indiana on 1 February). Overall, 35 distinct location categories were provided. To obtain country-wide temporal indicators, we first applied a min-max normalization to the vs,j values:

$${v}_{s,j}(t)=\frac{{v}_{s,j}(t)-\mathop{\min }\limits_{[0,T]}({v}_{s,j})}{\mathop{\max }\limits_{[0,T]}({v}_{s,j})-\mathop{\min }\limits_{[0,T]}({v}_{s,j})}.$$
(8)

We then averaged the values across all states:

$${v}_{j}(t)=\frac{1}{S}{\sum }_{s}{v}_{s,j}(t),$$
(9)

where S is the total number of states. By weighting each state equally, we obtained a measure that is more representative of the whole U.S. territory, rather than being biased towards high-density regions.

### Time series smoothing

All our temporal indicators are affected by large day-to-day fluctuations. To extract more consistent trends out of our time series, we applied a smoothing function—a common practice when analyzing temporal data extracted from social media (O’Connor et al., 2010). Given a time-varying signal x(t), we apply a “boxcar” moving average over a window of the previous k days:

$${x}^{* }(t)=\frac{\mathop{\sum }\nolimits_{i = t-k}^{t}x(i)}{k};$$
(10)

We selected a window of one week (k = 7). Weekly time windows are typically used to smooth out both day-to-day variations as well as weekly periodicities (O’Connor et al., 2010). We applied the smoothing to all the time series: the language categories ($${z}_{c}^{* }(t)$$), the mentions of medical entities ($${f}_{e}^{* }(t)$$), the interaction types ($${f}_{i}^{* }(t)$$), and the foursquare visits ($${v}_{j}^{* }(t)$$).

### Change-point detection

To identify phases characterized by different combinations of the language categories, we identified change-points—periods in which the values of all categories varied considerably at once. To quantify such variations, for each language category c, we computed $$\nabla ({z}_{c}^{* }(t))$$, namely the daily average squared gradient (Lütkepohl, 2005) of the smoothed standardized fractions of that category. To calculate the gradient, we used the Python function numpy.gradient. The gradient provides a measure of the rate of increase or decrease of the signal; we consider the absolute value of the gradient, to account for the magnitude of change rather than the direction of change. To identify periods of consistent change as opposed to quick instantaneous shifts, we apply temporal smoothing (Eq. (10)) also to the time-series of gradients, and we denote the smoothed squared gradients with *. Last, we average the gradients of all language categories to obtain the overall gradient over time:

$$\nabla (t)=\frac{1}{D}\mathop{\sum }\limits_{d}{\nabla }^{* }({z}_{c}^{* }(t)).$$
(11)

Peaks in the time series (t) represent the days of highest variation, and we marked them as change-points. Using the Python function scipy.signal.find_peaks, we identified peaks as the local maxima whose values is higher than the average plus two standard deviations, as it is common practice (Palshikar, 2009).

## Results

### Language use until the first contagion wave

During the first wave, U.S. residents experienced what the pandemic entailed for the first time, and did so by going through an entire life-cycle, making this contagion wave self-contained: arrival of an unknown virus, skepticism, isolation measures, full lock-down, and first reopening. Figure 1A–C shows how the standardized fractions of all the language categories as per Formula (2) changed from 1 February to 15 April, the day in which restrictions in most states were lifted. The cell color encodes values higher than the average in red, and lower in blue. We partitioned the language categories according to the three psycho-social epidemics. Figure 1D shows the value of the average squared gradient over time (as per Formula (11)); peaks in the curve represent the days of high local variation. We marked the peaks above two standard deviations from the mean as change-points. We found two change-points that coincide with two key events: 27 February, the day of the announcement of the first infection in the country; and 24 March, the day of the announcement of the ‘stay at home’ orders. These change-points identify three phases, which are described next by dwelling on the peaks of the different language categories (days when their standardized fractions reached the maximum) and reporting the percentage increase at peak (the increase is compared to the average from 1 February to 15 April, and its peak is denoted by ‘max peak’ in Table 1). The first phase (refusal phase) was characterized by anxiety and fear. Death was frequently mentioned, with a peak on 11 February of +45% compared to its average during the whole time period. The pronoun they was used in this temporal state more than average; this suggests that the focus of discussion was on the implications of the viral epidemic on ‘others’, as this was when no infection had been discovered in the U.S. yet. All other language categories exhibited no significant variations, which reflected an overall situation of ‘business-as-usual.’

The second phase (anger phase) began on 27 February with an outburst of negative emotions (predominantly anger), right after the first COVID-19 contagion in the U.S. was announced. The abstract fear of death was replaced by expressions of concrete health concerns, such as words expressing risk, and mentions of how body parts did feel. On 13 March, the federal government announced the state of national emergency, followed by the enforcement of state-level ‘stay at home’ orders. During those days, we observed a sharp increase of the use of the pronounIand of swear words (with a peak of +54% on 18 March), which hints at a climate of discussion characterized by conflict and polarization. At the same time, we observed an increase in the use of words related to the daily habits affected by the impending restriction policies, such as motion, social activities, and leisure. The mentions of words related to home peaked on 16 March (+38%), the day when the federal government announced social distancing guidelines to be in place for at least two weeks.

The third phase (acceptance phase) started on 24 March, the day after the first physical-distancing measures were imposed by law. The increased use of words of power and authority likely reflected the emergence of discussion around the new policies enforced by government officials and public agencies. As the death toll raised steadily—hitting the mark of 1000 deaths on 26 March—expressions of conflict faded away, and words of sadness became predominant. In those days of hardship, a sentiment of care for others and expressions of prosocial behavior became more frequent (+19% and +25%, respectively). Last, mentions of work-related activities peaked as many people either lost their job, or were compelled to work from home as result of the lockdown.

### Thematic analysis

The language categories capture broad concepts related to Strong’s epidemic psychology theory, but they do not allow for an analysis of the fine-grained topics within each category. To study them, for each of the 87 combinations of language category and phase (29 language categories, for 3 phases), we listed the 100 most retweeted tweets (e.g., most popular tweets containing anxiety posted in the refusal phase). To identify overarching themes, we followed two steps that are commonly adopted in thematic analysis (Braun and Clarke, 2006; Smith and Shinebourne, 2012). We first applied open coding to identify key concepts that emerged across multiple tweets; specifically, one of the authors read all the tweets and marked them with keywords that reflected the key concepts expressed in the text. We then used axial coding to identify relationships between the most frequent keywords to summarize them in semantically cohesive themes. Themes were reviewed in a recursive manner rather than linear, by re-evaluating and adjusting them as new tweets were parsed. Table 2 summarizes the most recurring themes, together with some of their representative tweets. In the refusal phase, statements of skepticism were re-tweeted widely (Table 2, row 1). The epidemic was frequently depicted as a “foreign” problem (r. 2) and all activities kept business as usual (r. 3).

In the anger phase, the discussion was characterized by outrage against three main categories: foreigners—especially Chinese individuals, as Supplementary Materials detail—(r. 4), political opponents (r. 5), and people who adopted different behavioral responses to the outbreak (r. 6). This level of conflict corroborates Strong’s postulate of the “war against each other”. Science and religion were two prominent topics of discussion. A lively debate raged around the validity of scientists’ recommendations (r. 7). Some social groups put their hopes on God rather than on science (r. 8). Mentions of people self-isolating at home became very frequent, and highlighted the contrast between judicious individuals and careless crowds (r. 9).

Finally, during the acceptance phase, the outburst of anger gave in to the sorrow caused by the mourning of thousands of people (r. 10). By accepting the real threat of the virus, our Twitter users were more open to find collective solutions to the problem and overcome fear with hope (r. 11). Although the positive attitude towards the authorities seemed prevalent, some users expressed disappointment against the restrictions imposed (r. 12). Those who were isolated at home started imagining a life beyond the isolation, especially in relation to reopening businesses (r. 13).

### Comparison with other behavioral markers

To assess the validity of our approach, we compared the previous results with the output of alternative text-mining techniques applied to the same data (internal validity), and with real-world mobility traces (external validity).

#### Comparison with other text mining techniques

We processed the very same social media posts with three alternative text-mining techniques (Fig. 1E–G). In Table 3, we reported the three language categories with the strongest correlations with each behavioral marker.

First, to allow for interpretable and explainable results, we applied a simple word-matching method that relies on a custom lexicon containing three categories of words reflecting consumption of alcohol, physical exercising, and economic concerns, as those aspects have been found to characterize the COVID-19 pandemic (The Economist, 2020). We measured the daily fraction of users mentioning words in each of those categories (Fig. 1E). In the refusal phase, the frequency of any of these words did not significantly increase. In the anger phase, the frequency of words related to economy peaked, and that related to alcohol consumption peaked shortly after that. Table 3 shows that economy-related words were highly correlated with the use of anxiety words (r = 0.73), which is in line with studies indicating that the degree of apprehension for the declining economy was comparable to that of health-hazard concerns (Bareket-Bojmel et al., 2020; Fetzer et al., 2020). Words of alcohol consumption were most correlated with the language dimensions of body (r = 0.70), feel (r = 0.62), home (r = 0.58); in the period were health concerns were at their peak, home isolation caused a rising tide of alcohol use (Da et al., 2020; Finlay and Gilmore, 2020). Finally, in the acceptance phase, the frequency of words related to physical exercise was significant; this happened at the same time when the use of positive words expressing togetherness was at its highest—affiliation (r = 0.95), posemo (r = 0.93), we (r = 0.92). All these results match our previous interpretations of the peaks for our language categories.

Second, since it is unclear whether a simple word count approach is effective in studying how the three psycho-social epidemics unfolded over time, we additionally applied the deep-learning approach that extracts mentions of expressions of conflict, social support, and power. Figure 1F shows the min-max normalized scores of the fraction of users posting tweets labeled with each of these three interaction types (as per Formula (5)). In the refusal phase, conflict increased—this is when anxiety and blaming foreigners were recurring themes in Twitter. In the anger phase, conflict peaked (similar to anxiety words, r = 0.88), yet, since the first lock-down measures were announced, initial expressions of power and of social support gradually increased as well. Finally, in the acceptance phase, social support peaked. Support was most correlated with the categories of affiliation (r = 0.98), positive emotions (r = 0.96), and we (r = 0.94) (Table 3); power was most correlated with prosocial (r = 0.95), care (r = 0.94), and authority (r = 0.94). Again, our previous interpretations concerning the existence of a phase of conflict followed by a phase of social support were further confirmed by the deep-learning tool, which, as opposed to our dictionary-based approaches, does not rely on word matching.

Third, we used the deep-learning tool that extracts mentions of medical entities from text (Scepanovic et al., 2020). Out of all the entities extracted, we focused on the 100 most frequently mentioned and grouped them into two families of symptoms, respectively, those related to physical health (e.g., “fever”, “cough”, “sick”) and those related to mental health (e.g., “depression”, “stress”) (Brooks et al., 2020). The min−max normalized fractions of users posting tweets containing mentions of these symptoms (as per Formula (7)) are shown in Fig. 1G. In refusal phase, the frequency of symptom mentions did not change. In the anger phase, instead, physical symptoms started to be mentioned, and they were correlated with the language categories expressing panic and physical health concerns—swear (r = 0.83), feel (r = 0.77), and negate (r = 0.67). In the acceptance phase, mentions of mental symptoms became most frequent. Interestingly, mental symptoms peaked when the Twitter discourse was characterized by positive feelings and prosocial interactions—affiliation (r = 0.91), we (r = 0.88), and posemo (r = 0.85); this is in line with recent studies that found that the psychological toll of COVID-19 has similar traits to post-traumatic stress disorders and its symptoms might lag several weeks from the period of initial panic and forced isolation (Dutheil et al., 2020; Galea et al., 2020; Liang et al., 2020).

#### Comparison with mobility traces

To test for the external validity of our language categories, we compared their temporal trends with the mobility data from Foursquare. We picked three venue categories: Grocery shops, Travel & Transport, and Outdoors & Recreation to reflect three different types of fundamental human needs (Maslow, 1943): the primary need of getting food supplies, the secondary need of moving around freely (or to limit mobility for safety), and the higher-level need of being entertained. In Fig. 1H, we show the min-max normalized number of visits over time (as per Formula (9)). The periods of higher variations of the normalized number of visits match the transitions between the three phases. In the refusal phase, mobility patterns did not change. In the anger phase, instead, travel started to drop, and grocery shopping peaked, supporting the interpretation of a phase characterized by a wave of panic-induced stockpiling and a compulsion to save oneself—it co-occurred with the peak of use of the pronoun I (r = 0.80)—rather than helping others. Finally, in the acceptance phase, the panic around grocery shopping faded away, and the number of visits to parks and outdoor spaces increased.

### Embedding epidemic psychology in real-time models

To embed our operationalization of epidemic psychology into real-time models (e.g., epidemiological models, urban mobility models), our measures need to work at any point in time during a new pandemic; yet, given their current definitions, they do not: that is because they are normalized values over the whole period of study (Fig. 1A–C). To fix that, we designed a new composite measure that does not rely on full temporal knowledge, and a corresponding detection method that determines which of the three phases one is in at any given point in time.

First, for each language category c, we computed the average value of fc (as per Formula (1)) during the first day of the epidemic, specifically μ[0, 1](fc). During the first day, 86k users tweeted. We experimented with longer periods (up to a week and 0.4M users), and obtained qualitatively similar results. We used the averages computed on this initial period as reference values for later measurements. The assumption behind this approach is that the modeler would know the set of relevant hashtags in the initial stages of the pandemic, which is reasonable considering that this was the case for all the major pandemics occurred in the last decade (Chew and Eysenbach, 2010; Fu et al., 2016; Oyeyemi et al., 2014). Starting from the second day, we then calculated the percent change of the fc values compared to the reference values:

$${{\Delta }}{ \% }_{c}(t)=\frac{{f}_{c}(t)-{\mu }_{[0,1]}({f}_{c})}{{\mu }_{[0,1]}({f}_{c})}.$$
(12)

For each phase, we defined a parsimonious measure composed only of two dimensions: the dimension most positively associated with the phase (expressed in percent change) minus that most negatively associated with it (e.g., (death-I) for the refusal phase). To identify such dimensions, we trained three logistic regression binary classifiers (one per phase). For each phase, we marked with label 1 all the days that were included in that phase and with 0 those that were not. Then, we trained a classifier to estimate the probability $${P}_{\mathrm {{phas{e}}}_{i}}(t)$$ that day t belongs to phase i, out of the Δ%c(t) values for all categories. During training, the logistic regression learned coefficients for each of the categories. On average, the classifiers were able to identify the correct phase for 98% of the days.

The regressions coefficients were then used to rank the language category by their predictive power. Table 4 shows the top three positive beta coefficients and bottom three negative ones for each of the three phases. The top and bottom categories of all phases belong to the LIWC lexicon. For each phase, we subtracted the top category from the bottom category without considering their beta coefficients, as these would require, again, full temporal knowledge:

$$\begin{array}{ll}&{{\Delta }}{ \% }_{Refusal}={{\Delta }}{ \% }_{death}-{{\Delta }}{ \% }_{I}\\ &{{\Delta }}{ \% }_{Anger}={{\Delta }}{ \% }_{swear}-{{\Delta }}{ \% }_{death}\\ &{{\Delta }}{ \% }_{Acceptance}={{\Delta }}{ \% }_{sad}-{{\Delta }}{ \% }_{anxiety}\end{array}$$
(13)

The resulting composite measure has the same change-points (Fig. 3A) as the full-knowledge measure’s (Fig. 1), suggesting that the real-time and parsimonious computation does not compromise the original trends. In a real-time scenario, transition between phases are captured by the changes of the dominant measure; for example, when the refusal curve is overtaken by the anger curve. In addition, we correlated the composite measures with each of the behavioral markers we used for validation (Fig. 1E–H) to find which are the markers most typically associated with each of the phases. We reported the correlations in Table 3. During the refusal phase, conflictual interactions were frequent (r = 0.58) and long-range mobility was common (r = 0.62); during the anger phase, as mobility reduced (Engle et al., 2020; Gao et al., 2020), some people hoarded groceries and alcohol (Da et al., 2020; Finlay and Gilmore, 2020) and expressed concerns for their physical health (r = 0.81) and for the economy (Bareket-Bojmel et al., 2020; Fetzer et al., 2020); last, during the acceptance phase, some people ventured outdoors, started exercising more, and expressed a stronger will to support each other (r = 0.90), in the wake of a rising tide of deaths and mental health symptoms (r = 0.85) (Dutheil et al., 2020; Galea et al., 2020; Liang et al., 2020).

### Language use after the first contagion wave

After the first wave and the “stay-at-home” lock-down at the end of March, for the remaining part of the year, there were two other contagion waves (Fig. 2A): one at the beginning of June, and the other at the beginning of October. In a way similar to the first wave, these two others were both associated with significant changes in the use of language (Fig. 2B): we had two change peaks for the second wave (i.e., 27 May and 9 June), and one for the third (2 October). This comes at no surprise as these two periods corresponded to two widely discussed events: the first 100K deaths in the U.S., and President Donald Trump testing positive for COVID-19. In particular, Fig. 3B shows that these changes were both to do with the rumping up of the categories associated with the anger phase: discussions of changes in mobility, and posts characterized by anger and, more generally, by negative emotions were predominant. By contrast, the categories associated with refusal and acceptance diverged from each other: unsurprisingly, throughout the year, refusal gradually died off, while acceptance increasingly took hold. Overall, we observed two classes of pattern (Fig. 3B). First, the three phases were not always orthogonal but blended together at times. During the second contagion wave, for example, anger and acceptance were both predominant, and had been so for several months. Second, the use of language had a cyclical nature. During each contagion wave, three consecutive local peaks (local maxima) were observed: refusal first, then anger, finally acceptance. This can be observed for all the three contagion waves. Such a cyclical nature was also reflected in our behavioral markers (Fig. 4E–H). For each wave, mentions of conflict peaked to then be followed by mentions of support and power (Fig. 4F). The same went for medical conditions: mentions of physical health peaked to then be followed by mentions of mental health (Fig. 4G).

## Discussion

### Findings beyond Strong’s model

Strong’s theory offers a framework upon which to operationalize the three psycho-social epidemics from the use of language but does not specifically describe how these epidemics unfold. Based on our data-driven results, we confirmed that the three epidemics are indeed present in our social conversations spanning almost one year, and they unfolded in ways that allowed us to enrich what Strong initially hypothesized in relation to four main aspects.

First, Strong’s theory predicts the presence of the three psycho-social epidemics but does not describe how they would be related to each other over time. We found that the three epidemics, as expressed by the relative presence of their relevant language categories, simultaneously raised and fell over time, and did so in relation to one another, demarcating three specific temporal phases. Over time, we identified three specific combinations of the epidemics that generated three phases our Twitter users went through: an initial refusal phase, an anger phase, and a final acceptance phase. Since these temporal phases partly resemble the stages of grief by Kübler-Ross et al. (1972), a promising direction for future work is to explore the relationship between these phases and the stages of grief.

Second, Strong’s narration slightly hints at a typical sequence of events according to which the epidemic of fear activates before that of morality, which is then followed by that of action. We indeed observed a similar set of events yet these events were not strictly sequential but rather cyclical. Interestingly, every new cycle started in conjunction with the same specific event: the diffusion rate of the virus reaching a local maximum. Shortly after every sharp increase in the diffusion rate, a cycle of refusal, anger, and acceptance unfolded among our Twitter users.

Third, Strong’s framework does not make any explicit distinction between the initial stages of an epidemic and its final stages. We found two regimes that considerably separated the initial stages from the later stages: in the initial cycle, the variation of the three epidemics was of a larger magnitude than those in the two subsequent cycles.

Last, Strong’s theory should be used as a descriptive framework and, as such, cannot be used to explore short-term variations. By contrast, a data-driven experimental work such as ours is able to point out when such variations potentially took place, as Fig. 4 shows. In future work, these points of change could be subject to qualitative inquiry, which might well enrich the original formulation of the theory.

### Implications

New infectious diseases break out abruptly, and public health agencies try to rely on detailed planning yet often find themselves to improvise around their playbook. They are constantly confronting not only the health epidemic but also the three psycho-social epidemics. Measuring the effects of epidemics on societal dynamics and population mental health has been an open research problem for a long time, and multidisciplinary approaches have been called for (Holmes et al., 2020). We contributed to this line of research by operationalizing Strong’s model and successfully testing it on Twitter in the U.S. Since our methodology can be applied to any textual data, future work may well study alternative (even cross-cultural) population segments. Since our language categories are not tailored to a specific epidemic (e.g., they do not reflect any specific symptom an epidemic is associated with), our approach can be applied to a future epidemic, provided that the set of relevant hashtags associated with the epidemic is known; this is a reasonable assumption to make though, considering that the consensus on Twitter hashtags is reached quickly (Baronchelli, 2018), and that several epidemics that occurred in the last decade sparked discussions on Twitter since their early days (Chew and Eysenbach, 2010; Fu et al., 2016; Oyeyemi et al., 2014).

Our method complements the numerous cross-sectional studies on the psychological impact of health epidemics conducted on representative population samples (Brooks et al., 2020; Shultz et al., 2015), not least because it collects real-time statistics of implicit behavioral signals, which are orthogonal to survey responses.

For computer science researchers, our method could provide a starting point for developing more sophisticated tools for monitoring psycho-social epidemics. Furthermore, from the theoretical standpoint, our work provides the first operationalization of Strong’s model of the epidemic psychology, and widens its theoretical implications by observing cyclical phases of diffusion of the psycho-social epidemics.

Finally, the ability to systematically characterize the three psycho-social epidemics from the use of language on social media makes it possible to embed epidemic psychology into models currently used to tackle epidemics such as mobility models (Bansal et al., 2016). To see how, consider that, in digital epidemiology (Bauch and Galvani, 2013; Salathe et al., 2012), some parameters of epidemic models are initialized or adjusted based on a variety of digital data to account for co-determinants of the spreading process that are hard to quantify with traditional data sources, especially in the first stages of the outbreak. This is particularly useful when modeling social and psychological processes such as risk perception (Bagnoli et al., 2007; Moinet et al., 2018). Interestingly, these approaches are designed to deal with partial data, therefore they can benefit even from digital data that is incomplete and—like in the case of our Twitter-based study—not necessarily representative of the whole population (Salathe et al., 2012).

### Limitations

Future work could improve our work in five main aspects. First, we focused only on one viral epidemic, without being able to compare it to others. Yet, if one were to obtain past social media data during the outbreaks of diseases like Zika (Fu et al., 2016), Ebola (Oyeyemi et al., 2014), and the H1N1 influenza (Chew and Eysenbach, 2010), one could apply our methodology in those contexts as well, and identify similarities and differences. For example, one could study how mortality rates or speed of spreading influence the representation of Strong’s epidemic psychology on social media.

Second, our geographical focus was the entire United States and, as such, was coarse and limited in scope. In Supplementary Materials, we broke-down the analysis of temporal phases for individual states in the U.S., and observed no substantial differences across states. In the future, one could conduct a more systematic analysis on a finer geographical granularity, relate differences between states to known events (e.g., a governor’s decisions, prevalence of cases, media landscape, and residents’ cultural traits). In particular, recent studies suggested that the public reaction to COVID-19 varied across the U.S. states depending on their political leaning (Grossman et al., 2020; Painter and Qiu, 2020). One could also apply our methodology to other English-speaking countries, to investigate how cultural dimensions (Hofstede et al., 2005) and cross-cultural personality trait variations (Bleidorn et al., 2013) might influence the three psycho-social epidemics.

Third, the three psycho-social epidemics were not always orthogonal to each other but did blend together at times. Future work could focus on those particular periods of time to determine whether the use either of finer-grained categorizations of language or of event detection techniques other than our change-point detection (Aiello et al., 2013) could disentangle those periods in theoretically meaningful ways.

Fourth, our study is limited to Twitter, mainly because Twitter is the largest open stream of real-time social media data. The practice of using Twitter as a way of modeling the psychological state of a country carries its own limitations. Despite having a rather high penetration in the U.S. (around 20% of adults, according to the latest estimates (Perrin and Anderson, 2018)), its user base is not representative of the general population (Li et al., 2013). Additionally, Twitter is notoriously populated by bots (Ferrara et al., 2016; Varol et al., 2017), automated accounts that are often used to amplify specific topics or view points. Bots played an important role to steer the discussion on several events of broad public interest (Bessi and Ferrara, 2016; Broniatowski et al., 2018), and it is reasonable to expect that they have a role in COVID-related discussions too, as some studies seem to suggest (Yang et al., 2020). To partly discount their impact, since they tend to have anomalous levels of activity (especially retweeting (Bessi and Ferrara, 2016)), we performed two tests. First, we computed all our measures at user-level rather than tweet-level, which counter anomalous levels of activity. Second, we replicated our temporal analysis excluding retweets, and obtained very similar results. In the future, one could attempt to adapt our framework to different sources of online data, for example to web search queries—which have proven useful to identify different phases of the public reactions to the COVID-19 pandemic (Husnayain et al., 2020).

Last, as Strong himself acknowledged in his seminal paper: “any sharp separation between different types of epidemic psychology is a dubious business.” Our work has operationalized each psycho-social epidemic independently. In the future, modeling the relationships among the three epidemics might identify hitherto hidden emergent properties.

## Data availability

The daily aggregates of the measurements are available at: https://doi.org/10.6084/m9.figshare.14892642.v1. The tweet IDs we used are available at https://github.com/echen102/COVID-19-TweetIDs. Other datasets and visualizations are available on the project’s site http://social-dynamics.net/EpidemicPsychology.

## References

1. Aiello LM, Petkos G, Martin C, Corney D, Papadopoulos S, Skraba R, Göker A, Kompatsiaris I, Jaimes A (2013) Sensing trending topics in Twitter. IEEE Trans Multimed 15(6):1268–1282

2. Akbik A, Bergmann T, Blythe D, Rasul K, Schweter S, Vollgraf R (2019) FLAIR: an easy-to-use framework for state-of-the-art NLP. In: Proceedings of the conference of the North American Chapter of the Association for Computational Linguistics. ACL, 54–59

3. Alpers GW, Winzelberg AJ, Classen C, Roberts H, Dev P, Koopman C, Taylor CB (2005) Evaluation of computerized text analysis in an internet breast cancer support group. Comput Hum Behav 21(2):361–376

4. Arguello J, Butler BS, Joyce E, Kraut R, Ling KS, Rosé C, Wang X (2006) Talk to me: foundations for successful individual–group interactions in online communities. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 959–968

5. Bagnoli F, Lio P, Sguanci L (2007) Risk perception in epidemic modeling. Phys Rev E 76(6):061904

6. Bansal S, Chowell G, Simonsen L, Vespignani A, Viboud C (2016) Big data for infectious disease surveillance and modeling. J Infect Dis 214(suppl_4):S375–S379

7. Bareket-Bojmel L, Shahar G, Margalit M (2020) Covid-19-related economic anxiety is as high as health anxiety: findings from the USA, the UK, and Israel. Int J Cogn Ther 1:1-9

8. Baronchelli A (2018) The emergence of consensus: a primer. R Soc Open Sci 5(2):172189

9. Bauch CT, Galvani AP (2013) Social factors in epidemiology. Science 342(6154):47–49

10. Bento AI, Nguyen T, Wing C, Lozano-Rojas F, Ahn Y-Y, Simon K (2020) Evidence from internet search data shows information-seeking responses to news of local covid-19 cases. Proc Natl Acad Sci USA. 11220–11222

11. Bessi A, Ferrara E (2016) Social bots distort the 2016 US presidential election online discussion. First Monday 21(11-7):1-14

12. Blau PM (1964) Exchange and power in social life. Transaction Publishers

13. Bleidorn W, Klimstra TA, Denissen JJ, Rentfrow PJ, Potter J, Gosling SD (2013) Personality maturation around the world: a cross-cultural examination of social-investment theory. Psychol Sci 24(12):2530–2540

14. Borelli JL, Sbarra DA (2011) Trauma history and linguistic self-focus moderate the course of psychological adjustment to divorce. J Soc Clin Psychol 30(7):667–698

15. Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3(2):77–101

16. Broniatowski DA, Jamison AM, Qi S, AlKulaib L, Chen T, Benton A, Quinn SC, Dredze M (2018) Weaponized health communication: Twitter bots and russian trolls amplify the vaccine debate. Am J Public Health 108(10):1378–1384

17. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, Rubin GJ (2020) The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet. 912–920

18. Cap P (2016) The language of fear: communicating threat in public discourse. Springer

19. Chen E, Lerman K, Ferrara E (2020) Tracking social media discourse about the covid-19 pandemic: development of a public coronavirus twitter data set. JMIR Public Health Surveill 6:e19273

20. Chew C, Eysenbach G (2010) Pandemics in the age of twitter: content analysis of tweets during the 2009 h1n1 outbreak. PLoS ONE 5(11):e14118

21. Choi M, Aiello LM, Varga KZ, Quercia D (2020) Ten social dimensions of conversations and relationships. In: Proceedings of the web conference. ACM

22. Cinelli M, Quattrociocchi W, Galeazzi A, Valensise CM, Brugnoli E, Schmidt AL, Zola P, Zollo F, Scala A (2020) The COVID-19 social media infodemic. Scientific Reports, 10(1):1-10

23. Coppersmith G, Dredze M, Harman C (2014) Quantifying mental health signals in twitter. In: Proceedings of the workshop on computational linguistics and clinical psychology: from linguistic signal to clinical reality. ACL, 51–60

24. Da BL, Im GY, Schiano TD (2020) Covid-19 hangover: a rising tide of alcohol use disorder and alcohol-associated liver disease. Hepatology. 1102–1108

25. Deri S, Rappaz J, Aiello LM, Quercia D (2018) Coloring in the links: capturing social ties as they are perceived. In: Proceedings of the ACM conference on Computer Supported Cooperative Work and Social Computing, CSCW. ACM, 1–18

26. Deutsch M (1958) Trust and suspicion. J Confl Resolut 2(4):265–279

27. Dredze M, Paul MJ, Bergsma S, Tran H (2013) Carmen: a twitter geolocation system with applications to public health. In: Workshops at the 27th AAAI conference on artificial intelligence. AAAI

28. Dutheil F, Mondillon L, Navel V (2020) Ptsd as the second tsunami of the sars-cov-2 pandemic. Psychol Med 1–2

29. ElSherief M, Kulkarni V, Nguyen D, Wang WY, Belding E (2018) Hate lingo: a target-based linguistic analysis of hate speech in social media. In: Twelfth international AAAI conference on Web and Social Media. AAAI

30. Engle S, Stromme J, Zhou A (2020) Staying at home: mobility effects of covid-19. Available at SSRN

31. Ferrara E (2020) #covid-19 on twitter: bots, conspiracies, and social media activism. arXiv preprint arXiv:2004.09531

32. Ferrara E, Varol O, Davis C, Menczer F, Flammini A (2016) The rise of social bots. Commun ACM 59(7):96–104

33. Fetzer T, Hensel L, Hermle J, Roth C (2020) Coronavirus perceptions and economic anxiety. Rev Econ Stat, 6:1–36

34. Figea L, Kaati L, Scrivens R (2016) Measuring online affects in a white supremacy forum. In: 2016 IEEE conference on intelligence and security informatics (ISI). IEEE, 85–90

35. Finlay I, Gilmore I (2020) Covid-19 and alcohol—a dangerous cocktail. BMJ, 369

36. Fiske ST, Cuddy AJ, Glick P (2007) Universal dimensions of social cognition: warmth and competence. Trends Cogn Sci 11(2):77–83

37. Frimer JA, Schaefer NK, Oakes H (2014) Moral actor, selfish agent. J Personal Soc Psychol 106(5):790

38. Fu K-W, Liang H, Saroha N, Tse ZTH, Ip P, Fung IC-H (2016) How people react to zika virus outbreaks on twitter? a computational content analysis. Am J Infect Control 44(12):1700–1702

39. Galea S, Merchant RM, Lurie N (2020) The mental health consequences of covid-19 and physical distancing: the need for prevention and early intervention. JAMA Intern Med 180(6):817–818

40. Gao S, Rao J, Kang Y, Liang Y, Kruse J (2020) Mapping county-level mobility pattern changes in the united states in response to covid-19. SIGSPATIAL Special 12(1):16–26

41. Gibbs GR (2007) Thematic coding and categorizing. In: Analyzing qualitative data. Ed. Flick U. Sage, London, 38–56

42. Gill AJ, French RM, Gergle D, Oberlander J (2008) The language of emotion in short blog texts. In: Proceedings of the 2008 ACM conference on computer supported cooperative work. ACM, 299–302

43. Goffman E (2009) Stigma: notes on the management of spoiled identity. Simon and Schuster

44. Gonzalez MC, Hidalgo CA, Barabasi A-L (2008) Understanding individual human mobility patterns. Nature 453(7196):779–782

45. Graham J, Haidt J, Nosek BA (2009) Liberals and conservatives rely on different sets of moral foundations. J Personal Soc Psychol 96(5):1029

46. Graham J, Haidt J, Koleva S, Motyl M, Iyer R, Wojcik SP, Ditto PH (2013) Moral foundations theory: the pragmatic validity of moral pluralism. In: Advances in experimental social psychology. Eds. Devine P, Plant A. Elsevier, 47:55–130

47. Grossman G, Kim S, Rexer J, Thirumurthy H (2020) Political partisanship influences behavioral responses to governors’ recommendations for covid-19 prevention in the united states. PNAS, 117:24144–24153

48. Hebert JR, Clemow L, Pbert L, Ockene IS, Ockene JK (1995) Social desirability bias in dietary self-report may compromise the validity of dietary intake measures. Int J Epidemiol 24(2):389–398

49. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

50. Hofstede GH, Hofstede GJ, Minkov M (2005) Cultures and organizations: software of the mind, 2. McGraw-Hill, New York

51. Holmes EA, O’Connor RC, Perry VH, Tracey I, Wessely S, Arseneault L, Ballard C, Christensen H, Silver RC, Everall I et al. (2020) Multidisciplinary research priorities for the covid-19 pandemic: a call for action for mental health science. Lancet Psychiatry. AAAI, 547–560

52. Hou Z, Du F, Jiang H, Zhou X, Lin L (2020) Assessment of public attention, risk perception, emotional and behavioural responses to the covid-19 outbreak: social media surveillance in China. Preprint at https://doi.org/10.1101/2020.03.14.20035956

53. Huang Z, Xu W, Yu K (2015) Bidirectional LSTM-CRF models for sequence tagging. Preprint at arXiv:1508.01991

54. Hunt M, Auriemma J, Cashaw A (2003) Self-report bias and underreporting of depression on the BDI-II. J Personal Assess 80(1):26–30

55. Husnayain A, Fuad A, Su EC-Y (2020) Applications of google search trends for risk communication in infectious disease management: a case study of covid-19 outbreak in taiwan. Int J Infect Dis. 221–223

56. Jimeno-Yepes A, MacKinlay A, Han B, Chen Q (2015) Identifying diseases, drugs, and symptoms in twitter. Stud Health Technol Inform 216:643

57. Johnson T, Fendrich M (2005) Modeling sources of self-report bias in a survey of drug use epidemiology. Ann Epidemiol 15(5):381–389

58. Kahn JH, Tobin RM, Massey AE, Anderson JA (2007) Measuring emotional expression with the linguistic inquiry and word count. Am J Psychol, 120:263–286

59. Khan S, Huremović D (2019) Psychology of the pandemic. In: Psychiatry of pandemics. Ed. Huremović D. Springer, 37–44

60. Kornfield R, Toma CL, Shah DV, Moon TJ, Gustafson DH (2018) What do you say before you relapse? how language use in a peer-to-peer online discussion forum predicts risky drinking among those in recovery. Health Commun 33(9):1184–1193

61. Kouzy R, Abi Jaoude J, Kraitem A, El Alam MB, Karam B, Adib E, Zarka J, Traboulsi C, Akl EW, Baddour K (2020) Coronavirus goes viral: quantifying the covid-19 misinformation epidemic on twitter. Cureus 12(3):e7255

62. Kübler-Ross E, Wessler S, Avioli LV (1972) On death and dying. JAMA 221(2):174–179

63. Li L, Goodchild MF, Xu B (2013) Spatial, temporal, and socioeconomic patterns in the use of twitter and flickr. Cartogr Geogr Inf Sci 40(2):61–77

64. Li S, Wang Y, Xue J, Zhao N, Zhu T (2020) The impact of covid-19 epidemic declaration on psychological consequences: a study on active weibo users. Int J Environ Res Public Health 17(6):2032

65. Liang L, Ren H, Cao R, Hu Y, Qin Z, Li C, Mei S (2020) The effect of covid-19 on youth mental health. Psychiatr Q, 91:1–12

66. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M,. Zettlemoyer L, Stoyanov V (2019) Roberta: A robustly optimized bert pretraining approach. Preprint at arXiv:1907.11692

67. Lütkepohl H (2005) New introduction to multiple time series analysis. Springer Science, Business Media

68. Maslow AH (1943) A theory of human motivation. Psychol Rev 50(4):370

69. McConnell P (2005) Banks and avian flu: planning for a possible pandemic. Risk Trading Technology. Continuity Central

70. Mohammad SM, Turney PD (2013) Crowdsourcing a word-emotion association lexicon. Comput Intell 29(3):436–465

71. Moinet A, Pastor-Satorras R, Barrat A (2018) Effect of risk perception on epidemic spreading in temporal networks. Phys Rev E 97(1):012313

72. O’Connor B, Balasubramanyan R, Routledge BR, Smith NA (2010) From tweets to polls: linking text sentiment to public opinion time series. In: Fourth international AAAI conference on weblogs and social media. AAAI

73. Oyeyemi SO, Gabarron E, Wynn R (2014) Ebola, twitter, and misinformation: a dangerous combination? BMJ 349:g6178

74. Painter M, Qiu T (2020) Political beliefs affect compliance with covid-19 social distancing orders. Covid Economics, 4:103-123

75. Palshikar G et al. (2009) Simple algorithms for peak detection in time-series. In: Proceedings of the 1st international conference on advanced data analysis, business analytics and intelligence, IIMA, 122

76. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in PyTorch. In: Proceedings of the advances in neural information processing systems autodiff workshop. NIPS

77. Pennington J, Socher R, Manning C (2014) GloVe: global vectors for word representation. In: Proceedings of the conference on empirical methods in natural language processing. ACL, 1532–1543

78. Perrin A, Anderson M (2018) Share of U.S. adults using social media, including facebook, is mostly unchanged since. Pew Research Center. https://www.pewresearch.org/fact-tank/2019/04/10/share-of-u-s-adults-using-social-media-including-facebook-is-mostly-unchanged-since-2018/

79. Plutchik R (1991) The emotions. University Press of America

80. Pulido CM, Villarejo-Carballido B, Redondo-Sama G, Gómez A (2020) Covid-19 infodemic: more retweets for science-based information on coronavirus than for false information. Int J Sociol, 35:377–392

81. Qiu J, Shen B, Zhao M, Wang Z, Xie B, Xu Y (2020) A nationwide survey of psychological distress among Chinese people in the covid-19 epidemic: implications and policy recommendations. Gen Psychiatry 33(2):e100213

82. Rezapour R, Shah SH, Diesner J (2019) Enhancing the measurement of social effects by capturing morality. In: Proceedings of the 10th workshop on computational approaches to subjectivity, sentiment and social media analysis. ACL, 35–45

84. Saif H, He Y, Alani H (2012) Alleviating data sparsity for twitter sentiment analysis. In Proceedings of the 2nd Workshop on Making Sense of Microposts, 2-9. CEUR

85. Salathe M, Bengtsson L, Bodnar TJ, Brewer DD, Brownstein JS, Buckee C, Campbell EM, Cattuto C, Khandelwal S, Mabry PL et al. (2012) Digital epidemiology. PLoS Comput Biol 8(7):e1002616

86. Scepanovic S, Martin-Lopez E, Quercia D, Baykaner K (2020) Extracting medical entities from social media. In: Proceedings of the ACM conference on health, inference, and learning. ACM, 170–181

87. Schutz A, Luckmann T, Zaner R, Engelhardt J (1973) The structures of the life-world. Northwestern University studies in phenomenology & existential philosophy, vol. 1. Northwestern University Press

88. Shaw B, Han JY, Kim E, Gustafson D, Hawkins R, Cleary J, McTavish F, Pingree S, Eliason P, Lumpkins C (2007) Effects of prayer and religious expression within computer support groups on women with breast cancer. Psycho-Oncology 16(7):676–687

89. Shen JH, Rudzicz F (2017) Detecting anxiety through reddit. In: Proceedings of the fourth workshop on computational linguistics and clinical psychology-from linguistic signal to clinical reality. ACL, 58–65

90. Shultz JM, Baingana F, Neria Y (2015) The 2014 ebola outbreak and mental health: current status and recommended response. Jama 313(6):567–568

91. Simms T, Ramstedt C, Rich M, Richards M, Martinez T, Giraud-Carrier C (2017) Detecting cognitive distortions through machine learning text analytics. In Proceedings of: IEEE International Conference On Healthcare Informatics (ICHI). IEEE, 508–512

92. Smith JA, Shinebourne P (2012) Interpretative phenomenological analysis. American Psychological Association

93. Sommariva S, Vamos C, Mantzarlis A, Đào LU-L, Martinez Tyson D (2018) Spreading the (fake) news: exploring health messages on social media and the implications for health professionals using a case study. American journal of health education 49(4):246–255

94. Strong P (1990) Epidemic psychology: a model. Sociology of Health, Illness 12(3):249–259

95. Tajfel H, Turner JC, Austin WG, Worchel S (1979) An integrative theory of intergroup conflict. Organizational Identity. Eds. Hatch MJ, Schultz M

96. Tausczik YR, Pennebaker JW (2010) The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology 29(1):24–54

97. Merchlinsky C. With millions stuck at home, the online wellness industry is booming. The Economist. https://www.economist.com/international/2020/04/04/with-millions-stuck-at-home-the-online-wellness-industry-is-booming

98. Van Bavel JJ, Baicker K, Boggio PS, Capraro V, Cichocka A, Cikara M, Crockett MJ, Crum AJ, Douglas KM, Druckman JN et al. (2020) Using social and behavioural science to support covid-19 pandemic response. Nat Hum Behav, 4:460–471

99. Varol O, Ferrara E, Davis CA, Menczer F, Flammini A (2017) Online human–bot interactions: detection, estimation, and characterization. In: Eleventh international AAAI conference on web and social media. AAAI

100. Wang C, Pan R, Wan X, Tan Y, Xu L, Ho CS, Ho RC (2020) Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. Int J Environ Res Public Health 17(5):1729

101. Waterloo SF, Baumgartner SE, Peter J, Valkenburg PM (2018) Norms of online expressions of emotion: Comparing facebook, twitter, instagram, and whatsapp. New Media Soc 20(5):1813–1831

102. Windsor LC, Dowell N, Graesser A (2014) The language of autocrats: leaders’ language in natural disaster crises. Risk Hazards Crisis Public Policy 5(4):446–467

103. Wojcik S, Hughes A (2019) Sizing up twitter users. Pew Research Center, Washington

104. Wolf M, Theis F, Kordy H (2013) Language use in eating disorder blogs: psychological implications of social online activity. J Lang Soc Psychol 32(2):212–226

105. Wood MJ (2018) Propagating and debunking conspiracy theories on twitter during the 2015-2016 zika virus outbreak. Cyberpsychol Behav Soc Network 21(8):485–490

106. Yang K-C, Torres-Lugo C, Menczer F (2020) Prevalence of low-credibility information on twitter during the covid-19 outbreak. Proceedings of the ICWSM International Workshop on Cyber Social Threats (CySoc). AAAI

## Acknowledgements

We thank Sarah Konrath, Rosta Farzan, and Licia Capra for their useful feedback on the manuscript. This research was partly supported by the EU Grant “GO GREEN ROUTES” no. 869764.

## Author information

Authors

### Corresponding author

Correspondence to Daniele Quercia.

## Ethics declarations

### Competing interests

The authors declare no competing interests.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Aiello, L.M., Quercia, D., Zhou, K. et al. How epidemic psychology works on Twitter: evolution of responses to the COVID-19 pandemic in the U.S.. Humanit Soc Sci Commun 8, 179 (2021). https://doi.org/10.1057/s41599-021-00861-3