Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

A multi-country test of brief reappraisal interventions on emotions during the COVID-19 pandemic

Abstract

The COVID-19 pandemic has increased negative emotions and decreased positive emotions globally. Left unchecked, these emotional changes might have a wide array of adverse impacts. To reduce negative emotions and increase positive emotions, we tested the effectiveness of reappraisal, an emotion-regulation strategy that modifies how one thinks about a situation. Participants from 87 countries and regions (n = 21,644) were randomly assigned to one of two brief reappraisal interventions (reconstrual or repurposing) or one of two control conditions (active or passive). Results revealed that both reappraisal interventions (vesus both control conditions) consistently reduced negative emotions and increased positive emotions across different measures. Reconstrual and repurposing interventions had similar effects. Importantly, planned exploratory analyses indicated that reappraisal interventions did not reduce intentions to practice preventive health behaviours. The findings demonstrate the viability of creating scalable, low-cost interventions for use around the world.

Protocol registration

The stage 1 protocol for this Registered Report was accepted in principle on 12 May 2020. The protocol, as accepted by the journal, can be found at https://doi.org/10.6084/m9.figshare.c.4878591.v1

Main

The COVID-19 pandemic is increasing negative emotions and decreasing positive emotions around the globe1,2,3,4,5,6,7,8,9,10. Concurrently, individuals are reporting that COVID-19 is having a negative impact on their psychological functioning and mental health4,11,12. For example, individuals report sleeping less, consuming more alcohol or other drugs or substances, having trouble concentrating because their mind is occupied by COVID-19, and having more fights with their partner or loved ones, some escalating to domestic violence1,9,13.

These disturbing trends are caused partly by heightened levels of negative emotion and diminished levels of positive emotion, which have been found to contribute to a number of negative psychological, behavioural and health consequences. These include increased risk of anxiety and depressive disorders as well as other forms of psychopathology14; impaired social connections15; increased substance use16,17,18; compromised immune system functioning19,20,21; disturbed sleep22; increased maladaptive eating23,24; increased aggressive behaviour25,26; impaired learning27; worse job performance28,29; and impaired economic decision-making30,31.

As the COVID-19 pandemic unfolds around the world, we believe it is crucial to mitigate expected adverse outcomes by reducing negative emotions and increasing positive emotions. Such a change in emotions is central to increasing psychological resilience, a multifaceted concept that involves adaptive emotional responses in the face of adversity32,33,34. Reappraisal—an emotion regulation strategy that involves changing how one thinks about a situation with the goal of influencing one’s emotional response35—is a promising candidate as an intervention to increase psychological resilience due to its adaptability, simplicity and efficiency34,36,37,38. In contrast to less effective emotion-regulation strategies such as suppression, reappraisal generally leads to more successful regulation (d = 0.45, 95% confidence interval (CI) = [0.35, 0.56] in changing emotion experience in a meta-analysis39; see caveats about interpreting effect sizes in past research in Methods, ‘Sampling plan’). In particular, over the short term, reappraisal leads to decreased reports of negative emotion and increased reports of positive emotion40,41,42, as well as corresponding changes both in peripheral physiological responses43,44,45 and central physiological responses46,47,48,49,50,51,52,53. Over the longer term, reappraisal is associated with stronger social connections54; higher academic achievement55,56; enhanced psychological well-being57; fewer psychopathological symptoms58,59; better cardiovascular health60,61, and greater resilience during the COVID-19 pandemic62.

Despite these shorter-term and longer-term benefits, most people do not reappraise consistently63,64, which has motivated efforts to teach people to use reappraisal (reviewed in refs. 65,66). For example, in the context of anxiety, reappraisal training led to reduced intrusive memories67 and increased emotion-regulation self-efficacy68,69. Reappraisal training also led to long-lasting changes in the neural representation of unpleasant events70.

Although demand characteristics are always a concern when examining the effects of reappraisal (given that one is teaching people to change their thinking in order to change how they’re feeling, and then asking them how they feel)71, the wide array of self-report and non-self-report outcomes39,40,41,42,43,44,45,46,47,48,49,50,51,52,53 that show reappraisal effects across studies increases confidence that these effects are real. It is also encouraging to note that reappraisal generally outperforms other types of emotion regulation such as suppression, even though demand characteristics appear comparable across regulation conditions39. In addition, evidence indicates that reappraisal interventions can influence emotional outcomes even in intensely challenging contexts in which people are often unmotivated to regulate their emotions72. For example, a brief reappraisal training conducted in the context of the Israeli–Palestinian conflict and replicated in the context of the Colombian conflict73, has been found to contribute to reduced intergroup anger and increased support for conciliatory political policies74.

As part of the attempt of the Psychological Science Accelerator (PSA) to address pressing questions related to the psychological impact of COVID-19, the current study aimed to use reappraisal interventions to enhance psychological resilience in response to the pandemic. To maximize the impact of these interventions, this project had a global reach of large, diverse samples via the PSA’s network75, and employed highly scalable methods that were translated for use around the world. In order to make stronger and clearer inferences, our design included two reappraisal interventions that were compared with two control conditions, an active control and a passive control.

For our reappraisal interventions, we examined two theoretically defined forms of reappraisal76—reconstrual and repurposing. Reconstrual involves changing how a situation was construed or mentally represented in a way that changes the emotional responses related to the situation. Examples of reconstrual in response to COVID-19 include: “Washing hands, avoiding touching my face, keeping a safe distance…There are simple and effective things I can do to protect myself and my loved ones from getting sick and to stop the spread of the virus” and “I know from world history that keeping calm and carrying on gets us through tough times”. Repurposing involves focusing on a potentially positive outcome that could come from the current situation in a way that changes the emotional response to it. Examples of repurposing in response to COVID-19 include: “This situation is helping us realize the importance of meaningful social connections, and helping us understand who the most important people in our lives are” and “Medical systems are now learning to deal with amazing challenges, which will make them much more resilient in the future”. For our active control condition, we asked participants to reflect on their thoughts and feelings as they unfolded. Meta-analyses have revealed that reflecting on one’s thoughts and feelings produces small but reliable salutary effects (d = 0.07, 9% CI = [0.05, 0.17] in improving psychological health, including emotional responses77,78). Examples of reflecting in response to COVID-19 are: “I really wish we could find a vaccine soon” and “This situation is changing so fast, and I don’t know how the future will develop”. By asking participants in this condition to actively use a strategy that is likely to have a positive effect, we sought to match expectancy and demand across reappraisal and active control conditions. For our passive control condition, we asked participants to respond as they naturally do, which is a commonly used passive control condition in prior research on emotion regulation (for a meta-analysis, see ref. 39).

In comparing conditions, we chose to distinguish between negative and positive emotional responses, as previous evidence suggests that the two are clearly separable79,80. Specifically, we hypothesized that our reappraisal interventions would lead to reduced negative emotional responses (hypothesis 1) and increased positive emotional responses (hypothesis 2) compared with both control conditions combined. While both reconstrual and repurposing strategies involve changing thinking, we hypothesized that the reconstrual intervention would lead to greater decreases in negative emotional responses than the repurposing intervention (hypothesis 3) and that the repurposing intervention would lead to greater increases in positive emotional responses than the reconstrual intervention (hypothesis 4). We theorized that reconstruing one’s situation should primarily decrease negative emotions, because it typically focuses on ameliorating the problem at hand. The reconstrual intervention is most similar to a previously studied subtype of reappraisal called reappraising emotional stimulus, which has been investigated mainly on negative emotions and has a d = 0.38 and 95% CI = [0.21, 0.55] for changing emotion experience39. Repurposing one’s situation, by contrast, should primarily increase positive emotions because it usually calls to mind positive experiences. Repurposing is similar to a few previously examined types of reappraisals, such as benefit finding and positive reappraisal, both of which are primarily associated with positive outcomes81,82 (Methods, ‘Sampling plan’ provides further detail).

In testing these hypotheses, we planned to use orthogonal contrasts that make two primary comparisons, while keeping all other comparisons exploratory (Table 1 provides further detail). The first comparison contrasted both the reappraisal conditions combined with both the active control condition and the passive control condition combined for negative (hypothesis 1) and positive (hypothesis 2) emotions. The second comparison contrasted the reconstrual and repurposing interventions for negative (hypothesis 3) and positive (hypothesis 4) emotions. One attractive feature of comparisons between reappraisal conditions is that there is no reason to assume that demand or expectancies would differ across these reappraisal conditions.

Table 1 Contrast structure of testing hypotheses 1–4 (with unit weighting)

One potential concern about the current design was that the emotion-regulation interventions might reduce preventive health behaviours (for example, maintaining social distance and washing hands) that could potentially be motivated by negative emotions. Some research on the connection between emotions and health behaviour suggests that increased negative emotions such as fear do not seem to be a strong motivator for changing one’s health behaviour83. Furthermore, positive emotions augmented by the reappraisal interventions may contribute to a greater tendency to undertake health behaviours84,85. For example, positive emotions can lead to higher medication adherence86. To ensure that our interventions would not adversely impact any relevant health behaviours, we took two steps. First, during the instructions, we clarified that—in some cases—negative emotions such as fear and sadness may be helpful, and that it is up to each person to determine when an emotion is unhelpful or not and to downregulate only those emotions that are unhelpful. Second, to assess whether our training would lead to reduced vigilance, we specifically measured and examined intentions to follow stay-at-home orders and wash hands in exploratory analyses.

In addition, we conducted other exploratory analyses. These analyses included testing the impact of our reappraisal interventions on negative and positive anticipated emotions and intentions to enact potentially harmful versus beneficial behaviours associated with these emotions (details described in Methods, ‘Measures’), and assessed whether the effects of our reappraisal interventions, if any, were moderated by motivation to use the given strategy71, belief in the effectiveness of the given strategy87, or demographics (gender39, socioeconomic status88,89 or country or region90 (hereafter country/region) (particularly in light of the differing levels of impact of COVID-19 in any given country/region at any given point in time)).

Results

Final sample size and demographics

We collected 27,989 responses between May 2020 and October 2020. After implementing preregistered exclusions (see detail in https://doi.org/10.6084/m9.figshare.c.4878591.v1) and an additional exclusion of nine duplicate IDs, our final sample included 21,644 participants from 87 countries/regions (63.41% female, 35.34% male, 0.45% other genders, 0.56% preferred not to say and 0.24% missing responses to the gender question; participants were aged 31.91 ± 14.52 yr (mean ± s.d.); see Supplementary Table 1 for sample size per country/region and Supplementary Table 2 for sample size per month). Of the 87 countries/regions represented, 37 had more than 200 participants, surpassing our 95% power criterion based on simulations in our power analysis (see detail in Methods, ‘Power analysis’).

We preregistered two exclusion criteria. First, as planned, we excluded participants who answered both multiple choice manipulation check questions incorrectly, and found that conditions had similar proportions of such participants (0.55%), Holm’s adjusted P values > 0.999. Second, as planned, we excluded participants who completed fewer than 50% of the questions in the study, and found that the passive control condition had fewer such participants (16.17%) than the other three conditions (23.86% in the active control condition, 24.41% in the reconstrual condition and 23.90% in the repurposing condition), Holm’s adjusted P < 0.001. One possible explanation for this difference is that the instructions given to participants in the passive control condition were shorter than those given in the other conditions, requiring less cognitive effort to read and less time to complete the study. Applying both exclusion criteria, the overall exclusion rate was significantly lower in the passive control condition (16.71%) than in the other three conditions (24.47% in the active control condition, 24.99% in the reconstrual condition and 24.37% in the repurposing condition), Holm’s adjusted P < 0.001. To rule out concerns related to differences in exclusion rates, we repeated all preregistered analyses on the full sample. Reassuringly, all patterns, statistical significance and conclusions remained unchanged when analyses were repeated on the full sample (Supplementary Table 3).

Preregistered analyses

We included all 87 countries/regions in all analyses regardless of their sample sizes, except for Fig. 1, Supplementary Fig. 1 and Supplementary Fig. 2, where the 37 countries/regions with n ≥ 200 were analysed separately by country/region. Effect sizes, frequentist statistics and Bayes factors for each of our hypotheses are presented in Table 2. Raw means and standard deviations for each relevant measure are provided in Table 3. Details of analytical models are described in Methods.

Fig. 1: Effect sizes of both reappraisal interventions combined (versus both control conditions combined) on primary outcomes by country/region.
figure1

In almost all of the 37 countries/regions in which there were more than 200 participants, both reappraisal interventions combined (versus both control conditions combined) decreased negative emotional responses and increased positive emotional responses for primary outcome measures (emotions in response to the photos, state emotions after viewing all the photos, and emotions about the COVID-19 situation). Effect sizes are raw mean differences on five-point scales without adjusting for covariates. Confidence intervals are based on the t distribution. Countries/regions are ordered by decreasing effect sizes of negative emotions in response to the photos, and larger dots reflect larger samples (Supplementary Fig. 1 presents the countries/regions in alphabetical order.).

Table 2 Effect sizes, frequentist statistics and Bayes factors for each preregistered hypothesis
Table 3 Raw mean and s.d. values for outcomes

Hypotheses regarding the shared effects of two brief reappraisal interventions

Consistent with the main hypotheses of the study, both reappraisal interventions combined (versus both control conditions combined) significantly decreased negative emotional responses (hypothesis 1) and significantly increased positive emotional responses (hypothesis 2) across all primary outcome measures (emotions in response to the photos related to COVID-19 from various news sources, state emotions after viewing all the photos and emotions about the COVID-19 situation; Table 2, rows 2–7; details of these measures are described in Methods). As shown in Fig. 1, this finding was consistent across the 37 countries/regions in which there were more than 200 participants (although all 87 countries/regions were included in the analysis testing hypotheses regardless of their sample size, only the 37 countries/regions with n ≥ 200 were analysed separately by country/region for Fig. 1). For example, in comparing participants’ immediate negative emotional responses to the photos related to the COVID-19 situation, data from 33 out of the 37 (89%) countries/regions showed significant effects of the reappraisal interventions in the hypothesized direction. None of the 37 countries/regions’ data revealed a statistically significant result in the opposite direction.

Hypotheses regarding the unique effects of the two reappraisal interventions

Results revealed little to no support for our hypotheses regarding the differences between reconstrual and repurposing, as neither was reliably better than the other at reducing negative emotions or increasing positive emotions across outcomes (Table 2, rows 8–13; Supplementary Fig. 2). We hypothesized that reconstrual would produce greater decreases in negative emotional responses than repurposing (hypothesis 3), and data revealed supportive evidence for only one outcome (negative emotions about the COVID-19 situation; Table 2, row 10) out of the three measures of negative emotions. The other two negative emotion outcome measures did not support that hypothesis. One outcome (negative emotions in response to the photos; Table 2, row 8) revealed that repurposing had significantly stronger effects in decreasing negative emotional responses than reconstrual, whereas the Bayes factor indicated inconclusive evidence. Another outcome (negative state emotions; Table 2, row 9) revealed no significant difference between types of reappraisal, and the Bayes factor indicated strong evidence in favour of the null hypothesis.

We also hypothesized that repurposing would produce greater increases in positive emotional responses than reconstrual (hypothesis 4), and data revealed supportive evidence for only one outcome (positive emotions in response to the photos; Table 2, row 11) out of the three measures of positive emotions. The other two outcome measures of positive emotions revealed no significant differences between the two reappraisal conditions. The Bayes factors indicated strong evidence in favour of the null hypothesis for one outcome (positive state emotions; Table 2, row 12) and inconclusive evidence for another outcome (positive emotions about the COVID-19 situation; Table 2, row 13). Overall, there were no consistent differences across outcomes between reconstrual and repurposing in reducing negative emotions or increasing positive emotions in the current experimental context. We examined potential reasons for these findings in the exploratory analyses and in the discussion section.

Exploratory analyses

To better understand the impact of the reappraisal interventions, we conducted four sets of exploratory analyses. First, we examined pairwise comparisons between conditions (each of the reappraisal conditions versus each of the control conditions, and the active control condition versus the passive control condition) for our primary outcomes (emotions in response to the photos, state emotions after viewing all the photos and emotions about the COVID-19 situation). Second, we assessed the effect of reappraisal interventions on four exploratory outcomes (behavioural intentions to practice preventive health behaviours, participants’ engagement with emotion regulation strategies, global change in emotions, and anticipated emotions). Third, we assessed four sets of potential moderators of reappraisal interventions’ effects (motivation to use the given strategy71, belief in the effectiveness of the given strategy87, demographics39,88,89,90 and lockdown status). Finally, we contextualised reappraisal interventions’ effect sizes on negative emotions by comparing them with effect sizes of lockdown status and self-isolation due to symptoms. Details of analytical models are reported in Supplementary Information (Supplementary Tables 4 and 5).

Pairwise comparisons of conditions on primary outcomes

In the first set of exploratory analyses, we examined the extent to which each of the reappraisal conditions differed from each of the control conditions for our primary outcomes (emotions in response to the photos, state emotions after viewing all the photos and emotions about the COVID-19 situation). Pairwise comparisons for all primary outcomes produced results consistent with the pattern of evidence for hypothesis 1 and hypothesis 2. Each of the repurposing and reconstrual conditions (versus each of the control conditions) significantly decreased negative emotional responses and significantly increased positive emotional responses (Ps < 0.001; Table 3).

We also examined whether the active and passive control conditions differed from each other at the level of pairwise comparisons. Among the three primary outcome measures of negative emotional responses, one was significantly higher in the active control condition than in the passive control condition (negative emotions in response to the photos: B = 0.091 ± 0.015, t(20,740) = 6.192, P < 0.001, d = 0.070, 95% CI = [0.048, 0.093]), while the other two showed no significant differences (negative state emotions: B = 0.022 ± 0.011, t(20,400) = 1.933, P = 0.053, d = 0.037, 95% CI = [−0.001, 0.075]; negative emotions about the COVID-19 situation: B = 0.005 ± 0.022, t(26.01) = 0.221, P = 0.827, d = 0.005, 95% CI = [−0.040, 0.047]). Among the three primary outcome measures of positive emotional responses, two were significantly higher in the active control condition than in the passive control condition (positive emotions in response to the photos: B = 0.039 ± 0.013, t(20,740) = 2.918, P = 0.004, d = 0.033, 95% CI = [0.011, 0.054]; positive emotions about the COVID-19 situation: B = 0.053 ± 0.019, t(233.7) = 2.805, P = 0.005, d = 0.053, 95% CI = [0.015, 0.091]), while one showed no significant differences (positive state emotions: B = 0.009 ± 0.010, t(20,350) = 0.858, P = 0.391, d = 0.017, 95% CI = [−0.021, 0.054]). Thus, effects produced by the active control condition versus the passive control condition differed infrequently. When they did differ, differences were small in magnitude, inconsistent in direction, and slightly smaller in effect size than was suggested by previous meta-analyses77 (d = 0.07, 95% CI = [0.05, 0.17]).

Effects of reappraisal interventions on four exploratory outcomes

Details of exploratory outcomes can be found in Methods and Fig. 2. Descriptive statistics and pairwise comparisons for exploratory outcomes can be found in Table 3. Here we focus on the contrast between the two reappraisal interventions combined and the two control conditions combined.

Fig. 2: Overview of the experiment.
figure2

Participants in the passive control condition did not have the fourth step in the practice trials.

Behavioural intentions to practice preventive health behaviours

To address the concern that reappraisal interventions might reduce preventive health behaviours (by reducing negative emotions such as fear), we asked about participants’ behavioural intentions to follow stay-at-home orders stringently and to wash their hands regularly for at least 20 s the following week. We found that reappraisal interventions (versus both control conditions combined) did not significantly change intentions to follow stay-at-home orders (B = 0.009 ± 0.024, t(15.04) = 0.38, P = 0.709, d = 0.005, 95% CI = [−0.023, 0.032]) or to wash hands (B = 0.034 ± 0.020, t(20,740) = 1.69, P = 0.091, d = 0.022, 95% CI = [−0.004, 0.048]). Pairwise comparisons revealed that the only significant difference was that participants in the reconstrual condition reported higher intentions to wash their hands than those in the passive control condition (B = 0.077 ± 0.028, t(20,740) = 2.714, Holm’s adjusted P = 0.040, d = 0.051, 95% CI = [0.014, 0.087]). These results thus provide preliminary evidence that reappraisal interventions did not significantly reduce intentions to practice preventive health behaviours.

Participants’ engagement with emotion-regulation strategies

To better understand participants’ engagement with emotion-regulation strategies when viewing the photos related to COVID-19, we examined participants’ self-reported frequency of using different strategies when viewing the photos, motivation to use their given strategy, and belief in the effectiveness of their given strategy.

Providing confidence in the effectiveness of the manipulation, we found that participants in each of the four conditions reported using the strategy instructed in their condition more frequently than using the other strategies (see Table 3). It is noteworthy that participants in the two reappraisal conditions reported using both reconstrual and repurposing more frequently than those in either control conditions rather than primarily using only the form of reappraisal instructed in their condition. This finding may help explain the lack of differences between the two reappraisal conditions on our primary outcomes.

Next, we examined participants’ motivation to follow the given instructions, as well as participants’ belief that the given strategy could influence their emotions. We found that participants in the two reappraisal interventions (versus both control conditions combined) reported being significantly less motivated to follow their given instructions while viewing the photos (B = −0.192 ± 0.016, t(20.87) = −11.62, P < 0.001, d = −0.183, 95% CI = [−0.215, −0.152]), but reported significantly greater belief in the effectiveness of their given strategy (B = 0.420 ± 0.053, t(52.05) = 7.97, P < 0.001, d = 0.233, 95% CI = [0.175, 0.290]). Thus, the reappraisal conditions were effective in changing emotions despite the fact that participants in reappraisal conditions reported being less motivated to follow the instructions than participants in the control conditions.

Global change of emotions

At the end of the study, we asked participants how they felt compared with at the beginning of the study. We found that reappraisal interventions (versus both control conditions combined) significantly reduced global negative feelings (B = −0.397 ± 0.026, t(45.29) = 15.30, P < 0.001, d = −0.432, 95% CI = [−0.489, −0.377]) and significantly increased global positive feelings (B = 0.378 ± 0.023, t(45.49) = 16.75, P < 0.001, d = 0.423, 95% CI = [0.373, 0.473]). These findings suggest that the effects are not specific to items in the immediate proximity of the manipulations.

Anticipated emotions

To gain insight into the potential longer-term effects of reappraisal interventions, we asked participants how they anticipated they would feel the following week. We found that reappraisal interventions (versus both control conditions combined) significantly reduced negative anticipated emotions (B = −0.125 ± 0.012, t(41.99) = −10.27, P < 0.001, d = −0.205, 95% CI = [−0.245, −0.166]) and significantly increased positive anticipated emotions (B = 0.125 ± 0.008, t(13.07) = 15.58, P < 0.001, d = 0.227, 95% CI = [0.197, 0.256]). These findings suggest that participants anticipated that reappraisal strategies would be useful in improving their emotional well-being in the future.

Exploratory moderators of intervention effects

Prior research suggests that emotion-regulation interventions lead to better results when the participants are: motivated to regulate their emotions71, led to believe in the effectiveness of regulation87, female (versus male)39, from lower (versus higher) socioeconomic status88,89, and from Western (versus Eastern) cultures90. We examined these as well as lockdown status (as a proxy for differing levels of impact of COVID-19) as potential moderators on our primary outcomes (emotions in response to the photos, state emotions after viewing all the photos, and emotions about the COVID-19 situation).

Controlling for baseline emotions, results of multilevel models revealed that two of the variables moderated intervention effects across all six primary outcomes. Specifically, the higher the scores on motivation to use the given strategy and on belief in the effectiveness of the given strategy were, the more effective the interventions were (Supplementary Figs. 3 and 4 and Supplementary Tables 6 and 7). Two variables (gender and employment status) moderated intervention effects on four of the six primary outcomes: Females (versus males) and individuals who had no employment and no income (versus those who had employment and income or versus those with no employment but with income) showed stronger effects of the intervention (Supplementary Tables 9 and 10). One variable moderated intervention effects on two of the six outcomes: the higher a country/region scored on Hofstede’s91 index of individualism, the more effective the intervention was in increasing positive emotions in response to the photos and increasing positive emotions about the COVID-19 situation among participants from that country/region (Supplementary Table 8). Subjective socioeconomic status, education level, and lockdown status significantly moderated no more than one of the six outcomes, which would be unlikely to hold after correction for multiple comparisons (Supplementary Tables 1113). Full, detailed results are reported in the Supplementary Information.

Contextualising reappraisal interventions’ effect sizes

To facilitate interpretation of reappraisal effect sizes, it is helpful to compare them to effect sizes of other factors that may have also contributed to differences in participants’ emotions. One such candidate for comparison is differences in emotional experience as a function of lockdown status and of self-isolation due to symptoms. Assuming that lockdown or self-isolation due to symptoms impacted participants’ emotions, emotional changes caused by these factors could be compared to the ones caused by our interventions in order to get a sense of the impact of our intervention.

With negative state emotions as the outcome variable, we examined lockdown status and self-isolation due to symptoms, respectively, as a fixed variable in two separate multilevel models with random by-country/region slopes and random by-country/region intercepts to estimate the pure effect size of each variable (as lockdown status and self-isolation due to symptoms were correlated, entering both variables simultaneously in the same model may generate biased estimates). We found that participants whose areas were in full lockdown reported more negative state emotions than participants whose areas were not in lockdown (B = 0.154 ± 0.040, t(37.56) = 3.812, P < 0.001, d = 0.159, 95% CI = [0.075, 0.243]), and participants whose areas were in partial lockdown reported more negative state emotions than participants whose areas were not in lockdown (B = 0.094 ± 0.027, t(27.25) = 3.531, P = 0.001, d = 0.097, 95% CI = [0.041, 0.155]). We also found that participants who were self-isolating due to flu-like or cold-like symptoms reported more negative state emotions than participants who were not self-isolating due to flu-like or cold-like symptoms (B = 0.175 ± 0.044, t(25.83) = 3.981, P < 0.001, d = 0.183, 95% CI = [0.092, 0.283]). As shown in Table 2 for hypothesis 1b, participants who were in the two reappraisal conditions reported less negative state emotions than participants who were in the two control conditions (B = 0.185 ± 0.013, t(36.39) = 14.401, P < 0.001, d = 0.313, 95% CI = [0.270, 357]). In addition, the amount of variance explained by fixed effects in a model with only lockdown status as a fixed variable is marginal92 R2 = 0.003. The amount of variance explained by fixed effects in a model with only self-isolation due to symptoms as a fixed variable is marginal92 R2 = 0.001. The amount of variance explained by fixed effects in a model with only the contrast between the two reappraisal conditions and the two control conditions as the fixed variable is marginal92 R2 = 0.013. Across different measures of effect size, it is notable that the effects of reappraisal interventions on state negative emotions were of similar or even larger magnitude than the effects of lockdown status or self-isolation due to symptoms. This comparison suggests that reappraisal interventions could help to alleviate the emotional toll caused by lockdown and self-isolation. Thus, we believe that the effects of reappraisal interventions are not only statistically significant but also practically meaningful.

Discussion

The current study had two main goals. The first was to examine the shared effects of two brief reappraisal interventions (versus both passive and active control conditions) on negative and positive emotions in response to the COVID-19 pandemic, and to determine whether these effects were similar or different across countries/regions and COVID-19 situations. The second goal was to examine the potentially unique effects of the two reappraisal interventions—reconstrual and repurposing—on negative and positive emotions.

Regarding the first goal, we predicted and found that both reappraisal interventions (versus both control conditions combined) consistently decreased negative emotional responses (hypothesis 1) and consistently increased positive emotional responses (hypothesis 2) across all primary outcome measures: immediate emotions in response to each photo about the COVID-19 situation, state emotions after viewing all the photos related to the COVID-19 situation and overall emotions about the COVID-19 situation. Exploratory analyses suggested that both reappraisal interventions also improved participants’ reported emotions compared with at the beginning of the study and the emotions they anticipated feeling in the future.

Further exploratory analyses suggested that despite substantial local variations in how severe the pandemic was at the time data were collected and cultural differences in how people understand and respond to emotions90,93, the intervention effects appeared in almost all of the countries/regions we studied. For example, in comparing participants’ immediate negative emotional responses to the photos related to the COVID-19 situation, 33 out of the 37 (89%) countries/regions with high statistical power (over 200 participants) showed statistically significant effects of reappraisal interventions. Although reappraisal interventions tended to have larger effects among females (versus males), and among unemployed individuals without income, the effects were largely unqualified by education level, subjective socioeconomic status, and whether a participant’s country/region was under lockdown.

Regarding the second goal, we predicted that reconstrual would be more effective at reducing negative emotions than repurposing (hypothesis 3), but repurposing would be more effective at increasing positive emotions than reconstrual (hypothesis 4). We found little to no support for these hypotheses, as neither was reliably better than the other at reducing negative emotions or increasing positive emotions across outcomes. The finding that the two forms of reappraisal were similarly effective at regulating emotions in the context of COVID-19 is consistent with the idea that the pandemic offers a wide array of affordances both for construing emotional situations in different ways, thus enabling reconstrual, and for evaluating these situations in light of different goals, thus enabling repurposing76. This implies that it may be beneficial to combine both strategies, a hypothesis that future studies can be designed to test. It also remains to be investigated whether reconstrual and repurposing offer similarly comparable benefits in other contexts.

The comparable effectiveness of reconstrual and repurposing in this context raises interesting questions about these two forms of reappraisal. We found that even though participants learned only one form of reappraisal, they reported using both strategies more often than in either control condition. This overlap might have stemmed from insufficient differentiation between the reappraisal instructions used in this study. It may also mean that the distinction between repurposing and reconstrual, although useful theoretically, is not readily accessible to lay people. Alternatively, this overlap may have stemmed from reconstrual and repurposing being mutually associated to a degree that being instructed to use one strategy primes the other strategy. Future research is needed to more directly investigate these possibilities.

After assessing results related to the primary goals, an important question was whether reducing negative emotions and increasing positive emotions in response to the pandemic might inadvertently come at the cost of decreasing intentions to engage in preventive health behaviour (reviewed in ref. 94). Reassuringly, the reappraisal interventions improved emotions without significantly reducing intentions to practice preventive health behaviours. This is consistent with recent findings that there are many paths to motivate preventive health behaviours during the COVID-19 pandemic without inducing negative emotions95,96,97,98.

Our results highlight the benefits of applying reappraisal interventions at scale to increase psychological resilience and to mitigate the adverse impacts of the COVID-19 pandemic—benefits that could potentially be applied in other contexts that elicit negative emotions. Importantly, the effects of the intervention were not meagre: the extent to which emotions were changed by our reappraisal interventions was comparable in magnitude to the extent to which emotions differed between people who faced extreme hardships (lockdowns or symptom-induced isolations) and people who experienced neither of these hardships. Thus, contextualising the effect sizes of reappraisal interventions in this manner suggests that the interventions are practically meaningful. This practical meaning matters in light of findings that people on average do not appear to fully recover their emotional well-being even after six months into the COVID-19 pandemic99, that stress and depression can impair vaccine efficacy100, and that negative emotions predispose morbidity and mortality via increases in substance use and other risky behaviours101. Essential workers, nurses and doctors, students, patients and many other populations whose work and life are highly affected by the pandemic could potentially benefit from reappraisal interventions, although more research is needed to establish the effectiveness of reappraisal for groups facing distinct challenges. Because these interventions are inexpensive, brief and scalable, they could be implemented through a variety of media and communication mechanisms, such as advertising campaigns102, speeches, courses, apps and mobile games103.

Our results also have important implications for the science of emotion (reviewed in ref. 104) and for emotion regulation (reviewed in refs. 35,39) in particular. Despite the fact that reappraisal is one of the most researched topics in psychology35, this study is the largest cross-cultural investigation of reappraisal that has been conducted to date, drawing diverse samples from well beyond the WEIRD (western, educated, industrialized, rich and democratic) societies105 that have been heavily represented to date in social science. Thus, the findings reveal the generalizability of reappraisal effects across many countries/regions even in the context of substantial, protracted stressors. The present study also extends understanding of how contextual moderators influence reappraisal processes (for example, individualism, lockdown status and demographics) while deepening understanding of distinct forms of reappraisal (that is, comparing them in relation to multiple outcomes). Finally, our study provides a rich dataset for examining many other questions related to emotions, emotion regulation and cultural differences. We look forward to seeing what other insights can be generated from this dataset.

Despite the encouraging findings, several limitations should be noted. One limitation is the use of convenience sampling and a limited set of photos. Our sample was not nationally representative within each country/region, and it appeared to over-represent females, younger people and people with internet access. The photos used in the study, although carefully chosen, might not represent local situations for different groups of participants. Future research is needed to assess generalizability using nationally representative samples and more personally emotionally evocative stimuli. A second limitation is that we cannot fully rule out the influence of demand characteristics and expectancies. Although we attempted to match demand characteristics and expectancies in the reappraisal conditions using our active control condition, we did not quantify the extent to which they were comparable, and we measured perceived strategy effectiveness after participants had used the strategies, which is different from expectancies formed upon reading the instructions but before using the strategies. Future research should assess the influence of demand characteristics and expectancies. A third limitation relates to the fact that the current study examined only the immediate and proximal effects of the interventions. Future research employing longitudinal designs is needed to examine whether the effects persist over time and at what intervals individuals might optimally engage in reappraisal. A fourth limitation is that the current study examined only a limited number of outcomes via self-report measures. More comprehensive evaluations, including assessments of actual behaviours (rather than intentions) and health outcomes, are necessary to determine whether there are any additional benefits or unintended consequences of the interventions. Finally, before implementing reappraisal interventions for practice, more research is needed to better evaluate the intervention (for example, via formal cost-benefit analysis and/or using the ‘reach, efficacy, adoption, implementation and maintenance’ framework106,107).

In conclusion, our findings demonstrated that two brief reappraisal interventions had robust and generalizable effects in reducing negative emotions and increasing positive emotions during the COVID-19 pandemic across countries/regions, without reducing intentions to practice preventive health behaviours. We hope this study will inform efforts to create scalable interventions for use around the world to build resilience during the pandemic and beyond.

Methods

Ethics information and participants

This study is one of three studies in the PSA COVID-19 Rapid Project. The other two studies investigated the effects of loss and gain message framing and self-determination theory-guided message framing, respectively. The other two studies are reported elsewhere. The study was conducted online, and participants clicked a single data collection link that led to either the current study or the other two studies in the COVID-19 Rapid Project. A comprehensive summary of the PSA COVID-19 Rapid Project—including descriptions of the study selection procedure, the other selected studies, the internal peer review process, and implementation plans—can be found at https://psyarxiv.com/x976j/.

Participants were recruited by the PSA network. The PSA recruited 186 member laboratories from 55 countries/regions speaking 42 languages. Of the 27,989 participants recruited to complete the current study (not counting participants for the other two studies in the PSA COVID-19 Rapid Project), 4,050 were recruited through semi-representative panelling (on the basis of sex, age and sometimes ethnicity) from the following countries/regions: Egypt, Kenya, Nigeria, South Africa, Mexico, United States, Austria, Romania, Russia, Sweden, Switzerland, United Kingdom, China, Japan and South Korea (270 participants per country/region). The remaining participants were recruited through the research groups by convenience sampling. Each research group obtained approval from their local Ethics Committee or IRB to conduct the study, explicitly indicated that their institution did not require approval for the researchers to conduct this type of task, or explicitly indicated that the current study was covered by a pre-existing approval. Although the specifics of the consent procedure differed across research groups, all participants provided informed consent. The style and the amount of compensation varied with local conventions (a common practice in PSA). More information regarding participant compensation and sample size can be found at https://psyarxiv.com/x976j/.

Procedure

An overview of the experiment is depicted in Fig. 2.

Pre-measure

Before reading the instructions, participants reported emotions they felt in the moment (details for all study measures are described in the next section). These ratings constituted a baseline emotional measure.

Randomization to condition

Following the pre-measure, participants were randomly assigned to one of four between-subjects experimental conditions: two reappraisal intervention conditions (reconstrual and repurposing), one active control condition and one passive control condition. Because the study was conducted online, data collection was performed blind to the conditions of the participants. The content of the instructions in each condition differed, but the lengths were matched except for the passive control condition, which had a shorter set of instructions.

Participants in the two reappraisal intervention conditions (reconstrual and repurposing) and the active control condition received the following instructions: “In this study, we will show you photographs related to COVID-19 from various news sources. Our goal is to better understand how people respond to such photos, which may include feelings of fear, anger, and sadness. Sometimes emotions like these are helpful. At other times, however, these emotions can be unhelpful to us. Researchers have found that when people think their emotions are unhelpful, they can take steps to influence their emotions.”

In the reconstrual condition, participants were told that (emphasis in original) “One strategy that some people find helpful for influencing their emotions is rethinking. This strategy involves changing one’s thinking in order to change one’s emotions. This strategy is based on the insight that different ways of interpreting or thinking about any situation can lead to different emotions. This means that finding new ways of thinking about a situation can change how you feel about the situation. For example, consider someone who stays at home under lockdown due to COVID-19 and is feeling anxious, sad, or angry. In this case, rethinking might involve realizing that the situation is only temporary because dedicated people across the world are working hard to find a vaccine.” Participants were then given four examples of how rethinking might be employed for the COVID-19 situation (Example 1: “I know from world history that keeping calm and carrying on gets us through tough times.”; Example 2: “Scientists across the world are working hard to find treatment and vaccines. Throughout history, humans have been resourceful in finding solutions to new challenges.”; Example 3: “Washing hands, avoiding touching my face, keeping a safe distance…There are simple and effective things I can do to protect myself and my loved ones from getting sick and to stop the spread of the virus.”; Example 4: “In the past, people have overcome many challenges that seemed overwhelming at the time, and we will overcome COVID-19 related challenges too.”).

In the repurposing condition, participants were told that (emphasis in original) “One strategy that some people find helpful for influencing their emotions is refocusing. This strategy involves changing one’s thinking in order to change one’s emotions. This strategy is based on the insight that finding something good in even the most challenging situations can lead to different emotional responses. This means that refocusing on whatever good aspects may be found in a situation can change how you feel about the situation. For example, consider someone who stays at home under lockdown due to COVID-19 and is feeling anxious, sad, or angry. In this case, refocusing might involve realizing that staying at home gives them time to do things that they may not have been able to do before, like reading, painting, and spending time with family.” Participants were then given four examples of how refocusing might be employed for the COVID-19 situation (Example 1: “This situation is helping us realize the importance of meaningful social connections, and helping us understand who the most important people in our lives are.”; Example 2: “Medical systems are now learning to deal with amazing challenges, which will make them much more resilient in the future.”; Example 3: “Even though we are physically apart, we are finding creative ways to stay connected and our hearts are more connected than ever.”; Example 4: “I have been inspired by the way that frontline health care workers have responded with resilience, generosity, determination, and deep commitment.”).

In the active control condition, participants were asked to reflect on their emotions as they unfold. This condition is inspired by the literature on expressive writing and experimental disclosure, which shows that asking people to reflect about their very deepest thoughts and feelings can improve psychological health77,78. By having an active control condition, which was likely to lead to some benefit to participants, we can make stronger inferences regarding the impact of reappraisal interventions relative to a potentially useful strategy designed to equate demand characteristics and expectancies. In the instructions, participants were told that (emphasis in original) “One strategy that some people find helpful for influencing their emotions is reflecting. This strategy involves allowing oneself to freely experience and reflect on one’s thoughts and feelings. This strategy is based on the insight that reflecting on your thoughts and feelings about any situation can lead to different emotional responses. This means that exploring your thoughts and emotions can change how you feel about the situation. For example, consider someone who stays at home under lockdown due to COVID-19 and is feeling anxious, sad, or angry. In this case, reflecting might involve allowing oneself to experience these feelings and be fully immersed in the lockdown experience, reflecting on the meaning this situation has for the person and their loved ones.” Participants were then given four examples of how reflecting might be employed for the COVID-19 situation (Example 1: “This situation is changing so fast, and I don’t know how the future will develop.”; Example 2: “People are struggling to cope with these unprecedented and overwhelming challenges.”; Example 3: “Someone I love might get sick and there might not even be ventilators to help them.”; Example 4: “I really wish we could find a vaccine soon.”).

To reinforce what they had learned, participants in the two reappraisal conditions and the active control condition were then asked to summarize, in one or two sentences, the strategy they had just learned. This text response was collected only for exploratory purposes and was not used in confirmatory analysis.

In the passive control condition, participants received the following instructions: “In this study, we will show you photographs related to COVID-19 from various news sources. Our goal is to better understand how people respond to such photos, which may include feelings of fear, anger, and sadness. As you view these photographs, please respond as you naturally would.” Having a passive control condition allowed us to have clear interpretations in the case that we found no significant difference in our contrast between both the reappraisal conditions combined and both the control conditions combined. If this was the case, we would have compared each reappraisal condition against the passive control condition and compared the active control condition against the passive control condition in the exploratory analysis to determine whether each strategy had a non-zero impact relative to individuals’ natural responses.

Practice trials

After receiving instructions by condition, participants were asked to practice the strategy in two trials designed to facilitate their understanding of the strategy. The practice trials included providing ratings and written responses to two photographs (per prior research108). The photographs in this study were selected by our research team from major media news sources (CNN, New York Times, The Guardian and Reuters) and present situations in Asia, Europe and North America. They were rated by our team to evoke either sadness or anxiety above the midpoint on a seven-point scale ranging from ‘not at all’ to ‘very’ and to score close to or above the midpoint on a seven-point scale ranging from ‘not at all’ to ‘very’ on the question “How much do you recommend using this picture?” (photographs available at https://osf.io/8bjnz/). In each practice trial, participants saw a ‘negative’ photo related to the COVID-19 situation (for example, an exhausted doctor or medical workers in hazmat suits) and a reminder above the photo to use the strategy that was presented to them. In the reconstrual condition, the reminder was “As you view the photo, draw on the examples we gave you earlier in order to interpret the situation in a new way.” In the repurposing condition, the reminder was “As you view the photo, draw on the examples we gave you earlier in order to focus on any good you can find in the situation.” In the active control condition, the reminder was “As you view the photo, draw on the examples we gave you earlier in order to reflect on your thoughts and feelings.” In the passive control condition, the reminder was “As you view the photo, respond as you naturally would.” After 10 s, participants were asked to rate their emotions in response to the photo using two corresponding unipolar five-point Likert scales, one for negative emotion and one for positive emotion. These ratings were designed to familiarize participants with the task, and were not used in the confirmatory analyses. After each photo, participants in the two reappraisal conditions and the active control condition were asked to write (in text) how they applied the strategy while observing the photo. Participants in the passive control condition were asked to write (in text) anything that comes naturally to their mind about the photo. The text response was also collected only for exploratory purposes and was not used in the confirmatory analysis. Participants in the two reappraisal conditions and the active control condition were then given one example of how the photo might be viewed (examples varied by condition). Note that the two reappraisal conditions and the active control condition were designed to be matched for demand characteristics and expectancy.

Experimental trials

Following the two practice trials, participants viewed additional photos related to the COVID-19 situation in ten experimental trials. Participants in the two reappraisal conditions and the active control condition were asked to use the strategy that they practiced, and participants in the passive control condition were asked to respond naturally. All participants saw exactly the same ten photos, but the order of the presentation was randomized across the ten experimental trials. Each photo was presented to participants with the same reminder used in the practice trials. After observing each photo for ten seconds, participants were asked to rate both their negative and positive emotions in response to the photo using the same five-point Likert scales from the practice trials.

Post-measures

In the final section of the study, participants completed several measures, including (1) negative and positive state emotions, (2) negative and positive emotions about the COVID-19 situation, (3) negative and positive anticipated emotions, (4) behavioural intentions, (5) motivation/beliefs, and (6) manipulation check.

Measures

Demographics

At the beginning of the study, participants completed a general survey that included demographic questions and some questions related to COVID-19 shared by all three studies in the PSA COVID-19 Rapid Project. Details about the general survey can be found at https://osf.io/7axc4/. While we originally planned for the general survey to appear at the end of the study, it was necessary for recruitment purposes (selecting representative panels) that it appear at the beginning of the study.

Baseline emotions

To assess baseline emotion, we asked participants how they were feeling right now at the beginning of the session on a five-point scale ranging from 1 (not at all) to 5 (extremely) (all response options were labelled and numbers were not displayed to participants for clarity). For negative baseline emotions, we measured five items on fear, anger, sadness, distrust and stress from the modified differential emotions scale109. For positive baseline emotions, we measured five items on hope, gratitude, love, inspiration and serenity from the modified differential emotions scale109 (details for all scoring rules are described in ‘Analysis plan’). We also measured three items on loneliness110 and three items on social connectedness111. These six items also were included in the assessment of post-photo state emotions and in the assessment of anticipated emotions (at each assessment point, these six items were used in exploratory analyses).

Negative emotional responses

In order to capture descriptively rich, nuanced data, we measured negative emotional responses in four ways. The first way is to measure negative emotions in response to the photos. For each photo, we asked participants how negative the photo made them feel using a unipolar scale ranging from 1 (not at all) to 5 (extremely). The second way is to measure negative state emotions after viewing all ten photos. We asked participants “how you are feeling right now” with the same set of items used to measure baseline emotions, which included five negative state emotions of fear, anger, sadness, distrust and stress. The third way is to measure negative emotions about the COVID-19 situation. We asked participants how negative/hopeless they were feeling about the COVID-19 situation right now on a unipolar scale ranging from 1 (not at all) to 5 (extremely). The fourth way is to measure negative anticipated emotions, which were an exploratory outcome. We asked participants “In the next week, to what extent, if at all, do you think you will feel each of the following?” with the same set of items used to measure baseline emotions, which included five negative anticipated emotions of fear, anger, sadness, distrust and stress.

Positive emotional responses

Following a parallel procedure, we measured positive emotional responses in four ways. The first way is to measure positive emotions in response to the photos. For each photo, we asked participants how positive the photo made them feel using a unipolar scale ranging from 1 (not at all) to 5 (extremely). The second way is to measure positive state emotions after viewing all ten photos. We asked participants “how you are feeling right now” with the same set of items used to measure baseline emotions, which included five positive state emotions of hope, gratitude, love, inspiration, and serenity. The third way is to measure positive emotions about the COVID-19 situation. We asked participants how positive/hopeful they were feeling about the COVID-19 situation right now on a unipolar scale ranging from 1 (not at all) to 5 (extremely). The fourth way is to measure positive anticipated emotions, which were an exploratory outcome. We asked participants “In the next week, to what extent, if at all, do you think you will feel each of the following?” with the same set of items used to measure baseline emotions, which included five positive anticipated emotions of hope, gratitude, love, inspiration and serenity.

Behavioural intentions

In addition to the emotional responses that are central to our four confirmatory hypotheses in this study, we also examined exploratory outcomes concerning behavioural intentions. Such intentions matter because they have been shown to predict actual behaviours112,113. Following protocols from Fishbein and Ajzen114, we asked participants to indicate on a 7-point scale ranging from 1 (extremely unlikely) to 7 (extremely likely) their intentions to engage in each of 10 different behaviours within the next week. Five of the items concern potentially harmful behaviour, which we chose based on documented links between negative emotions and substance use, aggressive behaviour and excessive information seeking17,25,115. Items included: drinking too much alcohol, using too much tobacco (for example, smoking or vaping) or other recreational drugs, yelling at someone, taking anger out online and spending too much time on media. The other five items concerned beneficial behaviour, which we chose based on evidence that positive emotions contribute to more health behaviours84,85. Items include: eating healthy food, getting enough physical activity, practicing healthy sleep habits (for example, going to bed and waking at regular hours), washing hands regularly for at least 20 s, and following a stay-at-home order stringently (if there isn’t an order in your region now, assume that one is imposed).

Motivation and beliefs

We measured both the motivation to use the emotion regulatory strategy and the belief in the effectiveness of the emotion regulatory strategy as exploratory moderators71,87. We asked “Recall the instructions we gave you for viewing the photos. To what extent, if at all, do you agree or disagree with the following statements?” Motivation to use the emotion regulatory strategy was measured with the item: “I tried my hardest to follow the instructions I was given while viewing the photos.” Belief in the effectiveness of the emotion regulatory strategy employed by participants was measured with the item “I believed that following the instructions would influence my emotions.” Participants rated their answers using a 7-point scale ranging from 1 (strongly disagree) to 7 (strongly agree).

Manipulation check

We planned to evaluate participants’ attention to our instructions and photos using two multiple-choice questions. The first question asked participants to choose the instructions they had at the beginning of the survey from among four options. The second question asked participants to choose the photo that was not shown to them in the survey from among three options.

For exploratory purposes, we also asked how often participants actually used each approach when viewing the photographs and their global change of emotions compared to the beginning of the study. Participants were asked, “When viewing the ten photographs related to COVID-19 earlier, how often did you use each of the following approaches?” and rated four approaches: “responding as I naturally would,” “reflecting on my thoughts and feelings,” “interpreting the situation in a new way,” and “focusing on any good I could find in the situation.” Participants rated their answers using a 5-point scale ranging from 1 (never) to 5 (always). To measure global change of emotion, participants were asked, “Overall, compared to the beginning of this study, how negative do you feel right now?” using a 5-point scale ranging from 1 (much more negative) to 5 (much less negative) and “Overall, compared to the beginning of this study, how positive do you feel right now?” using a 5-point scale ranging from 1 (much more positive) to 5 (much less positive).

Order of items

For measures above, items belonging to the negative category (that is, negative emotional responses and intentions for harmful behaviour) and to the positive category (that is, positive emotional responses and intentions for beneficial behaviour) were presented in a counterbalanced order within each measure across participants. In other words, half of the participants always rated an item from the negative category first and then an item from the positive category, whereas the other half always rated an item from the positive category first and then an item from the negative category. For measures that have multiple items, items belonging to the negative category were randomized within the negative category, and items belonging to the positive category were randomized within the positive category. When the same set of items used to measure baseline emotions was repeated, the set had the same order for every given participant.

Analysis plan

Pre-processing

Exclusion

We planned to exclude (1) participants who answered both multiple-choice manipulation check questions incorrectly, and (2) participants who completed fewer than 50% of the questions in the study.

Reliability of measures

For items from the modified differential emotions scale109, we planned to create overall negative emotion scores at each time point by averaging the five negative emotions (fear, anger, sadness, distrust and stress) and overall positive emotion scores at each time point by averaging the five positive emotions (hope, gratitude, love, inspiration and serenity) if the average inter-item correlation was above 0.40 for negative emotions and for positive emotions, respectively. If the average inter-item correlation was below 0.40, we would conduct an exploratory factor analysis with oblique rotation and maintain factors with an eigenvalue above 1.00. If no factors had an eigenvalue above 1, we would report results by item rather than as a composite. The actual average inter-item correlation was 0.50 for negative baseline emotions and 0.48 for positive baseline emotions. Therefore, we created overall negative emotion scores at each time point by averaging the five negative emotions and overall positive emotion scores at each time point by averaging the five positive emotions.

Missing data

We dropped incomplete cases on an analysis-by-analysis basis. Given our sampling plan described below, we should have power of 0.95 or above.

Outliers

In order to be maximally conservative, we did not define or identify outliers.

Analytic plan for hypotheses

Since negative emotional responses and positive emotional responses are separable79,80, we examined negative emotional responses and positive emotional responses separately. To control family-wise error rates in multiple comparisons, we used the Holm–Bonferroni method within each of the four hypotheses separately. For all analyses testing negative emotional responses (hypothesis 1 and hypothesis 3), we planned to control for the participants’ negative baseline emotions. As originally intended by the scale109, we planned to create an overall negative baseline emotion score by averaging the five negative emotions (fear, anger, sadness, distrust and stress). For all analyses testing positive emotional responses (hypothesis 2 and hypothesis 4), we planned to control for the participants’ positive baseline emotions. As originally intended by the scale109, we planned to create an overall positive baseline emotion score by averaging the five positive emotions (hope, gratitude, love, inspiration and serenity). To account for the nested structure in our data (for example, participant nested by country/region), we fitted multilevel models with the condition using the contrast in Table 1, random by-country/region slopes, and random by-country/region intercepts. If a model failed to converge, we planned to explore other reasonable models113 and report results of all explored models in an appendix. We visually assessed assumptions of heteroscedasticity and normality of residuals and found no severe deviations. All tests were two-tailed.

Although we used the frequentist approach for confirmatory analyses, we also reported Bayes factors for every result to gain information about the strength of evidence provided by the data comparing the null and alternative hypotheses116. If we obtained non-significant results from the frequentist approach, we used Bayes factors to help us interpret non-significant results and differentiate between insensitive results and those that reveal good enough evidence supporting the null hypothesis. We set these evidence thresholds to BF10 >10 for H1 and BF10 <0.1 for H0. If Bayes factors did not cross the evidence thresholds, we think our sample size is sufficiently large that inconclusive results at this sample size would be an important message for the field. We used informed priors for the alternative model: a one-tailed Cauchy distribution with a mode of zero and a scale r = 0.18 (hypotheses 1 and 2), r = 0.17 (hypothesis 3) and r = 0.25 (hypothesis 4) on the standardized effect size. These priors were based on the lowest available estimates of effect sizes in past research (more information in ‘Sampling plan’). At stage 1, we wrote the code for the Bayesian part of our analysis plan using the BayesFactor package117 in R. We also planned to investigate the sensitivity of our conclusions to priors using robustness regions118, which involves calculating a Bayes factor under a large number of different priors to see how the Bayes factor changes. After we collected our data, we made the following adjustments to our plans for our Bayesian analysis. First, to estimate the Bayesian models, we switched from the BayesFactor package to the brms package119 because of its superior handling of random effects. Our brms models used four chains, each with 1,000 warm-up samples, 10,000 post-warm-up samples and a thinning rate of 1. To calculate Bayes factors, we used bridge sampling, as implemented in the bayestestR120 and bridgesampling121 packages, to compare the marginal likelihoods of the full model versus a null model that does not contain one of our two focal contrasts. Second, we discovered that the Bayesian versions of our models involving emotional responses to the photos had high computational requirements due to the inclusion of two sources of random effect (country/region and participant) rather than one. To make these models more computationally manageable we simplified the dataset by computing the average emotional response to each photo for each participant and using this as the outcome variable. This allowed us to omit the by-participant random effect in these models and drastically reduce the resource requirements and compute time. Although these simplified models do not separate participant-specific variance from error variance, our analysis plan had no plans to interpret these sources of variation separately, so we reasoned this simplification was a fair way to obtain the same mathematical results as required by our analysis plan at a lower computational cost. Finally, we simplified the robustness analyses by only investigating how the Bayes factors change with one very large prior (r = 1.0) rather than computing full robustness regions. We made this last change to once again reduce the compute time to manageable levels. If the Bayes factors under the large prior are in line with those generated by the pre-registered priors (which are already very small), the results should be robust to other reasonable priors.

Tests for hypotheses 1 and 3

Overall, we expected that reappraisal interventions (versus control) would reduce negative emotional responses (hypothesis 1), and that reconstrual would lead to greater decreases in negative emotional responses than repurposing (hypothesis 3). We tested hypothesis 1 and hypothesis 3 using two orthogonal contrasts (Table 1). The first contrast is between both reappraisal conditions combined and both control conditions combined for hypothesis 1. The second contrast is between the reconstrual condition and the repurposing condition for hypothesis 3. Negative emotional responses were measured in four ways (negative emotions in response to the photos, negative state emotions after viewing the photos, negative emotions about the COVID-19 situation, and negative anticipated emotions). We had confirmatory hypotheses regarding the first three outcomes and examined negative anticipated emotions in the exploratory analysis. Therefore, hypothesis 1 can be subdivided into hypotheses 1a to 1c, and hypothesis 3 can be subdivided into hypotheses 3a to 3c. We planned to consider a hypothesis to be supported if at least 1 of the 3 sub-hypotheses is significant after Holm–Bonferroni correction (controlling for 3 comparisons within each hypothesis). If we found non-significant results for any sub-hypothesis, we compared each reappraisal condition against the passive control condition and compared the active control condition against the passive control condition in the exploratory analysis to determine whether each strategy had a non-zero impact relative to individuals’ natural responses.

Testing effects on negative emotions in response to the photos

We expected that reappraisal interventions (versus control) would reduce negative emotions in response to the photos (hypothesis 1a), and reconstrual would lead to greater decreases in negative emotional responses in response to the photos than repurposing (hypothesis 3a). We modelled ratings of negativity in response to each photo in the experimental trials as a function of the fixed effects of condition using our contrast. We included by-participant random intercepts, by-country/region random intercepts, as well as by-country/region random slopes for each contrast.

Testing effects on negative state emotions

We expected that reappraisal interventions (versus control) would reduce negative state emotions (hypothesis 1b) and reconstrual would lead to greater decreases in negative state emotions than repurposing (hypothesis 3b). Similar to creating the overall negative baseline emotion score, we planned to create an overall negative state emotion score by averaging the five negative emotions (fear, anger, sadness, distrust and stress). We modelled the overall negative state emotion score as a function of the fixed effects of condition using our contrast. We included by-country/region random intercepts, as well as by-country/region random slopes for each contrast.

Testing effects on negative emotions about the COVID-19 situation

We expected that reappraisal interventions (versus control) would reduce negative emotions about the COVID-19 situation (hypothesis 1c), and reconstrual would lead to greater decreases in negative emotions about the COVID-19 situation than repurposing (hypothesis 3c). We modelled negative emotions about the COVID-19 situation as a function of the fixed effects of condition using our contrast. We included by-country/region random intercepts, as well as by-country/region random slopes for each contrast.

Tests for hypotheses 2 and 4

Overall, we expected that reappraisal interventions (versus control) would increase positive emotional responses (hypothesis 2), and repurposing would lead to greater increases in positive emotional responses than reconstrual (hypothesis 4). We tested hypothesis 2 and hypothesis 4 using two orthogonal contrasts (Table 1). The first contrast is between both reappraisal conditions combined and both control conditions combined for hypothesis 2. The second contrast is between the reconstrual condition and the repurposing condition for hypothesis 4. Positive emotional responses were measured in four ways (positive emotions in response to the photos, positive state emotions after viewing the photos, positive emotions about the COVID-19 situation, and positive anticipated emotions). We had confirmatory hypotheses regarding the first three outcomes and examined positive anticipated emotions in an exploratory analysis. Therefore, hypothesis 2 can be subdivided into hypotheses 2a to 2c, and hypothesis 4 can be subdivided into hypotheses 4a to 4c. We planned to consider a hypothesis to be supported if at least 1 of the 3 sub-hypotheses is significant after Holm–Bonferroni correction (controlling for 3 comparisons within each hypothesis). If we found non-significant results for any sub-hypothesis, we would compare each reappraisal condition against the passive control condition and compare the active control condition against the passive control condition in the exploratory analysis to determine whether each strategy had a non-zero impact relative to individuals’ natural responses.

Testing effects on positive emotions in response to the photos

We expected that reappraisal interventions (versus control) would increase positive emotions in response to the photos (hypothesis 2a), and that repurposing would lead to greater increases in positive emotions in response to the photos than reconstrual (hypothesis 4a). We modelled ratings of positivity in response to each photo in the experimental trials as a function of the fixed effects of condition using our contrast. We included by-participant random intercepts, by-country/region random intercepts, as well as by-country/region random slopes for each contrast.

Testing effects on positive state emotions

We expected that reappraisal interventions (versus control) would increase positive state emotions (hypothesis 2b), and repurposing would lead to greater increases in positive state emotions in response to the photos than reconstrual (hypothesis 4b). Similar to creating the overall positive baseline emotion score, we planned to create an overall positive state emotion score by averaging the five positive emotions (hope, gratitude, love, inspiration and serenity). We modelled the overall positive state emotion score as a function of the fixed effects of condition using our contrast. We planned to include by-country/region random intercepts, as well as by-country/region random slopes for each contrast. However, the model could not converge when we included by-country/region random slopes for contrast 2. To make the model converge, we did not include by-country/region random slopes for contrast 2.

Testing effects on positive emotions about the COVID-19 situation

We expected that reappraisal interventions (versus control) would increase positive emotions about the COVID-19 situation (hypothesis 2c), and repurposing would lead to greater increases in positive emotions about the COVID-19 situation than reconstrual (hypothesis 4c). We modelled positive emotions about the COVID-19 situation as a function of the fixed effects of condition using our contrast. We included by-country/region random intercepts, as well as by-country/region random slopes for each contrast.

Exploratory analyses

We conducted a series of exploratory analyses to address supplemental questions regarding our hypotheses, including, but not limited to: (1) Were there any differences in other pairwise comparisons in testing hypotheses 1–2? (2) Were there emotion-specific effects of reappraisal122? (3) Were the effects on emotions subjectively detectable by participants123? Did the effects of strategy use vary by (4) motivation to use the strategy71; (5) beliefs in the strategy’s effectiveness87; or (6) the participant’s country of residence90?

We investigated the impacts of strategy use on other outcomes, including, but not limited to: (1) positive and negative anticipated emotions; (2) intentions to enact potentially harmful versus beneficial behaviours (results in Supplementary Table 14); and (3) loneliness and social connectedness (results in Supplementary Table 15).

Sampling plan

Expected effect sizes

In order to compare effect sizes across studies, below we report values of Cohen’s d, which in some cases were transformed or calculated from the results reported in the original studies (see Supplementary Table 16 for details). Several caveats are in order regarding the effect sizes that follow. First, meta-analyses tend to overestimate effect sizes, although the size of overestimation varies considerably across studies and sometimes shows no overestimation124. Second, most previous studies were conducted in the laboratory, whereas the current study was conducted online. Third, the current crisis is likely to lead to strong emotional responses, especially for participants who are facing financial or health-related setbacks, although strong negative emotions also motivate people to regulate emotions more64. These caveats suggest uncertainty in effect sizes.

In general, reappraisal has an average effect size of d = 0.45, 95% CI = [0.35, 0.56] in changing emotion experience relative to passive control conditions (that is, no instruction, instructions to experience naturally, instructions to not regulate in a certain manner, or instructions to enhance or maintain the focal emotion) (a meta-analysis39 finds no evidence of publication bias). Experimental disclosure and expressive writing, which inspired the instruction in the active control condition, have an average effect size of d = 0.07, 95% CI = [0.05, 0.17] in improving psychological health (including emotional responses), relative to engaging in non-treatment neutral activities (for example, describing what they have done in the past 24 h) or no activities (a meta-analysis77 finds evidence of publication bias). These works suggest the lowest available estimate of the effect size to be d = 0.18 (subtracting the upper bound of 95% CI d = 0.17 for experimental disclosure and expressive writing from the lower bound of 95% CI of d = 0.35 for the reappraisal interventions) between our reappraisal interventions and the control conditions for hypothesis 1 and hypothesis 2.

In relation to the comparison between reconstrual and repurposing, although prior research has not used the same theoretical framework76 to empirically contrast reconstrual and repurposing as we did in the current study, research on closely related constructs can provide estimates of effect sizes. Reconstrual is most similar to a previously studied subtype of reappraisal called ‘reappraising emotional stimulus’ in Webb, Miles and Sheeran’s meta-analysis39, which has a d = 0.38, 95% CI = [0.21, 0.55] in changing emotion experience (this effect size is primarily for negative emotions, as all but one study examined negative emotions). Repurposing is similar to the construct ‘benefit finding’ (perceiving positive consequences that resulted from a traumatic event), which is associated with positive well-being, d = 0.45, 95% CI = [0.37, 0.52], but not global distress, d = 0.00, 95% CI = [−0.04, 0.04] (meta-analysis81). Repurposing is also similar to the subtype of reappraisal called ‘positive reappraisal’, which is more effective in increasing positive thoughts than other types of reappraisals, d = 0.49, 95% CI = [0.25, 0.72] relative to detached reappraisal125. These works suggest the lowest available estimate of the effect size to be d = 0.17 (subtracting the upper bound of 95% CI d = 0.04 for the association between benefit finding and global distress from the lower bound of 95% CI of d = 0.21 for ‘reappraising emotional stimulus’39 between reconstrual and repurposing in changing negative emotions for hypothesis 3), and d = 0.25 (the lower bound of 95% CI of positive reappraisal in increasing positive thoughts than detached reappraisal125) between reconstrual and repurposing in changing positive emotions for hypothesis 4.

Sample size

For practical reasons, sample size was decided primarily on the basis of the availability of resources among members of the PSA.

Adjusted alpha levels

The tests of each hypothesis involved three comparisons, with α for the smallest P value being 0.017 (that is, 0.05/3), α for the second-smallest P value being 0.025 (that is, 0.05/2), and α for the largest P value being 0.05 (Holm–Bonferroni corrections).

Power analysis

We conducted a simulation study to estimate power for a variety of potential effect sizes (|d| = 0.05 to 0.29, separated by increments of 0.02), number of countries/regions (Ncountry/region = 30, 35, 40, 45, 50, 55, 60), within-country/region sample sizes (N = 200, 400, 600, 800), by-country/region intercept variances (σ2intercept = 0.05, 0.30, 0.55, 0.80), and by-country/region slope variances (σ2slope = 0.0, 0.02, 0.03, 0.04) at α = .017. The lowest level of intercept variances in our simulation was chosen on the basis of an ongoing multi-country/region project tracking rates of depression (σ2intercept = 0.04) and worries about the COVID-19 (σ2intercept = 0.06) across countries/regions during the COVID-19 outbreak126 (details in Supplementary Table 16). The lowest level of slope variances in our simulation was chosen on the basis of the average slope variance (σ2slope < 0.01) in a large multi-site, multi-country/region project involving 28 psychological manipulations127. The slope variances capture the variability of the effect of psychological manipulations, and there is no apparent reason to expect that the effect of reappraisal interventions on emotions is more variable than most other psychological manipulations reported in Klein et al.127. In fact, appraisal theories of emotion argue that the relationship between appraisals and emotions is culturally universal128, suggesting low variability. As one example to show that similar appraisals associate with similar emotional experiences, we found the associations varied little across countries/regions between perceived insufficient government response and depression (σ2slope = 0.003) and between perceived insufficient government response and worries (σ2slope = 0.003) during the COVID-19 pandemic126 (details in Supplementary Table 16), consistent with the observation of low slope variances (σ2slope < 0.01) in Klein et al. 127. Despite expecting low variability from empirical findings and theories, we tested a variety of intercept variances and slope variances in our power simulation, some of which were much higher than those reporrted in refs. 127,126 to be maximally conservative. We conducted 1,000 simulations for each set of simulation parameters using the simr package129 using computing power harnessed through the Open Science Grid130,131.

We show comprehensive results for our simulation study at https://osf.io/mf5z4/. In our final sample after pre-registered exclusion, 37 countries/regions had over 200 participants, surpassing the 95% power criterion based on simulations.

Reporting Summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

The analytic dataset is available at https://osf.io/jeu73/. Materials are available at https://osf.io/4yf9d/, with additional relevant materials for the PSA’s rapid-response COVID-19 projects at https://osf.io/s4hj2/.

Code availability

All analysis code (completed in R) is available at https://osf.io/jeu73/.

References

  1. 1.

    New poll: COVID-19 impacting mental well-being: Americans feeling anxious, especially for loved ones; older adults are less anxious. American Psychiatric Association https://www.psychiatry.org/newsroom/news-releases/new-poll-covid-19-impacting-mental-well-being-americans-feeling-anxious-especially-for-loved-ones-older-adults-are-less-anxious (5 May 2020).

  2. 2.

    Barari, S. et al. Evaluating COVID-19 public health messaging in Italy: self-reported compliance and growing mental health concerns. Preprint at medRxiv https://doi.org/10.1101/2020.03.27.20042820 (2020).

  3. 3.

    Hou, Z., Du, F., Jiang, H., Zhou, X. & Lin, L. Assessment of public attention, risk perception, emotional and behavioural responses to the COVID-19 outbreak: social media surveillance in China. Preprint at SSRN https://doi.org/10.2139/ssrn.3551338 (2020).

  4. 4.

    Kirzinger, A., Kearney, A., Hamel, L. & Brodie, M. KFF health tracking poll—early April 2020: the impact of coronavirus on life in America. The Henry J. Kaiser Family Foundation https://www.kff.org/health-reform/report/kff-health-tracking-poll-early-april-2020/ (2020).

  5. 5.

    Langer, G. Coronavirus impacts: disrupted lives, elevated stress, and soaring worry: Poll. ABC News https://abcnews.go.com/Politics/coronavirus-impacts-disrupted-lives-elevated-stress-soaring-worry/story?id=69812058 (5 May 2020).

  6. 6.

    Li, S., Wang, Y., Xue, J., Zhao, N. & Zhu, T. The impact of COVID-19 epidemic declaration on psychological consequences: a study on active Weibo users. Int. J. Environ. Res. Public Health 17, 2032 (2020).

    CAS  PubMed Central  Article  PubMed  Google Scholar 

  7. 7.

    McCarthy, J. Three in four Americans worry about COVID-19 exposure. Gallup.com https://news.gallup.com/poll/307646/three-four-americans-worry-covid-exposure.aspx (5 May 2020).

  8. 8.

    Pinnington, B. Britain’s view of COVID-19 as ‘high threat’ to their business jumps 19 points in a week. Ipsos MORI https://www.ipsos.com/ipsos-mori/en-uk/britains-view-covid-19-high-threat-their-business-jumps-19-points-week (5 May 2020).

  9. 9.

    Smith, M. How is COVID-19 affecting British opinions, jobs and wellbeing? YouGov https://yougov.co.uk/topics/politics/articles-reports/2020/03/24/how-covid-19-affecting-british-opinions-jobs-and-w (5 May 2020).

  10. 10.

    Zhao, Y., Cheng, S., Yu, X. & Xu, H. Chinese public’s attention to the COVID-19 epidemic on social media: observational descriptive study. J. Med. Internet Res. 22, e18825 (2020).

    PubMed  PubMed Central  Article  Google Scholar 

  11. 11.

    Jackson, C. & Newall, M. Partisanship reemerging in American views of coronavirus. Ipsos https://www.ipsos.com/en-us/news-polls/axios-ipsos-coronavirus-index (5 May 2020).

  12. 12.

    Stremikis, K. In COVID-19 tracking poll, Californians report worsening mental health, cost worries. California Health Care Foundation https://www.chcf.org/blog/covid-19-tracking-poll-californians-report-worsening-mental-health-cost-worries/ (2020).

  13. 13.

    COVID-19 and ending violence against women and girls. UN Women https://www.unwomen.org/en/digital-library/publications/2020/04/issue-brief-covid-19-and-ending-violence-against-women-and-girls (5 May 2020).

  14. 14.

    Sheppes, G., Suri, G. & Gross, J. J. Emotion regulation and psychopathology. Annu. Rev. Clin. Psychol. 11, 379–405 (2015).

    PubMed  Article  Google Scholar 

  15. 15.

    Cacioppo, J. T. & Cacioppo, S. The growing problem of loneliness. Lancet 391, 426 (2018).

    PubMed  PubMed Central  Article  Google Scholar 

  16. 16.

    Cooper, M. L., Frone, M. R., Russell, M. & Mudar, P. Drinking to regulate positive and negative emotions: a motivational model of alcohol use. J. Pers. Soc. Psychol. 69, 990–1005 (1995).

    CAS  PubMed  Article  Google Scholar 

  17. 17.

    Dorison, C. A. et al. Sadness, but not all negative emotions, heightens addictive substance use. Proc. Natl Acad. Sci. USA 117, 943–949 (2020).

    CAS  PubMed  Article  Google Scholar 

  18. 18.

    Kassel, J. D., Stroud, L. R. & Paronis, C. A. Smoking, stress, and negative affect: correlation, causation, and context across stages of smoking. Psychol. Bull. 129, 270–304 (2003).

    PubMed  Article  PubMed Central  Google Scholar 

  19. 19.

    Brod, S., Rattazzi, L., Piras, G. & D’Acquisto, F. ‘As above, so below’ examining the interplay between emotion and the immune system. Immunology 143, 311–318 (2014).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  20. 20.

    Denson, T. F., Spanovic, M. & Miller, N. Cognitive appraisals and emotions predict cortisol and immune responses: a meta-analysis of acute laboratory social stressors and emotion inductions. Psychol. Bull. 135, 823–853 (2009).

    PubMed  Article  Google Scholar 

  21. 21.

    Moons, W. G. & Shields, G. S. Anxiety, not anger, induces inflammatory activity: an avoidance/approach model of immune system activation. Emotion 15, 463–476 (2015).

    PubMed  Article  Google Scholar 

  22. 22.

    Alvaro, P. K., Roberts, R. M. & Harris, J. K. A systematic review assessing bidirectionality between sleep disturbances, anxiety, and depression. Sleep 36, 1059–1068 (2013).

    PubMed  PubMed Central  Article  Google Scholar 

  23. 23.

    Adam, T. C. & Epel, E. S. Stress, eating and the reward system. Physiol. Behav. 91, 449–458 (2007).

    CAS  PubMed  Article  Google Scholar 

  24. 24.

    Groesz, L. M. et al. What is eating you? Stress and the drive to eat. Appetite 58, 717–721 (2012).

    PubMed  Article  Google Scholar 

  25. 25.

    Birkley, E. L. & Eckhardt, C. I. Anger, hostility, internalizing negative emotions, and intimate partner violence perpetration: a meta-analytic review. Clin. Psychol. Rev. 37, 40–56 (2015).

    PubMed  PubMed Central  Article  Google Scholar 

  26. 26.

    Roberton, T., Daffern, M. & Bucks, R. S. Emotion regulation and aggression. Aggress. Violent Behav. 17, 72–82 (2012).

    Article  Google Scholar 

  27. 27.

    Foley, A. E. et al. The math anxiety–performance link: a global phenomenon. Curr. Dir. Psychol. Sci. 26, 52–58 (2017).

    Article  Google Scholar 

  28. 28.

    Diener, E., Thapa, S. & Tay, L. Positive emotions at work. Annu. Rev. Organ. Psychol. Organ. Behav. 7, 451–477 (2020).

    Article  Google Scholar 

  29. 29.

    Shockley, K. M., Ispas, D., Rossi, M. E. & Levine, E. L. A meta-analytic investigation of the relationship between state affect, discrete emotions, and job performance. Hum. Perform. 25, 377–411 (2012).

    Article  Google Scholar 

  30. 30.

    Haushofer, J. & Fehr, E. On the psychology of poverty. Science 344, 862–867 (2014).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  31. 31.

    Lerner, J. S., Li, Y., Valdesolo, P. & Kassam, K. S. Emotion and decision making. Annu. Rev. Psychol. 66, 799–823 (2015).

    PubMed  Article  PubMed Central  Google Scholar 

  32. 32.

    Bonanno, G. A. Loss, trauma, and human resilience: have we underestimated the human capacity to thrive after extremely aversive events? Am. Psychol. 59, 20–28 (2004).

    PubMed  Article  PubMed Central  Google Scholar 

  33. 33.

    Fletcher, D. & Sarkar, M. Psychological resilience. Eur. Psychol. 18, 12–23 (2013).

    Article  Google Scholar 

  34. 34.

    Southwick, S. M. & Charney, D. S. The science of resilience: implications for the prevention and treatment of depression. Science 338, 79–82 (2012).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  35. 35.

    McRae, K. & Gross, J. J. Emotion regulation. Emotion 20, 1–9 (2020).

    PubMed  Article  PubMed Central  Google Scholar 

  36. 36.

    Kalisch, R., Müller, M. B. & Tuescher, O. A conceptual framework for the neurobiological study of resilience. Behav. Brain Sci. 38, e92 (2014).

    PubMed  Article  PubMed Central  Google Scholar 

  37. 37.

    McRae, K. & Mauss, I. B. Increasing Positive Emotion in Negative Contexts: Emotional Consequences, Neural Correlates, and Implications for Resilience. Positive Neuroscience (Oxford Univ. Press, 2016).

  38. 38.

    Tugade, M. M. & Fredrickson, B. L. Regulation of positive emotions: emotion regulation strategies that promote resilience. J. Happiness Stud. 8, 311–333 (2007).

    Article  Google Scholar 

  39. 39.

    Webb, T. L., Miles, E. & Sheeran, P. Dealing with feeling: a meta-analysis of the effectiveness of strategies derived from the process model of emotion regulation. Psychol. Bull. 138, 775–808 (2012).

    PubMed  Article  PubMed Central  Google Scholar 

  40. 40.

    Denny, B. T. & Ochsner, K. N. Behavioral effects of longitudinal training in cognitive reappraisal. Emotion 14, 425–433 (2014).

    PubMed  Article  PubMed Central  Google Scholar 

  41. 41.

    Gross, J. J. Antecedent- and response-focused emotion regulation: divergent consequences for experience, expression, and physiology. J. Pers. Soc. Psychol. 74, 224–237 (1998).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  42. 42.

    Jackson, D. C., Malmstadt, J. R., Larson, C. L. & Davidson, R. J. Suppression and enhancement of emotional responses to unpleasant pictures. Psychophysiology 37, 515–522 (2000).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  43. 43.

    Denson, T. F., Grisham, J. R. & Moulds, M. L. Cognitive reappraisal increases heart rate variability in response to an anger provocation. Motiv. Emot. 35, 14–22 (2011).

    Article  Google Scholar 

  44. 44.

    Dillon, D. G. & LaBar, K. S. Startle modulation during conscious emotion regulation is arousal-dependent. Behav. Neurosci. 119, 1118–1124 (2005).

    PubMed  Article  PubMed Central  Google Scholar 

  45. 45.

    Ray, R. D., McRae, K., Ochsner, K. N. & Gross, J. J. Cognitive reappraisal of negative affect: converging evidence from EMG and self-report. Emotion 10, 587–592 (2010).

    PubMed  PubMed Central  Article  Google Scholar 

  46. 46.

    Chang, L. J., Gianaros, P. J., Manuck, S. B., Krishnan, A. & Wager, T. D. A sensitive and specific neural signature for picture-induced negative affect. PLoS Biol. 13, e1002180 (2015).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  47. 47.

    Dörfel, D. et al. Common and differential neural networks of emotion regulation by detachment, reinterpretation, distraction, and expressive suppression: a comparative fMRI investigation. NeuroImage 101, 298–309 (2014).

    PubMed  Article  PubMed Central  Google Scholar 

  48. 48.

    Hajcak, G. & Nieuwenhuis, S. Reappraisal modulates the electrocortical response to unpleasant pictures. Cogn. Affect. Behav. Neurosci. 6, 291–297 (2006).

    PubMed  Article  PubMed Central  Google Scholar 

  49. 49.

    Kalisch, R. et al. Anxiety reduction through detachment: subjective, physiological, and neural effects. J. Cogn. Neurosci. 17, 874–883 (2005).

    PubMed  Article  PubMed Central  Google Scholar 

  50. 50.

    Lohani, M. & Isaacowitz, D. M. Age differences in managing response to sadness elicitors using attentional deployment, positive reappraisal and suppression. Cogn. Emot. 28, 678–697 (2014).

    PubMed  Article  PubMed Central  Google Scholar 

  51. 51.

    Ochsner, K. N., Bunge, S. A., Gross, J. J. & Gabrieli, J. D. E. Rethinking feelings: an fmri study of the cognitive regulation of emotion. J. Cogn. Neurosci. 14, 1215–1229 (2002).

    PubMed  Article  PubMed Central  Google Scholar 

  52. 52.

    Schaefer, S. M. et al. Modulation of amygdalar activity by the conscious regulation of negative emotion. J. Cogn. Neurosci. 14, 913–921 (2002).

    PubMed  Article  PubMed Central  Google Scholar 

  53. 53.

    Shahane, A. D., Lopez, R. B. & Denny, B. T. Implicit reappraisal as an emotional buffer: reappraisal-related neural activity moderates the relationship between inattention and perceived stress during exposure to negative stimuli. Cogn. Affect. Behav. Neurosci. 19, 355–365 (2019).

    PubMed  Article  PubMed Central  Google Scholar 

  54. 54.

    English, T., John, O. P., Srivastava, S. & Gross, J. J. Emotion regulation and peer-rated social functioning: a 4-year longitudinal study. J. Res. Personal. 46, 780–784 (2012).

    Article  Google Scholar 

  55. 55.

    Davis, E. L. & Levine, L. J. Emotion regulation strategies that promote learning: reappraisal enhances children’s memory for educational information. Child Dev. 84, 361–374 (2013).

    PubMed  Article  PubMed Central  Google Scholar 

  56. 56.

    Ivcevic, Z. & Brackett, M. Predicting school success: comparing conscientiousness, grit, and emotion regulation ability. J. Res. Personal. 52, 29–36 (2014).

    Article  Google Scholar 

  57. 57.

    Gross, J. J. & John, O. P. Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. J. Pers. Soc. Psychol. 85, 348–362 (2003).

    PubMed  Article  PubMed Central  Google Scholar 

  58. 58.

    Aldao, A., Nolen-Hoeksema, S. & Schweizer, S. Emotion-regulation strategies across psychopathology: a meta-analytic review. Clin. Psychol. Rev. 30, 217–237 (2010).

    PubMed  Article  PubMed Central  Google Scholar 

  59. 59.

    Cludius, B., Mennin, D. & Ehring, T. Emotion regulation as a transdiagnostic process. Emotion 20, 37–42 (2020).

    PubMed  Article  PubMed Central  Google Scholar 

  60. 60.

    Appleton, A. A., Buka, S. L., Loucks, E. B., Gilman, S. E. & Kubzansky, L. D. Divergent associations of adaptive and maladaptive emotion regulation strategies with inflammation. Health Psychol. 32, 748–756 (2013).

    PubMed  PubMed Central  Article  Google Scholar 

  61. 61.

    Appleton, A. A., Loucks, E. B., Buka, S. L. & Kubzansky, L. D. Divergent associations of antecedent- and response-focused emotion regulation strategies with midlife cardiovascular disease risk. Ann. Behav. Med. 48, 246–255 (2014).

    PubMed  Article  PubMed Central  Google Scholar 

  62. 62.

    Veer, I. M. et al. Psycho-social factors associated with mental resilience in the corona lockdown. Transl. Psychiatry 11, 67 (2021).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  63. 63.

    Suri, G., Whittaker, K. & Gross, J. J. Launching reappraisal: it’s less common than you might think. Emotion 15, 73–77 (2015).

    PubMed  Article  PubMed Central  Google Scholar 

  64. 64.

    Milyavsky, M. et al. To reappraise or not to reappraise? Emotion regulation choice and cognitive energetics. Emotion 19, 964–981 (2019).

    PubMed  Article  PubMed Central  Google Scholar 

  65. 65.

    Cohen, N. & Ochsner, K. N. From surviving to thriving in the face of threats: the emerging science of emotion regulation training. Curr. Opin. Behav. Sci. 24, 143–155 (2018).

    PubMed  PubMed Central  Article  Google Scholar 

  66. 66.

    Denny, B. T. Getting better over time: a framework for examining the impact of emotion regulation training. Emotion 20, 110–114 (2020).

    PubMed  Article  PubMed Central  Google Scholar 

  67. 67.

    Woud, M. L., Holmes, E. A., Postma, P., Dalgleish, T. & Mackintosh, B. Ameliorating intrusive memories of distressing experiences using computerized reappraisal training. Emotion 12, 778–784 (2012).

    PubMed  Article  PubMed Central  Google Scholar 

  68. 68.

    Goldin, P. R. et al. Cognitive reappraisal self-efficacy mediates the effects of individual cognitive-behavioral therapy for social anxiety disorder. J. Consult. Clin. Psychol. 80, 1034–1040 (2012).

    PubMed  PubMed Central  Article  Google Scholar 

  69. 69.

    Kivity, Y. & Huppert, J. D. Does cognitive reappraisal reduce anxiety? A daily diary study of a micro-intervention with individuals with high social anxiety. J. Consult. Clin. Psychol. 84, 269–283 (2016).

    PubMed  Article  PubMed Central  Google Scholar 

  70. 70.

    Denny, B. T., Inhoff, M. C., Zerubavel, N., Davachi, L. & Ochsner, K. N. Getting over it: long-lasting effects of emotion regulation on amygdala response. Psychol. Sci. 26, 1377–1388 (2015).

    PubMed  Article  Google Scholar 

  71. 71.

    Tamir, M., Halperin, E., Porat, R., Bigman, Y. E. & Hasson, Y. When there’s a will, there’s a way: disentangling the effects of goals and means in emotion regulation. J. Pers. Soc. Psychol. 116, 795–816 (2019).

    PubMed  Article  Google Scholar 

  72. 72.

    Porat, R., Halperin, E. & Tamir, M. What we want is what we get: group-based emotional preferences and conflict resolution. J. Pers. Soc. Psychol. 110, 167–190 (2016).

    PubMed  Article  Google Scholar 

  73. 73.

    Hurtado-Parrado, C. et al. Emotion regulation and attitudes toward conflict in Colombia: effects of reappraisal training on negative emotions and support for conciliatory and aggressive statements. Front. Psychol. 10, 908 (2019).

    PubMed  PubMed Central  Article  Google Scholar 

  74. 74.

    Halperin, E., Porat, R., Tamir, M. & Gross, J. J. Can emotion regulation change political attitudes in intractable conflicts? From the laboratory to the field. Psychol. Sci. 24, 106–111 (2012).

    PubMed  Article  Google Scholar 

  75. 75.

    Moshontz, H. et al. The Psychological Science Accelerator: advancing psychology through a distributed collaborative network. Adv. Methods Pract. Psychol. Sci. 1, 501–515 (2018).

    PubMed  PubMed Central  Article  Google Scholar 

  76. 76.

    Uusberg, A., Taxer, J. L., Yih, J., Uusberg, H. & Gross, J. J. Reappraising reappraisal. Emot. Rev. 11, 267–282 (2019).

    Article  Google Scholar 

  77. 77.

    Frattaroli, J. Experimental disclosure and its moderators: a meta-analysis. Psychol. Bull. 132, 823–865 (2006).

    PubMed  Article  Google Scholar 

  78. 78.

    Pennebaker, J. W. & Smyth, J. M. Opening Up by Writing It Down: How Expressive Writing Improves Health and Eases Emotional Pain 3rd edn (Guilford Publications, 2016).

  79. 79.

    Kreibig, S. D. & Gross, J. J. Understanding mixed emotions: paradigms and measures. Curr. Opin. Behav. Sci. 15, 62–71 (2017).

    PubMed  PubMed Central  Article  Google Scholar 

  80. 80.

    Larsen, J. T. & McGraw, A. P. Further evidence for mixed emotions. J. Pers. Soc. Psychol. 100, 1095–1110 (2011).

    PubMed  Article  PubMed Central  Google Scholar 

  81. 81.

    Helgeson, V. S., Reynolds, K. A. & Tomich, P. L. A meta-analytic review of benefit finding and growth. J. Consult. Clin. Psychol. 74, 797–816 (2006).

    PubMed  Article  PubMed Central  Google Scholar 

  82. 82.

    Shiota, M. N. Silver linings and candles in the dark: differences among positive coping strategies in predicting subjective well-being. Emotion 6, 335–339 (2006).

    PubMed  Article  PubMed Central  Google Scholar 

  83. 83.

    Kok, G., Peters, G.-J. Y., Kessels, L. T. E., ten Hoor, G. A. & Ruiter, R. A. C. Ignoring theory and misinterpreting evidence: the false belief in fear appeals. Health Psychol. Rev. 12, 111–125 (2018).

    PubMed  Article  PubMed Central  Google Scholar 

  84. 84.

    Fredrickson, B. L. & Joiner, T. Reflections on positive emotions and upward spirals. Perspect. Psychol. Sci. 13, 194–199 (2018).

    PubMed  PubMed Central  Article  Google Scholar 

  85. 85.

    Pressman, S. D., Jenkins, B. N. & Moskowitz, J. T. Positive affect and health: what do we know and where next should we go? Annu. Rev. Psychol. 70, 627–650 (2019).

    PubMed  Article  Google Scholar 

  86. 86.

    Ogedegbe, G. O. et al. A randomized controlled trial of positive-affect intervention and medication adherence in hypertensive African Americans. Arch. Intern. Med. 172, 322–326 (2012).

    PubMed  PubMed Central  Article  Google Scholar 

  87. 87.

    Bigman, Y. E., Mauss, I. B., Gross, J. J. & Tamir, M. Yes I can: expected success promotes actual success in emotion regulation. Cogn. Emot. 30, 1380–1387 (2016).

    PubMed  Article  Google Scholar 

  88. 88.

    Hittner, E. F., Rim, K. L. & Haase, C. M. Socioeconomic status as a moderator of the link between reappraisal and anxiety: laboratory-based and longitudinal evidence. Emotion 19, 1478–1489 (2019).

    PubMed  Article  Google Scholar 

  89. 89.

    Troy, A. S., Ford, B. Q., McRae, K., Zarolia, P. & Mauss, I. Change the things you can: emotion regulation is more beneficial for people from lower than from higher socioeconomic status. Emotion 17, 141–154 (2017).

    PubMed  Article  Google Scholar 

  90. 90.

    Miyamoto, Y., Ma, X. & Petermann, A. G. Cultural differences in hedonic emotion regulation after a negative event. Emotion 14, 804–815 (2014).

    PubMed  Article  Google Scholar 

  91. 91.

    Hofstede, G., Hofstede, G. J. & Minkov, M. Cultures and Organizations: Software of the Mind 3rd edn, (McGraw-Hill, New York, 2010).

  92. 92.

    Nakagawa, S. & Schielzeth, H. A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods Ecol. Evol. 4, 133–142 (2013).

    Article  Google Scholar 

  93. 93.

    Eid, M. & Diener, E. Norms for experiencing emotions in different cultures: inter- and intranational differences. J. Pers. Soc. Psychol. 81, 869–885 (2001).

    CAS  PubMed  Article  Google Scholar 

  94. 94.

    Ferrer, R., Klein, W., Lerner, J. S., Reyna, V. F. & Keltner, D. in Behavioral Economics and Public Health (eds Roberto, C. & Kawachi, I.) (Harvard Univ. Press, 2016).

  95. 95.

    Smith, A. M. et al. Coping with health threats: the costs and benefits of managing emotions. Psychol. Sci. https://doi.org/10.1177/09567976211024260 (2021).

  96. 96.

    Pfattheicher, S., Nockur, L., Böhm, R., Sassenrath, C. & Petersen, M. B. The emotional path to action: empathy promotes physical distancing and wearing of face masks during the COVID-19 pandemic. Psychol. Sci. 31, 1363–1373 (2020).

    PubMed  Article  Google Scholar 

  97. 97.

    Jørgensen, F., Bor, A. & Petersen, M. B. Compliance without fear: individual-level protective behaviour during the first wave of the COVID-19 pandemic. Br. J. Health Psychol. 26, 679–696 (2021).

    PubMed  PubMed Central  Article  Google Scholar 

  98. 98.

    Jordan, J., Yoeli, E. & Rand, D. Don’t get it or don’t spread it? Comparing self-interested versus prosocial motivations for COVID-19 prevention behaviors. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/yuq7x (2020).

  99. 99.

    Willroth, E. C. et al. A test of resilience theory in the context of a global crisis. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/g3jr7 (2021).

  100. 100.

    Madison, A. A., Shrout, M. R., Renna, M. E. & Kiecolt-Glaser, J. K. Psychological and behavioral predictors of vaccine efficacy: considerations for COVID-19. Perspect. Psychol. Sci. 16, 191–203 (2021).

    PubMed  PubMed Central  Article  Google Scholar 

  101. 101.

    Ferrer, R. A. et al. The role of incidental affective states in appetitive risk behavior: a meta-analysis. Health Psychol. 39, 1109–1124 (2020).

    PubMed  Article  Google Scholar 

  102. 102.

    Kemp, E., Cowart, K. & Bui, M. Promoting consumer well-being: examining emotion regulation strategies in social advertising messages. J. Bus. Res. 112, 200–209 (2020).

    Article  Google Scholar 

  103. 103.

    Porat, R., Erel, L., Pnueli, V. & Halperin, E. Developing ReApp: an emotion regulation mobile intervention for intergroup conflict. Cogn. Emot. 34, 1326–1342 (2020).

    PubMed  Article  Google Scholar 

  104. 104.

    Keltner, D. & Lerner, J. S. in Handbook of Social Psychology (eds Fiske, S. T. et al.) (John Wiley & Sons, 2010).

  105. 105.

    Henrich, J., Heine, S. J. & Norenzayan, A. The weirdest people in the world. Behav. Brain Sci. 33, 61–83 (2010).

    PubMed  Article  Google Scholar 

  106. 106.

    Glasgow, R. E., Vogt, T. M. & Boles, S. M. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am. J. Public Health 89, 1322–1327 (1999).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  107. 107.

    Premachandra, B. & Lewis, N. A. J. Do we report the information that is necessary to give psychology away? A scoping review of the psychological intervention literature 2000-2018. Perspect. Psychol. Sci. https://doi.org/10.1177/1745691620974774 (2021).

  108. 108.

    Goldin, P. R. & Gross, J. J. Effects of mindfulness-based stress reduction (MBSR) on emotion regulation in social anxiety disorder. Emotion 10, 83–91 (2010).

    PubMed  PubMed Central  Article  Google Scholar 

  109. 109.

    Fredrickson, B. L. in Advances in Experimental Social Psychology Volume 47 (eds. Devine, P. & Plant, A.) 1–53 (Academic Press, 2013).

  110. 110.

    Hughes, M. E., Waite, L. J., Hawkley, L. C. & Cacioppo, J. T. A short scale for measuring loneliness in large surveys: results from two population-based studies. Res. Aging 26, 655–672 (2016).

    Article  Google Scholar 

  111. 111.

    Lytle, M. C., Blosnich, J. R., Luca, S. M. D. & Brownson, C. Association of religiosity with sexual minority suicide ideation and attempt. Am. J. Prev. Med. 54, 644–651 (2018).

    PubMed  Article  Google Scholar 

  112. 112.

    Ajzen, I. & Madden, T. J. Prediction of goal-directed behavior: attitudes, intentions, and perceived behavioral control. J. Exp. Soc. Psychol. 22, 453–474 (1986).

    Article  Google Scholar 

  113. 113.

    Webb, T. L. & Sheeran, P. Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychol. Bull. 132, 249–268 (2006).

    PubMed  Article  Google Scholar 

  114. 114.

    Fishbein, M. & Ajzen, I. Predicting and Changing Behavior: The Reasoned Action Approach (Psychology Press, 2011).

  115. 115.

    Thompson, R. R., Jones, N. M., Holman, E. A. & Silver, R. C. Media exposure to mass violence events can fuel a cycle of distress. Sci. Adv. 5, eaav3502 (2019).

    PubMed  PubMed Central  Article  Google Scholar 

  116. 116.

    Dienes, Z. Using Bayes to get the most out of non-significant results. Front. Psychol. 5, 781 (2014).

    PubMed  PubMed Central  Article  Google Scholar 

  117. 117.

    Morey, R. D. & Rouder, J. N. Baysefactor: computation of Bayes Factors for common designs. R package version 0.9.12-4.2 https://cran.r-project.org/package=BayesFactor (2018).

  118. 118.

    Dienes, Z. How do I know what my theory predicts? Adv. Methods Pract. Psychol. Sci. 4, 364–377 (2019).

    Article  Google Scholar 

  119. 119.

    Bürkner, P.-C. brms: an R package for Bayesian multilevel models using Stan. J. Stat. Softw. 80, 1–28 (2017).

    Article  Google Scholar 

  120. 120.

    Makowski, D., Ben-Shachar, M. & Lüdecke, D. bayestestR: describing effects and their uncertainty, existence and significance within the Bayesian framework. J. Open Source Softw. 4, 1541 (2019).

    Article  Google Scholar 

  121. 121.

    Gronau, Q. F., Singmann, H. & Wagenmakers, E.-J. bridgesampling: an R package for estimating normalizing constants. J. Stat. Softw. 92, 1–29 (2020).

    Article  Google Scholar 

  122. 122.

    Vishkin, A., Hasson, Y., Millgram, Y. & Tamir, M. One size does not fit all: tailoring cognitive reappraisal to different emotions. Pers. Soc. Psychol. Bull. 46, 469–484 (2019).

    PubMed  Article  PubMed Central  Google Scholar 

  123. 123.

    Anvari, F. & Lakens, D. Using anchor-based methods to determine the smallest effect size of interest. J. Exp. Soc. Psychol. 96, 104159 (2021).

    Article  Google Scholar 

  124. 124.

    Kvarven, A., Strømland, E. & Johannesson, M. Comparing meta-analyses and preregistered multiple-laboratory replication projects. Nat. Hum. Behav. 4, 423–434 (2020).

    PubMed  Article  Google Scholar 

  125. 125.

    Shiota, M. N. & Levenson, R. W. Turn down the volume or change the channel? Emotional effects of detached versus positive reappraisal. J. Pers. Soc. Psychol. 103, 416–429 (2012).

    PubMed  PubMed Central  Article  Google Scholar 

  126. 126.

    Fetzer, T. et al. Global behaviors and perceptions in the COVID-19 pandemic. Preprint at SSRN https://ssrn.com/abstract=3594262 (2020).

  127. 127.

    Klein, R. A. et al. Many Labs 2: investigating variation in replicability across samples and settings. Adv. Methods Pract. Psychol. Sci. 1, 443–490 (2018).

    Article  Google Scholar 

  128. 128.

    Ellsworth, P. C. & Scherer, K. R. in Handbook of affective sciences 572–595 (Oxford Univ. Press, 2003).

  129. 129.

    Green, P. & MacLeod, C. J. SIMR: an R package for power analysis of generalized linear mixed models by simulation. Methods Ecol. Evol. 7, 493–498 (2016).

    Article  Google Scholar 

  130. 130.

    Pordes, R. et al. The open science grid. J. Phys. Conf. Ser. 78, 012057 (2007).

    Article  Google Scholar 

  131. 131.

    Sfligoi, I. et al. The pilot way to grid resources using glideinWMS. In 2009 WRI World Congress on Computer Science and Information Engineering (eds Burgin, M. et al.) 428–432 (IEEE, 2009).

  132. 132.

    Jeffreys, H. Theory of Probability (Oxford Univ. Press, 1961).

  133. 133.

    Brauer, M. & Curtin, J. J. Linear mixed-effects models and the analysis of nonindependent data: a unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. Psychol. Methods 23, 389–411 (2018).

    PubMed  Article  PubMed Central  Google Scholar 

Download references

Acknowledgements

This project was supported by funds from: the Amazon Web Services (AWS) Imagine Grant (to E.M.B.); the Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research (JSPS KAKENHI; 16h03079, 17h00875, 18k12015, and 20h04581 to Y.Y.); the research programme Dipartimenti di Eccellenza from the Ministry of Education University and Research (MIUR to N. Cellini and G.M. and the Department of General Psychology of the University of Padua); statutory funds of the University of Wroclaw (to A. Sorokowska); the Charles University Research Programme PROGRES (Q18 to M. Vranka); the Knut and Alice Wallenberg Foundation (2016:0229 to J.K.O.); the Rubicon Grant (019.183sg.007 to K.v.S.) from the Netherlands Organisation for Scientific Research; the Australian Research Council (dp180102384 to R.M.R.); the US National Institutes of Health (NIMH111640 to M.N.-D.), the Huo Family Foundation to N.J.; the NSF Directorate for Social, Behavioral and Economic Sciences, Division of Social and Economic Sciences (1559511 to J.S.L.); the US National Institutes of Health (RO1-CA-224545 to J.S.L.); Eesti Teadusagentuur–Estonian Research Council (PSG525 to A. Uusberg); the J. William Fulbright Program (to F. Azevedo); the HSE Basic Research Program (to D. Dubrov); Dominican University (a Faculty Development Grant to A. Krafnick); and the French National Research Agency Investissements d’avenir supporting PSF (ANR-15-IDEX-02 to H.I.); the Slovak Research and Development Agency (project no. APVV-20-0319 to M. Adamkovič); the programme FUTURE LEADER of Lorraine Université d’Excellence within the French National Research Agency Investissements d’avenir (ANR-15-IDEX-04-LUE to S.M.). Computation for this research was assisted by: the Harvard Business School compute cluster (HBSGrid); and the Open Science Grid. The Open Science Grid is supported by the National Science Foundation award 1148698 and the US Department of Energy’s Office of Science, as well as by the compute resources and assistance of the UW-Madison Center For High Throughput Computing (CHTC) in the Department of Computer Sciences. The CHTC is supported by UW-Madison, the Advanced Computing Initiative, the Wisconsin Alumni Research Foundation, the Wisconsin Institutes for Discovery, and the National Science Foundation, and is an active member of the Open Science Grid, which is supported by the National Science Foundation and the U.S. Department of Energy’s Office of Science. We thank data science specialist S. Worthington and the research computing environment at the Institute for Quantitative Social Science, Harvard University and V. Ivanchuk for research assistantship. Our semi-representative panels were made possible by an in-kind purchase from the Leibniz Institute for Psychology (protocol https://doi.org/10.23668/psycharchives.3012); a special grant from the Association for Psychological Science and a fee waiver from Prolific. This work was supported by a grant from the American Psychological Society (granted to the PSA). Further financial support was provided by the PSA and a special crowdfunding campaign initiated by the PSA. We thank Amazon Web Services for help with server needs, the Leibniz Institute for Psychology (ZPID) for help with data collection via the organization and implementation of semi-representative panels, Prolific Inc. for offering discounted recruitment, and Harvard University’s Institute for Quantitative Social Sciences for statistical consulting. Finally, this research was supported by resources from the Open Science Grid, which is supported by National Science Foundation award 1148698, and the U.S. Department of Energy’s Office of Science. Beyond those roles already acknowledged, the funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Affiliations

Authors