Systematic review of mental health symptom changes by sex or gender in early-COVID-19 compared to pre-pandemic

Women and gender-diverse individuals have faced disproportionate socioeconomic burden during COVID-19. There have been reports of greater negative mental health changes compared to men based on cross-sectional research that has not accounted for pre-COVID-19 differences. We compared mental health changes from pre-COVID-19 to during COVID-19 by sex or gender. MEDLINE (Ovid), PsycINFO (Ovid), CINAHL (EBSCO), EMBASE (Ovid), Web of Science Core Collection: Citation Indexes, China National Knowledge Infrastructure, Wanfang, medRxiv (preprints), and Open Science Framework Preprints (preprint server aggregator) were searched to August 30, 2021. Eligible studies included mental health symptom change data by sex or gender. 12 studies (10 unique cohorts) were included, all of which reported dichotomized sex or gender data. 9 cohorts reported results from March to June 2020, and 2 of these also reported on September or November to December 2020. One cohort included data pre-November 2020 data but did not provide dates. Continuous symptom change differences were not statistically significant for depression (standardized mean difference [SMD] = 0.12, 95% CI -0.09–0.33; 4 studies, 4,475 participants; I2 = 69.0%) and stress (SMD = − 0.10, 95% CI -0.21–0.01; 4 studies, 1,533 participants; I2 = 0.0%), but anxiety (SMD = 0.15, 95% CI 0.07–0.22; 4 studies, 4,344 participants; I2 = 3.0%) and general mental health (SMD = 0.15, 95% CI 0.12–0.18; 3 studies, 15,692 participants; I2 = 0.0%) worsened more among females/women than males/men. There were no significant differences in changes in proportions above cut-offs: anxiety (difference = − 0.05, 95% CI − 0.20–0.11; 1 study, 217 participants), depression (difference = 0.12, 95% CI -0.03–0.28; 1 study, 217 participants), general mental health (difference = − 0.03, 95% CI − 0.09–0.04; 3 studies, 18,985 participants; I2 = 94.0%), stress (difference = 0.04, 95% CI − 0.10–0.17; 1 study, 217 participants). Mental health outcomes did not differ or were worse by small amounts among women than men during early COVID-19.

Statistical analyses. For continuous outcomes, separately for each sex or gender group, we extracted a standardized mean difference (SMD) effect size with 95% confidence intervals (CIs) for change from pre-COVID-19 to COVID-19. If not provided, we extracted pre-COVID-19 and COVID-19 means and standard deviations (SDs) for each group, calculated raw change scores (SD), and calculated SMD for change using Hedges' g for each group 43 , as described by Borenstein et al. 44 . Raw change scores were presented in scale units and direction, whereas SMD change scores were presented as positive when mental health worsened from pre-COVID-19 to COVID-19 and negative when it improved. We then calculated a Hedges' g difference in change between sex or gender groups with 95% CI. Positive numbers represented greater negative change in females or women compared to males or men.
For studies that reported proportions of participants above a scale cut-off, for pre-COVID-19 and COVID-19 proportions, if not provided, we calculated a 95% CI using Agresti and Coull's approximate method for binomial proportions 45 . We then extracted or calculated the proportion change in participants above the cut-off, along with 95% CI, for each sex or gender group. Proportion changes were presented as positive when mental health worsened from pre-COVID-19 to COVID-19 and negative when it improved. If 95% CIs were not reported, we generated them using Newcombe's method for differences between binomial proportions based on paired data 46 . To do this, which requires the number of cases at both assessments, which is not typically available, we assumed that 50% of pre-COVID-19 cases continued to be cases during COVID-19 and confirmed that results did not differ substantively if we used values from 30 to 70% (all 95% CI end points within 0.02; see Supplementary Table S1). Finally, we calculated a difference of the proportion change between sex or gender groups with 95% CI 47 . Positive numbers reflected greater negative change in females or women compared to males or men.
Meta-analyses were done to synthesize differences between sex or gender groups in SMD change for continuous outcomes and in proportion change for dichotomous outcomes via restricted maximum-likelihood random-effects meta-analysis. Heterogeneity was assessed with the I 2 statistic. Meta-analysis was performed in R (R version 3.6.3, RStudio Version 1.2.5042), using the metacont and metagen functions in the meta package 48 . Forest plots were generated using the forest function in meta. Positive values indicated more relatively worse changes in mental health for females or women compared to males or men.

Results
Search results and selection of eligible studies. As of August 30, 2021, there were 64,496 unique references identified and screened for potential eligibility, of which 63,534 were excluded after title and abstract review and 741 after full-text review. Of 221 remaining articles, 209 were excluded, leaving 12 included studies that reported data from 10 cohorts. Supplementary Fig. S1 shows the flow of article review and reasons for exclusion.
Adequacy of study methods and reporting. Two studies (1 cohort) 51,52 were rated as "yes" for adequacy for all items. Other studies were rated "no" for 1-3 items (plus 0-3 unclear ratings) 50 19,067 participants; I 2 = 67.0%). Anxiety, measured continuously, worsened significantly more for females or women than for males or men during COVID-19 ( Fig. 1a; SMD change difference = 0.15, 95% CI 0.07 to 0.22; N = 4 studies 54,56,58,60 , 4,344 participants; I 2 = 3.0%). General mental health, measured continuously, also worsened more for females or women than for males or men in early COVID-19 ( Fig. 1c; SMD difference in change = 0.15, 95% CI 0.12 to 0.18; N = 3 studies 49,53,59 , 15,692 participants; I 2 = 0.0%). This was predominantly based on a large population-based study from the United Kingdom 49 . That study did not report results from fall 2020 for continuous outcomes, but as shown in Table 2 and Figs. 2c and e, the difference in change between females or women and males or men decreased between early and late 2020 for dichotomous outcomes in the same cohort 50 . The magnitude of both statistically significant differences was small (see Fig. 3).

Discussion
The COVID-19 pandemic has affected women and gender minorities disproportionately [8][9][10][11][12][13][14][15][16][17] . There has been an assumption, seemingly confirmed by cross-sectional data collected during COVID-19, that overall mental health has worsened and that there have been even greater negative changes in mental health among women than for men [21][22][23][24][25][26][27][28][29][30][31] . We reviewed evidence from 12 studies (10 cohorts) that reported mental health changes from pre-COVID-19 to COVID-19 separately by sex or gender. We compared females or women with males or men; no studies compared gender minorities with any other group. Data were largely from March to June 2020, early in the pandemic. Syntheses of continuously measured anxiety symptoms (SMD = 0.15, 95% CI 0.07 to 0.22) and general mental health (SMD = 0.15, 95% CI 0.12 to 0.18) found that mental health worsened more for females or women than males or men, but the magnitude was small and far below thresholds that are typically considered clinically important (e.g., SMD = 0.50) 61 . None of the other 6 mental health outcomes that we examined (continuous depression symptoms and stress; dichotomous anxiety symptoms, depression symptoms, general mental health, and stress) differed by sex or gender.
Sex and gender differences in mental health disorder prevalence, symptoms, and risk factors are well-established [62][63][64][65] . Likely risk factors include gender inequities and discrimination, economic disadvantage and poverty, higher rates of interpersonal stressors, and violence 66,67 , and many of these risk factors have been exacerbated for women during COVID-19 [8][9][10][11][12][13][14][15] . We did not identify any differences in mental health by sex or gender, however, that appeared to be substantive; all were 0.15 SMD or smaller, which is considered to be a small difference based on commonly used metrics (e.g., < 0.20 SMD) 68 and below thresholds for clinical meaningfulness 61 .
Based on our findings, it is possible that despite the challenges women have faced, many have been resilient and that the mental health disaster that has been predicted by many has not occurred 69 . Overall, across populations, expected negative changes in mental health during the pandemic compared to pre-pandemic levels have not been as dramatic as might have been expected 3,[70][71][72] . To the best of our knowledge, there have been two systematic reviews that have compared symptoms prior to COVID-19 and after the start of the pandemic. The reviews used somewhat different methods, including study inclusion and exclusion criteria, but findings were consistent. Both reported that symptom scores on measures of general mental health, depression, and anxiety were stable or had worsened by small amounts during the pandemic 4,5 . This is consistent with the only study, to the best of our knowledge, that has evaluated prevalence of mental health disorders using validated diagnostic interviews rather than symptom changes. That study, which probabilistically sampled Norwegian adults in January to early March 2020 (pre-pandemic), mid-March to May 2020, and June to July 2020, reported that the prevalence of current mental disorders, assessed using the Composite International Diagnostic Interview (version 5.0), was stable across time periods 73 . Similarly, a study on suicide in 21 countries during early COVID-19 found that observed numbers of deaths from suicide was stable or decreased from pre-pandemic to the early pandemic months in all included jurisdictions based on an interrupted time-series analysis 72 .
Our findings, as well as those from other studies that have reported that mental health implications early in the pandemic may not have been as substantial as expected depart from what has been reported in some research and by the media. Three factors may feed this discrepancy. One is the publication of many cross-sectional studies that report proportions above cut-offs on self-report measures, which are not designed for that purpose [74][75][76][77][78] , and assume that what are perceived as high numbers, generally, or sex differences, comparatively, must not have been present pre-COVID-19 5 . A second is the use of surveys that ask questions about well-being with COVID-19 explicitly assigned as a cause; illustrating the pitfalls of this, a study of over 2,000 young Swiss adult men found significant angst when questions were asked in this way, but no changes in validated measures of depression www.nature.com/scientificreports/ symptoms and stress from pre-COVID 79 . A third reason relates to news media reports that emphasize dramatic events and anecdotes without evidence that demonstrates changes 69 . Strengths of our study include the use of rigorous systematic review methods. We searched 9 databases, including Chinese-language databases, without language restrictions and included studies that enabled the direct comparison of mental health changes by sex or gender. Our findings emphasize that we should not assume that mental health effects of COVID-19 have been much greater for females or women than for males or men during the pandemic. Indeed, across the 21 analyses we conducted, differences were consistently null or very small and no individual studies stood out as deviating from this overall finding. Nonetheless, one should be cautious about generalizing our findings to all populations and subgroups. First, included studies were conducted in 8 countries, and it is possible that there could have been differences in other countries, given that the pandemic has manifested itself differently across countries and that countries have managed the pandemic differently (e.g., length and severity of restrictions). Second, all but one of the included studies was on adults, and the findings may not be generalizable to children or adolescents. Third, there were not enough studies to attempt subgroup analyses by sociodemographic or other factors, such as professional groups, for example. Cross-sectional studies have reported that there could be differences in mental health by sex or gender that are related to sociodemographic variables (e.g., age, race or ethnicity) and professional roles (e.g., health care workers) 80,81 . Cross-sectional analyses, however, do not allow us to determine if any identified associations or differences may have been present prior to the pandemic, and if so, to what degree. Fourth, we were not able to evaluate the influence of potential risk and protective factors that may differ between sex or gender and if these might potentially explain some of the results observed. The information needed to do this was not provided in included studies. Fifth, we did not identify any studies that compared results from gender-diverse individuals to other gender groups. This highlights an important evidence gap in the literature, and indicates the need for more research on this population, especially given that several studies suggest that the mental health of this population group may have been affected negatively since pre-COVID-19 16,82,83 . There are other limitations to consider in addition to generalizability. First, this review only included 12 studies from 10 cohorts, and many had limitations related to study sampling frames and recruitment methods, follow-up rates, and management of missing data. Second, our review only included studies with mental health outcomes early in the pandemic. This did not permit us to examine long-term trends in mental health as the pandemic progressed. It is possible that sex or gender differences absent in the early pandemic may have developed. For example, according to a United States Centers for Disease Control report on suicide-related weekly emergency department visits, the numbers for teenage females (aged 12-17 years) increased minimally in 2020, but were over 51% higher in 2021 compared to the same period in 2019, versus an increase of 4% among teenage males 84 . Analyses of overall mental health that have been reported and the results in our study are based on data from early in the pandemic, and it is not clear to what degree these findings would apply to later stages of the pandemic. Third, heterogeneity was high for some meta-analyses; it was low, however, for others, and results across 8 analyses did not differ substantively. Fourth, in calculating 95% CIs for within-group changes in proportions with the information provided in publications (pre-COVID-19 and COVID-19 group proportions), we assumed that 50% of pre-COVID-19 cases continued to be cases during COVID-19. However, the maximum difference in any end point of a 95% CIs across analyses was 0.02 when we varied our assumption from 30 to 70%.
In sum, we identified small sex-or gender-based differences for anxiety symptoms and general mental health, continuously measured, but other outcomes (continuous depression symptoms and stress; dichotomous anxiety symptoms, depression symptoms, general mental health, and stress) did not differ by sex or gender. This finding diverges from what has been reported from cross-sectional studies. These are aggregate results, though, and  www.nature.com/scientificreports/ many individuals have certainly experienced negative mental health changes related to increased socioeconomic burden. It seems plausible, given the divergent ways that the pandemic has affected different people that many people are experiencing improved mental health, whereas large numbers of others may be experiencing worsened mental health, including new onset mental disorders among people without previous morbidity. Thus, mental health changes should continue to be monitored longitudinally in COVID-19, taking into consideration sex and gender, particularly in younger populations. Our research underlines that few studies report results by sex