Abstract
We document mental and social distress of children, adolescents and adults, using data on 3 million calls to German helplines between January 2019 and May 2022. High-frequency data from crisis helpline logs offer rich information on the evolution of “revealed distress” among the most vulnerable, unaffected by researchers’ study design and framing. Distress of adults, measured by the volume of calls, rose significantly after both the outbreak of the pandemic and the Russian invasion of Ukraine. In contrast, the overall revealed distress of children and adolescents did not increase during those crises. The nature of young people’s concerns, however, changed more strongly than for adults after the COVID-19 outbreak. Consistent with the effects of social distancing, call topics of young people shifted from problems with school and peers to problems with family and mental health. We find the share of severe mental health problems among young people to have increased with a delay, in the second and third year of the pandemic.
Similar content being viewed by others
Introduction
Information on psychological distress in the population is a prerequisite for effective crisis management. It contributes to a comprehensive understanding of the costs and benefits of policy measures and helps to identify vulnerable groups in need of targeted support1,2,3,4,5,6.
The COVID-19 pandemic is a case in point: while the general state of physical health and economic activity could be measured fairly accurately and at high frequency7,8, population-level information on mental suffering was much sparser. Evidence suggests that mental health problems in the overall population sharply increased around the pandemic outbreak in 2020 and gradually decreased again thereafter2,9, with economic aid alleviating some pressure10 and the availability of vaccines further reducing distress in the population11,12. Less is still known about longer-lasting mental health trajectories, as pandemic fatigue and economic scarring might have set in13,14,15,16,17.
A large-scale war such as Russia’s invasion of Ukraine represents another event that could affect mental health well beyond the population immediately affected by the crisis18. With the prospect of recurrent natural, economic, and social crises linked to climate change, risks to population mental health look set to remain acute19,20,21,22,23,24,25,26.
In this paper, we focus on children and adolescents. It has been argued that lockdowns and associated pandemic mitigation measures weigh particularly hard on the young, for two main reasons. First, the health risks from which those measures are designed to offer protection were much smaller for young people than for the elderly. Second, constraints on social contacts and educational opportunities could be expected to weigh disproportionately on the young.
Survey-based evidence indeed shows children and adolescent mental health to have deteriorated after the outbreak of the pandemic27, with girls suffering disproportionately28, and school closures being associated with marked negative effects on mental health29,30,31,32. On the other hand, there is evidence of the pandemic reducing bullying, at school and online33. Overall, the evidence is mixed: while children and adolescents reported increased anxiety and depression in surveys, administrative data on suicides, substance abuse and clinical mental health diagnoses rather suggest a decrease in distress during the pandemic28,34,35,36. Some experts have speculated that this apparent paradox may be explained by most administrative data referring to the early months of the pandemic, during which many services were pared back, and that mental stressors, being cumulative over time, will only manifest themselves in “hard” data with a lag34. Our long span of high-frequency, large-sample observational data can shed light on this open question.
Most evidence on population mental health is drawn from surveys. In the COVID-19 context, longitudinal studies have been conducted to monitor self-reported mental health and well-being37,38,39,40,41,42,43. Surveys are a rich and valuable source of information, but they have limitations. First, survey evidence is typically available only with a lag and at a relatively low frequency, as surveys are expensive to implement at regular intervals with an informative sample size4. Moreover, survey responses are inevitably sensitive to framing and reporting biases sometimes referred to as “demand effects”44,45,46,47,48, sampling issues49, and conceptualization challenges50,51.
Researchers have therefore tapped complementary data sources to assess population distress at a higher frequency and in user-generated data that are unaffected by study design. The main approaches include analyses of social media sentiment52,53,54,55,56, online search behavior57,58,59,60,61, and, like this paper, crisis helplines10,62,63,64,65,66,67,68.
Helplines are proven tools for mental health protection and suicide prevention69,70. They were particularly important when lockdown measures restricted social contact and access to regular psychological and social support services71,72. Helpline counselors systematically collect anonymous data on caller characteristics and conversation topics. The resulting data provide detailed information on psychological distress in vulnerable populations. Callers to helplines do so on their own initiative, thereby incurring a mental and time cost. In analogy to the concept of revealed preferences observed through consumer choices in the field, we therefore consider helpline calls to be a measure of revealed distress. On the spectrum of mental health outcomes, helpline data arguably capture a stronger form of distress than survey-based measures, but in contrast to clinical data, they also offer a window on sub-clinical distress.
In this study, we exploit data from the two nationwide German helplines for young people: Nummer gegen Kummer, a service aimed specifically at children, adolescents, and their parents, and TelefonSeelsorge, Germany’s biggest helpline, aimed at all demographics. We have data on over 3 million phone calls and online conversations between January 2019 and February 2023, allowing us to track the evolution of distress as revealed in calls made by young people before the COVID-19 pandemic, over the course of the pandemic and after the start of the Russia-Ukraine war.
Results
Call volumes over time
The solid black line in Fig. 1a tracks the weekly volume of total helpline calls by children and adolescents from January 2020 to May 2022. This data series includes all calls to the specific children and youth helpline (75% of the overall total of young callers, see Supplementary Table S1 for details) as well as all calls to the general helpline by callers below 20 years of age (25%), and it covers conversations over the telephone (76%) as well as via text-based chat (14%) and email (11%). The dashed red line illustrates the evolution of the pandemic in Germany as measured by COVID-19-related deaths, the dashed blue line shows the stringency of containment measures8,73, and the gray areas illustrate periods of general (dark shading) or partial (light shading) school closures. More detailed data series on call and chat volumes, separately by helpline, are shown in Supplementary Figs. S1 and S2.
Figure 1a reveals no strong correlation between (a) the up-and-down of the pandemic and associated containment measures and (b) the consolidated volume of helpline calls by young people. In fact, the volume of calls, while quite volatile week-to-week, is stable across the entire period. The pandemic thus appears to have had no sustained impact on the volume of helpline calls by young people. As can be seen in Supplementary Fig. S1, this trend is not shared equally across the two helplines, with calls by young people to the general-purpose helpline increasing, and calls to the young people’s helpline decreasing slightly. Given that the young people’s helpline is numerically more important, accounting for 78.7% of all calls by Under-20 s (Supplementary Table S1), we can rule out that the volume of helpline calls by children and young people has increased more since the outbreak of the pandemic period than calls by adults.
The strongest apparent regularity is a gradual reduction in the volume of calls after the reopening of schools in 2020 and 2021. School closures, on the other hand, were associated with small increases in call volumes. The observed drop in call volumes after the lifting of school closures could, however, also be a summer effect. This interpretation is confirmed by Fig. 1c, where we overlay the evolutions of our four sample years: a decrease in call volumes by late spring and summer is visible also for the pre-pandemic year 2019.
In Fig. 1b, we show changes in monthly call volumes relative to February 2020, separately by age category. This figure confirms that call volumes by children and adolescents did not systematically exceed the pre-pandemic level, and in some months were even significantly lower. The only indication of a pandemic effect is a statistically significant increase in call volumes by adolescents in the first three months of the pandemic as well as during the second COVID-19 wave of winter 2020–2021.
However, adolescent call volumes reverted to pre-pandemic levels both after the initial three months of the pandemic and after the second wave. Helpline call volumes by children were below pre-pandemic levels in 24 of the 27 pandemic months covered by our data. These patterns contrast strongly with the evolution of call volumes by adult callers: those volumes increased sharply after the outbreak of the pandemic and never reverted to pre-pandemic levels thereafter. The increase in the volume of calls made by adults is also evident from Fig. 1d, which contrasts markedly with Fig. 1c for children and adolescents.
One potential issue with analyses of call volumes is call-answering capacity: unchanged volumes could be consistent with increased demand that could not be met as the supply of counselors did not keep up. Indeed, helplines had to adjust to remote work after the outbreak of the pandemic10,62. However, capacity constraints are not a plausible explanation for the sustained stagnation or even reduction in calls that were placed and answered, as shown in Fig. 1. Most helplines were given additional resources to cope with expected increases in demand after the outbreak of the pandemic. Importantly, the increase in recorded adult calls clearly shows that the capacity existed to serve at least part of the increased demand for helpline counseling.
Overall, the evolution of call volumes before and during the pandemic suggests that, while revealed distress noticeably increased among adults, no comparable increase occurred for children and adolescents. This observation runs counter to the often-held view of young people as the main victims of the pandemic and associated mitigation measures. This is broadly consistent with early evidence based on clinical data29,34,74,75,76.
Conversation topics
Helpline call data provide population-level information not only on the amount but also on the nature of distress. After every call, counselors electronically tick topics from organization-wide pre-defined lists. This generalized procedure results in consistent data on the type of concerns raised by helpline callers.
As we combine data from two large helplines, each with its own topic list, we need to concord them into a harmonized nomenclature. We propose a categorization of 20 topics, each of which contains at least one item from both of the two original topic lists (for details, see Supplementary Table S6). To filter out seasonal fluctuations, we compute mean deviations from the corresponding calendar week of 2019 for every conversation topic (see “Methods” Section, Eq. 2).
Figure 2 summarizes the evolution of seasonally corrected harmonized topic prevalence from January 2020 through May 2022, separately for children, adolescents and adults. Changes in helpline-specific, and thus even more detailed, topics are shown in Supplementary Figs. S3 and S4. In Fig. 2, we list topics in decreasing order of their average sample prevalence in 2019 summed across children and adolescents, with numbers in brackets stating the average prevalence for the given age group. Prevalence percentages sum to more than 100% because calls can touch on more than one topic.
Prior to the pandemic, the topics most frequently raised by children were problems with school (22.2%), with family (21.5%), and with peers and friends (19.4%). Adolescents most often sought help for severe mental health problems (depression, self-harm, etc.; 24.2%), family relations (21.4%), and issues with sexuality (20.7%).
Our focus is on changes over time, against the background of the pandemic and the Russian invasion. We note the following, particularly striking observations.
The conversation topics raised by children and adolescents after the outbreak of the pandemic changed more strongly than those raised by adult callers. This is visually evident from a comparison of the three panels of Fig. 2. The mean absolute deviation of weekly percentage-point changes in topic prevalence from January 2020 was 1.79 for children (95% CI 1.71, 1.87, n = 2520 coefficient estimates), 1.28 for adolescents (95% CI 1.23, 1.33), and 0.73 for adult callers (95% CI 0.69, 0.76).
For children and adolescents, the pandemic was associated with a significant decrease in the prevalence of calls due to problems with school (most pronounced for children, with − 13.7 p.p. in the week of 15 April 2020, 95% CI − 14.3, − 13.1, detailed numerical estimation results are available in the Figure Source Data), sexuality (most pronounced for adolescents, − 6.8 p.p. in the week of 9 September 2020, 95% CI − 11.1, − 2.5) and peer relationships (− 3.6 p.p. in the week of 18 March 2020, 95% CI − 4.6, − 2.6). These evolutions are consistent with same-age social contacts being reduced during the pandemic, which also limits the potential for relationship problems.
Conversely, the weight of problems within the family markedly increased after the outbreak of the pandemic, both for children (+ 12.0 p.p. in the week of 25 March 2020, 95% CI 9.6, 14.5 and + 14.2 p.p. in the week of 5 Feb. 2021, 95% CI 13.4, 15.0) and for adolescents (+ 7.1 p.p. in the week of 24 June 2020, 95% CI 5.6, 8.7). This and the previous observations together are consistent with social interactions shifting toward the family, away from peers.
Another evident change during the pandemic period is an increase in mental-health-related calls, especially by children but also, with somewhat lesser intensity, by adolescents. In the initial weeks of the pandemic, calls by children related to “moderate” mental-health problems (e.g. stress, boredom) rose sharply (+ 13.1 p.p. in the week of 1 April 2020, 95% CI 10.2, 15.9), but were later superseded by an increase in calls due to mental health problems that we class as severe (e.g. depression, self-harm, + 10.9 p.p. in the last week of 2021, 95% CI 10.1, 11.8). This is indicative of a certain segment of children callers suffering from increasingly serious mental health issues as the pandemic lasted beyond the initial wave. For both children and adolescents, severe mental health issues peaked in the winter of 2021–2022, consistent with pandemic fatigue/exhaustion.
The evolution of topics raised by adult callers is very different from that of younger callers. Calls by adults mainly increased due to fears and anxiety (including fear of infection (+ 6.7 p.p. in the week of 18 March 2020, 95% CI 6.6, 6.9) and due to loneliness (+ 4.4 p.p. in the week of 6 April 2020, 95% CI 4.20, 4.6). This observation is consistent with patterns found internationally10. Fears by adults spiked around the outbreak of the pandemic in spring 2020, and again around the Russian invasion of Ukraine in spring 2022.
Young callers also differ from adult callers in terms of what might be considered the most serious topics, violence and suicidality. Among adult callers, the prevalence of those topics has fallen below its pre-pandemic level throughout our observation period. In contrast, we observe increases in the prevalence of calls linked to violence for both children (+ 5.5 p.p. in the week of 16 September 2020, 95% CI 4.0, 7.1) and adolescents (+ 5.0 p.p. in the first week of 2021, 95% CI 2.29, 7.68). For children, we moreover detect an increase in the share of calls related to suicidality, even though this is from a low base (4% of calls). The share of suicide-related calls by adolescents, however, remained roughly unchanged relative to before the pandemic.
When we split the data by caller gender, we find that the share of calls made by girls increased during the pandemic (Supplementary Table S5 and Fig. S5). Among children, the share of girls increased from 50% in 2019 to 56% in 2021 and remained high (54%) in 2022. The increase was even stronger among adolescents, where the share of calls by females rose from 52 to 58% between 2019 and 2021, and remained high (57%) in 2022. In contrast, the gender composition of callers remained stable among adults (67% in 2019, 68% in 2021, and 67% in 2022).
The increase in severe mental health issues is mostly driven by girls (up to + 15.2 p.p. for girls younger than 15 during the last week of 2021 and + 7.8 p.p. for adolescent girls of age 15–19 in the week of 23 April 2022, 95% CI 7.7, 22.7 and 1.4, 14.1, respectively; compared to a maximum increase of + 3.4 p.p. for boys below the age of 15 in the week of 28 May 2021, and + 5 p.p. for boys aged 15–19 in the week of 2 July 2021, 95% CI 1.1, 5.6 and 2.7, 7.2, respectively), see Supplementary Figures S7–S8). Girls and young women appear to have been particularly hard hit by the pandemic and associated restrictions.
Parent helpline
Most children and adolescents live with their parents. This implies (a) that parents can offer a complementary source of information on the distress and mental health of young people, and (b) that the well-being of children and adolescents has strong externalities, as it immediately affects their parents. Data from a parent helpline allow us to address these two dimensions: an alternative angle on the well-being of children as viewed by their parents, and insights into problems linked to parents’ responsibilities for their children.
We observe a sharp and sustained increase in calls to the parent helpline after the outbreak of the pandemic contrasts with the largely unchanged volume of calls by children and adolescents themselves (Supplementary Fig. S1a). The surge in call volumes in 2020, however, is likely to have been driven at least in part by supply rather than by demand, as that helpline was publicized more actively and operating hours were extended after the outbreak of the pandemic77, see “Methods” Section for details.
Supply-side factors, however, cannot affect changes over time in the relative prevalence of conversation topics. We illustrate the evolution of topic prevalence in Fig. 3, analogous to Fig. 2. Here, we do not need to harmonize topic nomenclatures and can instead take them as designed by the helpline.
In Fig. 3a, we present topics related to callers’ children (see Supplementary Figure S9 for a decomposition into age groups 0–14 and > 15). The most striking pattern is an initial decrease of calls related to child mental health, which is the topic raised most frequently by parents (− 6.4 p.p. in the week of 22 April 2020, 95% CI − 9.4, − 3.3 ). That decrease, however, was followed in 2021 by a marked increase (+ 7.5 p.p. in the week of 7 May 2021, 95% CI 2.7, 12.2). This evolution is consistent with the pandemic initially alleviating mental-health problems but increasing them as it dragged on after about a year, confirming the patterns observed in calls from children and adolescents themselves.
A similar delayed increase in distress is evident in parents’ problems with their own lives (Fig. 3b): the topic “overload, helplessness” sharply increased in the second year of the pandemic (+ 10.7 p.p. in the week of 16 April 2021, 95% CI 4.9, 16.5), whereas the strongest increase in the prevalence of calls due to “burnout” only appeared in the third year of the pandemic (+ 9.4 p.p. in the week of 7 May 2022, 95% CI 5.6, 13.1). Problems linked to “child care”, in contrast, spiked in the early months of the pandemic (+ 9.7 p.p. in the week of 15 April 2020, 95% CI 6.7, 12.8) but returned to pre-pandemic levels thereafter, despite renewed school closures during the second COVID-19 wave in Germany (see Fig. 1a).
Overall, more calls to the parent helpline concern problems of the parents themselves (73%) than problems of their children (53%, see Supplementary Table S4). The share of parent-related calls increased by 4.1 percentage points between 2019 and 2022, and that of child-related calls to the parent helpline increased by 2.4 percentage points. This suggests that the pandemic was associated with a somewhat stronger increase in parents’ own problems.
Overall, we read the evidence from the parent helpline as suggesting that the toll of the pandemic weighed mainly on the parents themselves and grew gradually more severe as the pandemic lasted. This mirrors the evolution towards more severe mental health issues observed in children’s and adolescents’ own calls and is thus consistent with significant intra-family externalities.
War and pandemic
The data allow us to compare the mental and social effects of two large-scale crises: the outbreak of the COVID-19 pandemic in early 2020, with direct ramifications on everyday life, and the Russian attack on Ukraine in early 2022, which did not affect daily life immediately outside Ukraine, Russia and neighboring countries. Call volumes by children and adolescents did not noticeably increase in the wake of the Russian invasion (see Fig. 1). Calls by adults increased slightly in February 2022 (from an average of 2350 calls per day in January to 2417 in February), but most of that increase occurred prior to the invasion on February 24. This is in contrast to the pronounced increase observed after the outbreak of the pandemic (from 1703 average calls per day in February to 1897 in March, and 2080 in April 2020) or during the second COVID-19 wave (from, on average, 1915 calls per day during October 2020 to 2262 daily calls in January 2021). Judging by those data, both the war and the pandemic have affected adults more than the young.
In this context too, we can look beyond call volumes and search for patterns in the nature of calls. In Fig. 4, we compare changes in topic prevalence in the first month of the pandemic and in the first month of the war, in both cases scaled relative to the topic prevalence of the relevant age group in 2019.
We find that calls due to fears and anxiety increased in the wake of both the pandemic and the war. This effect is statistically significant and of very similar magnitude across the two crises for both children (+ 3 p.p. after the pandemic outbreak, 95% CI 1.3, 4.7; and + 2.8 p.p. after the invasion of Ukraine, 95% CI 1.1, 4.55) and adults (+ 4 p.p. after the pandemic outbreak, 95% CI 3.0, 4.9; and + 2.7 p.p. after the invasion of Ukraine, 95% CI 1.4, 4.0). Adolescents, however, were different: at the start of neither of the two crises did they call more often out of fear or anxiety (+ 0.4 p.p. after the pandemic outbreak, 95% CI -0.7, 1.5; and + 0.5 p.p. after the invasion of Ukraine, 95% CI − 1.0, 2.1).
Some differences between the two crises also emerge. Family-related problems increased (+ 5.8 p.p. for children and + 3.2 p.p. for adolescents, 95% CI 2.1, 9.5 and 1.0, 5.4, respectively), and school-related and peer-related problems decreased significantly in the early weeks of the pandemic, both for children (− 11.1 and − 6.0 p.p., 95% CI − 12.2, − 9.9 and − 8.1, − 3.8, respectively) and adolescents (− 4.4 and − 3.3 p.p., 95% CI − 6.2, − 2.5 and − 4.1, − 2.6, respectively). Those issues do not seem to have been affected by the outbreak of the war. This difference is easy to rationalize with the school closures and lockdown of early 2020.
For adults, the increase in calls out of fear and anxiety was similar after the outbreak of the pandemic (+ 4.0 p.p., 95% CI 3.0, 4.9) and after the outbreak of the war (+ 2.7 p.p., 95% CI 1.4, 4.0). However, calls due to loneliness increased only after the outbreak of the pandemic (+ 2.9 p.p., 95% CI 1.8, 3.9) but not after the outbreak of the war (− 0.4 p.p., 95% CI − 0.9, 0.0).
In summary, our analysis suggests that COVID-19 was the bigger of the two shocks when measured by the volume of calls made by adults. Neither of the two crises was followed by a sustained change in the volume of overall helpline calls placed by children and adolescents. Differences in the nature of calls during the two crises, in terms of the relative prevalence of conversation topics, have a straightforward explanation in the unequal nature of those crises (lockdown and school closures early in the pandemic but not in the war) and therefore validate the informativeness of helpline call data.
Discussion
We analyze calls to crisis helplines as a way to monitor trends in revealed distress by children, adolescents, and adults, over the course of the COVID-19 pandemic that had started in early 2020 and after the Russian invasion of Ukraine in February 2022. Data collected routinely by crisis helpline counselors provide a detailed and high-frequency measure of the prevalence of distress, as measured by call volumes, and of its nature, as measured by call topics, across different segments of the population.
We find that the volume of calls by children and adolescents increased less in the wake of either of those international crises than calls by adults. To the extent that call volumes reflect the level of social and mental distress in the population, this suggests that children and adolescents were less impacted by those crises than adults. It seems plausible to attribute this finding at least partly to different affectedness: adults faced greater health risks from COVID-19 and are more likely to live alone. They also were more likely to be informed about the wider security concerns after the Russian attack on Ukraine. Our data show that the increase in adult call volumes was mainly driven by fear (including fear of infection) and by loneliness.
The composition of calls by young people, however, changed differently from that of calls by adults. Since the outbreak of the pandemic, we observe a decrease in young people raising problems at school and with peers and an increase in young callers talking about problems with family and mental health. The mental health issues appear to have become more severe over the course of the pandemic. This pattern too has a natural explanation, as school closures and social distancing reduced opportunities for peer contact and forced closer contact with family members.
In short, our analysis suggests that the overall level of distress among young people as well as among adults (net of the proximate effects of the pandemic) has remained roughly unchanged, even if some subgroups of the population (e.g. girls) were more strongly affected.
Analyses of administrative data are consistent with no or small effects of the pandemic on mental health. For instance, suicide rates do not appear to have been significantly affected by the pandemic9,74,75,76,78 and hospital admissions for mental health issues have been flat or decreasing79. Police reports of child maltreatment and domestic violence did not increase substantially after the onset of the pandemic80,81.
Survey evidence, however, has on the whole pointed to a significantly raised level of distress throughout the pandemic, in clear contrast to clinical data34. Numerous surveys have reported increases in mental health problems after the outbreak of the pandemic2,37,82,83,84. Surveys targeted at children and adolescents found similar increases27,28. Those survey findings match our observations that, on aggregate, girls suffered more distress than boys and that for the affected subpopulation the severity of mental health issues increased over time.
One interpretation of the apparent differences is that it is difficult for surveys to cover the whole range of possible problems. Hence, they may struggle to uncover potential “offsetting” trends in problems other than those queried by the investigator. As a result of this issue, and also due to the difficulty of weighting different problem types, it is challenging to aggregate survey items into a combined measure of total distress. Helpline data allow us to circumvent this aggregation problem by considering call volumes, which provide an indication of the number of people for whom the level of distress exceeded the time and inhibition threshold for calling a helpline, regardless of the nature of the problem. This, in addition to offering a measure of revealed distress independent from potential experimenter demand effects, is a key advantage of helpline data.
Our analysis offers numerous avenues for further research. Child and youth helplines of other countries could be analyzed. If multiple countries, or regions within a country, could be covered, then panel data econometric techniques could be applied to disentangle the effects of different overlapping causes10. Another, even more challenging, objective could be to characterize the composition and representativeness of helpline caller populations. Such an analysis would require close collaboration with helpline managers as well as a solution to extract the required information without compromising caller anonymity.
Methods
Helpline statistics
We received data on calls with the two largest crisis helpline services in Germany, collected since January 2019. For calls that develop into conversations, counselors fill in an electronic report on individual caller characteristics and issues discussed, selecting relevant items from a pre-defined, helpline-specific list of topics. The helplines guarantee anonymity to their callers, and it is impossible to identify callers from the conversation-level data. Callers are informed that anonymous call data are collected for reporting and statistical purposes, explicitly in the terms and conditions and implicitly in annual reports and online publications.
Two of the helplines analyzed are operated by Nummer gegen Kummer e.V. (NgK), a non-profit organization established in 1980 that coordinates dedicated helpline services for children and adolescents–the Kinder- und Jugendtelefon (KJT) phone service and the chat/email-based Online-Beratung (OB)—as well as a telephone helpline for parents, the Elterntelefon (ET). NgK services have been supported by Deutsche Telekom AG since 1991, with additional funding provided by the German Federal Ministry for Family Affairs, Senior Citizens, Women and Youth, as well as by the European Union and other donors. The KJT service is free of charge, currently provided by 77 centers across Germany, where contacts are answered from Monday to Saturday between 2 and 8 pm by around 3200 trained volunteer counselors. The OB support via live chat is available on Wednesdays and Thursdays from 2 to 6 pm, E-mails are usually answered within 1–2 days. Data on chats are collected only from January 2020 onwards; for 2019 the OB data cover only E-mail contacts. The dedicated ET parent helpline has been operational nationwide since 2001 and currently comprises a network of 38 helpline centers. The minimum operating hours across all centers are Monday–Friday between 9 and 11 am, as well as Tuesdays and Thursdays from 5 to 7 pm. As the service gained popularity after the outbreak of the COVID-19 pandemic, ET operating hours were extended to cover Monday–Friday 9 am to 5 pm, and additionally from 9 am to 7 pm on Tuesdays and Thursdays. Further details are available on the NgK website at www.nummergegenkummer.de.
The third helpline analyzed, TelefonSeelsorge e.V. (TS) is the largest service in Germany, providing around-the-clock free general-purpose support by telephone, mail and online chat. The helpline is run by the Protestant and the Catholic churches and supported by the German Federal Ministry for Family Affairs, Senior Citizens, Women and Youth, as well as Deutsche Telekom AG. Of the over 100 active local centers, 82 started to report information about conversations in 2019; 6 additional centers started to collect data in January 2020. For additional information, see www.telefonseelsorge.de. Supplementary Tables S2–S4 present descriptive statistics.
Call volumes
For our main analyses, we combine data from the NgK child helpline and the general-purpose TS (both phone and chat/mail services, see Supplementary Table S1), exploiting the available information on caller characteristics to consistently distinguish sub-groups by age and gender. After every conversation, counselors record information on caller’s sex and age, as stated during the conversation or as inferred by the counselor. We distinguish three age groups: children up to age 14, adolescents aged 15–19, and adults aged 20 and older. In the Supplementary Information we show details separately for the two helpline services: Supplementary Fig. S1 shows the evolution of conversation volumes over time, Supplementary Fig. S2 shows the distribution of caller age.
Figure 1 shows the development of daily call volumes across age groups. Panel A shows the time series of daily calls identified as coming from young callers, between January 2020 and May 2022. Panel B shows the results of a regression specified as in Eq. 1, where the dependent variable is defined as the natural logarithm of the number of calls reported for day \(t\):
The indicator variable \(Jan\,2020_{t}\) is equal to one for days in January 2020 and zero otherwise. The variables \(Month_{t}\) denote the running month \(\tau\), counting from March 2020 (\(Month_{t}^{1}\)), so that the coefficients \(\gamma\) can be interpreted as the percentage deviation in monthly call volumes compared to February 2020. \(\theta_{t}\) denotes weekday indicators that account for variation through lower capacity at the KJT on weekends. We estimate separate regressions for each of the three age groups.
Deviation in conversation topic prevalence
For the analysis of conversation topics in Fig. 2, we drop 227,453 calls for which no information on the age of callers is available, as well as 41,683 calls without information on at least one conversation topic. This leaves us with a sample of 2,654,160 calls. To summarize the development across different helpline services, we have built a list of 20 topics each of which covers topics as defined in either helpline-specific nomenclature. Supplementary Table S6 shows the mapping of helpline-specific topics to our common categorization of non-exclusive topics; Supplementary Table S7 lists the original German descriptions of the detailed helpline-specific topics.
To capture deviations in the relative prevalence of conversation topics while controlling for seasonal patterns, we use the conversation-level data and estimate separate linear probability models for each of the three age groups. Specifically, we estimate Eq. 2, where the dependent variable \(T_{i,t}\) is set equal to one if topic \(T\) has come up in conversation \(i\) that took place in week \(t\), and zero otherwise:
The indicator variables \(Week_{t}^{\tau }\) for the running week \(\tau\), counting from the first week of 2019, are equal to one for calls during the respective week, and zero otherwise. Using the year 2019 as a reference, we include indicator variables from week 53 (January 2020) onwards. Calendar month-indicators \(\mu_{t}\) control for seasonal regularities, and helpline-specific intercepts \(\xi_{j\left( i \right)}\) capture time-invariant differences across helplines, where \(j\left( i \right)\) identifies the helpline which answered call \(i\). Estimates of the coefficients \(\delta_{\tau }\) can be interpreted as percentage-point deviations in relative prevalence as compared to calls to the same helpline in the corresponding calendar month of 2019. Standard errors are clustered at the helpline-month level.
Due to a technical problem at the TS helpline, information on conversation topics was recorded only for a relatively small share of contacts during the second week of September 2021. For the illustration in Fig. 2, we display instead the deviation for this week through interpolation based on the average coefficient estimates for the previous and subsequent week.
Using the same methodology, we obtain changes in topic prevalence from separate regressions separately for female and male callers. Calls for which information on caller sex is unavailable are dropped from the sample (18,117 calls), as well as 3317 callers labeled “diverse”. Results are shown in the Supplementary Figs. S6–S8. Supplementary Figs. S3 and S4 present results for the full list of topics at the helplines separately.
Figure 3 shows the results obtain for the 56,150 calls recorded at the NgK ET parent line between 1 January 2019 and 31 May 2022. Supplementary Table S4 shows descriptive statistics. Using the information on the age of children that adult callers refer to during calls, we repeat the analysis splitting the sample according to whether callers talk about children below the age of 15 or adolescents older than 14. These results are shown in Supplementary Fig. S9. Here, we drop 7464 calls for which no information on children’s age is available. Of the remaining 48,686 calls, 3570 (6.4%) refer to both children and adolescents. 2057 (3.7%) of the calls to the parent helpline do not refer to children of either age group.
Short-run changes after pandemic outbreak and Russian invasion
The results presented in Fig. 4 are based on separate regressions using data on calls around the two crisis events to isolate short-term changes in topic prevalence.
For the pandemic outbreak, we use data covering the time from 1 January 2019 to 13 April 2019, and the same time in 2020. We define the indicator variable \(Post\,\,out\,break_{t}\) set to one for calls on/after 13 March 2020, when the first significant lockdown measures were enacted73, and we estimate linear probability models for every topic according to Eq. 3:
We include week-of-year indicators \(\vartheta_{w\left( t \right)}\), where \(w\left( t \right)\) identifies the week-of-year of calendar time \(t\) to capture seasonal trends, and indicators for the year \(\Theta_{y} \left( t \right)\), where \(y\left( t \right)\) identifies the year of calendar time \(t\), and helpline indicators \(\xi_{j\left( i \right)}\) and age-group \(\alpha_{a\left( i \right)}\), where \(a\left( i \right)\) identifies the age group of the person who placed call \(i\). Estimates of the coefficient \(\gamma\) quantify the age-specific change in topic prevalence during the time from 13 March 2020 to 13 April 2020, compared to the time from 1 January 2020 to 12 March 2020, relative to the change observed between the periods 13 March 2019 to 13 April 2019 and 1 January 2019 to 12 March 2019. Standard errors are clustered at the helpline-week level. We repeat the analysis using data from 1 January up to 24 March in 2019 and 2022, where we define the indicator \(Post\,invasion_{t}\) as equal to one for calls on/after 24 February 2022.
Data availability
Data were provided by helplines for the sole purpose of this research project, subject to confidentiality agreements. Researchers can obtain (updated) helpline data after signing agreements with the individual helplines. For further information, contact info@nummergegenkummer.de and presse@telefonseelsorge.de. The full data underlying specific parts of the analysis are available from the authors at rafael.lalive@unil.ch upon reasonable request, conditional on permission of the respective helplines. All analyses were conducted using Stata/SE 17.0. Do files and Figure source data are available online at https://doi.org/10.5281/zenodo.7090520. The data on infection rates and policy measures shown in Fig. 1 are available online from the JHU CSSE COVID-19 Dataset8, and the Oxford COVID-19 Government Response Tracker73.
References
Jawaid, A., Gomolka, M. & Timmer, A. Neuroscience of trauma and the Russian invasion of Ukraine. Nat. Hum. Behav. https://doi.org/10.1038/s41562-022-01344- (2022).
Robinson, E., Sutin, A. R., Daly, M. & Jones, A. A systematic review and meta-analysis of longitudinal cohort studies comparing mental health before versus during the COVID-19 pandemic in 2020. J. Affect. Disord. 296, 567–576 (2022).
Knox, L., Karantzas, G. C., Romano, D., Feeney, J. A. & Simpson, J. A. One year on: What we have learned about the psychological effects of COVID-19 social restrictions–a meta-analysis. Curr. Opinion Psychol. 46, 101315 (2022).
Santomauro, D. F. et al. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. Lancet 398, 1700–1712 (2021).
Giuntella, O., Hyde, K., Saccardo, S. & Sadoff, S. Lifestyle and mental health disruptions during COVID-19. Proc. Natl. Acad. Sci. https://doi.org/10.1073/pnas.2016632118 (2021).
Petersen, M. B. et al. Communicate hope to motivate the public during the COVID-19 pandemic. Sci. Rep. 12, 1–7 (2022).
Chetty, R., Friedman, J. N., Hendren, N., Stepner, M. & Opportunity Insights. The economic impacts of COVID-19: Evidence from a new public database built using private sector data. (2020).
Dong, E., Du, H. & Gardner, L. An interactive web-based dashboard to track COVID-19 in real time. Lancet. Infect. Dis 20, 533–534 (2020).
Aknin, L. B. et al. Mental health during the first year of the COVID-19 pandemic: A review and recommendations for moving forward. Perspect. Psychol. Sci. 17, 915–936 (2022).
Brülhart, M., Klotzbücher, V., Lalive, R. & Reich, S. K. Mental health concerns during the COVID-19 pandemic as revealed by helpline calls. Nature 600, 121–126 (2021).
Perez-Arce, F. et al. COVID-19 vaccines and mental distress. PLoS ONE 16, e0256406 (2021).
Agrawal, V., Cantor, J. H., Sood, N. & Whaley, C. M. The impact of the COVID-19 vaccine distribution on mental health outcomes. (2021).
Witteveen, D. & Velthorst, E. Economic hardship and mental health complaints during COVID-19. Proc. Natl. Acad. Sci. 117, 27277–27284 (2020).
Ridley, M., Rao, G., Schilbach, F. & Patel, V. Poverty, depression, and anxiety: Causal evidence and mechanisms. Science 370, eaay0214 (2020).
Avdic, D., Sonja, C. & Kamhöfer, D. A. Economic downturns and mental health in Germany. Eur. Econ. Rev. 140, 103915 (2021).
Petherick, A. et al. A worldwide assessment of changes in adherence to COVID-19 protective behaviours and hypothesized pandemic fatigue. Nat. Hum. Behav. 5, 1145–1160 (2021).
Haktanir, A., Can, N., Seki, T., Kurnaz, M. F. & Dilmaç, B. Do we experience pandemic fatigue? Current state, predictors, and prevention. Curr. Psychol. 41, 7314 (2021).
Thompson, R. R., Jones, N. M., Holman, E. A. & Silver, R. C. Media exposure to mass violence events can fuel a cycle of distress. Sci. Adv. 5, eaav3502 (2019).
Cissé, G. et al. Health, wellbeing, and the changing structure of communities. in IPCC sixth assessment report: Climate change 2022 – impacts, adaptation and vulnerability (Intergovernmental Panel on Climate Change, 2022).
Romanello, M. et al. The 2021 report of the lancet countdown on health and climate change: Code red for a healthy future. Lancet 398, 1619–1662 (2021).
Burke, M. et al. Higher temperatures increase suicide rates in the United States and Mexico. Nat. Clim. Chang. 8, 723–729 (2018).
Burke, M. et al. Exposures and behavioural responses to wildfire smoke. Nat. Hum. Beh. 6, 1351 (2022).
Baylis, P. et al. Weather impacts expressed sentiment. PLoS ONE 13, e0195750 (2018).
Meierrieks, D. Weather shocks, climate change and human health. World Dev. 138, 105228 (2021).
Mullins, J. T. & White, C. Temperature and mental health: Evidence from the spectrum of mental health outcomes. J. Health Econ. 68, 102240 (2019).
Ma, T., Moore, J. & Cleary, A. Climate change impacts on the mental health and wellbeing of young people: A scoping review of risk and protective factors. Social Sci. Med. 301, 114888 (2022).
Ravens-Sieberer, U. et al. Three years into the pandemic: Results of the longitudinal german COPSY study on youth mental health and health-related quality of life. Front. Public Health 11, 1129073 (2023).
Racine, N. et al. Global prevalence of depressive and anxiety symptoms in children and adolescents during COVID-19: A meta-analysis. JAMA Pediatr. 175, 1142–1150 (2021).
Viner, R. et al. School closures during social lockdown and mental health, health behaviors, and well-being among children and adolescents during the first COVID-19 wave: A systematic review. JAMA Pediatr. 176, 400 (2022).
Takaku, R. & Yokoyama, I. What the COVID-19 school closure left in its wake: Evidence from a regression discontinuity analysis in Japan. J. Public Econ. 195, 104364 (2021).
Yamamura, E. & Tsustsui, Y. School closures and mental health during the COVID-19 pandemic in Japan. J. Popul. Econ. 34, 1261 (2021).
Huebener, M., Waights, S., Spiess, C. K., Siegel, N. A. & Wagner, G. G. Parental well-being in times of COVID-19 in Germany. Rev. Econ. Household 19, 91–122 (2021).
Bacher-Hicks, A., Goodman, J. & Holt, M. K. The COVID-19 pandemic disrupted both school bullying and cyberbullying. Am. Econ. Rev. Insights 4, 353–370 (2022).
Korczak, D. J., Madigan, S. & Vaillancourt, T. Data divide–disentangling the role of the COVID-19 pandemic on child mental health and well-being. JAMA Pediatr. 176, 635 (2022).
von Soest, T. et al. Adolescents’ psychosocial well-being one year after the outbreak of the COVID-19 pandemic in Norway. Nat. Hum. Behav. 6, 217–228 (2022).
Butterworth, P., Schurer, S., Trinh, T.-A., Vera-Toscano, E. & Wooden, M. The effect of lockdowns on mental health: Evidence from a natural experiment analysing an Australian longitudinal probability sample survey. SSRN Electron. J. https://doi.org/10.2139/ssrn.3990827 (2021).
Beutel, M. E. et al. Mental health and loneliness in the German general population during the COVID-19 pandemic compared to a representative pre-pandemic assessment. Sci. Rep. 11, 1–9 (2021).
Rattay, P. et al. Differences in risk perception, knowledge and protective behaviour regarding COVID-19 by education level among women and men in Germany: Results from the COVID-19 Snapshot Monitoring (COSMO) study. PLoS ONE 16, 0251694 (2021).
Mata, J. et al. Health behaviors and mental health during the COVID-19 pandemic: A longitudinal population-based survey in Germany. Soc. Sci. Med. 287, 114333 (2021).
Buecker, S. et al. Changes in daily loneliness for German residents during the first four weeks of the COVID-19 pandemic. Soc. Sci. Med. 265, 113541 (2020).
Benke, C., Autenrieth, L. K., Asselmann, E. & Pané-Farré, C. A. Lockdown, quarantine measures, and social distancing: Associations with depression, anxiety and distress at the beginning of the COVID-19 pandemic among adults from Germany. Psychiatry Res. 293, 113462 (2020).
Altindag, O., Erten, B. & Keskin, P. Mental health costs of lockdowns: Evidence from age-specific curfews in Turkey. Am. Econ. J. Appl. Econ. 14, 320–343 (2022).
Abreu, L. et al. Life with corona: Increased gender differences in aggression and depression symptoms due to the COVID-19 pandemic burden in Germany. Front. Psychol. https://doi.org/10.3389/fpsyg.2021.689396 (2021).
Bertrand, M. & Mullainathan, S. Do people mean what they say? Implications for subjective survey data. Am. Econ. Rev. 91, 67–72 (2001).
Hamermesh, D. S. Subjective outcomes in economics. South. Econ. J. 71, 1–11 (2004).
Brown, S., Harris, M. N., Srivastava, P. & Taylor, K. Mental health, reporting bias and economic transitions. Oxf. Econ. Pap. 74, 541–564 (2022).
Jahedi, S. & Méndez, F. On the advantages and disadvantages of subjective measures. J. Econ. Behav. Organ. 98, 97–114 (2014).
Carlsson, F., Kataria, M. & Lampi, E. Demand effects in stated preference surveys. J. Environ. Econ. Manag. 90, 294–302 (2018).
Pierce, M. et al. Says who? The significance of sampling in mental health surveys during COVID-19. Lancet Psychiatry 7, 567–568 (2020).
Hainmueller, J., Hangartner, D. & Yamamoto, T. Validating vignette and conjoint survey experiments against real-world behavior. Proc. Natl. Acad. Sci. 112, 2395–2400 (2015).
Fried, E. I., Flake, J. K. & Robinaugh, D. J. Revisiting the theoretical and methodological foundations of depression measurement. Nat. Rev. Psychol. 1, 358–368 (2022).
Wang, J. et al. Global evidence of expressed sentiment alterations during the COVID-19 pandemic. Nat. Hum. Behav. 6, 349–358 (2022).
Metzler, H. et al. Collective emotions during the COVID-19 outbreak. Emotion (2022).
Kryvasheyeu, Y. et al. Rapid assessment of disaster damage using social media activity. Sci. Adv. 2, e1500779 (2016).
Pellert, M., Schweighofer, S. & Garcia, D. Social media data in affective science. In Handbook of Computational Social Science 240–255 (Routledge, 2021).
Pellert, M., Metzler, H., Matzenberger, M. & Garcia, D. Validating daily social media macroscopes of emotions. Sci. Rep. https://doi.org/10.1038/s41598-022-14579-y (2022).
Fetzer, T., Hensel, L., Hermle, J. & Roth, C. Coronavirus perceptions and economic anxiety. Rev. Econ. Stat. 103, 968–978 (2020).
Brodeur, A., Clark, A. E., Fleche, S. & Powdthavee, N. COVID-19, lockdowns and well-being: Evidence from google trends. J. Public Econ. 193, 104346 (2020).
Foa, R. S., Fabian, M. & Gilbert, S. Subjective well-being during the 2020–21 global coronavirus pandemic: Evidence from high frequency time series data. PLoS ONE 17, e0263570 (2022).
Halford, E. A., Lake, A. M. & Gould, M. S. Google searches for suicide and suicide risk factors in the early stages of the COVID-19 pandemic. PLoS ONE 15, e0236777 (2020).
Gimbrone, C. et al. Associations between COVID-19 mobility restrictions and economic, mental health, and suicide-related concerns in the US using cellular phone GPS and Google search volume data. PLoS ONE 16, e0260931 (2021).
Batchelor, S., Stoyanov, S., Pirkis, J. & Kõlves, K. Use of kids helpline by children and young people in Australia during the COVID-19 pandemic. J. Adolesc. Health 68, 1067–1074 (2021).
Salmi, S., Mérelle, S., Gilissen, R., van der Mei, R. & Bhulai, S. Detecting changes in help seeker conversations on a suicide prevention helpline during the COVID-19 pandemic: In-depth analysis using encoder representations from transformers. BMC Public Health 22, 1–10 (2022).
Zalsman, G. et al. Suicide-related calls to a national crisis chat hotline service during the COVID-19 pandemic and lockdown. J. Psychiatr. Res. 139, 193–196 (2021).
Turkington, R. et al. Behavior of callers to a crisis helpline before and during the COVID-19 pandemic: Quantitative data analysis. JMIR Mental Health 7, e22984 (2020).
Arendt, F., Markiewitz, A., Mestas, M. & Scherr, S. COVID-19 pandemic, government responses, and public mental health: Investigating consequences through crisis hotline calls in two countries. Soc. Sci. Med. 265, 113532 (2020).
Burgt, M. C. van der, Mérelle, S., Beekman, A. T. & Gilissen, R. The impact of COVID-19 on the suicide prevention helpline in The Netherlands. Crisis (2022).
Jacobsen, A. L. et al. Level of suicidal ideation among callers to the Danish suicide prevention helpline. Arch. Suicide Res. https://doi.org/10.1080/13811118.2022.2084005 (2022).
Choi, D. et al. Development of a machine learning model using multiple, heterogeneous data sources to estimate weekly US suicide fatalities. JAMA Netw. Open 3, e2030932 (2020).
Niederkrotenthaler, T. et al. Association of Logic’s hip hop song ‘1-800-273-8255’ with Lifeline calls and suicides in the United States: Interrupted time series analysis. BMJ 375, e067726 (2021).
Port, M. S. et al. Perceived impact of COVID-19 among callers to the national suicide prevention lifeline. Crisis https://doi.org/10.1027/0227-5910/a000881 (2022).
Schleider, J. L. et al. A randomized trial of online single-session interventions for adolescent depression during COVID-19. Nat. Hum. Behav. 6, 258–268 (2022).
Hale, T. et al. A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat. Hum. Behav. 5, 529–538 (2021).
Radeloff, D. et al. Trends in suicide rates during the COVID-19 pandemic restrictions in a major German city. Epidemiol. Psychiatr. Sci. https://doi.org/10.1017/S2045796021000019 (2021).
Kastner, U. W. et al. Effects of the COVID-19 pandemic on suicide attempts in a rural region in Germany, a 5-year observational study. SSRN Electron. J. https://doi.org/10.2139/ssrn.4065611 (2022).
Kim, S. Y., Kim, H.-R., Park, B. & Choi, H. G. Comparison of stress and suicide-related behaviors among Korean youths before and during the COVID-19 pandemic. JAMA Netw. Open 4, e2136137 (2021).
Nummer gegen Kummer e.V. Statistik 2021—Elterntelefon. (2022).
Pirkis, J. et al. Suicide numbers during the first 9–15 months of the COVID-19 pandemic compared with pre-existing trends: An interrupted time series analysis in 33 countries. EClinicalMedicine 51, 101573 (2022).
Fasshauer, J. M. et al. Impact of COVID-19 pandemic on involuntary and urgent inpatient admissions for psychiatric disorders in a German-wide hospital network. J. Psychiatr. Res. 142, 140–143 (2021).
Baron, E. J., Goldstein, E. G. & Wallace, C. T. Suffering in silence: How COVID-19 school closures inhibit the reporting of child maltreatment. J. Public Econ. 190, 1–13 (2020).
Cappa, C. & Jijon, I. COVID-19 and violence against children: A review of early studies. Child Abuse Negl. 116, 105053 (2021).
Daly, M. & Macchia, L. Global trends in emotional distress. Proc. Natl. Acad. Sci. https://doi.org/10.1073/pnas.2216207120 (2023).
Riehm, K. E. et al. Trajectories of mental distress among US adults during the COVID-19 pandemic. Ann. Behav. Med. 55, 93–102 (2021).
Pierce, M. et al. Mental health responses to the COVID-19 pandemic: A latent class trajectory analysis using longitudinal UK data. Lancet Psychiatry 8, 610–619 (2021).
Acknowledgements
We thank Heidi Schütz and Anna Zacharias (Nummer gegen Kummer), Ludger Storch and Lydia Seifert (TelefonSeelsorge) for sharing their data with us, as well as for their continued support in analyzing and interpreting the data. We are grateful to the Swiss National Science Foundation (Grant No. 100018_204575) and NCCR LIVES—‘Overcoming Vulnerability: Life Course Perspectives’ for financial support.
Author information
Authors and Affiliations
Contributions
M.B., V.K. and R.L. designed the analysis. V.K. implemented the analysis. M.B. wrote the main text. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Brülhart, M., Klotzbücher, V. & Lalive, R. Young people’s mental and social distress in times of international crisis: evidence from helpline calls, 2019–2022. Sci Rep 13, 11858 (2023). https://doi.org/10.1038/s41598-023-39064-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-023-39064-y
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.