Introduction

The prevalence of depression in the United States has increased multiple fold to approximately 32% during the COVID-19 pandemic [1]. Correspondingly, >40 million adults (~19%) have anxiety disorders [2], often with co-morbid depressive symptoms. Efficacious psychotherapies such as problem-solving treatment (PST) exist [3], and in-person PST is a proven intervention for treating both depression and anxiety, which often manifest as comorbid conditions [4,5,6]. However, access to such therapies is affected by shortages in mental healthcare professionals and high out-of-pocket costs. Digital mental health applications offer viable solutions [7, 8]; consumer-based voice assistants that leverage artificial intelligence to deliver psychotherapy is a nascent and underexplored area with considerable potential for behavioral counseling and to promote emotional well-being [9, 10].

With the integration of voice assistants in mobile devices, their use is pervasive; recent reports have highlighted that nearly a third of search queries rely on voice input [11]. However, their use in healthcare delivery is limited [10], with current research largely focusing on information seeking activities on topics including healthy lifestyle [12, 13], medication names [14], and mental health [15, 16]. Although prototypes of voice-based applications for behavioral assessment and counseling have been developed (e.g., [17, 18]), high-quality clinical research on their therapeutic potential is currently lacking.

Relying on user-centered design principles, and aligned with the treatment fidelity of PST, we developed a virtual voice-based coach, Lumen, that delivers PST for patients with mild-to-moderate depression and/or anxiety [19, 20]. We conducted a pilot randomized clinical trial (RCT) to obtain initial evidence on the effects of PST delivery using Lumen on brain function and clinical outcomes, as is consistent with a theory-driven, mechanism-focused approach to treatment evaluation. The primary objectives were to assess the magnitude of treatment effects on: (a) the activation of a priori neural targets involved in emotional reactivity and cognitive control, the two core theoretical constructs for PST, and (b) depression and anxiety symptoms. We also assessed associations between neural targets and self-reported surveys of emotional reactivity and cognitive control.

Method

The Institutional Review Board for the University of Illinois Chicago (UIC) approved the study. All participants provided written consent. The study was registered on ClinicalTrials.gov (NCT# 04524104).

Participants

Enrollment followed a multi-step process (Fig. 1). Participants were recruited between April 5, 2021, and October 7, 2021, from the outpatient care clinics at the University of Illinois Hospital and Health Sciences System (UI Health) and employee email listservs at UIC, a minority-serving institution.

Fig. 1: Consort chart.
figure 1

Flowchart regarding the enrollment and randomization of participants.

Full size image

Adults were deemed eligible if they had a Patient Health Questionnaire-9 (PHQ-9) score of 10–19 and/or a Generalized Anxiety Disorder Scale (GAD-7) score of 10–14, without serious medical or psychiatric comorbidities or other exclusions (see Supplementary Material, Section A; also see full protocol in Supplementary Material, Section H). Participants were asked to self-identify their race and ethnicity based on fixed categories to comply with National Institutes of Health’s reporting requirements.

Participants were compensated for this study. As part of this study, participants made two visits for neuroimaging (baseline [visit 1], and at 16 weeks [visit 2]). Upon completion of visit 1, all participants received $50 in compensation. At visit 2, participants in the Lumen intervention arm could choose to receive $100 in compensation or keep the iPad with their access to Lumen deactivated. For those in the waitlist control arm, at visit 2, participants could choose to receive $100 in compensation or choose to attend a Lumen orientation session and receive a Lumen PST-enabled iPad (which they could keep in lieu of the $100 compensation).

Randomization and masking

Participants were randomly assigned in a 2:1 ratio to receive the Lumen intervention or to be in a waitlist control group using a validated online system [21] based on Pocock’s covariate-adaptive minimization [22]. The 2:1 allocation allowed more participants to receive the Lumen intervention without substantially reducing statistical power [23]. Pocock’s minimization method was used to achieve better-than-chance marginal balance across multiple baseline characteristics: age, sex, race/ethnicity, education, PHQ-9 score, GAD-7 score, and Digital Health Literacy [24]. Investigators, the safety monitor, outcome assessors, and data analysts were blinded to participants’ treatment assignment.

Lumen

Lumen is a virtual voice-based coach developed on Amazon’s Alexa platform. Lumen delivers an evidence-based PST program [5, 6] consisting of eight sessions (four weekly, followed by four biweekly sessions) for patients with mild-to-moderate depression and/or anxiety. PST is patient-driven, where the coach acts as a guide to identify a problem, set a goal, brainstorm solutions, choose a solution, develop an action plan, and to implement and evaluate the plan [25]. This stepwise approach makes PST appropriate for therapy delivery using a virtual voice-based coach.

Lumen was designed through an iterative user-centered process that involved software developers, interaction designers, psychiatrists, PST experts, and behavioral scientists. Several iterations of the prototype were internally tested; a fully functional prototype underwent feasibility and usability testing with 26 users [19]. The design was driven by two key principles: (a) aligning participants’ voice-based interaction with Lumen similar to the cognitive processes of human communicative interactions [26], and (b) configuring the content of the interactions with the principles and process of evidence-based PST. Towards this end, the Lumen architecture included multiple, interacting components that managed voice-based therapy delivery (a conversation manager), and ascertaining persistence and consistency across the eight therapy sessions (a context manager; see additional information in Supplementary Material sections B, C, and D; Figure S1, Table S1, and Table S2).

For this study, Lumen was integrated within the Alexa app on an iPad provided to all participants. Lumen participants attended an in-person orientation session with a trained health coach where they received their iPad, intervention workbook, and completed a tutorial on how to interact with Lumen. Participants were instructed to begin their PST right away, within 1 week of their orientation session and the health coach helped schedule their 4 weekly and the following 4 biweekly PST sessions. Within 3 days of their first scheduled PST session, the health coach called participants to inquire about any technical issues and helped troubleshoot these issues (if any). Participants received reminder text messages about their upcoming and overdue (if any) PST sessions. Participants with overdue sessions, even after their reminders, were called by the health coach and encouraged to complete their outstanding session(s). Participants also had the opportunity to reach out to the health coach if they faced any issues as part of their study.

For each session, participants instantiated Lumen PST through the Alexa app with a “Launch Lumen Coach” voice instruction and completed their assigned PST sessions. A typical Lumen session lasted ~12 min. Between sessions, participants completed surveys and ecological momentary assessments (EMAs, see Supplementary Materials, Section D, Table S2).

Waitlist control

Participants in the waitlist control arm received automated text messages to complete surveys and EMAs at intervals similar to the intervention arm. These participants could choose to receive a Lumen-enabled iPad after their end-of-study assessments at 16 weeks.

Neural target measures

Blinded outcome assessors conducted standardized assessments at baseline and 16 weeks. Task-based functional magnetic resonance imaging (fMRI) data were collected utilizing previously-established standardized fMRI sequences and parameters [27, 28] that inform transdiagnostic phenotypes of neural circuit dysfunction for depression and anxiety. These fMRI methods, including facial expressions task and Go-NoGo tasks, have been standardized in previous work designed for application to precision psychiatry and target engagement studies [29, 30]. A brief description of these tasks are provided below, and additional details can be found in the Supplementary Materials (Section E).

Facial expressions task

A standardized set of 3D evoked facial expression stimuli was presented in pseudorandom order, with 5 repeated blocks of 8 stimuli per block for sad, fear, anger, and happy relative to neutral blocks [29]. Participants were instructed to continuously view the faces and were informed beforehand that they would be asked post-scan questions about the faces they were viewing. To assess amygdala activation for the negative affect circuit, our analysis focused on threatening faces only, given our prior research showing threat-related amygdala activation mediating the effect of in-person PST on depression and problem-solving outcomes [31]. Threat stimuli included a combination of fear and anger stimuli relative to neutral blocks. During the conscious viewing condition, each face was presented for 500 ms, with an interstimulus interval of 750 ms. To elicit the negative affect circuit in response to non-conscious threat stimuli, the same fear and anger stimuli were presented in a backward-masking design to prevent awareness. In this non-conscious condition, face stimuli were presented for 10 ms followed immediately by a neutral face mask stimulus for 150 ms, and with a stimulus onset asynchrony of 1250 ms to match that of the conscious condition [32].

Go-NoGo task

For the Go-NoGo paradigm, the ‘Go’ and ‘NoGo’ stimuli were presented for 500 ms each with an interstimulus interval of 750 ms. The Go-NoGo paradigm allowed for event-related analysis and is used to assess impulsivity (automatically generated ‘Go’ responses) versus inhibition (‘NoGo’ responses). In the ‘Go’ trials, participants were asked to press a button on GREEN stimuli as quickly as possible (with the word “press” displayed in green); in the ‘NoGo’ trials, participants should withhold button presses on RED stimuli (with the word “press” displayed in red). The probability of ‘NoGo’ stimuli was 0.33. A total of 180 ‘Go’ and 60 ‘NoGo’ stimuli were presented in a pseudorandom order with a constraint to ensure that ‘NoGo’ stimuli were not repeated more than 3 times in a row. Reaction times and number of errors on task were used to evaluate task performance [29].

Informed by previous findings [6, 31] identifying neural targets engaged by in-person PST, the primary target regions of interest (ROIs) were the amygdala (bilaterally) representing a key node in the negative affect circuit, and the dorsal lateral prefrontal cortex (dlPFC) (bilaterally), a key node in the cognitive control circuit. The negative affect circuit was engaged by the viewing of threat faces in the non-conscious viewing condition. The cognitive control circuit was engaged using the Go/No-go task.

Person-level activation of the ROIs for each contrast of interest for each task (e.g., threat versus neutral faces, no-go versus go) was derived in a manner consistent with the methods used in prior studies [27].

Clinical outcome measures

On the Hospital Anxiety and Depression Scale (HADS) [33, 34], depression and anxiety symptom scores ranged from 0 to 21, with 0–7 indicating normal; 8–10 indicating borderline abnormal (borderline); and 11–21 indicating abnormal (case). HADS total scores were computed as the sum of depression and anxiety scores, indicating overall psychological distress.

Self-reported measures

Validated self-report surveys of PST theory-based constructs of emotion (affect, worry) and cognition (problem-solving, dysfunctional attitudes) were also completed at baseline and 16 weeks. The Positive and Negative Affect Schedule (PANAS) assessed positive and negative affect [35], with scores ranging from 10 to 50 and higher scores representing higher levels of positive or negative affect. Worry was measured using the Penn State Worry Questionnaire (PSWQ), with a higher total score indicating more worry (range 16–80) [36]. The Social Problem-solving Index-Revised Short Form (SPSI-R:S) assessed total problem-solving ability, with the higher score indicating more productive problem-solving skills, and 5 subscales including problem orientation (positive, negative) and problem-solving styles (rational, impulsive/careless, and avoidant) [37]. Each subscale was scored by summing the respective 5 items (each from 0 to 4), and the total problem-solving ability score ranged from 0 to 20 by averaging the subscale scores. Dysfunctional Attitudes Scale (DAS) measured the presence and intensity of dysfunctional attitudes, with higher scores indicating more dysfunctional attitudes (range 40–280) [38].

Statistical analysis

The intervention vs. control effects on changes in neural targets and self-reported measures of emotional reactivity and cognitive control from baseline to 16 weeks were assessed using t tests. Correlations of changes in neural targets with changes in self-reported measures were estimated using Pearson’s correlation tests.

The intervention vs. control effects on changes in HADS scores from baseline to 16 weeks were tested using ordinary least square regression with adjustment of baseline values of the outcome measure. Each model included all participants with follow-up data on the outcome at 16 weeks, and participants were analyzed based on the group to which they were assigned. Moderation analysis used the same models as above plus the main effect of each potential effect modifier (e.g., sex) and its interaction with the group; the latter, if significant, rejected the null hypothesis of no moderation. Model-adjusted between-treatment mean differences with 95% confidence intervals (CIs) for the overall sample and the subgroups defined by the effect modifiers were reported. Cohen’s d was calculated by the mean difference between the two groups divided by the pooled standard deviation.

Given that this study was a pilot RCT, the primary purpose was to establish a reliable signal regarding the impact of Lumen on neural targets and clinical outcomes that would be promising enough to warrant further research. Towards this end, we used Cohen’s d ≥ 0.3 to define the meaningful mean difference between the intervention and control groups in neural target and symptom changes from baseline to 16 weeks. Moreover, our approach to data reporting and interpretation regarding the intervention effects on neural targets and symptom outcomes was focused on the magnitude and precision (95% CI) of the effect estimates, and not on p-values [5]. Similarly, we were not focused on smaller correlations (Pearson’s r < 0.4) between the neural targets and self-reported measures as it would have limited clinical relevance.

All analyses were conducted using SAS, version 9.4 (SAS Institute Inc., Cary, North Carolina).

Sample size calculation

The sample size of this pilot RCT was calculated using a confidence interval approach. To obtain a precision interval with a standardized half-width of 0.50 (akin to a medium effect) with 90% assurance, we had planned a sample size of 60 (nTreatment = 40, nControl = 20), assuming ≥85% retention at 16 weeks. A precision interval approach was used where we defined that, compared with the waitlist control group, the intervention group will demonstrate a meaningful improvement in outcomes (in both neural targets and symptoms) if the standardized between-group mean difference was at least Cohen’s d = 0.3 in favor of intervention. At this effect size, the upper limit of the precision interval overlaps with d = 0.8 (large effect) given a standardized half-width of 0.5 with 90% assurance that the interval contains the true mean difference based on power analysis. For the correlation of change in neural targets with change in self-reported measures, a sample size of 51 (i.e., 60 × 85%) would be sufficient to detect a coefficient of r = 0.4 with 80% power and 2-sided α = 0.05.

Results

Sample characteristics and retention

Of 1049 individuals who completed screening, 936 were ineligible and 50 declined or were unable to participate (Fig. 1). Randomized participants included 42 in the intervention and 21 in the waitlist control. Participants had a mean age of 37.8 years (SD = 12.4), 68% were women, 25% were Black, 24% were Latino, 59% had a high school or college (1 to 4+ years) education, and 51% had an annual income less than $55,000 (Table 1). On average, participants had moderate depression (mean PHQ-9 Score=12.8 [SD = 3.1]) and anxiety (mean GAD-7 Score=9.7 [SD = 2.7]), and borderline abnormal HADS depression scores (mean HADS depression = 7.3 [SD = 3.0]), and abnormal HADS anxiety scores (mean HADS anxiety Score=10.6 [SD = 3.3]). Participants had a total HADS score of 17.9 [SD = 5.2]. Based on PHQ-9 and GAD-7 categories of symptom severity, a majority of the 63 participants had moderate to moderately severe depression or moderate anxiety, and 29 had both. All 63 participants had complete baseline data, and 61 (97%) were assessed at 16 weeks. Of the 42 Lumen participants, 38 (90.5%) completed at least 4 PST sessions, and 34 (81.0%) completed all 8 PST sessions.

Table 1 Eligibility screening and sociodemographic measuresa; the prognostic factorsb for randomization included age, sex, race/ethnicity, educational level, PHQ-9 score, and GAD-7 score.
Full size table

Intervention effect on activation of neural targets

Between-group mean differences in changes in the primary ROIs (i.e., activation decrease in the control group > intervention in L. amygdala, activation decrease in intervention group > control group) engaged in the non-conscious threat stimuli (i.e., emotional reactivity) from baseline to 16 weeks did not meet the Cohen’s d = 0.3 threshold. There was, however, a meaningful change in a primary cognitive control target: the activation of both the right and left dlPFC decreased in the intervention from baseline to 16 weeks compared with the control, while the between-group mean difference in activation of the right dlPFC (−0.20 [95%CI: −0.61, 0.22]) met Cohen’s d = 0.3 (Table 2).

Table 2 Treatment effects on primary neural target regions of interest and self-reported measuresa,b.
Full size table

Intervention effect on clinical outcomes

At 16 weeks, intervention participants had greater improvements in their HADS depression, anxiety and total scores compared with control participants, with a medium effect size. Model-adjusted between-group mean difference was −1.33 (95%CI: −3.26, 0.60; Cohen’s d = 0.49) for the HADS depression score, −1.58 (95%CI: −3.82, 0.66; Cohen’s d = 0.51) for the HADS anxiety score, and −2.89 (95%CI: −6.76, 0.99; Cohen’s d = 0.55) for the HADS total score (Table 3).

Table 3 Treatment effects on depression and anxiety symptoms.
Full size table

The treatment effect on the HADS depression score was significantly moderated by sex (p = 0.048), education (p = 0.048), and digital health literacy score (p = 0.03). The Lumen group had consistently greater improvements than the control group in HADS depression, anxiety, and total scores than control participants among women (between-group mean difference [95% CI]: −2.5 [−4.8, −0.3], −2.2 [−4.9, 0.5], −4.7 [−9.3, −0.2], respectively), non-White (−2.4 [−4.7, −0.0], −2.7 [−5.4, 0.0], −5.1 [−9.8, −0.4], respectively), and those with college or less education (−2.9 [−5.6, −0.2], −3.1 [−6.3, 0.1], −6.0 [−11.4, −0.6], respectively) at 16 weeks (Fig. 2).

Fig. 2: Treatment effect on symptoms.
figure 2

Treatment effects on HADS depression score (HADS_D), HADS anxiety score (HADS_A) and total HADS score (HADS_T), overall and by sub-group.

Full size image

In addition, participants with lower digital health literacy scores had a greater mean decrease in their HADS depression score in the Lumen vs. control group (Supplementary Material, Section F, Figure S2).

Intervention effect on self-reports

Each of the self-reports associated with emotional reactivity (positive affect score, negative affect score, Penn State worry questionnaire) showed improvement in the intervention and control groups from baseline to 16 weeks, but did not show meaningful between-group differences (all Cohen’s d < 0.3). Similarly, all self-reports associated with cognitive control showed improvement over time, with the rational problem-solving (1.43 [95%CI: −1.25, 4.1], Cohen’s d = 0.3) and impulsive/careless raw scores (−1.47, [95%CI: −3.57, 0.63], Cohen’s d = 0.4) having meaningful differences in improvement in the intervention group compared with the control (Table 2).

Association of neural targets and self-reports

In the intervention group, an increase in the activation of the right dlPFC was positively correlated with an increased self-reported SPSI-R score indicating improved problem-solving ability (r = 0.4, p = 0.02), and negatively associated with the self-reported avoidant score indicating reduced avoidance (r = −0.5, p = 0.01), from baseline to 16 weeks. In the waitlist control group, the right dlPFC was negatively correlated with dysfunctional attitudes score (r = −0.5, p = 0.04). Moreover, associations were in the opposite direction in the control group for several of the other considered scales, although they were not statistically significant: negative correlation with the self-reported SPSI-R score (r = −0.5, p = 0.050), and positive correlation with the self-reported avoidant score (r = 0.3, p = 0.20) (see Supplementary Materials, Section G, Tables S3 and S4; Figure S3).

Discussion

A virtual voice-based coach intervention showed meaningful changes in a subset of select neural targets, with a decrease in the activation of the primary neural target related to cognitive control—the right dlPFC—in the intervention group compared with the control. Related self-reported rational problem-solving and impulsive/carelessness scores also showed meaningful improvements with Lumen. The change in the right dlPFC activation was also correlated positively with self-reported problem-solving ability scores and negatively with the avoidance scores in the Lumen group. Moreover, participants in the Lumen intervention group showed improvements in both depression and anxiety symptoms as well as total psychological distress at 16 weeks, compared with the waitlist control group. The between-group differences in the HADS-D and HADS-A scores were consistently greater than clinically important differences defined as 1.5–1.7 in prior studies [39, 40]. These treatment effects were moderated by participant sex, race/ethnicity, and educational status. These findings offer a consistent signal and support the pragmatic viability of Lumen as a promising digital intervention to address mild-to-moderate depression and/or anxiety.

To the best of our knowledge, this is the first clinical trial of a virtual voice-based coach for behavioral therapy, coupled with neuroimaging, that was delivered using a consumer-based voice platform (i.e., Amazon’s Alexa). The demonstration of neural target engagement for cognitive control and improved clinical outcomes is promising and offers considerable opportunity for delivering PST. Moreover, more than 80% of the Lumen participants completed all 8 PST sessions, suggesting high feasibility and acceptability among a highly diverse group of participants.

The neural mechanistic findings supplement the validated self-report clinical outcomes, and may help elucidate the theory-based therapeutic underpinnings of this novel form of PST delivery. Meaningful decrease in the activation of right dlPFC in the Lumen group is consistent with other studies suggesting that right hemisphere hyperactivation is associated with depressive disorders [41]. Furthermore, the right dlPFC is frequently used as a target for low-frequency inhibitory repetitive transcranial magnetic stimulation, suggesting that a reduction of activity in this region has an antidepressant effect [42]. The association of right dlPFC activation with increased problem-solving skills and decreased avoidant scores in the intervention group may represent efficient processing by a key node involved in the cognitive control circuit. This finding, in combination with our prior work from the ENGAGE-2 study [6], suggests that the malleability of dlPFC activity may serve as a prognostic biomarker for identifying patients who would likely respond to this type of intervention.

The presence of moderators in our models suggests that women, minorities, those college or less educated or those with lower digital health literacy may likely benefit more from using Lumen. Among marginalized groups with lower resources and limited access to mental health services, Lumen offers a potential resource for easy and on-demand access. This is especially the case, given the significant proliferation of mobile phones with voice applications. A recent qualitative study [43] found similar results, showing the potential benefits and opportunities for using virtual technology for health management among Black men, further emphasizing the role of digital tools among minority populations.

There is considerable published literature and on-going research on the use of text-based applications (“chatbots”) for mental health support [44, 45]; however, a recent meta-analysis showed that text-based chatbots had mixed results [46]. In this context, our study using a virtual voice-based coach offers a novel mental health therapy delivery mechanism. The absence of full-fledged RCTs of voice-based applications is likely due to the challenges high-quality natural language understanding on these platforms. However, it must be stated that these platforms have had greater success in directed and specific conversations utilizing the extensive modern machine learning and natural language processing algorithms. As such, Lumen was designed to string together multiple short, directed conversations (e.g., “what is your goal?”) that reflect the therapeutic approach underlying the delivery of PST [19].

This study has several limitations. First, this was a pilot RCT with a small study sample, among those with mild-to-moderate depression and/or anxiety, increasing the probability of false discovery and failure to detect uncommon problems or adverse events. Nonetheless, this study provides the foundational evidence for a planned confirmatory study (NCT05603923). Second, task-based neuroimaging studies have had varying within-subjects reliability, which may have reduced the capacity to detect changes in certain neural targets. Future studies will also complement data-driven approaches such as whole-brain network analysis. Third, the comparison group did not receive any treatment, and it remains unclear how Lumen differs from PST delivered by human coaches. Fourth, we considered several moderator variables in our analysis; given the small sample size the findings should be considered preliminary. Finally, although education and digital literacy were moderating variables, the overall sample was well-educated and digitally literate.

In summary, this pilot RCT provides preliminary evidence that a virtual voice-based coach may represent an alternative option of PST delivery for managing mild-to-moderate depression and anxiety. This innovative approach may reduce barriers to mental health care access, particularly for vulnerable populations.