Associations between aversive learning processes and transdiagnostic psychiatric symptoms revealed by large-scale phenotyping

Background Aversive learning processes are a candidate source of dysfunction in psychiatric disorders. Here symptom expression in a range of conditions is linked to altered threat perception, manifesting particularly in uncertain environments. How precise computational mechanisms that support aversive learning, and uncertainty estimation, relate to the presence of specific psychiatric symptoms remains undetermined. Methods 400 subjects completed a novel online game-based aversive learning task, requiring avoidance of negative outcomes, in conjunction with completing measures of common psychiatric symptoms. We used a probabilistic computational model to measure distinct processes involved in learning, in addition to inferred estimates of safety likelihood and uncertainty. We tested for associations between learning processes and traditional psychiatric constructs alongside transdiagnostic factors using linear models. We used partial least squares regression to identify components of psychopathology grounded in both aversive learning behaviour and symptom self-report. Results State anxiety and a transdiagnostic compulsivity-related factor were associated with enhanced learning from safety. However, data-driven analysis using partial least squares regression indicated the presence of two separable components across our behavioural and questionnaire data: one linked enhanced safety learning and lower estimated uncertainty to physiological anxiety, compulsivity, and impulsivity; the other linked enhanced threat learning and heightened uncertainty estimation to symptoms of depression and social anxiety. Conclusions Our findings implicate aversive learning processes under uncertainty to the expression of psychiatric symptoms that cut across traditional diagnostic boundaries. These relationships are more complex than previously conceptualised. Future research should focus on understanding the neural mechanisms underlying alterations in aversive learning and how these lead to the development of symptoms and disorder.


37
Many core symptoms of mental illness are linked to learning about unpleasant events in our 38 environment. In particular, symptoms of mood and anxiety disorders, such as apprehension, 39 worry, and low mood can intuitively be related to altered perception of the likelihood of aversive 40 outcomes. Indeed, the importance of altered threat perception is a feature of many disorders 41 that extend beyond disorders of mood to encompass conditions such as psychosis (1) and 42 eating disorders (2). As a result, research into how individuals learn about aversive events 43 holds great promise for enhancing our understanding across a diverse range of mental health 44 problems. 45 Computational approaches are a powerful means to characterise the precise mechanisms 46 underpinning learning, as well as uncovering how these relate to psychiatric symptom information, updating faster in response to negative than positive outcomes (12), a bias that 55 might engender a negative view of the environment. In healthy individuals, this bias is 56 associated with trait optimism (15). However, it is unclear to extent such biased learning relates 57 to the spectrum of mental health problems. 58 One process implicated in the genesis of psychiatric disorders is that of uncertainty estimation. 59 Uncertainty plays a fundamental role in learning, and computational formulations optimise 60 learning in the face of non-stationary probabilistic outcomes based on uncertainty (11,(16)(17)(18)(19). 61 While psychiatric traits, including anxiety, are linked to an inability to adapt learning in response 62 to environmental statistics such as volatility (5,9), little research has investigated whether how 63 individuals estimate, or respond to, uncertainty in aversive environments is associated with 64 psychiatric traits. This is a critical question given that core features of anxiety revolve around a 65 concept of uncertainty; individuals with anxiety disorders report feeling more uncertain about 66 threat and being less comfortable in situations involving uncertainty (20-23). 67 Existing work on aversive learning has had a particular focus on symptoms of anxiety and 68 depression (7, 12). However, these approaches have not been designed optimally for 69 identifying mechanisms that span traditional diagnostic boundaries. This assumes importance 70 in light of recent studies, using large samples, that show several aspects of learning and 71 decision making relate more strongly to transdiagnostic factors than to any specific categorical 72 psychiatric disorder (6,8,24,25). Applying such an approach to aversive learning may yield 73 better insights into the role of learning in psychiatric disorders.

74
Here, we aimed to identify specific aversive learning processes that relate to both traditional 75 measures of anxiety, and transdiagnostic psychiatric traits, in a large sample collected online. 76 Specifically, we used a computational approach to test whether anxiety and transdiagnostic 77 symptoms are associated with biased learning from safety and threat, whether these traits 78 relate to altered estimates of threat likelihood, and whether they are associated with different 79 levels of uncertainty during threat learning. Given difficulties in using traditional aversive stimuli 80 in an online setting, we developed a novel game-based avoidance task designed to engage 81 threat and avoidance processes without the need for administration of painful or noxious We recruited 400 participants through Prolific (26). Subjects were selected based on being 90 aged 18-65 and having at least a 90% approval rate across studies they had previously 91 participated in. spaceship through asteroid belts. Each asteroid belt featured two locations that could potentially contain escape 95 holes (safety zones), and subjects were instructed to aim to fly their spaceship through these to gain the highest 96 number of points. Subjects were only able to move the spaceship in the Y-dimension, while asteroid belts moved 97 towards the spaceship. The probability of each zone being safe varied over the course of the task but this could be 98 learned, and learning this probability facilitated performance. B) Screenshot of the task, showing the spaceship, an 99 asteroid belt with a hole in the lower safety zone (safety zone B), a representation of the spaceship's integrity (shown 100 by the coloured bar in the top left corner) and the current score. 101 Avoidance learning task 102 Traditional lab-based threat learning tasks typically use aversive stimuli such as electric shocks 103 as outcomes to be avoided. As it is not possible to use these stimuli online, we developed a 104 game-based task in which subjects' goal was to avoid negative outcomes. In this game, 105 participants were tasked with flying a spaceship through asteroid belts. Subjects were able to 106 move the spaceship in the Y-axis alone, and this resulted in a one dimensional behavioural 107 output. Crashing into asteroids diminished the spaceship's integrity, and if enough asteroids 108 were hit the game finished. In this eventuality subjects were able to restart and continue where 109 they left off. The overarching goal was to maximise the number of points scored, where the 110 latter accumulated continuously for as long as the game was ongoing, and reset if the 111 spaceship was destroyed. Subjects were shown the current integrity of the spaceship by a bar 112 displayed in the corner of the screen, along with by a display of their current score. No actual 113 monetary reward was given to the subjects for performance on the task.  Figure 1A), and the probability of safety associated with either location varied independently 120 over the course of the task. Thus, it was possible to learn the safety probability associated with 121 each safety zone and adapt one's behaviour accordingly. Participants also completed a control 122 task that required avoidance that was not dependent on learning, enabling us to control for 123 general motor-related avoidance ability in further analyses (described in supplementary 124 material). We elected a priori to exclude subjects with limited response variability (indicated by 125 a standard deviation of their positions below 0.05) so as to remove subjects who did not move 126 the spaceship. However, no subject met this exclusion criterion.

127
After completing the task, subjects were asked to provide ratings indicating how anxious the 128 task made them feel and how motivated they were to avoid the asteroids, using visual analogue 129 scales ranging from 0 to 100. Our modelling approach focused on models that allowed the quantification of subjective 137 uncertainty. To this end, we modelled behaviour using approximate Bayesian models that 138 assume subjects estimate safety probability using a beta distribution. This approach is naturally 139 suited to probability estimation tasks, as the beta distribution is bounded between zero and 140 one, and provides a measure of uncertainty through the variance of the distribution. While  The basic premise underlying these models is that evidence for a given outcome is dependent 148 on the number of times this outcome has occurred previously. For example, evidence for safety 149 in a given location should then be highest when safety has been encountered many times in 150 this location. This count can be represented by a parameter A, which is then incremented by a   This formed the basis of all the probabilistic models tested: We also observed in pilot data that subjects tended to be influenced more by outcomes 173 occurring in the zone they had previously chosen, an effect likely due to attention. On this basis, 174 we incorporated a weighting parameter that allowed the outcome of the unchosen option to 175 be down-weighted by an amount shown in the above equation (W) determined by an 176 additional free parameter, ω.
We can calculate the estimated safety probability for each zone (P) by taking the mean of this 179 distribution: Similarly, we can derive a measure of uncertainty on each trial by taking the variance of this 182 distribution.
Our best fitting model included a "stickiness" parameter, which caused the chosen safety zone For completeness, we also tested two reinforcement learning models, a Rescorla-Wagner  Crucially, the fact that task outcomes were identical for every subject ensured these values 234 were dependent only on the manner by which subjects learned about safety, not the task itself.

284
Due to an interest in asymmetric learning about safety and punishment, the two parameters of 285 the preferred model with greatest relevance to our hypotheses were update rates for safety 286 and danger. To verify that the estimated parameters truly reflected behavioural tendencies, we 287 examined correlations between their estimated values and the tendency to stay following 288 safety and move following danger across subjects ( Figure 2D). These were robustly correlated, 289 providing confidence that these model-derived parameters related to relevant behavioural 290 measures. Interestingly, we observed a negative bias in the values of these parameters 291 whereby subjects tended to update to a greater extent in response to danger than safety 292 (t(400) = 26.76, p < .001, Figure 2E). (represented by the grey and red lines) across the duration of the task, generated by simulating data from the model. 303 D) Correlations between estimated update parameters for danger (left) and safety (right) and our behavioural 304 measure of position switching after these outcomes across subjects, demonstrating that parameters from our model 305 reflect purely behavioural characteristics. E) Distributions of estimated parameter values for τ + and τ -, representing 306 update rates following danger and safety outcomes respectively, showing a bias in updating whereby subjects 307 update to a greater extent in response to danger than safety. 308 Relationships with psychiatric measures 309 First, as a validation of our task, we tested whether task-induced anxiety was predicted by Highest posterior density estimate 318 We next asked whether our four behavioural variables of interest (threat and danger update 319 parameters, mean estimated safety probability, and mean estimated uncertainty) were 320 associated with anxiety (both state and trait) and intolerance of uncertainty respectively.

321
Contrary to our hypotheses, the only relationships with HPD intervals that did not include zero 322 were positive effects of state anxiety on safety update rates and mean estimated safety 323 probability ( Figure 3E, Table 1), indicating more anxious individuals learned faster about safety 324 and perceived safety as more likely overall. 325 We then tested whether behaviour in the task was associated with three transdiagnostic 326 factors identified in previous research (6). Here, we observed unexpected relationships with a 327 factor labelled compulsivity and intrusive thought ( Figure 3F, Table 2), reflecting the fact that 328 subjects scoring higher on this factor learned faster about safety and had higher safety 329 probability estimates. No other effects had HPDIs that excluded zero.  Perceptions of danger and safety have been linked to key symptoms of psychiatric disorders. 374 Here, we use computational modelling to show that when subjects learn to avoiding threat, 375 transdiagnostic components of psychopathology relate to how they learn about both safety 376 likelihood, and uncertainty. 377 We found an unexpected relationship between biases in learning and anxiety. Contrary to our 378 a priori hypotheses, subjects scoring higher on state anxiety tended to update their predictions 379 to a greater extent in response to safety, as well as perceiving safety to be more likely overall, 380 than those scoring low on this measure. These results diverge from previous findings that 381 reported individuals diagnosed with clinical anxiety and depression learn faster from 382 punishment (12). One potential explanation for this discrepancy is that the aforementioned 383 work included subjects with clinical diagnoses, involving a mix of anxiety and depressive 384 disorders. When we examined transdiagnostic factors, we found that symptoms of depression 385 were associated with elevated learning from threat, suggesting that such a bias in learning is 386 associated with depressive symptoms. It is also possible that the nature of our game-based  We found a similar pattern of enhanced learning from threat when examining a transdiagnostic 392 factor representing compulsivity and intrusive thought. Although this factor has been shown 393 to be associated with more model-free behaviour (6, 24), altered confidence judgements (25), 394 and action-confidence coupling (8) in large-scale samples, to date it has not been investigated 395 with regard to threat learning. Notably, we found a weak relationship between this factor and 396 uncertainty, whereby more compulsive individuals had higher certainty in their safety 397 estimates, echoing previous work in perceptual decision making that showed this factor is 398 associated with higher confidence estimates (8, 25). We only found weak relationships with 399 the other two factors, representing anxious-depression and social withdrawal, in a direction 400 indicative of lower safety probability estimates and higher uncertainty. categories or in factors identified using self-report measures alone, are associated with distinct 408 patterns of learning behaviour. This data-driven analysis also revealed relationships between 409 aversive learning and impulsive behaviour, encompassing a symptom dimension that is 410 typically studied in the context of reward processing (34). Individuals scoring higher on these 411 symptoms exhibited higher safety learning, which may explain previously observed 412 relationships between impulsivity and risk tolerance (35).

413
A feature of this study is our development of a novel online task for measuring aversive learning.

414
A number of studies examining other aspects of learning and decision making in the context of 415 psychiatric disorders have also availed of large samples recruited through online services (6, 8, 416 24, 25). However, it has been difficult to examine aversive learning in online environments, as 417 aversive lab stimuli such as shock cannot be easily administered online. A game-based design 418 allowed us to design a task that required avoidance behaviour as well as evoke feelings of 419 anxiety. Although qualitatively different from standard lab-based tasks, we observed similar 420 patterns of biased learning to that seen in lab-based work (36). An added benefit of our task is 421 that it is highly engaging, and subjects reported feeling motivated to perform well.

422
One potential limitation of this study is a focus on a general population sample. While this might