Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

# Ageing is associated with disrupted reinforcement learning whilst learning to help others is preserved

## Abstract

Reinforcement learning is a fundamental mechanism displayed by many species. However, adaptive behaviour depends not only on learning about actions and outcomes that affect ourselves, but also those that affect others. Using computational reinforcement learning models, we tested whether young (age 18–36) and older (age 60–80, total n = 152) adults learn to gain rewards for themselves, another person (prosocial), or neither individual (control). Detailed model comparison showed that a model with separate learning rates for each recipient best explained behaviour. Young adults learned faster when their actions benefitted themselves, compared to others. Compared to young adults, older adults showed reduced self-relevant learning rates but preserved prosocial learning. Moreover, levels of subclinical self-reported psychopathic traits (including lack of concern for others) were lower in older adults and the core affective-interpersonal component of this measure negatively correlated with prosocial learning. These findings suggest learning to benefit others is preserved across the lifespan with implications for reinforcement learning and theories of healthy ageing.

## Introduction

Learning associations between actions and their outcomes is fundamental for adaptive behaviour. To date, the majority of studies examining reinforcement learning have tested how we learn associations between actions and outcomes that affect ourselves, and largely focused on these processes at a young age, both in humans and other species1,2,3,4,5. However, such self-relevant learning may be computationally separable from learning about actions that help other people. Studies suggest slower learning of associations between actions and outcomes when they are about6 or affect others7, henceforth referred to as ‘prosocial learning’.

Senescence is associated with a multitude of changes including declines in cognitive functioning and perception, but perhaps the preservation of affective processing and social cognitive abilities8,9,10. However, less is known about how ageing affects social behaviour, despite the critical importance of this question. Social isolation has been found to be as damaging to physical health as smoking or excessive drinking11. Social behaviours that benefit others—prosocial behaviours—are vital for maintaining social bonds and relationships12 across the lifespan. In addition to the benefits for others, prosociality has been linked to improved life satisfaction13, mental wellbeing14 and physical health15 for the person being prosocial, all of which could contribute to healthy ageing. A key aspect of prosocial behaviour is the ability to learn associations between our own actions and outcomes for other people7. Here, we use computational models of reinforcement learning in young and older participants to examine the mechanisms that underpin self-relevant and prosocial learning and associations with healthy individual differences in socio-cognitive ability.

Reinforcement Learning Theory (RLT) provides a powerful framework for understanding and precisely modelling learning16. In RLT, prediction errors signal the unexpectedness of outcomes and affect the choices we make in the future. The influence that prediction errors have on choices can be modelled individually through the learning rate, which quantifies the effect of past outcomes on subsequent behaviour. The plausibility of reinforcement learning as a core biological mechanism for action–outcome associations is bolstered by our understanding of neurobiology, with prediction errors encoded by single neurons in the ventral tegmental area17. Although essential for successful adaptive behaviour, several studies suggest that our propensity for reinforcement learning declines in later life10. Compared to younger adults, older adults show learning impairments particularly when action–outcome associations are probabilistic18 or reverse19. Age-related declines in learning ability have been linked to functional and structural changes in frontostriatal circuits20,21 and dopamine transmission18,22, which shows a significant age-related decrease23,24,25 and has a key role in coding prediction errors2,26,27. Indeed, one study showed that administering L-DOPA, a dopamine precursor, to older adults increased their learning rate28. Therefore, if reinforcement learning in general declines in older age, we would hypothesise lower learning rates for both self-relevant and prosocial learning in older, compared to younger, adults.

Alternatively, prosocial learning may depend not only on our learning ability but also our motivation to help others29. Results from experiments using economic games to measure prosociality have found that older adults tend to be more generous30,31. There is also evidence of an age-related increase in charitable donations to individuals in need32. At work, older adults engage in more prosocial behaviours than younger adults, according to both self-report data and colleagues’ ratings33. Finally, self-reported altruism and decisions to donate to others have been shown to increase with age34. However, one limitation of these studies is that the paradigms often place self and other reward preferences in conflict. Money for the other person depends on less money for oneself. Moreover, older adults generally have higher accumulated wealth, which would be an important confound in studies of monetary exchange35. Prosocial learning avoids this confound by separating outcomes for oneself from outcomes for others. If older adults do indeed value outcomes for others more than young adults, we might expect that whilst self-relevant learning declines with ageing, prosocial learning could be preserved. Comparing young and older adults on self-relevant and prosocial learning provides an opportunity to dissociate how possible age-related changes in cognitive ability and social behaviour impact on learning.

While studies point to potential group differences between young and older adults, there is also substantial individual variability in concern for others, which can be captured by trait measures. Decreased prosocial behaviour and altered self and other reward processing are key features of psychopathic traits36,37. Psychopathic traits are characterised by dysfunctional affective-interpersonal features38,39 and also include an antisocial behavioural dimension40. Levels of these traits range from clinically defined psychopathy, a severe personality condition linked to poor life outcomes, violence and criminality41,42,43, to lower subclinical levels of psychopathic traits in the general population. This conceptualisation of psychopathy as dimensional, rather than categorical, reflects the Research Domains of Criteria (RDoC) approach to psychiatry44. Evidence of similar behavioural and neural profiles between community samples with high levels of psychopathic traits and those with clinical diagnoses of psychopathy45 is consistent with this RDoC approach. Self-report measures of psychopathic traits mirror the latent structure of clinical psychopathy measures as comprising socio-emotional and behavioural dimensions but are distinct from a clinical diagnosis of psychopathy. These subclinical self-report measures, such as the Self-Report Psychopathy Scale (SRP)40 used here, can be administered in behavioural studies to index the range of scores found in the general population. The SRP captures psychopathic traits on two dimensions labelled: ‘affective-interpersonal’ (lack of empathy and guilt) and ‘lifestyle-antisocial’ (impulsive and antisocial behaviours)40.

Intriguingly, preliminary evidence suggests that ageing may also be associated with changes in self-reported psychopathic traits46, which could have important implications for our understanding of an ageing population. In community samples, ageing is associated with a decrease in the socio-emotional and behavioural dimensions of the SRP47. These studies highlight the importance of assessing how differences in self-reported psychopathic traits could map on to differences in prosocial behaviours.

Taken together, previous research supports opposing hypotheses for how ageing is associated with self-relevant and prosocial reinforcement learning. On the one hand, evidence suggests that older adults should be impaired at learning, regardless of the recipient, consistent with ageing-related declines in learning ability and dopamine transmission. On the other hand, potential increases in valuing outcomes for others in older, compared to younger, adults would predict preserved prosocial learning ability but reduced self-relevant learning ability. Finally, we expected variation in self-reported psychopathic traits to be associated with learning for others but not self in both age groups.

To distinguish between these competing hypotheses, we tested 75 young (aged 18–36, mean = 23.07, 44 females) and 77 older (aged 60–80, mean = 69.84, 40 females) adults carefully matched on gender, years of education and IQ test performance. Participants completed a probabilistic reinforcement-learning task (Fig. 1) designed to separate self-relevant (rewards for self) from prosocial learning (rewards for another person), as well as controlling for the general valence of receiving positive outcomes (rewards for neither self nor other).

The detailed model comparison showed that a computational model with separate learning rates best explained how people learn associations for different recipients (Fig. 2). Young adults were faster to learn when their actions benefitted themselves, compared to when they helped others. Strikingly, however, this was not the case for older adults, who showed a relative increase in the rate at which they learnt about actions that helped others, compared to themselves (Fig. 3a, b). Older adults had significantly reduced levels of self-reported psychopathic traits (mean = 21.09) compared to younger adults (mean = 24.36) and in older adults, lower self-reported psychopathic trait scores correlated with prosocial learning rates (Fig. 4a, b). These effects were not explained by individual differences in IQ test performance, memory or attention abilities. Overall, we show that despite age-related disruption to self-relevant learning, prosocial learning was similar between young and older adults. Ageing was also associated with a decline in self-reported psychopathic traits. These findings suggest learning how our actions help others is preserved across the lifespan.

## Results

We analysed the behaviour of 75 younger adults and 77 older adults who completed the probabilistic reinforcement-learning task (Fig. 1b), neuropsychological tests, and a self-report measure of psychopathic traits (see ‘Methods'). To ensure comparability, older adults with dementia, as diagnosed by Addenbrooke’s Cognitive Examination (ACE)48, were not included in the study. The two age groups were matched on gender (χ2(1) = 0.45, P = 0.50) and did not differ in years of education or IQ test performance (Supplementary Table 1). IQ test performance was quantified using age-standardised scores on the Wechsler Test of Adult Reading (WTAR)49. We conducted additional analyses controlling for IQ test performance (standardised WTAR score, measured for young and older adults), and memory and attention (memory and attention subscales of the ACE, older adults only). These control analyses showed that our results are not accounted for by IQ test performance or executive function (see ‘Methods' and Supplementary Information).

### Learning occurs for all recipients for both age groups

We first examined whether participants were able to learn for all three recipients to validate their ability to complete the task. We quantified performance as selecting the option associated with a high chance of receiving the reward. Participants in both age groups were able to learn to obtain rewards for themselves, another person and no one. This was demonstrated through average performance above chance level (50%; all ts > 15.46, all Ps < 0.001) and a significant effect of trial number in predicting trial-by-trial performance (all Zs > 4.48, Ps < 0.001) for each separate recipient and age-group combination.

### Learning rate depends on who receives the reward

Next, to quantify learning, we used computational models of reinforcement learning to estimate learning rates (α) and temperature parameters (β), key indices for the speed by which people update their estimates of reward, and the precision with which they make choices, respectively. Models were fitted using a hierarchical approach and compared using Bayesian methods50,51, with model selection based on the exceedance probability and integrated Bayesian Information Criterion (BIC; see ‘Methods'). We tested multiple models that varied with respect to whether learning could be explained by shared or separate free parameters across recipients (self, other no one). Based on our previous results7, we examined whether shared or separate learning rates, in particular, resulted in a better model fit. We used four candidate models:

(i) 1α1β: one α for all three recipients and one β for all three recipients

(ii) 3α1β: αself, αother & αno one, one β

(iii) 2α1β: αself & αnot-self [other + no one], one β

(iv) 3α3β: αself, αother, αno one, βself, βother & βno one (see Supplementary Table 2)

Initially, we aimed to establish that both our experimental schedule and our models were constructed in a way that allowed us to disentangle recipient-specific learning rates. To this end, we created synthetic choices using simulations based on each of our four models (see ‘Methods'). We fitted the models to the data and assured that the best-fitting model was the one that had been used to create the data. In such a way, we established model identifiability, both when considering the exceedance probability (Fig. 2a and see ‘Methods') and the number of times a model was identified as the best one (Fig. 2b). As a second prerequisite for testing for agent-specific learning rates, we performed parameter recovery using our key model of interest, the 3α1β model. Over a wide parameter space, we were able to recover the parameters underlying our choice simulation (Fig. 2c).

Having established the models were identifiable and parameters recoverable, we performed Bayesian model selection on the data from our participants. Participant’s choices were best characterised by the 3α1β model. This indicated that the learning process underlying the choices is most accurately captured by assuming separate learning rates for each recipient (αself, αother and αno one). This model fit the data best (exceedance probability = 97%; ΔBICint = 122; Fig. 2d) and predicted choices well (R2 = 51%; see ‘Methods' for further details).

### Higher learning rates are associated with better performance

As a final check of the robustness of our model and to enable clear interpretation of any differences in learning rate between recipients and age groups (see ref. 52), we conducted an additional simulation experiment. We simulated data from 10,000 participants using the 3α1β model. This created 30,000 values of α from the three recipient conditions, spanning the full range of possible values from 0 to 1 (see ‘Methods'). For each, we quantified the associated performance as the percentage of times the synthetic participant chose the high probability of reward option, averaged across the blocks for the relevant recipient. Plotting the learning rates against performance (Fig. 2e) shows that the optimal value of α is ~0.55. This is higher than all the values of α found on our task in any recipient condition for either age group. Therefore, higher learning rates were associated with better performance. We further established this link by correlating learning rates and performance in the empirical data from our participants. We found a strong correlation between learning rates and performance overall (rs(150) [95% confidence interval] = 0.58 [0.46, 0.68], P < 0.001; Fig. 2f) and in each recipient and age-group combination (Supplementary Table 3).

### Older adults are slower at learning for themselves, but prosocial learning is preserved

Next, we used this validated computational model to test our hypotheses as to whether there were group differences in learning rates when learning to reward self, other or no one. Two participants had learning rates for two of the three recipients more than three standard deviations (SDs) above the mean (αself 6.68 and αno one 9.64; αself 7.96 and αother 3.78 SDs above the mean) and were excluded from all analysis of learning rates. We analysed the condition-specific learning rates from our best-fitting computational model using a robust linear mixed-effects model (RLMM; see ‘Methods'). The RLMM fixed effects were age group (young, older), recipient (self, other and no one), as well as the age group * recipient interaction. While this RLMM includes all three recipient conditions, it generates coefficients (main effect and the interaction with age group) contrasting pairs of recipient conditions—[self vs. other] and [self vs. no one]. These are more interpretable than an omnibus test that would not show which recipient conditions were driving an effect or interaction. We followed up these results with planned comparisons, between the older and younger group in each recipient condition, and between pairs of recipient conditions within each age group.

Across age groups, participants showed a higher learning rate when rewards were for themselves, compared to for another person (recipient [self vs. other]: b = −0.02 [−0.03, −0.01], Z = −4.79, P < 0.001). Importantly, however, this pattern differed between age groups. The difference between self-relevant and prosocial learning rates was reduced in older compared to younger adults (recipient [self vs. other] * age-group interaction: b = 0.02 [0.002, 0.03], Z = 2.29, P = 0.02). Between-group comparisons showed older adults learnt more slowly for themselves compared to younger adults (W = 3512, Z = −2.63, r(150) = 0.22 [0.06, 0.36], P = 0.009). However, prosocial learning was preserved, with a Bayes factor suggesting strong evidence of no difference in αother between young and older adults (W = 3042, Z = −0.86, r(150) = 0.07 [0.003, 0.24], P = 0.39, BF01 = 4.26). Within-subject comparisons of αself and αother in each age group showed that young adults had higher learning rates for themselves, relative to another person (V = 659, Z = −4.04, r(75) = 0.47 [0.26, 0.63], P < 0.001). In contrast, older adults showed no significant difference between learning rates for self and other (V = 1150, Z = −1.45, r(75) = 0.17 [0.01, 0.38], P = 0.15, BF01 = 1.08).

As expected, across age groups learning was slower for no one than self (recipient [self vs. no one]: b = −0.02 [−0.03, −0.01], Z = −4.57, P < 0.001). Unlike αself vs. αother, learning for self compared to no one did not interact with age group (recipient [self vs. no one] * age-group interaction: b = 0.008 [−0.006, 0.02], Z = 1.15, P = 0.25). Within-subject comparisons between αself and αno one in each age group showed that both groups learnt preferentially for themselves compared to no one (young adults: V = 928, Z = −2.62, r(75) = 0.30 [0.07, 0.51], P = 0.009; older adults V = 901, Z = −2.76, r(75) = 0.32 [0.09, 0.53], P = 0.006). There was no significant difference between the age groups in αno one but also no evidence in support of the null (W = 3241, Z = −1.61, r(150) = 0.13 [0.01, 0.29], P = 0.11, BF01 = 2.04).

Considering differences in learning between αother and αno one, young adults did not differentiate between another person and no one, with strong Bayesian evidence for no difference (V = 1533, Z = −0.57, r(75) = 0.07 [0.003, 0.31], P = 0.57, BF01 = 5.08). In contrast, older adults had higher learning rates for another person, compared to no one (V = 976, Z = −2.37, r(75) = 0.27 [0.05, 0.49], P = 0.02). Crucially, this shows that older adults’ lack of differentiation between self and other was not simply because they were insensitive to the recipient condition.

Finally, we also observed an effect of age on both learning rates overall and temperature parameters. Older adults showed slower learning overall compared to younger adults (b = −0.02 [−0.03, −0.01], Z = −3.73, P < 0.001) and higher levels of exploration of choice options (median β young: 0.05, older: 0.19, W = 1511, Z = −4.89, r(150) = 0.40 [0.26, 0.53], P < 0.001; Supplementary Fig. 1).

In summary, older adults prosocial learning was preserved at the same rate as young adults, despite age-related declines in self-relevant learning rates. In other words, young adults learned faster for themselves than others, but older adults distinguished between themselves and others significantly less than the young participants. Only older adults, not young adults, distinguished between rewards for another person and no one.

### Participants perform better for themselves, compared to no one

For completeness, we also tested the effects of recipient and age group on trial-by-trial tendency to pick the high-reward stimuli (Fig. 3c). In addition to the main effect of trial number (b = 1.70 [1.29, 2.13], Z = 7.97, P < 0.001), showing learning, these models revealed older adults chose the high-reward option less frequently (mean for young: 80%, older: 64%, b = −1.20 [−1.65, −0.70], Z = −4.84, P < 0.001), and improved less during the task (trial number * age-group interaction b = −0.81 [−1.34, −0.27], Z = −2.95, P = 0.003) across recipients. When averaging across age groups, performance was better for the self (75%), compared to no one (70%; b = −0.36 [−0.68, −0.05], Z = −2.28, P = 0.02). However, there was not a significant difference between accuracy for other (72%) and self (b = −0.22 [−0.49, 0.05], Z = −1.63, P = 0.10), or any significant interactions between the age group and the recipient (bs < 0.16, Zs < 0.96, Ps > 0.34).

### Self-reported psychopathic traits are lower and associated with prosocial learning in older adults

Finally, we examined individual variability in self-reported psychopathic traits, considering age-related differences and influence on prosocial learning. Several studies have suggested that individual differences in subclinical self-reported psychopathic traits can be meaningfully and accurately captured in community samples and often parallel findings in criminal offenders45. Critically, self-reported psychopathic traits are closely linked to alterations in social behaviour and willingness to help others. Therefore, we also asked participants to complete the Self-Report Psychopathy Scale (SRP-IV-SF)40. The SRP is a self-report measure of psychopathic traits in community samples that assesses traits with the same factor structure as in measures used for clinical psychopathies, such as antisociality and interpersonal effect. The measure robustly captures the latent structure of clinical psychopathy to ensure parallels can be drawn between normal individual differences in the community and clinical samples (see ‘Methods'). One participant in each age group had missing questionnaire data and are not included in these analyses. Self-reported psychopathic traits are consistently divided into two components that this scale measures: core ‘affective-interpersonal’ traits, which capture lack of empathy and guilt; and ‘lifestyle-antisocial’ traits, which capture impulsivity and antisocial tendencies. Comparing the two age groups on these scales showed that older participants had significantly lower scores than young participants on both the core affective-interpersonal (young mean = 24.36, older mean = 21.09, W = 3558, Z = −3.15, r(148) = 0.26 [0.11, 0.40], P = 0.002) and the lifestyle-antisocial subscales (young mean = 22.89, older mean = 20.27, W = 3471, Z = −2.82, r(148) = 0.23 [0.09, 0.38], P = 0.005). These findings suggest that both components of self-reported psychopathic traits were reduced in older, compared to younger, adults.

Next, we sought to test our hypothesis that individual differences in self-reported core psychopathic traits would explain variability in learning rates, specifically for prosocial learning. We observed a significant negative relationship between αother and self-reported core psychopathic traits among older participants (﻿rs(74) = −0.33 [−0.52, −0.11], P = 0.005; Fig. 4a). Intriguingly, this relationship was significantly more negative (Z = 3.28, P = 0.001) than the equivalent correlation in young adults, which was not significant (﻿rs(74) = 0.21 [−0.02, 0.42], P = 0.07; Fig. 4b). This pattern of results was the same when correlating relative prosocial learning rate (αother − αself) with self-reported psychopathic traits. We also conducted control analyses, correlating the same pairs of variables but using partial correlations controlling for β. The negative relationship between prosocial learning (when quantified as αother or αother − αself) and self-reported psychopathic traits was still present for older adults, showing that the correlations with learning rates were independent of individual choice exploration (all Ps < 0.05; see Supplementary Table 4). The negative relationship between αother and self-reported psychopathic traits for older adults also remained significant after applying false discovery rate (FDR) correction for multiple comparisons across this correlation and the five other age group and recipient combinations (see Supplementary Table 5). Moreover, the finding that no significant correlations were apparent between self-reported psychopathic traits and αself or αno one (Ps > 0.15; Supplementary Table 5), suggests a specificity in the relevance of psychopathic traits to prosocial learning in older adults.

Given the age-group differences in levels of self-reported psychopathic traits and their correlations with prosocial learning, our final analysis considered possible indirect effects of age group on relative prosocial learning rate (αother − αself), through scores on the core affective-interpersonal subscale of the psychopathic traits measure. A standard mediation model (Fig. 5a) did not show evidence for a significant indirect effect. However, as would be predicted if the link between self-reported psychopathic traits and prosocial learning depends on age, there was a significant indirect effect in a moderated mediation model. This revealed an indirect effect of age group on relative prosocial learning rates via self-reported psychopathic traits for older adults (unstandardised indirect effect = 0.006 [0.001, 0.01], P = 0.006, proportion mediated = 0.30) but not for young adults (unstandardised indirect effect = −0.001 [−0.007, 0.006], P = 0.67; see Fig. 5b for standardised coefficients). In summary, young and older adults differed in levels of self-reported psychopathic traits and whether or not self-reported psychopathic trait scores were associated with their prosocial learning rates.

## Discussion

Reinforcement learning is a fundamental process for adaptive behaviour in many species. However, existing studies have largely focused on young people and self-relevant learning, but the decisions we make often occur in a social context53 and our actions affect outcomes for others. Here, we applied computational models of reinforcement learning to the question of ageing-related changes in self-relevant and prosocial learning. We found a clear decrease in learning rates for self-relevant rewards in older, compared to younger, adults. Intriguingly, despite this reduction in self-relevant learning, learning rates for outcomes that affected others did not differ between older and young adults, with Bayesian analyses supporting no difference. Moreover, not only did older adults have preserved prosocial learning rates, but age was also associated with lower self-reported psychopathic traits, which were specifically linked to prosocial learning in older adults.

Models of learning are a powerful tool for understanding prosocial behaviour. By isolating the learning rate, we can precisely examine the influence of reward history on learning. We robustly replicated previous findings that self-relevant learning can be computationally separated from prosocial learning7, with different learning rates for different recipients providing the best explanation of behaviour. Including separate learning rates improved the model fit and, on average across participants, learning rates were higher for self-relevant rewards, compared to when someone else received the reward. However, this increase in learning rates for self, compared to other, was reduced in older adults who showed higher prosocial learning rates, relative to their own self-relevant learning rates, than young adults. As expected, older adults learnt more slowly for themselves than young adults but in the prosocial condition, the learning rates did not significantly differ between the age groups. Bayesian analyses additionally confirmed that prosocial learning was preserved between young and older adults.

As with much research on age-related changes in cognitive and social tasks, our key finding that older adults are relatively more prosocial on a learning task could be interpreted as due to changes in ability, motivation, or a combination of these factors. Importantly, learning rates were not associated with executive function or an age-standardised measure of intelligence. We also show that our results remain the same after controlling for these measures. These findings suggest that the observed decline in self-relevant learning rates, but the preservation of prosocial learning rates, for older adults are not explained by changes in these broad abilities. Considering learning more specifically, a recent comparison of motivation and ability as explanations for ageing-related reductions in model-based strategies during self-relevant learning supported the limited cognitive abilities account54. Our finding that learning rates for self-relevant outcomes were reduced in older adults is in line with alterations in the neurocognitive systems required for successful learning. Research combining models of learning with neuroimaging and pharmacological manipulations suggests ageing reduces the ability to generate reward prediction errors18 (but see refs. 55,56) due to declines in dopamine functioning28,57 (also see refs. 58,59). Differences in motivation could also be applicable for self-relevant learning as the subjective value of financial outcomes is also likely to decrease in older age, due to changes in wealth across the lifespan35.

Our findings suggest that despite declines in learning ability associated with ageing, prosocial learning—learning to help others—is preserved. This finding aligns with emerging literature showing older adults may be more prosocial than younger adults30,31,60,61. The possibility that relatively preserved prosocial learning is related to increased prosocial motivation is in line with our observed link between learning rates and psychopathic traits. Self-reported psychopathic traits were significantly reduced in our older adult sample, dovetailing with similar previous findings on this trait46,47 and broader trait benevolence34. Importantly, we found self-reported psychopathic traits in older adults negatively correlated with prosocial learning rates. The tendency to learn faster for oneself compared to another person was most reduced, and even reversed, in the older people lowest on psychopathic traits. Notably, this negative correlation between self-reported psychopathic traits and prosocial learning was only found for older adults. This suggests that age-related differences in prosocial learning could be linked to basic shifts in individual traits and motivations over the lifespan, not just to domain-general reductions in cognitive abilities. The idea that social motivations become more influential in learning and decision-making with age has also been suggested based on studies of social rewards such as smiling faces or hypothetical time spent with social partners62,63,64. Taken together, this work suggests that strategies to support healthy ageing might benefit from leveraging potentially preserved social motivations, a hypothesis future research could address.

Importantly, our task included a control condition where no one benefitted. This was essential to establish that the lack of difference between self and other learning rates for older adults was not simply due to an age-related reduction in the absolute dynamic range as maximum learning rates decrease. Older adults had higher learning rates for both others and themselves, compared to this control condition. In contrast, young adults did not differentiate another person from no one. Older adults, therefore, showed a relative increase in learning rates that was specific to the prosocial condition. This is also evidence against the idea that lower learning rates in older people are reflective of a general reduced sensitivity to who gets the reward. It is interesting to note that the magnitude of the decrease in self-relevant learning rates associated with being older, compared to younger, is similar to the decrease shown by the young participants when learning for someone else, compared to themselves. The preservation of prosocial learning rates between age groups may seem at odds with the decreased self-relevant learning rates in our sample of older adults and existing evidence of underlying neurobiological changes. However, learning from outcomes for self and other have been linked to distinct regions of the brain, shown through human neuroimaging7,65,66, and causally in monkeys with focal lesions67.

Taken together, our results add to a growing body of literature suggesting age-related increases in prosocial motivation30,31,60,61. If this is the case, the next question is how and why this happens, as there are many possible reasons to be prosocial. For example, prosocial behaviours can be motivated by reputational concerns68, the ‘warm glow’ of helping69,70,71, or vicarious reinforcement from positive outcomes for others72. In our procedure, we were very careful to prevent reputational concerns influencing learning to help others. Participants underwent an extensive procedure to introduce them to another participant but to hide information about their age and identity. This meant we could assess the tendency for prosocial learning in a situation where reputational motivations were excluded, and identity-based influences were controlled. Using a reinforcement-learning task, in which performance generates positive outcomes for others, also focuses on vicarious rewards from the outcome, rather than the warm glow associated with the action of helping. Thus, the increase in prosocial learning rates, relative to self-relevant learning rates, suggests older adults are reinforced by outcomes for others and themselves more similarly than younger adults. Many prosocial measures such as the dictator game73 are costly, requiring direct trade-offs between outcomes for oneself and others. This also makes it hard to detect whether changes are in the value of outcomes for oneself, or outcomes for others, or both. In contrast, separating self-relevant learning, prosocial learning, and the control condition allows us to differentiate increases in the value of prosocial outcomes from decreases in the value of outcomes for the self. Our results are consistent with older adults having both decreased self-relevant learning rates (compared to young adults) and increased prosocial learning rates (compared to their performance in the control condition).

Our results also support the advantages of a model-based approach for understanding both prosocial behaviour and ageing. The model-based parameters were more sensitive to the effects of interest than general measures of performance on the task. The model comparison process is able to provide important information about how the learning process takes place, which cannot be revealed from performance measures alone. We showed that learning was best represented by a separate learning rate for each recipient. Moreover, this approach can capture additional latent parameters that drive behaviour, such as the inverse temperature, which indexes how closely participants follow the stimulus value. We demonstrate that a single inverse-temperature parameter best explained behaviour during learning across recipient conditions, despite learning rates being distinct.

While our procedure and task have many benefits, it is important to also recognise limitations. To test for age-related differences in prosocial learning, we recruited a group of older adults and a group of younger adults. This increases the power to detect differences, but we are unable to assess at what age or how quickly changes take place. Also, while our age groups were matched on years of education and IQ test performance, the recruitment from university databases or issues around self-selection may mean that the levels of education and IQ test performance in our sample are not completely representative of the general population. Further studies could include samples recruited entirely from the community and participants across the whole adult lifespan. Future studies should also assess the timescale after which older participants reach similar ceiling levels of performance as younger adults, or whether they are never able to reach the same level of performance. Of similar importance is the question of whether older adults may be able to sustain higher levels of motivation over an extended period of time compared to younger adults, possibly compensating for slower learning speeds. In this study, due to time constraints and the presence of several conditions, we are only able to derive conclusions from a limited time window. Moreover, previous research has suggested that individual differences in empathy—the ability to vicariously experience and understand others’ affect—might relate to differences in prosocial learning7. Empathy is positively associated with affective-interpersonal psychopathic traits and might also relate to motivation to help others74,75. Further studies could also assess how empathy predicts changes in prosocial learning across the lifespan. Finally, future studies could manipulate the identity of the recipient as we show preservation in the tendency to help others when there are no particular characteristics known about the other person, but these effects might additionally be modulated by factors such as perceived social distance.

To conclude, we find evidence that despite declines in self-relevant learning in older adults, the ability to learn which actions benefit others is preserved. Moreover, the bias with which people favour self-relevant outcomes is reduced. Not only do older adults have relatively preserved prosocial learning, they also report lower levels of core psychopathic traits that index lack of empathy and guilt, and these traits are linked to variability in prosocial learning. These findings could have important implications for our understanding of reinforcement learning and theoretical accounts of healthy ageing.

## Methods

### Participants

We recruited 80 young participants and 80 older participants using the same recruitment methods in order to match the samples as closely as possible. Participants were recruited from university databases, which included students and members of the community, social media, and adverts in local newspapers. We excluded anyone who was currently studying or had previously studied psychology, and no one took part for course credit. Additional exclusion criteria were previous or current neurological or psychiatric disorder, non-normal or non-corrected to normal vision and, for the older sample, scores on the Addenbrooke’s Cognitive Examination that indicate potential dementia (cut-off score 82)48. This sample size gave us 88% power to detect a medium-size effect (d = 0.5).

Five young and three older participants were excluded due to: diagnosis of a psychiatric disorder at the time of testing (one young participant); previous study of psychology (two young participants); and incomplete or low-quality data (two young and three older participants). This left a final sample of 152 participants, 75 young adults (44 females aged 18–36, mean = 23.07) and 77 older adults (40 females aged 60–80, mean = 69.84). Two older participants were excluded from all analyses involving learning rates due to each having two learning rate estimates more than three standard deviations (SDs) above the mean (for one αself was 6.68 SDs above the mean and αno one 9.64 SDs above the mean; for the second αself was 7.96 and αother 3.78 SDs above the mean). One further participant from each age group was missing data on the SRP measure so are not included in the relevant analyses.

Participants were paid at a rate of £10 per hour plus an additional payment of up to £5 depending on the number of points they earned for themselves during the task. They were also told the number of points that they earned in the prosocial condition would translate into an additional payment of up to £5 for the other participant (see details of the task below). All participants provided written informed consent and the study was approved by the Oxford University Medical Sciences Inter Divisional Research Ethics Committee and National Health Service Ethics.

### Prosocial learning task

The prosocial learning task is a probabilistic reinforcement-learning task, with rewards in one of three recipient conditions: for the participant themselves (self), for another participant (other; prosocial condition), and for neither person (no one; control condition)7. Each trial presents two symbols, one associated with a high (75%) probability of gaining points and the other with a low (25%) probability of gaining points. The two symbols were randomly assigned to the left or right side of the screen and selected via a corresponding button press. Participants select a symbol then receive feedback on whether they obtained points or not (see Fig. 1b) so learn over time which symbol maximises rewards. The experiment was subdivided into blocks, i.e., 16 trials pairing the same two symbols for the same recipient. Participants completed three blocks, a total of 48 trials, in each recipient condition, resulting in 144 trials overall (see Supplementary Information for trial structure). Blocks for different recipients were pseudo-randomly ordered such that the same recipient block did not occur twice in a row.

On trials in the self condition, points translated into increased payment for the participant themselves. These blocks started with ‘play for you' displayed and had the word ‘you' at the top of each screen. Blocks in the no one condition had ‘no one' in place of ‘you' and points were not converted into any extra payment for anyone. In the prosocial other condition, participants earned points that translated into additional payment for a second participant, actually a confederate. Participants were told that this payment would be given anonymously, they would never meet the other person, and that the person was not even aware of them completing this task (see Supplementary Information). The name of the confederate, gender-matched to the participant, was displayed on these blocks at the start and on each screen (Fig. 1b). Thus, participants were explicitly aware of how their decisions affected on each trial. Stimuli were presented using Presentation v17 (Neurobehavioral Systems— https://www.neurobs.com/).

### Questionnaire measures

#### Dementia screening and executive function

Older adults were screened for dementia using the Addenbrooke’s Cognitive Examination (ACE-III)48. The ACE examines five cognitive domains; attention, memory, language, fluency and visuospatial abilities. The ACE-III is scored out of 100 and as a screening tool, a cut-off score of 82/100 denotes significant cognitive impairment. We also used scores on the attention and memory domains in control analyses as proxies for executive function in older adults.

#### General intelligence

All participants completed the Wechsler Test of Adult Reading (WTAR)49 as a measure of IQ test performance. The WTAR requires participants to pronounce 50 words that have an unusual grapheme-to-phoneme translation. This means the test measures reading recognition or existing knowledge of the words, rather than the ability to apply rules for pronunciation. The WTAR was developed and standardised with the Wechsler Memory and Adult Intelligence Scales and correlates highly with these measures76. Standardisation involves adjusting for healthy age-related differences. The test is suitable for participants aged 16–89, covering our full sample, and scores in older age have been shown to correlate with cognitive ability earlier in life77.

#### Psychopathic traits

Participants completed the short form of the Self-Report Psychopathy Scale 4th Edition (SRP-IV-SF)40. This scale consists of 29 items, 7 each measuring: interpersonal, affective, lifestyle and antisocial tendencies (plus ‘I have been convicted of a serious crime’). We used the two-factor grouping, summing the core, affective-interpersonal items and separately, the lifestyle-antisocial items for use in the analysis. The robust psychometric properties of this measure have been established in community78 and offender populations through construct and convergent validity79, internal consistency and reliability40.

### Procedure

#### Role assignment

To enhance the belief that points earned in the prosocial condition benefitted another person, we conducted a role-assignment procedure based on a set-up used in several studies of social decision-making80,81. Participants were instructed not to speak and wore a glove to hide their identity. A second experimenter brought the confederate, also wearing a glove, to the other side of the door. Participants only ever saw the gloved hand of the confederate, but they waved to each other to make it clear there was another person there (Fig. 1a). The experimenter tossed a coin to determine who picked a ball from the box first and then told the participants which roles they had been assigned to, based on the ball they picked. Our procedure ensured that participants always ended up in the role of the person performing the prosocial learning experiment. Participants were unaware of the age of the other person, but the experimenter used a name for them, suggesting their gender was the same as the participant.

Participants received instructions for the learning task and how the points they earned would be converted into money for themselves and for the other participant. Instructions included that the two symbols were different in how likely it was that choosing them lead to points but that which side they appeared on the screen was irrelevant. Participants then completed one block of practice trials before the main task and were aware outcomes during the practice did not affect payment for anyone. After the task, participants completed the self-report measure of psychopathic traits and the dementia screening.

### Computational modelling

We modelled learning during the task with a reinforcement-learning algorithm16, creating variations of the models through the number of parameters used to explain the learning rate and temperature parameters in the task82. The basis of the reinforcement-learning algorithm is the expectation that an action (or stimulus) a will provide reward on the following trial. This expected value, Qt+1(a) is quantified as a function of current expectations Qt(a) and the prediction error δt, which is scaled by the learning rate α:

$${Q}_{t+1}(a)\,=\,{Q}_{t}(a)\,+\,\alpha \,\times \,\underbrace{[{r}_{t}\,-\,{Q}_{t}(a)]}_{{{{\rm{Predictionerror}}}}\,{\delta}_{t}}$$
(1)

Where δt, the prediction error, is the difference between the actual reward experienced on the current trial rt (1 for reward and 0 for no reward) minus the expected reward on the current trial Qt(a).

The learning rate α therefore determines the influence of the prediction error. A low learning rate means new information affects expected value to a lesser extent. The softmax link function quantifies the relationship between the expected value of the action Qt(a) and the probability of choosing that action on trial t:

$${p}_{t}[(a|{Q}_{t}(a))]\,=\,\frac{{e}^{({Q}_{t}(a)/\beta )}}{{\sum }_{a{\prime} }{e}^{({Q}_{t}(a{\prime} )/\beta )}}$$
(2)

The temperature parameter β represents the noisiness of decisions—whether the participant explores or always chooses the option with the highest expected value. A high value for β means choices seems random as they are equally likely irrespective of the expected value of each option. A low β leads to choosing the option with the greatest expected value on all trials.

### Model fitting

We used MATLAB 2019b (The MathWorks Inc) for all model fitting and comparison. To fit the variations of the learning model (see below) to (real and simulated) participant data, we used an iterative maximum a posteriori (MAP) approach50,51. This approach involves implementing two levels: the lower level of the individual subjects and the higher level reflecting our full sample. An MAP approach provides a better estimation than a single-step maximum likelihood estimation (MLE) alone by being less susceptible to the influence of outliers. For the real participant data, we fit the model across groups to provide the most conservative comparison, so this full sample combined young and older participants.

For the MAP procedure, we initialised group-level Gaussians as uninformative priors with means of 0.1 (plus some added noise) and variance of 100. During the expectation, we estimated the model parameters (α and β) for each participant using an MLE approach, calculating the log-likelihood of the subject’s series of choices given the model. We then computed the maximum posterior probability estimate, given the observed choices and given the prior computed from the group-level Gaussian, and recomputed the Gaussian distribution over parameters during the maximisation step. We repeated expectation and maximisation steps iteratively until convergence of the posterior likelihood summed over the group or a maximum of 800 steps. Convergence was defined as a change in posterior likelihood <0.001 from one iteration to the next. Note that bounded free parameters were transformed from the Gaussian space into the native model space via appropriate link functions (e.g., a sigmoid function in the case of the learning rates) to ensure accurate parameter estimation near the bounds. The detailed code for the models and implementation of the fitting algorithm83 can be found at https://doi.org/10.17605/osf.io/xgw7h.

### Model comparison

Our hypotheses generated four models to compare that differed in whether the model parameters (α and β) for each participant had one value across recipient conditions or depended on the recipient (self, other and no one). For model comparison, we calculated the Laplace approximation of the log model evidence (more positive values indicating better model fit84) and submitted these to a random-effects analysis using the spm_BMS routine85 from SPM 8 (http://www.fil.ion.ucl.ac.uk/spm/software/spm8/). This generates the exceedance probability: the posterior probability that each model is the most likely of the model set in the population (higher is better, over 0.95 indicates strong evidence in favour of a model). For the models of real participant data, we also calculated the integrated BIC (lower is better50,51) and R2 as additional measures of model fit. To calculate the model R2, we extracted the choice probabilities generated for each participant on each trial from the winning model. We then took the squared median choice probability across participants. The 3α1β model had the best evidence on all measures (see Supplementary Table 2).

### Simulation experiments

We used simulation experiments to assess that our experiment allowed us to dissociate models of interest, as well as parameters of interest within the winning model. We simulated data from all four models to establish that our model comparison procedure (see above) could accurately identify the best model across a wide range of parameter values. For this model identifiability analysis, we simulated data from 150 participants, drawing parameters from distributions commonly used in the reinforcement-learning literature86,87. Learning rates (α) were drawn from a beta distribution (betapdf(parameter,1.1,1.1)) and softmax temperature parameters (β) from a gamma distribution (gampdf(parameter,1.2,5)). We fitted the models to this simulated data set using the same MAP process as applied to the experimental participants’ data and repeated this whole procedure ten times. By plotting the confusion matrices of average exceedance probability (across the ten runs; Fig. 2a) and how many times each model won (Fig. 2b), we show the models are identifiable using our model comparison process.

Our winning model contained four free parameters (αself, αother, αno one, β). To assess the reliability of this model and the interpretability of the free parameters, we also performed parameter recovery on simulated data (see Supplementary Information for procedure) as recommended for modelling analyses that use a ‘data first’ approach82,88. We simulated choices 1296 times using our experimental schedule and fitted them using MAP. We found strong Pearson’s correlations between the true simulated and fitted parameter values (all rs > 0.7, see Fig. 2c), suggesting our experiment was well suited to estimate the model’s parameters.

Finally, we conducted a principled simulation experiment to identify the optimal learning rate in our task and examine the link between learning rates and performance. We simulated data from 10,000 participants with learning rates (α) and softmax temperature parameters (β) drawn from beta and gamma distributions, respectively, as described above. We bounded the β parameters at 0 and 0.3 to reflect the range shown by our participants. In line with our winning model, simulated participants had a separate learning rate for each recipient condition, and these spanned the full range of possible α values from 0 to 1. This generated 30,000 learning rate values (10,000 participants and three conditions). For each, we quantified performance as the proportion of times the simulated participant chose the ‘correct’ high-reward option in the relevant condition. Results showed that average performance improved as learning rates increased, up to learning rates of approximately 0.55 (Fig. 2e). Crucially, this optimal alpha was above the highest learning rate shown by any of our participants in any recipient condition, meaning a higher learning rate on our task was associated with better performance.

### Statistical analysis

Analysis of group and recipient differences in the fitted model parameters and behavioural data was run in R89 v3.6.2 with R Studio90. We used a robust linear mixed-effect model (RLMM; rlmer function; robustlmm package91 to predict learning rates and generalised linear mixed-effects models (glmer function; lme4 package92) for the trial-by-trial data (binary outcome of choosing the high vs. low reward option). We used (robust/generalised) linear mixed-effects models as these account for the within-subject nature of the recipient manipulation and do not rely on parametric assumptions. In addition, unlike an ANOVA model or omnibus test, an RLMM generates coefficients (with confidence intervals and significance values) for terms that compare pairs of factor levels (e.g., self vs. other). This approach is more informative than an ANOVA when the factor has more than two levels, which is our case as the recipient factor has three levels (self, other and no one). Each linear mixed-effects model had fixed effects of age group, recipient (self, other, no one) and their interaction, plus a random subject-level intercept. Analysis of trial-by-trial choices also included the trial number in the fixed terms, interacting with recipient and group (including the three-way interaction), and in the random terms, interacting with the recipient. In the analysis of learning rates controlling for IQ test performance, standardised scores on the WTAR were also included as a fixed term (Supplementary Table 8). Correlations of learning rates with self-reported psychopathic traits (Supplementary Tables 4 and 5) and neuropsychological measures (Supplementary Table 9) were calculated with Spearman’s Rho nonparametric tests. Comparisons between independent correlations were calculated with Z tests (paired.r function; psych package93) To control for IQ test performance and executive function in the associations between older adults’ prosocial learning rates and self-reported psychopathic traits, we ran partial correlations each controlling for one of WTAR, ACE memory and ACE attention scores (Supplementary Table 10).

For simple and post hoc comparisons, we used two-sided nonparametric tests as outcome variables that violated normality assumptions. Effect sizes and confidence intervals for paired and independent nonparametric comparisons were calculated using the cohens_d and wilcox_effsize functions, respectively, from the rstatix package94. Bayes factors (BF01) for non-significant results were calculated using nonparametric paired and independent t tests in JASP95 v0.14.1 with the default prior. BF01 corresponds to how many times more likely the data are under the null hypothesis of no difference than under the alternative hypothesis that there is a difference. A BF01 >3 (equal to BF10 <1/3) is considered substantial evidence in favour of the null hypothesis whereas a BF01 between 1/3 and 3 indicates the data cannot clearly differentiate between hypotheses96. Median learning rates and their standard errors for plotting were calculated using bootstrapping with 1000 samples.

For the analysis of indirect paths, we used the mediate function (mediation package97) combined with robust linear models (rlm function, MASS package98). This method estimates the unstandardised indirect effects, with 95% confidence intervals and significance, through a bootstrapping procedure with 10,000 bootstrapped samples. The outcome in the models was relative prosocial learning rate (αother − αself), the predictor was age group, and the indirect path was through self-reported psychopathic traits on the core affective-interpersonal subscale of the SRP-IV-SF. We calculated two models. The first, a standard mediation model, included an indirect path from age group to relative prosocial learning rate via self-reported psychopathic traits (Fig. 5a). The second included an interaction between age group and self-reported psychopathic traits in predicting relative prosocial learning rate to allow the possibility of a statistically moderated mediation99. Specifically, this type of moderated mediation examines whether the indirect path (in this case via self-reported psychopathic traits) on the outcome (αother − αself) is moderated by the predictor (age group)100,101. In other words, self-reported psychopathic traits could explain some of the variances in relative prosocial learning for one age group but not the other (Fig. 5b).

### Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

## Data availability

The data generated in this study are available on the Open Science Framework at https://doi.org/10.17605/osf.io/xgw7hSource data are provided with this paper.

## Code availability

Code for modelling and analysis is available at https://doi.org/10.17605/osf.io/xgw7h.

## References

1. 1.

Cohen, M. X. Neurocomputational mechanisms of reinforcement-guided learning in humans: a review. Cogn. Affect. Behav. Neurosci. 8, 113–125 (2008).

2. 2.

Glimcher, P. W. Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc. Natl Acad. Sci. USA 108, 15647–15654 (2011).

3. 3.

Niv, Y. Reinforcement learning in the brain. J. Math. Psychol. 53, 139–154 (2009).

4. 4.

O’Doherty, J. P., Lee, S. W. & McNamee, D. The structure of reinforcement-learning mechanisms in the human brain. Curr. Opin. Behav. Sci. 1, 94–100 (2015).

5. 5.

Schultz, W. Neuronal reward and decision signals: from theories to data. Physiol. Rev. 95, 853–951 (2015).

6. 6.

Lockwood, P. L. et al. Neural mechanisms for learning self and other ownership. Nat. Commun. 9, 4747 (2018).

7. 7.

Lockwood, P. L., Apps, M. A. J., Valton, V., Viding, E. & Roiser, J. P. Neurocomputational mechanisms of prosocial learning and links to empathy. Proc. Natl Acad. Sci. USA 113, 201603198–201603198 (2016).

8. 8.

Blanchard-Fields, F. Everyday problem-solving and emotion: an adult developmental perspective. Psychol. Sci. 16, 26–31 (2007).

9. 9.

Blanchard-Fields, F. & Hess, T. M. The social cognitive perspective and the study of aging. in Social Cognition and Aging (eds Hess, T. M. & Blanchard-Fields, F.) 1–14 (Academic Press, 1999).

10. 10.

Samanez-Larkin, G. R. & Knutson, B. Decision making in the ageing brain: changes in affective and motivational circuits. Nat. Rev. Neurosci. 16, 278–289 (2015).

11. 11.

Holt-Lunstad, J., Smith, T. B., Baker, M., Harris, T. & Stephenson, D. Loneliness and social isolation as risk factors for mortality: a meta-analytic review. Perspect Psychol. Sci. J. Assoc. Psychol. Sci. 10, 227–237 (2015).

12. 12.

Fehr, E. & Fischbacher, U. The nature of human altruism. Nature 425, 785–791 (2003).

13. 13.

Buchanan, K. E. & Bardi, A. Acts of kindness and acts of novelty affect life satisfaction. J. Soc. Psychol. 150, 235–237 (2010).

14. 14.

Raposa, E. B., Laws, H. B. & Ansell, E. B. Prosocial behavior mitigates the negative effects of stress in everyday life. Clin. Psychol. Sci. 4, 691–698 (2016).

15. 15.

Post, S. G. Altruism, happiness, and health: it’s good to be good. Int. J. Behav. Med. 12, 66–77 (2005).

16. 16.

Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 2011).

17. 17.

Schultz, W. Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 30, 259–288 (2007).

18. 18.

Eppinger, B., Hämmerer, D. & Li, S.-C. Neuromodulation of reward-based learning and decision making in human aging. Ann. N. Y. Acad. Sci. 1235, 1–17 (2011).

19. 19.

Mell, T. et al. Effect of aging on stimulus-reward association learning. Neuropsychologia 43, 554–563 (2005).

20. 20.

Samanez-Larkin, G. R., Levens, S. M., Perry, L. M., Dougherty, R. F. & Knutson, B. Frontostriatal white matter integrity mediates adult age differences in probabilistic reward learning. J. Neurosci. 32, 5333–5337 (2012).

21. 21.

Samanez-Larkin, G. R., Worthy, D. A., Mata, R., McClure, S. M. & Knutson, B. Adult age differences in frontostriatal representation of prediction error but not reward outcome. Cogn. Affect. Behav. Neurosci. 14, 672–682 (2014).

22. 22.

Hammerer, D. & Eppinger, B. Dopaminergic and prefrontal contributions to reward-based learning and outcome monitoring during child development and aging. Dev. Psychol. 48, 826–874 (2012).

23. 23.

Bäckman, L., Nyberg, L., Lindenberger, U., Li, S.-C. & Farde, L. The correlative triad among aging, dopamine, and cognition: current status and future prospects. Neurosci. Biobehav. Rev. 30, 791–807 (2006).

24. 24.

Dreher, J.-C., Meyer-Lindenberg, A., Kohn, P. & Berman, K. F. Age-related changes in midbrain dopaminergic regulation of the human reward system. Proc. Natl Acad. Sci. USA 105, 15106–15111 (2008).

25. 25.

Li, S.-C., Lindenberger, U. & Bäckman, L. Dopaminergic modulation of cognition across the life span. Neurosci. Biobehav. Rev. 34, 625–630 (2010).

26. 26.

Schultz, W. Updating dopamine reward signals. Curr. Opin. Neurobiol. 23, 229–238 (2013).

27. 27.

Schultz, W. Dopamine reward prediction-error signalling: a two-component response. Nat. Rev. Neurosci. 17, 183–195 (2016).

28. 28.

Chowdhury, R. et al. Dopamine restores reward prediction errors in old age. Nat. Neurosci. 16, 648–653 (2013).

29. 29.

Contreras-Huerta, L. S., Pisauro, M. A. & Apps, M. A. J. Effort shapes social cognition and behaviour: A neuro-cognitive framework. Neurosci. Biobehav. Rev. 118, 426–439 (2020).

30. 30.

Engel, C. Dictator games: a meta study. Exp. Econ. 14, 583–610 (2011).

31. 31.

Matsumoto, Y., Yamagishi, T., Li, Y. & Kiyonari, T. Prosocial behavior increases with age across five economic games. PLoS ONE 11, e0158671 (2016).

32. 32.

Sze, J. A., Gyurak, A., Goodkind, M. S. & Levenson, R. W. Greater emotional empathy and prosocial behavior in late life. Emotion 12, 1129–1140 (2012).

33. 33.

Ng, T. W. H. & Feldman, D. C. The relationship of age to ten dimensions of job performance. J. Appl. Psychol. 93, 392–423 (2008).

34. 34.

Hubbard, J., Harbaugh, W. T., Srivastava, S., Degras, D. & Mayr, U. A general benevolence dimension that links neural, psychological, economic, and life-span data on altruistic tendencies. J. Exp. Psychol. Gen. 145, 1351–1358 (2016).

35. 35.

Cheung, F. & Lucas, R. E. When does money matter most? Examining the association between income and life satisfaction over the life course. Psychol. Aging 30, 120–135 (2015).

36. 36.

Buckholtz, J. W. et al. Mesolimbic dopamine reward system hypersensitivity in individuals with psychopathic traits. Nat. Neurosci. 13, 419–421 (2010).

37. 37.

White, B. A. Who cares when nobody is watching? Psychopathic traits and empathy in prosocial behaviors. Personal. Individ. Differ. 56, 116–121 (2014).

38. 38.

White, S. F. et al. Reduced amygdala response in youths with disruptive behavior disorders and psychopathic traits: decreased emotional response versus increased top-down attention to nonemotional features. Am. J. Psychiatry 169, 750–758 (2012).

39. 39.

Marsh, A. A. et al. Empathic responsiveness in amygdala and anterior cingulate cortex in youths with psychopathic traits. J. Child Psychol. Psychiatry 54, 900–910 (2013).

40. 40.

Paulhus, D. L., Neumann, C. S., Hare, R. D., Williams, K. M. & Hemphill, J. F. Self-report Psychopathy Scale 4th Edition (SRP 4) Manual (MHS, Multi-Health Systems Incorporated, 2017).

41. 41.

Leistico, A.-M. R., Salekin, R. T., DeCoster, J. & Rogers, R. A large-scale meta-analysis relating the Hare measures of psychopathy to antisocial conduct. Law Hum. Behav. 32, 28–45 (2008).

42. 42.

Blais, J., Solodukhin, E. & Forth, A. E. A meta-analysis exploring the relationship between psychopathy and instrumental versus reactive violence. Crim. Justice Behav. 41, 797–821 (2014).

43. 43.

Asscher, J. J. et al. The relationship between juvenile psychopathic traits, delinquency and (violent) recidivism: a meta-analysis. J. Child Psychol. Psychiatry 52, 1134–1143 (2011).

44. 44.

Insel, T. R. The NIMH Research Domain Criteria (RDoC) Project: precision medicine for psychiatry. Am. J. Psychiatry 171, 395–397 (2014).

45. 45.

Seara-Cardoso, A., Neumann, C., Roiser, J., McCrory, E. & Viding, E. Investigating associations between empathy, morality and psychopathic personality traits in the general population. Personal. Individ. Differ. 52, 67–71 (2012).

46. 46.

Huchzermeier, C. et al. Are there age-related effects in antisocial personality disorders and psychopathy? J. Forensic Leg. Med. 15, 213–218 (2008).

47. 47.

Gill, D. J. & Crino, R. D. The relationship between psychopathy and age in a non-clinical community convenience sample. Psychiatry Psychol. Law 19, 547–557 (2012).

48. 48.

Hsieh, S., Schubert, S., Hoon, C., Mioshi, E. & Hodges, J. R. Validation of the Addenbrooke’s cognitive examination III in frontotemporal dementia and Alzheimer’s disease. Dement. Geriatr. Cogn. Disord. 36, 242–250 (2013).

49. 49.

The Psychological Corporation. Wechsler Test of Adult Reading (Harcourt Assessment, 2001).

50. 50.

Huys, Q. J. et al. Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding. PLoS Comput. Biol. 7, e1002028 (2011).

51. 51.

Wittmann, M. K. et al. Global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys. Nat. Commun. 11, 3771 (2020).

52. 52.

Zhang, L., Lengersdorff, L., Mikus, N., Gläscher, J. & Lamm, C. Using reinforcement learning models in social neuroscience: frameworks, pitfalls and suggestions of best practices. Soc. Cogn. Affect. Neurosci. 15, 695–707 (2020).

53. 53.

Olsson, A., Knapska, E. & Lindström, B. The neural and computational systems of social learning. Nat. Rev. Neurosci. 21, 197–212 (2020).

54. 54.

Bolenz, F., Kool, W., Reiter, A. M. & Eppinger, B. Metacontrol of decision-making strategies in human aging. eLife 8, e49154 (2019).

55. 55.

Lighthall, N. R., Pearson, J. M., Huettel, S. A. & Cabeza, R. Feedback-based learning in aging: contributions and trajectories of change in striatal and hippocampal systems. J. Neurosci. 38, 8453–8462 (2018).

56. 56.

Daniel, R., Radulescu, A. & Niv, Y. Intact reinforcement learning but impaired attentional control during multidimensional probabilistic learning in older adults. J. Neurosci. 40, 1084–1096 (2020).

57. 57.

Nieuwenhuis, S. et al. A computational account of altered error processing in older age: dopamine and the error-related negativity. Cogn. Affect. Behav. Neurosci. 2, 19–36 (2002).

58. 58.

Karrer, T. M., Josef, A. K., Mata, R., Morris, E. D. & Samanez-Larkin, G. R. Reduced dopamine receptors and transporters but not synthesis capacity in normal aging adults: a meta-analysis. Neurobiol. Aging 57, 36–46 (2017).

59. 59.

Seaman, K. L. et al. Differential regional decline in dopamine receptor availability across adulthood: Linear and nonlinear effects of age. Hum. Brain Mapp. 40, 3125–3138 (2019).

60. 60.

Mayr, U. & Freund, A. M. Do we become more prosocial as we age, and if so, why? Curr. Dir. Psychol. Sci. 29, 248–254 (2020).

61. 61.

Lockwood, P. L. et al. Aging Increases Prosocial Motivation for Effort. Psychol. Sci. 32, 668–681 (2021).

62. 62.

Seaman, K. L. et al. Adult age differences in decision making across domains: Increased discounting of social and health-related rewards. Psychol. Aging 31, 737–746 (2016).

63. 63.

Rademacher, L., Salama, A., Gründer, G. & Spreckelmeyer, K. N. Differential patterns of nucleus accumbens activation during anticipation of monetary and social reward in young and older adults. Soc. Cogn. Affect. Neurosci. 9, 825–831 (2014).

64. 64.

Gorlick, M. A. et al. Attenuating age-related learning deficits: emotional valenced feedback interacts with task complexity. Emotion 13, 250–261 (2013).

65. 65.

Lockwood, P. L., O’Nell, K. C. & Apps, M. A. J. Anterior cingulate cortex: a brain system necessary for learning to reward others? PLOS Biol. 18, e3000735 (2020).

66. 66.

Lockwood, P. L., Apps, M. A. J. & Chang, S. W. C. Is There a ‘Social’ Brain? Implementations and Algorithms. Trends Cogn. Sci. 24, 802–813 (2020).

67. 67.

Basile, B. M., Schafroth, J. L., Karaskiewicz, C. L., Chang, S. W. C. & Murray, E. A. The anterior cingulate cortex is necessary for forming prosocial preferences from vicarious reinforcement in monkeys. PLoS Biol. 18, e3000677 (2020).

68. 68.

Izuma, K. The social neuroscience of reputation. Neurosci. Res. 72, 283–288 (2012).

69. 69.

Andreoni, J. Giving with impure altruism: applications to charity and Ricardian equivalence. J. Polit. Econ. 97, 1447–1447 (1989).

70. 70.

Andreoni, J. Impure altruism and donations to public goods: a theory of warm-glow giving. Econ. J. 100, 464–477 (1990).

71. 71.

Kuss, K. et al. A reward prediction error for charitable donations reveals outcome orientation of donators. Soc. Cogn. Affect. Neurosci. 8, 216–223 (2013).

72. 72.

Mobbs, D. et al. A key role for similarity in vicarious reward. Science 324, 900–900 (2009).

73. 73.

Camerer, C. F. & Fehr, E. Measuring social norms and preferences using experimental games: a guide for social scientists. Found. Hum. Sociality Econ. Exp. Ethnogr. Evid. Fifteen-. Small-Scale Soc. 97, 55–95 (2004).

74. 74.

Contreras-Huerta, L. S., Lockwood, P. L., Bird, G., Apps, M. A. J. & Crockett, M. J. Prosocial behavior is associated with transdiagnostic markers of affective sensitivity in multiple domains. Emotion. https://doi.org/10.1037/emo0000813 (2020).

75. 75.

Lockwood, P. L., Ang, Y.-S., Husain, M. & Crockett, M. J. Individual differences in empathy are associated with apathy-motivation. Sci. Rep. 7, 17293 (2017).

76. 76.

Venegas, J. & Clark, E. Wechsler test of adult reading. in Encyclopedia of Clinical Neuropsychology (eds Kreutzer, J. S., DeLuca, J. & Caplan, B.) 2693–2694 (Springer, 2011).

77. 77.

Dykiert, D. & Deary, I. J. Retrospective validation of WTAR and NART scores as estimators of prior cognitive ability using the Lothian Birth Cohort 1936. Psychol. Assess. 25, 1361–1366 (2013).

78. 78.

Gordts, S., Uzieblo, K., Neumann, C., Van den Bussche, E. & Rossi, G. Validity of the self-report psychopathy scales (SRP-III Full and Short Versions) in a community sample. Assessment 24, 308–325 (2017).

79. 79.

Neumann, C. S. & Pardini, D. Factor structure and construct validity of the self-report psychopathy (SRP) scale and the youth psychopathic traits inventory (YPI) in young men. J. Personal. Disord. 28, 419–433 (2012).

80. 80.

Crockett, M. J. et al. Dissociable effects of serotonin and dopamine on the valuation of harm in moral decision making. Curr. Biol. 25, 1852–1859 (2015).

81. 81.

Lockwood, P. L. et al. Prosocial apathy for helping others when effort is required. Nat. Hum. Behav. 1, 0131 (2017).

82. 82.

Lockwood, P. L. & Klein-Flügge, M. C. Computational modelling of social cognition and behaviour—a reinforcement learning primer. Soc. Cogn. Affect. Neurosci. https://doi.org/10.1093/scan/nsaa040 (2020).

83. 83.

Cutler, J. et al. Ageing is associated with disrupted reinforcement learning whilst learning to help others is preserved. Open Sci. Framew. https://doi.org/10.17605/osf.io/xgw7h (2021).

84. 84.

MacKay, D. J. C. Information Theory, Inference and Learning Algorithms (Cambridge University Press, 2003).

85. 85.

Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J. & Friston, K. J. Bayesian model selection for group studies. NeuroImage 46, 1004–1017 (2009).

86. 86.

Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P. & Dolan, R. J. Model-based influences on humans’ choices and striatal prediction errors. Neuron 69, 1204–1215 (2011).

87. 87.

Palminteri, S., Khamassi, M., Joffily, M. & Coricelli, G. Contextual modulation of value signals in reward and punishment learning. Nat. Commun. 6, 8096 (2015).

88. 88.

Palminteri, S., Wyart, V. & Koechlin, E. The importance of falsification in computational cognitive modeling. Trends Cogn. Sci. 21, 425–433 (2017).

89. 89.

R Core Team (R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, 2017).

90. 90.

RStudio Team (RStudio: Integrated Development for R. RStudio, Inc., 2015).

91. 91.

Koller, M. robustlmm: an R package for robust estimation of linear mixed-effects models. J. Stat. Softw. 75, 1–24 (2016).

92. 92.

Bates, D., Mächler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2015).

93. 93.

Revelle, W. psych: Procedures for Psychological, Psychometric, and Personality Research (Northwestern University, 2020).

94. 94.

Kassambara, A. rstatix: Pipe-Friendly Framework for Basic Statistical Tests. R package version 0.5.0. (2020).

95. 95.

JASP Team. JASP (Version 0.12.2). (Eric-Jan Wagenmakers, University of Amsterdam, 2020).

96. 96.

Marsman, M. & Wagenmakers, E.-J. Bayesian benefits with JASP. Eur. J. Dev. Psychol. 14, 545–555 (2017).

97. 97.

Tingley, D., Yamamoto, T., Hirose, K., Keele, L. & Imai, K. mediation: R package for causal mediation analysis. J. Stat. Softw. 59, 1–38 (2014).

98. 98.

Venables, W. N. & Ripley, B. D. Modern Applied Statistics with S. (Springer, 2002).

99. 99.

Imai, K., Keele, L. & Tingley, D. A general approach to causal mediation analysis. Psychol. Methods 15, 309–334 (2010).

100. 100.

Preacher, K. J., Rucker, D. D. & Hayes, A. F. Addressing moderated mediation hypotheses: theory, methods, and prescriptions. Multivar. Behav. Res. 42, 185–227 (2007).

101. 101.

Judd, C. M. & Kenny, D. A. Process analysis: estimating mediation in treatment evaluations. Eval. Rev. 5, 602–619 (1981).

## Acknowledgements

This work was supported by a Medical Research Council Fellowship (MR/P014097/1), a Christ Church Junior Research Fellowship, a Christ Church Research Centre Grant, and a Jacobs Foundation Research Fellowship to P.L.; a Wellcome Trust Principal Fellowship to M.H.; NIHR Biomedical Research Centre, Oxford. The Wellcome Centre for Integrative Neuroimaging is supported by core funding from the Wellcome Trust (203139/Z/16/Z). We thank Ellena Crane for assistance with data collection. We are grateful to Matthew Apps and Miriam Klein-Flugge for helpful disucssions and to  Craig Neumann for assistance with the Self-Report Psychopathy Scale. We are also grateful to our colleagues who acted as the other participant during the study.

## Author information

Authors

### Contributions

P.L. designed the study. A.A., L.H., D.D. and P.L. collected the data. J.C., M.W. and P.L. analysed the data. J.C., M.W., M.H. and P.L. wrote the paper.

### Corresponding authors

Correspondence to Jo Cutler or Patricia L. Lockwood.

## Ethics declarations

### Competing interests

The authors declare no competing interests

Peer review information Nature Communications thanks Gregory Samanez-Larkin and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Cutler, J., Wittmann, M.K., Abdurahman, A. et al. Ageing is associated with disrupted reinforcement learning whilst learning to help others is preserved. Nat Commun 12, 4440 (2021). https://doi.org/10.1038/s41467-021-24576-w

• Accepted:

• Published: