Main

The behavioral and psychological symptoms of dementia (BPSD) include delusions, hallucinations, aggression, activity disturbance (e.g., wandering), affective disturbances, anxieties, diurnal rhythm disturbances and apathy (Raskind 1993; Cummings et al. 1994; Sclan et al. 1996). These behaviors are common, and have negative impacts on both the patient and the caregiver (Cohen et al. 1993; Cohen-Mansfield and Billig 1986). Despite this known importance, the neurobiology underlying agitation and aggression in AD remains to be elucidated, complicating rational treatment attempts.

Alzheimer's disease (AD) is associated with an extensive serotonergic deficit (Adolffson et al. 1979; Arai et al. 1984; Cross et al. 1984; Mann and Yates 1983; Yamamoto and Hirano 1985). As serotonin (5-HT) generally plays an inhibitory role on motivated behaviors and other neurotransmitter systems, loss of inhibiting tone from the 5-HT system may be important in BPSD (Herrmann and Lanctôt 1997; Lanctôt et al. 2001). In fact, the serotonergic system is involved in many psychobiological functions such as aggression, anxiety, depression, sleep, sexual behavior, appetite, anxiety, learning, aversion and psychosis (Dubovsky and Thomas 1995) that are relevant to BPSD. Furthermore, reduced 5-HT has been associated with hostility, impulsivity and aggression in other, non-demented psychiatric populations (Coccaro and Astill 1990; Linnoila and Virkkunen 1992; van Praag 1991). Three post-mortem studies have addressed the relationship between serotonin and aggression finding both decreased serotonin and it metabolite (Palmer et al. 1988), decreased serotonin 5-HT2 receptors (Procter et al. 1992) and no relationship (Chen et al. 1996). The in vivo functional significance of underlying dysfunctions in serotonergic neurotransmission with respect to behaviors and their potential for treatment is unknown.

Open label trial treatment with selective serotonin reuptake inhibitors (SSRIs) such as alaproclate (Bergman et al. 1983), sertraline (Burke et al. 1997; Burke et al. 1994) and citalopram (Pollock et al. 1997), and the 5-HT1A partial agonist buspirone (Herrmann and Eryavec 1993) have been associated with improvements in BPSD in a high proportion of patients. Randomized controlled trials with the SSRIs (Auchus and Bissey-Black 1997; Burke et al. 1994; Cutler et al. 1985; Dehlin et al. 1985; Katona et al. 1998; Lanctôt et al. 2002; Nyth and Gottfries 1990; Olafsson et al. 1992; Pollock et al. 2002; Taragano et al. 1997), buspirone (Cantillon et al. 1996; Lawlor et al. 1994) and the serotonin norepinephrine reuptake inhibitor trazodone (Lawlor et al. 1994; Sultzer et al. 1997) have also been performed for the treatment of BPSD with more equivocal results. Of the SSRIs, citalopram, which has the highest selectivity for serotonin versus norepinephrine (Hyttel 1994), shows the greatest efficacy in randomized controlled trials (Nyth and Gottfries 1990; Pollock et al. 2002). This clinical evidence, albeit indirect, is supportive of a link between serotonergic dysfunction and some BPSD.

The hypothesis of this study was that reduced central serotonergic activity is etiologically related to aggression in patients with AD. To test this hypothesis, central serotonergic tone (e.g., the prolactin (PRL) response to the 5-HT releaser-uptake inhibitor fenfluramine) in demented patients with aggressive behaviors was compared with that of demented patients without aggressive behaviors.

METHODS

Subjects

All subjects were inpatients in the long-term care facilities associated with three hospitals. The Research Ethics Board of each hospital approved the study. After complete description of the study, the legal substitute decision-maker for each subject provided written informed consent.

Each subject received a comprehensive diagnostic assessment including physical, neurological, and psychiatric examinations. To be included in the study, patients had to meet the DSM-IV criteria for primary degenerative dementia (American Psychiatric Association 1994) and the NINCDS-ADRDA criteria for probable Alzheimer's disease (McKhann et al. 1984) of at least one year's duration. As well, patients had to have significant behavioral problems as demonstrated by a score of at least eight on the Neuropsychiatric Inventory (NPI) (Cummings et al. 1994), be at least 55 years of age and score less than 24 on the Mini-Mental State Examination (MMSE) (Folstein and Folstein 1975). Patients were recruited so that half of them had a clinical notation of aggression and showed aggressive behaviors and the remainder were nonaggressive. Patients were excluded if they had clinical or laboratory evidence of a significant medical illness or other medical/neurological condition which could diminish cognitive function, a Hachinski ischemic score >3 (Hachinski et al. 1975), or a brain computed tomographic (CT) scan that could not be interpreted as consistent with AD. In addition patients were excluded if they had evidence of significant cardiovascular disease or hypertension (> 160/100 mmHg), a premorbid or current psychiatric diagnosis (including major depression), contraindications to receiving fenfluramine, or administration of a depot antipsychotic within one treatment cycle of the baseline assessment.

Study Design

Patients who met the study entrance criteria at the screening visit as described above began a washout, during which all psychotropic and antiparkinsonian medications were discontinued. This washout was necessary in anticipation of initiating a new psychotropic trial. The length of washout was approximately one week, based on five half-lives of the psychotropic medication (or its active metabolite). After the washout, patients were given a fenfluramine challenge test to assess central serotonergic function.

Patients were assessed twice: at a screening visit (baseline) and after washout (test day). Patients were assessed for behaviors, cognition, function and depression. Behavioral disturbances were the primary outcome variable and were assessed with the Neuropsychiatric Inventory (NPI) (Cummings et al. 1994) that assesses 10 behavioral disturbances occurring in dementia patients: delusions, hallucinations, dysphoria, anxiety, agitation/aggression, euphoria, disinhibition, irritability/lability, apathy, and aberrant motor activity. Depression was assessed with the Cornell Scale for Depression in Dementia (Cornell) (Alexopoulos et al. 1988). Cognition was measured with the MMSE (Folstein and Folstein 1975). Functional ability was measured with the Functional Assessment Staging (FAST) scale, which measures functional loss in AD (Reisberg 1988). Clinical Global Impression (CGI) for the overall severity of the behavioral disturbances was assessed by both the physician and primary nurse. Secondary behavior scales included the Behavioral Pathology in Alzheimer's Disease (BEHAVE-AD) rating scale (Sclan et al. 1996) to measure overall psychopathology and the Cohen-Mansfield Agitation Inventory (CMAI) (Cohen-Mansfield and Billig 1986) which rates the frequencies of 29 types of agitated behaviors on a 7-point scale. Except for fenfluramine, no psychotropic medication was administered during the trial. There was provision for the administration of lorazepam if needed during washout, but use could not exceed four times per week.

Fenfluramine Challenge Test

While there are a variety of neuroimaging techniques available to visualize the serotonergic system in vivo, these are impractical in highly agitated individuals with severe cognitive impairment. The level of agitation in this group of severely impaired institutionalized individuals makes it highly unlikely that they would tolerate 60 or more minutes of lying motionless in a PET or SPECT scanner. Neuroendocrine challenge studies are advantageous in that they are minimally invasive, require cooperation for less time, and can be performed in the patient's familiar environment.

Changes in the plasma levels of anterior pituitary hormones in response to various serotonergic agonists have been widely used to study brain serotonergic function in psychiatric disorders (Risch et al. 1981). In this study, the prolactin response to d,l-fenfluramine challenge was used. Fenfluramine has been associated with cardiac valvulopathy after prolonged therapy (Connolly et al. 1997), but not single dose use, and behavioral side-effects have not been reported in patients with dementia (Mintzer et al. 1998). Food and fluids other than water were not permitted after midnight on the night before the fenfluramine challenge test day. On the morning of the test day patients were served a light breakfast and they were provided with juice and light snacks throughout the test period as both dehydration and hypoglycemia can affect prolactin secretion. Patients were encouraged to remain awake, recumbent and relaxed throughout the procedure as sleep, stress, and exercise can also alter prolactin secretion.

Tests commenced at approximately 8:00 A.M. and were conducted in the patients' own rooms. Two blood samples were obtained 15 min apart for determination of baseline prolactin concentration. Five minutes after the second blood sample (time 0) 60 mg of d,l-fenfluramine was administered orally. Blood samples for prolactin concentration were obtained 3 and 4 h later. Peak concentration is expected at 4 h, but the effect should be significant from 2 to 8 h after fenfluramine administration (Quattrone et al. 1983). The blood samples were collected into tubes containing aprotonin (Trasylol) to prevent degradation of hormones by plasma proteases. Samples were kept cool and serum was separated by centrifugation (1,000 × g for 10 min) within 2 h of being drawn. Serum was refrigerated until the determination of hormone levels by a two-site chemiluminometric (sandwich) immunoassay.

Blood samples for determination of plasma fenfluramine and norfenfluramine (active metabolite) levels were also collected at time 0 and 4 h after fenfluramine administration. These samples were collected into heparin coated tubes, refrigerated, and centrifuged within 2 h. Plasma was stored at −20°C. Concentrations of fenfluramine and norfenfluramine were determined by GCMS (gas chromatography mass spectography) by one of the authors who was blind to aggressiveness status of the patients (KR).

Data Analysis

Baseline plasma prolactin concentration (the mean prolactin concentration of the two blood samples taken before fenfluramine administration) and peak prolactin concentration (the maximum measured concentration after fenfluramine administration) were calculated for each patient. Prolactin response was calculated as percent change from baseline (peak prolactin concentration/baseline prolactin concentration) to adjust for differences in baseline prolactin. Relationships between baseline prolactin concentration, prolactin response, aggression and scores on the assessment instruments were determined using Spearman's rank correlation tests and multiple regression analyses. Aggression was also considered as a dichotomous variable. On the basis of the NPI aggression/agitation subscale (maximum 12) after washout, patients who scored 6 or higher on the NPI aggression subscale (at least once per week and at least moderate severity) at baseline were classified as “aggressive” (n = 11), while those who scored 4 or lower were classified as “non-aggressive” (n = 11). Prolactin response of the aggressive group of patients were compared with those of the non-aggressive group using independent samples t-tests.

RESULTS

Study Subjects

Forty-one patients who met preliminary inclusion criteria were referred or identified. Consent to participate was obtained for 26 of these patients. Of those 26, three patients dropped out before the fenfluramine challenge test and a fourth patient did not complete it because of problems drawing blood. Therefore, 22 patients completed the fenfluramine test day.

Baseline Characteristics

The baseline characteristics of the patients are shown in Table 1. The 22 patients were severely impaired as shown by both the MMSE and the FAST. They had no notable depression as demonstrated by their low scores on the Cornell scale for depression in dementia. There were no significant differences between the groups in any of the subject characteristics (Table 1), but the groups differed in level of aggression (Table 2). Table 3 shows the other behaviors that were present at the time of fenfluramine challenge.

Table 1 Demographics, Patient Characteristics at Baseline and Fenfluramine and Norfenfluramine Concentrations (n = 22)
Full size table
Table 2 Aggression after Washout in 22 Study Subjects
Full size table
Table 3 Correlation of NPI Subscales with Prolactin Response in 22 Study Subjects
Full size table

Prolactin Response to Fenfluramine Challenge

Relationship with Aggression

The mean prolactin concentrations as a percent of baseline are shown in Figure 1, panel A. The aggressive group had a significantly greater increase in prolactin following fenfluramine administration (215 ± 60%) than the non-aggressive group (123 ± 54%) (p < .01).

Figure 1
figure 1

A. Mean prolactin (PRL) concentration as % of baseline at each time point (n = 22). Baseline prolactin (prefenfluramine) measurements were taken at −15 and 0 min; fenfluramine was given at time 0 and prolactin concentrations were measured 3 and 4 h after fenfluramine challenge. ♦ 5 aggressive (n = 11), ▪ = nonaggressive (n = 11). B. Mean prolactin (PRL) concentration as % of baseline-gender effect. Baseline prolactin (prefenfluramine) measurements were taken at −15 and 0 min, fenfluramine was given at time 0 and prolactin concentrations were measured 3 and 4 h after fenfluramine challenge. ♦ = aggressive females (n = 4), ▪ = aggressive males (n = 7), ▴ = nonaggressive males (n = 5), • = non aggressive females (n = 6).

Full size image

The relationship between prolactin response following fenfluramine challenge and aggression was confirmed by significant correlations between prolactin response (expressed as percentage of baseline) and scores on the Neuropsychiatric Inventory aggression/agitation subscale (rs = 0.61, p = .003) (Figure 2) and irritability subscale (rS = .53, p = .012) and the aggressiveness axes of the BEHAVE-AD (rs = 0.47, p = .03). Score on the CMAI aggressiveness subscales (sum of verbal/aggressive and physical/aggressive axes) was not significantly correlated to prolactin response.

Figure 2
figure 2

Scatterplot of prolactin response as % of baseline versus NPI aggression agitation subscale, rho = 0.61, p = .003.

Full size image

Specificity for Aggression

Although overall level of psychopathology (NPI total) was also correlated with prolactin response, the aggression subscale of the NPI accounted for more variability (Table 3). Furthermore, there were no correlations between prolactin response following fenfluramine challenge and the nonaggressive subscales of the NPI except irritability (Table 3).

Gender Effect

There were significant differences between the responses of the male and female subjects in the fenfluramine challenge test. Female aggressive subjects had a more rapid and robust increase in prolactin concentration as a percentage of baseline (216 ± 70%, n = 4) than nonaggressive females (97% ± 40, n = 6). In contrast, while the aggressive males had a robust increase in prolactin concentration (215% ± 60, n = 7), nonaggressive males also had a small increase in prolactin concentration (155% ± 53, n = 5) (Figure 1, panel B). Regression analysis confirmed that the prolactin response to fenfluramine challenge was significantly different for males and females (p = .003) and also the effect of gender was different for aggressive and non-aggressive patients, and the effect of aggression was different for males and females (p = .0036 for interaction in regression analysis). The overall model was significant (p = .006) and accounted for 49% of the variance (r = 0.70).

Secondary Outcome Variables

Prolactin response was not significantly correlated with baseline measures of sedation, Clinical Global Impression by caregiver or investigator or body weight (Spearman rank rho, all p > .3). Prolactin response was however, correlated with baseline scores on the Cornell scale for depression in dementia (rS = .47, p = .03), perhaps reflecting the Cornell's assessment of symptoms of agitation, and MMSE (rS = .54, p = .01). In a regression analysis, MMSE was a strong predictor of prolactin response (p = .0004). In the same regression model, there was again a significant interaction between the effects of gender and aggression (p = .015) but neither gender (p = .08) or aggression (p = .13) alone were significant. The overall model with MMSE was significant (F = 11.9, p = .0001) and accounted for 74% of the variance (r = 0.86).

Plasma Fenfluramine Concentrations

Blood samples to measure plasma fenfluramine and norfenfluramine concentrations were taken 4 h after fenfluramine administration (see Table 3 for values). Plasma concentrations of fenfluramine or norfenfluramine were not significantly different for the two levels of aggression, sizes of prolactin response or gender (3-way ANOVA, for main effects p = .18 to .74). Thus, the relationships between these variables were not due to pharmacokinetic differences between the groups.

Baseline Prolactin Concentration

Baseline prolactin levels (mean of two pre-fenfluramine concentrations) for the aggressive group (x ± SD = 8.6 ± 3.4 μg/L) were significantly lower than those of the non-aggressive group (16.5 ± 9.2 μg/L) (p < .05). Baseline prolactin concentrations were also correlated with scores on the Neuropsychiatric Inventory aggression/agitation subscale (rS = −0.53, p = .012) and the aggressiveness axis of the BEHAVE-AD (rS = −0.66, p = .001). There were no significant relationships between baseline prolactin levels and scores on the irritability subscale of the NPI or the aggressiveness subscales of the CMAI.

DISCUSSION

This study demonstrated that prolactin response to fenfluramine challenge may be directly related to aggression in patients with behavioral disorders and severe cognitive impairment associated with Alzheimer's disease. The finding of a greater prolactin response to fenfluramine challenge in aggressive AD patients compared with nonaggressive patients is contrary to the results of other studies in males with personality disorders in which impulsive aggression was associated with a blunted prolactin response (Coccaro et al. 1997). However, increased response to fenfluramine challenge was found in impulsive/aggressive polysubstance abusers with antisocial personality disorder, and was interpreted as evidence of post-synaptic supersensitivity to fenfluramine (Fishbein et al. 1989). Importantly, these results are also consistent with those of another group who found a similar relationship between agitation and prolactin response in 10 AD patients (Mintzer et al. 1998). However, that study had a smaller number of patients, did not look at aggression specifically and their sample was predominantly female.

Since fenfluramine releases serotonin from presynaptic stores (van Praag 1991), greater functional serotonergic responsivity to fenfluramine stimulation, indicated by higher plasma concentrations of prolactin, could reflect either increased intracellular presynaptic availability of 5-HT, changes in postsynaptic receptor sensitivity, or a combination of both. This study in AD patients cannot determine whether the responses of the aggressive patients were high or the responses of the nonaggressive patients were blunted without an age-matched control group but it is more plausible that aggressive AD patients have a functionally hyperresponsive 5-HT system leading to an increased response. This could be due to intrinsic damage to the serotonergic system with compensatory upregulation of the remaining postsynaptic receptors, similar to denervation supersensitivity seen in animals after 5-HT lesioning (Siever et al. 1991). The possibility of denervation supersensitivity in AD is supported by evidence of loss of serotonergic neurons (Mann and Yates 1983; Yamamoto and Hirano 1985), decreased concentrations of 5-HT and its major metabolite in AD (Adolffson et al. 1979; Arai et al. 1984; Cross et al. 1984) and previous neuroendocrine studies (Lawlor et al. 1989; McLoughlin et al. 1994). None of those studies specifically assessed behavioral disorders. There is also evidence that normal aging leads to a blunted prolactin response (McBride et al. 1990), further supporting the hypothesis that aggressive patients had an increased response.

This study also found that the prolactin response to fenfluramine challenge was greater in females than in males. Other studies (Coccaro and Kavoussi 1997; Malone et al. 1996; McBride et al. 1990) have found females to have an elevated response to fenfluramine challenge. Response is postulated to be related to estrogen levels and menstrual cycle in pre-menopausal women (Markianos et al. 1996; O′Keane et al. 1991). However, all females were post-menopausal and only one of the female AD patients in the present study was on hormone replacement therapy (medroxyprogesterone 2.5 mg/d). Females are thought to have increased serotonin function, storage capacity and synthesis compared with males (Charney et al. 1988). These results suggest that gender-related difference in prolactin response persist throughout the life-time and are independent of menstrual status.

Although aggression subscales on the NPI and BEHAVE-AD were correlated with prolactin response results, aggression scores on the CMAI (p = .05) was not. The Cohen Mansfield Agitation Inventory (CMAI) (Cohen-Mansfield and Billig 1986; Cohen-Mansfield et al. 1989) is designed specifically for nursing home residents and assesses verbal, vocal and motor agitation. It does not have a subscale specifically for aggressive behaviors. Whereas both the NPI and the BEHAVE-AD measure severity of behaviors, the CMAI measures frequency, but not severity. Thus, severe aggression may not be differentiated from mild aggression sufficiently on this scale to be correlated with central serotonergic function. Aggression can occur in a variety of situations (Ware et al. 1990) including only during care (“provoked”) and aggression that is more impulsive such as attacking a sleeping patient (“unprovoked”). It may be that only certain types of aggressive behavior, those on the more impulsive side, are closely linked to serotonergic function. The NPI irritability scale, which looks at some behaviors that could be considered impulsive, was also correlated with central serotonergic function supporting this.

The concentrations of prolactin prior to fenfluramine challenge were lower in both male and female aggressive AD patients than in nonaggressive AD patients. Prolactin secretion is influenced by several neurotransmitters, neuropeptides and hormones, and by external factors such as medications, stress, sleep and exercise. It is unlikely that prechallenge prolactin could reliably reflect central serotonergic tone.

Of note, in this group of patients, serotonergic hyperresponsivity was greater in patients with less severe cognitive impairment. Although there are no studies looking at the serotonergic system over the course of AD, evidence from post-mortem tissue (where AD has usually run its full course) combined with ante-mortem evidence (where the AD progression is incomplete) suggests that disruptions in the serotonergic system increase with severity of AD (Palmer 1996). This may have implications for treatment suggesting that the serotonergic system in patients with advanced AD may not be intact enough to respond to serotonergic replacement strategies. Recent evidence from randomized controlled trials are compatible with this interpretation. The randomized placebo-controlled trials that showed positive results with SSRIs in the treatment of behavioral disturbances involved patients with mild AD (Nyth and Gottfries 1990) and moderate-severe dementia (Pollock et al. 2002). A randomized controlled trial of institutionalized patients with advanced AD treated with sertraline found that only 36% (8/22) of patients were responders (Lanctôt et al. 2002). Thus, serotonin uptake inhibitors may be more effective earlier in the course of AD.

In this study, patients were free of psychotropic medications for at least one week prior to fenfluramine challenge, and this interval was greater than five half-lives of their previous psychotropic medications. Despite washout, there is a possibility that prior use of psychotropic medications confounded fenfluramine challenge results. In this population, since both aggressive and nonaggressive patients had BPSD, all patients had recent histories of psychotropic drug utilization except three (two nonaggressive, one aggressive). While the impact of previous drug use on the CNS cannot be measured, this potential confounder was present in both aggressives and nonaggressives.

CONCLUSIONS

In conclusion, the results of this study suggest that altered central 5-HT activity is related to aggression in AD patients. The increased prolactin response to fenfluramine may be evidence for serotonergic hyperresponsivity in these patients, and suggests the possibility that serotonergic strategies may be efficacious for this condition. However, if the denervation supersensitivity hypothesis is correct, exacerbation of aggression may initially be seen with such treatment, and decreases on aggression would be expected only as the supersensitivity decreased. Furthermore, patients with advanced Alzheimer's disease may not benefit from serotonergic replacement therapies. This study links aggression with serotonergic differences in vivo and as such contributes to the knowledge of the neurobiology of behavioral disorders associated with dementia.