Main

Schizophrenia is a disease with a highly variable course and symptoms. On broad-ranging symptom inventories such as the Brief Psychiatric Rating Scale (BPRS) (Overall and Gorham 1988) and Positive and Negative Symptom Scale (PANSS), patients typically display a wide variety of disordered behavior of varying severity. With some variation in how symptoms are clustered together, factor analytic studies have suggested the presence of certain broad symptom domains (reality distortion, disorganization, negative symptoms and anxiety/depression) underlying this heterogeneity (Buchanan and Carpenter 1994). Sources of variability in the symptom domains reported include: (1) potential differences in pathology and symptom associations among distinct patient populations; (2) differences in the instruments used, with some (e.g., Scale for Assessment of Positive Symptoms (Andreasen 1984)) and Scale for the Assessment of Negative Symptoms (Andreasen 1989)) omitting the affective domain entirely, despite the common presence of these symptoms in patients with schizophrenia; and (3) variations in factor analytic methods. In addition, only a few reports (Arndt et al. 1995; Harvey et al. 1996; Goldman et al. 1991) have addressed whether factor solutions are stable over time despite medication changes.

The effectiveness of medications may vary across these symptom domains, so that domain-specific symptom scores are logical outcome measures for clinical trials. Symptom domain scores used in psychiatric clinical trials, however, have been inconsistent in the symptoms included. Nicholson et al. (1995) found no less than ten BPRS-based definitions of positive symptoms and four definitions of negative symptoms in recent psychiatric literature. Commonly used BPRS scorings group symptoms of disorganization and reality distortion together in a “positive” symptom score, despite the emerging evidence that these symptoms fall into two distinct domains.

The goal of the current study is to develop and validate domain-specific BPRS symptom scores suitable for use as outcome measures in clinical trials in patients with treatment resistant schizophrenia. The 20–30% of patients with schizophrenia who do not show positive symptom response to first generation antipsychotic medications (Prien and Cole 1968; Davis and Casper 1977; Essock et al. 1996) may have a distinct form of the disease. Using explicit criteria for treatment resistant schizophrenia to compare such patients to those with responsive illness, patients with treatment resistant disease have been found to have increased cortical atrophy (Ota et al. 1987; Stern et al. 1993; Bilder et al. 1994) and lower levels of catecholamines in cerebrospinal fluid (van Kammen and Schooler 1990). Lack of response to early treatment is predictive of non-response with other treatments (Lieberman 1993; Stern et al. 1993). A decade ago treatment resistance was largely defined as persistent positive symptoms (Kane et al. 1988), and while this remains true it is well recognized that patients with this characterization demonstrate other persistent symptoms as well. Recently, disorganization has been reported to be higher in patients with treatment resistance than those who are partial or full responders to antipsychotic medication (Rodriguez et al.1998). We focused on the BPRS because, with its wide coverage of symptom domains and relatively low patient and staff burden to collect, the BPRS is among the most commonly used outcome measures in clinical trials in schizophrenia (Thornley and Adams 1998). Because the symptom patterns and neurobiology of treatment resistant schizophrenia may be distinct from those found in other schizophrenic populations, BPRS factor models from mixed patient samples (Overall and Klett 1972; Guy 1976) may not represent the optimal symptom cluster scores for assessing treatment response in this population.

The current study used two large data sets of BPRS ratings from patients with treatment resistant schizophrenia. Both data sets were used to evaluate the fit of several widely-used BPRS factor models using confirmatory factor analysis (CFA). The first data set was used to develop new symptom scorings appropriate to this population and to evaluate the longitudinal stability of this factor solution six months post-clozapine. The second data set was used to provide an independent validation of the novel factor model developed in the Clozapine Authorization and Monitoring Program (CAMP) patients’ data. Data used for these analyses was collected in compliance with the principles of the Declaration of Helsinki, as described below for each patient group.

METHODS

Maryland Clozapine Data Base

All patients prescribed clozapine in Maryland state inpatient facilities, state-supported outpatient clinics, or through the state Medical Assistance or Pharmacy Assistance programs were included in the CAMP database. To receive clozapine, a patient had to meet the following criteria: (1) DSM-III-R or DSM-IV criteria for the diagnosis of schizophrenia or schizoaffective disorder; (2) a clinical trial of at least two different chemical classes of antipsychotic agents; and (3) treatment resistance determined by the prescribing physician. In a minority of cases, patients who were intolerant of conventional neuroleptics were offered a trial of clozapine. Candidates were excluded for a history of a drug-induced blood dyscrasia, an uncontrolled seizure disorder, a white blood cell count of less than 3500 per ml, or a history of a myeloproliferative disorder.

Data collected on CAMP patients included patient demographics (age, gender, ethnicity), clinical data (BPRS scores at baseline and various time points, monitoring parameters, diagnosis), and antipsychotic medication usage. A 20-item version of the BPRS (the standard Overall and Gorham (1988) 18-item instrument plus ratings of poverty of thought and inappropriate affect) was used to assess changes in psychiatric symptoms. Clinicians who began treating patients with clozapine prior to June 30, 1993, were required to participate in one-day training sessions on clozapine administration guidelines and BPRS scoring. Videotapes of patient interviews were available to clinicians to practice use of BPRS at study initiation. Anchor guidelines for BPRS scoring were given to each clinician participating in the clozapine program. Treating clinicians were required to complete the BPRS at baseline, at three and six months and every six months thereafter.

Of the 1542 patients included in the CAMP database who received clozapine between January 1, 1989 and December 31, 1997, 10 patients less than 12 years of age were excluded from this study, as were patients without a baseline BPRS score (n = 227) or with missing ratings for some items (n = 83). In addition, 52 patients with diagnoses other than schizophrenia/schizoaffective disorder were dropped from the analysis, and 92 patients who also had BPRS ratings from the TRU (see below) were excluded from this analysis to eliminate overlap between patients in the two data sets. Thus, the current report includes 1074 CAMP patients, of whom 834 also had complete follow-up (collected an average of 231 days after baseline) BPRS ratings. Of these, 750 (70%) baseline and 601 (72%) follow-up records were completed by psychiatrists who had completed the CAMP BPRS training program.

In accordance with Federal Regulation 45 CFR 46.101.b, analyses of the CAMP data base were determined to be exempt from requirements for formal IRB approval.

MPRC Treatment Research Unit Data Base

The Treatment Research Unit (TRU) is an in-patient research facility at the Maryland Psychiatric Research Center, on the grounds of Spring Grove State Hospital in Catonsville, Maryland. BPRS scores from 197 patients admitted to the TRU between January 1990 and February 2000 meeting criteria for treatment resistant schizophrenia were used for this analysis. Diagnoses were determined by Structured Clinical Interview for DSM-III-R or DSM-IV Axis I Disorders (SCID) criteria. Patients were between the ages of 18 and 65 and were determined competent to sign informed consent. Treatment resistance was defined as: persistent positive symptoms (item score 4 on at least two of four positive symptom items on the BPRS; a total score ⩾45 on the (18 item) BPRS and a score ⩾4 on the Clinical Global Impression (CGI); no period of good functioning in the last five years; and at least two failed trials of antipsychotic medication (600 mg CPZ units for at least six weeks). Of these 197 patients, 135 subjects from the TRU participated in a clinical trial with traditional antipsychotic given as a lead-in to verify treatment resistance. The BPRS score used in this analysis was obtained subsequent to the antipsychotic lead-in. Fifty-two TRU patients used in this analysis were not included in clinical research protocols for various reasons (e.g. noncompliance, elopement, clozapine nonresponse, uncooperative, suicidal, current substance abuse, serious medical condition). The initial BPRS completed upon admission was utilized for this analysis in these patients. TRU staff performing BPRS ratings participated in an on-going program to maintain inter-rater comparability, requiring interclass correlations on BPRS total scores ⩾0.8 among raters and versus “gold standard” consensus ratings of test tapes.

All patients included in this analysis had given informed consent to collection of data for research purposes in protocols approved by the University of Maryland Baltimore Institutional Review Board. In each case, after complete description of the study in question to the participants, written informed consent was obtained.

Statistical Methods

This analysis used only the original 18 BPRS items (Overall and Gorham 1988). Baseline BPRS scores among CAMP and TRU patients were compared with Wilcoxon tests (Conover 1980). We used confirmatory factor analysis (CFA) to: (1) evaluate how well several published factor models fit our data (Overall and Klett 1972; Guy 1976; Harvey et al. 1996 (see Table 1 for details of each model)); and (2) find a modification of these models with better fit. With CFA, pre-specified models determined how many factors were fitted, and which items would have zero or non-zero loadings on each factor. Subject to these constraints, estimates were made of the loadings for all variables on their assigned factors using a weighted least squares method that does not assume multivariate normality. Deviations between the observed correlations among items and those specified in the factor model contribute to a global lack of fit chi-square. In large data sets, this lack of fit chi-square is almost always significant, and other indices of goodness of fit—Bentler's comparative fit index and Bentler and Bonnet's non-normed fit index (NNFI)—are more useful for comparing the fit of pre-specified models. Bentler and Bonnet suggest that factor models could be considered well-fitting if the CFI and NNFI are 0.90 or higher. In addition to global measures of fit, CFA can also be used to identify the specific items which make the largest contributions to the global lack of fit chi-square. Modified models, which delete an item or make it load on a different factor, can be considered, and the goodness of fit measures recalculated. We carried out this model modification process one variable at a time, dropping the worst-fitting item at each step. If only one item was retained for a given factor, the factor and associated items were dropped. The exploratory search for an improved factor model was performed using pre-clozapine baseline BPRS records from CAMP patients. Because measuring fit of the final model on the same data used to explore alternate models may over-estimate the fit of the final model, BPRS records from TRU research participants were used to independently validate the model fit. Principal components analysis was performed to estimate the number of important factors present using the scree plot and eigenvalue >1.0 criteria (Hatcher 1994). To assess the effect of treatment with clozapine on the BPRS factor structure, CFA was used to calculate the CFI and NNFI measures of fit on the six-month post-clozapine BPRS scores from CAMP patients for each of the factor models considered. CFA models were fitted with the SAS® statistical package, using the CALIS procedure (SAS Institute Inc. 1990, 1997; Hatcher 1994).

Table 1 Models for Scoring Schizophrenic Syndromes Based Upon BPRS Items
Full size table

RESULTS

CAMP patients with baseline BPRS ratings were similar in mean age to TRU patients (37.6 ± 10.3 versus 38.5 ± 9.3 years), although there was a wider age range among CAMP patients (14.4 to 81.3 years) than TRU research participants (18 to 62 years). Males were 63.5% of CAMP patients, compared with 66.0% of TRU research participants. The percentages of CAMP patients classified as white, black and “other” were 68.2%, 28.1% and 3.7%, respectively, compared with 66.0%, 33.0% and 1.0% among TRU research participants.

Mean baseline BPRS total scores (± s.d.) were 62.8 ± 16.8 among CAMP patients and 57.1 ± 12.1 among TRU research participants (χ2 = 19.12, df = 1, p < .001). On individual BPRS items (Table 2), CAMP and TRU patients were comparable in mean scores on psychotic symptoms (hallucinations, unusual thought content), but CAMP patients had higher scores on average, in such areas as conceptual disorganization, emotional withdrawal, blunted affect, depression, tension and uncooperativeness. With few exceptions (hostility, suspiciousness), a higher percentage of CAMP than TRU patients were rated as “severe” or “very severe” for each of 18 symptoms on the BPRS. Few patients were rated as having core psychotic symptoms (hallucinations, unusual thought content) “not present.”

Table 2 Brief Psychiatric Rating Scale (BPRS) Item Scores in Maryland Clozapine Data Base (CAMP) Patients at Pre-Clozapine Baseline And Treatment Research Unit (TRU) Research Participants
Full size table

Compared with the 834 CAMP patients with BPRS follow-up data, the 240 patients without six month follow-up BPRS scores did not differ significantly from those with follow-up data in BPRS total score (mean ± s.d., 59.4 ± 16.2 versus 57.6 ± 14.4, Wilcoxon χ2 = 0.04, df = 1, p = .84), or on any BPRS item (data not shown), but were on average older (39.1 ± 10.6 versus 37.1 ± 10.2 years, Wilcoxon χ2 = 7.49, df = 1, p = .006), and were more likely to be female (43.6% versus 34.4%, χ2 = 6.73, df = 1, p = .01) or black (38.0% versus 25.0%, χ2 = 16.83, df = 5, p = .005). Of the 240 patients without follow-up data, 33 were discontinued for non-compliance or patient refusal, 11 for lack of efficacy, 49 for possible adverse effects, 12 for other reasons, and 135 for unknown reasons.

Principal components analysis of the 18-item BPRS in the baseline CAMP patient data suggested the presence of four components with eigenvalues >1.0 (factors which would account for more of the combined variance than any individual item) (Figure 1 ). Exploratory factor analysis of the baseline CAMP patient data did not reveal a factor solution with simple structure (i.e, each item loading on a single factor). Similar results were found in six-month follow-up assessments from CAMP patients, and in TRU participant data. Accordingly, confirmatory factor analysis was used to see whether published BPRS factor models which assign items to a single symptom domain adequately fit to the data.

Figure 1
figure 1

Lines connect scree plots of eigenvalues from the 18-item BPRS data from CAMP patients at baseline and six months, and from TRU research participants. The size of the eigenvalues is proportional to the percentage of the total variance of the 18 items accounted for by the corresponding principal components. A reference line is drawn at an eigenvalue of 1.0, the level at which a principal component accounts for no more of the total variance than one of the original items.

Full size image

The factor models considered are listed in Table 1, and measures of fit for each are summarized in Table 3. In the model-building analyses in the baseline CAMP patient data, the best fitting models identified were: the Overall and Klett 4-factor model (1972) (CFI = 0.90, NNFI = 0.86); a 4-factor “treatment resistant” schizophrenia model (CFI = 0.91, NNFI = 0.88), which retained factors for: anxiety/depression, negative symptoms, reality distortion and disorganization; and a treatment resistant 3-factor model, which dropped the anxiety/depression factor (CFI = 0.93, NNFI = 0.90).

Table 3 Summary of Measures of Factor Model Fit in Maryland Clozapine Data Base and Treatment Research Unit Data Base
Full size table

In the model validation analyses of the TRU patient data, the fit of all the models considered showed little change compared with that in the baseline CAMP patient data used for model development. The 3-factor treatment resistant model showed the most improvement in fit in the TRU data base. The goodness of fit statistics (Table 3) and estimated factor loadings (Table 4) for the 4-factor treatment resistant model are largely comparable in the data from CAMP and TRU patients.

Table 4 Estimated Factor Loadings for 4-Factor Treatment-resistant Model
Full size table

In CAMP patients, factor-based scores for reality distortion showed weak correlations with anxiety/depression (r = 0.27, p < .001) and negative symptoms (r = 0.17, p < .001). Disorganization scores were correlated with scores for both negative symptoms (r = 0.46, p < .001) and reality distortion (r = 0.42, p < .001). In the TRU participant data, correlations were generally similar, although smaller, with the exception that a negative correlation (r = –0.17, p < .001) was found between negative symptoms and anxiety/depression.

Longitudinal Stability of the BPRS Factor Structure

The fit of all the factor models improved between the baseline and six months post-clozapine ratings for CAMP patients (Table 3), and the relative ordering of fit for the models considered did not change. Factor loadings for the 4-factor treatment resistant model were very similar at pre-treatment baseline and six months post-clozapine (Table 4).

DISCUSSION

This is the first study to report on the factor structure of schizophenic symptoms in large samples of patients with treatment resistant schizophrenia unresponsive to conventional antipsychotic medications. Despite the evidence that the underlying neurobiology of schizophrenia may be different in patients with treatment resistant schizophrenia compared with patients whose symptoms respond to conventional antipsychotics, in two large, independent samples of patients with treatment resistant schizophrenia, the best fitting model identified was a 3-factor model which appears to correspond to the “three syndrome” model (reality distortion, disorganization and negative symptoms) for core features of schizophrenia discussed by Buchanan and Carpenter (1994), Arndt et al. (1991) and others. A model adding an anxiety/depression factor to the first three fit almost as well. The Overall and Klett (1972) model was the best fitting of the three published models considered. The fit of these showed little change between baseline and six months after starting treatment with clozapine, suggesting that the correlations among schizophrenic symptoms rated on the BPRS are stable over time despite treatment with a novel antipsychotic.

Comparison to Other Factor Ratings

The major difference between the new factor structures developed in this study and other widely used BPRS factor solutions (Overall and Klett 1972; Guy 1976) is that our model does not cluster hallucinations, delusions and conceptual disorganization together in one “thought disturbance” or “positive symptom” factor, but assigns these items to two distinct domains, reality distortion (hallucinations, delusions, grandiosity, suspiciousness) and disorganization (conceptual disorganization, disorientation, mannerisms and posturing). A similar differentiation between reality distortion and disorganization has been suggested from analyses of symptoms clusters assessed with more extensive symptom inventories (Arndt et al. 1991 (SAPS); Lancon et al. 2000 (PANSS)) in patient samples not selected for treatment resistance, as well as from consideration of the association of symptoms with their anatomic, functional and other correlates (Buchanan and Carpenter 1994). If the distinction between reality distortion and disorganization reflects distinct biological processes, treatment response could be assessed more informatively by separate measures of these two domains, rather than by a “positive symptom” factor which includes elements of both. Harvey et al. (1996) reported a similar division in a model derived from exploratory factor analysis. The symptoms loading on several of their factors differed from our model, and their five-factor model did not fit well in their original data set; nor did it fit as well in our data as the other models considered.

Aside from this major difference, our solution has a number of other changes in how symptoms are grouped together, which contribute to the improvement of fit between our model and other published BPRS factor solutions. Our reality distortion factor includes suspiciousness, in line with the proposal of Kane et al. (1988) that in patients with treatment resistant schizophrenia suspiciousness should be scored as a “positive” symptom. The negative symptom and anxiety/depression factors identified in this study are similar to those found in other studies of symptom clusters in schizophrenia (Overall and Klett 1972; Guy 1976), although we drop somatic concerns from the anxiety/depression factor. A factor corresponding to the anxiety/depression symptom cluster is commonly found in ratings of schizophrenic patients if the instrument used (e.g., PANSS, BPRS) assesses this domain. Therefore, despite slightly better fit of the three factor model, we do not recommend dropping the anxiety/depression factor. In our data, tension, hostility, uncooperativeness and excitement were no more correlated with each other than each was with several of factors retained in the four factor treatment resistant model (data not shown). We suggest that symptom response in these clinically important areas be assessed using the individual items, since they did not cluster together as a distinct symptom domain.

The stability of our factor structure before and after treatment with clozapine supports the assertion that these factors retain their internal validity despite treatment effects, and may be used to give meaningful summaries of treatment effects on symptoms. Stability of schizophrenic symptom clusters over time despite medication effects has also been reported in schizophrenic patients not selected for treatment resistance (Arndt et al. 1995; Goldman et al. 1991; Harvey et al. 1996).

Limitations

Some CAMP patients were selected for intolerance to conventional neuroleptics, rather than unresponseness of psychotic symptoms to treatment. Not all psychiatrists performing BPRS ratings on CAMP patients received standard training, although if the data analyses were restricted to BPRS scores from psychiatrists completing the CAMP training program, the results were virtually identical to those presented (data not shown). Inter-rater reliability was not assessed. In view of these limitations, it is important that the findings in the CAMP patient data were fully validated in the TRU participant data. TRU participants met more rigorous criteria for true treatment resistance, and in the TRU an on-going program of inter-rater reliability assessment was conducted to maintain comparability. More rigorous inter-rater comparability may account for the rare finding that in the TRU validation sample the treatment resistant factor model fit even better than in the CAMP baseline data used to develop this model. Improved assessment of the same patient over time (reduced rating error) likewise could account for the better fit of the models in the CAMP patients at six months post-clozapine. An alternate explanation might be diminished patient heterogeneity in the group followed longer; however, this is not supported by the absence of significant baseline symptom differences on any BPRS item between those with and without follow-up data. None of the factor models for treatment resistant schizophrenia fully met both statistical criteria for excellent fit (NNFI and CFI > 0.9) in all analyses. This may reflect the limits of the symptoms rated on the BPRS in discriminating between symptom domains.

In general, mean symptom scores were higher in CAMP patients than TRU research participants, especially on tension, uncooperativeness, conceptual disorganization and negative symptoms. Patients who are severely symptomatic on these items may be less likely to be able or willing to give consent to participate in clinical research. Inclusion of such patients may give the finding from the CAMP data some advantages in generalizability over data gathered from participants in clinical research, despite the lesser training of raters and inclusion of some patients selected for intolerance to conventional neuroleptics rather than treatment resistant schizophrenia.

The 18-item BPRS does not measure some symptom patterns common in patients with schizophrenia (e.g., deficits in cognition or premorbid social adjustment), and may offer only a few items measuring a particular symptom complex. It is possible that a more extensive battery with more specific symptom items might have detected additional factors not included in our model or subdivided some of our symptom domains, such as negative symptoms (Stuart et al. 1999).

CONCLUSIONS

We found that symptoms rated on the BPRS may be reliably grouped into four clusters: reality distortion, disorganization, negative symptoms, and anxiety/depression (see Table 4 for items in each domain). Clinical trial outcomes are often analyzed using a “positive symptom” score which groups together elements of reality distortion and disorganization, even though these domains are widely thought to reflect distinct underlying neurobiological processes in schizophrenia. In two large, independent groups of patients with treatment resistant schizophrenia, our factor structure fit the data as well or better than other published BPRS factorings, and was stable over a six-month period before and after administration of clozapine. We recommend that future clinical trials among patients with treatment resistant schizophrenia use these four symptom clusters to measure treatment response.