Introduction

Neuropathological discoveries of the early 1900s1, 2 identified gross and cellular neuropathology changes with the classical degenerative diseases. Despite extensive research no single neuropathological signature has been found with the mental illnesses. Although there may be no gross or cellular neuropathology with mental illnesses, the issue of molecular neuropathology remains a question. With improvements in the level of investigative technology there has been renewed interest in identifying this pathology with some success. Both schizophrenia and bipolar disorder gene expression analyses have identified changes in genes encoding mitochondrial3 and synaptic proteins.4, 5, 6 However, evidence of molecular neuropathology that is consistently replicated in different cohorts is lacking in the field. One hindrance to achieving additional insights on molecular pathology is the limited post-mortem research being conducted on mental illness, most likely due to the scarcity of available tissue. An additional confound is the lack of reliable clinical information, which is important to interpreting the meaning of the biological results. In clinical psychiatric research, there are many well-validated clinical instruments to measure a wide variety of psychiatric symptoms. The same cannot be said of instruments used to collect retrospective information. It is difficult to address this issue because the nature of data collection relies, in part, on information obtained by an informant that is retrospective and subject to the vagaries of memory, as well as the closeness of relationship between subject and informant. Thus, obtaining accurate informant descriptions of lifetime psychiatric diagnoses, behavioral traits and clinical symptoms for donors of post-mortem brain tissue can be extremely difficult.

The goal of this project was to better understand if individuals who know a subject well and are often the informants for post-mortem brain research can accurately describe the mood state and identify lifetime psychiatric symptoms of their family member. To accomplish this goal, we interviewed a subject with a known axis 1 diagnosis and an informant who was their next-of-kin (NOK) using the same diagnostic and symptom severity scales. The goal was to determine the level of concordance of answers between subject and NOK pairs of well-established clinical instruments.

Materials and methods

All research was approved by the University of Texas Health Science Center Institutional Review Board and was performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki. The interview with the subject occurred in person in a University office and the NOK interview was conducted either by telephone or in person. Subject recruitment was from the patient Mood Disorders Clinic at the University of Texas Health Science Center San Antonio or via advertisement. The inclusion criteria for subject participation were (1) a psychiatric diagnosis of bipolar disorder 1, major depression or schizophrenia and (2) a NOK who had regular contact with them and was willing to participate in the research. In the first set of 10 subject–NOK pairs, PMT interviewed the subject and CGB interviewed the NOK. For the second 10 sets, the interviewers were reversed.

All subjects and informants were administered the following instruments; Mini International Neuropsychiatric Interview (MINI),7 Barratt Impulsiveness Scale,8 Childhood Trauma Questionnaire (CTQ),9 Montgomery Asberg Depression Rating Scale,10 Hamilton Depression Rating Scale, 31 question (Ham-D31),11, 12 Brief Psychiatric Rating Scale (BPRS)13 and the Bipolar Inventory of Symptoms Scale (BISS).14 With the seven NOK who were not married or engaged to the subjects, we did not ask sex-related questions. One subject did not complete the BPRS, and two subjects did not complete the Barratt Impulsiveness Scale. Nine NOK did not have knowledge of childhood events in the subjects and did not complete the CTQ.

Statistics

Based on the MINI, we categorized subjects and informants according to psychiatric diagnosis and we assessed diagnostic agreement. We reported results as proportions of either positive or negative agreement15, 16 with 95% confidence intervals.17 That is, with a 95% level of confidence the range of values contained the ‘true’ proportion. As a result of the limited sample, we combined the alcohol abuse and alcohol-dependent results into alcohol use disorder, and the drug abuse and drug dependence into drug use disorder. We analyzed summative scores for the seven symptom severity scales administered to both subject and informant. We assessed agreement between subject and informant for the BPRS, BISS, BISS subscales, Montgomery Asberg Depression Rating Scale, Ham-D, Young Mania Rating Scale, CTQ and Barratt Impulsiveness Scale using intra-class correlation coefficients (ICC) and 95% confidence intervals.18, 19 We additionally assessed agreement between mean scale scores for subjects and informants using two one-sided equivalence testing.20

Results

Table 1 shows the demographic information. The subjects were 30% Hispanic and 70% Anglo with 60% female and 40% male. Although in the NOK, 35% were Hispanic and 55% Anglo. Table 2 shows the proportion of positive and negative agreement between subject and informant based on responses to the MINI. The diagnostic positive agreement values range from 0.25 for alcohol use disorder to 0.97 with major depression. Diagnostic negative agreement ranged from 0 with major depression to 0.97 with obsessive compulsive disorder.

Table 1 Demographic information on subjects and NOK
Full size table
Table 2 Proportion of positive and negative agreement between subjects and NOK with 95% confidence interval using the MINI semistructured exam for DSM-IV diagnoses
Full size table

Agreement between subject and informant symptom severity scale scores is shown in Table 3. Concordance rates were as follows: BISS=0.59 (0.23, 0.81), Ham-D=0.44 (0.03, 0.72), Young Mania Rating Scale=0.0.61 (0.26, 0.82), Montgomery Asberg Depression Rating Scale =0.0.44 (0.03, 0.72), BPRS=0.58 (0.19, 0.81), CTQ=0.48 (−0.15, 0.83) and Barratt Impulsiveness Scale=0.36 (−0.11, 0.70). Subdividing the BISS into its factor components21 showed moderate concordance, with the mania, irritability and anxiety factors showing the greatest concordance and the depression factor the least (Table 3). We found the Barratt Impulsiveness Scale mean scores and the BPRS mean scores for subject and informant statistically equivalent (P=0.005 and P=0.02, respectively).

Table 3 Intra-class correlation coefficients with 95% confidence intervals between the subject and NOK for BISS, BPRS, MADRS, Ham-D, YMRS, CTQ, BISS illness subscale and Barratt Impulsivity Scale
Full size table

Discussion

We report that informant-gathered information on individuals with a major mental illness can identify most severe lifetime Diagnostic and Statistical Manual (DSM) diagnosis. The notable diagnostic exceptions are generalized anxiety disorder, agoraphobia without panic disorder, alcohol abuse, alcohol dependence and drug dependence, for which there was a moderate level of disagreement between the subject and NOK. Psychiatric symptoms experienced in the last week and childhood trauma scores were concordant between the subject and his or her NOK. Although the ICC for the Barratt Impulsiveness Scale was low, the mean scores were found to be statistically equivalent.

The reliability of psychiatric diagnoses in living individuals generated by a variety of instruments has been demonstrated by the SCID-I (Structured Clinical Interview for Axis-1),22 SCID-II (Structured Clinical Interview for Axis-2),23 MINI7 and Diagnostic Interview for Genetic Studies.24 However, there is very limited information regarding the reliability of retrospective diagnoses, especially as they apply to post-mortem research. The general approach to establish post-mortem psychiatric diagnoses includes a review of medical records and conducing a psychological autopsy about the decedent with the NOK (Table 4). Sundqvist et al.25 reported a kappa coefficient of agreement for diagnoses solely from chart review between the ante and post-mortem diagnoses ranging from 0.35 for schizoaffective disorder to 0.95 with major depression. The inclusion of an interview with the NOK, in addition to the review of medical records, increases the information reliability across diagnostic classifications. Most research of this type relies on using a semistructured information gathering process to organize medical and psychological autopsy material. The two common ones are the Diagnostic Interview After Death,26, 27 Diagnostic Instrument for Brain Studies28 and their variants.29, 30

Table 4 Previously published reliability assessments for post-mortem diagnoses
Full size table

Deep-Scoboslay et al.,31 Kelly and Mann32 and Lehrmann et al.33 used SCID-P (axis 1) and the SCID-II with either DSM-III-R and DSM-IV criteria. They combined this information with antemortem data organized through the Diagnostic Interview After Death and found the instruments demonstrate good reliability when compared with medical records. This study also shows good reliability of informant information for a majority of diagnoses. Because our sample was limited to primary diagnoses of mood disorders, the reliability determination of the other diagnoses such as schizophrenia was incomplete. For example, three subjects endorsed generalized anxiety disorder symptoms and two post-traumatic disorder symptoms but these symptom sets were not observed by the NOK. The subject–NOK interview provided the greatest discordance in the alcohol use disorders with four subjects reporting misuse but not by the NOK. This is consistent with the clinical experience of patients often under reporting their drinking. There was higher concordance with drug use, but the frequency of any positive response was low with only three NOK or subject reported misuse. Lehrmann et al.33 looked at substance misuse in a post-mortem sample identified by medical examiner records, NOK interviews and toxicology. They showed that when medical records and toxicology data are combined, the detection rate drastically increases. Clearly, increasing the number of sources of information allows for greater reliability for all diagnoses. Two other studies looked at the concordance between psychiatric diagnosis generated by an informant compared with that of a subject and found high concordance.34, 35 Schneider et al.34 found kappa correlation coefficients for mood disorders at 0.79, anxiety at 0.79 and any personality disorder at 0.92, which are comparable to our findings. Zhang et al.35 also found high concordance with SCID-based diagnosis and also conducted a Ham-D with a subject and two informants. They found that the results were significantly correlated (Spearman’s rho=0.57). Their results had substantially higher concordance than we report and this is most likely because they used two informants for each subject.

Genetic and family studies using the family history method also collect and utilize informant-based information. Rougemont-Buecking et al.36 in a large well-designed study showed fair to good agreement between a family member and direct interview for panic disorder and obsessive compulsive disorder, whereas poor agreement was seen with overall anxiety disorder and generalized anxiety disorder.36 Mendlewicz et al.37 reported an agreement kappa of 0.5 to identify affective disorders between a direct psychiatric interview and probands recollection.37 Gershon and Guroff38 reported kappa’s for bipolar disorder 1=0.63, major depression=0.42, whereas for bipolar disorder 2 and schizoaffective disorder the kappa=0.38 One possibility that our values showed greater agreement than the genetic studies is that all of our subjects were long-term psychiatric patients with family that were knowledgeable of their medical history. In this report, we show that the MINI can also provide an accurate psychiatric diagnosis, and can be completed in a shorter amount of time in comparison with the SCID. This is important as it limits the intrusiveness of the NOK interview.

In this work, we attempted to simulate a typical NOK post-mortem interview with clinic patients and their NOK to see if the NOK were aware of the severity of psychiatric symptoms and mood state in the subjects. The ICC of all the scales ranged from 0.66 to 0.44 with the exception of the BISS depression subscale (0.28) and Barratt Impulsiveness Scale (0.36). Using Landi and Koch39 interpretation of the similar kappa, these scores showed at least a ‘moderate’ level of agreement. Although Barratt Impulsiveness Score had a poor level of agreement by ICC, the mean scores were found to be statistically equivalent.

This study gathered retrospective information obtained by an informant. The reliability of this type of data has several areas of potential confounds. Most NOK under report symptoms and when interpreting the results, care must be given to who the informant is. For example, parents may not be aware of their children's alcohol/drug use nor of their sexual drive and many spouses may not have detailed history of the other spouse's childhood abuse. This study uses a small sample size of non-randomly selected subjects; even so, our results are similar to other reports using a variety of instruments.34, 35, 40 Because our focus was on mood disorders not all DSM axis 1 diagnoses were encountered and we were unable to report positive agreement data for the diagnoses of: obsessive compulsive disorder, post-traumatic stress disorder, schizophrenia and adult attention deficit hyperactivity disorder. Additional work is needed to study the concordance of these disorders and understand how long after death a NOK can provide reliable mood symptom ratings.

Overall, we show in a group with severe mental illness that an informant interview of the NOK can provide useful information, which can be used to better analyze post-mortem biological information. There are significant caveats to reliable post-mortem data collection: (1) the interviewer must have extensive experience conducting clinical interviews, (2) the informant must have regular contact with the subject, (3) multiple informant interviews should be conducted if available and (4) the psychometric instruments used must be geared toward clinically obvious symptom levels.