Introduction

The Spinal Cord Independence Measure version III (SCIM III) is a scale that rates the independence of patients with spinal cord injuries (SCIs)1 with regard to performing activities of daily living. This tool was developed specifically for SCI patients and has become a widely used instrument to measure functioning in activities of daily living. The chief advantage of this instrument, which was first published in 1997,2 is its sensitivity to changes in the performance on tasks that are relevant to patients with SCIs. The validity and reliability of the third version of the SCIM have been tested in multicenter studies, which have demonstrated it to have satisfactory psychometric properties.3, 4, 5

SCIM III comprises items on 19 daily tasks that are grouped into three subscales that assess ‘Self-care’ (6 items, range 0–20), ‘Respiration and sphincter management’ (4 items, range 0–40) and ‘Mobility’ (9 items, range 0–40). The total SCIM III score ranges from 0 to 100, and higher scores reflect better performance or greater independence of a person.3 Item scores are graded for increasing difficulty and weighted by clinical relevance. Each item has between 2 and 9 grades. SCIM III is ranked by a multidisciplinary team, primarily in inpatient settings; this procedure is time-consuming.

Recently, a self-reported version of the SCIM III (SCIM SR) was created6 and translated into German, Italian and French, but only the German version has been validated.

Self-administered questionnaires are the instrument of choice for data collection outside the hospital, because they require few resources, are faster and can be applied independently from the evaluation settings.6, 7, 8

Thus, the SCIM SR is invaluable because it monitors and detects any special health needs in SCI patients after discharge from the hospital and because it helps in the collection of data for research. The purpose of this study was to compare the Italian versions of the SCIM SR and SCIM III and assess their validity.

Materials and methods

Subjects

Consecutive patients who were admitted to the Spinal Unit of Montecatone Rehabilitation Institute (MRI) and the Spinal Unit of IRCCS Fondazione S. Lucia, Italy, were recruited between February and December 2013. The criteria for eligibility were a traumatic or a nontraumatic SCI (time since injury ⩾1 month), sufficient Italian language skills (as assessed by the clinical staff), and the ability to read and answer the self-report questions independently. People with severe health conditions or cognitive impairments were excluded.

The following information was collected from their medical records: age, gender, etiology, date of injury, level of lesion (paraplegia/tetraplegia) and American Spinal Injury Association (ASIA) Impairment Scale (AIS) grade.

Procedure and instrument development

Participants were asked to fill in a paper-pencil version of the SCIM SR questionnaire. If they had difficulties writing owing to limited hand function, their caregiver helped them complete the form but did not explain the items or assist in choosing an answer. On the same day, health professionals (including physicians, physiotherapists and nursing staff) with experience with SCIM III completed the Italian SCIM III9 (which is used routinely in both of the rehabilitation centers in this research) through observation of the participants.3

The English version of the SCIM SR has been translated into Italian by professional scientific translators (www.five.ch). Before being administered to the study subjects, the Italian draft version of the SCIM SR was tested in cognitive interviews of five individuals with SCIs who were recruited at IRCCS Fondazione S. Lucia. In addition, five Italian-speaking experts who have used the SCIM III in a clinical setting have been asked to comment on the draft version of SCIM SR. Cognitive interviews are important to determine the comprehensibility of questionnaires before being applied in surveys.10

The results of this pretest guided the final changes to the Italian draft version of the SCIM SR (Appendix A and B). Then, final modifications were made to establish the test version for the validation phase. We certify that all applicable institutional and governmental regulations that concern the ethical use of human volunteers were followed during this study.

Statistical analyses

Descriptive statistics on patient characteristics were expressed as mean and standard deviation or frequency and percentage. Descriptive statistics were also reported for the SCIM III and SCIM SR total and subscale scores, and the nonparametric Wilcoxon signed-rank test was used to analyze differences in medians.

Pearson’s correlation coefficients were calculated to analyze the linear relationship between SCIM III and SCIM SR. Moreover, intraclass correlation (ICC) was performed to assess the conformity of SCIM III and SCIM SR scores, because they represent two quantitative measurements by different observers (clinician and patient) on the same subject. The ICC coefficient takes into account interobserver and intraobserver variability and is a composite measure of these factors.11 According to previous work,12 a correlation of 0.80 is considered a fairly strong relationship, a correlation of 0.60 signifies a moderate relationship and a value of 0.20 indicates that the scores are weakly associated.

The Bland-Altman method was used to calculate the mean and limit of agreement (LOA) to describe the differences between SCIM III and SCIM SR scores. The mean difference and LOAs are reported in Bland-Altman plots that show the difference between SCIM SR and SCIM III scores against the mean scores of both measurements for each subject, which is useful for detecting outliers.13

For formal analyses, stratified mean differences between total SCIM III and SCIM SR scores were calculated and linear regression was performed, after the assumption of normal distribution was tested by the Shapiro test and normal QQ plot. In a stratified analysis, the mean difference is compared between variable categories; thus, age was grouped into three classes: youngest (<35 years), middle (35–54 years) and oldest (>54 years). In the regression analysis, we regressed the difference between the two total scores as continuous outcomes on select potential predictors, introduced as continuous (age and time) or categorical variables (gender, lesion level, AIS grade and etiology). Univariate and multivariate models were constructed. For each explanatory variable, the coefficient and its 95% confidence interval were computed, and the F test was performed to determine the significance of the association between predictors and outcome. Factors were considered to be meaningful at a significance level of 0.05.

The data were analyzed using R v3.0.2.

Results

We enrolled 116 patients (80 males, mean age 45.5 years), the characteristics of whom are presented in Table 1.

Table 1 Patients characteristics
Full size table

Overall, the cognitive interviews with patients and professionals demonstrated that the Italian SCIM SR was understandable and acceptable.

Table 2 lists the descriptive statistics of the SCIM III and SCIM SR scores and their P-values by the nonparametric Wilcoxon signed-rank test. There were no missing values for any scale score. The scores were not normally distributed (see Figure 1); thus, we could only compare the median values by the nonparametric Wilcoxon signed-rank test instead of the means. The means, medians and interquartile ranges were similar, and median total and subscale SCIM III and SCIM SR scores did not differ significantly (P-values>0.05).

Table 2 Descriptive statistics on total and subscale scores of SCIM III and SCIM SR
Full size table
Figure 1
figure 1

Total score distribution of SCIM III and SCIM SR. The graph shows the distribution of the total scores of the two scales. The scores were not normally distributed. A full color version of this figure is available at the Spinal Cord journal online.

Full size image

Pearson's and ICC correlation coefficients between SCIM III and SCIM SR scores are shown in Table 3. Pearson's correlation coefficients ranged from 0.806 for the 'Respiration and sphincter management' subscale to 0.934 for the total score. Although 'Respiration and sphincter management' had the lowest coefficient, it was sufficiently high (>0.7) to demonstrate a linear relationship between SCIM III and SCIM SR. Also, the ICC values were high, approximating the linear correlation coefficients. The scores of the two measurements are plotted in Figure 2—the evaluations by the SCIM III and SCIM SR strongly agree, because the points are uniformly distributed around the line of equality (the bisect of each figure).

Table 3 Pearson and intraclass correlation coefficients
Full size table
Figure 2
figure 2

SCIM III and SCIM SR scores. The graph shows the plotting of the scores of the two measurements. The points are uniformly distributed around the line of equality (the bisect of each figure), thus demonstrating that the evaluations by the SCIM III and SCIM SR strongly agree. A full color version of this figure is available at the Spinal Cord journal online.

Full size image

The difference between scores was estimated by the Bland-Altman method, as shown in Table 4. As expected, the mean differences were close to 0, confirming the agreement between SCIM III and SCIM SR scores. The LOAs can be interpreted as a reference interval—that is, the range in which most differences lie.13 12 In Table 4, we can see that this range is about ±5 for the subscale ‘Self-Care’, ±10 for the subScale ‘Respiration and sphincter management’, ±9 for ‘Mobility’ and ±16 for the total score. The ranges vary from 16.3%, 22.7% and 24.2% to 25.6%, respectively. The LOA included over 92% of the differences in all scales, and the Bland-Altman plots (Figure 3) showed that there were few outliers for each scale. In addition, the LOA for the total score included 95% of the observations, which were normally distributed as indicated in the normal Q-Q plot (Figure 4) and confirmed by the Shapiro test (P-value=0.102).

Table 4 Bland-Altman analysis
Full size table
Figure 3
figure 3

Bland-Altman plots on subscales score. The Bland-Altman plots show that there are few outliers for each subscale and the total scale. A full color version of this figure is available at the Spinal Cord journal online.

Full size image
Figure 4
figure 4

Q-Q plot for differences on total scores. The normal Q-Q plot demonstrates the normal distribution of the observations

Full size image

With the assumption of a normal distribution, we calculated the 95% confidence interval of the mean difference in total scores in the stratified analysis and generated regression models (Table 5). The difference between total scores was small for all classes and had a range of ±1. The largest disparity arose between the mean difference for AIS A (−1.33) and AIS C (1.54). However, because all 95% confidence intervals contain the 0 value, we conclude that there were no significant differences between total SCIM SR and SCIM III scores in any category. None of the explanatory variables was a significant predictor of the difference between SCIM III and SCIM SR scores based on both the models of the regression analysis (P-values).

Table 5 Stratified and regression analyses on difference between SCIM SR and SCIM III total score
Full size table

Discussion

Our results support the validity of the criteria of the SCIM SR compared with the SCIM III.6 The patient feedback and cognitive interviews provided valuable data that we used to establish preliminary validity of the content of the SCIM SR in the self-evaluation of patients with SCIs.

The mean and median SCIM III and SCIM SR scores were similar for each scale, indicating that patients and clinicians judged the patient’s performance to be the same. The Pearson correlation and ICC coefficients of the total and subscale scores met the quality criterion (>0.7). 6, 14

Moreover, by Bland-Altman analysis, with mean differences of approximately 0, the self-evaluation by SCIM SR and the evaluations by professionals with SCIM III were very close for the total and the sub-items scores. This result differs from the validation study of the German version of SCIM SR,6 in which patients systematically rated their ‘mobility’ to be higher than did professionals.

In the regression analysis, we observed no association between the differences in scores and any explanatory variable—that is, there was no class patients who were at risk of completing the SCIM SR form differently from professionals. This result is an important finding, because it demonstrates the reliability of this tool for each type of patient, without regard to his sociodemographic or clinical characteristics.

As reported,6 item 6 (bladder management) has been changed in adapting SCIM SR to SCIM III. In the conversion table (Appendix B), score 3 is missing because of the inability to measure residual urine volume by self-report; score 6 is used in its place. This arrangement amplifies the score on item 6 in the SCIM SR and masks those cases in which patients judge themselves the same as clinicians do. Nevertheless, these results indicate high correlation between item 6 of the SCIM SR and SCIM III, as with the German version.6 However, this oversight can affect the results, because there are some cases in which a score of 6 is excessive and a score 0 is inappropriate. Thus, to increase the precision and accuracy, a new conversion table that takes into account a score of between 0 and 6 should be adopted.

Another drawback is that our study was limited to the time of discharge. To use this instrument for longer follow-ups, a specific and an appropriate validation study that at least performs test-retest correlation and comparison—with, for example, a quality of life measure—would be needed.

With regard to strengths, our sample size was sufficiently high for a validation study.6, 13 Moreover, our study included many patient characteristics and was thus representative of the study population and did not have missing data, increasing its reliability.

Finally, the number of negative differences, corresponding to patients who overestimated their functioning, compensated for the number of patients who underestimated their performance, supporting the hypothesis of contingency of the difference between measurements and highlighting the strength of SCIM SR instrument in assessing the performance of SCI patients after discharge from the hospital.

Conclusion

Our findings support the criterion validity of the SCIM SR, and, because SCIM III is the only comprehensive instrument that measures disability that has been developed specifically for individuals with SCIs,6 the self-reported version can be used for continued monitoring of daily activities in community-dwelling individuals and might detect special health-care needs in this population. The only precaution that must be taken concerns the reliability of the patients, which depends on their mental status.

Date archiving

There were no data to deposit.