Introduction

The Spinal Cord Independence Measure (SCIM) is a comprehensive disability rating scale that has been designed specifically for patients with spinal cord lesions (SCLs).1, 2 The demand for the scale is steadily increasing, and has been endorsed by key stakeholder groups. For instance, the international group for recovery outcome measures, which was sponsored by the US National Institute on Disability and Rehabilitation Research (NIDRR) framework for the appraisal of evidence of metric properties, recommended that the latest version (SCIM III) should be implemented worldwide as the primary functional recovery outcome measure for spinal cord injuries (SCIs).3, 4 As well, the expert panel of the Spinal Cord Injury Solutions Network (SCISN) concluded that the clinical utility and psychometric properties of the SCIM are appropriate for patients with acute traumatic SCI.5 SCIM III was tested for validity and reliability in an international multi-center clinical study, and the findings supported the reliability and the validity of the scale.1, 2

Nevertheless, it was recommended that the SCIM III should continue to undergo refinements and psychometric validation.4, 5 Although our multi-center study found that the patterns of scores on the SCIM-III between people with trauma and non-trauma etiologies were comparable,2 the use of a mixed population of trauma SCL (TSCL) and non-trauma SCL (NTSCL) has been highlighted as an argument for further validation.4 The doubts stem from the knowledge that outcomes and characteristics may vary between the two groups.3

Patients with TSCL are typically younger and have an earlier age of onset,6, 7, 8, 9 their male/female ratio is higher6, 8, 9, 10 and their neurological impairment as measured by the American Spinal Injury Association (ASIA) impairment scale (AIS) is often more severe.7, 8, 9, 10, 11 As well, the neurological recovery rate in patients with TSCL was found lower than in NTSCL individuals with similar neurological impairment.11 However, the length of stay in rehabilitation tended to be longer in TSCL,7, 8, 12, 13 functional gain was found better than in NTSCL8, 12 and incidence rates of secondary SCI medical complications differed between the groups.7

Based on the issues raised regarding the use of a mixed population for the validation of the tool, we evaluated the reliability and validity of SCIM III in a separate analysis for patients with TSCL.

Materials and methods

Participants

In this study we analyzed data of the subgroup of TSCL subjects from the multi-center international SCIM III study.1, 2 The original sample included 425 in-patients with SCL from 13 units in 6 countries. Inclusion criteria were a SCL (AIS grades A–D), age ⩾18 years and no concomitant impairments that might influence everyday function (for example, cognitive or mental impairments).1

A total of 261 patients with TSCL were retrieved from the entire sample for this study. The sample had a male/female ratio of 5:2, and a mean age of 40.1 years (s.d.=17.1). With regard to impairment, 55% had tetraplegia and 45% had paraplegia. As well, AIS grades were A, B, C and D in 49.2, 13.5, 19.6, and 17.7%, respectively.

Procedure

The patients’ functional status was assessed with the SCIM III questionnaire by two expert professionals on the first week after admission to rehabilitation and on the last week before discharge from rehabilitation. A third staff member evaluated the patients at the same time intervals with the Functional Independence Measure (FIM).14 Each of the three examiners scored the patients independently and was blind to the other examiners’ results. All assessments were based on direct observations of patients’ performance, except for those items where direct observation was not practical (for example, bowel habits, voiding and wheelchair/ground transfers). Information on these tasks was obtained from a staff member who observed the patient during routine care.

Data analysis

The following measures were used to evaluate inter-rater reliability of SCIMIII: (1) the percentage of total agreement between the paired raters; (2) the κ coefficients of SCIM tasks (chance-corrected agreement); (3) Pearson's correlation and paired t-test of SCIM subscales; and (4) intraclass correlation coefficient (ICC), which estimates the proportion of variability between subjects within the total variability in scores. The desired ICC value is >0.75.15

The internal consistency of the scale was tested using Cronbach's α coefficient. The internal consistency of a scale stands for the extent to which the different items of a scale relate to the same underlying dimension. Values of Cronbach's α exceeding 0.7 support reasonable internal consistency.15 All the above analyses were done on data obtained on admission to rehabilitation. Validity of SCIM III was evaluated by: (1) the correlation between SCIM III and FIM (criterion validity), using Pearson's coefficient on admission to rehabilitation14 and (2) comparing the responsiveness to change of SCIM III and FIM subscales between admission and discharge, by McNemar test. Any score change represents a change in the functional status and was considered significant responsiveness. The statistical analysis was performed using SPSS 15 for Windows (SPSS Inc., Chicago, IL, USA).

Results

SCIM III reliability

(1) Reliability between two raters: In the different SCIM III tasks, total agreement between the paired raters was 79.1–98.7%. The κ coefficients ranged between 0.649 and 0.858, and were statistically significant for all tasks (Table 1).

Table 1 Total agreement between raters and κ coefficients for SCIM III tasks
Full size table

In the SCIM III subscales scores and total score, Pearson's correlation coefficient values were >0.9 (P<0.001), and the mean differences between the paired raters were nonsignificant (Table 2). ICC values were >0.95 for the SCIM III subscales and for the total SCIM III score (Table 3).

Table 2 Correlation and differences between paired raters
Full size table
Table 3 ICCs between paired raters within subscales and for total scores
Full size table

(2) Cronbach's α values for the entire SCIM III scale were 0.835 and 0.833 for the first and the second raters, respectively. For the subscales, Cronbach's α coefficients were 0.65–0.88, and when subscales were eliminated, Cronbach's α for the entire scale decreased, except for the respiration and sphincter management subscale. Within the subscales, elimination of most of the items decreased the α coefficient of the subscale, except for the tasks of dressing lower body, respiration, mobility in bed and ground/wheelchair transfers (Table 4).

Table 4 Internal consistency (Cronbach's coefficient α) within subscales
Full size table

SCIM III validity

(1) A significant correlation was found between SCIM III and FIM scores. Pearson's coefficients for the two raters, whose SCIM scores were examined for correlation with the FIM scores, were 0.839 (P<0.001, n=231) and 0.835 (P<0.001, n=228), respectively.

(2) Responsiveness of SCIM III to changes in function between admission to rehabilitation and discharge was better than that of FIM. In all subscales, SCIM III identified more changes in function than FIM. In the ‘respiration and sphincter management’ and in the ‘mobility indoors and outdoors’ subscales, the difference in responsiveness between SCIM III and FIM was statistically significant for both raters (P<0.001). In the ‘self-care’ subscale, the difference was significant only for one of the raters (P<0.02) and in the subscale ‘mobility in the room and toilet’ the differences between the two scales was not statistically significant (Table 5). For most of the patients, the change in function was an improvement. A reduction in SCIM III score throughout rehabilitation was indicated by both raters in one patient only, and by one of the raters in three additional patients.

Table 5 Responsiveness to functional changes of FIM and SCIM III within subscales
Full size table

Discussion

The results showed favorable psychometric properties of SCIM III in a separate examination of patients with TSCL: total agreement between raters, κ coefficients and ICC values were high. SCIM III subscales’ scores and total scores of the two raters were in good correlation, and they had similar mean values. The internal consistency of the scale was more than reasonable. As well, SCIM III and FIM scores were highly correlated, and SCIM III responsiveness to changes in function was better than that of the FIM. The last finding demonstrates that in addition to being valid, the SCIM III has a psychometric advantage over FIM.

The items, dressing lower body, respiration, mobility in bed and ground/wheelchair transfers, did not contribute to SCIM III internal consistency. The same was found in mixed TSCL and NTSCL population for respiration, mobility in bed and ground/wheelchair transfers.1 This may indicate that these items relate to somewhat different underlying dimensions than the other items of their subscales. Such a difference is clear regarding respiration, which shares the same subscale with sphincter management only for convenience, because both were scored by nursing staff in the original SCIM study.1, 2 The difference in underlying dimension of respiration assessment probably reduced the contribution of the respiration and sphincter management subscale to the internal consistency of the entire scale. These exceptions, however, had little influence on the overall SCIM III internal consistency.

A recent publication provided a comparison of the psychometric properties of disability assessment instruments used for patients with acute traumatic SCI.5 Based on a systematic review of the literature on FIM, SCIM, Walking Index for Spinal Cord Injury (WISCI), Quadriplegia Index of Function (QIF), Modified Barthel Index (MBI), Timed Up and Go (TUG), 6-min walk test (6MWT) and 10-meter walk test (10MWT), the authors recommended for the use of SCIM III in the classification and evaluation of patients with acute SCI. Despite this recommendation, they also noted the need for further investigations to confirm the performance of the SCIM in the acute care setting in a multi-center trial. Upcoming studies, including a US multi-center study for the validation of SCIM III, and a study of the predictive ability of SCIM III, may further confirm SCIM III validity and widen the basis for the development of an improved 4th version of the scale.

Conclusion

Despite less than desired internal consistency for a few items, the results confirm the reliability and validity of SCIM III for TSCL patients in a number of countries.