Introduction

Myriad consequences of sensorimotor and autonomic deteriorations following spinal cord injury (SCI) result in the patients requiring contributions from many rehabilitation professionals including physicians, nurses, occupational therapists, and physical therapists working corporately to assess, promote, monitor, and transfer patients’ independence [1]. These consequences may be significantly exacerbated during the current coronavirus (COVID-19) pandemic, as hospital services are limited for these individuals and the rehabilitation period is dramatically shortened [2]. This situation may result in a paradigm shift with the greater demands on community-based rehabilitation (CBR) and home healthcare services for the distribution of standard healthcare services to individuals’ homes and communities [3, 4]. This healthcare modification suggests the need for a thorough standard assessment to promote effective communication regarding goal settings, monitoring and data transferring among rehabilitation professionals in various clinical, community and home-based settings.

The Spinal Cord Independence Measure (SCIM) has been specifically designed to cover all aspects relating to individuals with SCI, including self-care ability, respiration and sphincter management, and daily mobility, to be rated exclusively by an occupational therapist, nurse, and physical therapist, respectively [5]. The SCIM outcomes reflect the independence of individuals with SCI, whereby a higher score indicates the requirement of little assistance, or fewer aids, to complete basic daily-living and life-support activities [5]. Since its development, the tool has gradually been modified for appropriateness to reach the most recent version (SCIM III), and it has also been translated into various languages, including Thai [5, 6]. However, the reliability of this tool has been reported only when used by particular rehabilitation professionals who are responsible for specific domain, with intraclass correlation coefficients (ICCs) range from 0.89 to 0.97 [7].

To the best of the researchers’ knowledge, there is no clear evidence supporting the use of SCIM III interchangeably among rehabilitation professionals of various domains in order to extend the clinical utility of the tool for primary hospitals, CBR and home healthcare services that might not have all the professionals needed for a particular domain of the SCIM III. The researchers hypothesized that, after training, rehabilitation professionals of various domains can rate the items of the SCIM III interchangeably. Furthermore, with myriad components of SCIM III items, the researchers hypothesized that, in addition to balance control, hand functions and functional mobility, which have been investigated in existing reports [7,8,9,10], the outcomes of the SCIM III could reflect other important insights for individuals with SCI. Therefore, this study investigates the rater reliability of the SCIM III including both the total scores and that of each domain among various rehabilitation professionals, including occupational therapists, nurses and physical therapists. In addition, this study assessed the concurrent validity of the SCIM III outcomes and standard measures that have already been proven for their validity to reflect muscle strength, balance ability, functional endurance and ambulatory (AM) ability among individuals with SCI, both wheelchair users (WU) and AM individuals. The findings may extend clinical implication of the SCIM III to promote standardization of assessments and data transferring for individuals with SCI among various healthcare services.

Methods

Design and participants

This observational study was conducted among rehabilitation professional raters and individuals with SCI from a tertiary rehabilitation center and communities from January 2020 to March 2021. Their characteristics are described in the following sections.

Raters

Seven rehabilitation professional raters were involved in this study. Among them, one was an expert that is, a rehabilitation professional who had more than 3 years of experience in using the SCIM III. The other six raters included two occupational therapists, two nurses, and two physical therapists, with one experience and one novice rater in each profession. Experience raters were those with more than 3 years of experience with individuals with SCI, while novice raters were those who rarely work with individuals with SCI [11, 12].

Individuals with SCI

Individuals with SCI, with either complete or incomplete lesions, were recruited from a rehabilitation ward of a tertiary hospital and communities. The eligible participants were aged at least 18 years old, had an SCI from traumatic and non-progressive causes, and were at a subacute and chronic stage of injury. Individuals with SCI were excluded if they presented with any conditions that might affect the outcomes of the study or if participation in this study might affect their conditions (e.g., unstable medical conditions, brain function disorders, visual deficits, joints deformity, leg length discrepancy of more than 2 cm [13, 14], pain in the musculoskeletal system with a pain score of more than 5 out of 10 on a visual analog scale or other medical conditions that might affect participation in the study) [14].

The sample size was estimated using previous studies [9, 15, 16]. Morrow and Jackson [15] suggest that 30 participants are needed for a reliability study. For the correlation study, this study applied a correlation level from a previous study (0.5) [9], with an alpha level of 0.05 and power of test at 90% [16]. The data indicated that the study required at least 38 participants. Thus the study recruited 76 participants for the validity study (38 WU and 38 AM participants). The eligible participants read and signed an informed consent document that was approved by the Institutional Ethics Committee for Human Research prior to participation in the study (HE611371).

Research protocols

Rater training for the reliability study

All raters were trained to use the SCIM III by an expert prior to score the participants. The contents included the aim and characteristics of the SCIM III, its scoring system and the scoring details of each item for approximately an hour. Then, they were given a practice session in four individuals with SCI who did not involve in this study. Subsequently, all raters individually rated SCIM III scores of SCI participants from video data over two sessions, with a 14-day interval [11].

Outcome measures

The eligible participants with SCI were interviewed and assessed for their demographics (i.e., bodyweight, height, sex, age and underlying disease, if any), vital signs (i.e., heat rate and blood pressure) and SCI characteristics (i.e., causes, post injury time, severity and levels of injury using the criteria from the American Spinal Injury Association Impairment Scale (AIS)) [13, 17]. The participants were divided into two groups: WU and AM individuals. Participants who were wheelchair users or WU were those who used a wheelchair in at least 75% of their daily living [13, 17], while AM individuals were those who had the ability to walk independently, with or without a walking device, over a distance of at least 10 m [14].

All participants were assessed for their SCIM III scores (Thai version) including self-care, respiratory and sphincter management, and mobility subscales by an expert (kappa value > 0.95) though observation and interview with data confirmation from their caregivers. Specific items within self-care and sphincter control domain that were unable to directly observe during examination were scored according to information obtained from a staff member or caregiver who observed the participants performing the tasks during the same week [14]. The data of the first 30 participants (15 WU and 15 AM participants) [15] were video recorded for the further analysis on rater reliability.

Subsequently, participants with SCI were assessed using standard measures to reflect their muscle strength, balance control, functional endurance and AM ability by an expert [18,19,20,21,22,23,24,25], as the details described in Supplementary 1. Then the data of SCIM III and standard measures from the expert were used to analyze the concurrent validity of the SCIM III. During the tests, a lightweight safety belt was fastened around each participant’s waist, and a physical therapist was always by their side to ensure their safety and the accuracy of the tests. Participants could take a period of rest between the trials and the tests, as required.

Data analyses

Descriptive statistics were applied to explain the demographics and SCI characteristics of the participants, and findings of the study. ICCs and kappa coefficients were used to analyze the intra- and inter-rater reliability and the agreement of the SCIM III scores among the raters, respectively. ICCs were interpreted as excellent (ICC > 0.9), good (ICC = 0.75–0.90), moderate (ICC = 0.5–0.75) or poor (ICC < 0.5) reliability [26]. Kappa coefficients were interpreted as excellent (≥0.75), good (0.40–0.74) or poor (<0.40) agreement [27]. The concurrent validity, i.e., the amount of correlation between two different assessments, an assessment to be validated for its outcome benefit (SCIM III) and the well-established validity measures for muscle strength, balance ability, functional endurance and AM ability [28], was analyzed using the Spearman rank correlation coefficients (rs) for ordinal data and Pearson’s correlation coefficients (r) for continuous data. The levels of correlation coefficients were interpreted as excellent (rs, r > 0.90–1.00), high (rs, r = 0.70–0.90), moderate (rs, r = 0.50–0.70), low (rs, r = 0.30–0.50) or negligible (rs, r = 0.00–0.30) correlation [29]. The level of statistical significance was set at p < 0.05.

Results

Participant characteristics

Eighty-two middle-aged, mostly males, participants with SCI (39 WU and 43 AM individuals) completed this study, and 30 participants were also involved in the reliability study (15 WU and AM participants). The participants had a level of SCI ranging from cervical (C3) to cauda equina, with an SCI severity ranging from AIS A to AIS D (Table 1). All participants could breathe independently using room air, used an indwelling catheter and needed assistance with bowel management. The other demographics and SCI characteristics for the participants are presented in Table 1.

Table 1 Demographics and spinal cord injury (SCI) characteristics of participants.
Full size table

Reliability of the SCIM III

The data indicated that the SCIM III had excellent intra-rater reliability for both novice and experience rehabilitation professional raters, when analyzed for the overall items (ICC > 0.90; Table 2) and separately for each subscale (kappa values > 0.80; Table 2). In addition, the SCIM III scores of all novice and experienced raters showed excellent inter-rater reliability when compared with the scores identified by the expert (ICCs for overall items >0.90; kappa values for each subscale > 0.80) except the scores given by a novice nurse for the bladder and bowel management items (kappa values = 0.68 and 0.62, respectively; Table 2).

Table 2 Reliability of the total and sub-items of the Spinal Cord Independence Measure (SCIM) III among rehabilitation professionals.
Full size table

Correlation between the SCIM III and standard measures

For WU individuals, the total SCIM scores were significantly correlated with all standard measures (rs = 0.343–0.528; p < 0.05; Table 3). In addition, their scores of the respiratory and sphincter management subscales were significantly related to the hand grip test (HG), modified functional reach test (mFRT) and 1-minute seated push-up test (1MinSPUT; rs = 0.395–0.436; p < 0.05; Table 3). Furthermore, the mobility subscale of these individuals was significantly associated with HG and t-shirt outcomes (rs = 0.342 and 0.379, respectively; p < 0.05; Table 3). For AM individuals, their mobility scores on the SCIM III showed clear correlation with all standard measures except the HG test (rs = 0.392–0.779; p ≤ 0.001; Table 3). Moreover, their total SCIM score and scores on self-care items were significantly correlated with the Timed Up and Go (TUG) test, the 6-minute walk test (6MinWT) and 10-meter walk test (10MWT; rs = 0.527–0.711; p < 0.05; Table 3).

Table 3 Correlation between the Spinal Cord Independence Measure (SCIM) III score and standard measures.
Full size table

Discussion

The dramatic decrease in rehabilitation length as a result of the limited hospital services available during the COVID-19 pandemic has made effective CBR and home healthcare services even more vital for many individuals, including those with SCI. This study investigated the rater reliability of the SCIM III among rehabilitation professionals, including nurses, occupational therapists and physical therapists. The findings indicated that the SCIM III had excellent rater reliability among these rehabilitation professionals, including both novice and experienced raters, when analyzed for the overall items (ICCs > 0.90; Table 2) and separately for each subscale (kappa values >0.80), with the exception for a novice nurse for the bladder and bowel management items (Kappa values = 0.68 and 0.62, respectively; Table 2). The present findings also extend the concurrent validity of the SCIM III scores for WU and AM individuals with SCI (rs > 0.340; p < 0.05; Table 3).

The excellent rater reliability of both novice and expert rehabilitation professionals (Table 2) reflects the characteristics of the SCIM III itself. The tool is a user-friendly clinical measure that was specifically designed to measure routine daily activities relating to the impairments found among individuals with SCI [7]. These tasks are very familiar to rehabilitation professionals. The scoring system of the SCIM III is also self-explanatory and weighted each task according to its clinical relevance with respect to the overall activities needed by individuals with SCI [7, 29]. Therefore, after training and practicing with four individuals who had different characteristics, novice and experience health professional raters could rate the SCIM III items in such a way that their scores showed excellent agreement and coherence with those of an expert (Table 2). The present findings are in line with those of a previous report in a multicultural setup that indicated ICC values above 0.94 for the overall items and all subscales [30]. However, the previous data were derived from rehabilitation professionals who were routinely responsible for performing relevant functions (i.e., self-care activities by occupational therapists; respiration and sphincter management by nurses, as well as mobility in room and bathrooms; and mobility indoors and outdoors by physical therapists) [30].

Nonetheless, an exception was noted in this study for the bladder and bowel management items when rated by a novice nurse, whereby the data indicated good, but lower agreement than other rater (Kappa values = 0.68 and 0.62, respectively; Table 2). This finding may reflect the nurse’s clinical experience, only 1 year of working experience and rarely worked with individuals with SCI, and this may subsequently affect the decision-making process when rating the SCIM items. The nurse commented about being unable to clearly determine the different scores for bladder and bowel management items. This finding suggests the need for special attention to be given to these items while training professionals on the use of the SCIM III, particularly for novice raters. Nonetheless, the kappa values of nurse’ s score were greater than 0.60, indicating acceptable and good agreement in reliability studies [31]. The nurse’s data were also similar to those given in a previous study [30], which reported kappa value for bladder and bowel management items of 0.70 and 0.63, respectively. Therefore, the current findings confirm the use of the SCIM III interchangeably among rehabilitation professionals after adequate training, with particular attention being given to the items related to sphincter management for bladder and bowel for novice health professional raters.

Moreover, the present findings extend clinical benefit of the SCIM III, as the outcomes showed significant correlation with standard measures needed in daily living for WU and AM participants (Table 3). In WU participants, the total SCIM scores significantly correlated with all standard measures (rs = 0.343–0.528; p < 0.05, Table 3). On the contrary, for AM individuals, only the mobility items showed significant correlation with most standard measures, except the HG test (rs = 0.392–0.779; p < 0.05; Table 3). These findings reflect the specific characteristics of the participants and SCIM III scores. Most WU participants were at a chronic stage and had good upper limb motor scores (49.3 ± 0.4; Table 1). Thus, they had been executing the daily activities measured through the SCIM III using their upper extremities. Consequently, their overall SCIM III scores were significantly correlated with the HG test, mFRT, t-shirt test and 1MinSPUT, which reflected individuals’ upper limb muscle strength, limit of stability, balance control, and functional endurance, respectively (rs = 0.343–0.528; p < 0.05; Table 3). Rudhe et al. [9] also found a high level of correlation between the total SCIM III scores and upper extremity motor score (r = 0.78), the manual muscle test used for upper extremity muscles (r = 0.78) in individuals with tetraplegia. The findings further suggest the crucial contribution of upper limb muscle function (measured by the HG test), limit of stability while sitting (measured by the mFRT) and functional endurance (measured by the 1MinSPUT) on respiratory and sphincter managements within these individuals (rs = 0.395–0.436; p < 0.05; Table 3).

In contrast, AM participants required both upper limbs and lower limbs to complete their daily tasks. Having good upper limb motor scores (48.8 ± 0.5; Table 1) indicated that the participants also had a good level of ability to complete self-care activities and respiratory and sphincter managements; as such, the HG test was the least demanding task for these individuals, and the data indicated no clear correlation in these items (i.e., self-care and respiratory and sphincter management). However, the various degrees of trunk and lower limb dysfunctions reduced their mobility at a different level; and afforded a significant correlation between the mobility items and standard mobility measures that reflected lower limb muscle strength (i.e., the five times sit-to-stand test), limit of stability (i.e., the functional reach test), balance control during walking [TUG], functional endurance [6MinWT], and walking ability [10MWT; rs = 0.392–0.779]; p < 0.001; Table 3). These findings are consistent with those of Wannapkake et al. [10], who reported a high level of correlation between the total SCIM III score with the TUG data (rs = −0.764) in AM individuals with SCI (AIS C and D). A recent study also reports the use of SCIM III scores to discriminate and monitor independence among AM individuals with SCI [32].

The present findings support the use of SCIM III interchangeably among rehabilitation professionals of various domains (Table 2). The significant correlation with standard measures also infer the increase or decrease in the SCIM III scores reflect the corresponding increased or decreased outcomes in the standard measures. However, the low to moderate correlation may reflect the different constructs (i.e., independence) between the SCIM III and the standard measures investigated in this study (Table 3). The findings extend the clinical utility of the SCIM III, covering various levels of healthcare settings. However, all participants in this study were at a chronic stage of injury, could breathe independently using room air, used an indwelling catheter and had good upper limb function, which may limit the generalizability of the findings. An additional study involving participants with an acute SCI who need a respirator and have poor upper limb function would further confirm the use of the SCIM III interchangeably among rehabilitation professionals as raters.

Conclusion

The current finding extend the clinical benefit of the SCIM III, as the increase or decrease in the SCIM III scores reflect the corresponding increased or decreased in the outcomes with different underlying constructs, including the upper limb muscle strength, limit of stability, balance control and functional endurance of WU and AM individuals with SCI. Moreover, the SCIM III can be used interchangeably among rehabilitation professionals. Thus, the tool can be used to promote the standardization of communication regarding goal setting, monitoring and data transferring among rehabilitation professionals in various clinical, community and home-based setting, particularly during the COVID-19 pandemic, when in-patient services are limited for individuals with SCI and the rehabilitation length is shortened.