Study Design:

Prospective repeated measures.

Objectives:

Evaluate intra-rater agreement of repeated motor and sensory scores at individual spinal levels.

Setting:

Non-profit pediatric rehabilitation center.

Methods:

Fifty-eight youth with complete spinal cord injury undergoing two neurological exams. Agreement between exams for each myotome and dermatome was evaluated for four neurological groups: C1–C4 (N=9); C5–C8 (N=8); T1–T6 (N=22); T7–T12 (N=19). Kappa (k) and weighted k (k_w) coefficients were calculated.

Results:

Agreement between strength scores was 99 and 100% in subjects with tetraplegia and paraplegia, respectively. C1–C4: pin prick (PP)=absolute agreement (AA) in 57% dermatomes (D) (k range=0.10–0.83; k_w range=0.36–0.93). Light touch (LT)=AA in 59% D (k range=0.35–0.77; k_w range=0.34–0.84). C5–C8: PP=AA in 86% D (k range=0.28–0.78; k_w range=0.43–0.93). LT=AA in 80% D (k range=0.10–0.80; k_w range=0.12–0.91). T1–T6: PP=AA in 82% D (k range=0.36–0.83; k_w=0.20–0.96). LT=AA in 77% D (k range=0.23–0.89; k_w range=0.23–0.89). T7–T12: PP=AA in 82% D (k range=0.46–0.90; k_w range=0.54–0.90). LT=AA in 84% D (k range=0.41–0.87; k_w range=0.52–0.94).

Conclusion:

Overall, agreement was excellent for myotome comparisons. For the C5–C8, T1–T6 and T7–T12 groups, there are variations in sensory scores within three levels of the neurological level (NL). For the C1–C4 group, variation in sensation extended well caudal to the NL. The International Standards for Neurological Classification of Spinal Cord Injury is a good outcome measure for a single-site, one-rater study but differences in repeated sensory scores at individual D were found, which should be considered in the interpretation of results of outcome studies.

Introduction

Much effort is directed toward the development of measures that evaluate recovery after spinal cord injury (SCI).¹ Muscle strength and sensation have been used as the primary outcomes for clinical trials² despite few studies on the variation of repeated measures at each neurological level (NL). Studies on natural recovery suggest changes in neurological function particularly at NL around the injury.^{3, 4, 5, 6} Treatment effectiveness, therefore, may be difficult to ascertain due to typical variation, spontaneous recovery and inadequate outcome measures.⁶

The International Standards for Neurological Classification of Spinal Cord Injury (ISCSCI)^{7, 8} is the standardized method for the evaluation of neurological impairment following SCI. The sensory and motor exams are both used to classify the NL, motor level, sensory level, zone of partial preservation and severity of injury. The zone of partial preservation is defined as those dermatomes and myotomes that are caudal to the NL that remain partially intact,⁸ and is only used in regard to complete injuries (American Spinal Injury Association Impairment Scale (AIS) 'A').^{7, 8} NL is defined as the most caudal segment of the cord with normal motor and sensory function bilaterally, whereas the motor level and sensory level are the caudal most segments of the cord where there is normal bilateral motor and sensory function, respectively. Standardization of testing techniques and classification guidelines are well described in the teaching manual.⁷

Studies have investigated the reliability and validity of previous versions of examination and classification techniques.^{9, 10} Studies^{11, 12, 13, 14} on the most recent revision⁸ found that training may improve accuracy of agreement; agreement between repeated measures was acceptable and utility may be limited for children.^{11, 12, 13, 14} Although these studies provide useful psychometrics about the ISCSCI, the majority evaluated the reliability of summed scores and provide little information about agreement of repeated scores at individual myotomes and dermatomes.

Purpose

The purpose of this study was to evaluate the agreement of repeated motor and sensory scores at individual myotomes and dermatomes in four neurological groups: C1–C4; C5–C8; T1–T6; and T7–T12. With only one rater and the inclusion of subjects with chronic and complete injuries, it was hypothesized that there would be little variation in strength and sensation at individual myotomes or dermatomes, and high agreement between repeated scores at each level.

Methods

This was a prospective repeated measures study.

Sample

The sample (Tables 1, 2, 3 ) consisted of 58 subjects between 9 and 19 years of age (average age=15.3 years), AIS 'A'. Individuals with incomplete injuries (AIS B, C, D) were excluded from the analysis. The average time between injury and exam was 3.6 years (range=1–13 years). Subjects underwent two motor and sensory examinations based on the 2002 ISCSCI standards⁷ 1–4 days apart. The first author performed all of the exams.

Table 1 - Summary of subjects with C1–C4 (N=9) and C5–C8 (N=8) injuries.

Full table

Table 2 - Summary of subjects with T1–T6 (N=22) injuries.

Full table

Table 3 - Summary of subjects with T7–T12 (N=19) injuries.

Full table

For reliability studies, sample size consideration is not based on statistical methodology but rather on the desired precision of the reliability estimates. The sample in this study represents a convenience sample of individuals willing to undergo two neurological exams. Although this study has a greater number of subjects than other reliability studies of the ISCSCI, sample size was too small to analyze variation at each level and, by study design, there were no subjects with incomplete or newly acquired injuries.

We certified that all applicable institutional and governmental regulations concerning ethical use of human volunteers were followed throughout the duration of this study.

Data collection and management

This study was a single-site, single-rater reliability study. The ISCSCI sensory scale involves testing to pin prick discrimination (PP) and light touch (LT) at 56 dermatomes (28 on each side). A score of 0 (absent), 1 (impaired) or 2 (normal) was assigned to each dermatome for each sensation (PP and LT). The motor scale involves strength testing of 10 upper and lower limb muscles (five muscles per side). A score between 0 and 5 was assigned to each muscle where '0' represents no movement, '1' represents trace movement, '2' represents full movement, gravity eliminated, '3' represents full movement against gravity, '4' represents movement against gravity, some resistance and '5' represents normal strength. Data were de-identified and double-entered into a secure database. Statistical analysis was performed under blinded conditions.

Data analysis

Subjects were grouped by NL: C1–C4 (N=9); C5–C8 (N=8); T1–T6 (N=22); T7–T12 (N=19). Agreement between the two exams for each group at each myotome and dermatome was evaluated by calculating the percent of myotomes and dermatomes with absolute agreement. For those without absolute agreement, kappa (k) and weighted k (k_w) coefficients, calculated using SAS V9.1 (SAS Institute, Cary, NC, USA),were used to correct for chance agreement. Interpretation of the strength of agreement for kappa coefficient was 0.40–0.60, fair; 0.60–0.75, good; and greater than 0.75, excellent.¹⁵ A minimal value of 0.40 was needed to assume adequate agreement.

Results

Myotomes

Agreement between repeated strength scores was strong with absolute agreement in 1153 out of 1160 myotomes tested (99%). In the T1–T6 and T7–T12 groups, there was absolute agreement in 100% of the myotomes. Among the seven myotomes without absolute agreement, all were in the C1–C4 and C5–C8 groups, and only one had a k value less than 0.40 (C6 dermatome). For the C1–C4 group, when disagreement between two motor scores was present (N=four myotomes), it was only by one muscle grade. Likewise in C5–C8 group, with the exception of one subject in which the C6 myotome demonstrated a 3/5–1/5 change, there were two times that repeated motor scores did not agree, and the difference was only one muscle grade. In the C1–C4 group, absolute agreement in muscle strength was achieved caudal to five levels (T1) of the lowest NL (C4) and within one level of the NL in the C5–C8 group.

Dermatomes

Neurological group: C1–C4

PP with the C1–C4 group (Table 4) demonstrated absolute agreement in 32 out of 56 (57%) dermatomes; k ranged from 0.10 to 0. 83 with two dermatomes (left C5; left C7) below 0.40. Weighted k for PP ranged from 0.36 to 0.93 with one dermatome (left T6) below 0.40. For LT (Table 4), there was absolute agreement in 33 out of 56 (59%) dermatomes. LT k values ranged from 0.35 to 0.77 with two dermatomes (right T8; left C3) below 0.40. Weighted k values for LT ranged from 0.34 to 0.84 with three dermatomes (right C5; right T8; left C3) less than 0.40.

Table 4 - Kappa (weighted kappa) values for myotomes (M) and dermatomes (PP=test of discrimination and LT=test for light touch) in subjects with C1–C4 tetraplegia (N=9).

Full table (81K)

Neurological group: C5–C8

As shown in Table 5, for PP, there was absolute agreement in 48 out of 56 (86%) dermatomes with k values ranging from 0.28 to 0.78 with three dermatomes (right C7; left T2; left T3) less than 0.40. Weighted k values for PP ranged from 0.43 to 0.93. For LT (Table 5), there was absolute agreement in 45 out of 56 (80%) dermatomes with k values between 0.10 and 0.80. Four dermatomes (right C6; right T3; left T2; left T3) had k values less than 0.40. Weighted k values ranged from 0.12 to 0.91, with three dermatomes (right T3; left T2; left T3) below 0.40.

Table 5 - Kappa (weighted kappa) values for myotomes (M) and dermatomes (PP=test of discrimination and LT=test for light touch) in subjects with C5–C8 tetraplegia (N=8).

Full table (47K)

Neurological group: T1–T6 paraplegia

For PP in the T1–T6 group (Table 6), there was absolute agreement in 46 out of 56 (82%) dermatomes. PP k values ranged from 0.36 to 0.83, with one dermatome (right T8) having a k value below 0.40. With the exception of one dermatome (right T8), k_w values for PP exceeded 0.40. For LT (Table 6), there was absolute agreement in 43 out of 56 (77%) dermatomes with k values ranging from 0.23 to 0.89, with one (left T8) having a k value below 0.40. For each dermatome, k_w values for LT were greater than 0.40, with the exception of T8 (0.14).

Table 6 - Kappa (weighted kappa) values for myotomes (M) and dermatomes (PP=test of discrimination and LT=test for light touch) in subjects with T1–T6 paraplegia (N=22).

Full table (49K)

Neurological group: T7–T12 paraplegia

For both PP and LT, all kappa values exceeded the minimal value (0.40) for adequate agreement. For PP (Table 7), there was absolute agreement in 46 out of 56 (82%) dermatomes with k values ranging from 0.46 to 0.90; k_w values ranged from 0.54 to 0.90. For LT (Table 7), there was absolute agreement in 47 out of 56 (84%) dermatomes with k values ranging from 0.41 to 0.87; k_w values ranged from 0.52 to 0.94.

Table 7 - Kappa (weighted kappa) values for myotomes (M) and dermatomes (PP=test of discrimination and LT=test for light touch) in subjects with T7–T12 paraplegia (N=19).

Full table (47K)

For the groups with paraplegia, there was absolute agreement in every dermatome caudal to three levels below the lowest NL. For the C5–C8 group, there was absolute agreement in 48 and 45 dermatomes for PP and LT, respectively. For PP and LT, the remaining dermatomes had acceptable agreement. For both sensations, absolute agreement was seen in every dermatome three levels caudal to the NL. The C1–C4 group (N=9) showed the highest disagreement in repeated sensory scores. Unlike the other three subgroups, variation in sensation extended well caudal to the NL, with variation in PP extending to T11 and variation in LT extending to T9.

Discussion

The purpose of this study was to determine the intra-rater agreement of the ISCSCI exam at every dermatome and myotome in four neurological groups. In an effort to minimize variation, the exams were administered by one rater in subjects with chronic and complete SCI.

The perfect agreement in motor scores in the groups with paraplegia was anticipated and is likely because the myotomes fell outside of the levels surrounding the NL such that muscles rostral to the NL (upper extremity) were normal (5/5 strength) and the muscles caudal to the NL (lower extremity) were paralyzed (0/5). For the cervical groups, although the frequency was low and sample size small, there were differences in muscle grades in myotomes at and around the NL. The C6 myotome (wrist extension) had the most discrepancy between exams with strength scores differing more than two muscle grades. All other disagreements involved muscles with strength grades of '2', '3' or '4'; similar to our findings in T1–T6 and T7–T12 groups, in C1–C4 and C5–C8 groups, there were never disagreements in muscles graded as '0' (absent, paralyzed) or '5' (normal, unimpaired).

The results of the sensory data were less straightforward. For PP in both groups with paraplegia, there was absolute agreement in 46 out of 56 (82%) dermatomes. For the remaining 20 dermatomes (10 per neurological group), there was fair agreement in seven of 20 (35%) dermatomes and good or excellent agreement in 12 of 20 (60%) dermatomes. One dermatome (right C8) in the T1–T6 group had inadequate agreement. Only seven comparisons produced results in which one exam suggested absent sensation ('0') and the other exam suggested normal sensation ('2'). No trends in dermatome patterns or subjects were observed in exams having this discrepancy.

For LT, there was absolute agreement in 43 and 47 dermatomes in the T1–T6 and T7–T12 groups, respectively. In the 13 remaining dermatomes in the T1–T6 group, all had good or excellent agreement except for right T6 that had fair agreement and right T8 that had poor agreement. For LT in the T7–T12 group, all dermatomes had fair-to-excellent agreement. A total of eight LT comparisons showed absent ('0') on one exam and normal ('2') on the other exam; four of these cases involved the T8 dermatome. No other dermatome or subject trends were noted.

The extent of disagreement on repeated scores, as seen in the group with high tetraplegia, was unanticipated, particularly as every subject in this subgroup who was more than 1 year postinjury was confirmed to have a complete injury, and a single person performed all exams. Although the sample is small, these data provide an indication that sensory evaluation in persons with high tetraplegia may not be as discrete as in other neurological groups.

Previous studies using variations of the ISCSCI have shown that natural recovery of motor and sensory function occurs even in cases of complete SCI.^{3, 4, 16, 17} Mange³ and Wu⁴ reported changes in upper limb strength 6 and 12 months following injury, respectively. Likewise, Waters et al.¹⁶ reported motor recovery throughout the first year postinjury in persons with paraplegia and tetraplegia.¹⁷ All four studies analyzed repeated strength scores of subjects with newly acquired injuries over their first year postinjury as a basis to establish models that reliably predict the recovery of function. McDonald et al.¹⁸ reported on a single subject with a complete (AIS A) C2 SCI who, 5 years after injury, had substantial motor and sensory recovery as defined by the ISCSCI motor and sensory exams. In contrast, it is unlikely that the subjects in this study had neurological improvement between the first and the second exam due to the short time interval of 2–4 days between testing. Thus, disagreement in scores between exams must be attributed to other sources of variation such as the rater, the subject, the test itself or a combination of all three.

This study differs from the report of Jonsson et al.¹⁹ of agreement for motor, PP and LT scores at individual myotomes and dermatomes. Jonsson's subjects all had incomplete injuries and there were multiple testers. Jonsson concluded that there was only poor-to-fair agreement in motor and sensory scores at individual spinal levels. The discrepancy between the findings of this study and those of Jonsson's is likely attributable to the inclusion of only subjects with AIS 'A' in this study and, in direct contrast, the inclusion of subjects with AIS 'B', 'C' and 'D' in Jonsson's work. More important, in this study, all exams were conducted by one rater. Studies that evaluate agreement among multiple raters, such as those by Jonsson¹⁹ and Savic,¹⁴ provide additional assurance that the results can be generalized to other raters and that they are not simply a reflection of the single rater's ability or bias.

The assignment of subjects into neurological groups is a limitation of this study. Analysis by neurological group rather than by each NL was conducted due to the inadequate sample size representing every single NL. A study with adequate number of subjects with injuries of each NL would negate the need for groupings and would provide a stronger indication of the extent of variation for each NL and more useful parameters to aid in the interpretation of treatment effectiveness.

Conclusion

Our findings support the use of the ISCSCI motor and sensory examinations in young persons with complete SCI. The study results provide useful parameters when interpreting treatment effectiveness of a single-site outcome study when the primary end point is sensation and motor recovery, and the data are collected by a single rater. On the basis of this study, for NL groups C5–C8, T1–T6 and T7–T12, changes in sensation in dermatomes that are three levels caudal to the NL may be more likely a result of a treatment effect as compared to sensory changes within three levels of the NL. This finding does not pertain to changes in sensation in subjects with high tetraplegia where variation in sensation extended well caudal to the NL. In the cervical groups, the C6 myotome (wrist extension) showed the weakest agreement. The results of this study cannot be extrapolated to multicenter studies in which there are subjects with incomplete injuries and newly acquired injuries or to studies that have multiple raters. Future work will build upon the few studies that lend evidence in support of the ISCSCI for evaluation of treatment effectiveness at individual spinal levels in multicenter clinical trials involving subjects with incomplete injuries. Such studies will require a multicenter effort to ensure adequate sample size at each NL.

References

1. Steeves JD, Lammertse D, Curt A, Fawcett JW, Tuszynski MH, Ditunno JF et al. Guidelines for the conduct of clinical trials in spinal cord injury (SCI) as developed by the ICCP panel: clinical trials outcome measures. Spinal Cord 2007; 45: 206–221. | Article | PubMed | ISI | ChemPort |
2. Bracken MB, Shepard MJ, Collins WF, Holford TR, Young W, Baskin DS et al. A randomized controlled trial of methylprednisolone or naloxone in the treatment of acute spinal cord injury. N Engl J Med 1990; 322: 1405–1411. | PubMed | ISI | ChemPort |
3. Mange KC, Ditunno JF, Herbison GJ, Jaweed MM. Recovery of strength at the zone of partial preservation in motor complete and motor incomplete cervical spinal cord injured patients. Arch Phys Med Rehabil 1990; 71: 562–565. | PubMed | ISI | ChemPort |
4. Wu L, Marino RJ, Herbison GJ, Ditunno JF. Recovery of zero-grade muscles in the zone of partial preservation in motor complete quadriplegia. Arch Phys Med Rehabil 1992; 73: 40–43. | PubMed | ISI | ChemPort |
5. Muslumanoglu L, Aki S, Ozturk Y, Filiz M, Karan A, Berker E. Motor, sensory and functional recovery in patients with spinal cord lesions. Spinal Cord 1997; 35: 386–389. | Article | PubMed | ISI | ChemPort |
6. Fawcett JW, Curt A, Steeves JD, Coleman WP, Tuszynski MH, Lammertse D et al. Guidelines for the conduct of clinical trials for spinal cord injury as developed by the ICCP panel: spontaneous recovery after spinal cord injury and statistical power needed for therapeutic clinical trials. Spinal Cord 2007; 45: 190–205. | Article | PubMed | ChemPort |
7. American Spinal Injury Association Reference. Manual for International Standards for Neurological Classification of Spinal Cord Injury. American Spinal Injury Association: Chicago, 2003.
8. Marino RJ, Barros T, Biering-Sorensen F, Burns S, Donovan WH, Graves D et al. International standards for neurological classification of spinal cord injury. J Spinal Cord Med 2003; 26 (supp 1): S50–S56. | PubMed |
9. Cohen ME, Ditunno JF, Donovan WH, Maynard FM. A test of the 1992 international standards for neurological classification of spinal cord injury. Spinal Cord 1998; 36: 554–560. | Article | PubMed | ChemPort |
10. Cohen ME, Sheehan TP, Herbison GJ. Content validity and reliability of the international standards for neurological classification of spinal cord injury. Top Spinal Cord Inj Rehabil 1996; 4: 15–31.
11. Marino RJ, Jones L, Kirshblum S, Tal J. Reliability of the ASIA motor and sensory examination. J Spinal Cord Med 2004; 27: 194.
12. Mulcahey MJ, Gaughan J, Betz RR, Johansen KJ. The international standards for neurological classification of spinal cord injury: reliability of data when applied to children and youths. Spinal Cord 2007; 45: 452–459. | Article | PubMed | ChemPort |
13. Mulcahey MJ, Gaughan J, Betz L, Vogel L. Rater agreement on the ISCSCI motor and sensory scores obtained before and after formal training in testing technique. J Spinal Cord Med 2007; 30: S146–S149. | PubMed |
14. Savic G, Bergstrom EMK, Frankel HL, Jamous MA, Jones PW. Inter-rater reliability of motor and sensory examinations performed according to American spinal injury association standards. Spinal Cord 2007; 45: 444–451. | Article | PubMed | ChemPort |
15. Portney LG, Watkins MP. Foundations of clinical research. applications to practice. (2nd edn). Statistical Measures of Reliability, Chapter 26. Prentice Hall Health: Upper Saddle River, NJ, 2000.
16. Waters RL, Yakura JS, Adkins RH, Sie I. Recovery following complete paraplegia. Arch Phys Med Rehabil 1992; 73: 784–789. | PubMed | ChemPort |
17. Waters RL, Yakura JS, Adkins RH, Sie I. Motor and sensory recovery following complete tetraplegia. Arch Phys Med Rehabil 1993; 74: 242–247. | PubMed | ISI | ChemPort |
18. McDonald JW, Becker D, Sadowsky CL, Jane JA, Conturo TE, Schultz LM. Later recovery following spinal cord injury. J Neurosurg 2002; 97: 252–265. | PubMed |
19. Jonsson M, Tollback A, Gonzales H, Borg J. Inter-rater reliability of the 1992 international standards for neurological and functional classification of incomplete spinal cord injury. Spinal Cord 2000; 38: 675–679. | Article | PubMed | ChemPort |

Acknowledgements

This study was supported by the Shriners Hospitals for Children Research Advisory Board Grant no. 8956.