Introduction

The International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) are widely used to determine motor and sensory functions after spinal cord injury (SCI). The ISNCSCI were initially developed as the ASIA (American Spinal Injury Association) Standards for the Classification of Spinal Cord Injuries in 19821 to provide guidelines for consistent classification of neurological level and extent of injury, thereby achieving reliable data for clinical care and research studies.2 These standards are now in their seventh edition.3 A major revision occurred in 1992 when the International Spinal Cord Society (ISCoS, formerly known as the International Medical Society of Paraplegia (IMSoP)) endorsed standards that were published.4 A booklet for the ISNCSCI was first published in 19825 and a Reference Manual in 1994,6 with a second revision in 2003.7 After the 2011 revision of ISNCSCI, the reference manual was no longer up-to-date.2

The ISNCSCI were tested both for their inter-rater reliability8, 9, 10, 11 and intra-rater reliability.11, 12, 13, 14, 15 These studies found that training may improve the accuracy of agreement. Thus, it was suggested that proper training should be done before the use of the ISNCSCI in clinical practice; otherwise, there would be a risk of inaccurate scoring and, hence, improper classification. Attempts have been made to directly test the training effectiveness16, 17, 18 of ISNCSCI. Training methods include the following: an instructional course during annual conference meetings16; lectures on testing techniques, viewing of the testing technique video published by ASIA and hands-on practice with immediate feedback from the instructor17; and, finally, formal presentation, didactics, case reviews and discussion.18 However, all of the attendees at these trainings were professionals in the SCI field, such as physicians, physical therapists, occupational therapists and nurses. Limited knowledge existed about the training effectiveness when teaching the ISNCSCI to individuals with limited experience.

In this study, the training effectiveness when teaching medical students to use the ISNCSCI was examined by comparing their overall performance on a questionnaire that was administered at different points during the training period. In addition, the accuracy in determining sensory level (SL), motor level (ML) and American Spinal Injury Association Impairment Scale (AIS) was calculated.

Materials and methods

In June 2010, during the teaching workshop of physical medicine and rehabilitation in accordance with the syllabus from the medical school, 46 medical students in their fifth year of education (that is, the final year of their preclinical training) were involved in the study. The students were from the Department of Clinical Medicine at the Medical Science Center in Peking University.

Training process

First, students had a 2-h self-study session using the ISNCSCI booklet, followed by a 10-item questionnaire (see Appendix 1). The booklet contains the sixth edition of the ISNCSCI translated into Chinese and published in a Chinese journal.19 The questionnaire was developed to reflect the key points of the ISNCSCI. In the first learning session, the students were allowed to use the booklet as a reference. The answers to the questions were not reviewed with the students at this time. Two days later, the questionnaire was repeated. However, no booklets were allowed for reference during this second assessment. Furthermore, the order of the questions with each consequent assessment was changed in order to avoid learning bias. Immediately following the second assessment, students had a session that involved case discussions for 1 h. The two cases used for discussion were training cases 1 and 4 from the International Spinal Cord Injury Core Data Set (http://www.iscos.org.uk/sitefiles/PageFile_20_coredataset.pdf), using the official translated Chinese version (http://www.iscos.org.uk/sitefiles/Chinese%20translation%20International%20SCI%20Core%20Data%20Set%20%20Version%201.0%20%20-%20%20incl.%20TRAINING%20Cases.pdf) on the ISCoS website. Training case 1 illustrates an injury of C4 AIS A at admission and C5 AIS A at discharge, whereas case 4 is an injury of C6 AIS C at admission and C7 AIS D at discharge. All key points of ISNCSCI were emphasized during these sessions. After discussion, a 30-min session to test the accuracy of determining SL, ML and AIS was performed using training case 3 from the same source. Training case 3 is an injury of C7 AIS B at admission and C8 AIS B at discharge. The data collected from this case contained five items: SL on the left and right, ML on the left and right and AIS. These five items were assessed at admission and discharge, respectively, for a total of 10 items. Finally, the students completed the questionnaire for the third time. In the interval between each of the three assessments, students did not receive information on the results of their evaluations. The correct answers to the questionnaires were given only after the final assessment. The complete training process is shown in Figure 1.

Figure 1
figure 1

Training process.

Full size image

Outcome measure

ISNCSCI key point questionnaire

The questionnaire (Appendix 1) includes 10 questions which were selected to represent key points within the ISNCSCI. Among the questions, two were about sensory examination, motor examination and rectal examination, respectively; the last four questions were about AIS. For sensory examination, key points included definitions of SL and sensory score. For motor examination, key points involved a definition of ML and determination of ML for C5 and L2 spinal cord injuries. For rectal examination, key points included the most caudal aspect of the sacral spinal cord and a determination of the voluntary anal contraction. For the AIS, key points included a determination of AIS during the spinal shock phase, definition of AIS A, determination of AIS C, and documentation of zone of partial preservation. Considering that the content within the ISNCSCI involves a lot of definitions, the questions were shown in the form of a true/false judgment in order to test the understanding of these concepts. The attendees were asked to determine whether each question was true or false. Each correct answer gained a 1 point score for a total of 10 points.

Accurately determining the SL, ML and ASIA Impairment Scale

Establishing the SL, ML and AIS is an important part of the ISNCSCI. In previous training studies with cases,16, 18 the determination of SL, ML and AIS was selected as the main outcome measures of training. Despite these parameters, previous studies also consisted of determining neurological level, severity of injury (complete vs incomplete) and zone of partial preservation. During the development of the training workshop, it was decided that neurological level could be deduced from SL and ML, and severity of injury could be directly determined from the AIS. In addition, training case 3 in the International SCI Core Data Set was an incomplete injury, meaning no zone of partial preservation can be given. This is why it was decided to examine students’ understanding of SL, ML and AIS only. The accuracy rate of each item was calculated from the number of students with a correct determination divided by the total number of students (N=46).

Statistical analyses

Statistical analysis was performed by using SPSS 17.0 (SPSS Inc, Chicago, IL, USA) as follows: differences between means of continuous variables were calculated with Student’s t-test. Significance level was set at P<0.01.

Results

Scores from the ISNCSCI key point questionnaire (Appendix 1) were collected at different times during the training period. After the initial self-study session, the mean score of the first assessment while allowing the use of the booklet as a reference was 7.67±1.49. Two days after the first session and before case discussion, the students answered the questionnaire without a booklet for reference. The mean score of this second test was 7.96±1.15. There was no significant difference between the first and second tests (P=0.14). After the self-study session, most of the errors were found in regard to the determination of ML in C5 or L2, AIS in the spinal shock phase and determination of AIS C via voluntary sphincter contraction. The latter error accounted for the majority of incorrect responses to the questionnaire. Following the introduction of the cases with SCI, the score of the final assessment increased significantly to 9.61±0.88 (P<0.01 and P<0.01, respectively) when compared with the previous two tests.

After the case presentation session, the student response accuracy for SL, ML and AIS is shown in Table 1. The correct answers for SL left and right at admission and discharge ranged from 87.0 to 91.3%, whereas those for ML left and right at admission and discharge ranged from 80.4–89.1%. The rates of correct answers for AIS were 91.3% both at admission and discharge. The overall correct response rates for SL, ML and AIS of the Core Data Set Training Case 3 were 89.1%, 84.8%, and 91.3%, respectively.

Table 1 Accuracy rates in determination of sensory level, motor level and American Spinal Injury Association Impairment Scale of a training case
Full size table

Discussion

The results of this study suggest that greater training effectiveness of ISNCSCI for inexperienced medical students could be achieved from the use of both self-study and case discussion. The response accuracy of SL, ML and AIS in our study ranged from 80.4 to 91.3%. The difference to previous studies is that we tested the training effectiveness in medical students who did not have any previous experience with the ISNCSCI and we used a questionnaire with 10 questions that reflected specific topics that were taught. Incorrect answers could thus show the trainer which topics the students had difficulty in understanding. In addition, we only measured the response accuracy rate of SL, ML and AIS once as the students did not have prior experience in this area. Conversely, previous studies16, 17, 18 typically included two examinations for both pre- and post-training as the trainee already had some experience with ISNCSCI.

For the determination of ML in C5 or L2, a specific section was added in 20095 to explain that whether the C4 (L1) light touch and pin prick are both normal on the right or left side, then this is equivalent to the C4 (L1) motor being grade 5. However, this point was not clearly specified in the sixth edition of the ISNCSCI used in this study. The determination of the AIS during the spinal shock phase could not be found word-for-word in the ISNCSCI booklet. Spinal shock is defined as depressed spinal reflexes caudal to SCI, which is characterized by loss of all neurological activity, including motor, sensory, reflex and autonomic function below the level of injury.20 According to the definition of AIS A, there is no sensory or motor function preserved in the sacral segments S4-5. Thus, the criteria for AIS A are present during the spinal shock phase. Compared with a complete SCI, it seems relatively more difficult to classify incomplete injuries.16 According to Chafetz et al.,18 individuals with motor incomplete injuries represent the greatest challenges for raters. The situation was the same in our study. In the booklet, the definition of AIS C and D is marked ** with the additional explanation: ‘For an individual to receive a grade of C or D, that is, motor incomplete status, they must have either (1) voluntary anal sphincter contraction or (2) sacral sensory sparing (at S4/5 or deep anal pressure) with sparing of motor function more than three levels below the ML for that side of the body’.3 The distinction of AIS C and D would be difficult for inexperienced medical students to understand. In this situation, theory should be in accordance with practice. Therefore, case presentation can help with understanding specialized knowledge that cannot be transmitted only through self-study. In addition, the explanations of these concepts are enhanced in the 2011 revision, thus it is suggested that retesting with the newest version of the ISNCSCI should be carried out in a future study.

The interval between the first and second assessments was based on the curriculum plan for medical students. In addition, the theory of knowledge retention also suggested this practice. According to a review by Custers,21 Ebbinghaus’ curve of forgetting is characterized by large losses at short retention intervals, after which the curve levels off to smaller losses at longer intervals. Most importantly, the form of the retention curve is the same for both meaningless and meaningful materials, but the level of retention is considerably higher in the latter case. At least in the present study, we found that most knowledge transmitted by self-study of the ISNCSCI booklet was not forgotten in 2 days, as indicated by the similar response accuracies in the first and second assessments.

The results of this study further suggest that training does improve effectiveness in learning the ISNCSCI. When teaching ISNCSCI to medical students, the workshop at the final year of their preclinical training is recommended. The method of training should include both self-study and case presentation for at least 4 h. The limitation of this study was the lack of a long-term follow up of knowledge acquisition. A retest interval of 1 or 2 weeks should be investigated in further studies.22 In addition, further studies should attempt to test the training effectiveness of e-Learning materials as these materials already can be obtained from the ASIA website (www.asialearningcenter.com).

Conclusion

The training effectiveness of teaching the ISNCSCI through self-study is reliable. The correct answers to key points could remain for at least 2 days and without the need to use a reference. Some specialized knowledge could not be transmitted only through self-study of the ISNCSCI booklet. When choosing appropriate training cases for presentation and discussion, more detailed key points could be obtained by the trainees. Thus, training cases are a valuable method in the introduction of the ISNCSCI to medical students and should be utilized to improve the effectiveness of training.

Data archiving

There were no data to deposit.