Introduction

Although there is agreement among epidemiologists and clinicians that pain after spinal cord injury (SCI) is common, there has been no consensus on how to define and classify its various types. This has led to an ever-increasing number of different classification schemes of pain after SCI reported in the literature.1 While Hicken et al.1 may have counted 29 different classifications, most of these had not been used more than once in research, if that often. In the years following 2000, only three classifications were being used and developed: the proposed classification of the International Association for the Study of Pain (IASP),2 the Cardenas classification3 and the Bryce–Ragnarsson classification.4, 5 The similarities and differences between these three are discussed by Richards et al.6

In order to address the lack of accord on a single scheme, in 2009 a worldwide group of clinicians and researchers who had written about SCI pain and SCI pain taxonomies in the last decade, convened to consider the development of a consensus classification of SCI pain. Their purpose was to replace the various existing classifications with one that was based on state-of-the-art science, simple and acceptable as the consensus taxonomy by all interested in treating and studying SCI pain.

The International Spinal Cord Injury Pain (ISCIP) Classification they created is designed to be comprehensive and to include pains that are directly related to the SCI, as well as pains that are common after SCI but are not necessarily pathologically and causally related to the injury itself. The complete ISCIP Classification description is published separately and is outlined in Figure 1.7 Figure 1 incorporates the one change made in the classification outline made as a result of this present study, namely addition of ‘autonomic dysreflexia headache’ within Tier 3 of Other nociceptive pain. In line with previous classifications, the ISCIP Classification organizes pain types into a three-tiered structure.2, 4, 8 Tier 1 includes the types of Nociceptive, Neuropathic, Other and Unknown pain. For the neuropathic and nociceptive categories, Tier 2 includes subtypes of pains identified in previous SCI pain Classifications,3, 9, 10, 11 whereas Tier 3 is used to specify the primary pain source at the organ level, as well as the pathology, if either is known. The category of Other pain is used to specify distinct recognized pain entities or syndromes, which do not fulfill the criteria for nociceptive or neuropathic pain, whereas Unknown pain refers to pains that cannot be assigned with any degree of certainty to any of the other categories. If a pain seems to have both nociceptive and neuropathic characteristics, and is not a pain which would be considered Other pain, the classification calls for characterizing separately its component etiologies, one nociceptive and one neuropathic.

Figure 1
figure 1

ISCIP Classification.

Full size image

The objective of this study was to investigate the utility and reliability of the ISCIP Classification as used by physicians and other clinicians who received minimal training in its use. It is essential that the reliability of the use of a classification system for pain is determined, because if two individuals, whether they are clinicians or researchers, cannot agree on what specific types of pain are present, then these different types of pain cannot be studied and specific treatments cannot be prescribed or evaluated with any degree of success. Disagreements in classifying particular instances of pain may be due to unusual features of the pain in question not foreseen in a classification, but also due to omissions and inconsistencies in a classification, or ambiguities in its instructions and descriptions. By having multiple typical users classify the same material in the terms of a taxonomy, many of the latter type of issues can be identified. Disagreements and inability to classify specific cases may even lead to further development of a taxonomy to make it more encompassing.

It was hypothesized in this study that (1) most pain reports can be assigned, with high levels of confidence, to one of the four major types and six subtypes of pain distinguished in the ISCIP Classification (Tiers 1 and 2), and (2) the classification has an inter-rater reliability of more than 95% for allocating pains to the eight (Tier 1 and 2) categories.

Materials and methods

In order to test these hypotheses, a survey was prepared asking physicians and other clinicians who treat patients with SCI and pain, to assign one of the eight Tier 1 and Tier 2 categories described in the classification to a variety of SCI pains presented in vignette format.

Participants

Members of the American Spinal Injury Association (ASIA) and the International Spinal Cord Society (ISCoS) were randomly selected from the membership rosters and invited through email to participate in the project. In addition, the ISCIP Classification development group members recruited colleagues. All received an email message explaining the study and the nature of their role. To be included in the study, participants were expected to be clinicians with some experience treating patients with SCI and pain. Those who agreed to participate received a second email with a more detailed description of the project, information on the Institutional Review Board approval status, specific instructions and an ISCIP Classification self-study guide. (The Institutional Review Board of Mount Sinai School of Medicine reviewed the information submitted using its standard instructions, and decided that this was an exempt study. All its standards for human subject research were adhered to.) The web link to access the survey containing the vignettes was provided individually by a third email sent from the Zoomerang (www.zoomerang.com; Palo Alto, CA, USA) website, which was used to field the survey.

All participants who failed to complete the survey within 4 weeks got at least one, and in most instances multiple, reminders. Figure 2 summarizes the number of invitations sent out and their yield in terms of completed surveys. Invitations were sent out to 536 clinicians of whom 56 fully completed the survey and 24 completed it partially; an arbitrary decision was made to disregard surveys with 10 or fewer vignettes completed.

Figure 2
figure 2

Source of participants, and yield of partial and ‘complete’ surveys.

Full size image

Procedure

Over 100 clinical vignettes (brief case histories) were prepared by the members of the ISCIP Classification group, and assigned to a category of the new classification by consensus of at least three experts, all of whom took part in the development of the taxonomy. Of these, 75 vignettes, selected to provide a varied distribution of pain types and complexity of presentation, were included in this study. Each case scenario presented a person with SCI suffering from one simple- or complex-etiology pain, and provided demographic and injury information (age, gender, neurological level and time since injury), as well as descriptions of symptoms, relevant physical findings, diagnostic test results and previous treatment. For an example of a vignette, see Figure 3. Vignettes varied in their ambiguity, with some vignettes deliberately allowing more than one correct answer or omitting information which would allow a definitive diagnosis. The vignettes were randomly divided into three sets of 25.

Figure 3
figure 3

Example of a vignette, and questions in the survey asked with respect to each vignette presented.

Full size image

Three parallel versions of a survey were designed and distributed over the Internet using Zoomerang to three groups of potential participants. Each contained 25 vignettes to be classified using the new taxonomy, and additional questions seeking information about the respondents and their experience treating patients with SCI and pain. (The only reason for the split into three groups was a desire to have a large number of vignettes evaluated, but not to overly burden any respondent. Vignettes and respondents were assigned to survey versions randomly.) For each vignette, clinicians were asked to decide first on the number of pain components present; then to classify it (or them) by selecting the correct type from a list of combinations of Tier 1 and 2 categories; and finally to type in corresponding Tier 3 information on pain source, pathology and/or specific pain syndrome. Participants used a five-point scale to rate their confidence in the correctness of their decision that there were one or two components to the pain, and in the correctness of their categorization of the pain component or components as to type and subtype.

Data analysis

Responses to completed surveys were downloaded from the Zoomerang site as an Excel file, and then converted into an SPSS data file. Statistical analysis was performed using SPSS version 18. For some results, averages or counts were calculated for each person—for instance, the mean level of confidence, or the percentage of vignettes where the correct pain type was provided. These means or counts were then averaged over individuals for the appropriate tabulations. For most tabulations, however, each vignette completed by each person was considered a separate ‘case’, and tabulations for all these cases or for subgroups of them are reported. Because these observations are not independent from one another (they are nested within individuals), no inferential statistics are presented.

While the hypothesis referred to correct classification at the Tier-2 level using strict criteria only, the finding of many erroneous classifications in the analysis led us to explore some alternatives. Calculation whether respondents were correct in identifying the pain correctly by Tier-1 category was performed as follows: if the experts’ classification was of pain type 1, 2, 3 or 4, and the respondent assigned the pain to any category 1–4, credit was given. Calculation whether respondents were correct in identifying the pain as neuropathic, was similar: if the experts’ classification was of pain type 5, 6, 7 or 8, and the respondent assigned the pain to any category 5–8, credit was given.

In order to give credit to those respondents who may have had difficulty with the Zoomerang site's functioning or otherwise may have had a problem providing the proper code or codes for a vignette where they may actually have known the type or possible types of pain involved, we performed a review that considered the category code(s) registered as well as the material typed in under ‘primary pain generator’, ‘pathology’ and ‘syndrome’. Credit was given under the following circumstances:

  1. 1)

    The respondent selected all the correct potential choices for a vignette for which not enough information was given to exclude one of these correct potential choices. (That is, s/he was over-inclusive.)

  2. 2)

    The respondent indicated that there were two pain components when only one was expected, and the code that was given was correct but it was either repeated under the second component, or the pain type for the second component was left blank. The latter exception potentially does justice to those respondents who initially chose two for the number of pain components for a vignette, but later realized that there was only one component, and who were unable to go back in the Zoomerang survey to cancel the choice that they had made. (The site presented the pain type questions based on the answers to earlier branching questions on the number of pain components; when respondents changed their mind on the branching question when pondering the follow-up question on pain type, it was not simple to go back and change the answer to the question on the number of pain components.) This secondary review also identified additional, numerically less common, patterns:

  3. 3)

    One correct category was chosen but the respondent also selected an additional unexpected pain type code.

  4. 4)

    Only a single correct code was chosen when two were expected, because the respondent had declared the pain to be a two-component pain. Finally, a fifth group was distinguished:

  5. 5)

    Those cases in which the code provided was not correct, but the typed-in cause of pain was thought to be correct (Tier 3).

Results

A total of 56 respondents completed more than 10 vignettes. Their educational and practice characteristics are summarized in Table 1, separately by the set of vignettes they completed. The typical respondent was a physician with Physical and Rehabilitation Medicine specialization, but a number of physical therapists completed a survey, as did some occupational therapists and nurses. Almost half saw 100 or more patients with SCI per year and a third treated over 50 patients with SCI per year for pain. The majority were a member of one or more SCI professional societies, but membership in a pain society was uncommon. Lastly, although almost half of the respondents practiced in the United States, there were significant numbers from Europe, the Middle East and Southeast Asia. Most stated that English was their primary language or they were very comfortable reading scientific material in English, like the survey information.

Table 1 Specialty, training and experience information, by respondent group
Full size table

Confidence in one's answers

The respondents in general had fairly high confidence in the judgments they made as to the number of pain components and in their judgment as to the nature of these pain components—that is, selection of the appropriate ISCIP Classification category (Table 2). For each judgment the mean confidence (on a scale ranging from 1 (not more than a guess) to 5 (absolutely certain)) was calculated. The means of these values across individuals are provided in Table 2. The mean for confidence in the number of components present on average is 4.1. Where respondents decided that a pain had one component, their confidence in their decision on the number of components was somewhat higher (4.2 on average) than when they decided that there were two components to a pain (mean of 3.8).

Table 2 Confidence (mean and s.d. of individual-level averages for up to 25 vignettes) in the correctness of judgments, as to number of pain components and as to appropriateness of the ISCIP Classification category assigned to pain component(s), for 56 respondentsa
Full size table

Confidence in the classification of one-component pains was fairly high too (4.2 on average; Table 2); only in classifying two-component pains were respondents less certain of themselves (3.8 and 3.4 on average, for the first and second component, respectively). In all confidence reports, ‘nothing more than a guess’ was an uncommon rating (1.1%), and even the next higher rating of confidence (2) was selected infrequently (3.5%).

Once a decision had been made as to type of pain described in the vignette, the specific type chosen did not seem to affect confidence on the correctness of one's classification to a significant degree. For one-component pains the mean confidence level was about equally high for most categories (mean of 4.0–4.4); it was, however somewhat lower (2.7–3.2) for types 4 (Nociceptive pain—unknown if musculoskeletal or visceral or other), 8 (Neuropathic pain—unknown if at-level or below-level or other) and 10 (Unknown pain—unknown if nociceptive or neuropathic or other pain; not a defined syndrome). For two-component pains the picture is not as clear; it should be kept in mind, however, that the numbers of judgments involved frequently are very small.

Correctness of answers

According to the experts on pain types, of the 75 vignettes, 70 (93%) presented one-component pains, and for the remaining five it was legitimate to consider them as either a one-component or a two-component pain, for a total of about 7% of the pains potentially having more than just one component (Table 3). The respondents more often thought that the pain described was a two-component one—14.6% of the time. (In addition, they did not answer the question 3.2% of the time, over 56 *25=1400 vignettes.) Consequently, they gave the incorrect answer to the question on the number of components 14.2% the time, if they gave an answer at all.

Table 3 Nature and correctness of responses to the question on the number of pain components, by group
Full size table

Table 4 presents information on the nature of the categorization made by the expert group; as can be seen, the vignettes tended to imitate life, in that a clear answer to the nature of the pain described was not always available, and multiple options were reasonable answers. This table also summarizes the correctness of the classifications made by the respondents, as based on the judgments of the experts. Over all, approximately two-thirds of the answers (65%) were correct. The table in addition presents the percent of correct classifications by the nature of the pain presented in the vignette (according to the experts), for those vignettes where only a single answer was correct and for all vignettes where multiple answers were seen as possible. This suggests that nociceptive musculoskeletal pain was often classified correctly (84% of the time), as was nociceptive visceral pain (85%). The percentages correct are lower for neuropathic below-level pain (73%), neuropathic at-level pain (57%) and ‘other neuropathic pain’ (65%). For the other pain types, incorrect answers are more common than correct ones. Generally, classification of pains for which two or more categories could be correct was not very accurate in spite of the fact that with more correct answers, guessing the right one was more likely: answers to the 19 vignettes involved were correct less often (59%) than answers for the 56 single-answer vignettes (67%).

Table 4 Percent correctly classified vignettes, by nature of pain depicted in a vignette
Full size table

Table 5 offers detail on the wrong answers that were given for the 58 vignettes where according to the experts there was one correct answer only. For instance, of 75 erroneous categorizations of ‘other nociceptive pain’, 10 were ‘nociceptive musculoskeletal pain’, 23 were ‘nociceptive visceral pain’ and 13 were ‘other pain’.

Table 5 Code or code combinations selected in error, by correct classification per the experts, for 58 vignettes with one possible correct answer
Full size table

Correctness of answers at the level of the individual

The average respondent had 86% of the questions on number of pain components correct (s.d. 9%, range 60–100%), and 65% of the questions on taxonomic category (or categories, in the case of a two-component pain) (s.d. 17%, range 24–92%). Differences by language proficiency were minimal. (There was only one person admitting some difficulty with English, so this category was omitted.) Physicians did better than non-physicians (the former had on average 70% of categorizations correct, versus 53% for the latter), but declared sub-specialization in SCI medicine (which both physicians and non-physicians claimed) did not appear to make a difference.

Correctness of answers at the Tier-1 level

The overall correctness in determining whether pain was nociceptive was 78.7% (out of 713 judgments), whereas the overall correctness in determining whether pain was neuropathic was 77.4% (out of 786 judgments).

Correctness of answers under relaxed criteria

In Table 6, the first row (‘Strict’) presents the percent of responses correct under the rule of correct number of components and correct categorization of pain type(s). The second row, labeled ‘Minimally relaxed’ includes under ‘correct’ all correct responses in the first column plus correct answers in groups 1 and 2 above. The overall percent correct increases from 65.2 to 70.3.

Table 6 Correctness of pain type determination by respondents, using experts’ type designation as criterion; strict and relaxed criteria (n=1400 potential responses)
Full size table

When groups 3 and 4 were included under the ‘Moderately relaxed’ criteria of correctness (third row in Table 6) the overall correctness percentage calculated was 79.0%. If we credited responses in which the code provided was not correct, but the cause of pain typed-in in Tier 3 was thought to be correct under the ‘Maximally relaxed’ criteria for correctness (row 4), the overall percent correct increased to 85.4%. These types of answers give some information about how certain pain syndromes are not perceived by the respondents to fit logically into any specific category; autonomic dysreflexia headache is a good example.

Discussion

These findings indicate a less than perfect level of correct use of the ISCIP Classification in a sample of clinicians who, by their own report, are familiar with SCI and pain occurring after SCI. When possible problems resulting from the fairly unforgiving nature of the Zoomerang framework were taken into account, the percent of correct pain classifications was 85%—well below the 95% anticipated.

Respondents were very confident in their choices. In addition, they rarely chose an available selection that indicated uncertainty in diagnosis, even when a vignette could have more than one potential diagnosis. This can be seen by the relatively low correctness percentages for the Other nociceptive (code 3) and Other neuropathic (code 7) subtypes of pain, as well as by the increase in correctness seen when vignettes with additional correct codes (two correct codes given when a single was expected) were counted as correct (the ‘Minimally relaxed’ criteria group in Table 6). In nearly all misdiagnosed cases for which, according to the experts, more information was needed to make a single correct diagnosis by ruling out other potential types, many respondents chose one or more specific diagnoses, either or both of which could have been correct but of which one cannot be certain. (Therefore, an ‘Other nociceptive’ or ‘Other neuropathic’ designation was the correct choice.) This respondent behavior likely mirrors clinical practice, where it is in the patient's best interest if a diagnosis is made so that an attempt at treatment may be undertaken. A tendency by healthcare providers, especially physicians, to be overconfident in making a diagnosis has been seen in other conditions ranging from the diagnosis of pneumonia to the diagnosis of skull fractures.12, 13

Some subtypes of pain proved more challenging to classify than others. Vignettes with autonomic dysreflexia headache as the identifiable type of pain had particularly low rates of correctness of type, even though nearly all respondents had typed in the correct pain source. For this type of pain, it seemed that rather than classifying it as a nociceptive type related to activation of nociceptors or as an Other nociceptive pain, respondents often chose to classify the presumed triggering stimulus or to choose Other pain (syndrome).

For vignettes with abdominal pain as the identifiable type of pain, many respondents also demonstrated poor agreement with the experts, with some characterizing it as visceral pain and others as neuropathic pain. This disagreement paralleled a dispute between the developers of the classification in their discussions, which had led to the decision by the group to include in the original unpublished ISCIP Classification self-study guide accompanying the classification the following statement: ‘If pain is localized to the thorax, abdomen, or pelvis but there is no relation of this pain to any visceral function and there is no evidence of visceral pathology, at-level SCI (neuropathic) pain or below-level (neuropathic) SCI pain should be considered as the cause’. Nevertheless, the ambiguity in the definition, representing differences in expert opinion, likely contributed to the poor classification in these situations, despite it being clear in many vignettes that there was no identifiable abdominal pathology. This seems to mirror the present lack of understanding of the root causes of abdominal pain in persons with SCI. This suggests as well that clinical misdiagnosis of visceral pain may be a frequent phenomenon; it could lead to grave adverse consequences as treatments of neuropathic pain and visceral pain are quite different.

Neuropathic pain attributable to damage to the nerve roots in cauda equina injuries, but experienced in areas of the body more than three levels below the neurological level of injury (NLI), was often incorrectly classified as below-level pain. The original ISCIP Classification self-study guide included the following definition: ‘At-level SCI (neuropathic) pain refers to neuropathic pain perceived in a segmental pattern anywhere within the dermatome of the NLI and/or within the three dermatomes below this level and not in any lower dermatomes, unless the pain is thought be caused by damage to the cauda equina, in which case it may be perceived in lower dermatomes.’ The reason for misclassification could be that it is not immediately intuitive that pain extending below three dermatomes below the NLI can be at-level pain and is not by definition below-level pain. In addition, there were a significant number of respondents who misdiagnosed at-level and below-level neuropathic pain when the cauda equina was not involved. Reasons for this could be the variability of dermatomal maps, especially as respondents mentally mapped pain locations to remembered dermatomes, as well as the non-dermatomal or regional presentations of pain within the vignettes, for which an area of the body may span several dermatomes.

The other group of pains which demonstrated less than optimal agreement were those which were identified as non-neuropathic and non-nociceptive by the experts, describing a syndrome such as complex regional pain syndrome type I (CRPS type I) or fibromyalgia. These were often not recognized as syndromes by the respondents. Conversely, pain which the experts had identified as at-level pain with autonomic features was sometimes diagnosed as CRPS type I, despite there being evidence of a neurological injury at the level, namely the SCI. The reasons for this could include the fact that the concept of non-neuropathic, non-nociceptive pain is a relatively new one, which may not have been encountered previously by the respondents, specifically with regard to the case of at-level pain with autonomic features; in the past this entity has been described extensively as CRPS type I in the literature.14, 15, 16, 17, 18, 19

The often poor performance of the respondents may be due to use of written vignettes, rather than videotaped presentations or actual live patients. The vignette does not allow follow-up testing or questioning if the initial material is ambiguous. As well, a vignette sometimes by necessity describes in clear terms what may be much more indefinite in a clinical presentation. In addition, the platform we chose for presenting the vignettes, Zoomerang, is rather unforgiving of errors; once respondents had committed themselves to either a one-component or a two-component diagnosis, changing their mind was complicated if not impossible. Thus, a test of the ISCIP Classification using either a more flexible (and possible richer) simulation using vignettes or a test using real patients is necessary before it can be concluded that this new classification of post-SCI pain should be used in preference to the taxonomies it was designed to replace.

Conclusions

The reliability of the use of the ISCIP Classification is moderate as tested using a clinical vignette approach by physicians and other clinicians who received minimal training in use of the classification. The ISCIP Classification should be tested for reliability with real patients with pain after SCI. The instructions for using the ISCIP Classification and its component descriptions have been rewritten7 to further clarify the issues identified by the present validation study.

Data Archiving

There were no data to deposit.