Abstract
Objectives:
To provide a Korean translation of the International Spinal Cord Injury Basic Pain Data Set (ISCIBPDS) and evaluate the interrater reliability of the translated version.
Setting:
Survey of community-dwelling people with spinal cord injury (SCI) in South Korea.
Methods:
The initial translation was performed by two translators with an in-depth knowledge of SCI, and was then checked by another person with a similar background. A total of 115 SCI participants (87 men, 28 women; 48.4±14.1 years) were evaluated using the Korean version of the ISCIBPDS by two different raters. Intraclass correlation coefficient (ICC) or Cohen’s kappa (κ) was used for analysis.
Results:
All 115 participants had at least one pain problem on both surveys. Seventeen (14.8%) participants described their pain as a single pain problem to one rater while reporting the same pain as two or more different pain problems to the other rater. Twenty-two (19.1%) other participants reported their pain problems in a different order of severity on the surveys. The Korean version of the ISCIBPDS had acceptable interrater reliability, except in the ‘limit activities (how much do you limit your activities in order to keep your pain from getting worse?)’ item (ICC=0.318).
Conclusion:
Provision of criteria for pain separation may facilitate the consistent application of ISCIBPDS. In addition, the ISCIBPDS, which evaluated pain problems separately, reflected the multiple and complex characteristics of SCI-related pain; this was a strength of this data set.
Similar content being viewed by others
Introduction
Most patients with spinal cord injury (SCI) report one or more types of concurrent chronic pain.1, 2, 3 SCI-related pain taxonomies classify pain as neuropathic or nociceptive and according to the level of injury.4, 5 Because many of the pain types experienced after an SCI are refractory to treatment, pain often persists and can even worsen over time; it can also interfere with the patient’s cognitive, emotional and physical functioning.6, 7 In addition, persistent pain superimposed over impairment caused by the SCI further diminishes the patient’s health-related quality of life.8, 9
The use of standardized sets of outcome measures in clinical practice and trials would facilitate research collaboration between clinical centres. The translation, interpretation and application of a standardized set could also improve the management of SCI-related pain. The International Spinal Cord Injury Basic Pain Data Set (ISCIBPDS) was developed to standardize the collection and reporting of pain in the SCI population.10 One of the purposes of developing this data set was to make it usable worldwide by translating it into various languages. This data set consists of pain type, average pain intensity and questions related to the temporal pattern for each specific pain type. The impact of pain on physical, social, and emotional functioning and sleep is also addressed.
Above all, the uniqueness of the ISCIBPDS lies in its design that allows recording of pain separately according to its characteristics (for example, type, average intensity and so on). This approach reflects the clinical picture of SCI-related pain, which is highly complex because multiple pains with different characteristics may be experienced simultaneously in different areas of the body.10
Jensen et al.11 were the first to investigate the internal consistency and concurrent validity of the ISCIBPDS as self-report measures. The aim of the present study was to provide a translation of the ISCIBPDS for Korean people and evaluate the interrater reliability of the translated version.
Materials and methods
The Korean translation of ISCIBPDS
Before initiating this study, permission for translating the data set into Korean was obtained from the Executive Committee on 22 March 2012 by e-mail. The translation procedure was performed according to the recommendations of the Executive Committee.12 The initial translation was performed by two medical doctors who had an in-depth knowledge of SCI and fluency in English, and was later checked by another medical doctor with a similar background. All contents, including introductory text, scoring guidelines, forms and training cases, were translated.
Participants
From March 2013 to August 2013, 125 community-dwelling people with SCI were enroled by recruitment notification. Inclusion criteria were as follows: (1) age ⩾18 years; (2) occurrence of SCI at least 3 months before enrolment; and (3) ability to cooperate. Exclusion criteria were as follows: (1) a history of depression or other mental illness; (2) alcohol abuse; or (3) history of central nervous system disease unrelated to SCI (for example, cerebral palsy and multiple sclerosis). An incentive of 20 000 Korean Won (∼14 Euro) was offered to each of the 125 participants after completion of the second survey.
From the 125 collected samples, 10 were excluded because they did not contain basic information or contained responses that rendered the participant testimony unreliable. Some examples of these are cases with different basic information between the two surveys and cases that were surveyed for a different number of pain problems with the question ‘how many different pain problems did you have?’. Thus, a total of 115 questionnaires were analysed.
Interrater reliability test
Both raters were occupational therapists who were experienced in SCI and were able to perform basic neurological examinations. These raters were instructed by the responsible researcher in the scoring guidelines of the ISCIBPDS. They were also tested using a training case of the data set.
These trained raters visited and interviewed participants individually. All participants were admitted to the study after providing written informed consent and records of personal information and medical history at initial assessment. At the time of each survey, neurological examinations were performed on every participant based on the International Standards for the Neurological Classification of Spinal Cord Injury, including the American Spinal Injury Association Impairment Scale.13 The participants were then evaluated twice using the Korean version of the ISCIBPDS with an interval of 7 days. Each survey was performed independently, and the two raters did not obtain any participant information from each other.
Statistical analysis
Intraclass correlation coefficient (ICC) or Cohen’s Kappa (κ) was calculated to test the interrater agreement. An ICC higher than 0.70 was considered ‘reliable’.14 A κ value higher than 0.60 was defined as ‘good’ agreement.15 All analyses were computed using SPSS 20.0 (IBM Corp., Armonk, NY, USA).
Statement of ethics
We certify that all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during the course of this research (IRB No. B-1208/166-003).
Results
The mean age of the 115 enrolees who participated in the survey was 48.4±14.1 (median: 48.0; range: 18–83) years, and 87 (75.7%) of these participants were men. The average SCI duration was 23.0±35.8 (median: 13.9; range: 3.3–343.3) months. Table 1 shows details of participant demographic data at baseline.
Number of pain problems
All 115 participants had at least one pain problem on both surveys. The mean number of pain problems was 2.2±1.1 on the first survey and 2.3±1.1 on the second survey. The number of pain problems showed good reliability (ICC=0.716; Table 2).
Seventeen (14.8%) participants described their pain as a single pain problem to one rater while reporting the same pain as two or more different pain problems to the other rater. Pain separation in these participants was categorized into two patterns. In the first pattern, 12 participants described pain in different locations as a single pain in one survey while depicting the same pain as two or more different pain problems according to the location in the second survey. For example, case 3 described his neck and shoulder pain as one in the first survey while reporting the same pain as two different problems in the next survey. In the second pattern, six participants described their below-level neuropathic pain as a diffuse type of pain to one rater while reporting multiple below-level pain points to the other rater. A cervical SCI person (case 11) described below-level neuropathic pain as one pain problem to one rater while reporting arm and leg neuropathic pain as two different pain problems to the other rater. One participant (case 28) had both patterns of pain separation.
Order of pain severity
Twenty-two (19.1%) participants reported their pain problems in a different order of severity on the surveys. Of these, 17 participants changed the order of severity on the worst and the second worst pains. The remaining five participants changed the order of severity on the worst, the second worst and third worst pains.
Interrater reliability of the worst pain
Among the 115 participants, 17 with different pain separation and 22 with altered order of pain severity were excluded. Therefore, 76 (66.1%) participants were available for reliability analysis of the worst pain problem. ‘Type of pain (nociceptive vs neuropathic)’ showed good agreement (κ=0.780). Neuropathic pain was more frequently reported than nociceptive pain on both assessments. ‘Average pain intensity’ was also reliable with an ICC of 0.858. Participants rated their average pain intensity as 6.3±1.7 on the first survey and 6.5±1.9 on the second survey. ‘Diurnal peak in intensity’ was analysed among three groups (morning and afternoon vs evening and night vs unpredictable) and showed good agreement (κ=0.693). However, ‘number of days with pain’ and ‘pain duration’ showed slightly less than significant reliability (ICC=0.697 and 0.696, respectively; Table 3).
Among the six pain interference items, ‘social/recreational: changed ability’, ‘activity interference’ and ‘sleep interference’ had good interrater reliability (ICC=0.742, 0.731 and 0.822, respectively). ‘Family-related: changed satisfaction’ and ‘mood interference’ showed slightly less than significant reliability (ICC=0.660 and 0.698, respectively). However, ‘limit activities (how much do you limit your activities in order to keep your pain from getting worse?)’ had the greatest score difference between surveys and did not show good interrater reliability (ICC=0.318; Table 3).
Discussion
Jensen et al.11 evaluated the psychometric properties of a subset of the ISCIBPDS items by self-report measures and confirmed the utility and validity of these items. However, interrater reliability and ‘type of pain (nociceptive vs neuropathic)’ were not included in this previous study because these items were inappropriate for self-report measures.
To the best of the our knowledge, this is the first study to investigate the interrater reliability of the ISCIBPDS. The most important finding in this study was that the number of pain problems reported by some of the participants was different for the two raters. Seventeen (14.8%) individuals in this study described their pain as a single pain problem to one rater while reporting the same pain as two or more different pain problems to the other rater. This difference in the numbering of pain problems was associated with ‘pain locations’; the ‘type of pain (nociceptive vs neuropathic)’ was not altered in most participants.
Because identification of pain locations is scarcely affected by language translation, this finding does not seem to be related to the translation process. The pain area of the data set is divided into eight principal areas, and each of these eight pain locations has 53 subdivisions for a more precise description.3 Although this is an advantage, it might lead to participants reporting different pain separation between surveys. In some participants who experience less localized pain, especially the neuropathic pain experienced by some participants in this study, the pain may be felt as a single and less localized pain problem or multiple and more localized pain problems in adjacent areas, depending on the timing of the interview.
Describing the three worst pain problems and evaluating pain problems separately are unique characteristics of the ISCIBPDS, reflecting the complexity of the pain experienced by SCI participants. Provision of pain separation criteria could help users of ISCIBPDS classify SCI-related pain into a single pain problem or multiple pain problems.
Twenty-two (19.1%) participants altered the order of pain severity (worst, second worst and third worst) at the second survey. If only the worst pain was evaluated, many significant pain experiences could not be evaluated. This finding supported the appropriateness of the ISCIBPDS in describing three worst pain problems rather than describing only the single worst pain to understand the pain experience in SCI participants.
‘Type of pain (nociceptive vs neuropathic)’ was analysed by two groups and showed agreement in 90.8% of the participants. This result was important as treatment of nociceptive and neuropathic pain is quite different.16, 17
Among six pain interference items, ‘limit activities’ had an ICC value of 0.318, whereas ‘pain interference (how much has pain interfered with our day-to-day activities in the last week?)’ had an ICC value of 0.731 (Table 3). ‘Limit activities’ is from the life interference subscale of the Multidimensional Pain Inventory SCI version.18, 19 In a previous study, the life interference subscale had excellent test-retest values (ICC=0.810), and ‘limit activities’ had a factor loading of 0.93 on the life interference factor.18
However, the ‘limit activities’ item of this study did not show good interrater reliability. Pain may interfere with activities and result in performance limitations in some participants. However, in other participants, pain may not cause any performance limitations despite the interference in activities. Although the question ‘how much do you limit your activities in order to keep your pain from getting worse’ is regarding pain-induced limitations in performance, the Korean word ‘jehan’ which corresponds to the word ‘limitation’ seems to be less clear. In other words, ‘jehan’ might mean both: (1) actual limitation in performance and (2) simple difficulties in performing some activities without actual limitation in performance. In version 2 of the ISCIBPDS, the ‘limit activities’ item has been deleted, and only three items asking about interference with day-to-day activities, mood and sleep were selected.20
One of the limitations of this study was that the intrarater reliability was not evaluated. A future study evaluating intra- and interrater reliability may provide more information on the ability of the ISCIBPDS to produce stable and consistent results. Use of a script for the interview might also improve the reliability. In addition, the validity of the ISCIBPDS was not measured. Assessment of validity using the measures of psychological functioning and sleep problems would have been meaningful.
Conclusion
Some of the SCI participants reported a different number of pain problems to the two raters. Providing detailed instructions for pain separation, especially regarding pain localization, may facilitate consistent application of ISCIBPDS. In some participants with a different order of pain severity, the ISCIBPDS, which evaluated the pain problems separately, reflected the multiple and complex characteristics of SCI-related pain; this was a strength of this data set. The Korean version of ISCIBPDS had a good interrater reliability for all items, except the ‘limit activities’ item, which indeed has been deleted in version 2 of the data set.
Data archiving
There were no data to deposit.
References
Rintala DH, Loubser PG, Castro J, Hart KA, Fuhrer MJ . Chronic pain in a community-based sample of men with spinal cord injury: prevalence, severity, and relationship with impairment, disability, handicap, and subjective well-being. Arch Phys Med Rehabil 1998; 79: 604–614.
Turner JA, Cardenas DD, Warms CA, McClellan CB . Chronic pain associated with spinal cord injuries: a community survey. Arch Phys Med Rehabil 2001; 82: 501–509.
Widerstrom-Noga EG, Felipe-Cuervo E, Yezierski RP . Relationships among clinical characteristics of chronic pain after spinal cord injury. Arch Phys Med Rehabil 2001; 82: 1191–1197.
Bryce TN, Ragnarsson KT . Epidemiology and classification of pain after spinal cord injury. Top Spinal Cord Inj Rehabil 2001; 7: 1–17.
Siddall PJ, Yezierski RP, Loeser JD . Pain following spinal cord injury: clinical features, prevalence and taxonomy. IASP Newslett 2001; 3: 3–7.
Murray RF, Asghari A, Egorov DD, Rutkowski SB, Siddall PJ, Soden RJ et al. Impact of spinal cord injury on self-perceived pre- and postmorbid cognitive, emotional and physical functioning. Spinal Cord 2007; 45: 429–436.
Siddall PJ, McClelland JM, Rutkowski SB, Cousins MJ . A longitudinal study of the prevalence and characteristics of pain in the first 5 years following spinal cord injury. Pain 2003; 103: 249–257.
Anson CA, Shepherd C . Incidence of secondary complications in spinal cord injury. Int J Rehabil Res 1996; 19: 55–66.
Westgren N, Levi R . Quality of life and traumatic spinal cord injury. Arch Phys Med Rehabil 1998; 79: 1433–1439.
Widerstrom-Noga E, Biering-Sorensen F, Bryce T, Cardenas DD, Finnerup NB, Jensen MP et al. The International Spinal Cord Injury Pain Basic Data Set. Spinal Cord 2008; 46: 818–823.
Jensen MP, Widerstrom-Noga E, Richards JS, Finnerup NB, Biering-Sorensen F, Cardenas DD . Reliability and validity of the International Spinal Cord Injury Basic Pain Data Set items as self-report measures. Spinal Cord 2010; 48: 230–238.
Biering-Sorensen F, Alexander MS, Burns S, Charlifue S, DeVivo M, Dietz V et al. Recommendations for translation and reliability testing of International Spinal Cord Injury Data Sets. Spinal Cord 2011; 49: 357–360.
Kirshblum SC, Burns SP, Biering-Sorensen F, Donovan W, Graves DE, Jha A et al. International standards for neurological classification of spinal cord injury (revised 2011). J Spinal Cord Med 2011; 34: 535–546.
Spitzer RL, Fleiss JL, Endicott J, Cohen J . Mental status schedule: properties of factor-analytically derived scales. Arch Gen Psychiatry 1967; 16: 479.
Fleiss JL, Nee JC, Landis JR . Large sample variance of kappa in the case of different sets of raters. Psychol Bull 1979; 86: 974.
Finnerup NB, Baastrup C . Spinal cord injury pain: mechanisms and management. Curr Pain Headache Rep 2012; 16: 207–216.
Cardenas DD, Felix ER . Pain after spinal cord injury: a review of classification, treatment approaches, and treatment assessment. PM R 2009; 1: 1077–1090.
Widerstrom-Noga EG, Cruz-Almeida Y, Martinez-Arizala A, Turk DC . Internal consistency, stability, and validity of the spinal cord injury version of the multidimensional pain inventory. Arch Phys Med Rehabil 2006; 87: 516–523.
Widerstrom-Noga EG, Duncan R, Felipe-Cuervo E, Turk DC . Assessment of the impact of pain and impairments associated with spinal cord injuries. Arch Phys Med Rehabil 2002; 83: 395–404.
Widerstrom-Noga E, Biering-Sorensen F, Bryce TN, Cardenas DD, Finnerup NB, Jensen MP et al. The International Spinal Cord Injury Pain Basic Data Set (version 2.0). Spinal Cord 2014; 52: 282–286.
Acknowledgements
This work was supported by the Seoul National University Bundang Hospital, grant number 11-2012-019.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no conflict of interest.
Rights and permissions
About this article
Cite this article
Kim, H., Kim, H., Lee, B. et al. Interrater reliability of the Korean version of the International Spinal Cord Injury Basic Pain Data Set. Spinal Cord 52, 855–858 (2014). https://doi.org/10.1038/sc.2014.105
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/sc.2014.105
This article is cited by
-
Reliability of the Italian version of the International Spinal Cord Injury Pain Basic Data Set
Spinal Cord (2019)
-
Reliability of the International Spinal Cord Injury Upper Extremity Basic Data Set
Spinal Cord (2018)
-
The CanPain SCI Clinical Practice Guidelines for Rehabilitation Management of Neuropathic Pain after Spinal Cord: screening and diagnosis recommendations
Spinal Cord (2016)