Original Article

Spinal Cord (2011) 49, 357–360; doi:10.1038/sc.2010.153; published online 9 November 2010

Recommendations for translation and reliability testing of international spinal cord injury data sets

F Biering-Sørensen1, M S Alexander2, S Burns3, S Charlifue4, M DeVivo5, V Dietz6, A Krassioukov7, R Marino8, V Noonan9, M W M Post10, T Stripling11, L Vogel12 and P Wing9

  1. 1Clinic for Spinal Cord Injuries, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
  2. 2Renown Rehabilitation Hospital, Reno, NV, USA
  3. 3Department of Rehabilitation Medicine, University of Washington, Seattle, WA, USA
  4. 4Craig Hospital, Englewood, CO, USA
  5. 5Department of Physical Medicine and Rehabilitation, University of Alabama at Birmingham, Birmingham, AL, USA
  6. 6Spinal Cord Injury Center Balgrist University Hospital, Zurich, Switzerland
  7. 7International Collaboration On Repair Discoveries (ICORD), University of British Columbia and Vancouver Coastal Health, Vancouver, British Columbia, Canada
  8. 8Department of Rehabilitation Medicine, Jefferson Medical College of Thomas Jefferson University, Philadelphia, PA, USA
  9. 9Division of Spine, University of British Columbia, Vancouver, British Columbia, Canada
  10. 10Rudolf Magnus Institute of Neuroscience and Centre of Excellence in Rehabilitation Medicine, University Medical Centre Utrecht and De Hoogstraat, Utrecht, The Netherlands
  11. 119614 Larkview Court, Fairfax Station, VA, USA
  12. 12Shriners Hospital for Children and Rush Medical College, Chicago, IL, USA

Correspondence: Professor F Biering-Sørensen, Clinic for Spinal Cord Injuries, Rigshospitalet, Copenhagen University Hospital, Rigshospitalet TH2091, Blegdamsvej 9, Copenhagen DK-2100, Denmark. E-mail: finbs@rh.regionh.dk

Received 22 June 2010; Accepted 28 September 2010; Published online 9 November 2010.

Top

Abstract

Objective:

 

To provide recommendations regarding translation and reliability testing of International Spinal Cord Injury (SCI) Data Sets.

Setting:

 

The Executive Committee for the International SCI Standards and Data Sets.

Recommendations:

 

Translations of any specific International SCI Data Set can be accomplished by translation from the English version into the target language, and be followed by a back-translation into English, to confirm that the original meaning has been preserved. Another approach is to have the initial translation performed by translators who have knowledge of SCI, and afterwards controlled by other person(s) with the same kind of knowledge. The translation process includes both language translation and cultural adaptation, and therefore shall not be made word for word, but will strive to include conceptual equivalence. At a minimum, the inter-rater reliability should be tested by no less than two independent observers, and preferably in multiple countries. Translations must include information on the name, role and background of everyone involved in the translation process, and shall be dated and noted with a version number.

Conclusion:

 

By following the proposed guidelines, translated data sets should assure comparability of data acquisition across countries and cultures. If the translation process identifies irregularities or misrepresentation in either the original English version or the target language, the working group for the particular International SCI Data Set shall revise the data set accordingly, which may include re-wording of the original English version in order to accomplish a compromise in the content of the data set.

Keywords:

spinal cord injury; international data sets; translation; reliability

Top

Introduction

At an international meeting of experts in spinal cord injury (SCI) data collection and analysis in 2002 the concept of developing International SCI Data Sets to facilitate a common language among SCI centres worldwide was agreed upon.1 Since then, a number of International SCI Data Sets have been developed and published (Table 1). All data sets are created according to an established consensus and approval process, which includes receiving comments from all relevant and interested organisations and individuals in an open iterative process.1 These data sets and accompanying syllabus for each are freely available on the websites of the International Spinal Cord Society (www.iscos.org.uk), and the American Spinal Injury Association (www.asia-spinalinjury.org). As this is an on-going process, several other International SCI Data Sets are in development with various working groups of topic-specific experts.


The rapid development and interest in the International SCI Data Sets has demonstrated the need for developing procedures regarding translation and reliability testing of the Data Sets to enhance their international utility. Therefore, during the Annual Scientific Meeting of International Spinal Cord Society in 2009, the Executive Committee for the International SCI Standards and Data Sets decided to develop recommendations for translating and testing the reliability of the International SCI Data Sets.

Reliability and validity issues

Types of data in the International SCI Data Sets include patient-reported symptoms (for example, location and severity of pain; frequency of involuntary urine leakage within the last 3 months), information from medical records (for example, type of surgical procedures on the gastrointestinal tract), as well as results of investigations (for example, urodynamic investigation). Data sets are not intended to be used as measures; nevertheless they need to be valid and reliable. Content validity is important, whereas construct and convergent validity are only relevant if a data set is used as a measure, as has been done with a self-report version of the International SCI Pain Basic Data Set.10 Content validity is sought by inviting experts to develop a data set and by an exhaustive review process.1 Reliability is a prerequisite to allow comparison of data from different facilities or countries. In the development of the data sets, detailed scoring guidelines and training cases have been provided to enhance reliability.

As data sets are developed to promote worldwide standardization of collection of clinical data, their cross-national comparability is of utmost importance. All data sets have been developed in English by teams of experts from high-resource countries. Content validity therefore has to be considered if a data set is applied in different circumstances, for example, in a low-resource country, or in a country where lay-medicine is frequently used.

Procedures for cross-cultural adaptations of measures have been proposed by different authors,11, 12, 13, 14 typically including independent forward and backward translations, consensus meetings, cognitive interviewing or lay panels, and extensive testing of the translated versions for psychometric properties. These recommendations, however, apply to the field of quality of life measures, where subtle cultural and semantic issues are highly relevant. Compared with quality of life, translation of medical terminology and patient-reported symptoms is relatively straightforward, and it was felt that an exhaustive translation and testing protocol would add an unnecessary barrier to the desired worldwide use of data sets. For this reason, it was decided to provide the following basic guidelines along with optional steps to be taken to translate and study the reliability of an International SCI Data Set.

Top

Translation

The Executive Committee for the International SCI Standards and Data Sets, in cooperation with the working groups for the particular International SCI Data Sets may initiate translations of particular International SCI Data Sets by using known and capable individuals within the SCI community. Specific groups may also notify the committee regarding their wish to translate one or more of the International SCI Data Sets. If the committee finds that the group has the necessary resources and knowledge to make an adequate translation the process can start.

Any translation of data sets should include the introductory text, the scoring guidelines, the form and the training cases (if present). The translation process includes both language translation and cultural adaptation, the latter is necessary if the construct being measured does not exist, does not have the same meaning, or does not have the same role in the target culture.14 Translations of the International SCI Data Sets from the original English version into other languages therefore shall not be made word for word, but will strive to include conceptual equivalence. It is the meaning of the content in the original English version of the data set that should be translated. The translation shall be concise and clear. Therefore it should be simple, and terms or jargon that are not easily understood should be avoided.

As the International SCI Data Sets generally address medical issues and use medical terminology, there may be fewer linguistic problems when they are translated from English to the specific target language. Nonetheless, linguistic differences may constitute cross-cultural barriers for the applicability of some data elements in an International SCI Data Set. Sometimes a term cannot be translated into a target language, because of lack of an exact equivalent idiom or concept in the target language, which may apply to variations in the English language as well. In those cases, finding the conceptual equivalent is more important than a word-for-word translation.

In addition, the meaning of an original term may be modified during translation, as only part of the original meaning may be available in the target language, that is, the item will have a narrower meaning. Likewise, an original meaning may be expanded in the target language, that is, the translated item may have a broader meaning than the original wording.

In a given situation where it is not possible to find a local solution to a translation problem, the chair for the working group of the specific International SCI Data Set should be contacted through the Executive Committee for the International SCI Standards and Data Sets to agree upon a solution for the differences and find a final and satisfactory translation. This process is to be done in cooperation with the working group for the particular International SCI Data Set.

The translation process may also uncover irregularities and misrepresentation in both the original English version and the target language. Therefore, in certain cases re-wording of the original English version may be recommended in order to accomplish a compromise in the content of the data set.

The final translation must include a paragraph indicating the name, role and background of everyone involved in the translation process. In addition, the translation shall be dated and noted with the version number of the original English International SCI Data Set and a translation version number (that is, the first translation is Version 1 and so on). This is to be able to trace any specific versions of translations in the future, particularly if other translations not accepted by the Executive Committee for the International SCI Standards and Data appear or already exist.

Translation procedure

Translation of any specific International SCI Data Set can be accomplished by translation from the English version into the target language, and be followed by a back-translation into English, to confirm that the original meaning has been preserved.

Expert(s) in SCI should perform the initial translation of the specific International SCI Data Set, that is, they must be familiar with the concepts to be translated, both in their own language and in the international literature. It is preferable that two independent translations are made and be compared by a broader expert committee.

The translation back from the target language to English should be performed by linguistic expert(s), with expertise in medical English, who should be blinded to the original English version of the specific International SCI Data Set and be independent of the initial translator(s). Likewise, it is preferable if two independent back translations are performed.

If differences are found between the original English version and the ‘back-translated’ version, the initial translator(s) and back translator(s) should contact the working group for the particular International SCI Data Set involved.

Another approach may also be used, when guidelines and training cases are available explaining the content of the data elements,15, 16 as is the case for the International SCI Data Sets. These guidelines offer the translator(s) in-depth knowledge about the topic. The guidelines take the translator step-by-step through the data elements and give the background for the wording of the particular data element.

Using translators who are working in the field of health and who have knowledge of SCI ensures they have an understanding of the goals of their task. The translators should use the target language as their primary language, can use English as a working language, and have an understanding of the health concepts used, that is, a health researcher or health professional.15, 16

When the translation of the particular International SCI Data Set is made it should be sent to one or more persons to check. The ‘checker(s)’ shall not provide another translation but find out if the translation of the data elements, guidelines and training cases is sufficient to convey the original concepts.15, 16

If there are disagreements between the initial translator(s) and the ‘checker(s)’ that are not solved easily between them, the working group for the particular International SCI Data Set is to be involved as described above.

Reliability

The reliability of an International SCI Data Set can be determined by the reproducibility of the data elements in the particular International SCI Data Set. It is, in essence, an approximation of stability of the elements of the Data Set, when the issue that has to be monitored does not change. For evaluation of the reliability of the International SCI Data Sets, the Executive Committee recommends that, at a minimum, inter-rater reliability testing is performed. The committee believes if inter-rater reliability is good, intra-rater reliability is also likely to be good.17 Guidelines on testing of reliability are available.18 The Executive Committee recommends, in brief, that inter-rater reliability of any International SCI Data Set should be tested with at least two independent observers of the same SCI participant. The time interval between both measures depends on the type of data collected. For information provided by patients, an interval of 1–2 weeks is usual.19 Further, a total of 50 participants should be included, or a minimum of 30 participants in each country, if inter-rater reliability is simultaneously studied in more than one country. Individuals included in reliability testing should be representative of the SCI population with respect to the distribution of difficulties addressed in the International SCI Data Set being evaluated. For example, to evaluate the ‘frequency of faecal incontinence’ in the International SCI Bowel Function Basic Data Set, individuals with daily, occasional and no faecal incontinence should be included.

Appropriate statistics are Kappa (κ) and Intra Class Coefficient as they provide an estimation of agreement corrected for chance. It is important to be aware that κ is reduced and may inaccurately reflect agreement when there is high prevalence of one response (due to the correction for chance agreement).20 In such instances it may be appropriate to consider the percentage agreement, and in these situations it is useful to set an a-priori minimum. For example, (more or less arbitrary) 90% is recommended for variables with two categories and 80% for variables with three or more categories.

The local ethics committee(s) or institutional review board(s) should approve protocols for reliability studies of any International SCI Data Set, and any other necessary legal approvals that may apply should be made and documented in the report of the study.

Generally, standards and regulations for the particular centre and country have to be adhered to.

Top

Epilogue

If future translations or inter-rater reliability studies point out data elements that may need to be worded differently to obtain better meaning or reliability. It is the duty of the working group for the particular International SCI Data Set to re-evaluate the particular data element and, if necessary, to revise the data set and publish the new version on the websites of International Spinal Cord Society (www.iscos.org.uk), and American Spinal Injury Association (www.asia-spinalinjury.org).

It is strongly advised to contact the Executive Committee for the International SCI Standards and Data Sets before making any translation to ensure that this is not already done or in process.

Top

Conflict of interest

The authors declare no conflict of interest.

Top

References

  1. Biering-Sørensen F, Charlifue S, DeVivo M, Noonan V, Post M, Stripling T et al. International spinal cord injury data sets. Spinal Cord 2006; 44: 530–534. | Article | PubMed | ChemPort |
  2. DeVivo M, Biering-Sørensen F, Charlifue S, Noonan V, Post M, Stripling T et al. International spinal cord injury core data set. Spinal Cord 2006; 44: 535–5340. | Article | PubMed | ISI | ChemPort |
  3. Biering-Sørensen F, Craggs M, Kennelly M, Schick E, Wyndaele JJ. International lower urinary tract function basic spinal cord injury data set. Spinal Cord 2008; 46: 325–330. | Article | PubMed | ChemPort |
  4. Biering-Sørensen F, Craggs M, Kennelly M, Schick E, Wyndaele JJ. International urodynamic basic spinal cord injury data set. Spinal Cord 2008; 46: 513–516. | Article | PubMed | ChemPort |
  5. Widerström-Noga E, Biering-Sørensen F, Bryce T, Cardenas DD, Finnerup NB, Jensen MP et al. The international spinal cord injury pain basic data set. Spinal Cord 2008; 46: 818–823. | Article | PubMed
  6. Krogh K, Perkash I, Stiens SA, Biering-Sørensen F. International bowel function basic spinal cord injury data set. Spinal Cord 2009; 47: 230–234. | Article | PubMed | ISI | ChemPort |
  7. Krogh K, Perkash I, Stiens SA, Biering-Sørensen F. International bowel function extended spinal cord injury data set. Spinal Cord 2009; 47: 235–241. | Article | PubMed | ISI | ChemPort |
  8. Biering-Sørensen F, Craggs M, Kennelly M, Schick E, Wyndaele J-J. International urinary tract imaging basic spinal cord injury data set. Spinal Cord 2009; 47: 379–383. | Article | PubMed
  9. Krassioukov A, Alexander MS, Karlsson AK, Donovan W, Mathias CJ, Biering-Sørensen F. International spinal cord injury cardiovascular function basic data set. Spinal Cord 2010; 48: 586–590. | Article | PubMed | ISI | ChemPort |
  10. Jensen MP, Widerström-Noga E, Richards JS, Finnerup NB, Biering-Sørensen F, Cardenas DD. Reliability and validity of the international spinal cord injury basic pain data set items as self-report measures. Spinal Cord 2010; 48: 230–238. | Article | PubMed | ISI
  11. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 1993; 46: 1417–1432. | Article | PubMed | ISI | ChemPort |
  12. Bullinger M, Alonso J, Apolone G, Leplege A, Sullivan M, Wood-Dauphinee S et al. Translating health status questionnaires and evaluating their quality: The IQOLA project approach. J Clin Epidemiol 1998; 51: 913–923. | Article | PubMed | ISI | ChemPort |
  13. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000; 25: 3186–3191. | Article | PubMed | ISI | ChemPort |
  14. Price VE, Klaassen RJ, Bolton-Maggs PHB, Grainger JD, Curtis C, Wakefield C et al. Measuring disease-specific quality of life in rare populations: a practical approach to cross-cultural translation. Health Qual Life Outcomes 2009; 7: 92.  | Article | PubMed
  15. Partnership on health statistics. Core group health and health related surveys (HIS core group). Guidelines for the Development and Criteria for the Adoption of Health Survey Instruments. European Commission: Eurostat. pp 27–32. http://ec.europa.eu/health/ph_information/dissemination/reporting/healthsurveys_en.pdf. (accessed 16 October 2010).
  16. Robine J-M, Jagger C. Development of a household survey module on health status measurement: Final report. Report to Eurostat on European Health Status Module. Montpellier (France), Euro-REVES, November 2003 p 13. http://circa.europa.eu/Public/irc/dsis/health/library?l=/reports/healthsinterviewssurvey/ehis_european_health/euroreves_102003/reportstoseurostatsonshe/_EN_1.0_&a=d.(accessed April 1 2010).
  17. Holmefur M, Krumlinde-Sundholm L, Eliasson AC. Interrater and intrarater reliability of the assisting hand assessment. Am J Occup Ther 2007; 61: 79–84. | PubMed | ISI |
  18. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60: 34–42. | Article | PubMed | ISI
  19. Streiner DL, Norman GR. Health measurementscales: a practical guide to their development and use. 2 ed Oxford University Press: Oxford, 1995.
  20. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005; 85: 257–268. | PubMed | ISI |