Introduction

At an international meeting of experts in spinal cord injury (SCI) data collection and analysis in 2002 the concept of developing International SCI Data Sets to facilitate a common language among SCI centres worldwide was agreed upon.1 Since then, a number of International SCI Data Sets have been developed and published (Table 1). All data sets are created according to an established consensus and approval process, which includes receiving comments from all relevant and interested organisations and individuals in an open iterative process.1 These data sets and accompanying syllabus for each are freely available on the websites of the International Spinal Cord Society (www.iscos.org.uk), and the American Spinal Injury Association (www.asia-spinalinjury.org). As this is an on-going process, several other International SCI Data Sets are in development with various working groups of topic-specific experts.

Table 1 International Spinal Cord Injury (SCI) Data Sets published
Full size table

The rapid development and interest in the International SCI Data Sets has demonstrated the need for developing procedures regarding translation and reliability testing of the Data Sets to enhance their international utility. Therefore, during the Annual Scientific Meeting of International Spinal Cord Society in 2009, the Executive Committee for the International SCI Standards and Data Sets decided to develop recommendations for translating and testing the reliability of the International SCI Data Sets.

Reliability and validity issues

Types of data in the International SCI Data Sets include patient-reported symptoms (for example, location and severity of pain; frequency of involuntary urine leakage within the last 3 months), information from medical records (for example, type of surgical procedures on the gastrointestinal tract), as well as results of investigations (for example, urodynamic investigation). Data sets are not intended to be used as measures; nevertheless they need to be valid and reliable. Content validity is important, whereas construct and convergent validity are only relevant if a data set is used as a measure, as has been done with a self-report version of the International SCI Pain Basic Data Set.10 Content validity is sought by inviting experts to develop a data set and by an exhaustive review process.1 Reliability is a prerequisite to allow comparison of data from different facilities or countries. In the development of the data sets, detailed scoring guidelines and training cases have been provided to enhance reliability.

As data sets are developed to promote worldwide standardization of collection of clinical data, their cross-national comparability is of utmost importance. All data sets have been developed in English by teams of experts from high-resource countries. Content validity therefore has to be considered if a data set is applied in different circumstances, for example, in a low-resource country, or in a country where lay-medicine is frequently used.

Procedures for cross-cultural adaptations of measures have been proposed by different authors,11, 12, 13, 14 typically including independent forward and backward translations, consensus meetings, cognitive interviewing or lay panels, and extensive testing of the translated versions for psychometric properties. These recommendations, however, apply to the field of quality of life measures, where subtle cultural and semantic issues are highly relevant. Compared with quality of life, translation of medical terminology and patient-reported symptoms is relatively straightforward, and it was felt that an exhaustive translation and testing protocol would add an unnecessary barrier to the desired worldwide use of data sets. For this reason, it was decided to provide the following basic guidelines along with optional steps to be taken to translate and study the reliability of an International SCI Data Set.

Translation

The Executive Committee for the International SCI Standards and Data Sets, in cooperation with the working groups for the particular International SCI Data Sets may initiate translations of particular International SCI Data Sets by using known and capable individuals within the SCI community. Specific groups may also notify the committee regarding their wish to translate one or more of the International SCI Data Sets. If the committee finds that the group has the necessary resources and knowledge to make an adequate translation the process can start.

Any translation of data sets should include the introductory text, the scoring guidelines, the form and the training cases (if present). The translation process includes both language translation and cultural adaptation, the latter is necessary if the construct being measured does not exist, does not have the same meaning, or does not have the same role in the target culture.14 Translations of the International SCI Data Sets from the original English version into other languages therefore shall not be made word for word, but will strive to include conceptual equivalence. It is the meaning of the content in the original English version of the data set that should be translated. The translation shall be concise and clear. Therefore it should be simple, and terms or jargon that are not easily understood should be avoided.

As the International SCI Data Sets generally address medical issues and use medical terminology, there may be fewer linguistic problems when they are translated from English to the specific target language. Nonetheless, linguistic differences may constitute cross-cultural barriers for the applicability of some data elements in an International SCI Data Set. Sometimes a term cannot be translated into a target language, because of lack of an exact equivalent idiom or concept in the target language, which may apply to variations in the English language as well. In those cases, finding the conceptual equivalent is more important than a word-for-word translation.

In addition, the meaning of an original term may be modified during translation, as only part of the original meaning may be available in the target language, that is, the item will have a narrower meaning. Likewise, an original meaning may be expanded in the target language, that is, the translated item may have a broader meaning than the original wording.

In a given situation where it is not possible to find a local solution to a translation problem, the chair for the working group of the specific International SCI Data Set should be contacted through the Executive Committee for the International SCI Standards and Data Sets to agree upon a solution for the differences and find a final and satisfactory translation. This process is to be done in cooperation with the working group for the particular International SCI Data Set.

The translation process may also uncover irregularities and misrepresentation in both the original English version and the target language. Therefore, in certain cases re-wording of the original English version may be recommended in order to accomplish a compromise in the content of the data set.

The final translation must include a paragraph indicating the name, role and background of everyone involved in the translation process. In addition, the translation shall be dated and noted with the version number of the original English International SCI Data Set and a translation version number (that is, the first translation is Version 1 and so on). This is to be able to trace any specific versions of translations in the future, particularly if other translations not accepted by the Executive Committee for the International SCI Standards and Data appear or already exist.

Translation procedure

Translation of any specific International SCI Data Set can be accomplished by translation from the English version into the target language, and be followed by a back-translation into English, to confirm that the original meaning has been preserved.

Expert(s) in SCI should perform the initial translation of the specific International SCI Data Set, that is, they must be familiar with the concepts to be translated, both in their own language and in the international literature. It is preferable that two independent translations are made and be compared by a broader expert committee.

The translation back from the target language to English should be performed by linguistic expert(s), with expertise in medical English, who should be blinded to the original English version of the specific International SCI Data Set and be independent of the initial translator(s). Likewise, it is preferable if two independent back translations are performed.

If differences are found between the original English version and the ‘back-translated’ version, the initial translator(s) and back translator(s) should contact the working group for the particular International SCI Data Set involved.

Another approach may also be used, when guidelines and training cases are available explaining the content of the data elements,15, 16 as is the case for the International SCI Data Sets. These guidelines offer the translator(s) in-depth knowledge about the topic. The guidelines take the translator step-by-step through the data elements and give the background for the wording of the particular data element.

Using translators who are working in the field of health and who have knowledge of SCI ensures they have an understanding of the goals of their task. The translators should use the target language as their primary language, can use English as a working language, and have an understanding of the health concepts used, that is, a health researcher or health professional.15, 16

When the translation of the particular International SCI Data Set is made it should be sent to one or more persons to check. The ‘checker(s)’ shall not provide another translation but find out if the translation of the data elements, guidelines and training cases is sufficient to convey the original concepts.15, 16

If there are disagreements between the initial translator(s) and the ‘checker(s)’ that are not solved easily between them, the working group for the particular International SCI Data Set is to be involved as described above.

Reliability

The reliability of an International SCI Data Set can be determined by the reproducibility of the data elements in the particular International SCI Data Set. It is, in essence, an approximation of stability of the elements of the Data Set, when the issue that has to be monitored does not change. For evaluation of the reliability of the International SCI Data Sets, the Executive Committee recommends that, at a minimum, inter-rater reliability testing is performed. The committee believes if inter-rater reliability is good, intra-rater reliability is also likely to be good.17 Guidelines on testing of reliability are available.18 The Executive Committee recommends, in brief, that inter-rater reliability of any International SCI Data Set should be tested with at least two independent observers of the same SCI participant. The time interval between both measures depends on the type of data collected. For information provided by patients, an interval of 1–2 weeks is usual.19 Further, a total of 50 participants should be included, or a minimum of 30 participants in each country, if inter-rater reliability is simultaneously studied in more than one country. Individuals included in reliability testing should be representative of the SCI population with respect to the distribution of difficulties addressed in the International SCI Data Set being evaluated. For example, to evaluate the ‘frequency of faecal incontinence’ in the International SCI Bowel Function Basic Data Set, individuals with daily, occasional and no faecal incontinence should be included.

Appropriate statistics are Kappa (κ) and Intra Class Coefficient as they provide an estimation of agreement corrected for chance. It is important to be aware that κ is reduced and may inaccurately reflect agreement when there is high prevalence of one response (due to the correction for chance agreement).20 In such instances it may be appropriate to consider the percentage agreement, and in these situations it is useful to set an a-priori minimum. For example, (more or less arbitrary) 90% is recommended for variables with two categories and 80% for variables with three or more categories.

The local ethics committee(s) or institutional review board(s) should approve protocols for reliability studies of any International SCI Data Set, and any other necessary legal approvals that may apply should be made and documented in the report of the study.

Generally, standards and regulations for the particular centre and country have to be adhered to.

Epilogue

If future translations or inter-rater reliability studies point out data elements that may need to be worded differently to obtain better meaning or reliability. It is the duty of the working group for the particular International SCI Data Set to re-evaluate the particular data element and, if necessary, to revise the data set and publish the new version on the websites of International Spinal Cord Society (www.iscos.org.uk), and American Spinal Injury Association (www.asia-spinalinjury.org).

It is strongly advised to contact the Executive Committee for the International SCI Standards and Data Sets before making any translation to ensure that this is not already done or in process.