Introduction

People with spinal cord injury (SCI) are susceptible to various secondary health conditions (that is, complications). These conditions have an impact on physical health, psychological well-being, quality of life and community participation.1, 2, 3, 4, 5 The common secondary health conditions after SCI include pressure ulcers, urinary tract infections, deep vein thrombosis, spasticity, pain, respiratory problems and autonomic dysreflexia.6 Some of these can be life threatening. In high-income countries, people with SCI are regularly followed up and assessed by clinicians for secondary health conditions. However, this level of service is often not feasible in low- and middle-income countries (LMIC). It is therefore important to find inexpensive ways to screen people for secondary health conditions that do not require costly and regular face-to-face assessments. If those at high risk of secondary health conditions could be identified, then they could be targeted for follow-up with clinicians and further investigation.

There is currently only one assessment scale that is quick and easy to use and may provide a way for clinicians in LMIC to screen people with SCI for secondary health conditions. The assessment scale is called the Spinal Cord Injury–Secondary Conditions Scale (SCI–SCS).7 It was first published in 2007 and is an adaptation of the generic Seekins Secondary Condition Questionnaire for people with injury-related disabilities.8, 9 The scale to date has received little attention, although a modified version of the scale has been used as an outcome measure in at least one study.10, 11 It is a 16-item questionnaire that covers common health conditions related to SCI. Some of the items focus on the skin, musculoskeletal system, pain, bowel function, bladder function and the cardiovascular system. Patients rate each item on a 0–3 scale where a score of 0 indicates that the condition has not been experienced or has not been a significant problem in the last 3 months, and a score of 3 indicates that the condition is chronic and/or a serious problem. A total score is derived by adding the scores for each item to a total possible score of 48. A higher score indicates more severe secondary conditions compared with a lower score.

The original version of the SCI–SCS was designed to be self-administered through a pencil and paper questionnaire given to people with SCI. However, there are problems with this mode of delivery in LMIC. First, it relies on literacy, and, second, compliance with posted questionnaires is often poor. The solution may be to administer the SCI–SCS by telephone. This is feasible because of the high rate of mobile phone usage in LMIC.12 Of course responses to any questions administered over the telephone will never replace a thorough face-to-face assessment with medical investigations, however, it may help identify patients who are at high risk and in need of further attention. In countries without follow-up services, this may provide a starting point for prioritising health-care services. If patients could be readily screened by telephone, then resources could be directed to those at high risk of secondary health conditions. However, before recommending widespread use of a telephone-based version of the SCI–SCS, it is important to determine its inter-rater reliability and validity. Therefore, the aim of this study was to determine the inter-rater reliability of the telephone-based version of the SCI–SCS administered twice by two different assessors. The secondary aim was to determine the validity of the SCI–SCS. This was done by comparing the telephone-based version of SCI–SCS with the original paper-based version of SCI–SCS. The paper-based version was considered the gold standard.

Methods

Participants

Forty inpatients and outpatients were recruited from the Royal North Shore Hospital, Sydney, Australia. Participants were included if they had a complete or an incomplete traumatic or non-traumatic SCI and had sustained their SCI more than 3 months prior. Participants were excluded if they had a cognitive or a verbal impairment. They were also excluded if they had a clinically significant or an unstable medical condition including psychiatric, behavioural or terminal illness that would have compromised participation in the study. We certify that all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during the course of this research. All participants gave consent to participate.

Procedure

Initially, demographic data to describe the population were collected. This included the American Spinal Injury Association Impairment Scale and the neurological level of injury according to the International Standards for Neurological Classification of Spinal Cord Injury.13 Then, the SCI–SCS was administered on three occasions: twice using the telephone-based version and once using the paper-based version. The order of testing was randomised, and all three assessments were conducted blinded to each other. The median (interquartile range) time between the first and second assessment was 5 (4–6) days and between the second and third assessment was 4 (3–5) days. Each assessment took ~8 min, and the questions of the SCI–SCS were presented in the same order.

The telephone-based version was administered by two different telephone assessors. The two assessors were blinded to each other’s assessment during the conduct of the study. They were also instructed and encouraged not to disclose or discuss any part of their assessments during the conduct of study. They read the assessment items to participants according to a formalised script. Participants were asked to provide a score for each item, and the assessor then marked the score on the assessment form.

The paper-based version of the SCI–SCS was completed by participants either at an inpatient department, an outpatient clinic or at home. Participants were asked to mark a score for each item. Participants were provided with assistance by family members or independent staff to complete the forms if they had limited or no hand function.

Data analysis

Data were captured using the Research Electronic Data Capture (REDCap) Software (Nashville, TN, USA)14 and analysed using STATA 13 for Windows. The inter-rater reliability and convergent validity of the telephone-based version of the SCI–SCS were determined using intraclass correlation coefficients (ICCs), per cent close agreements and Bland–Altman plots. Reliability was determined by comparing the results of the two telephone assessments of the SCI–SCS, and validity was determined by comparing the mean results of the two telephone-based assessments with the paper-based assessments of the SCI–SCS. The ICC values were interpreted according to a rating system suggested by Shrout and Fleiss15 (>0.75 excellent, 0.40–0.75 fair to good reliability and <0.40 poor reliability).

Results

The demographic characteristics of participants are shown in Table 1. The ICC (95% confidence interval) reflecting the agreement of the two telephone-based assessments was 0.96 (0.93–0.98). The agreement between these two assessments is displayed in the Bland–Altman plot (Figure 1), and the per cent close agreements are shown in Table 2. In summary, the two telephone-based scores were within one, two, three and four points (out of 48 points), 60%, 85%, 93% and 95% of the time, respectively, and there was very little systematic bias between the two assessments.

Table 1 Characteristics of participants (n=40)
Full size table
Figure 1
figure 1

Bland–Altman plot displaying the agreement between the two telephone-based assessments of SCI-SCS. The mean (95% CI) reflecting bias is indicated.

Full size image
Table 2 Per cent close agreement (cumulative percentages) of the repeat assessments using the Spinal Cord Injury–Secondary Condition Scale
Full size table

The ICC (95% confidence interval) reflecting the agreement between the paper-based assessments and telephone-based assessments of SCI–SCS was 0.90 (0.83–0.95). The agreement between the two assessments is displayed in the Bland–Altman plot (Figure 2), and the per cent close agreements are shown in Table 2. In summary, the paper-based and telephone-based scores were within one, two, three and four points (out of 48 points), 60%, 80%, 80% and 88% of the time, respectively, and, there was very little systematic bias between the two assessments, although there was one outlier in which a telephone-based score differed from the paper-based score by 16 points (see Figure 2). This data point was not removed from the analysis to ensure that the results reflected a conservative estimate of convergent validity.

Figure 2
figure 2

Bland–Altman plot displaying the agreement between the paper-based assessments and the telephone-based assessments of SCI-SCS. The mean (95% confidence interval: CI) reflecting bias is indicated.

Full size image

Discussion

The results of this study indicate that the telephone-based version of SCI–SCS is reliable and valid when compared with the paper-based version of the SCI–SCS. These findings have important practical implications because the scale could be administered by telephone in LMIC where it may be cost prohibitive to provide regular face-to-face follow-up services for all people with SCI.16 A telephone interview overcomes problems of illiteracy and may be associated with a better response rate than mailed questionnaires.

Clearly, no telephone interview that relies on self-report can replace a thorough clinical assessment. There will always be the potential for patients to underestimate the severity of a secondary health condition or be unaware to the existence of a secondary health condition. The SCI–SCS does, however, reflect the perceptions of people with SCI. For some secondary conditions at least, the perceptions of people with SCI may be important. For example, people’s perceptions about their spasticity will often drive their treatment.17 However, people’s perceptions are not always sufficient to detect serious underlying secondary health conditions. For example, people with SCI may have very poor perception about the severity of a skin problem; hence, this will nearly always need verifying through a clinical assessment. Similarly, some other common problems may not always be apparent to people with SCI but may need attention to avoid long-term and more severe secondary conditions.6 Hence, although the telephone-based version of SCI–SCS may be reliable, the question remains as to whether it can be relied upon to successfully screen people with SCI for problems or to help flag patients at high risk and in need of more attention and follow-up. To clarify these issues, research attention now needs to be directed at comparing patients’ scores on the telephone-based version of SCI–SCS with the results of an experienced clinician’s examination. Alternatively, the usefulness of a telephone-based version of the SCI–SCS to detect secondary health conditions in LMIC could be tested within a trial. These are, however, costly and complex types of studies that would not have been sensible to undertake before at least ensuring that the telephone-based version of SCI–SCS was reliable and gave comparable results to the paper-based version.

The original paper-based version of SCI–SCS was validated against the SF-12, although this is perhaps not the most appropriate ‘gold standard’. Therefore not surprisingly the associations were only moderate ranging from 0.3 to 0.6. However, the internal consistency of the scale exceeded 0.8 across three time points and the test-retest reliability ranged from 0.6 to 0.8.7 Our study extended the results of this study by demonstrating excellent inter-rater reliability (0.96) of the telephone-based version of the SCI–SCS. In addition, the telephone- based version of the SCI–SCS yielded very similar results to the paper-based version. Investigators from Canada have used the SCI–SCS as the basis for a similar scale.18 However, it relies on participants’ self-reports about accessing health-care services to rate the severity of secondary health conditions. This is problematic in LMIC because access to health care is often limited and not necessarily indicative of the severity of a health condition.

Some items of the SCI–SCS lack clarity, which caused ambiguities for participants of the study. For example, it is not clear whether the item titled ‘sexual dysfunction’ is referring to sexual dysfunction or sexual dissatisfaction.19 Similarly, for the item titled ‘bladder dysfunction’. Again, it is not clear whether this refers to the bladder dysfunction, which the majority of people with SCI would experience, or whether it is referring to secondary problems related to bladder dysfunction. Despite the limitations of the SCI–SCS, it has many appealing features, which make if worthy of further investigation and work. Notably, it has a summed overall score. If future studies can demonstrate the validity of using a summed overall score, then this may be useful for research purposes. However, most importantly, the telephone-based version of the SCI–SCS may provide a simple and an inexpensive way of screening for secondary health conditions in LMIC where literacy is a problem and where it is difficult to provide comprehensive face-to-face follow-up for all people.

Data archiving

There were no data to deposit.