Introduction

The Spinal Cord Independence Measure (SCIM-III)1 has gained recognition as a relevant outcomes instrument in spinal cord injury (SCI) and one that is recommended for use in SCI clinical trials.2 The SCIM-III is a ‘disease-specific' instrument that contains items that evaluate self-care, respiration, bladder and bowel management and mobility. Three large-scale adult studies3, 4, 5 established moderate-to-strong reliability and validity and showed that when compared with the FIM (Functional Independence Measure) the SCIM-III had broader content range and was more responsive to change. Additional work has examined clinically important differences in SCIM-III scores,6 established target values for neurological levels in complete SCI7 and has validated a self-report version.8 The self-report version offers a practical alternative to the performance-based version, especially in community-based centers and for longitudinal monitoring.

The content of the SCIM-III is highly relevant for children with SCI as pediatric rehabilitation focuses on restoring capabilities for self-care and mobility, as appropriate for age and development.9 Children as young as 6 years of age learn to manage their bladder and begin to participate in their bowel program.10 Children with SCI are taught skills for independent dressing, body transfers and upright and wheeled mobility.9 Current instruments to evaluate treatment outcomes on self-care and mobility have ceiling and floor effects and gaps in content when used with children with SCI.11 Although one way to address the limitations of outcomes instruments in pediatrics is to develop new instruments, the preferred way, where appropriate, is to establish the lower age limit for existing instruments that are robust and already in the field. Given the relevance of the SCIM-III content to pediatric SCI and its endorsement for use in adult SCI clinical trials,2 it is prudent to systematically examine the psychometric properties relevant to its utilization with youth.

Materials and methods

Mixed methods were used to evaluate the SCIM-III self report (SR). Specifically, SCIM-III SR items and response scales were exposed to formal cognitive testing with children with SCI, and in parallel a survey using the modified Delphi technique was conducted to engage content experts in an iterative critical review of the SCIM-III SR.

Cognitive testing

Cognitive testing ensures that items and response scales are written in a way that conveys their intent clearly, without complex words, medical jargon and ambiguity.12 Because the goal of this study was to evaluate reading and comprehension of SCIM-III SR items in youths with SCI, purposeful sampling recruited youths from one pediatric SCI program in the United States. We enrolled youths aged 7–16 with SCI of at least 3-month duration. Another enrollment requirement was that youths had to have finished their initial episode of rehabilitation and discharged to home. Parent and child consent and assent were obtained prior to study enrollment.

We employed techniques used in previous studies13, 14 such that every SCIM-III SR item and response scale were exposed to cognitive testing. Subjects were asked to read and answer each item aloud, using the response scale choices. Think aloud methodology15 was used to record details about how children understood and interpreted items and why they selected a given response. After each child participant read and answered each question, they responded to these questions: ‘In your words, tell me what this question is asking you' and ‘Using your own words, how would you ask this question to a friend?'. Open-ended questions were used to further appreciate the child’s understanding about items and response scale choices. The interviews, which were performed by one of two occupational therapists (OT) or by a physical therapist (PT), were conducted in a private room, audiotaped and transcribed verbatim. Content analysis focused on identifying replacement words and phrases suggested by children. The findings were used to modify the items and response scales for round 2 and 3 of the Modified Delphi Survey.

Modified delphi survey

The Modified Delphi Technique16 was used to develop consensus on the appropriateness of the content, structure and wording of SCIM-III SR items and response scales for child SR. Three questions for each of the SCIM-III items were presented to expert therapists using an internet-based survey: ‘Is the content of this item appropriate for children?'; ‘Is this item written well for children?'; and ‘Is this response scale written well for children?'. For any ‘no' response, therapists provided a rationale for their answer and suggested alternative wording or phrases. The modified Delphi survey was performed iteratively until 80% agreement was achieved for each of the three questions, for each of the 17 SCIM-III SR items.

E-mail invitations were sent to known pediatric SCI therapists asking them to participate in the survey (purposeful sampling) and to forward the e-mail invitation to their colleagues in pediatrics, pediatric SCI rehabilitation or adult SCI rehabilitation (snowball sampling). The e-mail invitations included a description of the study and a link to the survey. Each of the iterative surveys was ‘open' for a 3-week period. Therapists completed the surveys in anonymity.

For the first round of the Delphi survey, the exact wording and structure of the SCIM-III SR8 were exposed for critical review. Summary and descriptive statistics were used to calculate the frequency of yes\no responses for each SCIM-III SR item and to determine the percent agreement among therapists on each of the three questions. Any item below 80% agreement was modified by replacing words and phrases with the ones suggested by the Delphi therapists and by children participating in the cognitive testing interviews. Modified items and response scales were again sent out for expert panel review. This process continued iteratively until 80% agreement was reached for each of the three questions, for each of the 17 SCIM-III SR items.

Statement of ethics

The Institutional Review Board approved this study.

Results

Cognitive testing

As summarized in Table 1, 16 youths with SCI participated in cognitive testing. The mean time since injury was 4.4 years (range, 1–11 years). There were slightly more children with paraplegia (62.5%) and complete injuries (56%). The neurological characteristics of one child was unknown because of his inability to tolerate the standard examinations of the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI).17

Table 1 Subject characteristics in cognitive testing sample
Full size table

Table 2 lists the words that were repeatedly (in at least 10 interviews and within the same interview, multiple times) mispronounced and poorly understood. The majority of the words were coded as complex, vague or as medical jargon, whereas few were also coded as not relevant to children. We stopped enrollment after 16 interviews owing to the repetitive nature of the feedback and saturation of the data.

Table 2 Words and phrases that were repeatedly mispronounced and\or not understood by children
Full size table

Modified delphi technique

Salient characteristics of the 84, 64 and 33 survey respondents who participated in each round of the Delphi are shown in Table 3. In round 1, 80% agreement was reached for each of the 17 SCIM-III items for the question, ‘Is this content appropriate for children?' Because consensus at or above 80% was achieved, this question was not asked in subsequent rounds.

Table 3 Sample of occupational and physical therapists for each of the three Delphi Survey Rounds
Full size table

Table 4 shows the results of each round for the additional two questions (Is this item written well for children and Is this response scale written well for children?). In round 1, there was 80% agreement by both OT and PT that four items (4, 10, 15, 17) were written well for children. However, not one response scale reached the 80% requirement. Modifications, informed by feedback from Delphi round 1 and from cognitive testing interviews, were made to every SCIM-III item and response scale (Tables 5,6,7). For example, the original response scale for item 9 was eliminated and modified; the onus of scoring was shifted from the child to the person calculating the scores (Table 7). Youths and therapists all reported that children required different levels of help for tub and toilet transfers; hence, item 11 was modified into two questions (11A and 11B) (Table 7). As a way to preserve the original SCIM-III SR scoring, a single score for item 11 is calculated by assigning the lowest score given for 11A and 11B (see Supplementary material for scoring manual). For other items, medical jargon was eliminated, and preambles and definitions of words were embedded. As examples, for item 7 (Table 6), a preamble was added to convey to children what is meant by ‘needing assistance with bowel management'; it reads: ‘Help means having someone put medicine in my bottom or having someone put a finger in my bottom so that I can have a bowel movement (poop)….'. A preamble to item 5 (Table 6), ‘I need a respiratory (tracheal) tube…' reads, ‘Some kids with a spinal cord injury need help to breath, cough, and to clear their throat…'. With preambles, children understood items and were able to respond appropriately. Finally, for many of the SCIM-III SR items, we replaced complex words with simpler words that were suggested by the children (Table 4). Despite having had reached 80% agreement for three of the items in round 1, due to the extensive modifications that were made, all items were exposed to round two of the Delphi Survey.

Table 4 Results of Modified Delphi Technique
Full size table
Table 5 SCIM-III SR and corresponding SCIM-III SR-Y with scoring: self-care domain
Full size table
Table 6 SCIM-III SR and corresponding SCIM-III SR-Y with scoring: respiration and sphincter management domain
Full size table
Table 7 SCIM-III SR and corresponding SCIM-III SR-Y with scoring: mobility domain
Full size table

For round 2 (Table 4), 80% of the OT agreed that each item and response scale was written well for children, except for the response scale for item 1 (69.6% agreement). PT also reached agreement for all items, except for items 2A (75.9%), 2B (79.3%), 3A (77.8%), 3B (74.1%) and 6 (77.8%) and for all response scales except for 3 (73.1%) and 6 (69.2%). Because PT reached agreement for item 1 and the recommendations for modifications from OT would change the original intent of the item, we did not make further modifications. For items 2A and 2B, the phrases ‘using a water tap' (round 1) and ‘turning on and off water' (round 2) were dropped. Therapists felt the phrase added a degree of complexity and that regardless of having or not having a SCI children usually have help with this task. For items 3A and 3B, the sentence structure and complexity of the items were simplified by moving the concept of ‘easy- and hard-to-dress clothes' to the response scales. Despite not reaching 80% agreement by PT, Item 6, ‘Do you need help with urination or how you pee?' was not modified because the recommendations were not appropriate for children beyond pre-school age (‘Do you need help with going potty?'). The modified items and response scales were exposed to a third Delphi round with PT and achieved 80% agreement (Table 4).

Discussion

In this study, we exposed the SCIM-III SR to formal cognitive testing with youths with SCI and to scrutiny by experienced OT and PT. Our findings informed modifications to every SCIM-III SR item and response scale, producing a pediatric version of the tool that we formally refer to as the SCIM-III SR-Youth (SCIM-III SR-Y) (Supplementary Appendix A). Although the wording of the SCIM-III SR-Y is different from those in the SCIM-III SR, modifications were carefully made as to avoid changing the original intent and scoring of any item. Preserving the item intent and the scoring was a priority so that scores can be compared, regardless of the version used (pediatric or adult); the comparison of summed scores between pediatric and adult versions requires further testing. Some modifications made scoring more difficult for the person interpreting the questionnaire. As a way to ease the burden of scoring and interpretation, we developed a SCIM-III SR-Y manual that shows each item and response scale and that provides the rationale and steps for scoring (see Supplementary Material).

Young children are capable of reporting on their own health status,18 assuming that the items and response scales are written with clarity, simplicity and minimal cognitive burden.19 We previously showed that youths with SCI as young as age 8 were able to understand and respond to questions.13 A fundamental problem with the SCIM-III SR was that youths did not understand the majority of the words and phrases. Comprehension problems with single words and small phrases were easily addressed by replacing them with words children suggested. Engaging children to evaluate reading and comprehension of items on patient reported outcome instruments, as has been done in this study, is essential prior to using them in research or practice.19

Two other problems encountered by children when reading and responding to the SCIM-III SR were vagueness of terminology and unfamiliarity with medical jargon. Even though the youths in this study had been injured for approximately 4½ years, they were unfamiliar with the meaning of the medical words in the SCIM-III SR. As a way to eliminate medical jargon and provide clarity, preambles enabled children to understand and appropriately respond to items that asked about complex health situations.

We did not anticipate how similar the cognitive testing results would be among participants. On the basis of our prior work13, 14 and published literature,19 we estimated that we would need to perform cognitive testing interviews with at least five subjects of various ages to understand how children of different ages responded to the SCIM-III SR. In this study, the same words and phrases were repeatedly problematic, and youths’ suggestions for change were relatively consistent across the ages. As we reached data saturation with 16 subjects, we stopped then.

An important finding from round 1 of the Delphi was that therapists agreed that the content of the SCIM-III SR was relevant to children, and thus no items were dropped. However, three rounds were needed before therapists agreed with the remaining two questions, (‘Is this item written well?' and ‘Is this response scale written well?'). Feedback from therapists was similar to what was garnered from the cognitive interviews and included recommendations to simplify the items and response scales, reduce the complexity and difficulty of words and phrases and eliminate medical jargon. The therapists also provided valuable feedback on how best to structure response scales in a way to minimize cognitive burden and increase relevancy to pediatric response. Several therapists provided feedback that the SCIM-III SR-Y should be designed as either a child- or a parent report.

There are limitations to this study. We did not perform cognitive testing with children aged 6 or younger and only enrolled one 7-year-old child. Thus, the SCIM-III SR-Y is not recommended for children younger than age 8. Also, we did not enroll children with acute injuries who had little or no experience with performing activities with having a SCI. Thus, children with new injuries may not interpret or respond to SCIM-III SR items in the same way as found in our sample. Although some therapists may have participated in all three Delphi rounds, others may have participated in only one or two; likewise, therapists did not have to participate in the first round in order to participate in subsequent rounds. Engaging the same therapists in each of the three rounds may have been more systematic but would have also prevented input from therapists in the field who have rich experiences. We choose to be inclusive in an attempt to solicit feedback from a broad range of perspectives.

This study involved the initial validation of the SCIM-III SR-Y by examining face and content validity. We did not evaluate concurrent validity, responsiveness or reliability. We also take heed to the recommendation that the SCIM-III SR-Y should also be considered for parent report. As such, work is planned.

Finally, regardless of living with or without a chronic condition, children develop independence through a natural developmental process, which varies greatly. As a way to facilitate interpretation of the SCIM-III SR-Y, work is planned to establish norm references for typically developing children and for children with SCI.

Conclusion

The SCIM-III SR-Y is the pediatric version of the SCIM-III SR and can be administered to children aged 8 and older. Because the scoring algorithm was not modified, scores from the SCIM-III SR-Y can be compared with the SCIM-III SR enabling longitudinal monitoring across the life span and inclusion of children in clinical trials.

Data archiving

There were no data to deposit.