Introduction

Relevant, precise and efficient measures are needed to examine the efficacy of rehabilitation programs for youth with spinal cord injury (SCI) and to support evidence-based practice. Generic measures currently used in this population have significant conceptual and psychometric limitations.1 For example, the PedsQL2 includes items inappropriate for children with SCI (for example, ‘it’s hard for me to walk more than one block’ and ‘it’s hard for me to run’), and the WeeFIM may not detect clinically meaningful change.3 The Pediatric Spinal Cord Injury Activity Measure (PEDI-SCI AM) was developed specifically to assess activity outcomes in youth with SCI,2, 3, 4, 5 providing an alternative to generic pediatric outcome measures. The PEDI-SCI AM4, 5, 6, 7, 8 includes activities important to youth with SCI, and items assess a wide range of abilities in the following domains: general mobility, daily routines, wheeled mobility and ambulation.

Previous work determined item calibrations and verified PEDI-SCI AM item bank unidimensionality.8 PEDI-SCI AM calibrated item banks can be administered as computerized adaptive tests (CATs), which use a computer program to select items. A CAT begins with a mid-range item, and subsequent items are administered based on the individual’s responses. Items that are too hard, too easy or irrelevant are not administered.9 As each item is administered, the score estimate precision increases. The program terminates, based on pre-determined rules specifying a level of precision (s.e.) or maximum number of items, to yield a final score estimate. Scores for a given domain can be compared on repeated measures even though different items are administered because scores are based on the same metric. Although there are many benefits to using CAT administration, static short form (SFs), comprising items carefully selected from calibrated item banks, are used when computer administration is not feasible.

The objectives of this study are to (1) examine psychometric properties of PEDI-SCI AM item banks and 10-item CATs) and (2) develop PEDI-SCI AM SFs and evaluate their psychometric properties.

Materials and Methods

Participant-calibration sample

PEDI-SCI AM calibration data were obtained from a convenience sample of 381 youth with acquired SCI (traumatic, transverse myelitis, tumor and so on) for the child-reported version (age 8–21 years) and 322 parents/caregivers of children with SCI for the parent-reported version (age 4–21 years). Inclusion criteria were discharge from initial SCI rehabilitation with return to pre-injury environment for at least 3 months. As the measure was developed specifically for children with acquired SCI, children with congenital (for example, spina bifida) and progressive conditions (for example, spinal muscle atrophy) were excluded from the study. Other exclusion criteria were as follows: English not primary language; and brain injury that interfered with the ability to read, comprehend and respond to items. Data were collected within the Shriners Hospitals for Children System (Philadelphia and Chicago, and Northern California hospitals). The study was approved by the Institutional Review Board of each facility. Parents and children provided consent and assent, respectively. We certify that all applicable institutional and government regulations concerning the ethical use of human volunteers were followed during the course of this research.

PEDI-SCI CAT development and simulation

The CAT algorithms were developed at Boston University for each domain and programmed to administer the first item with highest Fisher Information value at the average sample score level. Post hoc simulations were conducted on the calibration data set to estimate the 10-item CAT scores and s.e. The person score and s.e. were estimated using the weighted likelihood estimation method.10 Each subsequent item was selected based on maximization item Fisher information value at the current score level, and the score and s.e. were recalculated after feeding the program with known response until 10 items had been administered. The final scores were transformed to the T-score scale, with a population mean and s.d. of 50 and 10, respectively. Higher scores represented higher function.

PEDI-SCI SF development

An expert panel comprising 12 professionals (1 medical doctor, 1 psychologist, 4 physical therapists and 6 occupational therapists) attended a 1-day meeting to select SF item candidates from PEDI-SCI AM item banks by reviewing two item parameters: item difficulty (measured in logits) and item discrimination (measured as the slope of the item characteristic curve). Groups identified SF item candidates using spreadsheets with PEDI-SCI items hierarchically organized based on logit scores with an additional column for item characteristic curve slopes. Logit scores were used to select items with an appropriate range of difficulty, and item characteristic curve slopes identified items that best discriminated among persons with different ability levels. Finally, SF item candidates were reviewed from a clinical perspective to ensure that important aspects of function were included in each SF. Groups presented initial recommendations and used an iterative process to identify 7–12 items for the four domains, with separate SFs for child/parent respondents (eight initial SFs). On the basis of the differences in abilities of youth with paraplegia and tetraplegia and experience developing the adult SCI-functional index (SCI-FI) SFs,11 separate paraplegia and tetraplegia SFs were developed for the daily routines and wheeled mobility domains (four additional SFs: two parents/two children). Bowel and bladder management is critical for youth with SCI, and items assessing these abilities are in the item bank but were not selected for inclusion in the SF because the items did not have a high information function. Also, these items are not relevant for youth with SCI not using bowel and bladder programs and could not be scored. Separate SFs were developed for manual and powered wheeled mobility for youth with tetraplegia to yield a total of 14 separate PEDI-SCI SFs. Iterative psychometric analyses to examine properties of the 14 SFs were conducted by two investigators (MDS and MJM) to ensure optimal item selection.

Psychometric analyses

Calibration sample data were used to calculate group-level reliability for the full-item banks, 10-item CATs and SFs for each domain (separate for child and parent) defined as follows:

where E(s.e.2) is the mean of estimated score s.e.’s in each group, and σ2θ is the variance of the estimated score for child and parent respondents. We also calculated Cronbach’s alpha for the full-item bank and SF items (but not for 10-item CATs as different items were administered). Intraclass correlation coefficients (ICCs) were calculated to determine agreement between the total item bank and 10-item CATs and SFs. We examined precision of score estimates by limiting the score range (0 through 80) and calculating the s.e. Finally, we calculated the percent of the sample with the highest (ceiling) and the lowest (floor) score for the full-item banks, 10-item CATs and SFs.

Results

PEDI-SCI sample

The mean age (s.d.) of the participants for the child-reported version was 15.5 (s.d.=3.5) years; most were boys (55%) and white (82%); 57.6% had paraplegia; and 54.2% had complete injuries, as defined by American Spinal Injury Association Impairment Scale (AIS). Mean age (s.d.) for the parent-reported version was 13.6 (s.d.=4.5) years; most were boys (55%) and white (82%); 56% had paraplegia; and 52% had complete injuries (AIS). Details are reported elsewhere.7

PEDI-SCI item banks

The PEDI-SCI AM item banks include four domains: general mobility (19 child items and 18 parent items), daily routines (192 child items and 185 parent items), wheeled mobility (62 child items and 64 parent items) and ambulation (25 child items and 25 parent items; see Supplementary Material8 for complete list of items). Core items (N=178) were administered to all participants; supplemental items (N=229) were administered based on responses to screener questions (for example, age-specific items, bowel and bladder program, and ambulation status).8

Descriptive statistics

Tables 1, 2, 3 present the range, mean values and s.d.’s for the full-item banks, 10-item CATs and SFs. Scores were calculated using the T metric (mean=50; s.d.=10). Mean values were near 50 with some exceptions. For ambulation, parent respondents were below the mean for CAT (33.0) and SF (33.4) administrations. Where separate SFs were developed for different levels of lesion, lower mean scores were noted for tetraplegic-specific SFs, as expected (child and parent daily routines tetraplegia SFs=44.4 and 43.6, respectively; child and parent power wheeled mobility tetraplegia SFs=34.4 and 35.6, respectively; and child and parent manual wheeled mobility tetraplegia SFs=42.9 and 43.9, respectively). Higher mean scores were noted for paraplegia-specific SFs (child and parent daily routines paraplegia SFs=54.7 and 55.5, respectively; and child wheeled mobility paraplegia SF=54.0).

Table 1 Descriptive statistics of the PEDI-SCI parent-reported and child-reported general mobility and ambulation scalesa
Full size table
Table 2 Descriptive statistics of the PEDI-SCI daily routine scales
Full size table
Table 3 Descriptive statistics of the PEDI-SCI wheeled mobility scalesa
Full size table

Group-level reliability (0.73–0.96) was moderate to high for all domains and versions (full-item banks, 10-item CATs and SFs). Internal consistency values for the full-item banks and SFs were also acceptable (Cronbach’s alpha=0.77–0.99). For general mobility and ambulation item banks (child and parent versions), agreement was high for the CATs and SFs (ICC range=0.97–0.99). For the daily routines and wheeled mobility item banks (child and parent versions), agreement was high for the CATs (ICC=0.97–0.99); however, for daily routines, agreement was lower for paraplegia-specific SFs for children (ICC=0.72) and parents (ICC=0.83) compared with tetraplegia-specific SFs for children (ICC=0.93) and parents (ICC=0.95). For wheeled mobility SFs, agreement was strong (ICC range=0.90–0.99).

Score estimate precision

Figures 1, 2, 3, 4 display the s.e. observed for each domain and version. The dashed line represents reliability at 0.90; points below the line indicate the score reliability >0.9 and above the line score reliability <0.9. The general mobility (Figure 1) and ambulation (Figure 2) plots demonstrate excellent precision for low mid-range score estimates that varies somewhat by version (full-item banks>10-item CATs>SFs). For general mobility, the precision for parent and child respondents is similar. For ambulation, precision is better for parent respondents, but it should be noted that the sample size is smaller as fewer children were ambulatory. Results for daily routines (Figure 3) should be viewed with caution because of the problems noted with ceiling effects; however, a similar pattern is noted. Finally, for wheeled mobility (Figure 4), precision estimates vary by mode of administration with high s.e.’s for tetraplegia SFs and 10-item CATs for both child and parent respondents; however, these instruments were completed by subsets of the sample (for example, only manual wheelchair users).

Figure 1
figure 1

s.e. (dashed line represents reliability=0.90). (a) General mobility (child reported). (b) General mobility (parent reported).

Full size image
Figure 2
figure 2

s.e. (dashed line represents reliability=0.90). (a) Ambulation (child reported). (b) Ambulation (parent reported).

Full size image
Figure 3
figure 3

s.e. (dashed line represents reliability=0.90). (a) Daily routine (child reported). (b) Daily routine (parent reported).

Full size image
Figure 4
figure 4

s.e. (dashed line represents reliability=0.90). (a) Wheeled mobility (child reported). (b) Wheeled mobility (parent reported).

Full size image

Ceiling and floor effects

Tables 4, 5, 6 present floor and ceiling effects for the PEDI-SCI banks, 10-item CATs and SFs. Floor effects are acceptable (<15%) across all domains, respondent types and administration versions for children with paraplegia and tetraplegia (range=0–10.29%). Ceiling effects for children with tetraplegia are acceptable (<15%) across all domains, respondent types and administration versions. For children with paraplegia, ceiling effects for the general mobility domain are acceptable (<15%) for parent respondents but high for child respondents across all versions (23.61–26.85%). A significant problem with ceiling effects is noted for daily routines in children with paraplegia; ceiling effects are unacceptable, except for the parent daily routine full-item bank (13.0%). For wheeled mobility, ceiling effects are acceptable for parent and child respondents for all versions, except the child wheeled mobility SF (15.43%).

Table 4 PEDI-SCI general mobility and daily routines scalesa
Full size table
Table 5 PEDI-SCI ambulation scales
Full size table
Table 6 PEDI-SCI wheeled mobility scalesa
Full size table

Discussion

The PEDI-SCI AM is the first activity limitation measure developed specifically for youth with SCI. Initial examination of the PEDI-SCI AM demonstrated strong reliability and internal consistency for full-item banks, 10-item CATs and SFs. Simulated 10-item CAT and most SF scores provide precise estimates and show strong agreement with the full-item bank, indicating that CAT programs select appropriate items and SF items that adequately represent the full-item banks. The exception is the daily routine SF for children with paraplegia where the lower agreement with the item bank may limit item selection due to high ceiling effects.

Analysis of floor and ceiling effects indicated that the range of difficulty represented in the ambulation and wheelchair items was appropriate for youth with paraplegia and tetraplegia. Floors effects are minimal (<15%) across all domains and respondent types and the full-item banks, 10-item CATs and SFs, indicating that all PEDI-SCI AM measures had sufficient content range for all children, regardless of their level of lesion.

The PEDI-SCI AM measures had the necessary content range at the upper end of the scale for youth with tetraplegia, as evidenced by small ceiling effects. Likewise, for youth with paraplegia, ceiling effects for the child and parent versions of the mobility and ambulation SFs were acceptable. However, for children with paraplegia, ceiling effects for the child and parent daily routines and the child general mobility SFs were unacceptably high for all versions and may underestimate a child’s full ability in these domains, especially if the child was functioning at a relatively high level.

The ability to improve the psychometric properties of a measure by adding and calibrating new items (referred to as item bank replenishment) is a significant advantage of Item Response Theory-based measures. The next step in refinement of the PEDI-SCI AM involves replenishment of the daily routines and general mobility item banks. Replenishment can enhance item banks without altering the underlying scale, and we have successfully replenished other item banks.12 We will develop new items to assess the upper range of abilities and calibrate these items in a sample of youth with SCI, and these new items will be added to the existing item banks and SFs.

Given the void in robust outcome instruments for pediatric SCI, the PEDI-SCI AM is an important contribution that can elevate pediatric SCI practices. PEDI-SCI AM is the first measure with items specifically developed for youth with SCI that uses sophisticated measurement approach. It can be administered as a SF or CAT; although the PEDI-SCI AM CATs performed better than the SFs, some SFs were comparable to the simulated 10-item CAT, providing evidence that the SFs can be an acceptable alternative to CAT administration. Young children are capable of self-reporting on their health conditions; however, discordance between child and parent reports of health-related quality of life outcomes has been established in clinical samples and typically developing children.13, 14 It is important to assess outcomes from both parent and child perspectives,14 and the PEDI-SCI AM offers parent and child versions. These versions are based on different metrics, and scores cannot be directly compared, but the ability to administer the measure to parent and child respondents provides clinicians and researchers with different options based on the specific situation and purpose of the assessment. Finally, PEDI-SCI AM can be used with its companion PEDI-SCI participation measure,4 and, as item banks are linked to the SCI-FI, an Item Response Theory-based measure developed for the adult population, function can be tracked across the lifespan.8 The next step is to validate these measures in a new sample and compare the responsiveness of the PEDI-SCI AM CATs and SFs to generic measures.

There are several limitations to this work: the sample drawn from three Shriners Hospitals for Children in the United States may not be representative; and the sample is relatively small but acceptable given the overall number of children with SCI (estimated number of new injuries is 1500 per year).15

Conclusion

The PEDI-SCI AM uses contemporary measurement approaches to create new instruments for pediatric SCI research and clinical practice. For the first time, activity outcomes can be assessed with a measure developed specifically for youth with SCI. The ability to administer any or all of the four domains and select child and/or parent as respondent(s) provides a comprehensive measure that can be customized for multiple purposes, and the ability to use PEDI-SCI AM CAT or SF versions provides administrative options for different settings.

Data Archiving

There were no data to deposit.