Introduction

Participation has received attention as a broad concept deserving of solid methods of assessment.1, 2, 3, 4 The CHART (Craig Handicap Assessment and Reporting Technique),5 Assessment of LIFE-H (Life Habits),6 and the IPA (Impact On Participation and Autonomy)7 are recognized as useful assessments for adults with spinal cord injury (SCI).1, 8, 9 However, both the CHART and IPA contain items that are not intended for child response7, 8 and the pediatric version of the LIFE-H10 has limited applicability. The Children’s Assessment of Participation and Enjoyment is designed for 6–21 years of age11 but is long and burdensome. The widely used Pediatric Quality of Life Inventory has concepts related to participation but has limitations when used with children with SCI.12 Hence, current instruments have limited utility for measuring participation outcomes in children with SCI resulting in a significant void in the understanding about how well children with SCI achieve the outcomes that are most meaningful.

Computer adaptive testing reflects modern measurement methodology and is uniquely positioned to eliminate problems associated with long and burdensome questionnaires, inappropriate items, ceiling and floor effects and inadequate content range. Development of a CAT involves pooling existing test items from a variety of sources and creating new items that, in combination, form an item pool. Item response theory provides a means to calibrate the item pool based on the information each item contributes towards characterizing the underlying test construct or latent trait (for example, participation); the outcome of the calibration is an item bank that can contain hundreds of items that fall along a continuum (for example, never-to-always). Although the development and validation of CAT requires a relatively large sample, there are tremendous advantages over traditional instruments, particularly efficiency, precision and low burden.

CAT assumes that all items in a scale measure a single, unitary concept, often referred to as assumptions of unidimensionality and local independence. The assumption of unidimensionality allows score estimates without having a particular set of fixed items administered to every person. This feature is directly responsible for the flexibility in administering test items in a dynamic approach. For example, with CAT, the first question drawn from the item bank is from the ‘middle’ are never presented. Thus, CAT provides an efficient method to obtain relevant information about a person’s function. Assessment of participation following SCI utilizing CAT would be ideal; children would respond to a minimal number of appropriate items, selected based on their responses to filter questions and previous items, and accurate estimates of their participation would be obtained.

The purpose of this study was to evaluate the assumptions of unidimensionality and local independence of two newly developed item pools of participation. The second purpose was to compare simulated CAT scores to the scores from the full-item banks.

Materials and methods

Subjects

Child-reported data were collected on a 381 children between 8–21 years of age with SCI. Children were enrolled if they had returned to pre-injury environments for at least 3 months. Children were not included if they did not speak English or if they were dependant upon mechanical ventilation resulting in the inability to communicate using verbal expression. Data were collected at three pediatric hospitals in the United States.

Participation item banks

Two item banks were developed: items that evaluated child-reported outcomes of participation relevant to what the child wanted to do and items that were relevant to what the child perceived his or her friends doing (Tables 1 and 2, respectively). The processes used to develop13 and test14 these items have been described elsewhere.

Table 1 Items in the Original Self-Participation Bank
Full size table
Table 2 Items in the Original Friend-Participation Bank
Full size table

The conceptual model of participation is shown in Figure 1. Accordingly, self-participation occurs on a continuum from ‘as much as I’d like’ to ‘less than I’d like’ and friend-participation occurs on a continuum from ‘as much as my friends’ to ‘less than my friends.’ Based on the model, ‘as much as I’d like’ and ‘as much as my friends’ participation occurs in physical environments that are accommodating to mobility limitations, when the social milieu has a familiarity with SCI, when there is a capacity for engaging in activities, when cost is not excessive and when transportation is readily available. When environments are less accommodating, as capacity for engagement decreases, as cost of participation increases and as transportation requirements exceed available methods, participation becomes ‘a little less than I’d like’ and ‘a little less than my friends.’ The model also recognizes parental choice, child autonomy and opportunity as key factors to childhood participation.

Figure 1
figure 1

Model of Participation used to develop the Participation Item Bank. Based largely on the concepts of the ICF, the model also assumes that youth participate as much as they want and as much as their peers in human and non-human environments that are familiar with spinal cord injury, accommodating to physical limitations and when cost of participation is manageable and when transportation is readily available. Based on the model, youth become less participative as environments become less familiar and less accommodating to spinal cord injury and when costs and transportation exceed what is available. While their contribution to participation are less understood, the model also acknowledges choice and autonomy as factors in children’s participation.

Full size image

Using this model, items were carefully constructed and placed along the continuum based on item content. For example, items such as ‘I go out to eat with my family’ and ‘I do my homework’ are anchored on one far side while items such as ‘I date’ and ‘I sleep over my friend’s house’ are anchored on the other far side (Figure 1). Tables 1 and 2 show the items hierarchically from ‘much’ participation to ‘less’ participation for self-participation and friend-participation, respectively.

The response scale for the participation item banks required initial selection of one of two choices: ‘I do it’ and ‘I don’t do it.’ When children responded ‘I do it’, they were then required to select one option from the self-participation menu: ‘as much as I want’; ‘a little less than I want’; and ‘a lot less than I want’ and another option from the friend-participation menu: ‘as much as my friends’; ‘a little less than my friends’; and ‘a lot less than my friends.’ When children responded ‘I don’t do it’, they were required to select either ‘because I can’t’ or ‘because I don’t want to.’

Procedures

Participation items along with mobility and daily routine items13 were read and answered by children using a PC-based tablet. Children completed the survey in a quiet room or remotely (N=17), independent from parental influence on responses.

Data analysis

Unidimensionality and local independence

The dimensionality of responses was evaluated using categorical confirmatory factor analysis and residual correlations. CFI and TLI compare the model to a baseline null model; possible values range from 0 to 1; 0.95 or higher suggests good fit and values above 0.90 suggest acceptable fit.16 RMSEA assesses misfit per degree of freedom; values close to or below 0.06 reflects acceptable fit. Exploratory factor analysis was also used to test unidimensionality. As the items were polytomous (contained more than two potential response choices), a weighted least squares estimator based on a polychoric correlation matrix was used. The magnitude of eigenvalues by the first factor (at least 20% of the total variance explained by the first factor is desirable)17 and the difference in the magnitude of eigenvalues between the first and the second factors (a ratio in excess of four is supportive of the unidimensionality assumption)17 were used to assess the dimensionality. Both EFA and CFA were conducted using the MPlus software.18 Local independence was evaluated by inspecting the residual correlations between items also using MPlus software. Items with residual correlations >±0.2 were considered showing local dependence.

Item calibration

Given the relatively small sample size for analyses, the one-parameter model was used for the main analyses. The item parameters and fit statistics were calculated using WinSteps,19 which is based on joint maximum likelihood estimation. Fit statistics for each item were calculated based on the comparison of expected and observed value. Values between 0.7 and 1.4 are considered to be within limits of the model.

CAT real data simulation

The first item with appropriate content for most children and with middle range difficulty was selected to be the first item administered. The selection of the next item was based on the item that can provide the highest information at the estimated score. In the present study, three CAT scores were developed in the simulation with stop-rules of 5, 10 and 15 items (CAT-5, CAT-10 and CAT-15). These simulated scores were then compared with the actual latent trait score for both self-participation and friends-participation estimated by the full-item banks to evaluate the strength of association between the simulated CATs and the full-item bank.

We certify all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during the course of this research.

Results

The sample demographics are summarized in Table 3. As shown, among the 381 participants, the majority were between 15 and 19 years of age with slightly more participants with complete A) as compared with incomplete injuries and slightly more participants with paraplegia. With respect to age and type of injury, this sample is representative of the larger population of pediatric SCI in which more youths between 15–21 years of age sustain SCI as compared with any other pediatric age group and their injuries are more likely to be at the thoracic level.20

Table 3 Sample demographics (values are based on n=381 unless otherwise specified)
Full size table

Unidimensionality and local independence

Tables 1 and 2 contain the self and friend-participation items, respectively. Initial CFA on the self-participation and friends-participation scale indicated that seven self-participation items and five friends-participation items showed local dependence with residual correlations >0.2. These items were removed from further analysis. The CFAs of the remaining items in the two scales provide acceptable model fits as evidenced by CFI=0.905, TLI=0.951 and RMSEA=0.089 and CFI=0.924, TLI=0.963 and RMSEA=0.083, for the self- and friends-participation scale, respectively. In the EFA of the remaining self-participation items, the first factor explained ∼76% of the total variance and the ratio between it and the eigenvalue of the second factor was 11; in the EFA of the remaining friends-participation items, the first factor explained about 78% of the total variance and the ratio between it and the eigenvalue of the second factor was 12.

Item calibration

As summarized in Tables 1 and 2, acceptable item fit statistics were found for both the self-participation and friends-participation scale. One out of 52 self-participation items and 2 out of 54 friends-participation items demonstrated some misfit. The item ‘When in PE, I do the same activity as the other kids’ was removed from both the self- and friend-participation item banks because of infit values of 1.59 and 1.61, respectively. Despite the unacceptable infit value of 1.53 for the item ‘I am on a school team or club’, this item was not removed from the friend-participation scale because of the importance of its content.

CAT real data simulation

As reported in Table 4, the high Pearson’s correlations between the 15-, 10- and 5-item CAT scores and full-item bank scores indicated that the simulated CAT scores for both self- and friend-participation accurately captured the information from the entire item banks.

Table 4 Comparison of Scores from simulated 15-, 10- and 5-item CAT and full-item bank
Full size table

Discussion

The results of the calibration work demonstrate the feasibility of using the participation item banks with youths with SCI for a child-reported CAT. Seven items had to be removed from the self-participation item bank and five items from the friend-participation bank, as they showed local dependence with residual correlations >0.2. Four of the same items were removed from both item banks. Two additional self-participation items and one additional friend- participation items were also removed due to local dependence.

Three items showed misfit; When in PE I do the same activity as other kids’ was removed from both item banks. The item ‘I am on a school team or club’ was kept in friend-participation because of the importance of its content and lack of redundant items. With the 51 and 53 final items, acceptable model fit that captures self- and friend-participation, respectively, was demonstrated.

When compared with the full-item bank, the results of the CAT simulations indicate that the 5-, 10- and 15-item models yield accurate estimates of participation in children with SCI. In future work, we will validate these estimates by administration of full-item banks along with CAT models using a prospective sample.

The response scale is aligned with concepts important to the construct of participation. Both subjective perspective and peer comparison of participation were addressed. The peer comparison component was designed to address the potential issues described by Van Brakel et al.2 The scale was also aligned with the understanding of participation as being dependant upon factors other than capacity. Hence, for each item scored as ‘I don’t do it’, children were also required to select either ‘because I can’t’ or ‘because I don’t want to’ (choice). Future work will further explore the reasoning behind ‘because I can’t’ to gain a better understanding of the factors that influence participation by children with SCI.

Although enrollment of 381 children with SCI represents a large sample size with respect to the number of children with SCI, it is relatively small for the type of analyses conducted. The effects of a small sample size were minimized by using a one-parameter model. The data are not fully representative of population-based studies in children with SCI, as children with high tetraplegia, incomplete injuries who ambulate and with concomitant brain injuries are underrepresented. Nevertheless, there were sufficient high and low level items for most of the children who were ambulatory and significantly impaired, respectively. Although this work is focused on a pediatric measure, the majority of youths in this study were teenagers between 15 and 19 years of age with only 21% under the age of 15. Ideally, the study would have benefited from inclusion of more younger children but the sample distribution follows the typical age distribution of the pediatric SCI population with the highest number of injuries occurring during adolescence.20 Important future work will involve studies on how best to link the pediatric SCI items with those currently being created for adults with SCI.

Finally, we acknowledge that the construct of participation differs among researchers and the model of participation used in this study does not necessarily represent all models of participation. Furthermore, the model of participation used in this study may not be as relevant to other cultures or populations as it is to the sample of children in this study. Work is planned to conduct differential item functioning studies with children with other diagnoses

Conclusion

The self- and friend-participation item banks met the required assumptions for CATs. CAT versions of the self and friend-participation scale yielded summary scores comparable to the scores estimated using the full-item banks.

Data archiving

There were no data to deposit.