Introduction

The pediatric measure of participation (PMoP) consists of calibrated item banks that assess participation in two domains: what the child does relevant to what he\she wants to do (Participation-Self); and what the child does relevant to what friends do (Participation-Friends). There are separate item banks for child and parent respondents to yield four calibrated item banks: child-report participation self (8–21 years), child-report participation friend (8–21 years), parent-report participation self (4–21), and parent-report participation friend (4–21). Development and validation of the PMoP item banks have been previously described [1,2,3,4,5]. Briefly, items that assess important aspects of participation for children were created through iterative focus groups and cognitive testing [1, 2]. These items were then administered to a sample of 381 children with spinal cord injury (SCI) and 322 parents of children with SCI. Item Response Theory (IRT) analyses with Graded Response Model (GRM) were conducted. Calibration study results confirmed that psychometric properties of the child reported [3] and parent reported [5] PMoP item banks were acceptable and met the assumptions necessary for administration via computer adaptive testing (CAT). While the four calibrated item banks share a number of common items, item characteristics differ in each scale. Parent report is a measure of parent perception of their child’s participation, and is not intended to be a proxy for child report.

There are several advantages of administering the PMoP via CAT. Namely, PMoP items are filtered by age and thus provide information about participation relevant to one’s development. Also, there is a decrease in administration burden [6], which is always important but perhaps more so when testing children [7], and there is increased precision of ability estimates [6]. However, since administration of an assessment via a computer is not always feasible, PMoP short forms (SFs) were created to provide an option for administering a paper-and-pencil version of the PMoP [8]. The PMoP SF items were selected from the calibrated item banks to ensure an appropriate range of content and to optimize the ability to discriminate among children with different participation levels. The response categories for the SFs are identical to those used for CAT administration (see Supplementary Material), and are available in English and Spanish. Using the original calibration sample data [3, 5] the psychometric properties of the PMoP SFs showed acceptable group reliability, minimal ceiling and floor effects, and acceptable agreement between SF and full item banks scores [8]. However, field-testing with 107 children with chronic (≥3 month) SCI and 96 parents of children with chronic SCI [5] revealed several limitations of administering the instrument using SFs compared to use of CATs. First, a relatively high percentage of the sample selected the response “Don’t Do, Don’t Want To” for several SF items. With CAT administration, if the respondent selects “Don’t Do, Don’t Want To”, the computer selects the next item from the item bank, and the response does not contribute to the participation score estimate. However, with SF administration this is not feasible. Second, the PMoP CAT uses a filter to select items deemed appropriate for different age groups. In an attempt to create one SF for each of the four domains rather than age-specific SFs, SF items had some relevance across all age groups, but the age filter was not considered in their selection. Third, when the PMoP is administered as a CAT, the program filter removes school-related items if the child does not attend school. When administered as a SF, these items would be skipped. In combination, these limitations reduced the clinical utility of the PMoP SFs.

The goal of this study was to develop PMoP SFs that were better able to replicate the ability of the CAT to customize assessments to match important child characteristics. Specifically, we sought to improve the original PMoP SFs by developing PMoP SFs Version 2.0 (V2.0) with the following features: (1) only include participation activities that children frequently do, (2) develop age-specific SFs, and (3) allow for optional administration of school-related items. The psychometric characteristics of the newly developed PMoP SFs V2.0 were also examined.

Methods

Development of PMoP SFs V2.0

Figure 1 presents an overview of the process used to identify item candidates for PMoP SFs V2.0. As a first step, we conducted an initial review of the four PMoP calibrated item banks (child participation-self; child participation-friend; parent participation-self; parent participation-friend) to: (1) identify core items, defined as items that are neither age dependent or school related, (2) examine the calibration data to identify and remove core items for which more than 10% of the calibration study respondents selected the response “Don’t Do, Don’t Want To”, and (3) categorize remaining items based on the PMoP CAT predefined age filter rules for the following age groups: 4–7 (parent-report only), 8–11, 12–15, and 16–21 years of age (parent- and child-report). We completed this process for both child-reported and parent-reported item banks.

Fig. 1: Process used to develop PMoP SFs V2.0.
figure 1

Process in blue reflects retrospective review of calibration data, used to select items for PMoP SFs V2.0. Process in green reflects analyses of psychometric properties of PMoPs SF V2.0. Gray shows the final newly developed PMoP SFs V2.0.

Full size image

After this preliminary work, we implemented an iterative process of item review and selection to develop age-specific PMoP SFs V2.0 (Fig. 1). If the item was associated with an age filter, it was only considered for the SF for that age group, while core items were considered for SFs for any age group. As items were considered for inclusion in the SFs, item parameters derived from IRT analyses were reviewed: (1) difficulty level (average value of the threshold parameters from GRM); (2) item discrimination function (slope), and (3). the item information function was generated to identify the range of the person score in which the item can provide the highest information value. Construction of each SF involved an iterative process to remove and replace items with the goal of ensuring that each SF was comprised of age-appropriate items with a range of difficulty that optimized the discrimination function of items. As a final step, items that assess school function were added as an option for children who attend school, and the response category “Don’t Do, don’t want to do” was eliminated from the SFs.

Psychometric evaluation of the PMoP SF V2.0

We used the entire data set consisting of 381 children with SCI and 322 parents of children with SCI from the calibration study [3, 5] to evaluate the psychometric properties of the PMOP SFs V2.0. First, for each of the child- and parent-reported PMoP SFs V2.0, we examined the degree to which the SFs cover the range of difficulty by calculating the mean values, standard deviations (SD) and score range on the T metric (mean = 50, SD = 10), and compared them with mean values and SD from the calibration sample. We calculated the percent of participants responding to all items on the SFs at the higher-end (ceiling effect) and at the lower-end (floor effect) of the categories. We examined group-level reliability [9] by calculating the ratio of the true score variance (observed short form score variance minus the average of the squared short form score standard error) and the observed score variance, and evaluated the internal consistency by calculating the Cronbach’s Alpha. Group-level reliability applied the value ≥0.5 as adequate, ≥0.8 as good, and ≥0.9 as high [10]. Cronbach’s Alpha is generated from the Pearson correlations among the items to measure the consistency of the scale, with values ranging between 0.7 and 0.9 desirable [11, 12]. We assessed the score agreement between the person scores estimated from the SFs and from the full item bank using the intraclass correlation coefficient (ICC3,1), with 95% confidence interval (CI).

For each child- and parent-reported PMoP SFs V2.0, we also examined how well the SFs estimated participation across a broad range, as reflected by the test information function (TIF). We created the plots to display the TIF for each SF along with the distribution of scores derived from the calibration sample. We calculated the percentage of sample with score reliability >90% (test information value > 10) and 80% (test information value > 5) based on the normal assumption (mean = 50, SD = 10) of person scores distribution.

Results

Table 1 shows the number of core items in the calibrated item banks and the number of core items that had a response rate of <10% as “don’t do, don’t want to” in the calibration study. As illustrated at the bottom of Fig. 1 and summarized in Table 2, the child-reported PMoP SFs V2.0 consist of six forms, one for each of the three child-reported age groups for participation-self and one for each of the three child-reported age groups for participation-friend. The parent-reported PMoP SFs V2.0 consist of eight forms, one for each of the same age groups as the child-reported forms for participation-self and participation-friend, and two additional forms for age group 4–7, one for participation-self and one for participation-friend. As shown in Table 2, each of the PMoP SFs V2.0 contain a combination of core (non-filtered items) and age-filtered items that had <10% of the calibration sample select the response “don’t do, don’t want to”, except for one item on the 16–21 age group child- and parent-PMoP SFs. The item “I date” had slightly greater than 10% response of “don’t do, don’t want to” for child (14.4%) and parent (10.9%) but was retained because of content relevance. Each form also has school items that are optional items for those who attend school (Table 2). SF items and response categories are provided in Supplementary Material.

Table 1 Number of total and core items in each of the four calibrated item banks (columns 2 and 3, respectively).
Full size table
Table 2 Summary of the number of core, age-filtered, and optional school items on each of the PMoP SFs V2.0.
Full size table

Table 3 presents mean values, SD, and score range for each of the child- and parent-reported PMoP SFs V2.0, based on scores derived from calibration sample data. Mean and standard deviation values for the SFs closely reflect the score distributions of the full item banks. Floor and ceiling effects were negligible (<15%) [13, 14] except for ceiling effects noted on parent-reported participation-self and participation-friend for the youngest age group (16.22%), and oldest age groups (19.4%, 18.66%). Group-level reliability ranged from 0.66 to 0.80, indicating that the SFs can distinguish high participation and low participation groups with adequate reliability. Internal consistency (Cronbach’s alpha 0.79–0.90) was good to excellent [14]. Agreement between SF scores and scores generated from the total item bank was also consistently high as evidenced by ICC values > 0.90, except for the parent-report SFs for self- and friend-participation 4–7 age group (ICC = 0.83 and 0.89, respectively).

Table 3 Psychometric evaluation of PMoP based on scores derived from the calibration sample.
Full size table

Figures 2, 3 illustrate TIF curves and sample distributions for each of the child-reported (Fig. 2) and parent-reported (Fig. 3) SFs. For all analyses, the maximum TIF is slightly below 50, and as expected, TIF is reduced at the upper and lower ends of the scale. Comparison of TIF to the sample distribution demonstrates that test information decreases at the upper end of the scale, indicating that score estimates in this range will be less precise. For Parent-reported SFs, score reliability of >0.8 was achieved in at least 75% of participants, except for the youngest age group where score reliability of >0.8 was in 66% and 74% of participants for participation-self and participation-friend, respectively. For child-reported SFs, the percentage of participants with score reliability >0.8 ranged from 61% (8–12 age group) to 74% (oldest age group).

Fig. 2: Test information function for child-reported short forms.
figure 2

Test information function curves and sample distributions for child-reported PMoP SFs V2.0.

Full size image
Fig. 3: Test information function for parent-reported short forms.
figure 3

Test information function curves and sample distributions for parent-reported PMoP SFs V2.0.

Full size image

Discussion

Based on findings from field testing, we revised the initial PMoP SFs to improve their relevancy for children of different age groups and school status. The PMoP SFs V2.0 consists of separate SFs for each of the three child-reported age groups (8–11; 12–15; 16–21) and for each of the four parent-reported age groups (4–7; 8–11; 12–15; 16–21). Each SF is comprised of core items (non-filtered) along with items appropriate for the specified age group, and includes optional school items. The PMoP SFs V2.0 more closely resemble items that would be selected via CAT administration due to the consideration of age and school filters in the selection of SF items.

Scores from the PMoP SFs V2.0 are closely aligned with scores from the full items banks, indicating that they can be used to estimate scores with little loss in precision. Except for parent report for the youngest and oldest age groups, content range is good with minimal floor or ceiling effects, and values for group reliability and internal consistency are acceptable. Test information, particularly for the oldest age group and at the higher end of the scales for each age group, is not optimal. Thus, when participation is on the upper end of the scale, the PMoP SFs V2.0 may be less precise in estimating participation. Replenishment [15] of the item banks can be done to address the ceiling effects and precision at the upper and lower ends of the SFs. In addition to replenishment, we recommend further psychometric testing of the PMoP SFs V2.0 with prospective samples of children with SCI and parents of children with SCI to build upon the simulated evaluation reported in this paper.

In the development of the initial PMoP SFs, we selected core items based on the item characteristics from the calibration study, namely, item difficulty and item slope. In doing so, we selected items that provided the greatest range, and that had strongest ability to discriminate across varying levels of participation. We strived to create one SF for child-reported participation-self and one SF for child-reported participation-friend that could be utilized across the pediatric age span of 8–21, and one SF for parented-report participation-self and one SF for parent-reported participation-friend for the age span of 4–21. Until field testing, we did not appreciate the implications of the response category “don’t do, don’t want to” on SF candidate selection. The response category of “don’t do, don’t want to” is an important option when assessing participation, as participation is based on a variety of factors including one’s life situation, enjoyment experience, and interests [16,17,18]. CAT administered is ideal for such a response category, as items scored as “don’t do, don’t want to” are treated as skipped by the computerized scoring algorithm, and the response does not contribute to the calculation of score estimates. Unlike SFs, CAT administration has access to the entire item bank and can continue to introduce items until relevant ones (those that children want to do) are responded to for score estimate. When field-testing revealed a response of “don’t do, don’t want to” to a high number of SF items, we realized that we needed to examine the calibration data to identify items with a high response rate (>10%) of “don’t do, don’t want to” and eliminate those items as candidates for the PMoP SFs V2.0. In doing so, we consider items as candidates for the PMoP SFs V2.0 only if they had less than a 10% response “don’t do, don’t want to” in the calibration study. The exception is the item “I Date” which was retained due to its importance in the participation of adolescents and young adults. Moreover, the response category “don’t do, don’t want to” is not available on the PMoP SFs V2.0. Rather, items that are not of interest can be skipped, and a total score can still be calculated given that at least half of the items have been answered [8].

One of the greatest advantages of the PMoP when administered as CAT is the age filter, which is used to guide the selection of items based on the age of the responder. With such filters, young children are never asked to respond to items relevant to older children, and older children are never asked to respond to items intended for younger children. Nonetheless, while items differ among children and on repeated administration, scores can be compared. Development of the initial PMoP SFs sought to reduce burden of selecting different SFs based on age. Creating one SF for each of the four scales (child-reported participation-self; child-reported participation-friend; parent-reported participation-self; parent-reported participation-friend), we did not consider the age filter and accepted that some items would be scored as “don’t do, don’t want to”. Examples include the item “I work” when answered by young children, and “I play with toys” when answered by adolescents. Since the SFs can be scored if at least half of the items are answered [8], we “tolerated” a few items rated as “don’t do, don’t want to” in return for one SF for each of the four scales to reduce administration burden. Although we thought this would be an acceptable trade-off, the combination of core items rated as “don’t do, don’t want to” and age-filtered items rated as “don’t do, don’t want to” was too high. Thus, in the development of the PMoP SFs V2.0, we examined items filtered by age, and for each of the PMoP SFs V2.0, items relevant to that age group were selected. While this approach increased the number of SFs from four (PMoP SFs V1.0) to six and eight for PMoP SFs V2.0 child- and parent-report, respectively, we believe the age specificity will increase the clinical utility.

With CAT administration, filters select school participation items based on school grade response (preschool, Grades 1–12, college) verse does not attend school. As examples, the items “I eat in the cafeteria with my friends.” and “When in PE (gym class), I do the same activity as the other kids” are available for those who indicate a grade in school but not for preschool or college, whereas the item “I do my homework” has no grade filter. The initial PMoP SFs contained five school items (I do the same thing in physical education as other kids; I stay after school for activities; I go on field trips with my class; I eat with my friends in the cafeteria; I do my homework) that were selected based on their item characteristics. Similar to the age-filtered items, we accepted that some school items may be rated as “don’t do, don’t want to” by some children. In developing the PMoP SFs V2.0, because we made the decision to create SFs for each of the age groups we decided to reexamine school items and select items based on the filters used in CAT administration. As to make the SFs more relevant to a larger group of children and youth with SCI, school items on the PMoP SFs V2.0 are optional; in other words, they can be administered as part of the SFs V2.0 or they can be omitted, based on whether the child is in school. This is not only relevant to older children who may have completed school and not pursued higher education, but also during times when administration occurs outside of the school year or when the child has not yet returned to school. Optional items introduces more complexity in scoring but given the importance of school participation, the complexity is needed.

The PMoP offers the opportunity to assess participation from both the child’s and parent’s perspective, owing to reports that children and parent reports may differ [19,20,21]. While the PMoP SFs V2.0 can be administered to only the parent or to only the child, we believe both perspectives are important to family-centered care, shared decision making, and addressing the unique needs of children with SCI and of parents of children with SCI.

Participation outcomes are important benchmarks in pediatric rehabilitation [22]. The PMoP SFs V2.0 can be used to monitor parent- and child-reported outcomes of participation, compare to what children want to do, and what friends do. Information about participation can be used to inform habilitation goals as children age through to adulthood. The PMoP SFs V2.0 manual contains scoring algorithms and conversion tables (https://www.jefferson.edu/university/rehabilitation-sciences/departments/outcomes-measurement/measures-assessments.html).

The PMoP SFs V2.0 show potential for developing customized SFs that match the capability of CAT administration in targeting items that are relevant and age appropriate. Because items are tailored for age and school and despite the limitations described in this paper, the PMoP SFs V2.0 offer a major advancement over the original PMOP SFs. It is important to note that scores from previously administered SFs can be compared to scores from PMoP SFs V2.0. However, the intentional omission of the response category “don’t do, don’t want to” from the PMoP SFs V2.0 may introduce variation when comparing scores. Likewise, the PMoP was calibrated on children with SCI, as noted on the titles of the SFs (Supplementary Material). Work is planned to examine the PMoP in other clinical populations such as cerebral palsy.

There are limitations of this study. We were not able to examine the validity of the of the PMoP SFs V2.0 when school items are omitted since all participants younger than 16 years of age in the calibration study were in school, and there was not a large enough sample size in the calibration study of youth not in school. However, the non-school SFs option is available and future work will examine the psychometric properties of these SFs in a cohort of children who do not attend school. Additionally, the measurement properties of the PMoP SFs V2.0 were generated using the calibration study sample. Prospective samples will contribute to the ongoing validation of these scales.

Conclusion

The PMoP SFs V2.0 consists of a SF for each of the three age groups for child report, and a SF for each of the four age groups for parent report. Each SF is tailored to meet age-related participation of each age group, and to school status. Simulated psychometric evaluation confirmed that scores generated from the SFs have moderate to high agreement with scores generated from the full item bank, and that the scales have acceptable group reliability and internal consistency. There is some loss of precision of the estimate score at the upper end of the scale. Content range is good with negligible ceiling or floor effects, except for the parent-reported SFs for the youngest and oldest age groups.