Introduction

The appraisal process is defined as the extent to which one’s interpretation of an event influences whether they perceive the situation as negative or stressful [1]. The appraisal process is a critical component to the psychological and psychosocial adjustment of individuals after a traumatic injury, in particular spinal cord injury (SCI) [1,2,3]. In the context of SCI, appraisal of the injury might include an assessment of one’s physical condition but also of secondary conditions, such as reduced mobility and persistent pain; recent research has likewise suggested that individuals with SCI can experience appraisals of injustice [4, 5]. Health or injury-related injustice appraisal comprises elements of unfairness and loss consequent to one’s injury [3, 6].

Sullivan et al. [6] developed the Injustice Experience Questionnaire (IEQ) as a way to assess injury-related injustice appraisals in people with musculoskeletal pain. The items of the IEQ were developed from the research team’s clinical experience in the treatment of individuals with musculoskeletal injuries and focus groups with psychologists who provided interventions for individuals with musculoskeletal injuries [6]. Together, the team created 12 items that make up the IEQ. Items in the scale are worded as closely as possible to actual expressions of clients [6]. To evaluate the structure of the measure, Sullivan et al. (2008) used principal components analysis. Results yielded a two-component solution which represent the two IEQ subscales: (1) blame and unfairness, and (2) severity and irreparability of loss.

Over the last decade, researchers have used the IEQ to assess injustice appraisals in populations affected by various conditions including, musculoskeletal pain [6], whiplash [7], fibromyalgia [8], traumatic brain injury (TBI) [9], and SCI [4, 5]. As a result, there is growing evidence to suggest that injustice appraisals contribute to poor physical and psychological outcomes. For example, across populations, higher injustice appraisals are associated with the development and maintenance of persistent pain [5], disability [5], and psychological distress [5, 6, 8].

After the initial validation in a musculoskeletal pain population [6], the IEQ has since been validated in additional populations, including fibromyalgia [8], whiplash [7], TBI [9], musculoskeletal disorders [10], and in a mixed trauma sample [11]. However, to date, the IEQ has not been validated in a sample of individuals with SCI. Given that injustice appraisals appear to be an important risk factor for poor outcomes after SCI [4], it is necessary to validate the IEQ for use in this population. Therefore, the purpose of the current study was twofold: (1) to verify the factor structure of the IEQ using a sample of individuals with SCI, and (2) to assess IEQ scale reliability and construct validity using the same population.

We hypothesized that our results would confirm the two-factor model reported in the original validation study [6] and subsequent studies [8, 10, 11]. If our initial results did not support a two-factor model, we planned to conduct an exploratory factor analysis (EFA) to determine the structure that best fits the data. We expected that the IEQ would demonstrate good convergent and discriminant validity such that that the IEQ would be positively associated with measures of perceived disability, depression symptoms, stigma, pain catastrophizing, pain severity, and fear of movement related to pain and negatively related to measures of self-efficacy and life satisfaction.

Methods

To conduct this secondary analysis, de-identified data from three datasets were combined. Recruitment and study procedures for two of the datasets are detailed in Monden et al. [5] and Monden et al. [12]. The third dataset contains unpublished data from an ongoing study [13]. Supplemental Table 1 provides a description of each dataset. The local Institutional Review Board (IRB) deemed this study exempt from IRB approval based on exemption category 4(ii).

Study sample

The combined dataset includes 341 adults with SCI who completed the IEQ, measures of psychological distress and pain, and provided sociodemographic and injury-related information. Further, none of the studies included individuals without neurological impairment (i.e., American Spinal Injury Association [ASIA] Impairment Scale [AIS] grade E—normal motor and sensory function).

Measures/materials

Sociodemographic and injury-related characteristics

Sociodemographic variables used to characterize the study population included: current age (years), sex (male, female), race/ethnicity (non-Hispanic white, non-Hispanic black, other) marital status (single, married, no longer married [unmarried couples, cohabitating couples]), education level (less than high school [HS], HS/GED, Associate’s degree, Bachelor’s degree, Master’s degree, Doctorate degree), employment status (employed, unemployed, student, retired - age-related, retired -disability), and annual household income (<$50,000, $50,000 or greater).

Variables used to characterize the injury profile include: time post-injury (years), length of rehabilitation stay (days), injury etiology (vehicular, violence, fall, medical, sports, other), injury type (traumatic, non-traumatic), injury level (paraplegia, tetraplegia), AIS (A, B, C, D), and type of wheelchair used (power wheelchair, manual wheelchair, no wheelchair).

Injustice appraisals

Appraisals of injustice were evaluated using the IEQ [6]. The IEQ consists of 12 items that reflect the thoughts and feelings a person experiences when thinking about their injury as they relate to loss, perceived unfairness, and externalized blame. Participants respond to items such as, “Most people don’t understand how severe my condition is” and “I am suffering because of some else’s negligence” using a 5-point Likert-type scale ranging from 0 (never) to 4 (all the time). Items on the IEQ are summed for a total score between 0 and 48. Each subscale (blame/unfairness, severity/irreparability of loss) can also be summed separately. Higher scores on the IEQ indicate higher appraisals of injustice.

Perceived disability

A modified version of the Pain Disability Index (PDI) [14] was used to assess the impact that an injury has on a person’s daily life. The PDI assesses seven areas of daily living: home, social, recreational, occupational, sexual, self-care, and life support. Participants rated their level of disability using an 11-point Likert-type scale ranging from 0 (no disability) to 10 (worst disability). Higher scores on the PDI indicate greater perceived disability.

Depression symptoms

The Patient Health Questionnaire-8 (PHQ-8) [15] was used to assess severity of depression symptoms. Participants rated how often they experience eight depressive symptoms using a 4-point Likert-type scale ranging from 0 (not at all) to 3 (nearly everyday). Higher scores indicate higher severity of depression symptoms.

Self-efficacy

The Moorong Self-Efficacy Scale (MSES) was used to measure self-efficacy as it relates to everyday life activities for individuals with SCI [16]. Participants rated their confidence in the ability to perform 16 everyday tasks using a 7-point Likert-type scale ranging from 1 (very uncertain) to 7 (very certain). Higher scores on the MSES indicate a greater sense of SCI-related self-efficacy.

Stigma

The 10-item Spinal Cord Injury Quality of Life Stigma Short Form (SCI QOL Stigma-SF) was used to assess SCI-related perceived stigma [17]. Participants rated the frequency to which they endorse feelings of perceived stigma using a 5-point Likert-type scale ranging from 0 (never) to 4 (always). Higher scores represent a greater degree of perceived stigma.

Pain catastrophizing

The 13-item Pain Catastrophizing Scale (PCS) [18] was used to assess the effects of pain catastrophizing in the form of rumination, magnification, and helplessness. Participants were asked to reflect on past painful experiences and indicate how often they experience certain thoughts or feelings using a 5-point Likert-type scale ranging from 0 (not at all) to 4 (all the time). Higher scores on the PCS indicate greater pain catastrophizing.

Pain severity

The Present Pain Intensity index of the Short Form - McGill Pain Questionnaire (SF-MPQ) [19] was used to assess current pain. Respondents indicated which of six words, ranging from 0 (no pain) to 5 (excruciating pain), best reflect their current pain experience. Higher scores on the SF-MPQ represent greater levels of current pain.

Fear of movement related to pain

The Tampa Scale for Kinesiophobia (TSK) [20] is a 17-item questionnaire used to assess the subjective rating of kinesiophobia (fear of movement) related to pain. Participants responded to items using a 4-point Likert-type scale ranging from 1 (strongly disagree) to 4 (strongly agree). Higher scores on the TSK represent greater fear of movement as it relates to pain.

Statistical analyses

All analyses were completed using R statistical software with the psych and lavaan packages. Statistical tests used a significance level of α = 0.05, unless otherwise specified. The sociodemographic and injury-related characteristics were summarized using means and standard deviations (SD) for the continuous variables and frequency counts and percentages for the categorical variables. Non-parametric continuous variables were summarized using medians and interquartile ranges (IQR). Reliability of the IEQ was assessed using Cronbach’s alpha, a calculated metric of internal consistency. An alpha score > 0.80 demonstrates acceptable reliability [21].

The structure of the IEQ was evaluated using a series of factor analyses. CFA was conducted first to evaluate fit of the two-factor model determined by Sullivan et al. [6]. The IEQ total score and each subscale (blame/unfairness and severity/irreparability of loss) were also evaluated as single-factor models. In order to capture the ordinal nature of each IEQ item, CFAs were conducted using weighted least squares (WLS) estimators and orthogonal rotations where assumptions of normality do not apply. In the event of poor CFA model fit, EFA was used to determine the best fitting factor structure for our data. Data were screened to determine suitability for EFA. First, Polychoric correlations were evaluated to assess for multicollinearity between all IEQ items [22]. Polychoric correlations measure the agreement of ordinal variables. Bartlett’s test for sphericity was then used to evaluate the factorability of the IEQ items; a significant value of p < 0.05 suggests that items can be appropriately factored [23]. WLS estimation and orthogonal rotation were used to extract factor solutions. The optimal number of meaningful factors was evaluated using eigenvalues and a scree plot. Eigenvalues >1 are considered desirable [24]. Item loadings were also considered in the suitability of factors, with loadings >0.3 considered suitable [25].

For both EFA and CFA, several global indices were used to assess fit. Absolute fit was assessed using a chi-square test. Significant results indicate good fit, however reaching significance is common with large sample sizes [26]. Comparative fit was assessed using the comparative fit index (CFI) and Tucker–Lewis index (TLI), with recommended thresholds > 0.95 indicating good fit compared to a null model. Residual-based fit was assessed using standardized root mean square residuals (SRMR) and the root mean square error of approximation (RMSEA), where thresholds < 0.06 for SRMR, and <0.08 for RMSEA indicate low discrepancies between the observed and predicted model [27]. 90% confidence intervals (CI) were also calculated for RMSEA. All newly determined factors were also assessed for internal consistency using Cronbach’s alpha.

Finally, bivariate correlations were calculated to assess construct validity. The association between IEQ total score and SF-MPQ, PDI, PHQ-8, PCS, SCI QOL Stigma-SF, and TSK were used to evaluate convergent validity, and the associations between IEQ total score and MSES and SWLS were used to assess discriminant validity. Correlations were interpreted as very high (r = 0.90–1.00), high (r = 0.70–0.90), moderate (r = 0.50–0.70), low (r = 0.30–0.50), and negligible (r = 0.00–0.30) [28].

Results

Study sample description

Sociodemographic and injury-related characteristics for the study sample are displayed in Table 1. Participants were on average 36 years old with a median time post-injury of 8 years. The majority of the sample was male, non-Hispanic white, married, and employed. Slightly more than half of the sample consisted of individuals with paraplegia (53.5%) and injuries that were a result of a motor vehicle collision (51.3%). The average total IEQ score was 15.9 (SD = 10.3) and blame/unfairness and severity/irreparability of loss subscale scores were 5.9 (SD = 5.8) and 10.0 (SD = 5.2), respectively.

Table 1 Sample sociodemographic and injury-related characteristics (N = 341).
Full size table

Confirmatory factor analysis

The two-factor CFA model for the IEQ was significant (p < 0.001), however, it did not demonstrate good fit, CFI = 0.941, TLI = 0.926, SRMR = 0.080, and RMSEA = 0.116. Similar model fit results were found with the single factor structure of the IEQ total score and the severity/irreparability of loss subscale. The blame/unfairness subscale score met the recommendations for good model fit with CFI, TLI, and SRMR, but not RMSEA (CFI = 0.988; TLI = 0.980; SRMR = 0.042; RMSEA = 0.093). Complete CFA model fit results are presented in Table 2.

Table 2 Model fit results for the confirmatory factor analyses and exploratory factor analyses.
Full size table

Exploratory factor analysis

Due to poor model fit in CFA, EFA was conducted to identify a more appropriate factor structure. Initial evaluations of the IEQ items are displayed in Supplemental Table 2. Correlations between all items were <0.8, therefore meeting the assumption of a lack of multicollinearity. Bartlett’s test was found to be significant (p < 0.001), indicating the IEQ was suitable to be factored. Results from the scree plot (Supplemental Fig. 1) suggested that a four-factor solution would be optimal for the IEQ; however, one- to four-factor EFA solutions were ultimately evaluated for comparison.

Model fit for each EFA can be found in Table 2, with all EFA loadings for the IEQ items presented in Table 3. None of the EFA models met all thresholds for good model fit. The CFI for the two- and three-factor solutions was the only measure that met the recommended thresholds. Due to only one IEQ item being present in two of the factors, the four-factor solution was deemed unsuitable. Further, the four-factor solution demonstrated the worst fit of all solutions. The three-factor solution comprised at least two IEQ items in each factor and demonstrated strong factor loadings. However, interpretation of the factors proved difficult to distinguish, and Cronbach’s alpha values from two of its factors did not meet the recommended threshold, indicating lack of internal reliability. Results of the two-factor EFA differed from Sullivan et al. [6], such that two items originally representing severity/irreparability of loss (item 4 and item 5) were grouped with items representing blame/unfairness. However, Cronbach’s alpha for this revised severity/irreparability of loss factor also demonstrated a lack of internal reliability (α = 0.68). EFA performed on the IEQ measure as a single factor also demonstrated poor fit, despite its acceptable factor loadings. The path diagrams for the two-factor and three-factor EFA are illustrated in Fig. 1, along with the path diagram for the two-factor model determined by Sullivan et al. [6] in Fig. 2.

Table 3 Factor loading results for all exploratory factor analyses.
Full size table
Fig. 1: IEQ Path Diagrams for the Two-Factor and Three-Factor EFA.
figure 1

The curved blocks refer to the latent constructs (factors), the square blocks refer to the observed IEQ item responses from the sample. The curved lines refer to the association of overlap between factors, the straight lines refers to the extent that the items load onto the given factor.

Full size image
Fig. 2: IEQ path Diagram for the Two-Factor Model Determined by Sullivan et al. [6].
figure 2

The curved blocks refer to the latent constructs (IEQ subscales), the square blocks refer to each IEQ item. The curved lines refer to the association of overlap between factors, the straight lines refers to the extent that the items load onto the given factor based on sample responses.

Full size image

Reliability and validity of the IEQ

The IEQ total and blame/unfairness subscale demonstrated strong internal consistency (α = 0.89 and α = 0.86, respectively), whereas the alpha for the severity/irreparability of loss subscale fell below the acceptable level (α = 0.74). Associations between the IEQ, PHQ-8, PCS, SF-MPQ, SCI QOL Stigma-SF, TSK, MSES, and SWLS were assessed to determine convergent and discriminant validity. Table 4 presents these correlations. Scores on the IEQ were positively associated with scores on the SF-MPQ, PHQ-8, PCS, PDI, and SCI QOL Stigma-SF; the strength of these associations ranged from r = 0.055 to r = 0.599. Scores on the IEQ were negatively associated with scores on the MSES, SWLS, and TSK; the strength of these associations ranged from r = −0.019 to r = −0.512.

Table 4 Correlations between IEQ total score and measures of pain and psychological distress (N = 341).
Full size table

Discussion

The objective of this study was to assess the factor structure and validity of the IEQ in a sample of individuals with SCI. To this end, a series of CFA and EFA were conducted in an attempt to replicate the two-factor structure originally described by Sullivan et al. [6]. Further, bivariate correlations were conducted to evaluate the association between IEQ and measures of pain and psychological distress.

As reported in the original validation study [6], the severity/irreparability of loss and blame/unfairness subscales of the IEQ did not replicate in this sample of individuals with SCI. Since subsequent validation studies reported considerable correlation between subscales [8,9,10,11], we also used CFA to examine a one-factor structure for the IEQ total score and for each of the subscales. While we observed poor fit statistics for the IEQ total score and for the severity/irreparability of loss subscale, the blame/unfairness subscale demonstrated acceptable model fit on all but one of the global indices, RMSEA. These results suggest that items comprising the blame/unfairness subscale most closely represent the construct of injustice among individuals with SCI. However, poor fit as indicated by RMSEA demonstrates that these results should be interpreted with caution.

Since we were unable to confirm the two-factor structure described by Sullivan et al. [6], we used EFA to identify a model suitable for our data. The three-factor structure demonstrated strong factor loadings; however, we could not find any clinical utility in the grouping of items. For example, factor 3 of the three-factor structure comprises IEQ item 3 (“I am suffering because of someone else’s negligence”) and IEQ item 8 (“I worry that my condition is not being taken seriously”). IEQ item 3 clearly represents an expression of external blame whereas IEQ item 8 appears to represent feelings that others lack an understanding of their condition. Taken together, we were unable to identify a meaningful underlying theme for this factor. Item loadings for the two-factor EFA were similar to Sullivan et al. [6], with the exception of IEQ item 4 (“No one should have to live this way”) and IEQ item 5 (“I just want to have my life back”). As opposed to loading with items representing severity and irreparability of loss, as described in the original study, the items loaded with those representing blame and unfairness. Conceptually, one could argue that the content of these items also reflect feelings of unfairness, however, given the poor internal consistency observed for one of the factors of this revised two-factor model, we would not recommend its use.

Overall, none of the models produced a factor structure that met the thresholds for goodness-of-fit, possibly indicating that the current IEQ does not reflect the construct of injustice as perceived by individuals with SCI. This is further evidenced by low endorsement for many of the items (e.g., items 3, 8, and 9), as well as by items that failed to hang well with others (e.g., item 1; Supplemental Table 1). Furthermore, some items are written from an ableist perspective, which implies that living with a disability is inherently negative (e.g., item 4: “No one should have to live this way”), which for individuals with SCI, may not capture the construct of injustice as intended. Taken together, this may be evidence for removing, rewording, and/or adding items to create a modified version of the IEQ specific to measuring injustice appraisals in the SCI population.

The second aim of the study was to assess the reliability and validity of the IEQ. As hypothesized, the IEQ total and the blame/unfairness subscale demonstrated good scale reliability, while the severity/irreparability of loss subscale did not. This is further evidence that the severity/irreparability of loss subscale may not resonate with individuals with SCI. This subscale comprises items related to the permanence of one’s condition, including statements such as, “My life will never be the same,” “I just want my life back,” and “I feel that this has affected me in a permanent way.” It is possible that responses to these items change over the course of recovery as people adjust to life with SCI. This sample was highly variable in terms of time since injury, ranging from ~1 year post-injury to almost 26 years post-injury. The poor scale reliability for this subscale may in fact be a reflection of the variability in time post-injury among individuals in this sample. Longitudinal research is needed to elucidate the influence of time on appraisals of injustice. Construct validity was confirmed; specifically, convergent validity was demonstrated by positive associations between injustice appraisals and increased pain, depression symptoms, perceived disability, and pain catastrophizing. The IEQ was negatively associated with life satisfaction and self-efficacy, demonstrating discriminant validity. These results are in line with previous studies demonstrating an association between high injustice appraisals and poor psychological health [5, 8, 11, 29] and pain outcomes [4, 6]. Contrary to our hypothesis, however, injustice appraisals were negatively associated with the fear of movement related to pain, though the strength of the association was negligible. Several of the aforementioned associations observed were weak or negligible in magnitude, thus these results should be interpreted with caution.

There are other limitations to consider when interpreting these results. Given the voluntary nature of research participation, it is possible that the group of individuals who elected to participate in the present study is fundamentally different from the group who declined to participate, which could meaningfully impact the variables of interest. It also bears noting that our sample lacked demographic diversity, which is a concern when determining how well the results generalize to the population of interest. In particular, data were collected from only two SCI rehabilitation sites in the United States and the sample is predominantly comprised of males and non-Hispanic whites. Given that racial differences in injustice appraisals have been previously documented [30] and given the lack of diversity in our sample, there is a clear need for additional research using a larger, more diverse sample of individuals with SCI to investigate the psychometric properties of the IEQ.

The average IEQ total score reported for the present study was 16, which is lower than previous reports in other pain and injury populations [6,7,8,9,10, 29]. This may be due, in part, to the large variance in time post-injury across our sample as ratings of injustice appraisals may change across time. Scott et al. [7], previously defined high injustice as an IEQ score > 19, which also may indicate that on average, the current sample experiences relatively infrequent appraisals of injustice. Also of note are the small sample sizes for several of the measures of interest (e.g., PCS, TSK, SF-MPQ) which may make it difficult to draw meaningful and reliable conclusions about the associations between variables.

In this preliminary validation study of the IEQ in a population of individuals with SCI, strong internal consistency and construct validity were demonstrated, however, the two-factor IEQ as described by Sullivan et al. [6], was not confirmed. Additional research will be necessary to refine the IEQ for future use in this population.