Introduction

The risk of pressure injury (PI) development begins early following SCI with approximately one-third of all patients developing a PI before being admitted to inpatient rehabilitation. PI prevalence has been reported as 22–33% following traumatic SCI [1,2,3], 31.3% following nontraumatic SCI [4], and 36.5% within a mixed SCI sample [5]. Salzberg reported that 38.5% of patients with traumatic SCI developed a PI during the first 30 days following traumatic SCI and 48.7% developed a PI during their initial hospitalization [6]. In a prior publication, we reported a prevalence of 21% at the time of rehabilitation admission with 32% of the study cohort having at least one PI reported prior to discharge [7].

PI incidence during inpatient SCI rehabilitation is less clear [8]; although one study reported that the risk of individuals with SCI developing PIs during rehabilitation is five times greater than that of individuals with strokes [9]. There has been considerable variability in reported PI incidence in different rehabilitation settings. A study from the Swiss Paraplegic Centre in Nottwil, Switzerland reported a 25.4% incidence in patients admitted without PIs [10]. In another study, DeJong et al. found a prevalence rate of 32.1% for PIs Stage II or greater (i.e., open wounds due to pressure) before or during inpatient rehabilitation; and an incidence rate of 13.1% during rehabilitation [11]. In our prior study in Canada, 18% of patients developed at least one PI during SCI rehabilitation, with 10% categorized as Stage II or greater [7]. Overall, PI incidence during SCI rehabilitation is not dramatically different than the 24% incidence reported in the late 1970s [3].

An essential part of implementing PI prevention strategies is the accurate determination of PI risk. Unfortunately, many existing risk assessment scales (e.g., Norton, Waterlow, and Braden) are not specific to SCI and their utility has been questioned. Specifically, the items comprising these scales have been criticized for being variable [12], over-inclusive [5, 13], under-inclusive [14], inter-correlated [15], and open to interpretation [16]. Despite this, the Braden and Waterlow have been recommended for use with the SCI population [17], even though their validity has yet to be established in this population [18]. In contrast, the SCI Pressure Ulcer Scale (SCIPUS) [19] is a PI risk assessment scale developed specifically for individuals with SCI; however, recent studies of its psychometric properties have revealed limitations for both acute hospitalization [20] and inpatient rehabilitation [7]. As a result, there is currently no risk screening instrument that can be strongly recommended for individuals with SCI. This paper describes the modeling and development of a new PI risk screening instrument, the SCI Pressure Sore Onset Risk Screening (SCI-PreSORS) instrument.

Methods

Study cohort

As part of the previously reported SCI Knowledge Mobilization Network (SCI-KMN) initiative [21], data for PI incidence and prevalence were abstracted from the health records of 807 inpatients at the Toronto Rehabilitation Institute—University Health Network (TRI-UHN) Spinal Cord Rehabilitation Program, admitted after January 3, 2012 and discharged before December 31, 2014.

Outcome measures

All outcome measures were completed by trained clinical staff within 72 h of patient admission and subsequently abstracted from medical records. The Functional Independence Measure (FIM) assesses the degree of assistance required for activities of daily living and mobility, and the accompanying burden of care [22]. It contains 13 items related to motor function and 5 items related to cognitive function, and has been used across multiple inpatient rehabilitation populations [23]. At TRI-UHN, it is scored through observable patient behavior as the lowest functional level observed within 72 h following admission. The Braden scale is a six-item scale, which assesses PI risk and is intended for use with the general population. Items address sensory perception, moisture, activity, mobility, nutrition, and friction/shear [24]. The SCIPUS is a 15-item SCI-specific PI risk assessment instrument developed with community dwelling individuals with SCI [19]. Items address demographics, mobility, secondary complications, and blood laboratory values.

Data collection

Due to the nature of the SCI-KMN implementation initiative [21], data collection was deemed quality improvement by the Research Ethics Board (REB). REB exemption was granted and later extended to this secondary data analysis. Trained staff systematically reviewed pre-identified sections of the electronic health record to collect PI-related data and outcome measures described above. PI staging was performed according to the guidelines of the National Pressure Ulcer Advisory Panel [25]. PIs initially classified as unstageable were re-classified if the stage became apparent at a later date. Complete SCIPUS data were available for 603 individuals whereas Braden data were complete for 100 individuals.

PI prevalence and incidence were calculated. Prevalence is the proportion of individuals with a PI at any time during rehabilitation. Incidence is the proportion of individuals who developed at least one new PI during rehabilitation, irrespective of whether or not a PI was present at admission. Additional variables included demographics (age, gender), and injury characteristics (duration, level, American Spinal Injury Association Impairment Scale (AIS) grade). AIS grades were limited to SCIs of traumatic etiology. Individuals were designated ambulatory if they met any of the following criteria: Braden Activity score = 4 (walks frequently), SCIPUS Activity score = 0 (ambulatory), FIM Stairs score ≥ 4, or FIM Locomotion score ≥ 6 with mode = walk.

Model generation and validation

Model development was completed using a modeling cohort (n = 615). Two approaches were explored to determine PI risk: stepwise multivariate logistic regression and recursive partitioning. For both models, the dependent variable was PI incidence (any stage) during rehabilitation. Recursive partitioning is a methodology, which iteratively partitions a sample into categories based on traits (variables). Multivariate logistic regression creates a model score-based on weighted associations of traits with the dependent variable. The summed model score can then be used to determine risk above and below a given cut-off score. Criteria for each model were created with a priority to minimize false negatives (FN, cases categorized as low risk that developed a PI). To ensure robustness, the models would not allow for FN ≥ 0.5%. Modeling was limited to FIM and demographic variables due to incomplete availability of SCIPUS and Braden data. The performance of both models was tested using a validation cohort (n = 192). For comparative purposes, model metrics were also determined for the subset of cases for which Braden data were available. Explained variance (r2) was calculated and reported as the squared Pearson correlation coefficient between the model and PI incidence [26]. Analyses were conducted using SAS 9.2 and SPSS v21. Student t tests and chi-square tests were calculated to determine significant differences for demographic and other select cohort characteristics (p value < 0.05).

Recursive partitioning

The first step of the recursive partitioning model designated participants with a current PI or history of prior PI as high risk; the rationale being that screening measures should only be used for asymptomatic individuals [27]. The remainder of the cases without PI prevalence or history (n = 472) were utilized to develop the model. Subsequent steps were based on statistical criteria. Sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), and area under the curve (AUC) were calculated using univariate logistic regression and receiver operating characteristic curve analysis. The AUC represented the accuracy of the approach for risk stratification with values ≥70% considered good [28].

A node was created in the recursive partitioning model for a given variable if a cut-off value met predefined criteria. Criteria were designed to minimize FNs while reducing the likelihood that nodes incorporate variables, which discriminate only a small number of new cases. Nodes were created for variables in the following priority sequence:

  1. (1)

    Variables with 100% PPV or 100% NPV, which discriminated at least 10% of cases.

  2. (2)

    Variables with a PPV ≥ 90% and FP < 10% (deemed high risk).

  3. (3)

    Variables with NPV ≥ 90% and PPV ≥ 30% (deemed low risk).

A variable was removed from the model if it no longer independently discriminated at least 1% of overall cases following the entry of a subsequent variable.

Multivariate logistic regression

Prior to conducting the multivariate logistic regression model, variables were dichotomized to create simple constructs which could be easily administered by clinical staff. Cut-off values were defined as the value achieving a NPV between 90–99% while maintaining a PPV ≥ 30%. A modest PPV was desired so as to minimize the likelihood that the cut-off value be defined by a few outliers in the variable’s distribution. A cut-off point corresponding to 100% NPV was not accepted since the resulting binary variable would not have variability and thus not enter into the model. If NPVs for two cut-off points were equal for a given cut-off score, the one with higher sensitivity was prioritized. For variables not meeting the above criterion, the median was used for dichotomization.

Following dichotomization, univariate logistic regression analyses were completed for all identified variables. Variables with a trending association (p ≤ 0.1) with PI incidence were included in the multivariate analysis. Regression coefficients (b) were used to calculate a model risk score for each individual in the modeling cohort. The optimal model cut-off score was defined as the score with the highest sensitivity while maintaining FN < 0.5%.

Results

The 807 participants had a mean (±SD) age of 54.3 ± 18.4 years (median 57 years, IQR 40.5–67.8) and a length of stay (LOS) in rehabilitation of 64.3 ± 39.4 days (median 58 days, IQR 36–83) (Table 1). Individuals with traumatic injury (n = 289) had an average duration of injury of 133 ± 634 days (median 27 days, IQR 14–57). There were no significant differences for age, gender, injury severity, traumatic etiology, level of injury (paraplegia vs. tetraplegia), FIM, SCIPUS, or injury duration (p > 0.05) between modeling and validation cohorts. Individuals in the modeling dataset had a significantly longer LOS than those in the validation dataset (p < 0.001). Compared with individuals without PI incidence, those with PI incidence had significantly longer LOS (p < 0.001), lower FIM scores (p < 0.001), higher SCIPUS scores (p < 0.001), and lower Braden scores (p < 0.01) (Table 1). The following demographic variables were associated with greater PI incidence (p < 0.05)—gender (16.4% male vs. 10.5% female), completeness (38.8% complete vs. 11.3% incomplete), and etiology (21.5% traumatic vs. 10.4% non-traumatic). There were no differences in age, duration of injury, or level of injury between individuals with and without PI incidence (p > 0.05).

Table 1 Admission demographic variables comparisons across modeling and validation datasets as well as PI status.
Full size table

Pressure ulcer incidence and prevalence

Four hundred PIs were documented in 229 patients (28.4%) (Table 2). At admission, 165 patients (20.4%) had PIs whereas 116 patients (14.4%) developed PIs during inpatient rehabilitation. The anatomical distribution of PIs was coccyx/sacrum (40.0%), ischial tuberosities (22.0%), heels (18.0%), hip (4.5%), and other (15.5%).

Table 2 PI incidence and prevalence during SCI rehabilitation.
Full size table

In the modeling dataset, 128 individuals were admitted with PIs and 36 had a PI history. Of these cases (n = 143), 29.4% developed a PI during rehabilitation. Of those admitted with a PI but no history of PI otherwise, 27.1% developed another PI during rehabilitation. Of those with PI history but not admitted with a PI (n = 15), 46.7% developed PIs during their stay.

Model development—recursive partitioning model

In the recursive partitioning model, cases with PI prevalence or history at admission (n = 143) were categorized as high risk. For the remaining cases (n = 472), no variables met the criterion for model inclusion based on 100% PPV. Ambulation status discriminated with 100% NPV, meaning none of the ambulators (n = 129) developed a PI. For the remaining cases (n = 343), FIM Toileting also discriminated with 100% NPV. No individuals having a FIM Toileting score ≥ 4 (n = 79) developed a PI. The application of ambulation status and FIM Toileting therefore identified 208 individuals (44.1%) who did not develop PI with 0% FN. No other variable discriminated with 100% NPV for ≥10% of the cohort cases.

Next, the application of the variable FIM Bed/Chair Transfers ≥ 4 to the remaining cohort (n = 264) excluded an additional 73 cases with a 92% NPV. There were two false negatives (0.76%). No additional variables discriminated at-risk individuals using the predefined criteria. The remaining cases (n = 191) were considered at risk of PI. When the model (Fig. 1) is assessed in totality, there were only two FNs, and only one individual developed a PI with stage ≥ 2. Additional model metrics were as follows: sensitivity of 98%, specificity of 53%, AUC of 75%, and 0.33% FN.

Fig. 1: Recursive Partitioning Model.
figure 1

White boxes indicate low risk. Dark gray boxes indicate high risk. Light gray boxes indicate intermediate steps.

Full size image

Variables demonstrating a high NPV (>90%) could be considered protective traits—ambulation, FIM Toileting ≥ 4, and FIM Bed/Chair Transfers ≥ 4. Of the 281 individuals deemed low-risk, 92 had all three traits, 90 had two of the three traits, and 99 had only a single trait. Investigating cases with only a single trait, it was found that all variables retained in the model independently discriminated at least 1% of cases. Twelve (2.5%) cases were due to ambulation, 14 (3.0%) due to FIM Toileting, and 73 (15.1%) due to FIM Bed/Chair Transfers.

Model development—logistic regression

Of the 18 FIM variables analyzed, most items met the dichotomization criteria of NPV = 90–99% and PPV ≥ 30%. Four items from the motor scale (eating, grooming, locomotion, and stairs) and all items from the cognition scale did not meet dichotomization criteria therefore medians were used as their cut-off values. Of the 24 variables analyzed, univariate logistic regression demonstrated that all but four FIM variables (expression, memory, social interaction, and problem solving) had trending relationships with PI incidence (p ≤ 0.1) (Table 3). The variable with the strongest independent relationship to PI incidence was FIM Bed/Chair Transfers (FIM score < 4) [odds ratio (OR) = 29.7, AUC = 74%; Table 3].

Table 3 Univariate logistic regression for candidate variables to be included in the multivariate analysis.
Full size table

Four of the dichotomized variables entered the multivariate logistic regression model: motor incompleteness (AIS C/D), FIM Toileting < 3, FIM Bed/Chair Transfers < 4, and FIM Comprehension < 7 (Table 4). To simplify the formula for clinical usage, regression coefficients were multiplied by 2 and rounded to the nearest integer. This modification had no impact on model metrics. For the purpose of analyses, the risk score was calculated as:

$${\mathrm{Risk}}\,{\mathrm{Score}} = \, 2 \cdot\, \hbox{'}{\mathrm{MotorComplete}}\hbox{'}+3\cdot\, \hbox{'}{\mathrm{Toileting}} \, {<} \, 3\hbox{'} \, \\ +5\cdot\, \hbox{'}{\mathrm{BedTransfers}} \, {<} \, 4\hbox{'}+2\cdot\, \hbox{'}{\mathrm{Comprehension}} \, {<} \, 7\hbox{'}$$

where ‘Motor Complete’ is AIS A or B (motor incomplete is AIS C, D, E, or unspecified); ‘Toileting < 3’ is FIM Toileting score < 3; ‘BedTransfers < 4’ is FIM Bed/Chair Transfers score < 4; and ‘Comprehension < 7’ is FIM Comprehension < 7.

Table 4 Multivariate logistic regression.
Full size table

To determine a risk cut-off value, risk scores were calculated for each case in the dataset and sensitivity/specificity analyses were completed. The cut-off score which maintained FN < 0.5% was determined to be a risk score > 5 and had a sensitivity of 97%, specificity of 54%, and three FN (0.49%). For the other outcome measures, cut-off scores were determined to be SCIPUS > 5 (n = 440, sensitivity 97%, specificity 14%, FN 0.45%) and Braden < 18 (n = 100, sensitivity 100%, specificity 49%, FN 0%) (Table 5).

Table 5 Variable metrics as determined for the various datasets.
Full size table

Validation

Using the validation dataset (n = 192), psychometric performance was determined for the new models and SCIPUS scale. For comparative purposes, the analysis was also performed for the FIM motor subscale and the single variable FIM Bed/Chair Transfers score < 4. Results are summarized in Table 5.

While the SCIPUS AUC did not exceed 70%, the recursive partitioning and logistic regression models had AUCs of 78% and 74% respectively in the validation dataset. Furthermore, both the recursive partitioning and logistic regression models designated fewer cases as at-risk (45% and 52% respectively) compared with the SCIPUS (88%); with recursive partitioning also designating fewer at-risk cases than Bed/Chair Transfers score < 4 (47%).

Braden data were not available for the validation dataset as it was only determined for a subset of the modeling dataset cases (n = 100). The Braden performed well with a perfect sensitivity of 100% and 0% FN. A comparative analysis was run using each model while using only the cases in this subset who did not have admission PI prevalence or history (n = 80). When analyses were limited to these 80 cases, recursive partitioning performed best with 100% sensitivity, 0% FN, and AUC = 82%; while also categorizing the fewest number of cases as at-risk (41%) (Table 5). The logistic regression model also performed well with 100% sensitivity, 0% FN, AUC = 79%, and 46% of cases categorized as at-risk.

Discussion

PI is a potentially devastating complication following SCI. Given the personal impact and associated costs on the individual and health care system, it is imperative that we improve our ability to identify at-risk individuals. Resources and interventions could then be directed to these individuals with greater efficiency. To address this need, we explored and evaluated two novel approaches (recursive partitioning and multivariate logistic regression) to determining PI risk during SCI rehabilitation. The performance of both approaches was compared with existing scales (SCIPUS, Braden). The recursive partitioning model, titled the SCI-PreSORS, performed best. In the modeling cohort, sensitivity, specificity, and FN were 98%, 53%, and 0.33% respectively. Of the 281 individuals deemed low-risk, only two developed PI (with only one being a PI stage ≥ 2). The multivariate logistic regression also performed well in the modeling cohort (sensitivity = 97%, specificity = 54%, FN = 0.49%). Observations were similar in the validation cohort with FNs (n = 2) limited to 1% for both models. Interestingly, a single variable, FIM Bed/Chair Transfers, accounted for the majority of the observed predictive value for PI incidence. Additional variables in the models served to improve metrics incrementally.

Although the explained variance (r2) of the SCI-PreSORS model was low, it still outperformed the other models. A low explained variance is not unexpected given that the SCI-PreSORS is intended primarily for screening. Given the primary goal of screening is to identify all at-risk individuals, the investigators prioritized minimizing the false negative  rate over accurately predicting true positives. Following the completion of screening, individuals deemed at risk should undergo further comprehensive assessment.

In comparison to the overall cohort, the four observed false negatives with recursive partitioning (two modeling cohort, two validation cohort) were older. Mean age was 79.4 year (range 67–92 years) with 50% being male. The etiology of the spinal cord injury (SCI) was non-traumatic (lumbar stenosis, vascular, and congenital) for three individuals. It is noteworthy that the one individual who developed a PI stage ≥ 2 (stage 4) was 92 years of age.

Braden data were limited to 100 cases collected as part of a previously published evaluation of the SCIPUS [7]. In order to evaluate approaches to risk assessment in the same individuals, a secondary analysis was limited to cases with Braden data admitted without PIs (n = 80). In this analysis, all models demonstrated 100% sensitivity; however, the SCI-PreSORS demonstrated the greatest specificity by identifying fewer patients at risk.

In determining the relative value of different approaches to PI risk assessment, a low tolerance for FN (0.5%) was employed during model development. This was due to FNs having the potential to end in harm. Incorrect classification of individuals as low-risk for PI has significant health and legal repercussions. In this context, a test with maximal sensitivity is preferred [27]. Given that individuals with SCI are already known to be at high-risk of PI compared with other clinical populations [29], the optimal approach to partitioning risk turned out to be the identification of individuals at low risk as opposed to high risk. This is reflected by the fact that, while no variables had 100% PPV, ambulation and minimal assistance in toileting (score ≥ 4) demonstrated 100% NPV (protective factors). Inclusion of these variables improved the performance of the SCI-PreSORS compared with FIM Bed/Chair Transfers alone; particularly in identifying fewer individuals as at-risk (specificity).

While all of the studied approaches to risk assessment demonstrated good sensitivity, enhanced specificity was a noteworthy advantage for the SCI-PreSORS (recursive partitioning model). While the SCIPUS attained high sensitivity, it identified >80% of patients as at-risk for PI development (low specificity). In comparison, the SCI-PreSORS was able to accurately identify a large number of cases as low risk while maintaining close to 100% sensitivity. By accurately identifying individuals at low-risk for PI, the burden on clinical staff and hospital resources can be reduced by facilitating their targeted and efficient allocation toward the patients who need them most; fundamentally the sole purpose of completing risk screening.

The variables that demonstrated value for PI risk stratification and which were ultimately incorporated into the SCI-PreSORS (ambulation, FIM Toileting, and FIM Bed/Chair Transfers), provide additional insight into factors contributing to PI incidence. All of the above address aspects of mobility, and individuals who require minimal assistance to complete these activities are at comparatively low-risk of PI. This could be attributable to the fact that such individuals likely reposition independently and achieve adequate pressure relief. Analysis also revealed that the ideal cut-off for the Braden (<18) is different for SCI rehabilitation, compared with previously reported risk thresholds (<17 or <19) [18, 24]. This is important to note for institutions that continue to utilize the Braden for this population.

While the accurate and efficient identification of at-risk individuals is critical, it does not guarantee PI prevention. This is dependent on the efficacy of accompanying prevention plans and interventions. This point is highlighted by a recent multi-site initiative, which systematically implemented risk assessment and inter-professional prevention plans at six SCI rehabilitation centers [30] Documentation rates were markedly increased for risk assessment and prevention plans, however there was no demonstrable change in PI incidence.

While practice guidelines have recommended risk assessment and inter-professional treatment plans [17, 31], there is still relatively little Level I evidence (randomized trials) supporting the efficacy of interventions targeting PI prevention in SCI [17] and a direct link has yet to be demonstrated between prevention strategies and PI incidence in SCI rehabilitation. There is therefore an ongoing need to strengthen the evidence base underpinning practice guidelines and to better understand the relationship of complex, multi-faceted rehabilitation practices, and their ultimate impact on patient outcomes.

Study limitations

A potential limitation of both the recursive partitioning and multivariate logistic regression approaches is their reliance on variables derived from the FIM. Although the FIM remains the most widely utilized measure of burden of care and independence and has been shown to be valid in SCI [32], alternative measures such as the Spinal Cord Independence Measure (SCIM) are increasingly utilized and have advantages in specific populations such as SCI. Many of the constructs derived from the FIM, however, such as transfer performance, etc. could potentially be derived from alternative measures such as the SCIM or even patient observation. The use of analogous variables derived from alternative source data should be explored and validated in future work, as this would enhance the utility and generalizability of the SCI-PreSORS.

It should also be acknowledged that the described models were developed at one center, which admits a broad representation of individuals with spinal cord pathology of traumatic and nontraumatic etiology. Historically, the majority of inpatients are admitted for initial rehabilitation, with <5% representing readmissions. The validity and utility of the PreSORS would therefore be enhanced further by confirming its clinimetric properties in other contexts (e.g., sites, patient demographics).

Conclusions

The current study employed two modeling methods to determine PI risk among individuals participating in SCI rehabilitation. The SCI-PreSORS (recursive partitioning model) is composed of presence of PI history, ambulation, and two FIM variables (Toileting, Bed/Chair Transfers). Initial development of the SCI-PreSORS, as well as validation, was performed using retrospective data from one site. Validity would be enhanced further by confirming findings in a prospective cohort. Generalizability also needs to be demonstrated by evaluating the SCI-PreSORS at additional sites and potentially in different SCI subpopulations, prior to it being used widely for clinical purposes.