Introduction

Traumatic spinal cord injuries (TSCI) negatively impacts patients’ physical, psychological, and social well-being with a high risk of developing secondary health conditions [1, 2]. There are many challenges in the continuum of care of individuals with TSCI. Proper immobilization and timely access to specialized care are the main challenges in pre-hospital care of patients especially in developing countries where there are issues about infrastructure and trained personnel [3]. The in-hospital care of TSCI is complex and needs timely, efficient and robust interventions to decompress the spinal cord, support the perfusion to spinal cord, and help the functional recovery [4]. Lack of healthcare providers with specialized knowledge and skills and poor accessibility to diagnostic and therapeutic facilities are challenges of in-hospital care in many low-resource regions [4, 5]. The post-hospital care including rehabilitation services is also important for patients’ functional independence, reintegration into society and prevention of secondary health conditions such as pressure ulcers, genitourinary complications (e.g., urinary tract infection (UTI)), and chronic pain [6]. These conditions not only limit patients’ functional ability but also increase healthcare utilization after acute care. Studies show that about 27–57 percent of individuals with SCI experience re-hospitalization in the first year after discharge [7,8,9,10]. However, most of the common causes of re-hospitalization after SCI (e.g., UTI, pressure ulcers, and pain) are potentially preventable with appropriate care. This highlights the importance of the Quality of Care (QoC) for individuals with TSCI.

Improving the QoC of individuals with TSCI requires a comprehensive evaluation to identify the vulnerabilities of the system in delivering high-quality care throughout the continuum of care, planning and implementing evidence-based interventions and monitoring the results. Identifying the weak points in delivering care necessitates an iterative process that measures the domains of care quality that patients receive in a standard and comprehensive manner. This can be achieved by developing an assessment tool that contains evidence-based QoC indicators.

From our team’s recent review we found that TSCI indicators of QoC had not been summarized in previous literature [11]. Therefore, we conducted a scoping review to summarize these indicators [11]. There is, however, lack of evidence in the literature with respect to a tool that compiles a core set of clinically relevant, feasible, and valid QoC indicators that can be used to evaluate the domains of care that patients with TSCI receive. Therefore, the purpose of this study was to develop a list of QoC indicators that are valid, feasible, and representative of the quality of care that individuals with TSCI receive throughout the patient-care continuum (pre-hospital, in-hospital, and post-hospital).

Methods

This study was composed of the steps described by Giesen et al. in the “Questionnaire Development” book and by Brancato et al., in their handbook [12, 13].

Conceptualization and study design

In the present study two approaches were utilized to collect the concepts of QoC for individuals with TSCI. First, we used the results of our previous scoping review to retrieve the concepts of QoC from literature. Second, we designed a qualitative interview using experts’ opinions in order to assure the completeness of the list of concepts that were collected in the scoping review.

For the qualitative interview, a physician interviewed four TSCI healthcare professionals who were not informed about the concepts retrieved in the scoping review. The professionals were identified based on their expertise in SCI management and included one physiotherapist, one general surgeon, one emergency department specialist and one intensive care unit specialist. The interviews were done in two steps, for a total of eight interviews. In the first step, to avoid any effect on the expert’s opinions, and without presenting our concept’s list, they were asked: “In your opinion, which concepts are important to assess the QoC of individuals with TSCI?” They were asked to define/describe the concepts they chose as being important indicators in the pre-hospital, in-hospital and post-hospital setting. In the second step, they were shown the list of collected concepts from the scoping review and asked to provide suggestions. The newly suggested concepts were highlighted.

The quality of care concepts included in the scoping review study were 87 items and were classified into three main groups based on the structure, process and outcome of care (Donabedian framework [14]). In the current study, the retrieved concepts were re-evaluated and some of them integrated into a more general concept. Then each concept was further categorized based on: time-frame (pre-hospital, in-hospital, and post-hospital) and data source (patient report by questionnaire, available national registry, request from care centers, other). Next, the relationship of each concept with six domains of health care quality: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity [15] was determined. Subsequently, the concepts were screened based on priority and importance. At this phase, we contacted the corresponding authors (N = 93) of the articles, which were found in our team’s published scoping review [11]. A Google form was created containing each concept’s category, as well as its data, and presented to the abovementioned experts via email. These experts were selected because they had the experience to publish at least one paper describing a QoC concept for the field of spinal cord injury and were therefore familiar with the process of care delivery. They were asked to provide a value for each concept on a 10-point scale, with 1 as the lowest value and 10 as the highest value for the priority and importance of each item to serve as a QoC concept for TSCI. The experts’ scores served as inclusion criteria for further evaluation of each concept. The concepts that had (1); a median score of 7.0 or higher, and (2); a score of seven or higher by at least 70% of participants, were determined as core concepts of care.

Operationalization

After determining the core concepts of care we translated each concept into one or more indicators that represent that concept. For example timely management of TSCI is a concept that is related to the timeliness domain of care. This concept was operationalized as an indicator as proportion of patients who undergo decompression surgery within a specific time frame (e.g., 24 h from injury). The concepts were operationalized into indicators by using expert comments. These experts (n = 11) were all Iranian clinician scientists in the field of SCI care and included six neurosurgery specialists with a focus on traumatic and non-traumatic spinal cord injury, one emergency medicine specialist, one professor of health sciences and medicine with a focus in rehabilitation services research, one spine surgeon, one trauma care specialist and one orthopedic surgeon with spinal trauma specialty. The experts then determined indicators’ face and content validity by completing a Likert scale on the relevance, simplicity, clarity and essentiality of the developed indicators. A 4-point Likert scale was used for the relevancy and simplicity scales, and responses included: 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = very relevant for relevancy scale, and: 1-not simple 2-somewhat simple 3-quite simple 4-very simple for simplicity scale. In 4-point Likert scales, ratings of 1 and 2 were considered content invalid while ratings of 3 and 4 were considered content valid. For the clarity end essentiality scales we used a 3-item likert: 1 = not clear, 2 = items need revision; 3 = very clear for clarity, and: 1 = not essential; 2 = useful, but not essential; and 3 = essential. We then calculated the content validity index (CVI) [16]. CVI was measured by adding the total points of indicators with high scores (scores three and four of the Likert scale) and dividing this sum by the total number of responders. Content validity ratio (CVR) was also calculated based on the Lawshe table [17] using the following formula: CVR = (Ne - N/2)/ (N/2), in which “Ne” was the number of experts indicating “essential,” and “N” was the total number of experts (Table 1). The minimum value of CVR needed to retain the indicators was determined based on the number of panelists, as described elsewhere [17, 18]. Since we had 11 experts in this phase, the value of 0.59 was considered the cut-off for indicator selection based on CVR. Therefore, acceptance of the indicator was based on the following criteria: CVI score higher than 0.79 and CVR higher than 0.59 = adequate, CVI score between 0.79 and 0.70 = questionable and requires revision, and CVI score less than 0.70 or CVR less than 0.59 = unacceptable and eliminated.

Table 1 Relevance, clarity, simplicity, content validity index, and content validity ratio of quality of TSCI care indicators.
Full size table

Revision, process monitoring, and evaluation

We used the method of expert review to redesign the tool [12]. The experts involved here were Iranian experts (n = 11) who were also involved in the operationalization phase of the study. All questions that were misleading or difficult to answer were removed. The tool was then revised and reset in terms of content and technical aspects. Finally, the questions were clarified and simplified in the revision step. The research group met several times to discuss the content, layout, and response options for indicators. Consensus methods were used to reach agreement [19]. Finally the accepted indicators formed the content of the QoC assessment tool for TSCI. At this stage, we designed a series of questions to represent these indicators and serve as a practical way to represent them. To address each indicator of the QoC assessment, it was necessary to ask one or more questions from different response sources. We used relevant questions from the National Spinal Cord Injury Registry of Iran (NSCIR-IR) or related standard questionnaires to design the final assessment tool (Supplementary Appendix 1). In fact NSCIR-IR is used as the main data source, to develop, format and design the questions that represent each quality of care indicator. In the end, the comprehensiveness of the tool was evaluated by asking 11 experts to fill out a Likert scale with four answer options; “Not comprehensive”, “To some extent comprehensive”, “Fairly comprehensive” and “Fully comprehensive”. Figure 1 illustrates the assessment tool development process.

Fig. 1: The assessment tool development process.
figure 1

The indicators of care quality were retrieved from a scoping review and qualitative interviews with experts (n = 75). Then 12 international experts rated the core concepts of care on a 0–10 scale. The concepts that had (1); a median score of 7.0 or higher, and (2); a score of seven or higher by at least 70% of participants were determined as core concepts of care and underwent operationalization (n = 44). Indicators with high content validity index and content validity ratio were retained and formed the content of the final assessment tool (n = 27). CVI content validity index, CVR content validity ratio, NSCIR-IR National Spinal Cord Injury Registry of Iran.

Full size image

Results

Indicator selection and development

Scoping review and qualitative interview

Overall, 94 concepts were identified for QoC from scoping review (87 concepts) and qualitative interviews with experts (7 concepts).

Conceptualization

The retrieved 94 concepts were shared with the Iranian experts for summarization and integration purposes. This process resulted in the combination of 19 concepts into a more general concept. Therefore, the process of conceptualization provided us with 75 total concepts affecting the QoC in TSCI. As mentioned above, to evaluate these concepts, we contacted 93 experts by email. Twelve of them actively collaborated. Based on the criteria, 31 concepts were excluded and 44 concepts were included (Table 1). This process showed that the core concepts of care for individuals living with TSCI which are clinically relevant and feasible are related to access to care, timeliness of care, adherence to standard SCI care guidelines, patient safety measures, specific outcomes of care, and patient’s satisfaction with health care as a whole.

Operationalization

Content validity scores

After operationalization of the concepts, all 44 indicators were presented to 11 Iranian experts, to evaluate the indicators’ CVI and CVR. CVI scores ranged from 0.72 to 0.96, and CVR scores ranged from −0.27 to 1. After revision of items with low score in clarity scale (n = 3), twenty-seven indicators had adequate validity, and 17 indicators had low validity and thus were removed (Table 1). The mean and standard deviation (SD) CVI for relevance clarity and simplicity for the entire tool were 0.91 ± 0.07, 0.84 ± 0.07, and 0.85 ± 0.08 respectively. The CVR for the entire tool was 0.79 ± 0.13. Finally, 91 percent of experts valued the assessment tool as fairly or fully comprehensive. Table 1 summarizes the CVI and CVR scores for each indicator.

Characteristics of the final indicators

Table 2 summarizes the characteristics of the final core set of quality indicators based on Six Aims of the Institute of Medicine for the 21st Century Health Care System [15] along with the phases of care. The quality indicators developed included pre-hospital (n = 3; eg, proper immobilization), in-hospital (n = 12; eg, in-hospital mortality rate), post-hospital (n = 9; eg, employment rate of patients within one year after injury) and mixed between in and post hospital (n = 3; eg, satisfaction with pain control). Measures designed to assess the effectiveness (n = 5; eg, readmission rate), timeliness (n = 3; e.g., early decompression), patient-centeredness (n = 3; e.g., Satisfaction with care), equity (n = 3; eg, Patients’ employment rate), efficiency (n = 2; e.g., length of hospital stay), and safety (n = 1; eg, proper immobilization). Some indicators were related to both domains of safety and effectiveness of care (n = 7; eg, rate of secondary complications) and some were related to equity and timeliness (n = 3; eg, proportion of SCI-equipped centers in a specific area have 24 h availability of Magnetic resonance Imaging). Supplementary Appendix 1 contains the information that is needed to address these indicators in an assessment tool format.

Table 2 Characteristics of finalized 27 indicators of QoC in individuals with TSCI.
Full size table

Discussion

This study represents the development and design of a healthcare-related assessment tool through extensive discussion among experts over an extended period of time. To the best of our knowledge, this is the first study that introduces a comprehensive assessment tool to evaluate the QoC in the care continuum of patients with TSCI.

We developed an assessment tool containing information that makes the health-related QoC assessment feasible in patients with TSCI. To achieve a unique assessment tool, we asked international experts to appraise each concept of care quality and then used the Iranian experts to design the assessment tool. We used only Iranian experts for the latter stage because we had to design a Farsi version of the assessment tool with meaningful, clear and feasible questions (practical to measure) for our outcome assessors (nurses, patients, data providers, etc). In fact we evaluated the face validity of our assessment tool with Iranian experts.

Although Khosravi et al. [11] initially identified 87 concepts for TSCI patient care quality; our final count was limited to 27 after expert panel appraisal and content validity analysis. For instance, the cost of care could be an indicator influencing patient outcomes, but because its evaluation was not feasible, the experts decided to exclude it. Ultimately, 27 indicators with high clinical relevance, clarity, and simplicity were included in the assessment tool. This collection of indicators is believed to be essential and sufficient to measure QoC for individuals with TSCI and reflects a need for integrated reporting of QoC information in the pre-hospital, in-hospital, and post-hospital phases. This tool provides a path to evaluate QoC in these patients, and can be used as a guide for decision-making and creating health policies [20, 21].

Developing QoC indicators has been the subject of recent on-going studies. The SCI-High project is among these studies. The project has prioritized 11 domains for rehabilitation care of SCI patients and aims to develop at least one indicator for structure, process and outcome of each prioritized domain [22, 23]. To date the SCI-high project has established 32 indicators for 7 domains of cardiometabolic health; emotional well-being; sexual health; tissue integrity; urinary tract infection; walking, and wheeled mobility for SCI patients [24,25,26,27,28,29,30]. Although these indicators can be used to evaluate rehabilitation care, ensure continuous quality improvement, and ultimately ensure evidence-based advancement of rehabilitation care for patients with spinal cord injuries, there has been no focus on other stages of care such as pre-hospital and inpatient care in the SCI-high project.

The strength of our assessment tool is that it contains core set of evidence-based, clinical relevant indicators regarding all phases of care of TSCI. In addition we designed our tool based on available information from NSCIR-IR in order to ensure the feasibility of data collection. We believe that the items of our tool are measurable in low-income, middle-income and high-income countries with a reasonable time and cost efficiency. This study has several limitations that need to be highlighted. First, only 12 from 93 experts (13%) who were invited to participate in indicator selection actively collaborated. However, we believe this low response rate has not caused a bias because these 12 experts were representative of the diversity of specialties, from different health settings in low-income, middle-income and high-income countries and all had expertise and experience in the care of individuals with TSCI. Second, although our Iranian experts were from diverse disciplines involved in specialized trauma services (neurosurgery, trauma rehabilitation, orthopedic trauma surgery and emergency medicine), surgical experts predominated, and we did not involve nurses, and patients. Third, there are limitations with respect to data availability, accuracy and completeness across different care facilities [31]. Although we have suggested a core set of indicators that are designed based on the availability of NSCIR-IR data, feasibility studies in diverse health settings are necessary to determine whether this assessment tool is applicable or not. An ongoing project is being carried out in Iran to check the feasibility of this assessment tool [32]. Fourth, we were not able to benchmark our QoC indicators against a “gold standard” partly due to a lack of information regarding our indicators. Future studies should measure and report these indicators to provide evidence in order to formulate a quantitative measure for evaluating and benchmarking the quality of care.

In conclusion, our study presents a QoC assessment tool with clinical utility for patients with TSCI. This tool, however, should be further validated in future studies.