Introduction

Challenges in translating research evidence into the clinical setting are well known. It has been reported that only 14% of research is translated into practice and it takes an average of 17 years for this to occur.1, 2 Frequently reported barriers in moving evidence into practice include limited time, expertise, administrative support and resources.3, 4

To overcome these barriers, Fixsen et al.5 (part of the National Implementation Research Network (NIRN)) describe three approaches in translating knowledge first in fields such as education and now in health care. The first is a passive approach referred to as ‘letting it happen’, whereby research findings are only published (diffusion); the second involves providing aids, such as toolkits, ‘helping it happen’ (dissemination); and finally, the most active approach involves focusing on the implementation process to ensure the environment supports ‘making it happen’ (implementation).5 Various implementation frameworks have identified methods to assist with the implementation, also known as key ‘drivers’ of effective implementation, that consider intervention characteristics, external and internal environments, participants and the active change process of implementation.5

Similarly, the ‘Knowledge to Action’ cycle proposed by Graham et al.6 differentiates knowledge creation (studies, toolkits) from the action cycle (application of knowledge) and outlines the relationship among the action phases within the cycle. Furthermore, the National Institutes of Health (NIH) Roadmap expands the process of moving new innovations into practice by describing the additional research laboratory of practice-based research as the needed translational step to improve incorporation of discoveries into the front line of clinical care.7

In the field of spinal cord injury (SCI), the literature reports variability in all aspects of care, including acute clinical care,8 the management of chronic pain9 as well as expectations regarding outcomes given by clinicians to patients.10 There are multiple challenges in a field such as SCI to identify and implement evidence-based practices. Randomized controlled trials (RCTs) are required to generate level 1 evidence, but are often difficult to implement in clinical practice, do not often produce generalizable results and may preclude some people with SCI from receiving innovative therapies.11 Various strategies have been developed to summarize the body of evidence available and to assist with translating existing evidence into the clinical setting. The Spinal Cord Injury Rehabilitation Evidence (SCIRE) Project synthesizes research evidence on rehabilitation practices to inform health-care professionals, scientists, policymakers and consumers with SCI.12 The Consortium for Spinal Cord Medicine also synthesizes research evidence and has developed clinical practice guidelines and consumer guides.13 Guidelines and evidence syntheses provide platforms upon which knowledge translation (KT) interventions can be built. In other words, they ‘let’ and ‘help’ KT to happen but do not ‘make’ it happen. The SCI Quality Enhancement Research Initiative (QUERI) takes KT a step further by striving to ‘make’ it happen by systematically implementing clinical research evidence into practice to enhance the quality of care and outcomes.14 Even with the development of these and other interventions to disseminate and implement best practice in order to improve patient care, there is a need to examine whether KT interventions have affected outcomes—whether KT ‘happened’ and if so, was it effective.

In this study we will refer to KT interventions as factors used to assist the process of implementing practice change as well as the targeted practice itself. The objective of this study was to conduct a systematic review of the literature that evaluates original research publications on KT interventions used throughout the SCI continuum of care (prehospital through community) and to determine the effect of the implementation on practice through clinician behavior change and the impact on patient outcomes.

Materials and methods

Search strategy

A preliminary PubMed search was conducted to ensure there were no other published systematic reviews on this topic. A protocol was developed after consultation with a research librarian. Four electronic databases, MEDLINE (1946–present; OvidSP), PubMed (1946–present; PubMed), CINAHL (1982–present; EBSCOhost), EMBASE (1974–present; OvidSP) and PsycINFO (1887–present; EBSCOhost) were searched for English studies published from January 1980 to July 2012. The search strategy was based on previously published protocols.15 Search terms related to SCI, such as ‘spinal cord injury’ or ‘tetraplegia’, were combined with search terms related to KT, such as ‘knowledge translation’ or ‘implementation’, to generate a broad list of articles. The terms were searched both as subject headings and as keywords (see Supplementary File 1). No additional searching from nonelectronic sources was done.

Article selection

All articles identified from the search strategy were imported into the reference software RefWorks and duplicates were removed. Two reviewers (VKN and SEP) examined the article titles and a list of relevant articles was generated. The same two reviewers then independently screened the abstracts for inclusion and consensus was obtained in cases where there was a disagreement. Finally, full articles were acquired and independently reviewed for inclusion by the same two reviewers. Any duplicates not identified by RefWorks were manually removed during the abstract and full article review.

The study inclusion criteria were: (1) the article described the process for implementing a KT intervention and the methodology for evaluating the implementation (actively implementing research evidence into practice); (2) the KT intervention targeted patients with either a traumatic or nontraumatic SCI; (3) the KT intervention was related to clinical practice or education in any phase of the continuum (prehospital, acute, rehabilitation, community); and (4) the article was original research written in English and published between January 1980 and July 2012.

The study exclusion criteria were: (1) the article described the development of a clinical tool or measure (for example, clinical practice guideline or outcome measure) but did not describe the implementation; (2) the article described the impact on patient outcomes but did not describe the implementation; (3) less than half of the sample was individuals with SCI or the study had a sample size less than three subjects; or (4) unpublished materials. Included studies had to describe the methodology used to evaluate the implementation process but did not have to include methodology used to evaluate the effect on patient outcomes, as the main focus of the study was on the former. To capture all relevant studies examining implementation related to SCI across the continuum of care, no articles were excluded based on study design or phase of care.

Data abstraction

Studies included in the final selection were abstracted by one of the reviewers (SEP) and verified by the other (VKN). The data abstraction form was developed by modifying an example provided by Scott et al.15 in their previously published protocol (Supplementary File 2). For all studies, data abstracted included describing (1) the study (setting, location, design, number of sites involved); (2) the sample (sample size, inclusion/exclusion criteria); (3) KT intervention and hypothesized outcome; (4) KT implementation methods (deliverer of KT intervention, evidence supporting its uptake, strategies used for implementation); and (5) the evaluation of the clinician behavior change and effect on patient outcome (description of evaluation procedure for implementation, outcome measures used, facilitators/barriers to implementation). No attempt to contact original study authors was made to obtain missing information.

Study quality rating

Study quality rating was independently assigned by two raters (NPT and SEP) using the Downs and Black checklist16 that assesses the study reporting quality, external validity, internal validity (bias and confounding) and power. The last item on the Downs and Black checklist, related to the power calculation, was modified. Rather than providing a score from 0 to 5, a score of 1 was assigned to studies that included a power calculation and a score of 0 was given to studies without any power calculation. This resulted in a maximum score of 28 instead of 32. A higher score reflects a more thorough reporting of results and more rigorous study methodology. Downs and Black score ranges were grouped into the following four quality levels according to the range suggested by Samoocha et al.:17 excellent (26–28), good (20–25), fair (15–19) and poor (⩽14). Consensus among the raters was obtained when there was a disagreement on ratings. In addition, based on the type of study design, raters assigned a level of reported evidence as per the methodology used by SCIRE.12 Level 1 evidence reflects the strongest evidence and includes high-quality RCTs. Conversely, level 5 is the lowest score for evidence, based solely on expert opinion, a case report or an observational study.

Results

The initial search identified 4650 articles. Following the reviews based on the title, abstract and full text, 13 articles18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30 related to 10 studies were included (see Figure 1). The articles included were published in a variety of journals; 7 of the 13 articles were published in the SCI journal20, 21, 23, 26, 28, 29, 30 and the remainder were published in rehabilitation (n= 3)18, 19, 22 or other types (n=3) (for example, Implementation Science).24, 25, 27

Figure 1
figure 1

Flow diagram of studies selected and included.

Full size image

An overview of the 10 studies is provided in Table 1. The studies were conducted primarily in the United States (n=5)20, 21, 23, 26, 27, 28 and within the acute phase of care (n=4).21, 25, 26, 29 Most studies used a pre–post design (n=8)21, 22, 23, 24, 25, 26, 28, 29 that resulted in level 4 evidence. The Downs and Black score ranged between 3 and 19, with a median rating of 12. The majority of 13 articles were classified as ‘poor’ with a rating of ⩽14.18, 19, 20, 21, 22, 23, 24, 25, 27, 29 There were two articles considered to be ‘fair’,26, 28 but none were considered ‘good’ or ‘excellent’.

Table 1 Overview of studies
Full size table

An overview of the KT intervention for each of the 10 studies is outlined in Table 2. The interventions all targeted clinician behavior and are classified into three categories based on the overall goal of the KT intervention; seven studies developed and implemented a patient care protocol that was not a current standard of care at their site,18, 19, 21, 23, 24, 25, 26, 29 two studies included a KT intervention to provide education20, 27, 28 one study incorporated outcome measures into the clinical setting.22 The KT methods used to assist implementation (drivers) included promoting clinician’s competency in 6 studies,18, 19, 20, 21, 22, 24, 27, 28 using organizational strategies in all 10 studies,18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29 and engaging the leadership in their organization in 5 studies20, 21, 24, 27, 28, 29 (see Table 3).

Table 2 Overview of KT interventions
Full size table
Table 3 Knowledge translation (KT) methods (drivers) to assist with the implementation
Full size table

Evaluation of clinician behavior change and the effects on patient outcomes are described in Table 4. The drivers identified in Table 3 were not consistently evaluated (see Table 4). The methods used to evaluate changes in clinician behavior included auditing documents (n=6/10 studies),18, 19, 21, 23, 25, 26, 29 conducting a self-reported survey of knowledge or change in practice based on the KT intervention (n=5/10 studies)18, 19, 20, 22, 24, 27, 28 and measuring attendance at training sessions (n=1/10 studies).20 Only 3 out of 10 studies evaluated the clinician behavior change using statistical analyses; all 3 studies reported some significant changes.21, 23, 25

Table 4 Evaluation of KT interventions
Full size table

All studies except three evaluated the effect of the implementation on patient outcomes,18, 19, 21, 22, 25, 26, 27, 28, 29 and six of these seven studies evaluated the effectiveness using statistical analyses.18, 19, 21, 25, 26, 27, 28, 29 Four studies reported statistically significant changes in patient outcomes;25, 26, 27, 28, 29 the implementation of a pathway decreased in-patient mortality25 and length of stay,26 formation of a multidisciplinary team reduced length of stay and improved clinical care26, 29 and a campaign to raise awareness of the importance of vaccination produced an increase in vaccination rates and the patients’ perception of their importance.27, 28

Barriers to implementation were reported in 5 of 10 studies18, 19, 21, 22, 23, 27, 28, 30 and facilitators to implementation were reported in 5 studies18, 19, 21, 22, 23, 25, 30 (see Table 5). The most commonly reported barrier was lack of resources, and communication was the most frequently described facilitator, both of which were at the clinician level.

Table 5 Barriers and facilitators of implementing knowledge translation (KT) interventions
Full size table

Discussion

To our knowledge, this is the first systematic review describing implementation in SCI care delivery. In comparison, research in health conditions such as cancer have conducted more reviews in this area31, 32 and have progressed to further synthesize implementation evidence in a systematic review of published reviews.31 Although evidence pertaining to implementation of research in SCI is in its infancy, results from this systematic review of KT strategies in SCI suggest that it is an emerging area of research, as 12 of the 13 articles included were published after the year 2000.18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 30 In only one study an implementation intervention using the QUERI approach in the United States was described,27, 28 in only two studies clinical practice guidelines developed by the Consortium for Spinal Cord Medicine were implemented21, 23 and no one reported using evidence from SCIRE, which was less than expected. Implementation of the KT interventions occurred primarily in the acute21, 25, 26, 29 and the rehabilitation phases.18, 19, 20, 22, 23 It is interesting to note that none occurred in the prehospital phase, possibly because of SCI being included in general trauma studies in this phase of care or the lack of interest to conduct SCI-specific studies focusing on prehospital care.

It is difficult to compare the study quality of implementation studies in SCI with other health conditions because of the different rating scales used. In this review, study quality was evaluated using the Downs and Black checklist16 as part of the SCIRE methodology, but this checklist was not commonly used in other reviews32, 33, 34 and it is not specific to implementation science. According to the categorization proposed by Samoocha et al.,17 the majority of the articles were considered to be poor (10 of 12 articles),18, 19, 20, 21, 22, 23, 24, 25, 27, 29 suggesting the need for more rigorous study methodology. In a broader systematic review examining KT strategies in allied health professions, Scott et al.35 used a different quality assessment tool, but similarly found that studies were of low methodological quality. Therefore, low-quality rating appears to be a problem in not only SCI, but also in the KT field in general. It should be acknowledged that controlled studies involving the implementation of clinical practices are extremely difficult to conduct, both from ethical and logistical perspectives.

A recent review by Boaz et al.36 recommended including only studies whose KT interventions are derived from evidence as this is a fundamental principle of evidence-based practice. Almost all of the studies included in this review implemented evidence based on multiple research studies and expert opinion.18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29 Other than the clinical practice guidelines21, 23, 26 or accreditation standards,20 which might have proposed recommendations based on RCT-generated evidences, it was interesting that only one appeared to cite evidence from RCTs.21 A best practice that is amenable to a RCT may not be appropriate for an individual clinical setting where there is need for initial implementation of more basic best practices. As well, resource availability may not allow for RCT-validated best practices that may involve costly equipment and/or staff training. The decision to implement evidence into practice should also consider factors such as feasibility, relevance to practice and impact on patient outcomes, but these factors are often overlooked and not consistently evaluated. Development of criteria for conducting and reporting implementation studies would enable study results to be compared within the same health condition, such as SCI, and across different types of health conditions.

In terms of the evidence generated from the implementation efforts included in this review, most of the studies produced level 4 evidence,18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29 and there were no RCTs to generate level 1 evidence. This is not surprising as there are ethical obligations to provide all patients with the best practice care, making it challenging to conduct RCTs. In addition, the usefulness of the RCT in evaluating KT interventions has been questioned because of the often highly complex nature of KT interventions,37, 38 the risk of contamination of the intervention and control groups37 and the expense.37 However, it has been suggested that to ensure the robustness of the results, researchers should consider using time series designs, action research, detailed case series and controlled before and after studies as alternatives to RCTs.38 Given the complexity of KT intervention studies, the traditional levels of evidence may not apply and a greater focus should be on the quality of the study methodology.

Drivers used to execute the implementation efforts included targeting clinician competency, organizational structure and leadership in 4 of the 10 studies.20, 21, 24, 27, 28 In contrast, 3 of the 10 studies23, 25, 26 only addressed organizational drivers. All studies used multiple implementation drivers.18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29 In the literature, the effectiveness of changing clinicians’ behavior with single versus multiple drivers is not well defined.36, 39 Among the studies included in this review, studies that succeeded in changing clinician behavior did not necessarily utilize a greater number of drivers. In addition, it was not possible to ascertain the effectiveness of the individual drivers based on the studies included in this review, as it is often a set of drivers chosen based on a local context that become integrated and complement each other that determine the success of an implementation.5 However, facilitators and barriers to implementation were noted.

The most commonly reported facilitator was communication within study groups.18, 19, 22, 25 This was as expected, because sufficient communication between team members needs to occur in order to ensure competency of its members and develop an effective team. It was surprising that no studies mentioned the importance of communicating with leaders in the organization, as this has been identified as a major facilitator in other studies.5 In this review, all 10 studies reported evaluating clinician behavior change indicators; however, only 3 studies evaluated the results on changes in clinician behavior empirically and demonstrated improvements.21, 23, 25 The lack of empirical evaluation makes it difficult to determine whether the changes were significant. Grimshaw et al.39 suggested that the rationale for the selection of drivers as well as documentation of the contextual data pertaining to the implementation need to be included in implementation studies. Future studies should address these points and include the theoretical justification for selecting the drivers and the resources to ensure lessons learned can be applied to SCI and other health areas with complex multifaceted interventions.

The ultimate goal of any implementation is to improve patient outcomes. In this systematic review, 7 of the 10 studies evaluated changes in patient outcomes.18, 19, 21, 22, 25, 26, 27, 28, 29 Patients acquiring new knowledge or changing attitude were most frequently reported.22, 27, 28 None of the implemented practices were successful in reducing secondary complications such as deep vein thrombosis,21 pressure ulcers18, 19, 26 or pneumonia25, 26 that will be of interest to clinicians. The implementation of the clinical pathway for managing patients with cervical and thoracic SCI admitted to intensive care units significantly reduced length of stay and in-patient mortality without a reduction in secondary complications, suggesting the need for further studies to determine whether this was a result of methodological issues (for example, measures used and sample size) or the effectiveness of clinical pathways developed.26 It is interesting to note that only three studies evaluated the impact of the KT intervention using patient’s self-reported outcomes, as this provides valuable information from the patients’ perspective.18, 19, 22, 27, 28 One study reported a significant increase in the self-reported vaccination rate and perception of vaccination importance,27, 28 whereas the other study reported no change in the quality of follow-up care received22 or number of pressure ulcers or urinary tract infections based on self-reported outcome measures.18, 19 Using more rigorous study methods, adequate sample sizes, appropriate outcome measures that assess patient outcomes and proper statistical analyses (for example, adjust for potential confounders) to measure the impact will enable the KT intervention to be more effectively evaluated. Furthermore, many studies implemented the targeted practices in a single center;20, 25, 26, 29 future studies should include multiple centers to evaluate team functioning. This will also assist with subject recruitment and provide the statistical power needed to evaluate the study results. Currently, the level and amount of evidence required to support a full multicenter implementation of a KT intervention do not exist which makes it difficult to translate the findings from this review and enable clinicians or policy makers to use the results. This information would assist in knowing what KT interventions should be adopted in SCI care settings and would be relevant to other health conditions.

In considering the results from this systematic review, it is important to recognize the limitations. There is very little published in SCI examining implementation, and therefore the sample size was small, with only 13 articles and 10 studies meeting the inclusion criteria. The study had a narrow publication range from 1980 to 2012 and only included original studies. As KT in SCI is a new field, this study likely captures most relevant publications using this time frame and excluding publications from earlier years does not severely affect the generalizability of the study result. Limiting the search to published studies may have an effect of inflating the overall quality of the studies and perhaps biasing our findings toward showing positive results, as such studies are more likely to be published, and thus included in this review. It could also result in the exclusion of ongoing projects studying the implementing of evidence into clinical practice that have not yet been published, thereby underestimating the amount of evidence available, or research done in this field. Even though this study used two independent raters and developed a protocol with inclusion/exclusion criteria, it was often difficult to classify studies as utilizing active implementation (‘making it happen’) as compared with more passive approaches (‘helping it happen’). As there are no standardized terms used in the field of KT, it is possible that the search strategy did not include all relevant terms and may not have identified all eligible studies. The need for well-defined and standardized terms has been raised by McKibbon et al.,40 who reported 100 different terms used within the field. Refinement in the terminology will improve the methodology of KT studies and the reporting of results in future studies. Despite these limitations, this review serves as a benchmark of the state of implementation in SCI. A current initiative, the Knowledge Mobilization Network has partnered with the NIRN to implement and evaluate pressure ulcer treatment guidelines.41 Results from this project will provide important evidence concerning the process of implementation and the effectiveness of this process to affect patient and clinical outcomes. With the growing need to incorporate KT as part of funding applications, it will create awareness for the importance of this kind of research and help advance the field.

Conclusion

This systematic review provides an overview of implementation research in SCI that will be of interest to clinicians and policy makers. A total of 13 articles reporting on 10 studies were included. Results from this review demonstrated promising results in both changing clinician behavior and having an impact on important patient outcomes, such as in-patient mortality. The results from this study urge future implementation efforts to include a theoretical justification for the KT intervention selected and to ensure rigorous study methods are used to evaluate the effectiveness of the implementation.

Data Archiving

There were no data to deposit.