Introduction

A spinal cord injury (SCI) is a health condition that has serious consequences for the individual injured, their family, and the health care system. The prevalence of SCI has been found to be the highest in the United States (US) and lowest in France, at 906 per million and 250 per million, respectively [1]. Globally, there are a greater number of males, compared to females, that are injured and the peak incidence of injury occurs between the ages of 15 and 30 years [1]. Given the number of years individuals live with the injury, medical care is important with significant implications for the health care system.

Health care research is conducted to advance the scientific field being studied and ultimately influence the care provided to, and outcomes of, specific health populations. Research production is stimulated by the prevalence of a disease or problem, potential funding opportunities, and availability of resources within the health care system. This research is largely driven by the execution of randomized controlled trials (RCTs). RCTs are considered the gold standard in terms of research design [2] and they are often used to guide the development of clinical practice guidelines [3]. It has been shown that the number of RCTs published in various neurorehabilitation populations has been increasing, for example, in brain injury [4] and stroke [5]; however, few studies have assessed publication patterns in SCI research.

Current studies have examined rates of study completion, rate of citation, influence of funding body, and priorities in SCI research. A study by DePasse et al. [6] assessed rates of study completion and publication within SCI research by examining ClinicalTrials.gov. Of 626 studies identified, the study completion rate was 39.9%, of which 47.6% had been published; this highlights that a large number of findings have not been publically disseminated. When assessing the most-cited publications within SCI, Nowrouzi et al. [7] found that articles which focused on pathology and treatment had the highest lifetime and annual citation rates, with the number of highly cited articles increasing significantly after 1995. An American media content analysis was completed to determine the trends pertaining to cure- versus rehabilitation-focused SCI research [8]. While the majority of research noted in the media focused on curing SCI (80%), mainly for the purpose of promotion and funding, rehabilitation research was referenced when discussing ongoing studies or final study results [8].

These analyses have begun to identify important issues with the development of research itself, such as the influence of funding bodies on the publication of results [6], the alignment of research priorities with those of persons with SCI [9], how research is being framed to the public [8], and where influential research is being conducted. Despite these findings, to the best of our knowledge, a thorough examination of the trends and quality of all RCTs in SCI has not been completed. The continuous improvement of study methodology has been stated as a prerequisite for furthering the rate of advancements [10]. A review of the methodological quality of studies across several medical fields was conducted and found that, in general, overall quality of studies had increased over time, with the most frequently evaluated criteria being concealed allocation and blinding [10]. There is value in evaluating the research that has already been conducted in terms of quality and interventions to determine future directions and evaluate the progress that has been made to date. Therefore, the objective of this study was to map out and characterize the quantity and quality of all published SCI RCTs with respect to number, sample size, and methodological quality between January 1970 and December 2016.

Methods

A mapping review was undertaken, which aims to characterize the quantity and quality of existing literature within a specified field.

Article selection

A literature search of multiple research databases (i.e., PubMed, EMBASE, PsycINFO, CINAHL, and Scopus) was conducted for articles published from database inception up to and including December 2016. In addition to the MeSH headings “spinal cord injury,” “paraplegia,” “quadriplegia,” and “tetraplegia,” a variety of key search terms, tailored to each database, were used to identify the extensive range of interventions studied in SCI. To be eligible for inclusion, studies must have adhered to the following a priori criteria:

  1. 1.

    the research design was an RCT;

  2. 2.

    the written language was English;

  3. 3.

    the population was adult (>18 years old), human subjects; and

  4. 4.

    at least 50% of the sample had a SCI.

Studies were included regardless of the setting in which the intervention was applied (e.g., acute, rehabilitation) or type of injury (i.e., traumatic or non-traumatic). Further, studies were included regardless of the control group used (i.e., no treatment, usual treatment, other treatment, and placebo).

Data extraction and methodological quality

The following data were extracted from each RCT: author(s), year of publication, country of origin (based on primary author’s affiliation), initial and final sample size, intervention, and control. Two experts assessed each study (AMc, BB), came to a consensus on their classification, and collaboratively categorized them according to a primary topic area based on both intervention applied and the outcome of interest. If there were any discrepancies between the two reviewers, a third reviewer (SJ) was consulted.

To quantify the methodological quality of each study, two reviewers scored each RCT using the Physiotherapy Evidence Database (PEDro; Table 1) tool. When using the PEDro tool, a study is rated on 11 items that require a “yes” or “no” response. Each “yes” response equates to one point for a total maximum score of 10 [11]. The first item on the checklist is a measure of external validity and excluded in the final score. The PEDro tool has demonstrated good reliability [12] and validity [13]. Total scores were compared between reviewers; however, inter-rater reliability was not measured. In cases where there was a scoring discrepancy, a third reviewer was consulted. Consensus in the literature [14] indicates that studies may be classified as poor, good, fair, or excellent depending on their total PEDro score (<4, 4–5, 6–7, 8–10, respectively); this classification was applied in the current study.

Table 1 Physiotherapy evidence database tool
Full size table

Statistical analysis

The total number of RCTs was tallied for each primary topic area and in combination. Continuous data, including sample sizes and PEDro scores, were summarized as either mean (standard deviation) or median (range), as appropriate. A one-way analysis of variance (ANOVA) was performed to compare continuous variables between groups; post hoc t-tests were conducted for comparisons when appropriate. Total number of RCTs, sample sizes, and PEDro scores were also organized by year to examine trends over time. Statistical significance was defined as p < 0.05 and all analyses were conducted using IBM SPSS Statistics (Version 21; 1989–2012).

Results

Number of RCTs

In total, 466 RCTs were published between 1971 and 2016 on interventions evaluated in the SCI population (Appendix). Articles were published in 172 different journals, with 123 journals each publishing just one SCI article, 19 journals publishing two SCI articles, and 10 different journals publishing three SCI articles. A total of 20 different journals each published more than three SCI RCTs. The two most common journals that published SCI RCTs were Spinal Cord (n = 68, 14.6%) and Archives of Physical Medicine and Rehabilitation (n = 63, 13.5%). RCTs were categorized into 29 groups by primary topic area (Fig. 1); the three most commonly studied topic areas were Bladder (n = 78, 16.7%), followed by Pain (n = 54, 11.6%), and Lower Limb (n = 45, 9.7%). The publications came from a total of 38 different countries. The three most common producers of SCI RCTs were the US (n = 191, 41.0%), followed by Canada (n = 52, 11.2%), and Australia (n = 30, 6.4%). Between 1971 and 2011, the rate of RCT publications increased linearly; however, between 2012 and 2016 the rate tripled, resulting in 40.8% of all studies being published during this time period (Fig. 2). In closer examination of this time period, the US doubled its publication rate in 2013 and 2014, thereby producing 20% of its total volume in these two years; no other country demonstrated this same trend.

Fig. 1
figure 1

Number of RCTs published by topic area

Full size image
Fig. 2
figure 2

Number of RCTs published each year between 1971 and 2016

Full size image

Sample size

For 73.4% of RCTs, there were reportedly no participant dropouts from the beginning of the study to its completion. Overall, the median final sample size was 28 (range 4–691). In total, 247 (59.4%) RCTs had less than 30 subjects. Among the various topic areas, Bone (81.3% vs. 18.2%, respectively) and Lower Limb (71.1% vs. 29.9%, respectively) had the largest difference in proportion of studies with less than or greater than 30 subjects. A one-way ANOVA demonstrated no significant change in sample size among all studies over time (F = 0.815, p = 0.770).

PEDro scores

The overall mean PEDro score was 5.56 (1.68), with 10.5% of studies having poor, 39.9% fair, 35.4% good, and 14.2% excellent methodological quality. Mean PEDro scores improved from 5.0 (1.4) in 1976 to 6.3 (1.9) in 2016 (F = 2.230, p < 0.001). In examining individual PEDro items, there were significant differences in the proportion of studies which satisfied each item. In general, adherence was higher for items relating to random allocation (98.1%), baseline similarity for groups (54.9%), key outcomes obtained for >85% of the sample (76.4%), intention-to-treat (53.9%), results presented for at least one outcome (88.8%), and point measures and variability (84.8%); however, <40% of studies adhered to items relating to concealed allocation (18.5%), subject blinding (28.3%), therapist blinding (15.0%), and assessor blinding (37.1%). It is possible that some criteria are impossible to satisfy given the type of intervention applied; therefore, for greater clarity, the number of RCTs satisfying items related to concealed allocation, subject blinding, therapist blinding, and assessor blinding were examined by topic area (Table 2).

Table 2 Percentage of RCTs in each topic area satisfying PEDro items 3, 5, 6, and 7
Full size table

Discussion

The current review found 466 published RCTs in the field of SCI over 45 years. Upon closer examination it was found that the rate tripling after 2012 was due to two publication rate anomalies in the US (2013–2014). Potential reasons for this may include increased funding for SCI research through the establishment of a national SCI research network; however, at this time the exact reason is unknown. RCTs were conducted in a diverse number of countries and published widespread in academic journals.

Both the US and Canada were among the top producers of SCI publications. This may be attributed to the fact that these two countries have two of the highest incidence rates of SCI in the world [1]. It may also speak to the expertise and strong research and funding agencies established in these two countries. As well, the inclusion criteria of the current study restricted articles to the English language, which is the first language of both the US and Canada; therefore, there may be an inherent bias towards uncovering articles from these countries. The finding that the US and Canada lead in SCI publications is confirmed by studies reporting that North America is at the forefront in terms of publishing the most cited traumatic SCI articles, and those with highest annual citation rates [7, 9].

The three most commonly studied areas of SCI were bladder, pain, and lower limb; impairments in these domains are highly prevalent post SCI. For example, chronic pain is a frequent secondary complication for individuals with SCI, with incidence rates of up to 80% [15] and 30–40% of people experiencing severe disabling pain [16]. Importantly, from a consumer perspective, these areas are considered of significant importance for the rehabilitation of individuals with SCI [9].

The current study found that there was no significant change in sample size over time and that a large proportion of RCTs had a 0% attrition rate. Although the absolute number of RCTs is increasing, in general, more participants are not being recruited. It was not clear in all studies whether sample sizes were determined by power calculations, as recommended by clinical trial guidelines [17], or simply by convenience sampling. The latter option is a limitation of rehabilitation research in general, as studies are often limited to a single site and have narrow inclusion criteria [18].

Compared to the other topic areas, Bone and Lower Limb, had fewer trials with samples over 30 subjects. Bone health encompasses various topics such as bone fragility, osteoporosis, and bone resorption. Early identification of these issues can prove cumbersome, as they arise from biochemical changes in bone mineral density that may be undetected to a person with SCI until they have had their first fall or sign of bone fragility. Lower limb interventions utilize expensive technologies such as exoskeletons, gait trainers, and body-weight supported treadmill training that often require the assistance of a therapist; as such, it may be costly to recruit a large number of participants.

Ensuring that RCTs are properly designed, conducted, analyzed, and reported is critical to minimizing bias and yielding an accurate estimation of treatment effect. The current review revealed a significant improvement in methodological quality over time, which may reflect the creation of reporting guidelines [19] and appeals for trial registration [20]. Similar trends have been reported for RCTs of interventions in other medical fields [10, 21].

Studies have shown that inadequate or unclear reporting in RCTs of concealed allocation [22,23,24,25] and blinding procedures is a common occurrence across a wide range of disciplines [26, 27]; this is in line with the findings of the current review. These issues may be attributable to a lack of proper reporting rather than poor methodology. Studies have found that many authors of RCTs stated that certain procedures not explicitly reported in their article had in fact been conducted [28, 29]. Similarly, Mhaskar et al. [30] reported that publications often omitted important methodological features that were present in a previously published protocol, which may have been uncited or inaccessible to readers.

Lack of blinding in RCTs may reflect issues of feasibility. Schulz and Grimes [31] noted that, while concealed allocation can always be implemented, blinding of all parties is not always possible. Certain interventions impede or preclude the blinding of participants and therapists, although it is generally agreed upon that outcome assessors must be blinded whenever possible [26, 31, 32]. Accordingly, the current review found that assessors were most blinded (37.1%), followed by participants (28.3%) and therapists (15.0%). SCI rehabilitation primarily involves active learning and training rather than passive manipulation or ingested medication, and so it is difficult to develop controls that are blinded when delivering active aspects of the treatment [32]. As well, self-reported outcome measures are common in rehabilitation research, which obscure the distinction between participants and assessors [33].

The current review also determined that the greatest adherence was to randomization (PEDro item 2) and between-group statistical comparisons (PEDro item 10). While it is concerning that complete adherence was not achieved for these items, several studies have shown that a small but notable portion of RCTs have inadequate or unclear randomization procedures [24, 34,35,36]. This issue could be attributed to a poor understanding of randomization methodology [37], or more likely, poor reporting [30]. Statistical analyses of RCTs frequently include within-group comparisons; however, in the absence of between-group comparisons, such analyses are highly misleading and do not provide an accurate representation of treatment effectiveness [38].

Limitations

The current review is limited in that we omitted studies published in other formats (e.g., conference abstracts, gray literature); therefore, our findings may not be fully representative of all SCI research. The PEDro tool is also subject to limitations in its evaluation of RCT quality. Its accuracy is limited by the breadth and clarity of the information reported in the appraised article [39]. Furthermore, the PEDro scale does not account for several factors that may influence study quality, including the presence of sample size calculations [32, 40], appropriateness of statistical analysis [41, 42], and type of outcome measure (i.e., objective versus subjective) [43, 44]. While each included clinical trial was categorized by topic area, their primary results were not evaluated nor summarized. This information would be useful for both researchers and clinicians; however, it was not the aim of this study to undertake this particular analysis.