Introduction

Traumatic spinal cord injury (SCI) results in diverse motor, sensory and autonomic impairments caused by some form of external physical impact (that is, fall, motor vehicle accident, and so on). The extent of impairment is determined by the anatomic location and severity of the injury to the spinal cord. In Canada, it is estimated that there are currently nearly 44 000 people living with traumatic SCI.1 In the year 2010, there were 1785 new cases of traumatic SCI in Canada,1 with ∼1500 new cases occurring every year.2 In Ontario, the average age-standardized incidence rate of SCI between the years 2003 and 2006 was 2.4 injuries per 100 000 people,3 with an estimated Canadian SCI prevalence of 1/10 000, whereas the United States reports an annual SCI incidence rate of about four injuries per 100 000 people.4

Based on this demographic information, traumatic SCI cannot be defined as a rare disease as conceptualized in the United States5 and Europe.6 Despite this, the recruiting challenges faced by researchers conducting SCI studies are similar to those faced in clinical research conducted among individuals with a rare disease, or one with low incidence and prevalence. In fact, finding eligible and willing research participants in the SCI population, and obtaining an adequate sample size, poses a unique set of challenges for researchers.7 One reason is that SCI is a very broad classification, and the population of individuals with SCI are heterogeneous with respect to many characteristics often used to identify multiple subgroups, including, but not limited to, the etiology of the injury, age at injury, the neurological level of injury and preservation of sublesional sensory function and motor ability.8 Obtaining eligible SCI participants from one of these subgroups may therefore require application of strict pathophysiological inclusion and/or exclusion criteria, thus further restricting the pool of potential SCI participants and prolonging the duration of the study.9 Prolongation of studies beyond the expected timeline may lead to draining of resources, and funding shortages associated with lack of recruitment progress.10 In an effort to avoid these issues, and boost recruitment, researchers may be tempted to amend inclusion/exclusion criteria midstudy; this practice, however, exposes the entire study to bias, and compromises the integrity of the results. Obtaining an adequate sample size is vital to the success and relevance of any study regardless of design or primary outcome of interest.11 This issue is not unique to SCI, 85% of clinical trials do not conclude on time due to inadequate participant recruitment and 60–80% of trials do not meet their temporal endpoint due to recruitment challenges.9 Insufficient sample size can decrease the ability of a study to evaluate treatment efficacy and decrease the chance of detecting statistically and/or clinically relevant interventions.11, 12 Recruiting an insufficient number of participants also affects the internal and external validity of a study, inhibits the development of reliable evidence and delays adoption of effective interventions. Consequently, inability to attain an adequate number of participants can derail even the most meticulously designed, well-executed study.

To optimize the number of subjects/participants enrolled in a study and minimize time and expense, researchers are advised to lay out a recruitment strategy that includes a well-defined and justified screening procedures.13 A screening procedure identifies individuals who meet the inclusion criteria and do not meet exclusion criteria laid out in the study design. Robust screening will help formulate an effective study and guard against patient dropout midstudy. Designing an appropriate screening process is therefore vital to establishing eligible patient populations and suitably approaching patient sources.7 The screening process will ensure willing participants meet safety guidelines agreed upon by investigators and ethics boards,9 and provide opportunities for participating in the research for which they are eligible.

It is therefore paramount to develop the necessary tools to support the implementation of the screening procedure. Identifying the ratio of people that should be screened to successfully enroll one participant would provide researchers the ability to estimate the time and resources required for optimal subject accrual in a predictable timeframe. The process typically involves calculating the screening to recruitment (S:R) ratio (or recruitment rate), which is a method used for other health populations where low recruitment and/or high attrition hinder the conduct or completion of research trials (for example, pediatrics,14 surgery15). S:R ratios enable researchers to make informed estimates regarding the timeline, budget and personnel needs to successfully complete studies in specific research populations.

This paper describes and compares the S:R ratios of different SCI rehabilitation study methodologies (randomized controlled trials (RCTs), cohort and panel studies) and exposure/intervention types (rehabilitation, drug, telephone interview/chart abstraction, and survey) conducted at a Canadian tertiary rehabilitation facility for persons with SCI. The purpose of this report is to provide estimates of S:R ratios for SCI research using a variety of designs, and discuss potential implications of the routine use of S:R ratios as a tool for planning and implementing SCI rehabilitation research.

Materials and methods

Twenty-five studies conducted in or affiliated with a SCI tertiary rehabilitation center in Toronto, Ontario, Canada, were pooled. All studies had comprehensive prescreening, screening and recruitment data. Primary or site investigators were contacted and asked to share their screening procedure, number of individuals screened, the number of individuals enrolled, study design and sample size. Of the 32 studies identified among group members, twenty-five studies had adequate documentation of recruitment, prescreening and consent procedures, and were included in the study. For multicenter studies, only local site-specific data are reported. Only participants with SCI were included in the analysis; able-bodied controls or controls with similar impairments were omitted. The S:R ratios were calculated based on the number of consenting individuals who underwent screening divided by the number of eligible individuals enrolled in the study. All ratios are reported with a denominator of one (that is, the number of individuals screened to enroll one participant). The 25 studies comprise 10 RCTs, 9 cohort studies and 6 panel studies. RCTs consisted of rehabilitation and drug interventions. Cohort and panel studies included medical chart abstraction as well as telephone, mail and face-to-face interviews. The S:R ratios for each study methodology and exposure/intervention type were calculated and the median ratios and average sample sizes were reported.

Study design

There were three study methodologies identified: RCTs, cohort studies and panel studies. Table 1 provides the operational definitions for each of these study methodologies. Figures 1, 2, 3 provide schematic representations of the three study methodologies.

Table 1 Operational definitions for study methodologies and exposure/intervention types
Full size table
Figure 1
figure 1

Schematic of RCT study methodology.23

Full size image
Figure 2
figure 2

Schematic of cohort study methodology.24

Full size image
Figure 3
figure 3

Schematic of panel study methodology.

Full size image

Exposure/intervention type

Studies were classified into four exposure/intervention types: rehabilitation (rehab), drug therapy, telephone interview combined with chart abstraction (TI/CA) and survey. Table 1 presents the operational definitions for each of these intervention types.

Results

Table 2 shows the design and intervention type, as well as the sample size and S:R ratio for each of the studies included.

Table 2 S:R ratio and sample size for all included studies, based on study exposure/intervention and methodology type
Full size table

Study methodology

Study methodologies included ten RCTs, nine cohort and six panel studies. Figure 4 shows that RCTs had the highest median S:R ratio, followed by cohort, and panel studies. Panel studies acquired the largest average sample size, whereas RCTs accrued the lowest average sample size.

Figure 4
figure 4

Box plots showing median, quartiles and range of S:R ratios by study methodology. Median sample sizes were 34 (range 8–66) for RCT, 84 (range 41–395) for cohort and 73 (range 35–781) for panel studies.

Full size image

Exposure/intervention type

There were seven rehab studies, three drug studies, ten TI/CAs and five surveys. As shown in Figure 5, drug studies had the largest median S:R ratio, followed in decreasing ratios by rehab studies and TI/CAs. Surveys had the lowest median S:R ratio. TI/CA studies had the largest average sample size, followed by surveys, drug studies and rehab studies.

Figure 5
figure 5

Box plots showing median, quartiles and range of S:R Ratios by exposure/intervention type. Median sample sizes were 22 (range 8–47) for rehab, 40 (range 35–66) for drug, 85.5 (range 41–781) for telephone interviews/chart abstractions (TI/CA) and 59 (range 35–100) for survey studies.

Full size image

Discussion

Participant recruitment is one of the most crucial and challenging factors in the success of a research study,9, 10 and the present study represents a first effort to investigate S:R ratios in different study types, for potential use in boosting participant recruitment. In the present study, we obtained a trend in the S:R ratios for specific study methodologies or exposure/intervention types. Although these findings must be replicated in other settings, using larger sample sizes, the results of the present study suggest that S:R ratios are a useful tool for estimating the number of participants that should be screened to meet enrollment goals. S:R ratios can be applied to the planning of budgets, timelines and personnel needs to successfully complete recruitment of a desired sample size. S:R ratio data can also be used to assess the effectiveness of recruitment strategies and as a gauge of study validity.

The S:R ratios in the SCI studies reviewed varied by study methodology and exposure/intervention type. In terms of study methodology, RCTs in drug and rehabilitation therapy required the highest median number of people to be screened to enroll a single participant. Correspondingly, RCTs had much smaller sample sizes compared with the other two design types (panel and cohort), reflecting the relatively greater invasiveness and time commitment involved in RCT procedures, as well as the more rigorous inclusion/exclusion criteria and validation process used.

In terms of exposure/intervention type, median S:R ratios were lowest for surveys. Phone interviews do not require travel to an institution, facilitating ease of participation and larger sample sizes. However, results from survey studies can often be difficult to interpret and are unable to show cause–effect relationships. Drug studies had the highest median S:R ratio, likely reflecting their greater perceived invasiveness and potential risks, compared with other study types. Although rehabilitation therapies do not have the stigma attached to drug therapy, they often require a substantial commitment of time and effort, perhaps explaining why rehab studies had the second highest median S:R ratio.

S:R ratios can be used to gauge how effectively researchers sample a study population, thus indicating external validity.16 Furthermore, obtaining S:R ratio estimates before initiation of a study increases the credibility of the study’s recruitment protocol by providing quantitative data to support estimations of the time, funding and resources needed to attain the desired sample size, based on study design and intervention type. These data can be included in proposals to research ethics boards, granting agencies and oversight organizations to improve the chances of study success. In addition, examining historical S:R values for studies of similar methodology and exposure/intervention type can provide valuable insight into recruitment pitfalls, as well as a reference by which to gauge current recruitment success.17

Recruitment effectiveness can be hindered by several factors, ranging from trepidation toward a new drug or experimental treatment,18 to the method of screening.7 These types of issues become even more important for persons with catastrophic conditions, such as SCI. Preservation of a study’s external validity through sufficient inclusion/exclusion criteria, imposed a priori, may assist in preserving S:R ratios, particularly in a population as heterogeneous as that of persons with SCI.

One means by which researchers have attempted to improve recruitment in clinical trials is through central recruitment (CR), a process which recognizes that multiple uncoordinated recruitment attempts burden patients with SCI. CR involves having a single recruiter introduce multiple ongoing studies to each new patient.19 The CR process begins with an initial suitability assessment and health record screening (with the patient’s consent). If eligible to participate in research, patients are approached by a research coordinator, provided with an inventory of available research studies, and informed of their eligibility for each relevant study. This allows patients to decide on their interest in, and availability for, participating in all current studies.19 The CR process aims to minimize patient burden, while providing individuals with all the information necessary for them to make informed decisions with regard to their potential participation in current available research studies.

The CR process is currently being pilot tested at Toronto Rehabilitation Institute - University Health Network’s Lyndhurst Centre. Preliminary results have shown that: (a) the majority of patients (80%) were interested in being approached about research, (b) the duration of time from patient admission to consent has improved by 30% and (c) the length of time from the first CR visit to informed consent decision was reduced in our site from 5.6 to 2.6 days in the first 6 months, representing a 50% improvement.20 The CR process therefore appears to reduce the logistical and ethical challenges associated with recruiting subacute SCI inpatients to participate in research. The process has also been associated with high recruitment rates and a smoother, more efficient recruitment procedure.20 These characteristics make CR a promising recruitment process for SCI research, and further evaluation of the CR process scalability at Lyndhurst Centre is planned.

Limitations of the present study include that the data provided are site-specific, and that significance testing was not conducted, thus purely providing data trends. These factors limit the generalizability of the present study. In addition, a wide variety of prescreening techniques were employed (that is, referrals from medical professionals, word of mouth, poster campaigns, extensive telephone prescreening by a third party vendor, and so on.), which were not accounted for or reported in the present study and represent a substantial bias in the data reported. A standardized site-specific study screening procedure could reduce this limitation and provide valuable insight into the factors that prevent persons with SCI from participating in future research studies. The relatively small number of study designs described and intervention types evaluated was another limitation of the present study, in terms of generalizability. It is also noteworthy, however, that this analysis was specific to studies in the post-acute or rehabilitation stage, and not in the acute stage (for example, spinal cord repair or neuroprotection projects).

Despite these limitations, this paper marks the first reporting of multiple S:R ratios in SCI rehabilitation studies to date. To further address these issues, future investigations should examine a wider range of studies, with samples obtained from differing locations and conduct significance testing. In addition, future studies should look into quantifying the reasons for which individuals fail screening or do not agree to informed consent. Furthermore, routine reporting of S:R ratios is advocated for all studies,16, 21 to enable accurate tracking of the flow of participants throughout the course of the study, and future study comparisons, as well as inform future study designs. Currently, information on S:R ratios is often omitted from study results.16 The practices outlined above would shed light onto sources of patient recruitment, some flaws in research designs and point out aspects of selection bias inadvertently accounted for when constructing study protocols. New recruitment methods, such as electronic screening,22 may further assist in the provision of individualized screening for participants.

Data Archiving

There were no data to deposit.