Introduction

Anxiety disorders are the most common class of mental disorders, affecting an estimated 20% of adults (40 million people) in the US.1, 2, 3, 4 Of these individuals, at least half experience multiple anxiety disorders and other comorbid conditions such as mood and substance use disorders. Accurate information on the generalizability of treatment outcome studies will therefore depend on having accurate information about how study samples were enrolled and what filters were applied.

The current diagnostic system, DSM-5, distinguishes anxiety disorders (specific phobia (SP), social anxiety disorder (social phobia; SO), panic disorder (PD), agoraphobia and generalized anxiety disorder (GAD)), obsessive-compulsive and related disorders, and trauma- and stressor-related disorders (for example, post-traumatic stress disorder (PTSD)) (see Appendix specific diagnoses). It is presumed that these are independent and discrete disorders, as is major depressive disorder (MDD). Yet, the symptoms often overlap across diagnoses and can vary substantially within diagnoses. The same treatments, spanning pharmacotherapy, behavioral therapy and their combination, are also indicated for the spectrum of anxiety disorders5 (Supplementary Table 1). Reflecting these concerns, the Research Domain Criteria (RDoC) initiative is fostering trans-diagnostic research that is explicitly agnostic with respect to filtering by traditional diagnoses to evaluate brain-based mechanisms of dysfunction that may cut across these diagnoses.6, 7, 8, 9 Pursuit of RDoC research relies on sample enrollment that is not filtered by diagnosis, but reflects, for example, all people seeking treatment at clinical service.

However, standard practice in treatment outcome studies has been to focus on a particular discrete diagnosis and rule out comorbid disorders. Although exclusion criteria are commonly applied to manage patient safety and confounds with the measurements of interest, exclusions due to comorbidity may limit the relevance of findings for clinical translation. For example, up to ~80% of patients with comorbid conditions are excluded from treatment studies of MDD10 and schizophrenia.11 A recent study applying typical criteria for clinical trial exclusion to the STAR*D trial showed that these criteria would exclude ~80% of patients seen in primary care, an important issue for prescribers given that that trial findings might not always translate to the patients being treated in usual practice.12

An important step towards a complementary RDoC framework for the spectrum of anxiety psychopathology and its comorbidities would be to characterize (i) the extent of comorbidity across anxiety disorders and (ii) the nature and impact of exclusion criteria in anxiety disorder studies of pharmacotherapy, psychotherapy and their combination. To address these issues, this review characterizes the prevalence of anxiety disorders comorbid with each other and with depression by undertaking a secondary analysis of data from the National Comorbidity Survey-Replication (NCS-R).13, 14 We then summarize the nature of exclusion criteria used in treatment studies of anxiety disorder; the percentage rates of patients ruled out due to these criteria; and the evidence for the potential impact of these exclusions on the treatment outcome results. Based on the findings, we recommend an approach for developing standardized exclusion criteria reporting relevant to RDoC-motivated and trans-diagnostic research.

Materials and methods

Characterizing the prevalence of anxiety disorder comorbidity using national survey data

We undertook a secondary analysis of the NCS-R data to characterize the comorbidity of anxiety disorders with each other and with MDD. The NCS-R survey includes lifetime diagnosis information for 9282 individuals aged ⩾18 years.13, 14 DSM-IV diagnosis information was available for five anxiety disorders (PD, GAD, PTSD, SO, and SP) and MDD. See Supplementary Table 2 for the lifetime prevalence of each disorder displayed independently and split by sex. The percentage of lifetime comorbidity across disorders was calculated in several ways. First, to assess the extent of comorbidity across pairs of diagnoses, we determined the proportion of individuals for each diagnostic pairing that had both disorders out of (a) the total number of individuals with at least one anxiety disorder, and (b) the number of individuals diagnosed with each member of the comorbidity pair (for example, out of the individuals with PD those who also had PTSD). Second, a similar set of analyses was conducted looking at diagnoses triplets (for example, number of individuals who were comorbid for GAD, as well as PTSD and PD). The percentages of comorbidity were then represented as heat maps (created in R; R Foundation for Statistical Computing, Vienna, Austria; URL http://www.R-project.org)15 with darker colors indicating higher percentage of comorbidity.

The nature and impact of exclusion criteria based on a meta-review of studies

We then completed a structured literature review of the impact of exclusion criteria on anxiety disorder treatment outcome research. This review was part of the Cross-Disease Review of Exclusion Across Medicine (CREAM) project, a structured literature review of studies of exclusion criteria and their impact across a range of medical specialties (for example, oncology, cardiology, rheumatology and psychiatry).

Search strategy and selection criteria

In the Cross-Disease Review of Exclusion Across Medicine (CREAM) project, literature was identified systematically by conducting English-language searches in PubMed (Date of Search: Oct 1, 2014 on the following terms: “Eligibility criteria and generalizability” (anywhere in paper), “exclusion criteria and generalizability” (anywhere in paper), “exclusion criteria” (in title of paper) and “eligibility criteria” (in title of paper). To be considered relevant, studies had to analyze data on i) the prevalence and nature of exclusion criteria, ii) the overall and specific rates of exclusion due to commonly used exclusion criteria, and/or iii) the impact of exclusion criteria on sample representativeness or outcomes. From this cross-disease pool of literature, evidence on individual diseases was extracted for focused reviews, in this case studies addressing anxiety disorders.11

Results

The prevalence of anxiety disorder comorbidity

Sixty percent of individuals with one anxiety disorder had at least one other anxiety disorder or depression diagnosis, and 27% of individuals had three or more of these disorders comorbid. Importantly, comorbidity rates differed across disorders (Figure 1). For diagnostic pairs, the proportion of individuals with two comorbidities ranged from 4.5 to 20.3% within those diagnosed with at least one anxiety disorder (Figure 1a). The greatest comorbidity rates were observed between pairs of SO-MDD (20.3%), MDD-SP (18.6%) and MDD-GAD (18.3%), and the lowest comorbidity rates between pairs of PD-PTSD (4.5%) and PD-GAD (5.3%). These percentages differed when looking at comorbidity profiles within each diagnosis separately (for example, percentage of individuals with PD who also had GAD, PTSD, SO, SP, or MDD), such that the proportion of individuals with a given disorder who also had a second disorder (for example, individuals with MDD who also had PTSD) was not the same as the proportion of individuals with the second disorder who also had the first disorder (for example, individuals with PTSD who also had MDD) (Figure 1b). Notably, individuals diagnosed with either PTSD or GAD had a high percentage of comorbidity with MDD (60.4% and 63.6%, respectively); however, those with MDD had a relatively lower percentage of comorbidity with PTSD or GAD (20.0% and 26.1%, respectively).

Figure 1
figure 1

Proportion of lifetime comorbidity heat maps. Proportion of individuals with (a) pairs of diagnoses out of those who had at least one anxiety disorder diagnosis (N=2611); (b) pairs of diagnoses using the disorder listed in the column header as the denominator; (c) three diagnoses out of those who had at least one anxiety disorder diagnosis and; (d) three diagnoses using the disorder listed in the column header as the denominator. For example, the top left cell represents the proportion of individuals with panic disorder who also had specific phobia and major depressive disorder. These results were generated from a secondary analysis of the National Comorbidity Study data.13, 14 GAD, generalized anxiety disorder; MDD, major depressive disorder; PD, panic disorder; PTSD, post-traumatic stress disorder; SO, social phobia; SP, specific phobia.

Full size image

Similar findings were also observed when considering comorbidity triplets (Figures 1c and d).

The nature and impact of exclusion criteria in anxiety disorder treatment research

The papers returned from the search results can be categorized roughly into three types: those that (i) reviewed the prevalence of exclusion criteria in treatment research studies, (ii) reviewed the rates of exclusion by applying commonly used exclusion criteria to an independent sample of treatment-seeking individuals, and (iii) evaluated the impact of exclusion criteria on outcomes or sample representativeness. Table 1 gives a specific breakdown of the studies that covered each of these topics, organized according to the specific anxiety disorder of focus in each study.

Table 1 Summary of the studies reviewed
Full size table

Prevalence of exclusion criteria

Eight publications provided information regarding the prevalence of exclusion criteria across studies of PTSD, PD, GAD, SO and obsessive-compulsive disorder (OCD)16, 17, 18, 19, 20, 21, 22, 23 (Table 2; further details in Supplementary Results). However, one study that reported commonly used exclusion criteria for GAD did not report the actual usage frequency of independent exclusion criteria.21 Studies varied greatly in how they explained exclusion criteria. Search dates are not readily available since some studies used other meta-analyses. Some studies only listed the most common exclusion criteria and did not include actual frequency of use. These studies are summarized and described in Table 2.

Table 2 Summary of the prevalence of exclusion criteria applied in treatment studies of anxiety disorders
Full size table

Together, these findings indicate that while numbers of exclusion criteria vary across empirical studies and disorders, several are used in greater frequency than others. The most commonly reported exclusion criteria in studies across disorders were psychosis (80% of all studies), substance dependence (69%; most were even more expansive excluding for any substance use/misuse: 41%), bipolar disorder (47%), comorbid MDD (39%) and suicide risk (36%).

Psychotherapy randomized controlled trials (RCT) have slightly fewer exclusion criteria than pharmacotherapies. This is apparent for both GAD and SO, for which the median numbers of exclusion criteria for psychotherapy were 5 and 7, respectively, whereas the median numbers for pharmacotherapy were 11 and 10, respectively.

Rates of exclusion

Nine publications provided information regarding how many patients are being excluded as a result of commonly applied exclusion criteria, reporting on exclusion rates across PTSD, OCD, PD, GAD and SO.16, 18, 19, 20, 21, 23, 24, 25, 26 Of these, four categorized the exclusion rates of specific exclusions by first performing a meta-analysis to determine commonly used exclusion criteria and then applying these exclusion criteria to an independent database of treatment-seeking individuals for OCD, PD, GAD and SO19, 20, 21, 23 (Table 3). Three reviewed the exclusion rates by aggregating percentages of exclusion across previously published studies.16, 18, 26 The remaining two presented exclusion rates from individual empirical studies assessing the impact of exclusion criteria on treatment efficacy.24, 25

Table 3 Summary of the rates of exclusion because of each exclusion criterion applied in in treatment studies of anxiety disorders
Full size table

Combining the findings from these studies yielded several clinically relevant observations regarding exclusion criteria rates. First, the percentage of individuals that would have been excluded due to at least one exclusion criteria was substantial, ranging from 72% of those with OCD19 up to as much as 92.4% of those with PD.20 Second, not surprisingly, the percentage of individuals who would have been excluded varied dramatically by the type of exclusion and the primary disorder studied. For example, the percentage of participants excluded due to a psychosis diagnosis was relatively small yet ranged from 2.8% of those with OCD19 to 21.2% of those with PD.20 Similarly, the percentage of those excluded due to comorbid depression varied from 16.3% of those with OCD to 70.6% of those with a PD diagnosis.

Further findings regarding the general and specific exclusion rates due to application of exclusion criteria are presented in more detail for each study across anxiety diagnoses in the Supplementary Results.

Impact of exclusion criteria on sample representativeness and outcomes

Seven publications provided information regarding the impact of exclusion criteria on sample representativeness and outcomes across PTSD, OCD, PD, GAD and SO.16, 22, 24, 25, 26, 27, 28 Of these, four were meta-analyses focused mostly on clinical outcomes that compared effect sizes across studies that included varying degrees of exclusion criteria.16, 22, 26, 28 The remaining three compared demographics and outcomes between patients who would have been excluded due to commonly used exclusion criteria and those who would have been included.25, 27, 29 One meta-analysis26—which collapsed across PD, GAD, and depression and was based on eight studies—reported a negative correlation between the number of exclusion criteria described in the methods section of each article and the percentage of treatment seekers at termination. Given the rather large differences in methodology and conclusions of the above studies, the impact of exclusion criteria on sample representativeness and outcomes are provided below for each diagnosis separately.

PTSD impact on representativeness: No study to date has examined differences in demographics or baseline clinical symptoms between patients typically excluded from RCTs and those included.

PTSD impact on clinical outcome: To determine the impact of exclusion criteria on clinical trial outcomes, Bradley et al.16 correlated the number of exclusion criteria from each of the 26 studies in their meta-analysis with the pre- versus post-treatment effects sizes. Number of exclusion criteria was significantly related to pre- versus post-treatment effect size (r=0.42, df=23, P=0.03), such that studies with more exclusion criteria reported higher effect sizes.

OCD impact on representativeness: In 2000, Franklin et al.24 compared demographic and outcome information from a group of 110 individuals who had either been excluded from (79%) or refused to participate in (21%) RCTs conducted at the Center for the Treatment and Study of Anxiety (CTSA) to that of individuals from four previously conducted RCTs. Although they did not separate the demographic information for refusers and excluded individuals, demographic information on mean ages (CTSA sample=34.2, RCT samples=34.8, 33.8, 30.5 and 31.6), gender ratios (CTSA=47% females, RCT=46%, 53%, 55%, 56% female) and education (CTSA=45% with undergraduate or graduate education, RCT=34 and 44%) were similar across studies. Many pieces of the demographic information for the previously conducted RCTs were not available for several or all of the included studies. Pretreatment OCD severity was comparable between samples.

OCD impact on clinical outcome: Franklin et al.24 found that change in pre- to post-treatment severity for OCD symptoms in their CTSA study was similar to two previously conducted RCTs, but greater than two others (CTSA mean reduction=60%, RCT samples reduction=62, 54, 40 and 32%). Mirroring the reduction in OCD symptoms, they also found that depression severity—as indexed by Beck Depression Inventory (BDI) scores—also improved to a greater extent than in the three available RCTs (CTSA mean reduction=57%, RCTs sample reduction=39%, 43%, and an increase in one of the RCTs of 15%). To further examine the effects of treatment across the samples, they calculated effect sizes from the change in pre- versus post-treatment OCD and depression severity. Regarding the OCD severity, the effect size for the CTSA sample was larger than three out of the four RCT samples, and slightly smaller than one sample (CTSA effect size=3.26, RCT effect sizes=2.31, 0.93, 3.88 and 1.00). The CTSA sample also had a larger effect size for the pre- to post-treatment change in depression severity (CTSA effect size=1.26, RCT effect sizes=0.93, 0.79 and −0.33).

Panic disorder impact on representativeness: In 2002, Mavissakalian and Guo25 examined the differences in demographic variables of patients who were excluded from a drug trial for OCD and those who were accepted. Those who were excluded were more likely to be male (88 vs 77%), have a later OCD onset (30.6 vs 26.8 years) and a shorter duration of illness (7.3 vs 10.0 years). However, a potential explanation for the differences in duration of illness and onset of OCD was that exclusion criteria for this study included a minimum number of panic attacks per month and at least moderate PD severity. Overall, likely as a consequence of these restrictions, the accepted group had significantly lower functioning and higher anxiety scores than the excluded group.

GAD: No study to date has examined differences in demographics or baseline clinical symptoms between patients typically excluded from RCTs and those included.

SO impact on representativeness: Juster et al.27 conducted an empirical study comparing demographic information and clinical outcomes of participants who were entered into an RCT (n=47) to those deemed ineligible to participate in the drug arm (n=28). Both groups received cognitive behavioral therapy (CBT) for SO. No differences were found between accepted and excluded patients regarding gender ratio, age, employment, education or number of years with SO. Similarly, accepted participants did not differ from those excluded regarding 8 questionnaires or 10 independent assessor measures of severity.

SO impact on clinical outcome: In terms of outcomes, those enrolled were significantly different than those excluded on only three measures of the 8 questionnaires and 10 independent assessor measures of severity.27 Specifically, enrolled participants showed a significantly larger improvement in global functioning, social anxiety and avoidance.

In 2003, Lincoln et al.28 calculated pre- to post-treatment effect sizes for subgroups characterized by exclusion criteria in a sample of 217 patients who received exposure therapy combined with cognitive restructuring treatment for SO. Common exclusion criteria included (1) comorbid depression (as measured by a BDI ⩾18), (2) prior psychological treatment for SO, (3) low symptom severity, (4) >50 or <20 years of age and (5) comorbid Axis I disorder. The effect size for the sample that excluded individuals with higher depression (effect size=0.76) was smaller than for the sample that included individuals with high levels of depression (effect size=1.28).

A second study by Lincoln et al.22 calculated effect sizes for the change in pre- to post-treatment symptom severity for 26 clinical trial investigations of CBT and exposure therapies for SO conducted between 1996 and 2002. They next categorized studies based on their usage of exclusion criteria and created averaged effect sizes for each category (studies were weighted with the square root of n). Exclusion criteria included in this investigation were (1) comorbid psychosis, substance misuse or bipolar disorder, (2) comorbid depression, (3) comorbid Axis 1 disorder, (4) comorbid antisocial personality disorder, (5) low symptom severity and (6) prior treatment. The difference in effect sizes between two groups of studies were considered meaningful if the weighted means differed by at least 0.3. The only meaningful difference was between those studies that excluded prior treatment (effect size=0.71) and those that allowed prior treatment (effect size=1.01). Although slightly below their threshold, the effect size increased from 0.77 to 0.94 when excluding comorbid psychosis, substance misuse or bipolar disorder.

In both studies by Lincoln et al.,22, 28 an accumulation of common exclusion criteria was not generally found to lead to higher effect sizes.

Discussion

Our review of exclusion criteria used in trials of anxiety disorders shows that restricting the study sample to a discrete homogenous diagnosis with limited comorbidities may exclude up to 92% of treatment seekers. As a consequence, existing trials may have limited applicability for guiding future RDoC-motivated trans-diagnostic studies of treatment mechanisms or for translation to real-world settings, where comorbidity is the norm rather than the exception. A step towards addressing these issues would be for authors to adopt a standardized way of reporting the rationale for all exclusion criteria and provide data on the likely translational impact of these criteria.

Multiple comorbidities

Comorbidity of anxiety disorders with each other and with mood disorders is the most immediate issue to consider in regard to the impact of exclusion criteria. Current comorbidity data tend to focus on pair-wise combinations of anxiety and mood disorders, and there are no hard numbers about the prevalence of multiple combinations of anxiety disorders. These numbers are important for guiding new approaches to defining trans-diagnostic samples and RDoC-motivated samples unfiltered by a discrete diagnosis. Our initial analysis of lifetime comorbidity survey data showed that at least 60% of participants with one anxiety disorder had one or more additional anxiety or depression diagnosis. Further we show that the pattern of lifetime comorbidity is not bi-directional. For example, 60% of those with PTSD also had a diagnosis of MDD, however, only 20% of those diagnosed with MDD additionally had a PTSD diagnosis. Similarly, when considering comorbidity triplets 26% of those with a lifetime diagnosis of PTSD also had a diagnosis of MDD and social phobia, where only 9% of those with MDD had an additional PTSD and social phobia diagnosis. These patterns suggest that use of exclusion criteria may be disproportionally impacting certain anxiety diagnoses, such as PTSD, more than others.

The impact of exclusion for comorbidity

Given that the same treatments are currently used for the spectrum of anxiety disorders, the rationale for excluding participants due to a second or third additional anxiety disorder and/or MDD is unclear. Studies that followed the excluded individuals found mixed pattern of effects, from no differences to both better and worse outcomes. Thus, the impact of additional anxiety disorders is unlikely to consistently produce heterogeneity that confounds the results or reduces statistical power. Pragmatically, recruitment periods might be shortened by the opportunity to enroll all treatment seekers meeting anxiety disorder criteria for one or more disorders. Our findings suggest that in future anxiety trials, a cross-cutting RDoC approach in which participants present all forms of anxiety disorder that occur within a real-world setting may be of value for informing clinical care. This would require a fresh approach to trial design to record heterogeneous clinical profiles that reflect population data1 rather than attempt to define a priori specific diagnostic boundaries.

Our review suggests that, because of their common co-occurence, excluding for multiple other anxiety and mood disorders in order to study a 'single' anxiety disorder has the greatest potential for impacting the interpretation of treatment outcomes. Other commonly applied exclusion criteria do not necessarily have a big impact, feasibly reflecting a lower frequency of comorbidity. Of the studies we reviewed, the most common exclusion criterion was psychosis. However, this criterion typically rules out a comparatively small number of potential participants (2.8–21.2%), reflecting a lower frequency of those with anxiety who also have psychosis and may therefore have less impact on results than other criteria. In some studies, it might also be relevant to include psychosis as a comorbid clinical feature relevant to the treatment of interest. For example, patients with severe comorbid disorders such as psychosis may still benefit from psychotherapy treatments such as CBT used to treat PTSD (see refs 17, 30). In contrast, depression comorbidity produced the highest rates of exclusion, even though these rates were also variable (8.7–70.6%).

The need for standardized reporting of exclusion criteria

Overall, our review highlighted the paucity of evidence concerning the nature and impact of exclusion criteria for anxiety disorder treatment studies. Because significant variation exists in the specific set of exclusion criteria used in each trial, the generalizability of the results is limited. Variation in the number and type of exclusion criteria used, and in specific cutoff scores on psychometric scales, can also make it difficult to compare the results of studies of the same treatment. Exclusion criteria tend to appear as if they are 'cut and pasted' into the methods section, and may not be systematically reported or may even be underreported.31, 32 Our findings encourage the systematic reporting of the rationale used to guide the choice of each exclusion criteria and the implications of the choice. A recommendation based on the current review is that for all exclusion criteria, manuscripts should report the criteria in a specific manner for it to be replicable, present the rationale for each criterion and provide data on its impact. The justification and rationale could include information on the choice of each criterion and focus on criteria that reflect the clinical purpose of the study (for example, to gather data relevant to routine clinical care, impose only exclusion criteria that are medically necessary or that a reasonable clinician in practice would impose). To consider the impact of exclusion criteria, authors might include data on the proportion of subjects ruled out by the criterion and whether anything is known about disproportionate impact of the criterion (for example, disproportionately excluded female or racial/ethnic minority subjects). A first step towards achieving such standardized reporting of exclusion criteria could be for authors to use the CONSORT guidelines, which require a thorough reporting of exclusion criteria for each trial, and for journals to adopt these guidelines.

Limitations of the current state of knowledge

Due to the dearth of current evidence (that is, only 13 reviews/meta-analyses to date), the current state of knowledge is not based on a systematic study of the topic. Specific limitations of the existing knowledge base include the lack of clarity regarding screening methods and the lack of information about comorbidity.16 Of course, these limitations reflect the inherent limitations of clinical trials that have tended to not include the details about exclusion criteria (or the rationale for exclusion). As a result, reviews of exclusion criteria have focused on a limited subset of anxiety disorders, the description of exclusion criteria is commonly vague (for example, 'major mental illness') and there is a lack of consistency in criteria due to the patchy rate of review (eight reviews prior to 2004 and five since) spanning three revisions to the DSM.

To address these limitations, systematic protocols must be developed for studying the impact of exclusion criteria.