Main

Clopidogrel, in combination with aspirin, is a standard treatment for patients with acute coronary syndrome and is one of the top-selling drugs in the world.1,2 However, there is substantial interindividual variability in response. Polymorphisms of the cytochrome P450 (CYP) gene have been identified as a potential risk factor for nonresponse.3 This gene plays a central role in drug-metabolizing processes in the liver, and clopidogrel makes use of these processes to transform into an active metabolite capable of inhibiting platelet aggregation. Identification of CYP2C19 polymorphisms could lead to personalized treatment based on genotype in patients with acute coronary syndrome, and therefore, the US Food and Drug Administration recommends CYP2C19 genotyping for individualized antiplatelet management.4

The association between CYP2C19 loss-of-function alleles and clinical efficacy of clopidogrel has been studied extensively, and several meta-analyses have summarized the results of those studies.5,6,7,8 Systematic reviews and meta-analyses are often considered the highest level of evidence,9,10,11 and their popularity in the cardiovascular field increased almost 13 times as fast as the increase in number of published randomized clinical trials.12 However, the interpretation of meta-analyses is confusing when the conclusions of overlapping meta-analyses are discordant. Some meta-analyses on CYP2C19 loss-of-function and clinical efficacy of clopidogrel conclude that the association is proven,7,8 whereas others conclude the opposite.5,6

Our objective was to systematically evaluate the discordant evidence for the association between CYP2C19 loss-of-function alleles and clinical efficacy of clopidogrel through a critical methodological appraisal of published meta-analyses.

Materials and Methods

Our review adheres to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement,13 and we consulted the Cochrane Handbook for Systematic Reviews of Interventions for methodological aspects of meta-analyses.14

Literature search and study selection

A MEDLINE search was performed in August 2013 using a combination of search terms: (clopidogrel OR Plavix OR Iscover OR thienop* OR P2Y12) AND (cytochrome OR cyp OR polymorph* OR genetic*) AND (review OR meta-analysis). The Cochrane Database of Systematic Reviews, The Web of Knowledge, EMBASE, and reference lists of retrieved systematic reviews were inspected for additional studies.

Two reviewers (R.L.O. and S.J.H.) independently assessed all titles and abstracts for eligibility and examined all full-text articles to find meta-analyses of CYP2C19 polymorphisms and clinical outcomes in clopidogrel users. Disagreements were resolved by discussion.

Data extraction

General, clinical, and methodological characteristics of each meta-analysis were extracted by two researchers working independently (R.L.O. and A.C.J.W.J.). General characteristics consisted of first author, year of publication, and the date (month/year) that the authors had performed their systematic search. Clinical characteristics comprised the population and outcome definitions, and methodological characteristics were the eligibility of abstracts, the number of included studies, the method of assessment of primary study quality, details of the statistical analysis (random/fixed-effects meta-analysis), and the main results with confidence intervals. The literal conclusion of the authors about the presence of an association was documented from the abstract and categorized as present or absent. When the literal conclusion was unclear or not reported in the abstract, the wording was obtained from the discussion in the main text.

We examined whether and how the meta-analyses had addressed between-study heterogeneity and publication bias. These methodological characteristics are important for grading the quality of the evidence in the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) and Assessment of multiple systematic reviews (AMSTAR) tools.15,16,17 Heterogeneity is an apparent difference between the results of the primary studies13,14 and may be present when study populations, interventions, outcomes, or methodologies differ across the studies. Heterogeneity is generally quantified by the I2 or Cochran’s Q-statistic. Values of <25% suggest little heterogeneity, 25–50% suggests moderate heterogeneity, and >50% means large heterogeneity.18 We extracted these metrics and documented the methods that the authors had used to examine the differences in the results. To evaluate heterogeneity, primary study characteristics were extracted, which included study design, follow-up duration, patient characteristics, and outcome definitions. Publication bias is the tendency by investigators, reviewers, and editors to submit or accept manuscripts for publication based on the direction or strength of the study findings.19 Tests that assess publication bias include funnel plots, Harbord–Egger test, and trim-and-fill analyses.20,21,22 We extracted the specific methods that the authors used and their conclusions on the presence of publication bias.

Results

Search results

The MEDLINE search yielded 347 articles from which 11 meta-analyses on the association between CYP2C19 loss-of-function alleles and clinical efficacy of clopidogrel were identified. The searches in other databases and reference lists yielded no additional meta-analyses. The 11 meta-analyses included a total of 30 primary studies, but not all studies were included in all meta-analyses.

Meta-analyses characteristics

The 11 meta-analyses were published within a time frame of 24 months, and their literature searches were performed between October 2009 and October 2011. All articles presented separate analyses for composite clinical end points and stent thrombosis (ST). The definitions of the composite clinical end point varied across meta-analyses but generally included death, myocardial infarction (MI), and stroke ( Table 1 ). ST was typically defined according to the Academic Research Consortium definitions.23 Five meta-analyses used a checklist to assess the quality of the primary studies,5,7,8,24,25 and three meta-analyses used their own approach6,26,27 or did not check the quality.28,29,30 The Newcastle–Ottawa scale is a formal quality-scoring list,31 and the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE)32 and its genetics extension STrengthening the REporting of Genetic Association studies (STREGA) also serve as reporting guidelines.33 All three checklists comprise methodological aspects of observational studies including patient selection, outcome assessment, and adequacy of follow-up.

Table 1 Characteristics of included meta-analyses, sorted by publication date
Full size table

The total number of included studies ranged from 7 to 27 for the clinical end point and from 4 to 14 for ST ( Table 1 ; Supplementary Tables S1 and S2; Supplementary Appendix S1 online). As expected, more recent meta-analyses generally included more primary studies. The percentage of all studies that were included in the meta-analysis, calculated as the percentage of all available studies at the date of the study selection, ranged from 25 to 90% for the clinical end point and from 9 to 82% for ST (Supplementary Tables S1 and S2; Supplementary Appendix S1 online). Five meta-analyses did not include conference abstracts,5,25,28,29,30 and all but one of the meta-analyses left out data from one or more full articles that were included in other meta-analyses (Supplementary Table S1 and Supplementary Appendix S1 online).34,35,36,37,38,39,40,41 For example, the post hoc genetic analysis of the ACTIVE-A trial was included in only two of the seven meta-analyses that did their literature searches after the publication of the trial,6,24 and one meta-analysis had limited the inclusion to only primary studies with a follow-up time of 6–12 months.30

Primary study characteristics

The characteristics of the 30 primary studies varied (Supplementary Table S3 and Supplementary Appendix S1 online). Follow-up duration ranged from 1 to 43 months, and there were 23 cohort studies and 7 post hoc analyses of randomized trials. The mean age ranged from 60 to 70 years in most studies, with the exception of one study that involved patients with a mean age of 40 years.42 The percentage of patients who had undergone percutaneous coronary intervention varied from 0 to 100%, and the proportion of smokers ranged from 8 to 56%, with three studies including more than 50% of smokers.42,43,44

All 30 studies addressed clinical end points, and 17 studies also investigated ST. The definition of the composite clinical outcome and the event rates varied between primary studies. MI was most commonly included in the composite clinical end point (24 studies), followed by cardiovascular death (18 studies), stroke (17 studies), and all-cause death (10 studies).

Outcomes and conclusions of the meta-analyses

Eight out of 11 meta-analyses on clinical end points reported a statistically significant association ( Table 2 ; Supplementary Table S4 and Supplementary Appendix S1 online).6,7,8,24,26–29 Mean effect sizes of the significant association (random-effects models) ranged from 1.26 to 1.96. Five of these eight concluded that there was an association between CYP2C19 loss-of-function alleles and the clinical end point,24,26,27,28,29 two inferred that there was a possible association,7,8 whereas one concluded that the association was not proven because of publication bias ( Table 2 ; Supplementary Table S4 and Supplementary Appendix S1 online).6 The remaining three meta-analyses found no statistically significant pooled effect5,25 or did not pool the data because of between-study heterogeneity.30 Of the four meta-analyses that concluded absence of association, three were among the five most recently published.5,6,30

Table 2 Main outcomes for clinical end points and stent thrombosis
Full size table

For ST, all 11 meta-analyses reported a statistically significant association with CYP2C19 loss-of-function alleles.5,6,7,8,24–30 Mean effect sizes (fixedeffects models) ranged from 1.77 to 3.82. One meta-analysis concluded that there was a possible association,6 and one other meta-analysis observed the presence of heterogeneity and publication bias and downgraded the evidence ( Table 2 ; Supplementary Table S4 and Supplementary Appendix S1 online).5

Heterogeneity

Heterogeneity assessment of each meta-analysis is presented in Table 3 . All meta-analyses reported significant heterogeneity between the primary studies for the clinical end points, but they handled and interpreted the presence of heterogeneity in different ways. Six meta-analyses reduced heterogeneity by excluding one or more studies,5,8,24,25,26,29 and four of these found that the resulting pooled effect estimate remained unchanged.5,8,24,26 Five meta-analyses performed stratified meta-analyses by study and population characteristics,7,8,25,26,29 two performed meta-regression,5,20 and one inspected primary study characteristics.5 Higher sample size and poorer quality of the primary studies were associated with lower effect sizes,7,25 but other studies did not find an impact of study characteristics.5,6 For ST, four meta-analyses observed moderate heterogeneity, with I2 ranging from 32 to 44%,5,6,24,27 but the degree of between-study heterogeneity was not statistically significant in any of the 11 meta-analyses. One meta-analysis discussed that exclusion of studies reduced heterogeneity but did not justify the exclusion on clinical grounds.5

Table 3 Analyses and reporting of between-study heterogeneity in the meta-analyses
Full size table

Bias assessment

The assessment of publication bias is presented in Table 4 . All but two meta-analyses investigated publication bias for clinical end points.8,28 Nine meta-analyses used funnel plots to explore publication bias,5,6,7,24–26,29 of which five applied additional analyses such as Harbord–Egger test, stratification analysis, and trim-and-fill analysis.5,6,7,26,27 Six of the nine meta-analyses that assessed publication bias concluded that there was at least some evidence for bias due to selective missing studies,5,6,24,25,26,29 two did not find evidence for bias,7,27 and in one meta-analysis, the results of the funnel plot were not discussed.30 For ST, only 3 of the 11 meta-analyses reported the analyses of publication bias and concluded that presence of bias could not be ruled out.5,7,27 One of these meta-analyses concluded that the epidemiological credibility for an association of CYP2C19 loss-of-function alleles with ST with was weak, due to publication bias.5

Table 4 Analyses and reporting of publication bias in the meta-analyses
Full size table

Discussion

This review of 11 meta-analyses on the association between CYP2C19 loss-of-function alleles and clinical efficacy of clopidogrel shows that results and conclusions of 11 overlapping meta-analyses were discordant. Effect sizes differed because some meta-analyses did not include data from conference abstracts and more recent meta-analyses included more primary studies. Yet, conclusions predominantly differed because between-study heterogeneity and bias were handled differently across meta-analyses.

Overall, the meta-analyses consistently showed a larger effect of CYP2C19 loss-of-function alleles on ST as compared with the effect on the clinical end point. ST is associated with higher risk of experiencing MI and death, both components of the composite clinical end point.45 The absence of consistent association with clinical end points might indicate that the alleles have no impact on distant outcomes such as MI and death and that the significant effect of ST is diluted when combining the various end points. Liu et al. had performed meta-analyses on the separate clinical end points, such as bleeding complications, MI, stroke, and death, and found that effect sizes ranged from 0.99 to 1.55, with the exception of 2.37 for stroke. None of these effect sizes were statistically significant, and most analyses showed substantial between-study heterogeneity.24 This suggests that CYP2C19 impacts ST, and maybe stroke, but that its effect may not be strong enough to also influence the more distant outcomes.

All meta-analyses on the clinical end points observed substantial between-study heterogeneity. The heterogeneity was most likely explained by differences between study populations and primary outcome measures (Supplementary Table S3 and Supplementary Appendix S1 online). For example, the mean age of the participants in the study by Collet et al.42 was 40 years, as compared with 60 to 68 years in all the other studies. In all studies, the percentage of smokers ranged from 8 to 56%, the percentage of men from 55 to 92%, and the percentage of participants who underwent percutaneous coronary intervention from 0 to 100%. The effect of clopidogrel response should preferably be in a more homogeneous subgroup because the influence of age, gender, and smoking status on CYP enzyme activity cannot be neglected.46,47,48 The study by Collet et al. had the highest effect size of all studies (5.4; 95% confidence interval: 2.3–12.5), and its exclusion from meta-analyses would have been justified based on its incomparable study characteristics.49 Also from a clinical perspective, exclusion of the study by Collet et al. is warranted: a decision about genotyping 60- to 70-year-old individuals should not be affected by a threefold higher effect size in young adults.

The substantial heterogeneity between the primary studies was handled differently across the meta-analyses. Several meta-analyses excluded studies with extreme effect sizes (outliers) one by one to reduce heterogeneity. The removal of studies for the mere fact of introducing heterogeneity is however unjustifiable because, by definition, heterogeneity is introduced by the studies with the most extreme effects and never by studies that have effect sizes similar to the pooled estimate. Moreover, only very large studies with extreme effects may lead to a change in the pooled estimate after exclusion. Exclusion of smaller studies easily reduces heterogeneity without changing the pooled effect. Yet exclusion of studies should not be based a posteriori on the effect sizes but a priori on the basis of patient or study characteristics, preferably by prespecified subgroup analysis that details and motivates the exclusion of specific studies.14,50,51 The presence of substantial unexplained heterogeneity should be a major factor in the interpretation of the evidence and a good reason to refrain from drawing conclusions based on the quantitative results.14

Another factor that affected the interpretation of the relationship between CYP2C19 variants and clinical efficacy of clopidogrel was the presence of bias. Meta-analyses are subject to various biases including small-study bias and publication bias.13,52 Small-study bias is present when the smaller studies show stronger effects than the larger studies, and publication bias might occur when studies with statistically significant results are more likely to be published than those with nonsignificant results.22 For the clinical end point, 6 of 11 meta-analyses found evidence for publication bias, and the 3 meta-analyses that investigated publication bias for the ST end point found that the results were biased due to missing studies.3,5,27 These bias checks suggest that the association of CYP2C19 loss-of-function alleles and clinical efficacy of clopidogrel is affected by missing studies, studies that would have shown smaller effects or even effects in the opposite direction.

Despite the substantial heterogeneity in the meta-analyses of the clinical end point and the suggestions of publication bias in both the clinical end point and ST, most meta-analyses did not seem to consider these data as problems in their conclusions. Only 1 of the 11 meta-analyses on the clinical end point explicitly refrained from quantitative analysis because of heterogeneity,30 one downgraded the statistically significant association to evidence of no association,6 and two others phrased their conclusions of associations cautiously.7,8 Yet, five out of eight meta-analyses that observed statistically significant effects concluded that the CYP2C19 genotype was associated to the clinical end point despite the presence of substantial between-study heterogeneity.24,26,27,28,29 For ST, 8 out of 11 meta-analyses did not report about the assessment of bias,6,8,24,25,26,28,29,30 whereas the 3 meta-analyses that did such assessment found evidence for publication bias. Inspections of heterogeneity and bias are integral parts of meta-analyses, and their results should impact the conclusions of the quantitative analysis.13,15,16,17

The latest meta-analysis was the only one that performed a separate effect-modification analysis using data from four randomized trials.6 There was no evidence for genotype–treatment interaction in clopidogrel users for the composite cardiovascular outcome. These results are in line with our conclusion and further substantiate that there is no evidence that CYP2C19 loss-of-function alleles are associated with worse outcomes in clopidogrel users.

Evaluating discordant meta-analyses

The popularity of meta-analyses in the cardiovascular field has increased by almost 1,800% over the past 20 years, whereas the number of randomized controlled trials only increased by 140% over the same time period.12 This practice leads to increasing duplication of meta-analyses on the same topic.53 A recent study showed that 67% of the reviewed meta-analyses had at least one overlapping and 5% of the topics were studied in at least eight overlapping meta-analyses.11 Although overlapping meta-analyses may seem unnecessary, our review shows that authors make different choices in their conduct. Because there is no single best way to define inclusion and exclusion criteria, to define and select study populations and end points, and to interpret heterogeneity and bias in light of the results, the variety in meta-analyses might be as wide ranging as that in primary studies. When the variety in meta-analyses results from informed choices about the definitions, selection criteria, and analyses, this variety reflects paradigms in the field and should not be seen as duplication. Greater awareness and understanding of the subjectivity of meta-analyses and the impact of methodological choices on their results will enhance the appreciation of meta-analyses as high level of evidence.12

In 2010, the US Food and Drug Administration recommended CYP2C19 genotyping for individualized antiplatelet management.4,6 Based on a reevaluation of then-current and later meta-analyses, we conclude that this recommendation is currently not evidence based. The GRAVITAS, ARCTIC, and TRILOGY-ACS trials have shown that bedside platelet reactivity testing did not result in clinical benefits,54,55,56 but recently, a trial randomized patients undergoing percutaneous coronary intervention to either point-of-care genotyping and subsequent personalized treatment or standard clopidogrel treatment.57 The point-of-care genotyping strategy showed significantly lower on-treatment platelet reactivity in CYP2C19 loss-of-function carriers than the standard treatment strategy. However, the genetic substudy of the ARCTIC trial showed no benefits of genotyping.58 In the absence of evidence that CYP2C19 loss-of-function alleles truly affect outcomes, it will be interesting to see whether larger future trials of personalized treatment based on platelet monitoring and genotyping can show improvement in clinical end points.

Conclusion

The current study systematically evaluated overlapping discordant meta-analyses on the same topic. The results and conclusions of 11 overlapping meta-analyses on the association between CYP2C19 loss-of-function alleles and clinical efficacy of clopidogrel are discordant. Effect sizes differed because some meta-analyses did not include data from conference abstracts and more recent meta-analyses included more primary studies. Yet, conclusions predominantly differed because between-study heterogeneity and bias were handled differently across meta-analyses. Confidence in the presence of an association is limited, and personalized antiplatelet management based on genotyping is not supported by the currently available evidence.

Disclosure

A.P.K. is a member of the RE-ALIGN trial steering committee. A.C.J.W.J. was supported by the Vidi grant from the Netherlands Organization for Scientific Research and by the Center for Medical Systems Biology within the framework of the Netherlands Genomics Initiative. J.M.t.B. has specified relationships with Bristol-Myers Squibb, Eli Lilly, GlaxoSmithKline, Merck, Sanofi-Aventis, and Schering-Plough. Other authors declare no conflict of interest.