CLINICAL CONSIDERATIONS

Definitions used by Evaluation of Genomic Applications in Practice and Prevention

  • Analytic validity refers to a test's ability to accurately and reliably measure the genotype or analyte of interest.

  • Clinical validity defines the ability of the test to accurately and reliably identify or predict the intermediate or final outcomes of interest. This is usually reported as clinical sensitivity and specificity.

  • Clinical utility defines the balance of benefits and harms associated with using the test in practice, including improvement in measurable clinical outcomes and added value in clinical management and decision making compared with not using the test.

Patient population under consideration

These recommendations apply to the general population of adults without known preexisting cardiovascular disease (CVD), regardless of family history.

Considerations for practice

These tests have become available through primary care clinician offices and through direct to consumer marketing. Patients may ask about such tests or bring results of completed tests to their physicians for advice or consultation. Physicians should routinely consider well-established recommendations for cardiac risk assessment in the primary care setting (e.g., smoking, blood pressure, and lipid screening). In addition, all patients should be consistently counseled regarding appropriate physical activity and nutrition behaviors to reduce cardiac risk. Based on the available evidence, it is unclear how the results of genomic profiling should modify patient care to improve outcomes.

BACKGROUND AND CLINICAL CONTEXT FOR THE RECOMMENDATION

CVD is a major contributor to morbidity and mortality in the United States. An estimated 80 million adults have one or more types of CVD (48% at 60 years of age or older), and preliminary 2006 mortality data indicate that CVD accounts for 1 in every 2.9 deaths.1 The 2005 overall death rate from CVD was 279/100,000, with death rates higher in men than women, and in blacks than whites. Consequently, the burden of CVD is high, and the cost (direct and indirect) in 2008 is estimated at 448.5 billion dollars.2 Prevention and management of CVD, particularly ischemic heart disease and stroke, present a difficult challenge for health care and public health.3 Major nonmodifiable risk factors include increasing age, male gender, and heredity, whereas modifiable risk factors include smoking, hypertension, dyslipidemia, obesity, physical inactivity, and diabetes.46 African Americans have more severe hypertension and increased risk of heart disease compared with whites.7 In men, the average annual rate of initial cardiovascular events increases from 3/1000 at 35–44 years to 74/1000 at 85–94 years. Similar increases occur in women but approximately a decade later in life.2

Descriptions of tests and intended use claims

Seven companies offering eight genomic tests for CVD/“heart health” were identified in February 2008. The test panels included between 1 and 12 genes, with a total of 29 different genes (Table 1) included on one or more panels. This review is restricted to these tests. Some tests were offered direct to consumers, with reports detailing “diet and lifestyle recommendations personalized to the individual tested.” Others required the test to be ordered by a physician and included the collection of detailed health information (e.g., lipid levels, family history, height, and weight). One of the companies reported the individuals' 10-year risk of having a cardiovascular event (using the Framingham risk multiplied by the odds ratio [OR] based on the genetic test results). The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Working Group (EWG) commissioned an evidence review to address an overarching question regarding the following specific clinical scenario:

What is the direct evidence that genomic profiling in the general population of men and women without known preexisting CVD leads to improvement in cardiovascular health?

REVIEW OF SCIENTIFIC EVIDENCE

This statement summarizes the supporting scientific evidence used by the EWG to make recommendations regarding the use of genomic profiling in the general population of adults.

Methods

EGAPP is a project developed by the Office of Public Health Genomics at the CDC to support a rigorous, evidence-based process for evaluating genetic tests and other genomic applications that are in transition from research to clinical and public health practice in the United States.8 The EWG commissioned evidence review was contracted by the Office of Public Health Genomics and performed by a collaboration of external consultants and four EGAPP staff members. A Technical Expert Panel that included four EWG members and two additional consultants provided expert guidance during the course of the review. Two main groups of outcomes were defined based on the International Statistical Classification of Diseases and Related Health Problems (I00–I99).9 The first group, coronary heart disease (CHD), includes coronary artery disease, ischemic heart disease, and myocardial infarction. The second group, stroke, includes intracerebral and subarachnoid hemorrhage, ischemic stroke, and other diseases (e.g., cerebral infarction and occlusion/stenosis of cerebral arteries). In addition to the larger review, a peer-reviewed article on 9p21 has been published with detailed information available.10 The final EWG recommendation statement was formulated based on magnitude of effect, certainty of evidence, and consideration of contextual factors.8

Technology description

In general, genotyping methods have involved discrimination of alleles by primer extension, hybridization, ligation or enzymatic cleavage, and detection using fluorescence, mass, gel electrophoresis, or chemiluminescence. Mistaken alleles, allelic dropout (i.e., amplification of only one of two alleles in a heterozygous individual), and other genotyping errors can result from a number of causes. These have included interaction with flanking DNA sequences, low quality/quantity of the DNA in samples, laboratory problems related to reagents/protocols/equipment, and human error (e.g., sample mislabeling or contamination, data entry, and interpretation mistakes). Less is known about causes of genotyping errors in newer technologies (e.g., multiplex assays, chips, and SNP arrays) used in routine clinical practice and their potential impact on patient results.

Analytic validity

For this review, analytic validity can be defined in terms of the identification of a specific gene variant. Eight genomic panels were identified that included 29 genes and 58 variants. The following conclusions concerning analytic validity were made:

  • In many instances, insufficient information was provided to identify the specific variant tested within a specific gene. In addition, no published literature was found on the testing platforms used by the laboratories offering the test.

  • Two of the genes included (F5 and F2) are tested for in other clinical settings, and information from external proficiency testing (Level 1 data) indicates that the variants in these genes can be reliably identified by available testing methodologies.

  • External proficiency testing was not available for the other 27 genes included on these genomic panels.

  • Two of the companies offering genomic panels provided in-house data on the analytic methodology used and estimates of analytic performance.

    • deCODE Genetics laboratory (Reykjavik, Iceland) is CLIA licensed and College of American Pathologists accredited. Representatives provided information about the platform and methodology used (including a methodology publication), test results compared with bidirectional sequencing, replication (short term and long term), and reported using blinded samples for internal quality assurance.

    • Interleukin Genetics laboratory (Waltham, MA) is licensed by CLIA and four states. Representatives provided information about the platform and methodology, test results compared with commercial cell lines, short-term replication, and failure rates.

Analytic validity conclusions

Testing for variants in two genes (F2 and F5) can be done reliably with currently available technologies. In-house data for analytic validity from two companies were encouraging but graded as Level 4 evidence. There is inadequate evidence that the genomic profiling tests identified in this report have analytic validity, but platforms exist that could allow at least satisfactory sensitivity and specificity.

Clinical validity

In this context, clinical validity, expressed as ORs, assesses how effectively the at-risk variants of the genes predict CVD risk; heart disease and stroke were examined as separate outcomes. In addition to individual ORs for each gene/variant/disease association, a “best-case scenario” model of the combination of markers with the strongest evidence was created for each of these two major outcomes. Findings include:

  • For heart disease, the quality of evidence for clinical validity varies widely among the 29 genes (58 genes/variants).

    • The most credible evidence of a gene-disease association is for the 9p21 SNP markers and heart disease (but not stroke). This association is highly reproducible, unlikely to be influenced by major biases, and has the largest effect size documented for any variant with at least moderate credibility (OR = 1.56).

    • The 23 other genes/variants have at least some credible evidence.

    • Several gene/variant combinations have associations based on only a few small, heterogeneous studies. These effect sizes are suspect due to important possible biases. Some of the strongest reported associations have weak credibility.

  • The cumulative effect (cumulative odds ratio), when displayed as a receiver operator characteristic curve, produced an area under the curve (AUC) of 64.7% for the 24 genes/variants associated with heart disease. An AUC of 100% is a perfect diagnostic test, whereas a value of 50% indicates the test provides no useful information. Using a cumulative odds ratio cutoff level of 1.38 results in a detection rate of 24% at a false-positive rate of 10%. Based on these performance characteristics, this combination of genomic markers would not be considered a useful stand-alone test for heart disease risk stratification.

  • Data for stroke are less convincing. The cumulative odds ratio for a model including 13 genes/variants results in an AUC of 55.2% and a detection rate of 14%.

  • Limited information is available on whether combining genomic markers with traditional risk factors (TRFs) improves prediction of heart disease. Improvement over TRF has only been evaluated for 9p21. We used the net reclassification index11 as a measure of how much improvement in risk prediction was gained by adding 9p21 marker assessment to TRF. Overall, the net reclassification indexes ranged from −0.2% (a nonsignificant decrease in prediction by adding 9p21 SNP information) to 4.9% (a clinically unimportant but statistically significant improvement in risk prediction).

Clinical validity conclusions

There is convincing evidence that 9p21 variants are associated with heart disease and that the improvement in prediction when added to TRFs is negligible. For heart disease or stroke, 24 other associations had adequate information; 34 had inadequate information. Modeling showed that combining the 24 most credible markers for heart disease would not provide a clinically useful stand-alone test. Modeling the 13 most credible markers for stroke is even less predictive.

Clinical utility

In the setting of adults without known CVD, clinical utility assesses the benefits and harms associated with using genomic profiling tests to estimate risk and guide management as ways of improving health-related outcomes. Benefits might include successfully motivating behavior changes and more appropriately treating patients whose CVD risk has been correctly reclassified. Harms might include false reassurance triggering negative or no behavior change in those incorrectly reclassified as being at low risk and unnecessary drug therapy for patients incorrectly reclassified as being at high risk. In summary:

  • No studies in the available literature assess the clinical utility of cardiogenomic profiles.

  • A systematic review found preliminary evidence that CHD risk scores may translate into modest benefits (e.g., increased drug treatment and short-term blood pressure reduction), without clinical harms. However, the need for higher quality evidence on long-term outcomes, and for replication of the results in different clinical settings, was emphasized.12,13

  • Some studies that assessed the clinical utility of genomic testing conducted in the fields of lung cancer and diabetes showed some short-term behavior change (e.g., adherence to risk-reducing behaviors).14,15

Clinical utility conclusions

There is inadequate evidence of clinical utility for genomic profiles.

Clinical studies

At least three observational clinical studies are actively recruiting and their results could affect this and future recommendations regarding CVD and genomic profiling. The following list is from www.clinicaltrials.gov:

  • Gene expression profiles in patients with permanent atrial fibrillation (AF) versus sinus rhythm conditions: The study will investigate the transcriptional profile of AF associated genes by oligonucleotide microarray methods and the gene expression profiles of patients with AF, when compared with controls.

  • The genetic basis of AF: The study will establish a DNA bank of 1000 individuals with AF and 1000 individuals without AF and directly test the hypothesis that known functional polymorphisms predispose individuals to AF.

  • Inflammation and acute coronary syndromes: The study will focus on major cardiovascular events and aims to discover novel genomic biomarkers of acute coronary syndrome in leukocyte subsets by means of analyzing gene expression profiles.

Contextual issues important to the recommendation

  • CVD is an important public health problem and improvements in outcomes associated with genomic testing could have important impacts.

  • The traditional modifiable CVD risk factors, such as those used in the Framingham Risk Scores, have an advantage in clinical screening and risk assessment strategies because they measure the actual targets for therapy (e.g., lipid levels and blood pressure).

  • It is important to recognize that there may be differences in the utility of genomic markers in predicting coronary risk compared with stroke risk. This is true for the TRFs used in Framingham, which has better utility for coronary artery disease than for stroke. Ultimately, it would be preferable to have research that evaluates and reports the utility for each condition separately.

  • To be useful, genomic testing should provide demonstrable improvement on the predictive value of TRFs.

  • The genetic mechanism of some candidate gene variants (e.g., 9p21) is unknown.

Cost-effectiveness

This review did not include any economic analyses.

Research gaps

The EGAPP Working Group found the research literature insufficient with important gaps in knowledge, including the following:

  • Little or no available information on the analytic validity of genomic panels, either in the published literature or on the company websites. Often, it was not possible to even determine the testing platform or assay methodology being used.

  • The specific genes and variant(s) that were included on the genomic panel(s).

  • Which of the gene/variant associations identified might benefit from further validation and/or analysis to improve their credibility.

  • How information gained from Genome Wide Association studies might be helpful in determining the effect size and credibility of existing gene/disease associations.

  • Which, if any, of the gene/disease associations identified with moderate or weak credibility might be overestimated due to potential biases (e.g., publication bias).

  • How multiple genomic markers for CVD should be combined and the types of data needed to inform these models.

  • What methodology should be used to determine the extent to which genomic (or nongenomic) markers add useful information to an existing risk model.

  • Alternative strategies for prevention of heart disease and how genomic markers might impact these strategies.

  • How genomic markers that modify the risk for CVD derived from TRFs will change the pattern of clinical practice.

  • Are there behavioral changes related to providing the results of genomic testing, and would these changes plausibly lead to improved health, and what factors might influence these changes (e.g., setting, method of delivery, and change in risk).

Recommendations of other groups

In 2007, the American Heart Association published a scientific statement on the relevance of genetics and genomics for prevention and treatment of CVD.16 That group stated “great potential is clearly within sight” but that “everyday utility remains just outside our grasp.” Overall, their recommendations focus on:

  • Continuing to use family history to identify susceptible individuals and families.

  • Developing a research infrastructure.

  • Prioritizing research agendas.

  • Preparing proactively for effective genetic screening programs.

  • Educating researchers, clinicians, public health professionals, and the general public.

  • Informing clinicians of the genetic tools at their disposal, and understanding and using the results of genetic screening for complex CVDs.

The American Heart Association has produced a Scientific Statement on the criteria for the evaluation of novel markers of cardiovascular risk.17 That statement recommends the following components of an adequate evaluation:

  • A sound research design.

  • A representative at-risk population.

  • An adequate number of outcome events.

  • The degree to which the novel markers add to the prognostic information provided by standard risk markers.

  • The inclusion of measures of both discrimination and accuracy.

  • The clinical value should be assessed by its effect on patient management and outcome.

In a recent clinical guideline,18 the US Preventive Services Task Force addressed the issue of using non-TRFs in CHD risk assessment. Those factors (e.g., high sensitivity C-reactive protein and carotid intima-media thickness) did not include genetic/genomic markers. The only nontraditional factor for which a net benefit could be identified was C-reactive protein in which 11% of men in the intermediate risk group would be reclassified as high risk, and 47.8 CHD events over 10 years/10,000 men aged 40–70 years could be prevented. This benefit was “felt to be of uncertain magnitude because of the lack of information on harms of testing and the unknown effect of intensive therapy on those who are defined as high-risk by virtue of CRP testing.” In general, the US Preventive Services Task Force found that “the current evidence is insufficient to assess the balance of benefits and harms of using the non-TRFs studies to screen asymptomatic men and women with no history of CHD to prevent CHD events.”