Clinical utility of genetic and genomic services: context matters

Should diagnoses, and corresponding changes in disease management, be sufficient demonstration of clinical utility, even in the absence of evidence for improved clinical outcomes? This question is posed to the health-care payer community in a recent American College of Medical Genetics and Genomics (ACMG) position statement on the clinical utility of genetic and genomic services.¹ Affirmative arguments could be drawn from examples of individually rare, highly penetrant, single-gene disorders. We fully support the ACMG’s call for inclusion of individual, familial, and societal levels of impact in the evaluation of testing. Nevertheless, broadening the definition of clinical utility for all cases may be less helpful in the evaluation of genetic tests than promoting more context-dependent and transparent decision-making, with less rigidity and dogmatic adherence to artificial logic models.

Clinical Utility

In the statement, the ACMG reports that “coverage decision-making policy is now driven by a narrowed perspective that clinical benefit accrues only to the individual receiving the services,” so neither etiological diagnosis nor changes in treatment that lack corresponding proven health-outcome benefit qualify as demonstration of clinical utility.¹ Furthermore, familial and societal level benefits are ignored.¹ To exemplify a narrow view of clinical utility, the ACMG¹ cites the MolDX Clinical Test Evaluation Process,² which is based on the Analytic Validity, Clinical Validity, Clinical Utility, Ethical, Legal, Social Implications (ACCE) model process.³ Although ACCE was not the first to characterize clinical utility in terms of health outcomes, it established a nested model in which clinical utility encompasses, and adds to, all other components assessed.

In addition to MolDX, groups such as the Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Working Group (EWG),⁴ the Secretary’s Discretionary Advisory Committee on Heritable Disorders of Newborns and Children,⁵ and the United Kingdom Genetic Testing Network⁶ have drawn on their own interpretations of ACCE, along with other sources, in developing methods for evaluation. Although the EWG used ACCE as an aid in organizing and better understanding questions for evaluation, they also leveraged the inherent flexibility of the model to create and customize their methods and outcome definitions.^4,7

The Centers for Disease Control and Prevention’s Office of Public Health Genomics founded the EGAPP initiative in 2004 and launched the EWG as an independent panel in 2005. As early as 2009, published EGAPP methods suggested consideration of information that may be helpful in personal decision making and for ending diagnostic odysseys in evaluating the clinical utility of tests used in clinical diagnostic scenarios.⁴ The following year, the EWG published its framework for considering relevant outcomes in four categories of impact: (i) diagnostic thinking/health information, (ii) therapeutic choice, (iii) patient outcome, and (iv) familial/societal.⁸ Likewise, the methods of the Secretary’s Discretionary Advisory Committee on Heritable Disorders of Newborns and Children stipulate that, in addition to reductions in morbidity and mortality, “Broader benefits to the individual infant, such as nonclinical interventions or benefits to family and community, such as avoiding a diagnostic odyssey or informing nonmedical decision making, may also be considered” in assessing clinical utility.⁵

Using Contextual and other Key Factors

To broaden the definition of clinical utility in a meaningful way, both benefits and potential harms at the individual, familial, and societal levels would need to be assessed. The weighting of these types of broader outcomes is often not amenable to quantitative description or analysis. Instead, these elements may sometimes be presented qualitatively, as important considerations or contextual factors. Nevertheless, the potential weight of familial and societal impact by the EWG was demonstrated in its 2009 recommendation that all patients with newly diagnosed colorectal cancer be tested for Lynch syndrome, to benefit family members through cascade screening.⁸ Potential benefits to family members include clear, “hard” outcomes of reduced morbidity and mortality, whereas individual-level benefits to patients are not as discernible (at least, based on the evidence considered). With recommended testing of individual patients as a means to improve outcomes in relatives, it follows that familial and societal level values would necessarily have been influential factors. The EWG reported the following as key contextual issues: consideration of potential for distress and related psychosocial outcomes following testing, intervention uptake and surveillance rates among relatives, and limitations in patient-level benefit; the last issue has been reported to have directly influenced the group’s recommendation for informed consent antecedent to immunohistochemistry and microsatellite instability testing.⁸

This recommendation represents a particularly complex clinical scenario, and it is noteworthy that EGAPP methods allow for different approaches to assessing clinical utility based on how the test is to be used (i.e., diagnostic, screening, prognostic, risk assessment, or pharmacogenomics scenarios).⁴ While testing would be diagnostic of Lynch syndrome for patients, for family members testing the colorectal cancer (CRC) patient would fall under the category of risk assessment. If the clinical scenario had involved screening for Lynch syndrome among people with a family history of CRC or diagnostic application of testing to the patient alone, then a different recommendation might result. Consideration of how testing is to be applied can be an important determinant of how clinical utility is assessed and therefore whether an actionable recommendation can be made.

Going Beyond Clinical Utility

Elements of clinical utility in the ACCE model extend beyond clinical effectiveness of interventions.³ The EWG has acknowledged viewing many ACCE elements pertaining to testing implementation (such as availability of appropriate facilities and educational materials) as “information that should not be included in the consideration of clinical utility, but may be considered as contextual factors in developing recommendation statements.”⁴ Assessing clinical utility in its Lynch syndrome recommendation, the EWG created a chain of indirect evidence consisting of assessments of studies showing alteration of patient management (considering both proband and family members) and the influence these changes had on outcomes; they found only “limited but promising evidence suggesting that testing can improve outcomes.”⁷ Together, these findings suggest that, although the EWG may take a more restrictive view of clinical utility than described in the ACCE model, in some respects the group takes a broader view on contextual issues, and the overall weighting of these issues can influence the outcome of recommendations. Rather than considering expansion of the definition of clinical utility, it may be more practicable to ask whether interventions whose clinical utility is not backed by adequately powered studies directly assessing health outcomes can be recommended through evidence-based evaluation processes. We believe that the example of the EGAPP Lynch syndrome recommendation suggests that the answer is—sometimes—yes. Practice associated with rare disorders, such as many inborn errors of metabolism, has also begun to innovate in addressing this question. There are frameworks incorporating more practice-based evidence in evaluating interventions at the levels of both health care systems and patients, to address shortcomings in more traditional evidence-based medicine in this area.⁹

Widening the scope of contextual and other issues considered in decision making, beyond the strict confines of clinical context, is an idea that may already be gaining traction. This type of approach could include, for example, utility of information in its own right and factors important in local context, such as inequities that increase variability in follow-up services. Reporting on exactly how such factors are considered is critical and can be more challenging than reporting more quantitatively assessable outcomes. For example, to arrive at the EWG Lynch syndrome recommendation, evaluators would have needed to take into account the fact that the mechanics of cascade screening remain controversial in terms of relative autonomy, privacy, and other issues at familial and societal levels, with different approaches in screening-program management potentially affecting the magnitude of potential harms.¹⁰ From the EGAPP recommendation, however, it is not possible to ascertain the influence of each of these variables on the ultimate decisions that were made. For the intended purpose of guiding practice, it may be sufficient to state that such factors were considered. For the purposes of informing future research and guideline development by other groups, more detail on the specific contextual variables considered, along with estimated ranges of potential and actual influence on decisions attributable to each of these factors, would be useful.

Instead of broadening definitions, we believe that it would be more helpful for guidelines to consistently include and describe consideration of contextual and other relevant issues, extending beyond clinical outcomes to factors such as the value of information in planning and in preventing additional unnecessary testing. Guidelines should be more specific in reporting the extent to which these kinds of issues are allowed to affect final decisions. Without this type of transparent reporting on individual decisions, allowing readers to see the sometimes messy application of formal methods, we have no reliable basis on which to determine the degree to which contextual factors influence results and a poorer ability to understand nuances that may be critical in implementing testing strategies. Professional societies should promote the rigorous, evidence-based evaluation of health technologies that are within their scope of influence while understanding that the results may not tell the complete story. In cases in which contextual and other relevant issues can be convincingly argued to outweigh clinical outcome–based demonstration of clinical utility, recommendations in favor of testing should be acceptable to proponents of evidence-based medicine.

Disclosure

All authors are employed by CDC Office of Public Health Genomics, which maintains the EGAPP initiative and provides support for the independent EGAPP Working Group.