Main

Genetic services stakeholders—including patients and their families, guideline developers, medical geneticists, genetic counselors, and health insurers—must make decisions about which genetic tests should be offered in the clinical setting, which should be covered by insurance, at what level, and with what restrictions. These tests typically become available to patients and clinicians before their clinical utility is fully understood;1,2,3 thus, a key challenge in this decision-making process is assessing the available evidence on how genetic testing affects health outcomes.4 Patient-level studies of clinical utility may be lengthy and prohibitively expensive.5,6 Therefore, genetic services stakeholders must often make coverage decisions in the absence of complete and reliable evidence of clinical utility and under budget constraints. This will become increasingly true in the era of next-generation sequencing (NGS).

Decision-analytic modeling, a methodology developed in the decision sciences and used in health care by outcomes researchers and health economists, is potentially a useful tool in helping genetic services decision-makers with policy choices. It is a formal, quantitative synthesis of available clinical data relevant to particular health interventions, and is used to better understand the evidence base for comparing two or more courses of action, often in the absence of a comparative clinical trial.7,8 Models combine a variety of data sources, including clinical trials, observational studies, insurance claim databases, case registries, public health statistics, and preference surveys.9,10 Model results are prescriptive in that they provide an estimate of the differences in costs and outcomes among strategies, but also exploratory in that they can reveal the logical connections among inputs through sensitivity analyses. Decision-analytic models cannot substitute for missing clinical utility evidence, but they can clarify uncertainty caused by a paucity of evidence. In the absence of definitive real-world evidence for or against a particular choice, models can be tailored to account for a variety of contingencies identified by stakeholders.

The objective of this study was to gauge the opinions of genetic services stakeholders on the potential usefulness of decision-analytic modeling to inform their reimbursement decision-making. A secondary objective was to identify potential topics for future decision-analytic modeling studies to inform policy decisions regarding genetic services coverage.

Materials and Methods

Western States Genetic Services Collaborative (WSGSC) Reimbursement Work Group members and University of Washington researchers jointly planned a 1-day meeting in Seattle to identify regional challenges to Medicaid reimbursement of genetic services and develop plans to address challenges. WSGSC is a federally funded multistate cooperative agreement between the Health Resources and Services Administration Maternal and Child Health Bureau Genetic Services Branch and the Hawaii Department of Health; the agreement seeks to improve access to newborn screening and genetic services among its states and territory. The participating states and territory are Alaska, California, Guam, Hawaii, Idaho, Oregon, and Washington. Members include public health genetic services and newborn screening leaders, medical geneticists, genetic counselors, primary-care providers, and family representatives.

The study participants consisted of individuals selected by the WSGSC leadership to represent a purposive sample of state public health genetics leaders, professional family advocates who are employed in advocacy organizations, and state Medicaid executive leadership. Invitees who could not attend were asked to suggest a substitute. Participants included two Medicaid and 17 genetic services stakeholders from Alaska, California, Hawaii, Oregon, and Washington ( Table 1 ). Proceedings of the full meeting are available at http://www.westernstatesgenetics.org/Proj_activities.htm#2. A portion of the WSGSC meeting was allotted to our investigation, and we recruited the same meeting participants with an e-mail explaining the study we were planning to conduct, its objectives, and its voluntary nature. Our study received University of Washington and Hawaii Department of Health institutional review board approval, and informed consent was obtained.

Table 1 Summary of study stakeholder types
Full size table

We first presented a 30-min overview of decision-analytic modeling to give participants a basic understanding of the concepts, methods, and metrics involved (e.g., decision trees, data sources, sensitivity analyses, quality-adjusted life years). Second, we presented a warfarin pharmacogenetics case study, which used the inputs and results of a published decision-analytic model, to demonstrate the previously described concepts and methods.11 After our presentation, we conducted a 45-min group survey and concurrent discussion session facilitated by anonymous electronic voting on 10 Likert-style survey questions ( Table 2 ). At the end of our allotted time, the WSGSC leadership directed a consensus development process with meeting participants to identify and prioritize a list of genetic services to which decision-analytic modeling might best be applied. Participants suggested wide-ranging priorities before voting on their relative order of importance by show of hands.

Table 2 Questions and participant responses from electronic voting session
Full size table

The authors developed the survey questions based on previous experience presenting decision-analytic modeling to stakeholders but did not test them for validity or reliability. In addition to voting results, we took written notes of participants’ discussion after each question and used directed content analysis methods to qualitatively evaluate stakeholder responses.12 Drawing from previous research,4 we identified three key content area codes ( Table 2 ) to frame the survey questions and stakeholder feedback elicitation and used this scheme to evaluate our notes of the participants’ discussion.

Results

The results of the survey (N = 19) are presented in Table 2 and described below along with the results of the directed content analysis.

Uncertainty in genetic services decision-making

Participants expressed dissatisfaction with the state of evidence-based review for genetic services interventions. In discussion, participants cited aggressive marketing of genetic tests, an over-reliance on non-geneticist physicians’ clinical judgment, and uncertain significance of genetic association tests as barriers to evidence-based decision-making. Participants indicated budget constraints limit their ability to prioritize resources and services, found questions focusing on single-gene tests to be outdated, and acknowledged the need to address the role of microarray and NGS tests in clinical genetics.

Applicability of decision-analytic modeling

Participants were in favor of considering decision-analytic modeling results in evidence assessments of genetic services interventions. Approximately half of the participants currently use such modeling to assess efficacy and cost-effectiveness of genetic services. Among the positive impressions of decision-analytic modeling that the participants expressed were the transparency and explicitness of the methodology, the ability to test uncertainty and assumptions about data, and the usefulness of combining multiple layers of information to achieve a simplified result. Participants expressed concern that the validity of modeling results depends on the validity of data inputs and assumptions.

Acceptability of decision-analytic modeling

Some participants were uncertain about their comprehension of decision-analytic modeling techniques and potential results. Reasons for this included inconsistencies in methods and transparency among published decision-analytic modeling analyses, uncertainty about validity of quality-adjusted life years as a metric of health, and the difficulty of applying a single methodology to different interventions and outcomes. Participants believed costs should be incorporated into decision-analytic models for stakeholders who desire cost-effectiveness information.

Application of decision-analytic modeling to a genomic test

Participants were most interested in application of decision-analytic modeling to NGS and multiple-disorder testing platforms such as chromosomal microarray testing for autism, developmental delay, and cardiomyopathy, because they believed these technologies represent the future of genomics and will be the most difficult to evaluate. Although participants understood the challenges of using a traditional decision-analytic model to assess a multiplexed test such as NGS, they did express interest in approaching NGS assessment with decision-analytic modeling concepts. In addition, participants expressed a desire to see decision-analytic modeling applied to genetic services such as telehealth, use of out-of-state laboratories for genetic testing services, and genetic counseling services to balance differential state coverage of genetic counseling.

Discussion

The goal of this study was to assess the perspectives of genetic services stakeholders on the use of decision-analytic modeling to inform their genetic testing reimbursement decisions. Overall, the stakeholders expressed a desire for decision tools for genetic services, moderated by an understanding and concern about the limitations of decision-analytic modeling. Although stakeholders were receptive to using decision-analytic models for single-gene disorders, they indicated multiplex genetic tests were the biggest reimbursement policy challenges they currently face.

Stakeholders’ responses suggest that they work in a challenging environment and are wary of the market availability of genetic tests that lack substantial evidence for clinical utility. They also indicated budget constraints limit their ability to prioritize resources and services, and they struggle with making the “correct” choice when forced to choose one intervention to cover versus another. The finding that 63% of participants indicated that a test that benefits a single patient should not necessarily mean it should be offered to everyone may indicate participants’ beliefs that choices for coverage must be made.

These findings imply that the needs of genetic services stakeholders for clinical utility evidence and structured decision processes are not being met, and the impending clinical genetics paradigm shift in focus to genomic technologies will likely exacerbate these concerns. Further research for adapting decision-analytic modeling approaches to genetic services is warranted, particularly as it pertains to multiplex genetic testing technologies. Furthermore, other decision-makers in the genetic services area may wish to consider the use of decision-analytic modeling in their current processes as appropriate.

Findings from this study are consistent with an earlier stakeholder study of attitudes toward risk–benefit (but not cost) modeling for genetic testing.4 That study involved a wider range of stakeholders, including consumers, developers, payers, clinicians, and researchers. Our current study included a less-diverse group that had a higher representation of genetic services decision-makers. In both studies, there was general support for quantitative decision-analytic modeling as a tool to structure the discussion of the clinical utility of genetic tests and its ability to make explicit the uncertainty surrounding a particular service’s benefits, risks, and costs. However, both studies revealed a lack of consensus on some methods issues (e.g., the use of the quality-adjusted life years). This study provides more detailed feedback on genetic services reimbursement from a more specialized group of decision-makers.

An important finding of this study was the concern genetic services stakeholders have regarding multiplex testing strategies. The implications of these concerns were clear: focusing future decision-analytic modeling efforts on single-gene test interventions was seen as inessential, and stakeholders preferred that our efforts be focused on modeling array-based or NGS technologies. However, creating a model for multiplex genomic testing of multiple disorders is a significant challenge for which traditional decision-analytic approaches are ill equipped.

One example of the types of frameworks needed for assessing NGS involves assigning genomic loci to clinically relevant categories that allow for a clinical-utility driven, streamlined approach to return of NGS results.13 Our study findings indicate a desire for synthesizing such frameworks with the concepts of decision-analytic modeling. In collaboration with the WSGSC leadership, we are in the process of developing a new framework that quantitatively combines patient preferences for return of results with clinical genetics expertise. This approach likely will not be a traditional decision-analytic model but will appropriate some of the concepts entailed therein.

Implications from this study must be interpreted with recognition of the limitations of the methods and study sample. First, the purposive sample of regional genetic services stakeholders limits the generalizability of our findings to all US genetic services stakeholders. Second, results were derived from the experience and expertise of the stakeholder group whose attitudes and beliefs about genetic tests may not reflect a complete range of genetic services experience. Third, questions used during the electronic voting sessions were not validated with other populations, so results may be biased by measurement error. Finally, the directed content analysis approach we used necessitates that researchers approach the data with an informed but strong bias. Therefore, researchers might be more likely to find evidence that supports their aims. In addition, participants might have perceived cues to answer in a certain way or agree with the questions to please researchers.12

In conclusion, our findings point to a need for more systematic, yet flexible, evidence evaluation frameworks to assist decision-makers with their challenging, “real world” decisions regarding the provision of genetic testing services. Decision-analytic models—or perhaps concepts from decision analysis—appear to have promise as a tool to address these needs given the paucity of direct evidence in genomics, but generalized frameworks will be needed to address the challenges of evaluating NGS technologies.

Disclosure

The authors declare no conflict of interest.