Introduction

The number of genetic tests that provide clinically actionable information is rapidly increasing, especially for adult-onset conditions;1,2 however, their integration into routine health care has been slower than anticipated.3,4 Several reasons have been cited for the slow adoption of genetic testing, including limited clinician knowledge about genetics and genetic risk assessment,5,6 clinician lack of confidence about when to use genetic tests,6,7,8,9,10 patient concerns about cost and discrimination,5,6 lack of access to genetic services,6,11,12 and lack of a sufficiently rigorous evidence base demonstrating clinical utility critical for clinical management.3 Clinical utility is an important measure of the value of a genetic test. It has been defined as leading to an improved health outcome, including diagnosis, treatment, management, or disease prevention, that will benefit a patient or his/her family members.13 Although evidence of clinical utility is always desirable, there are situations when strong evidence of clinical utility is not necessary for the results to be clinically actionable.14 The value of a test, or worthiness of its impact at the cost involved, is shaped by promises of the test and real-world practice.15 When the evidence demonstrating the clinical utility of an innovation is limited, the stakeholder’s perspective, the surrounding context, and ways in which the innovation is put to use become especially critical elements in determining its value.15,16

To help inform policy decisions made by health-care system leaders who are considering adoption or expansion of their genetic test offerings, we set out to assess, from clinical leaders’ perspectives, the value of genetic testing across a spectrum of clinically actionable test results. Genetic tests with clinically actionable results have clinical validity (i.e., the ability to consistently and accurately detect or predict the outcomes of interest) and also varying degrees of clinical utility (i.e., the likelihood that a test significantly affects patient health outcomes).17 Thus, there is a continuum of a test’s perceived value ranging from providing clinically actionable information based on a high degree of clinical validity and clinical utility to providing less actionable, but perhaps still useful, information based on clinical validity and limited or no clinical utility.18,19 Because the value of a genetic test can help shape health policy decisions regarding its adoption,4 we sought to identify factors associated with highly valued genetic test indications, including factors that may influence provider behavior to utilize genetic tests to inform health-care decisions within a health-care organization.

Materials and Methods

Design

We conducted an observational, cross-sectional study in which we administered an Internet-based, structured survey to study the value of genetic testing within the Department of Veterans Affairs medical centers (VAMCs) nationwide. The survey link was e-mailed to clinical chiefs who served as key informants representing their own clinical areas within their facilities.20 Responses were anonymous. The survey was informed by findings from semistructured interviews with VA clinical chiefs that sought to identify characteristics of genetic services that facilitate or hinder adoption.4 The survey was finalized following input from cognitive interviews with four primary-care clinicians and a subsequent review for content, language, and ease of use from a national VA committee comprising chief medical officers and other clinical and administrative leaders.

Setting and participants

VA clinical chiefs of primary care, neurology, medical oncology, cardiology, and gastroenterology were invited to complete the survey as key informants. Key informants are knowledgeable informants selected because of particular experience (and, thus, specialized knowledge) about a topic of interest.21 With administrative and supervisory responsibility for their clinical specialty at their respective VAMCs, the clinical chiefs were well-positioned to provide information—from an organizational leadership perspective—about the current genetic testing practices within their respective specialty areas and at their facility in general. We searched publicly available websites or contacted the chief of staff at VAMCs to identify the clinical chiefs. A primary care chief was identified for most (142) of the 153 VAMCs. However, not all VA facilities had chiefs of other specialties. We identified 110 neurology, 101 cardiology, 103 gastroenterology, and 103 medical oncology chiefs.

Data collection

Prior to the launch of the survey, an introductory e-mail was sent to eligible key informants from the national VA program director of each specialty and the VA program director of the national Genomic Medicine Service. Data collection occurred from November 2011 through September 2012, with each specialty having from 4 to 6 months of access to the survey. Up to six reminder e-mails were sent to nonresponders, followed by several phone calls when necessary. Prior to the close of data collection, a final e-mail was sent from the national director of each specialty encouraging the participation of nonrespondents.

Outcomes of interest

Considering their experience in the VA and their perceptions of the VA’s role in providing care to veterans, the chiefs were asked to rate the value of genetic testing performed solely for the following purposes: (i) confirm a suspected genetic diagnosis, (ii) confirm an established clinical diagnosis, (iii) inform clinical management or treatment, (iv) inform the selection of effective disease prevention, (v) assist patients in making reproductive decisions, and (vi) assist patients in making life-planning decisions, such as career, marriage, or finances. Using a 5-point Likert scale ranging from “not at all valuable” to “extremely valuable,” informants rated the value of using a genetic test for each of the six reasons.

Independent variables

We used an adapted version of the Determinants of Provider Behavior conceptual framework,22 which was originally designed for studying implementation and quality improvement in health care, to inform the selection of independent variables influencing the value of genetic testing. In our modified framework, four domains contribute to the delivery of genomic services: (i) environmental factors outside of the medical facility, (ii) characteristics internal to the organization, (iii) provider characteristics, and (iv) characteristics specific to the patient and the medical encounter. The domains and their individual constructs as operationalized for the survey are listed in Table 1 .

Table 1 Characteristics of VAMCs as described by key informants according to variables that influence provider adoption of genetic services
Full size table

Most variables were obtained from the survey responses. The Area Resource File (http://arf.hrsa.gov) sponsored by the Health Resources and Services Administration served as the source describing some external and internal VA facility characteristics, including the number of male and female veterans per 100,000 population and the number of medical geneticists per 100,000 population. Hospital complexity rating (a VA-generated rating comprising factors including patient population characteristics, clinical services complexity, research funding, and numbers of residents trained) was obtained from VA administrative data sources. Medical school affiliation (defined as an affiliation with a medical school or medical school training program for residents) was derived by examining national VA organizational data, the VA’s Office of Academic Affiliations website (http://www.va.gov/oaa), and local VA website facility descriptions.

Statistical analysis

We used descriptive statistics to summarize the characteristics of the responding chiefs and their facilities. Bivariate statistics, including chi square, Student’s t-tests, and ANOVA analyses, were used to identify independent variables meaningfully correlated with value ratings. For the bivariate analyses, the value ratings for a genetic test were dichotomized as “high value” (i.e., “very valuable” and “extremely valuable”) and “not high value” (i.e., “not at all valuable,” “somewhat valuable,” and “moderately valuable”). Hierarchical multivariate linear regression models based on the full 5-point scale calculated associations between each of the six reasons for genetic testing and factors influencing adoption of genetic services. Only those independent variables significantly associated with value ratings (P < 0.05) for at least half of the testing reasons were included to achieve parsimony in the regression models. A model was created for each of the six reasons for genetic testing. Cluster adjustment was incorporated into the model-fitting process because chiefs’ responses were assumed to be clustered by facility. If collinearity was detected among the independent variables (by examining correlations), then some of these were excluded from the regression analyses (e.g., importance of cost-effectiveness of genetic testing, usefulness of a genetic test results tab in the electronic health record, and providers having experience ordering genetic tests). Primary care was the reference to which the other specialties were compared. All analyses were conducted using STATA version 12.

Results

Key informant characteristics

We received 353 responses, resulting in an overall response rate of 63.1%, with 77.5% of the identified primary-care chiefs, 68.2% of the neurology chiefs, 58.3% of the medical oncology chiefs, 54.5% of the cardiology chiefs, and 51.5% of the gastroenterology chiefs participating. These chiefs represented 141 (92.2%) of the 153 facilities, with an average of 2.5 (range, 1–5) clinical specialties responding per facility. Of the participating chiefs, 31.2% represented primary care, 21.2% neurology, 17.0% medical oncology, 15.6% cardiology, and 15.0% gastroenterology.

Characteristics of the chiefs and the facilities they described are shown in Table 1 . There was good representation from the four US geographic regions. The majority of VAMCs were characterized as having high complexity (68.2%), were mostly in urban and suburban areas (86.5%), and had a medical school affiliation (90.3%). Only 3.4% reported that at least one provider was formally trained in genetics and only 13.3% had a genetic champion (i.e., a respected colleague who supports the use of genetic testing and services) in their specialty at their facility. A little more than half (57.5%) reported that clinicians in their department could obtain a genetic consult for their VA patients on-site, at another VAMC, or through referral to a non-VA facility.

The value ratings for each of the six reasons for genetic testing are shown in Table 2 . Genetic test results that inform management and treatment, and therefore have greater evidence of clinical utility, had the highest value ratings, followed by results that inform effective disease prevention, assist in reproductive decision making, assist in life-planning decisions by veterans, confirm a suspected genetic diagnosis, and confirm an established diagnosis. Considering high-value ratings, these ranged from a low of 32.3% for genetic test results that solely confirm an established diagnosis, requiring only evidence of clinical validity, to a high of 58.6% for results that can inform management or treatment, which require evidence of both clinical validity and utility. Thus, the perceived value of a genetic test increased with increasing clinically actionable information resulting from the test ( Figure 1 ).

Table 2 Key informant (n = 353) ratings of the value of genetic testing according to reason for testing
Full size table
Figure 1
figure 1

Clinical actionability relates to actions taken based on genetic tests, with increasing clinical actionability for tests with evidence for both clinical validity and clinical utility rather than clinical validity alone. As clinical actionability increases, there is a corresponding increase in value for genetic test results that inform actions based on clinical validity only (e.g., to confirm an established clinical diagnosis) compared to genetic test results that inform actions based on both clinical validity and utility (informing medical management and treatment). (Adapted from ref. 34.).

Full size image

Based on the criteria for inclusion in the multivariate analyses, 13 independent variables were included in the hierarchical linear regression model for each purpose for genetic testing. The independent variables included three factors external to the health-care organization (evidence demonstrating the utility of genetic tests, availability of guidelines for ordering genetic tests, and federal regulations protecting veterans from genetic discrimination); four internal organizational factors (an organizational culture that fosters the adoption of genetic services, at least one provider formally trained in genetics, the ability to obtain a genetics consult on- or off-site for a patient, and including commonly ordered genetic tests in the laboratory test menu); five provider characteristics (specialty, finds it difficult to recognize patients who may benefit from genetic testing, considers genetic services relevant to the health-care needs of their patients, knows when to refer patients for genetic consultation, and is able to apply the results of genetic testing to their patients’ management); and one patient and encounter characteristic (patient demand for genetic tests). The linear regression models were all highly significant (P ≤ 0.001), with adjusted R2 values ranging from 18% to 29%, indicating that the models explain a considerable amount of the variability of the reported values of genetic testing. Results for each of the six models are shown in Table 3 , and the contribution of individual variables to the outcomes of interest is described below.

Table 3 Associations between constructs influencing provider adoption of genetic services and value ratings for each of the reasons for genetic testing
Full size table

Genetic tests that inform clinical management

Only factors external to the organization and provider characteristics were positively associated with a high-value rating for this reason for genetic testing, including evidence demonstrating the utility of genetic testing (β = 0.47; SE = 0.12; P ≤ 0.001), guidelines for ordering genetic testing (β = 0.39; SE = 0.15; P = 0.012), having providers who know when to refer a patient for a genetic consultation (β = 0.35; SE = 0.12; P = 0.005), and specialty (e.g., cardiology chiefs were significantly less likely than primary-care chiefs to value genetic testing to inform patient management and treatment) (β = −0.373; SE = 0.16; P = 0.019). No internal organizational, patient/or encounter characteristics were significantly associated with high-value ratings for genetic tests that inform clinical management.

Genetic tests that inform selection of effective disease prevention

Internal organizational factors and provider characteristics were positively associated with high-value ratings, including: a culture that fosters the adoption of genetic services (β = 0.463; SE = 0.14; P = 0.002), having at least one provider with formal genetics training (β = 0.047; SE = 0.18; P = 0.011), and providers who know when to refer patients for a genetic consultation (β = 0.33; SE = 0.12; P = 0.008). Cardiology chiefs were significantly less likely than primary-care chiefs to give high-value ratings for this genetic testing reason (β = −0.66; SE = 0.17; P ≤ 0.001). No external organizational factors, patient or encounter characteristics were significantly associated with high-value ratings for genetic tests that inform disease prevention.

Assisting with reproductive decisions

External organizational factors positively associated with high value for this genetic testing reason included guidelines for ordering genetic testing (β = 0.34; SE = 0.15; P = 0.026) and federal regulations protecting veterans from genetic discrimination (β = 0.31; SE = 0.14; P = 0.031). The internal organizational factors positively associated with high-value ratings included an organizational culture that fosters the adoption of genetic services (β = 0.32; SE = 0.15; P = 0.042) and the ability to obtain a genetic consult for veterans (β = 0.27; SE = 0.12; P = 0.031). Specialty was the only provider characteristic associated with high-value ratings. Compared to primary-care chiefs, chiefs of medical oncology (β = −0.80; SE = 0.22; P ≤ 0.001), cardiology (β = −0.64; SE = 0.20; P ≤ 0.001), and gastroenterology (β = −0.78; SE = 0.21; P ≤ 0.001) were less likely to give high-value ratings for this genetic testing indication. There were no significant differences in value ratings between neurology chiefs and primary-care chiefs. Neither patient nor encounter characteristics were significantly associated with high-value ratings for genetic tests that inform reproductive decisions.

Genetic tests that assist with life-planning decisions

The only internal organization characteristic positively associated with high-value ratings for this genetic testing reason was having genetic tests included in the laboratory test menu in the electronic health record (EHR) (β = 0.30; SE = 0.13; P = 0.024.) The only provider characteristic was specialty; compared to primary-care chiefs, high-value ratings for this reason for genetic testing were less likely to be given by chiefs of medical oncology (β = −0.62; SE = 0.22; P = 0.006), cardiology (β = −0.60; SE = 0.19; P = 0.002), and gastroenterology (β = −0.60; SE = 0.22; P = 0.006). Neurology chiefs did not rate the value of genetic testing for this reason differently from the primary-care chiefs. External organizational characteristics and patient and encounter characteristics were not associated with high-value ratings for genetic tests that inform life-planning decisions.

Genetic tests to confirm a suspected diagnosis

Significantly associated factors included internal organizational and provider characteristics. Positive associations with high-value ratings included organizational culture (β = 0.27; SE = 0.12; P = 0.032), a provider in the department with formal genetics training (β = 0.91; SE = 0.21; P ≤ 0.001), genetic tests in the laboratory menu in the EHR (β = 0.35; SE = 0.10; P = 0.003), and provider specialty. Neurology chiefs were significantly more likely to give high-value ratings (β = 0.70; SE = 0.19; P ≤ 0.001) compared to primary-care chiefs. There were no significant differences in value ratings between the other specialties and primary care. External organizational factors and patient and encounter characteristics were not associated with high-value ratings for genetic tests that confirm a suspected genetic diagnosis.

Genetic tests to confirm an established diagnosis

The internal organizational factors of having a culture that fosters the adoption of genetic services (β = 0.42; SE = 0.12; P = 0.001) and a genetic test menu in the EHR (β = 0.43; SE = 0.12; P ≤ 0.001) were positively associated with high-value ratings for this genetic test reason. The only provider characteristic was specialty; neurology chiefs were more likely than primary-care chiefs to value genetic testing for this reason (β = 0.45; SE = 0.20; P = 0.029). The other specialty-area chiefs gave ratings similar to those of primary-care chiefs. External organizational factors and patient and encounter characteristics were not associated with high-value ratings for genetic tests that confirm an established genetic diagnosis.

Discussion

Our findings show that most clinical leaders in the VA, one of the largest integrated health-care systems in the United States,23 found all of the reasons for genetic testing to be at least moderately valuable. We found that the value ratings for a genetic testing reason were correlated with the degree of clinically actionable information resulting from a genetic test. For example, there were fewer high-value ratings for a test performed solely to confirm an established clinical diagnosis than for a test performed solely to inform medical management and treatment.

Genetic testing applications are rapidly evolving, with increasing availability to every medical specialty, although a rigorous evidence base demonstrating clinical utility is often lacking.3,9,24 When there is limited evidence of benefit during the early diffusion phase of an innovation, the perspective of stakeholders is key in determining a value assessment and implementation decisions.15 Our results demonstrate, for the first time, the value perspective of a health-care organization’s clinical leadership regarding genetic testing. Importantly, their perceptions of value were not restricted to genetic tests only demonstrating an evidence base of clinical utility. This perspective can support the adoption and continued use of genetic tests in this organization.

The conceptual framework we used was useful for identifying factors that influence how genetic testing is valued by stakeholders representing the perspective of a health-care organization’s clinical leadership. Internal organizational structures and processes, such as a culture that fosters adoption of genomic medicine or having genetic tests on the laboratory menu, and provider characteristics, such as specialty and knowing when to refer a patient for genetic consultation, were most frequently associated with high-value ratings for genetic testing. Notably, the strength of the association with these characteristics appeared to vary according to the reason for testing, with greater consistency in valuing testing for reasons having greater clinical actionability.

The external organizational factors of demonstrated utility of genetic test results and availability of guidelines for test ordering were significantly associated with reasons for testing that have critically important implications for immediate clinical decision-making, such as genetic testing that informs management decisions for patients with signs and symptoms of a genetic condition and genetic testing that informs reproductive decision-making. These results suggest that the evidence of utility and genetic testing guidelines could have a meaningful effect on the willingness of an organization’s leadership to support testing for these purposes.25,26

Patient and patient–clinician encounter characteristics were not associated with the value of genetic tests regardless of the reason for genetic testing. This implies that patient characteristics or constraints of the patient–clinician encounter do not enter into how a genetic test is valued by the leadership of a health-care organization. Thus, targeting patient-specific characteristics (e.g., age, gender, race/ethnicity) or changing features of the patient–clinician encounter (e.g., more time, different setting) are unlikely to result in increased adoption or expansion of genetic testing in this organization.

An organizational culture that fosters the adoption of genetic services was associated with high value ratings for many of the reasons for genetic testing. This is consistent with literature describing organizational culture as being important to the adoption of technologies and strategies within a health-care organization.27,28,29,30 Leadership within an organization has the power to effect cultural change.31,32 Therefore, engaging organizational leadership in understanding the various reasons for genetic testing and its value beyond clinical utility may increase adoption of genetic tests to support patient-centered care. The positive association we found between having genetic tests available on the laboratory test menu and high value ratings for genetic tests with relatively low clinical utility (i.e., to inform life-planning decisions) supports this assertion. Within the VA, the laboratory directors at each facility assume fiscal responsibility for all tests performed, including genetic tests sent out to reference laboratories; they are typically responsible for approval decisions for such testing.4,33 Unrestricted genetic test ordering could result in overutilization of genetic tests. To restrict access to costly and complex tests, such as genetic tests, VA laboratory directors often implement preapproval processes and/or omit these tests from the laboratory test menu.4 Thus, when genetic tests are available on the laboratory test menu, this signifies leadership approval for unrestricted test ordering by providers.

The value of genetic testing varied significantly by specialty. Neurologists highly valued genetic testing for all six reasons for testing, whereas the oncology and gastroenterology chiefs gave high value ratings only to tests that inform management and prevention of a genetic diagnosis. Primary care chiefs gave high value ratings for all testing reasons except for confirming a suspected or established diagnosis. Less than half of the cardiology chiefs rated any reason to use a genetic test as highly valued. The associations between specialty and value of genetic testing may be influenced by the availability of genetic tests for conditions relevant to each respective discipline and to the types of patients seen. For example, neurologists are often faced with making a diagnosis for a patient with neurological signs and symptoms, whereas oncologists are focused on treatment decisions for cancer and options for cancer prevention.

In addition to the importance of having clinical guidelines for genetic testing, access to a provider formally trained in genetics and having a provider who knows when to refer a patient for a genetic consultation were positively associated with highly valued genetic tests for a range of clinically actionable testing reasons. This suggests that clinical leaders recognize the importance of a knowledgeable workforce that knows how best to utilize genetic services. These findings suggest an important role for genetic specialists in the adoption of genetic tests within a health-care organization.

There are some limitations of our study that deserve mention. First, because the VA is an integrated health-care system, our findings may not be relevant to other health-care organizations, particularly solo or small group practices. Second, our findings are limited to the perspectives of clinical chiefs in the VA representing clinical leadership and the organizational viewpoint. Ascertainment of provider and patient input was beyond the scope of this study but would contribute to our understanding of the factors associated with the value of genetic testing; this should be considered for future research. Finally, in this study, we did not report on the value of genetic testing explicitly for the purpose to inform family members of their genetic risk. Although this is an important use of genetic testing, only clinically actionable results for the patient tested were the focus of this study.

From an organizational leadership perspective, as the clinical actionability of genetic test results increases, there is a corresponding increase in value for genetic tests. Internal organizational structures and processes, external organizational factors, and provider characteristics that determine how genetic services are delivered influence the value of genetic testing within a health-care organization. These findings can help guide the design of multifaceted implementation efforts to incorporate genetic testing within an integrated health-care system. Such efforts should focus on strengthening the organizational culture to foster adoption of genomic medicine by (i) engaging organizational leadership in understanding the various reasons for genetic testing, (ii) encouraging an organizational culture that supports the adoption of genetic services, (iii) addressing unique needs of different medical specialties based on their role in providing genetic services and the relevance of genetics to the patients they care for, (iv) educating providers to recognize when to refer patients for genetic consultation, and (v) ensuring access to genetics professionals to help guide appropriate utilization of genetic tests. In addition, these results can inform the debate on the integration of genetic testing services into mainstream health care in light of value-based assessments of innovations.15

Disclosure

The authors declare no conflict of interest.