Introduction

The arsenal of genetic and genomic diagnostic and therapeutic approaches has expanded rapidly,1 particularly in the field of adult genetics.2 Because of complex patient-, provider-, and system-level barriers, however, the adoption and implementation of evidence-based genomic medicine has been slow in health-care settings.3,4,5,6,7,8 In response to this limited integration of genetics and genomics into clinical care, translation research has been widely called for,9,10,11,12 with a recent emphasis on the need for multilevel interventions13 targeting implementation challenges across three or more levels of contextual influence14 on genetic health-care delivery. Levels of contextual influence can include patients, their families, and social supports; health-care providers; health-care organizations; and the state and national policy environment.14 The specific role of organizational barriers has been largely overlooked in studies identifying challenges to integrating genomic medicine into routine health care; the most recognized barriers relate to provider knowledge gaps, genetic workforce insufficiencies, limited existing models of successful clinical implementation, and the inadequacy of available consumer information.3

Research methods that illuminate the interactions between multiple levels of contextual influence to identify targets for organization-based translational interventions have not been adequately described. We describe the design of a health services research approach to assess organizational determinants of adoption and implementation of genomic medicine in the Veterans Health Administration (VHA). This investigation was conducted in response to a research initiative sponsored by the Department of Veterans Affairs (VA) Health Services Research and Development Service to promote examination of the potential of emerging genomic technologies to optimize and improve medical care for veterans. As a component of this initiative, development of tools and models for genomic translation throughout VHA was encouraged. Such development required a baseline understanding of existing practice patterns, an assessment of practice variation, and a description of the organization-, clinician-, and patient-level challenges to genetic health-care delivery in the VHA to understand where and how to target translational interventions. We focused on organization-level challenges to adoption and implementation of genomic medicine, given the paucity of studies addressing this level of contextual influence.

Organizational research has been used frequently to illuminate factors affecting implementation-level translation of evidence-based knowledge and practice into care.15,16,17,18 This article focuses primarily on presenting an organizational research methodology to support implementation and translation of genomic applications into health-care organizations; detailed findings of the described studies using these methods are reported in separate7,19 and forthcoming publications. Though applied herein to a large, national, integrated health-care system, this methodological approach may serve as a model for health-care organizations in general, providing strategies to identify organizational targets for improving uptake of genetic and genomic services.

Materials and Methods

Developing a research plan

Building on experience in prior organizational research20 and quality improvement processes,20,21 we assembled a research team with methodological and content expertise in clinical genetics, provider behavior theory, quality improvement, and implementation science to design an organizational research plan focused on defining existing genomic medicine practice patterns across the VHA nationwide and identifying barriers and facilitators of genomic medicine implementation. We chose expert panel22 and key informant interview23,24 and survey methods25 because they are well suited to studies in which organizations, rather than individuals, are the subjects of analysis. Key informants are uniquely prepared to make useful observations that describe the impact of multiple levels of influence on organizational behavior (e.g., a service chief may have insight into the influence of the age and gender mix of the organization’s patient population on the likelihood that providers value genetic testing). A conceptual framework was selected to inform our study design and the synthesis of our study findings. Each method and the conceptual framework are described below.

Expert panels

Developed initially for the social sciences, expert panel methods have been used in health care to develop clinical guidelines and standards for quality of care26,27 and have been modified for use in setting institutional priorities to increase the impact of quality improvement efforts on patient health.22 In the latter context, top-ranking organizational managers are brought together to examine existing data and institutional priorities to define a set of “products and processes that will enable the institution to move from the type of care it currently delivers toward its anticipated future.”22 (p. 422).

Historically, expert panel methodology has involved inviting experts with varying relevant backgrounds to participate in a consensus-building exercise, typically conducted in a three-round modified Delphi process involving a round 1 survey where each panelist ranks action priorities; a round 2 in-person meeting to discuss survey results and come to consensus on initial rankings; and a round 3 follow-up survey for final rankings of top-rated priorities.22,26 Expert panel methods lend themselves to being adapted for use in contexts requiring various stakeholder perspectives and expertise in selecting and developing action priorities.

Qualitative interviews with key informants

The key informant technique involves qualitative interviews with knowledgeable informants who are selected not based on population representativeness but rather because of particular experience with (and thus specialized knowledge about) a topic of interest.24 As such, key informants are not the subjects of the studies in which they participate; rather, they provide information about the subject of the study (e.g., the health-care organization or a set of its potentially immeasurable components that the informant has the expertise to describe and make summary judgments about). Used initially in anthropology and sociology for cultural and community research, key informant interviews have come into use for understanding the cultures and practices of organizations as a foundation for such activities as defining emerging practice arrangements; assessing organizational readiness for change; evaluating decision support tools; identifying organizational challenges and facilitators; and the impacts of specific program implementation.28,29,30

Organizational informants are identified using purposive sampling techniques to find individuals with the requisite knowledge to address the research question(s). For example, a study might target employees of a particular type (e.g., clinicians affected by new practice guidelines); employees exposed to a particular experience (e.g., users of a decision support tool); or leaders with relevant organizational oversight responsibilities (e.g., chiefs of units attempting to adopt a new health-care strategy). While traditional ethnographic research uses unstructured interviewing with key informants to describe whole cultures, sociological and organizational uses of the method tend to structure the interview with a limited set of open-ended questions on a range of targeted topics. In organizational research informants are typically interviewed in this semistructured manner about prevailing knowledge of, attitudes about, practices associated with, and experiences with a topic related to their work environment. Researchers qualitatively analyze interview data, leading to the identification of themes and insights on which to base decision making (possibly through feedback to an expert panel) or as a basis for designing a next research step, such as an observational study or structured survey.

Quantitative surveys of key informants

Although variables of interest in quantitative organizational research can sometimes be directly measured (e.g., numbers of patients served), important constructs—such as the complexity of a new task, communication between clinical and laboratory personnel for test ordering, or the accessibility of specific services within a given organization—often cannot.31 When targeted as survey respondents, however, key informants can provide proxy measurement of such constructs. In key informant surveys the unit of analysis is the organization, or a component or aspect of the organization, not the respondent. Informants are not research “subjects” sampled on the basis of population representativeness, whose survey responses are statistically aggregated to examine distributions in the frequency of measured attitudes and behaviors. Instead, survey informants, like key informant interviewees, are purposively selected based on their ability to provide information about organizational attributes and processes. They may descriptively report observed practices, attitudes, and opinions in a work group or organizational setting by responding to questions that characterize the setting, not their own personal attitudes and traits.31 Key informant surveys have been used in many organizational settings to examine such topics as implementing clinical practice guidelines,32 assessing care for veterans with HIV,17 assessing the impact of primary-care resources on prevention practices,15 and characterizing the organization and delivery of women’s health care.16 Challenges related to informant reliability and potential bias need to be addressed31 in survey design (e.g., types of questions), sampling (e.g., defining informant characteristics and the number of informants to be surveyed per organizational unit), and analysis (e.g., methods for aggregating responses from multiple informants). Results from key informant surveys can, however, provide valuable data on organizational variables that would be impossible, or prohibitively expensive, to obtain by other means.

We used the above-described organizational research methods to design and implement agenda-setting pilot work. Pilot study results then informed the direction and focus of a national organizational research effort that used panel methods and key informant interviews and surveys anchored in the conceptual framework described below.

Conceptual framework

Conceptual frameworks representing theorized relationships among organizational structures related to a research project’s goals often are used to guide data collection and analysis in organizational research. Because the focus of our study was the delivery of genomic medicine within a health-care organization, we used a conceptual framework that is grounded in a clinically anchored version of the structure-process-outcome model described by Donabedian33 ( Figure 1 ). The domains of internal organizational structures and processes, environmental or external factors, provider characteristics, and patient and clinical encounter characteristics are considered determinants of provider behavior,34 which in this case is the delivery of genomic medicine that influences health-care outcomes. Because genomics is an emerging field, representing a potentially disruptive set of technologies35 requiring reinvention of organizational processes, we incorporated into the model elements of diffusion theory,36 adding “organizational innovativeness” to the mix of factors affecting the delivery of genomic medicine through internal organizational and external determinants. Diffusion theory focuses on the factors that increase or decrease the likelihood that an innovation will be adopted by individuals in an organization36 and by service organizations as systems.37 Table 1 reviews the internal organizational factors and external factors viewed as promoting organizational innovativeness.

Figure 1
figure 1

Conceptual model of factors associated with the adoption and delivery of genetic/genomic medicine. The conceptual framework is based on Donabedian’s structure–process–outcome model.33 The domains of internal organizational structures and processes, environmental or external factors, provider characteristics, and patient and clinical encounter characteristics are considered determinants of provider behavior.34

Full size image
Table 1 Diffusion of innovation in service organizations applied to genomic medicine
Full size table

Results

Local pilot studies

Pilot work focused on examining genomic medicine practices in primary care at one large VHA health-care system. We used the qualitative key informant interview method to collect data from 15 primary-care clinicians to preliminarily assess prevailing practices and attitudes about collecting and documenting family history, ordering genetic tests, and making referrals for genetic consultation. Findings revealed the routine collection of only limited family history occurred primarily during initial visits, and genetic testing was limited primarily to factor V Leiden testing for hereditary thrombophilia and HFE mutation analysis for hemochromatosis. To improve the efficiency of collecting family history data, clinicians suggested developing a template for the electronic health record, better organization of the family history in the electronic health record, and using a previsit form or the VHA’s personal health record to collect patient-provided data.

These results, together with other informational material, were presented to an expert panel comprising VHA and non-VHA clinicians from relevant specialties (e.g., primary care, genetics, oncology, neurology, and pathology) and with various leadership perspectives (e.g., national, regional, and local). The panel used a three-round process to rate for the VHA research priorities regarding genetic conditions of importance in the next 5 years and to identify key factors for the adoption of genetic medicine in the VHA. Acquired genetic conditions (e.g., cancer), common single-gene and chromosomal disorders, pharmacogenetics, and multifactorial diseases were rated as having the greatest clinical importance and potential demand in the next 5 years; internal organizational structures and processes were rated as most important to studying health services genomics. Key factors for the adoption of genomic medicine in the VHA included the availability of prevention and management options, clinical guidelines, genetic tests, physician geneticists, a centralized genetics program within the VHA, clinical decision support for the electronic health record, and genetics education for primary-care and other specialists.

The pilot studies revealed gaps in the identification and referral of individuals at risk for hereditary conditions. As a result, a multicomponent toolkit was developed; it included patient-, provider- and organizational-level interventions designed to facilitate family history risk assessment by primary-care clinicians and referral of patients with appropriate indications for genetic consultation.38 Panel members and stakeholders representing differing viewpoints of the organizational hierarchy consulted on the design and implementation of this toolkit, helping to successfully tailor its implementation to local organizational conditions.

National organizational studies

National organizational studies, as shown in Figure 2 , took place over a 3-year data collection period and used all three methods—expert panels, as well as qualitative and quantitative key informant methods—to examine the organization and practice of genomic medicine in six specialties: primary care, medical oncology, neurology, cardiology, gastroenterology, and pathology and laboratory medicine.

Figure 2
figure 2

Application of a three-component methodological approach in pilot work and in a national organizational study. For our pilot project, we conducted qualitative, semistructured interviews with 15 primary care clinicians as key informants to assess prevailing attitudes and practices relating to the delivery of genomic medicine, such as collecting and documenting family history and ordering genetic tests. We convened an expert panel comprising 12 members, including leaders in primary care, clinical genetics, health services research, and quality improvement, to review these pilot project interviews and to set priorities for research related to the delivery of genetic medicine for the next 5 years. Our national organizational studies included interviews and a survey of key informants. In year 1, we convened our 12-member expert panel to help us develop our research goals and to advise us on development of our research instruments, including our interview guide and structured survey. We conducted qualitative interviews with 64 key informants from five clinical specialties, including primary care, medical oncology, cardiology, neurology, and pathology and laboratory medicine. Results from these interviews informed the development of survey items and response options. In years 2 and 3, we developed and administered the structured survey and analyzed, synthesized, and interpreted the results of the survey. In year 2, our expert panel grew to include six additional clinicians who provide clinical genetic services at Veterans Health Administration (VHA) facilities. This expert panel rated the importance of key factors, identified from our interviews, that relate to the adoption of genomic medicine, as well as barriers and facilitators for the adoption and implementation of these factors. In year 3, the expert panel grew to 23 members from multiple levels of VHA organizational influence, including the chiefs of the clinical specialties that were surveyed. Preliminary survey findings describing the availability of genetic consultation and the utilization of genetic testing were reviewed to help contextualize and synthesize the results.

Full size image

To guide our response to preliminary findings from qualitative and quantitative studies of key informants and to allow for the accrual of perspectives from additional experts (as they became known to us over time), we developed an iterative expert panel process, in which a growing set of panelists was convened more than once to rate tasks dictated by emerging concerns. The initial 12-member panel included VHA clinical experts from multiple specialties and VHA administrative leaders at the national and network levels; given the paucity of medical geneticists in the VHA at that time, we identified medical geneticists from outside the VHA with leadership roles at the local and national levels. The panel grew to 23 members with the addition of six VHA genetic professionals identified over time, as well as VHA experts in gastroenterology and surgery. Panelists not associated with the VHA were purposively selected based on genetics research expertise or leadership roles in the local and national genetics communities. The panel formally rated the importance of factors key to the adoption of genomic medicine and the extent to which organizational factors served as barriers to adoption. The VHA genetic service providers gave to the larger panel important information about the details of their practices. Panelists also gave feedback and direction based on preliminary findings presented at meetings.

Informed by our conceptual framework, the expert panel, and the results of our pilot studies, we conducted qualitative semistructured interviews with 64 key informants (service chiefs and other leaders) in five medical specialties. The key informants were purposively selected from VA facilities that we knew (either through colleagues, VA leaders, or snowball sampling nominations) had some experience with genomic medicine or had a knowledgeable key informant who could address issues pertaining to local adoption of genetic services. Facility locations represented all geographic regions served by the VHA nationwide. The interviews covered six general domains addressing organizational conditions before implementation: (i) existing genetic services and arrangements for providing genetic services; (ii) types, training, and roles of professionals who provide services; (iii) procedures for ordering genetic tests; (iv) barriers to the adoption and delivery of genetic services; (v) the relative priority of genetic services in specialty areas; and (vi) factors necessary for the further development of genetic services, including professional development activities.

A qualitative analysis of interview transcripts focused on the attributes of genetic services (testing and consultation) relevant to the diffusion of this innovation in the health-care system found that, in general, key informants did not typically characterize genetic services as advantageous for their facilities or their patients; they did not consider them compatible with organizational norms of low cost and high clinical impact or applicable to their patient populations or norms of clinical care. Genetic services were considered complex, knowledge of and expertise in genetic testing was limited, and organizational barriers to the utilization of genetic services were perceived as substantial.7 Interpretation of often ambiguous genetic test results was viewed as having complex dimensions that many clinicians were not trained to evaluate. Complex logistic “hoop-jumping” in the genetic test ordering and approval processes was frequently described as a frustrating barrier to the utilization of genetic testing.7 Navigating the leadership hierarchy to access genetic tests with proven medical benefit was described as a complicated process of working with VA laboratories responsible for the genetic testing budget (comparable to justifying medical necessity to third-party payers).

With guidance from qualitative work and expert panel input, we developed a structured quantitative survey addressing the delivery of genomic medicine in the VHA that was customized for six medical specialties: primary care, neurology, medical oncology, gastroenterology, cardiology, and pathology and laboratory medicine. We targeted the specialty service chiefs at each VA facility as the key informants representing that facility. From the interview results we gleaned priorities for factors to include as independent variables in the quantitative survey, such as VA facility affiliation with a local academic medical center as a potential predictor of access to genetic consultative services. The survey was fielded as a one-time, Web-based instrument to all service chiefs for the selected specialties across the entire census of VA facilities (i.e., not a sample of VA facilities), yielding an overall response from 449 chiefs (63%) representing 142 of the 153 VA facilities nationwide.

Preliminary survey results were presented to the expert panel at the last in-person meeting in year 3, which helped the research team synthesize the survey findings. The first analysis of findings assessed the models of genetic service delivery in the VHA and found that a traditional model (as defined by Battista et al.39) of multidisciplinary clinics led by geneticists or coordinated services between geneticists and other providers was most prevalent, although there was variability in the organization of these services, with consults available on site, at another VA facility, via tele-genetics (video-/teleconferencing), or at non-VA facilities. This analysis also found non–genetics providers had integrated genetics into their practices, though there was considerable variability in the degree of integration by specialty.19 Further analyses in support of additional specific research questions are underway.

Discussion

The described three-part health services research approach using expert panel and key informant methods enabled us to accumulate data toward our goals of acquiring a baseline assessment of genomic medicine in the VHA,19 including identification of barriers to and facilitators of the adoption and implementation of genomic medicine.7 In aggregate, our expert panel and key informant strategies created a communication network across providers, organizational representatives, and community leaders and a detailed organizational foundation from which to plan the development of tools and models for genomic translation in the VHA.

In 2007, Khoury et al.4 described four phases in the continuum of translation research: type 1, discovery to candidate application; type 2, health application to evidence-based practice guidelines; type 3, practice guidelines to health practice (implementation, dissemination, and diffusion research); and type 4, practice to population health impact (outcomes research). Khoury and colleagues estimated that no more than 3% of research on the translation of promising genomic applications to health practice focuses on type 2 and beyond. Organizational research can be particularly useful in accelerating type 3 translation.

Organizational methods can be designed to account for the reality that clinical applications are developed and delivered in complex environments where many factors unrelated to the application’s effectiveness influence whether patients benefit. For example, even if a confluence of separate population-based studies has shown that testing women for BRCA1/2 gene mutations can define an increased breast and ovarian cancer risk and can result in interventions that prevent these cancers, implementing this knowledge in the form of real-world patient-care procedures is a whole other matter. Although it may be “tempting to target the care provider as the ‘problem’ and direct remediation strategies at the provider to ‘correct’ the problem”40 (p. 115), successful translational activities must target additional levels of influence. Importantly, “it is not so much the number of levels…or sheer number of multilevel activities conducted, but the interplay and alignment of activities across levels that appears to be critical”40 (p. 116). The value of health services research methods, like those discussed here, lies in their ability to illuminate the interplay of activities required to successfully deliver genomic medicine from an organizational perspective. Relying on data collected from different types of experts and stakeholders, these methods can be adapted to include expert panelists and key informants who can describe the impact of multiple levels of influence on the functioning of a given care system.

Manolio et al.8 emphasize that institutional stakeholders are key in developing practical and efficient clinical workflow for obtaining testing and providing results. In addition, “identifying and engaging the myriad stakeholders within an institution, including those needed to conduct genetic testing, interpret it, integrate it with the [electronic medical record], provide results and recommendations to the clinician, and pay for it, require that senior leaders who are familiar with the institution and its culture be on the genomic medicine team”8 (p. 265). In our case, an expert panel—comprising not only clinical genetics experts from within and outside the organization but also senior leaders familiar with national, regional, and organizational VA policy and culture—participated with researchers on the “genomic medicine team.” Key informant interviews and surveys targeted chiefs and other leaders of six medical specialties who could describe practices as well as barriers and facilitators from multiple levels because of their responsibilities to meet national, community, and organizational standards, while also participating in and supervising provider-level clinical practice processes specific to their specialty, patient mix, and site. In addition, we interviewed and surveyed pathology laboratory directors who offered important insights into the interface between provider-level test ordering practices and practical and cost-related barriers influencing the feasibility of specimen processing and the effective routing of test result reports.

Expert panel and key informant methods can help elucidate points of interface between different levels of contextual influence, thus helping to identify the three or more levels a given translational intervention needs to target for optimal implementation outcomes. Like all research methods, those described here have limitations. Informants necessarily make judgments and summary assessments of complex issues that may be affected by limitations in their degree of knowledge or experience, or informants who are knowledgeable about a particular level of influence may be unavailable or difficult to access. In our case, because our primary goal was to describe genetics-related usual care from an organizational perspective, an initial limitation was the lack of patient-level stakeholder representation in our key informant research. However, we did include questions about patient interest in, and barriers to, genetic testing as observed by specialty chiefs, and interview informants identified patient characteristics (e.g., diagnoses encountered, age- and gender-mix of specialty-specific caseloads) relevant to the elucidation of potential gaps in services. With the understanding of usual care garnered though our baseline investigations, we can now meaningfully engage patients and caregivers directly as stakeholders.

In summary, expert panel and key informant strategies can be used to create a backdrop of stakeholder involvement and baseline organizational information within which to plan translation research to inform strategic planning and policies related to the adoption and implementation of genetic services in health-care organizations.

Disclosure

The authors declare no conflict of interest.