Main

Molecular diagnostic (MDx) tests are a key component of personalized cancer genomics. These tests identify mutations that drive cancer progression. Specific mutations predict response to therapies that may improve progression-free survival for patients.1,2,3 Recent reviews of cancer genomics cite widespread use of MDx.4,5,6 However, there is little evidence to support these claims, particularly in community hospitals, in which 85% of all patients with cancer seek care.7 Although underuse may be attributable to an uncertain evidence base,8,9,10 delays in translating health-care innovations to community hospitals is a longstanding problem. In the context of cancer genomics, such delays may widen the disparities in treatment outcomes.9,11,12

Historically, translational research has focused on discovery and development of early-stage innovations.13 Recently, public health researchers have proposed that translational research be considered as belonging to four phases, T0–T4.14 Bench to commercial product activities take place in stages T0–T2. Dissemination and implementation research is conducted in T3. Health outcomes research is conducted in T4. The majority of funding for cancer genomics has been directed toward T0–T2 research, with <2% of funding being directed toward T3 research that evaluates bedside implementation and dissemination.15 This dynamic has contributed to limited empirical data evaluating “real world” use of genomic applications in patient care. With only half of patients in the United States benefiting from advances incorporated into guidelines,16 there is a growing need to fund more studies to analyze the dissemination, implementation, and health outcomes of cancer genomics.17 This article, which represents T3 research, analyzes the dissemination of advances in theT0–T2 stages of translational research in lung cancer.

The history of MDx in lung cancer provides a context for understanding the barriers to T3 translation. In 2004, researchers at a National Cancer Institute cancer center (NCI CC) discovered a link between epidermal growth factor receptor (EGFR) mutations in lung tumors and the likelihood that these tumors would respond to a class of oral anticancer drugs, the tyrosine kinase inhibitors (TKIs).18,19 This significant T0–T2 innovation set the stage for a greater understanding of the molecular biology of lung cancer. By the end of 2004, erlotinib, an EGFR TKI, was approved for the treatment of patients with non–small cell lung cancer. The drug’s labeling cited prolonged survival for EGFR-positive patients. In 2005, quickly following erlotinib’s approval, Genzyme Genetics acquired the rights to commercialize the EGFR assay.20

The EGFR assay is an MDx test that is applied to tumor tissue to evaluate the presence of EGFR mutations. The results of the EGFR assay predict the likelihood that a patient will respond to EGFR TKIs. Patients with advanced lung cancer and EGFR mutations in their tumors are likely to benefit from first-line therapy with EGFR TKIs such as erlotinib and gefitinib. The EGFR assay is a highly complex test; it is required to be carried out only in laboratories accredited by the Clinical Laboratory Improvement Amendments (CLIA) program for cytogenetic testing. A limited number of large academic medical centers and NCI CCs are CLIA-accredited to conduct high-complexity testing. These centers perform EGFR mutation analysis in house, using laboratory-developed tests (LDTs).21 However, overall, <1% of laboratories are accredited to conduct genetic testing.22 The vast majority of hospitals that order MDx testing send patients’ paraffin-embedded tissue slides to a few commercial reference laboratories for analysis.

As early as 2007, the National Comprehensive Cancer Network (NCCN) encouraged oncologists to consider molecular testing to identify patients who may benefit from EGFR TKI treatment.23 In 2009, the American Society of Clinical Oncology (ASCO) recommended the use of EGFR TKIs to treat patients with EGFR-positive lung cancer. However, neither the NCCN nor ASCO made specific recommendations about criteria for selection of patients to undergo the test for EGFR mutations. It was not until 2011 that NCCN and ASCO updated their guidelines to specifically recommend the testing of non–small cell lung tumors for EGFR mutations.24,25 The timing of these guidelines is a significant factor to consider in our analysis. The data we analyzed for this study related to hospital use of the EGFR assay during calendar year 2010, the year before specific testing guidelines were issued.

The delay between approval of a drug for EGFR-positive tumors and EGFR mutation testing guidelines may have contributed to institutional and regional differences in the use of the EGFR assay and EGFR TKIs. This delay was caused, in part, by the immature state of lung cancer genomics and a lack of consensus about the diagnostic technology that was to be used. There was also some debate about the criteria for selection of patients for testing, and about whether payers should reimburse the costs of the testing. Early research indicated that the incidence of EGFR mutations varied with the patients’ ethnicity, sex, and smoking status.26 However, a recent study reported that 57% of the mutations would be missed if clinical characteristics alone were considered.27 By 2008, several third-party payers, including Medicare, had started reimbursing the costs of the assay. However, there was confusion about which billing codes to use, and concerns that, if the test were to be ordered within 14 days of an inpatient stay, it would be considered part of the diagnosis-related group bundled payment. These factors may have contributed to the observed institutional and regional differences in the use of the EGFR assay. A 2010 NCCN survey reported that only 21–41% of oncologists conducted EGFR mutation testing.28 Current ASCO and NCCN guidelines call for patients with non–small cell lung cancer, including adenocarcinoma, large-cell carcinoma, and “not otherwise specified,” to undergo EGFR mutation testing.24,25 This represents ~68% (i.e., ~138,462) of newly diagnosed cases of lung cancer per year.

Institutional and regional differences in the use of MDx testing are important factors to consider in translation of cancer genomics. Publications describing the incidence of EGFR mutations in black patients illustrate how differences in access to cancer genomics may exacerbate disparities in outcomes. Studies published in 2005 and 2009 analyzed 93 tumors from black patients and found a 2% incidence rate of EGFR mutations.29,30 This contributed to a misperception that blacks have a lower incidence of EGFR mutations as compared with Caucasians. Although blacks have the highest rates of incidence, morbidity, and mortality related to lung cancer, they have been underrepresented in lung cancer research. This is attributable, in part, to the fact that racial, regional, and socioeconomic factors are associated with the use of biopsies.16,31 Recent studies report that the incidence of EGFR mutations in blacks is similar to that in Caucasians.32,33

This example demonstrates the need for empirical, population-level data to inform the translational research process. By examining factors associated with hospital use of the EGFR assay, we aimed to inform the development of a model to predict T3 translation of cancer genomic applications.

Materials and Methods

This was a retrospective cross-sectional study. We merged one proprietary data set with seven public data sets. Genzyme Genetics, which owned the rights to distribute the EGFR assay, provided a data set containing name, city, state, zip code, and number of EGFR assays sold to each hospital, laboratory, or outpatient clinic in the United States during calendar year 2010. Public data sets included:

  • Centers for Medicare and Medicaid Services (CMS) Provider of Service file, which contained hospital characteristics including academic medical affiliation and services offered.

  • CMS CLIA specialty file, which identified accreditation to conduct cytogenetic testing.

  • NCI Provider of Services file, which identified hospital participation in NCI clinical research cooperative groups.

  • Census population file, which provided county-level sociodemographic indicators.

  • National Program of Cancer Registries and Centers for Disease Control and Prevention State Cancer Profiles, which provided county-level data on lung cancer incidence.

  • National Institute of Standards and Technology file, which provided zip code, longitude, and latitude.

  • Federal Information Processing Standards file, which provided the county code.

We aggregated and summarized orders at the hospital level, recording each hospital’s Medicare provider number from the CMS Provider of Service file. The aggregated data set was merged with public data sets, using hospital number, zip code, and Federal Information Processing Standards code as unique identifiers for each US hospital and county. The unit of observation for multivariate analysis was the nonfederal acute-care hospital. The outcome variable was whether a hospital ordered the EGFR assay, and this was treated dichotomously and coded as 1 if one or more assays were ordered.

Most of the independent variables were dichotomous and were coded as 1 if the hospital or region had that particular characteristic. The number of cases of lung cancer per county, the percentage of blacks in the county population, and the distance between the hospital and an NCI CC were treated as continuous variables. The distance between the hospital and the nearest NCI CC was measured in miles, calculated using the longitude and latitude of each institution. In order to interpret the odds ratio (OR) of distance accurately, we obtained the logit coefficient and multiplied it by increments of 100 miles.

The data set contained orders for the assay from 27 of the 60 NCI CCs. In addition, several NCI CCs also confirmed the use of LDTs to identify EGFR mutations. For consistency, these orders at NCI CCs were excluded from the data set. We also examined the CLIA specialty file to evaluate the potential use of LDTs by other hospitals.

Statistics

We conducted univariate and bivariate analyses, including t-tests for continuous variables and χ2 for categorical variables. Statistically significant explanatory variables (P values <0.05) were included in the logistic regression model. The number of cases of lung cancer and the percentage of blacks, both of which were not statistically significant in bivariate analysis, were retained as control variables. We performed multivariate logistic regression to identify characteristics associated with the ordering of an EGFR assay by a hospital. We performed sensitivity analysis to ensure that the observed effects were not attributable to an artifact of modeling. This process included using different cutoff points for independent variables and estimates of LDT use. The statistics program STATA version 12.0 (STATA, College Station, TX) was used in the analyses.

Results

In 2010, 743 institutions ordered 7,958 EGFR assays ( Table 1 ), representing 5.7% of newly diagnosed, guideline-directed patients. Nonfederal acute-care hospitals accounted for 76% of the tests ordered. Federal hospitals, NCI CCs, independent pathology laboratories, outpatient ambulatory cancer centers, and physicians’ offices accounted for the rest of the tests. Of 4,781 acute-care hospitals in the database, 12% ordered the assay; 148 of the hospitals ordered only one assay.

Table 1 Number and type of institutions ordering EGFR assay
Full size table

Four NCI CCs had contracted with Genzyme to be exclusive providers of EGFR testing. These orders (588) accounted for more than half of the total NCI CC orders. We used these data to develop an estimate of EGFR mutation analysis conducted by NCI CCs (Supplementary Table S1 online). Although it was not possible to estimate use of LDTs by non-NCI CCs, analysis of the CLIA specialty file revealed that only 4% (203) of US hospitals are accredited for cytogenetic testing. Of these 203 hospitals, 52 were NCI CCs. Of the 151 non-NCI hospitals accredited to conduct cytogenetic testing, 37 (18%) had ordered the EGFR assay.

State-wise analysis

Table 2 shows a state-wise summary of the number of hospitals, NCI CCs, cytogenetic hospital laboratories, and EGFR assays ordered. The last column shows estimates of the percentage of lung cancer cases for which the EGFR assay was ordered. In total, ~4.4% of guideline-directed patients accessed the Genzyme EGFR assay through acute-care hospitals. Estimates suggested that ~5.3% of patients accessed EGFR mutation analysis at NCI CCs, many of which were likely to have been carried out through LDTs.

Table 2 State-wise summary of access to EGFR mutation analysis
Full size table

No hospital in Alaska ordered the assay. One hospital each in Arkansas, Montana, New Mexico, South Dakota, Vermont, and Wyoming ordered the assay. California, Florida, Illinois, New York, and Pennsylvania had the greatest number of hospitals ordering the assay. These are also the most populous states and have the highest number of cases of lung cancer. Between 4.4 and 10.52% of guideline-directed patients with lung cancer in these states were tested for EGFR mutations using the Genzyme assay. These states also have the most hospitals accredited for cytogenetic testing. Patients may have undergone EGFR mutation analysis through LDTs. North Dakota had the highest percentage (17.6%) of cases of lung cancer tested, despite having neither an NCI CC nor a hospital accredited to conduct cytogenetic testing. However, three North Dakota hospitals that ordered the assay had academic medical school affiliations, participated in NCI cooperative groups, offered cardiothoracic surgery, and were located in metropolitan counties with above-average education and income. These were all characteristics that we hypothesized as predicting the likelihood that the assay would be ordered.

County analysis

There are 2,496 counties with acute-care hospitals. Hospitals in 349 (14%) of these counties ordered the EGFR assay; 96 counties (4%) have hospitals accredited to conduct cytogenetic testing; and 50 (2%) have NCI CCs. Most hospitals that ordered the EGFR assay (522 hospitals, 88%) are located within metropolitan counties; 497 (84%) are located in counties with populations having above-average education levels, and 511 (86%) are located in counties with populations having above-average income.

Diffusion of the trend toward usage of the EGFR assay appeared to emanate from NCI CCs ( Figure 1 ). This made NCI counties particularly relevant. In NCI counties, 19% of hospitals ordered the assay versus 11% of those in non-NCI counties. NCI CCs are located in metropolitan counties with above-average education and income levels in the population. Education and income differences between NCI and non-NCI county populations are mirrored in the results of comparing “EGFR counties (EGFR counties had a hospital which ordered an EGFR assay)” with “non-EGFR counties (In non-EGFR counties, there was no hospital which ordered the assay).” In EGFR counties, 24.4% of the population had a bachelor’s degree education, and 21.2% of the households had incomes of at least $75,000 annually. In non-EGFR counties only 16% of the population had a bachelor’s degree education, and only 13.3% of the households had annual incomes of at least $75,000. There were also race-related differences between NCI and non-NCI counties. NCI CCs are located in urban areas where 20% of the population is black. In non-NCI counties, only 9% of the population is black. There were fewer race-related differences between EGFR counties and non-EGFR counties. In EGFR counties, 12% of the population was black. In non-EGFR counties, only 9% of the population was black.

Figure 1
figure 1

Acute-care hospitals that ordered the EGFR assay in 2010. EGFR, epidermal growth factor receptor; NCI, National Cancer Institute.

Full size image

Bivariate analysis of hospital characteristics

The availability of cardiothoracic surgery appeared to be the strongest predictor of the ordering of the EGFR assay at a hospital; 56% of hospitals that ordered the assay reported having this service ( Figure 2a ) versus 21% of hospitals who did not order the assay. Only two hospitals designated by Medicare as Critical Access ordered the EGFR assay. These types of hospitals are usually located in rural areas.

Figure 2
figure 2

Hospital characteristics by status of ordering EGFR assay.(a) Hospital characteristics by status of ordering the EGFR assay. (b) Membership in cooperative group by status of ordering EGFR assay. EGFR, epidermal growth factor receptor; NCI, National Cancer Institute; PET, positron emission tomography.

Full size image

It was hypothesized that cooperative groups participating in EGFR TKI trials would have a higher percentage of hospitals ordering the assay. Figure 2b illustrates that the American College of Surgeons and the Cancer and Leukemia Group B had the highest percentage of members (36%) who ordered the assay. However, the differences among cooperative groups in this respect were relatively small. The National Surgical Adjuvant Breast Project, the largest cooperative group with 398 hospital members, had the greatest number of members ordering the EGFR assay, possibly reflecting the established role of molecular markers in breast cancer.

Prediction model

The relationships between each of the independent variables and the likelihood of an EGFR assay being ordered persisted after multivariate logistic regression analysis. Table 3 summarizes the ORs and 95% confidence intervals for each variable. Significant institutional predictors of ordering the assay included affiliation with an academic medical center (OR = 1.48), participation in an NCI cooperative group (OR = 2.06), and availability of positron emission tomography scan services (OR = 1.44) and cardiothoracic surgery services (OR = 1.90); the availability of positron emission tomography scan services (which are not routinely used in lung cancer) may be a proxy for early adopters of technology.

Table 3 OR and CI values for characteristics of hospitals that ordered the EGFR assay
Full size table

Significant regional predictors included metropolitan county (OR = 2.08) and above-average education and income levels (OR = 1.46). Distance from an NCI CC was a negative predictor of the likelihood of the assay being ordered. Analyzed independently, the logit coefficient of distance was −0.008 (OR = 0.992). To interpret the coefficient, we multiplied it by 100 (miles) and calculated the exponential, arriving at a value of 0.449. Hospitals located between 100 and 500 miles from an NCI CC had a 55% lower likelihood of ordering the EGFR assay; hospitals located ≥500 miles from an NCI CC had a <2% likelihood of doing so.

In the multivariate analysis, the effect of distance was reduced but remained significant (OR = 0.996, −0.004 logit coefficient). Holding hospital and regional characteristics constant, hospitals located between 100 and 500 miles from an NCI CC had a 33% lower likelihood of ordering the EGFR assay. When the distance was ≥500 miles, other factors being held unchanged, the likelihood decreased to 14%. The annual incidence of lung cancer cases, inpatient chemotherapy, and race were not statistically significant predictors of the assay being ordered.

Discussion

This is the first study to use empirical data to analyze dissemination of the use of the EGFR assay. In 2010, EGFR testing was underutilized; only 12% of nonfederal hospitals ordered the assay. This study supports recent findings showing that institutional and regional characteristics predict hospital use of innovations in cancer care.11,34 The total number of lung cancer cases in the county did not predict the rate of testing or whether hospitals ordered the assay. The significant underutilization of the EGFR assay emphasizes the need for increased investment in research at the stages of dissemination and implementation (T3) and health outcomes (T4).

Although the EGFR assay was commercialized in 2005,18,19 routine testing was not recommended by ASCO and NCCN guidelines until 2011. This was an important factor that impeded dissemination. Underutilization of the test was therefore an expected finding.

Advances in biomarker clinical trial design and in regulatory and reimbursement processes have improved the pace at which lung cancer MDx tests are being included in guidelines. These changes may also improve dissemination of the use of MDx testing. In 2011, under the US Food & Drug Administration companion diagnostic policy, the Vysis ALK test (for the anaplastic lymphoma kinase fusion gene) was approved in conjunction with crizotinib. ASCO and NCCN incorporated the ALK test into guidelines almost immediately after Food & Drug Administration approval. Third-party payers implemented policies requiring evidence of ALK testing for coverage of crizotinib. Our preliminary research suggests that these policies have had a significant impact on the rate of dissemination of ALK testing. In the first 5 months after approval of crizotinib, 12,000 patients underwent ALK testing.35 Although the use of lung cancer MDx tests has increased, continued monitoring is needed to ensure that patients in community and critical-access hospitals are getting access to the tests.

There are several challenges associated with research of this type. It is difficult to achieve accuracy in identifying patients who underwent testing. Insurance claims and electronic medical record data often do not consistently capture the use of genetic testing. Hospitals and laboratories use several different billing codes for similar tests. CMS has attempted to address this problem by issuing a single code for each specific genetic test. However, when genetic tests are ordered after a surgical procedure or during an inpatient stay, the claim continues to be bundled with the diagnosis-related group payment, making it very difficult to identify the use of genetic testing in the Medicare population. Electronic medical records may provide a more accurate source of data. However, within the Veterans Health Administration, for instance, the rapid pace of development of genetic testing has made it difficult to implement timely updates to the electronic medical record so as to accurately capture the use of specific genetic tests. The result is that providers use nonspecific test descriptors such as “DNA analysis.” When a laboratory or a test manufacturer holds an exclusive license for a test, as Genzyme Genetics did in the United States in 2010, the company may be a good source of data. However, many diagnostic companies license their tests to numerous commercial reference laboratories. Abbott Molecular has licensed its ALK test to >50 commercial reference laboratories, and this has made it difficult to capture all test orders.

The use of LDTs presents even greater challenges. There are an increasing number of hospitals and laboratories acquiring CLIA accreditation to conduct cytogenetic testing. Development and validation of LDTs are active areas of research. As the use of LDTs becomes more predominant, it will become increasingly important to understand and monitor the use and quality of LDT testing. Until there is more certainty surrounding regulation, oversight, and validation, LDTs will continue to be a confounding factor in T3 and T4 research.

Despite these challenges, given the potential improvements in health outcomes that genetic tests offer, timely T3 and T4 research is needed in order to increase the evidence base. This study provides an example of potential public–private collaborations that can inform implementation of guideline-directed genetic testing.

Our data set has several limitations. Hospital data do not capture patient-level clinical variables that may explain differences in patterns of use of the tests. Although this is an inherent limitation of hospital data, these data are not protected by the Health Insurance Portability and Accountability Act, thereby allowing timely identification of areas in which evidence-based recommendations for genetic testing have not been implemented. The data set did not capture laboratory-developed EGFR mutation analysis conducted as part of clinical trials or by hospitals accredited for cytogenetic testing. With ~85% of patients with cancer being treated in community hospitals and <2% of patients with lung cancer being enrolled in clinical trials,36 we believe that only a very small percentage of patients have access to LDTs. However, from the available data, it is impossible to know for certain how many patients actually accessed LDT testing.

A robust T3 research agenda requires analysis of population-level claims data, examination of medical records, and interviews with various stakeholders so as to elucidate factors that influence dissemination and implementation. Although T3 research is rare in cancer genomics, it is more robust in other areas of medicine and public health.17 There is very little published research work relating to the T2 phase and beyond. Investment has been focused on commercialization of innovations without assuring that those innovations are translated to patient care. Even the currently well-established genetic test for BRCA1/2, which was developed in 1994, was recommended by the US Preventive Services Task Force only in 2005. Disparities in access to BRCA1/2 counseling/testing persist.12 Real-world outcome data have been slow to develop.37 Recent ENCODE publications38 illustrate that rapid development of genomic applications in T0/T1 continues. Since September 2009, CancerGEM Kb, a Centers for Disease Control and Prevention NCI collaborative database, has tracked more than 270 newly developed tests.39 Additional T2 research is sorely needed to evaluate these applications. For tests with demonstrable clinical utility, T3–T4 research is needed to achieve maximum positive impact on population health. Ongoing study of factors that accelerate or impede translation is essential for the success of cancer genomic medicine.40

Cancer genomics is maturing.4 A growing number of clinically useful molecular tests are being developed. Technologies allow for higher-throughput genomic screening at competitive pricing. Academic medical centers are implementing next-generation sequencing in routine cancer care. As T0–T2 innovations are commercialized, guidelines need to be developed to encourage hospitals and manufacturers to make utilization data available to facilitate research in dissemination, implementation, and health outcomes. A robust translational research agenda will increase the pace of dissemination and decrease disparities in access that currently exist for patients seeking care in community and critical-access hospitals, while improving quality and outcome analyses.

Disclosure

G.A.M. reports that he is currently vice president and head of Strategy, Portfolio and Alliance for Personalized Healthcare & Biomarkers at AstraZeneca Pharmaceuticals. He was formerly vice president and general manager of a division of Genzyme ­Genetics, Genzyme Analytical Services. All other authors declare no conflict of interest.