The Cystic Fibrosis mutation “arms race”: when less is more

The implementation of population-based cystic fibrosis carrier screening in late 2001 represented the first application, at an all-inclusive, whole-population level, of molecular genetic testing. It also represented the product of 12 years of research, pilot studies, deliberation, and consensus building by the National Institutes of Health, the American College of Medical Genetics (ACMG), and the American College of Obstetricians and Gynecologists (ACOG). That long developmental time-span owed to the complexity of the gene, the large number (>1500) and heterogeneity of its mutations and variants, and ethical concerns about clinical variability of the disease and potential adverse psychosocial impacts.¹ Yet, its eventual launch was seen as a model for the thoughtful integration of preventive molecular medicine into routine primary care (in this case, predominantly obstetrics and family medicine) and an early fruit of the investment in genomic research. Indeed, it was both fitting and significant that the seven pilot studies, conducted in the mid-1990s, were funded by the National Center for Human Genome Research (now the National Human Genome Research Institute [NHGRI]) under the sponsorship of the Ethical, Legal and Social Implications program. These studies culminated in a consensus conference at NIH in 1997, which recommended offering cystic fibrosis (CF) carrier screening to all pregnant couples and those planning a pregnancy.^2,3 Details of exactly how such a program should be implemented were considered at a second consensus conference held in 1998⁴ and then worked out by a steering committee comprised of representatives from ACMG, ACOG, and NHGRI.⁵ Subcommittees were formed to work out the three essential prongs of the effort: (1) patient education and informed consent; (2) laboratory testing (including the minimum core panel of mutations to be screened), interpretation, and reporting; and (3) provider education. As is now well known, the second of these subcommittees recommended a universal (pan-ethnic) screening panel of 25 CFTR mutations, which met the dual criteria of known association with CF and having an allele frequency in the affected US population of ≥0.1%, based on data maintained by the Cystic Fibrosis Foundation and others. The detection rate of this panel in Caucasians of European descent (80%) and the other major racial and ethnic groups was presented in an appendix to the subcommittee's report⁶ for use in calculating residual risks in those who test negative, and other aspects of genetic counseling.

Recognizing that before these recommendations there was wide disparity in the number and identity of CFTR mutations tested by individual laboratories,⁷ with no single laboratory offering this precise panel, testing laboratories and reagent vendors were given several months to “ramp up” to this minimal requirement. Even then, there were challenges. With no FDA-cleared molecular test kits available at that time for any genetic disease, much less one as complex as CF, laboratories had been developing their own in-house methods. But, although these may have been adequate for four or six mutations, the prospect of developing a “home brew” assay for as many as 25 mutations (and corresponding normal allele sequences) was beyond the capabilities of most facilities. In addition, positive control samples for most of the recommended mutations, ostensibly required under Clinical Laboratories Improvement Amendment (CLIA) regulations for use in quality control, were not to be had for love or money; they were simply not available, even in the laboratories already testing for them.

Fortunately, the law of supply-and-demand soon intervened to provide solutions to both of these impediments. Perceiving a large market as CF screening was declared standard of care for the entire population, the first of any commercial consequence in the history of molecular genetics, reagent and equipment vendors quickly developed and began marketing test platforms that covered the 25 recommended mutations. Indeed, virtually overnight CF became the flagship test product offered by many established and start-up companies, employing techniques ranging from restriction endonuclease digestion and gel electrophoresis to oligonucleotide microarrays. These products were uniformly robust and freed the laboratories from having to develop their own assays from scratch, though until recently none was in the form of a complete kit. Instead, they were marketed as analyte-specific reagents, a sort of half-way point at which good manufacturing processes were used, but no clinical claims were made by the seller. Since then, two companies' products have gone through full FDA review, and more are expected. This is not surprising, since ACMG and ACOG had already taken up the challenge of establishing clinical validity of the targets, leaving the manufacturers with a purely analytic hurdle to surmount for the FDA. On the mutation control side, developmental projects funded by CDC have led to the accrual of positive human samples for all 25 mutations, which are now sold as a package by the Coriell repository.^8–10

The combination of the ACMG/ACOG recommendations and the availability of these products opened the floodgates, and CFTR mutation testing quickly grew from a boutique, esoteric operation (like many tests for monogenic disorders) to a routine service of mammoth proportions. Commercial reference laboratories in particular saw their test volumes grow from a couple hundred per month to many thousands per week, and their molecular genetic test sections suddenly became major contributors to the company's bottom line.

But because of this success and tremendous growth—with test volumes rivaling those of HIV viral load, a throughput previously unheard-of in clinical molecular genetics—CF carrier screening has now become a commodity, and a highly competitive one at that. This has taken the form of aggressive marketing and advertising, especially by biotechnology equipment and reagent vendors and large commercial reference laboratories, which is to be expected. Yet, experience to date suggests that all of these technical platforms and testing laboratories, including those located in academic settings, are performing well and delivering robust and accurate results, with little if any substantive difference between them.^11,12 The situation is perhaps analogous to the competition among the various statin drugs, which differ only subtly in their mechanism of action and efficacy, yet whose ubiquitous marketing efforts have grown in parallel with each successive consensus recommendation for ever-lower cholesterol targets and broader indications for the medications. Similarly, no sooner had the ACMG recommendations been issued than marketing aimed at distinguishing providers and manufacturers from one another appeared. But since it was difficult to argue for superiority based on quality alone, some other distinguishing factor had to be chosen. In the case of CFTR testing, it was the number of mutations screened, and the opening shot was fired in the very next issue of this journal following the one in which the recommendations had appeared,⁶ originating from one of the commercial reference laboratories that actually had a seat at the table when the ACMG panel of 25 mutations was agreed upon. This publication claimed appreciably enhanced carrier detection rates using an expanded panel of at least 64 mutations.¹³ And it represented the beginning of a rather unseemly mutation “arms race” that has continued to this day, in which testing laboratories and manufacturers have continually tried to one-up each other by claiming better coverage for Caucasians or other racial/ethnic groups with this or that expanded panel, beyond that which ACMG/ACOG had deemed standard of care.

Admittedly, the 25 ACMG-recommended mutations were designated as the minimum core panel for universal population carrier screening; laboratories were free to supplement it with additional mutations of their choosing if they could justify it for their own local situation using similar criteria that the subcommittee had applied. But an entire section in the ACMG recommendations⁶ dealt critically with these expanded panels, stating that they “should not be offered routinely” even to couples who test positive-negative in the initial screen (which are often the situations of most concern), let alone to all screened individuals as a first-tier test (which is how they are being marketed). And the reasons we gave then remain just as valid today, supplemented by others which could not have been known in 2001 but have since become evident from the vast nationwide experience in CF screening that has occurred since that time. These can be grouped under the following broad categories:

Clinical utility and quality of care

1. False sense of security

This was a primary argument in the original recommendation statement, in which it was felt that the additional yield of the existing expanded mutation panels, despite the claims, was unlikely to assuage much of the residual uncertainty or anxiety resulting from a negative screen on the first-tier test. Particularly in the situation in which one member of a couple has tested positive for one of the ACMG-25 mutations and the other negative, it was felt that arranging for and awaiting the results of the expanded panel would introduce even more potential anxiety, whereas the amount of relief upon testing negative even for 100 mutations would not seem to justify it, since the individual could still carry any of the other 1400 CFTR variants that had not yet been tested. Anxiety, or the potential avoidance thereof, was never a very convincing argument for any policy in CF screening, and was a major reason the subcommittee did not endorse the Wald couple screening model¹⁴; moreover, none of the seven pilot studies detected undue anxiety levels, even in positive-negative couples.

2. False sense of danger

This is the obverse of item #1 above. The implication that expanded panels offer more security is based on the premise that the standard panel offers more danger, indeed an unacceptable level of danger. This premise, too, is false. One must always keep in mind that we are talking about a screening test for a recessive disease. A missed CFTR mutation in a carrier screen of a pregnant woman with no family history of the disease does not carry anywhere near the peril of a missed BRCA mutation in a woman with a strong family history of breast/ovarian cancer. In contrast to that dominant disorder, the woman carrying the undetected CFTR mutation will never suffer from cystic fibrosis herself. Moreover, the odds are stacked decidedly in favor that her partner (even if not tested) is not a carrier. And even in the small chance that he is, there is only a 1 in 4 risk of having an affected child. Thus, it is quite debatable whether the minuscule fraction of unwanted CF births that would result from all the women who forego an expanded panel screen in favor of the smaller ACMG panel can justify the additional cost and the other downsides discussed here.

3. Uncertain allele frequencies and arbitrary selection of rare variants

As reviewed above, the original panel of 25 mutations was selected by virtue of their having allele frequencies at or above a 0.1% threshold in one or more databases. This is a rather low fraction, and its accuracy for any particular mutation cannot be assured. Indeed, slightly different values, say from 0.05% to 0.5%, could be seen for quite a number of the mutations in the different databases. The reason is simply that, given the relatively small number of individuals studied and the rarity of these alleles, 0.1% blends into the background noise of all the other rare alleles, laboratory errors, nomenclature problems, and so on. At that level, it could be argued that even some of the ACMG mutations are somewhat arbitrary, and could just as easily have been substituted with others at about the same frequency; indeed, since the original recommendation, a number of additional mutations having ≥0.1% frequency have been identified and would have qualified for inclusion if the same criteria were used now. In the 2004 revisitation of the panel recommendations,¹⁵ it was decided not to incorporate any of those additional mutations, and one could probably make a similar argument that the original panel criterion need not have been set so low, that ≥0.5% or even higher (which would have translated into a panel of 11 or 12 mutations) might have accomplished the same goals with less arduous hurdles for developing multiplex testing platforms and obtaining positive controls.

If allele frequencies become unavoidably inaccurate and “noisy” when one approaches the 0.1% level, it stands to reason that variants even more rare than those selected by ACMG will be even less accurate. Thus, if some of the ACMG-included mutations could be accused of being rather arbitrary, a large majority of mutations selected for expanded panels that go beyond the ACMG panel are, almost by definition, arbitrary. They were chosen because the testing laboratory happened to stumble upon one, or read about it in a research or clinical paper whose researcher or clinician author had likewise stumbled upon it. In other words, these are very rare events, arbitrary almost to the point of randomness as to when, or if, they are identified and reported in the literature. Most have very little known about them from a clinical perspective (see item #4 below), and there are undoubtedly others lurking out there which have not been discovered (or published) yet, but which may actually be of greater significance, either by allele frequency or clinical impact. In this way, the claims made for any particular expanded panel, as opposed to expanded panels in general, are somewhat disingenuous: who is to say that those particular extra 30 or 50 mutations are the next logically most important ones in the train, to the exclusion of others?

4. Paucity of genotype-phenotype correlation data

It has been clear since the cloning of the gene that CFTR is a very complex genetic element, replete with an ever-growing number of identified mutations and variants and subject to modification in its phenotypic effects by internal polymorphisms and distant gene loci. It has been a major undertaking just to characterize the molecular and functional effects of the more common mutations. When it comes to rare variants, especially those causing missense changes, much less is known. Often the clinical correlation is based on one patient or one of a few families. The potential for misattribution of effects and for false assumptions is manifest. There is no better illustration of this than the variant I148T, which was part of the original ACMG panel until it was shown, after much testing experience, to be a benign polymorphism not associated with any CF symptoms.^16,17 It was subsequently removed from the ACMG panel.¹⁵ Initial work suggested that it was linked to another, truly pathologic mutation, 3199del6,¹⁸ but subsequently even this turned out not to be the rule,¹⁹ and 3199del6 by itself was too rare to justify addition to the panel. Keep in mind that the I148T “mutation” was selected for the original panel only after 2 years of research, deliberation and vetting by an expert ACMG committee—and still it was misclassified. How many of the ultra-rare mutations now found on commercial expanded panels have been subjected to anywhere near this level of scrutiny? Some are included even though available evidence would suggest they are benign, mild, or at best variable (e.g., D1270N, D1252H, G662D, R117C); and for a great many others, little to nothing is known, except for the few CF patients who have been found to carry them, perhaps spuriously (unlike the current ACMG panel mutations whose haplotypes are now known, it is not even certain, when two of these rare mutations are detected in a symptomatic patient, whether they are in cis or trans). Mindful that the end result of this screening is often pregnancy termination based on finding two of these “mutations” in the fetus, it should give one pause.

5. Problems with ethnic mutation data

Much of the drive, and hence commercial claims, toward expanded panels has been that they will have a significantly better carrier detection rate in ethnic/racial minorities, since the original ACMG panel was presumably biased in favor of the European/Caucasian/Ashkenazi-Jewish mutations that were better known at the time. Although many of us would argue with that presumption, since the panel was developed by examining allele frequencies across all populations, one can appreciate the desire to enhance detection levels in certain minority populations, especially for certain localities where they may be over-represented (e.g., Hispanic-Americans in California ^20,21). However, it is not yet clear that such a thing is even possible. Hispanic-Americans are very diverse in their origins even within the United States, so how are they to be screened in a “targeted” way? Those who are of pure or mixed European descent (e.g., from Spain) are likely to do well with the standard ACMG panel, whereas those who are of largely Amerindian descent will not be covered adequately by any known set of mutations.²² Furthermore, will test sensitivity data for Hispanics in California be applicable to Hispanics in New York or Florida? A similar problem exists around the term “African-Americans,” which encompasses a genetically heterogeneous group as well. The major African mutation is already included in the ACMG panel, whereas the others are too uncommon to make a large difference in overall carrier pick-up rate.²³

Cost and access

6. Added cost

The false sense of security from expanded panels comes at a price: the added cost of sending a specimen out to a reference lab for the additional mutation testing, or the likely greater expense (though there are exceptions, since CF test costs span a wide range across the country) of testing a larger mutation panel up front. Given that this is a screening test, keeping the costs down, to better allow coverage and access, is paramount, even at the price of some reduction in test sensitivity. Such considerations apply to all population-screening tests, some of which result in much more significant reductions in sensitivity than the marginal degree we are talking about here.

7. Potential for monopolization

One problem with an arms race is that one side always ends up on top at any given time-point; i.e., it is not the most conducive way to assure a level playing field. Mention has been made of the large number of commercial products (whether FDA-approved or analyte-specific reagents) now available for CF mutation testing. Most of these were fashioned with the ACMG-25 panel in mind. Since these products are what give most laboratories, and hence most patients, access to this testing, arbitrarily raising the bar to 80 or 100 or 300 mutations has the effect of cutting out the vast majority of testing centers. The few, or perhaps only, testing center(s) capable of offering such expanded panels would in effect create a monopoly that could further damage cost constraints and limit access. Of course, laboratories are free to screen for as many mutations (beyond the standard panel) as they wish; the problem for society arises when they try to claim that their expanded panel should be the new standard.

Cost effectiveness

8. Dwindling predictive value

Owing to the low but inevitable error rate inherent in any laboratory procedure, test sensitivity in and of itself cannot be used to establish the clinical power or utility of the assay. While sensitivity is the attribute typically chosen as “bragging rights” for the marketing of expanded CFTR mutation panels, a more useful parameter is predictive value, in this case positive predictive value. This number refers to the proportion of individuals with a positive test result who in fact have the diagnosis (or in this case, carrier state) in question. Predictive value depends on the prevalence of the condition (or in this case, of the carrier state) in the population being tested, because it reflects the prior probability that the abnormality exists in the patient against the probability that the positive test result is due to a technical error.²⁴ As the prevalence of the abnormality becomes vanishingly low, as is certainly the case with many of the rare CFTR mutations in expanded screening panels, the probability that a positive result is due to a technical error (false-positive) begins to exceed the probability that the tested person is in fact a carrier (and the lack of positive controls for these ultra-rare mutations doesn't help matters). Like it or not, this is a mathematical property of every clinical test across the whole spectrum of laboratory medicine—almost a law of nature—and one can choose to ignore it in the testing of rare analytes only at one's peril.

9. Law of diminishing returns

At this point in our experience, five-plus years out from the launch of nationwide CF carrier screening, the arguments presented above concerning the low and arbitrary yield of expanded panel mutations are no longer just hypothetical; we are beginning to accrue empirical evidence to this effect, both anecdotal and published. In other words, we are now at the point where we can legitimately say that the increased sensitivity claims made by the vendors of expanded panels are belied by actual experience in the field thus far. Only a few published or accrued examples will be presented here; undoubtedly, there are many other anecdotal experiences from individual centers that have not been formally collated or presented yet. Tsongalis et al.²⁵ reported on 532 patient samples (510 for carrier screening, 22 for diagnostic testing) that had been sent out from their center (Dartmouth-Hitchcock Medical Center) for expanded panel testing of >80 mutations. A total of 8 mutations were identified in 29 patients, all of which would have been detected by the original ACMG-25 panel. They estimated that the additional cost spent by the institution for the expanded panel testing was >$80,000. Pratt et al.²⁶ monitored all CF test requests received at their commercial reference laboratory (LabCorp) over a two-year period. Of 13,821 specimens tested with their panel of 31 mutations (which is only slightly larger than the ACMG-25), 167 were referred out for expanded panel testing (>80 mutations) because of persistent diagnostic suspicion in those who were symptomatic or positive family history in those who were being carrier-screened. Of those, only 6 (3.6%) came back with a mutation not included in the 31-mutation panel. Considering that these specimens were all from patients with elevated risk to carry CF mutations, one can assume that the yield in a naíve carrier screening population with no family history would have been even lower, perhaps dramatically so. An even more comprehensive assessment was experienced by a commercial facility (Ambry Genetics) that routinely performs whole-gene scanning for CFTR mutation identification. In a sample of over 12,000 individuals tested by this method, only 32 of the mutations found beyond those covered by the ACMG panel would have been detected by a commercial 97-mutation panel, and most of these were seen only once or twice. Moreover, 11 mutations contained in the expanded panel have never been seen in this commercial laboratory's screen of the coding regions of about 25,000 CFTR genes (S. Keiles, personal communication). These data beg the following question:

10. Why not just go to complete gene scanning/sequencing?

Given that the commercially available expanded panels appear to render low additional yield and contain ultra-rare mutations selected based on arbitrary or spurious criteria, one could make the argument that any patient or couple so distressed about their residual carrier risk after testing negative for the ACMG panel should just skip the available expanded panels and proceed directly to full-gene sequencing. This service is indeed available in a number of centers, and clinical experience is substantial.²⁷ It is being used almost exclusively for diagnostic testing, since it is generally too expensive for carrier screening. But the accrued data from diagnostic cases can be extrapolated to reflect on the relative power of targeted carrier screening panels as well. For example, of 263 samples from patients who did not have both of their mutations picked up by the ACMG panel and were subsequently sequenced at a large reference laboratory (Quest), only 12 (4.5%) would have had both of their mutations detected by the commercial 97-mutation panel (C. Strom, personal communication). Two of these were compound heterozygotes for ΔF508 and D1152H, the latter of questionable phenotypic significance for targeted screening (see item #4 above), and indeed both patients were asymptomatic into adulthood (one was tested only because his partner was a CF carrier, the other presented late with acute pancreatitis). If these two cases were removed from the cohort, then the detection rate becomes 10 out of 261 (3.8%). Since the standard ACMG panel will identify both alleles in about two-thirds of CF patients and the vast majority of the other mutations are identified only by complete sequencing, one can extrapolate that extended panel screening will provide only an additional 1.3% yield. And in actual practice, it may be even less: of 1500 specimens sent to Ambry for diagnostic testing, 57 positives would have been detected by the ACMG-25 panel whereas only an additional two (0.1%) would have been detected on the 97-mutation panel; and of 1369 carrier screens, only 3 of the 67 positives not included on the ACMG panel (0.2%) would have been found by the 97-panel (S. Keiles, personal communication)—all others would have required scanning/sequencing of all coding regions of the CFTR gene (or a much larger targeted panel) to be detected.

No one is seriously proposing sequencing of all coding regions for screening asymptomatic individuals, primarily because the cost is too high for a population screening test and because interpretation of the disease-causing potential of the vast majority of rare CFTR mutations is not possible. Even if the cost were to be reduced, the sequencing or scanning approach would still detect rare missense variants of unknown clinical significance, which brings us back to the concerns of item #4. For that reason, especially, these comprehensive approaches have no place in routine carrier screening at this time, nor in the foreseeable future.

11. Why not expand to other genetic diseases?

If one accepts from the preceding discussion that much of the effort and expense invested in tracking down ultra-rare CF mutations is at best misdirected, the question arises of how these resources might be better spent. One obvious answer is to replace them with common mutations for other severe genetic diseases. Although these candidates will need to be vetted as well, one could make a case for significantly higher yield and public health impact from incorporation of mutations in the carrier-screening panel for hemoglobinopathies, spinal muscular atrophy, hearing loss, fragile X syndrome, and/or any number of autosomal recessive inborn errors of metabolism. This argument becomes even more compelling when proposing ethnic-specific screening panels.

12. Problems with ethnic-specific targeting

Aside from the problematic ethnic minority mutation frequency data (item #5 above), one should consider the social milieu in which this program is taking place. Is detecting every last ethnic-specific CF mutation really the major medical problem facing these groups? Might efforts at ethnic targeting backfire as they did notoriously in the sickle-cell screening program in the 1970s?²⁸ It has been known since the pilot studies that cost-effectiveness of a carrier screening program declines in proportion to the number of couples with an affected fetus who choose not to terminate the pregnancy (or even to be screened or tested).²⁹ Numerous surveys have documented these numbers to be much higher in the Hispanic-American and African American communities^30,31 because of cultural, religious, and economic factors, with specific adverse effects on the cost-effectiveness of CF carrier screening.^32,33 Until we have field data to counter these facts, it would seem premature to expend so much effort and expense, for uncertain ultimate benefit, in these carved-out populations.

Ethical and professional obligations

13. The “mutation arms race” is unseemly and unprofessional

We have seen in the previous paragraphs that the competitive marketing of ever-expanding targeted mutation panels lacks scientific rigor and is thus largely commercially driven. In this modern era of “evidence-based medicine,” it should be deemed unacceptable. Not only is it unscientific, but it presents an unprofessional and undignified outward face of the genetics community, from whence the initial recommendation for population carrier screening arose, to the clinician-providers and to the public at large. The providers on the clinical side—mostly obstetricians because the recommendation was for screening to take place in the prenatal setting—have enough trepidations and complexities to deal with in this program without getting barraged by competing CFTR mutation panels of all shapes and sizes. The medical genetics profession is not an airline, boasting that it has the most nonstop destinations from a given city, nor Baskin-Robbins with its expanding menu of ice cream flavors. Just because something is available does not mean that it should be routinely offered, and just because something is discoverable does not mean that it should be routinely investigated. We geneticists, owing to our familiarity with the human genome and the human proteome, should recognize, beyond other medical specialists, the infinite parameters of the human body, and we should be judicious in determining which ones to go after. We also have a responsibility to educate, and not obfuscate, our professional colleagues. Just because most obstetricians who have not studied the matter carefully may presume that an 80-mutation panel is three times more sensitive than a 25-mutation panel does not mean that we should deliberately capitalize on that misconception.

14. Standard of care

The previous 13 items have been laid out as arguments in support of the current⁶ and reaffirmed¹⁵ ACMG recommendation for a standard population screening panel of 25 (now 23) CFTR mutations that need not and should not be exceeded for most routine screening situations. None of them is absolute scientific fact, and each could be challenged with counter-arguments. But since the counter-arguments will have the same constraints, the laboratories, clinician-providers, and equipment/reagent vendors will still be left without a clear direction. Yet, in the final analysis, there is a clear direction: the ACMG recommendations themselves. For better or worse, these constitute the sole professional consensus statement in the field, carefully vetted and overseen through many years by a large panel of experts representing ACMG, ACOG, and NHGRI. With the buy-in of the clinical providers (ACOG), it is now recognized as standard of care, and would be deemed as such in any legal proceeding. We can argue about it, we can change it (through normal and established ACMG committee channels), we can even denounce it if we feel so inclined. But we should not misrepresent it to our colleagues or our patients through dubious claims and aggressive advertising. To do so undermines the thoughtful approach that was used to develop it, casts doubt in the public eye about the early fruits of genomic medicine, and bodes ill for the generation of similar consensus recommendations for population screening of other complex genes.