New quality assurance standards for rare disease testing

A primary tenet in the field of laboratory medicine is that research and clinical service activities should be kept separate both physically and operationally. This is because the quality assurance, record-keeping, and personnel requirements for clinical laboratory testing are very stringent, mandatory, and inflexible. Such conditions are deemed appropriate and necessary for assuring consistency, timeliness, and reliability of high-complexity assays directly affecting patient care, not to mention the potential for loss of laboratory accreditation associated with lax adherence and the risk of liability from resulting errors. On the other hand, such dogmatic and regimented attention to detail may be counterproductive for the research setting which thrives on inventiveness, imagination, and pliancy. The field of molecular diagnostics, by its very nature, has always represented a marriage of research advances translated into clinical applications. Clinical molecular genetics laboratories often work in close concert with basic molecular genetics research laboratories, and many molecular diagnostic laboratory directors are heavily involved in basic research themselves. As such, we have been especially cognizant of this need for separation and the difficulty we sometimes face in doing so.

There is one area of molecular genetic testing that is particularly caught in the middle ground between research and service: the detection of mutations in genes for ultra-rare disorders. Ultra-rare disorders have been defined in this guidance document as having a prevalence of less than 2000 affected individuals in the US population, although no formal statutory definition exists. Ultra-rare disorders are considered less common than rare disorders. Rare disorders have been defined by the Orphan Drug Act of 1983 as those that affect less than 200,000 individuals in the United States. Some examples include pantothenate kinase-associated neurodegeneration, infantile neuroaxonal dystrophy syndrome, and X-linked ocular albinism. Because of their rarity, test request volume is generally too low to be of interest to mainstream clinical molecular genetics laboratories, whether in the academic or commercial setting. Moreover, the involved genes are often studied by only one or a few researchers, and they are the only ones with the reagents, knowledge, and experience to perform the tests. This state of affairs is not surprising when one considers the typical evolution of genetic disease discovery. This typical chronology begins with the encounter by an interested clinical investigator of one or more patients with a particular genetic disease. Specimens (usually blood) are collected from these patients for diagnostic (e.g., biochemical) testing and disease characterization. At the same time, genomic DNA is isolated, and if sufficiently large families or numbers of families can be accrued, gene mapping studies by linkage analysis are commenced. With luck, the causative gene will be discovered and published. Often the very next publication from the laboratory describes a mutation survey of the sequence changes in all the patients whose DNA has been collected and stored. But in most cases these studies are not especially exciting or revealing, comprised predominantly of routine missense and nonsense variants that do not shed much light on the molecular mechanisms of the disease or the gene product. Naturally, the research laboratory would then prefer to move on to these more functional studies, but because the investigators may be the only ones in the world set up to do the mutation detection assay, the families keep returning to them with requests for carrier testing and prenatal diagnosis. Sometimes researchers are put under pressure by their own institutional review boards to reveal test results to study participants, but this simply reflects misunderstanding of the law (see below). Usually the researchers will accede to these requests, either out of altruism or a feeling of indebtedness to the families who assisted in the gene mapping. But because this is no longer a high priority activity of the laboratory, such requests are often assigned to lesser qualified members of the team (e.g., junior graduate students) or otherwise put on the back burner, leading to poor turnaround time and analytic or clerical errors.

Clearly, this situation is far from ideal and does not serve well either the patients or the researchers. And even beyond that, there are legal issues at hand. The Clinical Laboratory Improvement Amendments (CLIA) mandate that only a licensed clinical laboratory can issue test results to physicians or patients that will be used for medical management. Because the research laboratory does not have a CLIA license (or any other clinical certification), it is technically in violation of federal law every time it gives out a test result. Although the law is usually not enforced in a punitive way, especially for rare diseases where a “compassionate use” concern may override the legal concerns (just as for orphan drugs), it is not in anyone's best interest for the status quo to be perpetuated indefinitely. Still, what options are there at this point for the research laboratory? They range from continuing to test outside of CLIA (not desirable or even legal, as already described) to ceasing all testing (which leaves the patients out in the cold if this is the only laboratory offering the testing) to the research laboratory obtaining its own CLIA license (not easy or even feasible for most research laboratories). Alternatively, the research laboratory could try to partner with a clinical molecular laboratory in the same institution, as will exist in most large academic centers. But large clinical laboratories are not interested in rare disease tests that may generate only a few requests per year. At that rate, the validation and quality assurance efforts for the test will cost far more than the amount of income from the testing.

Obviously, none of these solutions is ideal. But there is one remaining alternative. The test could be set up and offered in a laboratory dedicated to rare disease testing. Such a laboratory would recognize that volume for this particular test will be low, but will be one component of a larger constellation of rare disease tests offered, which in aggregate can result in an economically viable (if not highly lucrative) operation. Laboratories of this type, unheard of just a few years ago, are now growing in number and have inspired external support from funding agencies, as described in the other articles in this volume. They represent a new direction for clinical molecular genetics services, and one which requires its own quality assurance standards—which by necessity will be somewhat different from those applied to high-volume molecular genetic tests such as cystic fibrosis and fragile X syndrome. The need for different standards was recognized years ago by the NIH-DOE Task Force on Genetic Testing which stated in its published recommendations:

CLIAC [the CLIA Advisory Committee] should consider developing regulatory language that is less stringent, but does not sacrifice quality for laboratories that only occasionally and in small volume perform tests whose results are made available to health care providers or patients.¹

In response to this need, the American College of Medical Genetics (ACMG), through its Laboratory Quality Assurance Committee, has developed such guidelines, and these were published last year.² The remainder of this article provides a summary of those recommendations (Table 1).

Table 1 Key recommendations for quality assurance in rare disease testing

WHY THE NEED FOR NEW STANDARDS?

Molecular genetic testing for rare disorders presents a number of challenges and obstacles to quality assurance efforts that are not readily met by the existing guidelines and programs. As mentioned earlier, clinical laboratories must be licensed under the CLIA program, but the standards promulgated therein are vague and minimal even for routine molecular diagnostics, so they are even less appropriate for rare disease testing. Virtually all reputable clinical molecular genetics laboratories subscribe in addition to a laboratory inspection and accreditation program with higher and more specific standards, namely that offered through the College of American Pathologists (CAP). This program encompasses biennial inspections based on a long list of criteria detailed in the Molecular Pathology Checklist.³ In addition, CAP offers jointly with ACMG a proficiency testing program in which unknown samples are mailed to participating laboratories twice a year, and the results are reported back to CAP where they are collated and graded. But despite the tremendous effort that has gone into setting up this program, only a small fraction of the universe of testable genetic diseases is included in these challenges. By necessity, owing to marketing and financial considerations, they are limited to the more popularly tested molecular genetic disorders (e.g., cystic fibrosis, Duchenne muscular dystrophy, fragile X syndrome, factor V Leiden), because a critical mass of participating laboratories is necessary for the program to support itself. Needless to say, ultra-rare disorders tested by only one or a few laboratories will never be included in this formal program.

The absence of formal proficiency testing is just one of the challenges facing rare disease laboratories. There is also a dearth of positive mutation controls needed for basic quality control, simply because the patients and mutations are so rare. Hence, there is a need for alternative approaches. The goal in writing the ACMG guidelines in this area was to develop alternative approaches that were feasible and practical for esoteric, low-volume laboratories, reducing unreachable benchmarks where necessary, but without sacrificing quality. Whatever standards were developed had to be applicable across a wide, virtually limitless, number of target analytes (genes and mutations), as opposed to the recurring trinucleotide repeats and subsets of point mutations tested in the more common genetic disorders. With the translational nature of these rare disease tests, special attention to validation was needed, maintaining stringency while also encouraging efficient translation so that the tests can migrate out of the research laboratory and into the clinical setting. Informed consent, not usually required for most routine clinical tests, deserved attention because of the possible research component, at least during the migration phase. Lastly, there are unique aspects of test interpretation, reporting and counseling, because these tests are often treading on unexplored genomic territory, without well-characterized mutations and genotype-phenotype correlations. Thus, we are experiencing a paradigm shift in genetic testing, and we must be creative in our approach to assure quality of testing in a rapidly fluid environment.

DETERMINING WHEN A RESEARCH TEST IS READY FOR CLINICAL TRANSLATION

Our goal in all of this work is to bring more research tests into the clinical laboratory so that they will be legally available to the patients who need them. We want this transition to occur without undue delays, but not so fast that accuracy is lost. For a test to be considered ready for clinical application, it must have demonstrated analytic and clinical validity, and clinical utility. Determining analytic validity is the most straightforward, because the techniques used (primarily DNA sequencing but sometimes also including mutation-scanning techniques such as denaturing high pressure liquid chromatography) will be much the same as the molecular genetics laboratory uses for more routine tests. However, it may be hampered by the paucity of positive control samples available for pilot testing. A lower, though still reasonable, number of control samples may be acceptable under such circumstances. Alternatively, artificially synthesized mutation controls may need to be used, and there are a number of accepted strategies for doing so.⁴ Determining clinical validity may be much more difficult, simply because there is so little experience correlating molecular findings with an ultra-rare disease. It may require comprehensive review of the literature to make these correlations, and in any case will require close and continued communication with the research laboratory that did the first studies on the gene. Although it would be desirable to have multiple publications in agreement, often there is only one publication showing gene mutations to be disease-causing. That is a primary reason the research laboratory is considered a necessary collaborator on the funded test-development projects administered under the Collaboration, Education, and Test Translation (CETT) program. The research laboratory, for example, may be able to perform functional studies of the gene product to establish genotype-phenotype correlations. Nevertheless, there will always be more unknowns in rare disease testing than in routine testing. The clinical laboratory must examine all the available data to decide whether the potential benefits of offering the testing outweigh the risks caused by unknown technical and interpretive factors. The CETT program also strives to synchronize and centralize the sites of rare disease testing, to avoid needless duplication and to have specific tests based at those centers possessing the greatest expertise and experience (see page 343, this issue).

ALTERNATIVES TO FORMAL PROFICIENCY TESTING

Orphan disease testing laboratories need to be CLIA-certified and CAP-accredited just like other high-complexity molecular diagnostic laboratories. For the latter, they will be inspected using the CAP Molecular Pathology Checklist, but the inspector will need to be familiar with the ACMG rare disease testing guidelines concerning areas where they diverge from routine molecular diagnostic practice, such as test validation requirements and proficiency testing. Fortunately, CAP already recognizes the need for alternative approaches to proficiency testing, because no program can encompass every possible test that is offered. These alternatives include informal blinded sample exchanges with another laboratory, blind reanalysis of previously tested samples in the same laboratory, comparison of results with a core sequencing facility or commercial sequencing provider, or possibly the development of a generic sequencing-based proficiency challenge in which an anonymous DNA fragment, not necessarily representing the disease gene in question, is used for the challenge. Regarding internal or outside sequencing vendors, it goes without saying that these resources can only be used for back-up proficiency evaluation, not for the actual testing because they are not CLIA-certified.

SPECIAL TECHNICAL CONSIDERATIONS

Because rare genetic disorders also have rare mutations, in most cases the mutation spectrum to be addressed will be broad and largely unknown. There will not, in most cases, be a subset of recurring “common” mutations that can be used as the test platform, as is now done for cystic fibrosis carrier screening. Instead, each new family being tested will represent a clean slate and a brand new search for unknown mutations. Thus, the vast bulk of rare disease molecular testing will be done using DNA sequencing, which is the only technique capable of detecting (almost) all possible mutations, at least in the regions being examined (i.e., exons). It is acceptable for sequencing to be preceded by a mutation scanning technique such as denaturing high pressure liquid chromatography as long as the laboratory has validated the sensitivity of that technique and the ability to subsequently confirm any anomalous results by DNA sequencing. As we now know from the familial cancer testing field, some proportion of patients may have large deletions rather than point mutations, and these can be invisible to DNA sequencing when in the heterozygous state. Therefore, the laboratory must have an alternate method for detecting such changes if they are known to be potentially present at a frequency that justifies the additional effort. In any case, it is prudent to include a cautionary note about this possibility in negative sequencing reports for those genes known to sometimes exhibit large deletions.

Positive mutation controls are not generally required for this work, because DNA sequencing is considered to be internally controlled based on the sequence visualized and the ready comparison to the published wild type genomic sequence. Also, it is not possible to obtain a positive control representing every potential nucleotide change. However, it is advisable to run a negative control sample so that the laboratory can ascertain the performance and profile of the wild-type gene sequence in comparison with any apparent abnormalities observed in the test sample. For those few cases where the gene may have mutation hotspots or recurring mutations and allele-specific test methodologies (e.g., hybridization) are used, positive controls for them should be included.

More so than in testing of well-established disorders, many of the mutations detected in rare disease testing are likely to be undescribed previously. When these are missense changes, the question always arises whether they represent true pathologic mutations or benign polymorphic variants. It is the responsibility of the testing laboratory, perhaps with consultation from the research laboratory, to try to make this distinction for purposes of accurate genetic counseling. This will not always be possible given current knowledge, but an attempt should be made to deduce the phenotypic effect of the change based on the nature of the amino acid substitution, the position of that residue in the protein product, the degree of evolutionary conservation of the codon sequence, the presence or absence of disease in other family members or control populations with the same variant, comparison with public mutation and sequence databases, and possibly functional studies in collaboration with the research laboratory. For most of this work, the standards for DNA sequencing as contained in the ACMG Standards and Guidelines for Genetic Laboratories⁵ and the ACMG guidelines for interpretation of sequence variants⁶ will apply. It is also important that newly discovered variants be reported (and published) using standard nomenclature⁷ so that clinical correlations can be made without confusion between laboratories and centers.

The immense clinical value of research publications and accurate locus-specific mutation databases that represent worldwide collections of data for rare disorders cannot be overstated. In June 2006, the Human Variome Project (HVP), dedicated to the worldwide collection of variation and corresponding phenotype for annotation onto the human genome, was launched in Melbourne, Australia. Some of the target goals of the HVP include the development and maintenance of locus-specific databases, development and validation of software tools to facilitate sequence variant interpretation, international standardization of nomenclature, and international collaborations between research and clinical laboratories to assist in test translation and quality assurance. The development of these tools will ultimately enable rare disorder clinical laboratories to better interpret sequence variants in the context of disease phenotype. Although the clinical laboratories benefit from the use of the publicly available databases, they also have a responsibility in return to submit their in-house collection of sequence variants along with any clinical phenotypic information as long as patient anonymity is preserved. Thus, we have included these comments in the rare disorders guidelines, and urge the CAP to include this requirement as a Phase I deficiency on their checklist for molecular pathology laboratories. Improvement in the accuracy of patient diagnosis for rare disorders through genetic testing and its clinical utility for such families will be one critical outcome of the HVP initiative.

“CUSTOM” MUTATION ANALYSIS

A somewhat different operational model used by some rare disease testing laboratories, or offered for a subset of disorders, involves confirmation of previously detected mutations in a patient or family, as opposed to comprehensive gene sequencing to search for unknown mutations. Often the purpose of testing is to ultimately perform prenatal diagnosis of a current pregnancy and predict whether or not the fetus will be affected by the particular disorder. Such testing is considered high liability for the laboratory and accurate diagnosis is critical. This approach has been designated in our guidelines as “custom” analysis. Most often it takes the form of confirming a mutation found in a referring research laboratory, so that the result can be released from a CLIA-certified facility. Although the validation process is less stringent than for mainstream high-volume genetic tests, it includes all components of validation and fulfills our quality assurance standards. In this case, the burden of clinical validation is deferred to the research laboratory, and often the interpretation of clinical test results is directly dependent on the research laboratory's input. The clinical laboratory is responsible only for analytic validation. The assay conditions, primer sequences, etc., may be obtained from the research laboratory, but the testing laboratory must demonstrate that the assay performs well in its hands. As for any allele-specific test method (even if it is done by sequencing), a positive control is required, preferably a sample from the proband in the family. And although the case is being referred from a research laboratory, it is important that the actual testing be done on a new patient sample sent directly to the clinical laboratory, just as it would for any clinical laboratory test.

PRE- AND POSTANALYTIC ISSUES

Most of the requirements covering requisition forms, sample collection and transport, genetic counseling, reporting, etc., will be the same for rare disorders as they are for the more common disorders, and are covered adequately in the general ACMG Standards and Guidelines.⁵ However, there are a few unique features for tests that are still in translation from the research to the clinical setting. Requisition forms may require more basic information about the gene and the clinical disorder, and the laboratory may find it necessary to have a greater interaction with the referring physician to collect additional clinical information. Family history and ethnicity information are also essential. Informed consent may be necessary if the test is still considered translational or not entirely separate from the research program. These aspects must also be reflected in the report, along with the best estimates at clinical correlation and potential impact of missense variants, as described earlier. It is recommended that even benign variants be reported, so that the community can accrue more data about the gene sequence and genotype-phenotype correlations. Alternatively, some laboratories have chosen to footnote the presence of benign variants and provide full information at the physician's request. Genetic counseling is obviously a crucial component of this activity and it is assumed that the testing laboratory will have access to this resource, though in most cases the pretest counseling will have occurred at the referral source. But in our experience there are many complicated logistics that arise in this setting, where each test request is unique, and a laboratory-based counselor can be very helpful in communicating with the referring physicians or research laboratory. Test reports must include a complete description of the method used and the limitations of the assay. The ACMG rare disease testing guidelines document contains sample reports that can serve as useful models.² These model reports can be adapted by clinical laboratories that are funded by the CETT program to a report style recommended by the Genetics and Synoptic Reporting Workgroup (see page 343, this issue).

CONCLUSION

The ACMG guidelines summarized here were developed to assist laboratories in developing, validating, performing, and reporting molecular genetic tests for ultra-rare disorders. This activity will not be appealing or appropriate for most high-volume laboratories, but there is clearly a need for a small group of dedicated facilities to offer these essential services for the community of rare disease patients and their physicians and collaborating research laboratories. It was for their sake, to make the job easier yet still maintain high quality, that these recommendations were written. We believe they will be of value beyond the rare disease setting, because many of the issues we grappled with can apply just as well to testing for rare or unusual mutations in more common disorders. Indeed, once high-throughput genome-wide sequencing becomes routine, every patient being tested will become a rare disorder patient, because all sorts of unexpected and previously undescribed sequence changes will be detected. We hope that the groundwork we have laid here will be a useful launch point for those even greater challenges.