Main

Genetic carrier and diagnostic testing for diseases prevalent in the Ashkenazi Jewish population have been available for many years.1 Initially, diagnostic testing was exclusively performed by enzymatic analysis,2 and accurate carrier testing for Tay–Sachs disease (TSD) was also developed using enzymatic analysis. This led to a significant decrease in the incidence of TSD in this specific population. However, molecular genetics–based targeted mutation testing for disorders besides TSD that are also prevalent in this population provided additional carrier-screening options, and thus, carrier testing was expanded.3

Twice per year, the College of American Pathologists (CAP) provides proficiency testing (PT) surveys for molecular genetic disorders. This includes the MGL-4 survey, which focuses on inherited genetic conditions that are relatively prevalent in the Ashkenazi Jewish population; the survey originally focused only on the disorders initially recommended (TSD, Canavan disease (CD), and familial dysautonomia (FD)),4 but additional disorders have recently been added. Approximately 90% of the 5.7 million Jews currently residing in the United States have Ashkenazi Jewish ancestry3 and are therefore at risk of being carriers of one or more of the founder mutations typical for these conditions. Such common mutations account for the vast majority of sequence changes in the affected genes that are identified in this population.

The American Congress (formerly College) of Obstetricians and Gynecologists has recommended population-based carrier screening for four serious inherited disorders that have an increased prevalence in Ashkenazi Jewish individuals: cystic fibrosis, TSD, CD, and FD. This recommendation was founded on a carrier frequency of at least 1 in 40 for each condition.4 Cystic fibrosis screening is offered separately to the general population and is recommended for all individuals who consider having children.5 The three remaining diseases are characterized by neurologic deterioration, intellectual disability, and premature death. For TSD, this usually occurs within the first few years of childhood,6 whereas for CD, the course is more variable but also eventually fatal.7 FD is a multisystem condition that primarily affects the nervous system and results in insensitivity to pain and temperature, periodic vomiting, hypertension, and lack of coordination. These patients typically have unaffected intellectual abilities.8

Existing data from the CAP/American College of Medical Genetics and Genomics (ACMG) MGL-4 PT survey from 2006 through the first distribution of 2013 were analyzed. In addition to estimating the analytic sensitivity and specificity of genotyping, we evaluated the postanalytic interpretation of those genotypes (e.g., “normal noncarrier” through “affected”). This information can provide evidence of overall molecular laboratory performance in identifying TSD, CD, and FD.

Materials and Methods

The MGL-4 survey

The ACMG and CAP jointly sponsor external PT for targeted molecular analysis of the common mutations found in the Ashkenazi Jewish population for TSD, CD, and FD. PT for Gaucher disease, mucolipidosis type IV, Niemann–Pick disease type A, glycogen storage disease type 1A, Bloom syndrome, and Fanconi anemia type C were added in 2012, but insufficient data exist for the analysis of these mutations. Three challenges were distributed twice per year. Members of the CAP/ACMG Biochemical and Molecular Genetics Committee selected samples of human cell lines derived from patients with specific genotypes for the analysis of each of these disorders. The selected cell lines were grown, and each participating laboratory was provided with isolated genomic DNA. Fortunately, only a small number of mutations have been identified, accounting for at least 94–97% of the affected alleles for each of these three disorders in this population ( Table 1 ).6,7,8

Table 1 Details of the three single-gene disorders included in the MGL-4 external proficiency testing survey
Full size table

Sample challenges and grading

For each PT distribution, participants receive three samples and perform the testing for the condition(s) included in their test menu. Because of the American Congress of Obstetricians and Gynecologists recommendations, laboratories will typically offer testing for all three disorders. The results of the PT challenges are reported to the CAP, where the data are reviewed, analyzed, graded, and summarized by CAP staff and members of the Biochemical and Molecular Genetics Committee. Laboratories are graded on the accuracy of the identified genotype as well as on the postanalytic interpretation and phenotypic consequences of their results. PT is a critical parameter for quality assessment/assurance, and suboptimal survey performance requires investigation and corrective action. Laboratories receive a warning from the CAP and will be at risk of losing their CAP accreditation if they fail multiple surveys.

Collation and analysis of survey results

Data were reviewed from 2006 through the first distribution of 2013 for TSD, CD, and FD and were stratified by the participant’s location (United States, international). Nonclinical laboratories (i.e., those that listed themselves as research laboratories) were removed, along with participants who did not report since 2008 or earlier, assuming that these laboratories were no longer performing clinical testing. Laboratory responses for the Ashkenazi Jewish disorders listed above were submitted via written or electronic reporting to the CAP. Each laboratory was assigned a unique internal identifier. Responses were collated by the CAP, and data from the multiple years were summarized by identifier and provided to the Biochemical and Molecular Genetics Committee in a blinded manner via an Excel spreadsheet (Microsoft). The results reported by each laboratory for the genotype (normal, heterozygous, compound heterozygous, or homozygous mutant) for each of the proficiency survey samples were analyzed. In addition, the biannual participant summary reports provided by the Biochemical and Molecular Genetics Committee to participating laboratories were reviewed. The number of samples tested per month and the estimated turnaround times were provided by participants. The percentage of responding laboratories reporting the correct genotype result for each sample and the analytical sensitivity and specificity were calculated. The analytic interpretations were also analyzed. However, for a limited period, a few laboratories did not report analytic interpretations when mutations were present because a specific commercially available platform was unable to differentiate a single mutation (heterozygote carrier) from two identical mutations (homozygous affected). In real-time patient testing, these laboratories either (i) accepted samples only for carrier (not diagnostic) testing and therefore would not have expected to identify affected (homozygous or compound heterozygous) individuals or (ii) referred such samples to a reference laboratory for confirmatory genotyping. Because that platform is no longer commercially available, that is no longer an issue. The 95% confidence intervals (CIs) were calculated using the binomial proportion. Two-by-two contingency tables were analyzed by the Fisher’s exact test. Statistical significance was two tailed at P ≤ 0.05.

Results

Number of participants and tests performed, along with turnaround times

The number of total participants increased from 12 (11 from the United States) in 2006 to a high of 39 (33 from the United States) in 2009. In 2013, there were 33 participants (27 from the United States). The average numbers of samples tested for TSD over the past 2 years were computed and ranged from a low of 1 sample per month to more than 1,000 samples per month ( Figure 1a ). Turnaround times were longer for participants who tested 10 or fewer samples (10 of 24 had a turnaround time >14 days) than those for participants testing larger numbers of samples (1 of 15 participants had a turnaround time >14 days; P = 0.03). Of interest is the changing trend in numbers of tests performed over time ( Figure 1b ). Historically, this testing was restricted to the Ashkenazi Jewish population and, between 2006 and early 2011, the median number of TSD tests per month was 2,900 (with an annual rate of 35,000). Since then, there has been a consistent and large increase reported in each distribution such that in early 2013 the number tested per month was 12,500 (annual rate: 150,000). For CD and FD, these numbers were 12,100 and 12,000 per month, respectively . These increases are not due to earlier missing data or an increased interest in carrier screening in the Ashkenazi Jewish population but are traceable to a small number of laboratories reporting very large volumes of testing of non–Ashkenazi Jewish individuals using expanded panels of testing without regard to ethnic or racial background. Extrapolating these results suggests that at least 15,000 samples per month may be tested by 2015 (annual rate: 180,000). Overall, 98% or more of the sample testing was performed in US laboratories.

Figure 1
figure 1

Estimated yearly numbers of samples tested in the United States for Tay–Sachs disease from 2006 to 2013. (a) The average numbers of samples tested since the 2011-A distribution (logarithmic x-axis) versus the participants’ reported turnaround time in days (y-axis). The horizontal dashed line is at 14 days. Filled circles indicate international laboratories. Three participants are not shown; two did not provide numbers of samples tested, and the third did not provide turnaround time. (b) The estimated number of samples tested by participants per month (y-axis) in each distribution from 2006 through the first distribution of 2013 (x-axis). Between 2006-A and 2011-A, ~2,900 tests were performed per month. An abrupt increase began in late 2011, reaching12,500 samples per month by early 2013.

Full size image

Analytic sensitivity and specificity

Table 2 shows the Tay–Sachs genotype challenges and laboratory performance for each challenge from 2006 through the first distribution of 2013 for participants located in the United States. Data from challenges that were either homozygous affected (12) or heterozygous unaffected carrier (2) were used to compute analytic sensitivity, whereas the remaining wild-type unaffected normal challenges (30) were used to compute analytic specificity. A total of 321 sensitivity challenges occured with 312 correct responses, resulting in an analytic sensitivity of 97.2% (95% CI: 94.7–98.7%). All nine incorrect responses occurred in different laboratories. In four instances, the laboratory correctly identified that two mutations existed but incorrectly reported one of the mutations (e.g., reported c.1278insTATC/c.1421_1G>C, whereas the correct genotype was c.805G>A/c.1278insTATC). In three other instances, a homozygous result was reported as being heterozygous, and one heterozygous challenge was reported as wild type. None of these errors appeared to be due to a sample mix-up. However, one additional error (a compound heterozygous sample reported as wild type) did appear to be due to either a sample mix-up or reporting error. This mix-up was also involved in the only error among the 642 wild-type challenges, leading to an analytic specificity of 99.8% (95% CI: 99.1–99.9%).

Table 2 Tay–Sachs disease: external proficiency testing genotype challenges, participating US laboratory performance, and estimates of analytic sensitivity and specificity
Full size table

Analytic interpretation

Participants were also asked to provide the analytic interpretation (affected, carrier, and normal noncarrier). Among the 963 sensitivity challenges, there were 7 incorrect interpretations (99.3%; 95% CI: 98.5–99.7%). This is fewer than the number of genotyping errors because reporting an incorrect mutation is a genotyping error but may not result in an interpretation error. Thus, only five of the seven incorrect interpretations were associated with genotyping errors, always agreeing with the genotype. Since 2011, however, one participant from the United States has twice correctly genotyped samples as being compound heterozygotes but incorrectly reported the interpretation as being an “unaffected carrier.”

There were only six international participants in the MGL-4 survey, and their performance was similar to that of participants in the United States, with one exception. One participant was responsible for five analytic sensitivity errors. There were no other errors, so removing that one laboratory would result in an analytic sensitivity of 100% (50/50; 95% CI: 92.9–100%) and analytic specificity of 100% (102/102; 95% CI: 96.4–100%). No interpretation errors were identified.

Laboratory performance for CD testing

Table 3 shows the data relevant for analytic validity estimates for CD testing over the same time period within most of the same laboratories. Unlike the TSD challenges, in which most samples were either affected or normal, the CD challenges were all heterozygous carriers or normal. The analytic sensitivity for identifying a CD carrier was 98.8% (321/326; 95% CI: 96.9–99.7%). Among the five errors, only one was due to a sample mix-up. The corresponding analytic specificity was 99.7% (604/606; 95% CI: 98.8–99.9%). Of the two errors, one was due to a sample mix-up. Among the 919 clinical interpretations, only two errors occurred (99.2%; 95% CI: 99.2–99.9%). One had a correct interpretation (unaffected carrier), but the genotype was actually homozygous normal. The second was part of the sample mix-up.

Table 3 Canavan disease: external proficiency testing geno-type challenges, participating US laboratory performance, and estimates of analytic sensitivity and specificity
Full size table

Among the six international laboratories, there were no sensitivity or specificity errors (61/61, 100%; 95% CI: 94.1–100% and 113/113, 100%, 95% CI: 96.8–100%, respectively). No interpretation errors were identified.

Laboratory performance for FD testing

Table 4 contains the data relevant for analytic validity estimates for FD testing. Five of the 11 analytic sensitivity challenges were heterozygotes; the rest being homozygous or compound heterozygotes. The analytic sensitivity for identifying a FD carrier was 99.2% (246/248; 95% CI: 97.1–99.9%). Among the two errors, one was due to a sample mix-up. The corresponding analytic specificity was 99.7% (646/648; 95% CI: 98.9–99.9%). Of the two errors, one was due to a sample mix-up. Among the 882 clinical interpretations, only three errors occurred (99.7%; 95% CI: 99.0–99.9%). Two were correct interpretations of an incorrect genotype, and one was a correct genotype (homozygous 2507) that was reported as an unaffected carrier (no sample mix-up was involved).

Table 4 Familial dysautonomia: external proficiency testing genotype challenges, participating US laboratory performance, and estimates of analytic sensitivity and specificity
Full size table

Among the six international laboratories, there were no sensitivity or specificity errors (52/52; 100%, 95% CI: 93.1–100% and 135/135; 100%, 95% CI: 97.3–100%, respectively). No interpretation errors were identified.

Summary of assay methodology

The testing methods for all three disorders were nearly identical, and therefore only the methodology reported for Tay–Sachs testing was summarized. Seventeen laboratories reported their test methodologies as part of the 2013-A survey. Five participants (29%) reported using a bead array assay, four (24%) reported using a PCR-based assay, and three reported using sequencing (one of these used next-generation sequencing). Two participants reported using mass spectrometry, and one reported using microarray technology. The final two reported using “another” unspecified technology. There were too few laboratories and too few discrepancies to determine whether any one methodology was associated with a higher (or lower) analytic performance. All of the methodologies used were laboratory-developed tests; there are no US Food and Drug Administration–approved test platforms for these disorders.

Discussion

The information collected and presented here represents the first complete review of the external laboratory PT offered for TSD, CD, and FD. For carrier screening, molecular testing for TSD has essentially replaced the enzymatic methodology because of the ease of targeted mutation testing for multiple different disorders prevalent in the Ashkenazi Jewish population, making the performance of laboratories that provide such testing of paramount importance. Clinical Laboratory Improvement Amendments–certified laboratories must participate in external assessment, and although some laboratories may choose alternative methods, a large majority chose to participate in the CAP/ACMG MGL-4 survey. The analytic sensitivity and specificity over the 8-year-period is remarkably consistent, even though multiple different commercially available and laboratory-developed testing platforms were used. It is important to note that none of the methodologies used are approved by the Food and Drug Administration. This provides compelling data that laboratories provide accurate genetic test results using methodologies validated within each individual laboratory.

These survey results have also documented a recent trend over the past 2 years of increasing numbers of tests performed in the United States. The numbers of tests performed per month for these conditions increased by more than fourfold between 2010 and 2013. The rapid increase in the number of samples tested is probably due to “single-gene disorder panels” that are now being offered for carrier screening to a general population rather than to the target population of Ashkenazi Jews. The Tay–Sachs mutations included in the MGL-4 survey are, however, focused on those prevalent in the Ashkenazi Jewish population, which are not recommended for the general population. Laboratories offering panels for use in the general population would need to ensure wider coverage to establish relevant residual carrier risks following a negative test. However, such mutations are not currently included in this survey and are typically not available for PT.

Estimating a laboratory’s analytic performance through PT challenges is useful because the sample used has a known, correct genotype and interpretation. However, it should not be the sole measure of laboratory performance. For example, this type of analysis may overestimate the number of laboratory errors because the PT samples are not part of the routine testing and automated reporting present in most laboratories. Transcription and manual re-entry of data into the online CAP data system may introduce new errors. On the other hand, laboratories know that their performance is being assessed as part of these PT surveys and may, appropriately, exercise extra oversight that could lead to a reduction in the error rate compared with routine clinical testing.

Turnaround time is a critical component to a genetic test. Some laboratories have turnaround times that exceed 2 weeks. Long turnaround times would not be appropriate for carrier screening in a pregnant population, for which decision making is time critical. However, were a laboratory to perform only diagnostic testing, then turnaround time would be less critical. Thus, laboratories with larger numbers of samples tested (indicating that at least some samples are for carrier screening) would be expected to have quicker turnaround times. We documented that only 1 of 14 larger-volume laboratories had turnaround times greater than 2 weeks.

The results of this study are consistent with other previously published studies that reviewed similar CAP/ACMG PT results9,10 and demonstrate that US laboratories provide high-quality service for molecular genetic testing. These results should be reassuring to both consumers and public health advocates. In 2012, CAP PT for additional disorders prevalent in the Ashkenazi Jewish population was expanded to include Gaucher disease, Bloom syndrome, Niemann–Pick disease type A, glycogen storage disease type IA, mucolipidosis type IV, and Fanconi anemia type C. In the 2013-A challenges, between 20 and 25 participants reported results for these additional disorders. The performance of laboratories in identifying the genotypes and interpretations associated with those disorders will be the topic of a future report.

Disclosure

G.L.F., I.S., and E.L. are directors of diagnostic laboratories that perform mutation testing and participated in the CAP/ACMG Proficiency Testing Surveys for the disorders included in this report; G.L.F. is the president-elect of the ACMG; G.E.P. declares no conflict of interest.