Main

BRCA1 and BRCA2 (BRCA1/2) genes are associated with hereditary breast and ovarian cancer syndrome. The risk of breast and ovarian cancer varies with the gene involved and the type of mutation. BRCA1/2 carriers have an estimated 56–84% lifetime risk of breast cancer. For ovarian cancer, BRCA1 is associated with a 36–63% lifetime risk as opposed to 10–27% for BRCA2 carriers.1 Various mutations in BRCA1/2 specific to particular ethnic populations have been described. About 2.5% of Ashkenazi Jews carry one of the three founder mutations: BRCA1 185delAG (c.66_67delAG, p.Glu23fs), BRCA1 5382insC (c.5263insC, p.Gln1756fs), and BRCA2 6174delT (c.5946delT, p.Ser1982fs).2 Studies have shown that these three founder mutations represent 95% of all BRCA1/2 mutations identified in this population via DNA sequencing analysis.3 Testing for just these three mutations is substantially more affordable than gene sequencing and has also been free of the gene patent issues surrounding BRCA1/2 whole-gene sequencing, making it more accessible to clinical laboratories.4

As part of the College of American Pathologists (CAP) accreditation, molecular diagnostic laboratories that offer clinical testing are required to participate in proficiency testing (PT) twice annually to evaluate their performance. Grading of the PT is based on the accuracy of the genotype calls and the postanalytic interpretation. The data are collected by CAP and are then summarized by the CAP/American College of Medical Genetics and Genomics (ACMG) Biochemical and Molecular Genetics Committee. Since 2003, PT for the three Ashkenazi Jewish BRCA1/2 mutations has been offered. This report summarizes laboratory performance from 2003 through 2012.

Materials and Methods

The ACMG and CAP jointly sponsor biannual PT for three BRCA1/2 founder mutations in the Ashkenazi Jewish population. The PT surveys consist of three unknown samples distributed twice a year (designated A and B), for a total of six challenges per year. Isolated genomic DNA is provided to participants. The results are compiled by the CAP, and the committee members provide a summary of the data, intended responses, and educational material for each PT. This information is included in the participant summary reports that are returned to laboratories along with their score sheet.

For each sample, participating laboratories are asked to provide a genotype result and an interpretation on the clinical significance of those results; both elements are graded. The survey also captures additional information on a participant’s testing, including (i) whether they provide clinical testing (as opposed to testing in a research setting); (ii) the number of tests for the BRCA1/2 founder mutations performed each month; (iii) average turnaround time for clinical tests; and (iv) the testing methodology used. Based on the shipping address provided to CAP, participants were categorized as being within the United States or international. A unique identifier was created for each laboratory by CAP so that the authors did not have access to the location or identification of participants. Error rates, types of errors (genotype versus interpretation), analytical sensitivity, and specificity were computed. Confidence intervals were based on the binomial distribution. Comparison of rates was performed by Fisher’s exact test. Statistical significance was two tailed at P < 0.05.

Results

A total of 23 US and 39 international laboratories participated in the PT surveys at some time between 2003 and 2012. All participants indicated that they performed clinical testing for at least the three founder mutations in BRCA1/2. The number of laboratories participating in a single survey has more than doubled over time. In 2003, there were 14 subscribers, whereas in 2012, there were 38. There was almost a threefold increase in US-based participants in contrast to the sixfold increase in the number of international laboratories over the decade. The median number of samples tested per month was ~100 (range: 1–2,500). The turnaround time ranged from 3 to 225 days with turnaround times of ≤10 days reported for more than half of the US-based laboratories ( Figure 1 ).

Figure 1
figure 1

Laboratories participating in the CAP/ACMG proficiency testing survey for BRCA1/2 mutation testing: samples tested per month versus turnaround time. Each data point indicates the number of BRCA1/2 tests performed each month (logarithmic x-axis) versus their turnaround time, for a single participating laboratory. The laboratories located within the United States are represented by the small filled circles, and international laboratories are represented by the larger open circles. The horizontal dotted line indicates a turnaround time of 3 weeks. The small arrows indicate that the actual value is higher than represented. ACMG, American College of Medical Genetics and Genomics; BRCA1/2, breast cancer genes 1 and 2; CAP, College of American Pathologists.

Full size image

The testing modalities used by the 38 participants in the 2012B survey are listed in Table 1 . All participants reported using laboratory-developed tests, with the majority (76%) employing sequencing methods (Sanger) as their primary testing method. Other primary methods used were amplification coupled with electrophoresis with or without restriction enzyme digestion, mutation scanning, and next-generation sequencing (NGS). The secondary methods included multiplex ligation-dependent probe amplification, restriction fragment length polymorphism, gel electrophoresis, and NGS.

Table 1 Molecular testing techniques utilized by the 38 laboratories participating in the 2012B CAP/ACMG BRCA1/2 proficiency testing survey
Full size table

The clinical vignette for all of the PT samples was provided as being “obtained from individuals of Ashkenazi background with family history of early-onset breast cancer.” Sixty challenges were distributed and graded during the 10-year period. Thirty-eight were homozygous wild (wt/wt), seven were heterozygous for 185delAG/wt, nine were heterozygous for 5382insC/wt, and six were heterozygous for 6174delT/wt. There were 871 responses to “negative” challenges (wt/wt) and 484 responses to “positive” challenges (wt/mutation). Six genotyping errors were identified over the 10 years, with three identified in the first 5 years and three in the past 5 years ( Figure 2 ). There was no clustering of genotyping errors by time or by laboratory, with the exception of the two most recent errors that occurred in the same survey (2012B) by the same laboratory.

Figure 2
figure 2

The analytical and interpretative performance of laboratories participating in the CAP/ACMG proficiency testing survey from 2003 to 2012. Representation of (a) individual laboratory genotyping and (b) interpretation results. Each row represents 1 of the 23 laboratories located in the United States (upper rows) or 1 of the 39 international laboratories (lower rows). Each column represents a single distribution of three DNA samples. The PT surveys consist of three unknown samples distributed twice a year (designated A and B), for a total of six challenges per year. Each “box” indicates that the laboratory participated in the PT surveys as a clinical laboratory. The dark gray boxes indicate the surveys in which an individual laboratory participated; the light gray boxes indicate the surveys in which the laboratory did not participate; and the black boxes indicate errors. ACMG, American College of Medical Genetics and Genomics; CAP, College of American Pathologists; PT, proficiency testing.

Full size image

Among the six errors, five were false negatives. There were three instances in which a mutation was present, but it was not detected or reported by the laboratory. In another two, a mutation was reported, but it was not the one present in the sample (wrong mutation was detected). Specifics for these five errors are provided in Table 2 and Figure 2 . The analytical sensitivity among the 484 positive challenges was, therefore, 99.0% (479/484; 95% confidence interval: 97.6–99.7%).

Table 2 The intended and actual genotype result for all six genotyping errors identified in the CAP/ACMG BRCA1/2 proficiency testing survey
Full size table

One false positive also occurred. A mutation (5382insC) was reported when none was present in the sample ( Table 2 ). This was not likely to be a mix-up because no other sample in that distribution contained this mutation. The analytical specificity was 99.9% (870/871; 95% confidence interval: 99.4–99.9%).

In addition to correctly genotyping the samples, participants are also graded on the interpretation of the results. Three intended responses have remained unchanged over the 10 years: (1) lifetime risk of breast cancer cannot be determined without BRCA mutation testing of the affected relative; (2) lifetime risk of breast cancer is the same as that in the general population; and (3) no risk of breast cancer. The fourth option (2003 through 2011A) was lifetime risk of breast cancer is ~50–85%. Since 2011B, this was divided into (4a) lifetime risk of breast cancer is increased above that in the general population and (4b) lifetime risk of breast cancer is virtually 100%. Out of 1,325 interpretations, 103 did not match the intended response. Four of these were reported by laboratories that also reported a genotyping error. The associated interpretations were correct for the genotypes provided. Among the remaining 99 errors (7%), 80 (6%; 81% of all errors) were due to laboratories incorrectly interpreting the risk of a negative result as being “lifetime risk of breast cancer is the same as that in the general population” instead of “lifetime risk of breast cancer cannot be determined without BRCA mutation testing of the affected relative” ( Figure 2 ).

Figure 3 illustrates the interpretation results of the participants by survey year ( Figure 3a ), stratified by location (US and international). The pattern of incorrect responses is similar for the two groups and does not, on average, change over time (test of slope = 0; P = 0.61 and 0.80 for US and international participants, respectively). To explore this further, the interpretation of results was reanalyzed based on the number of years that each laboratory had been participating in the PT survey ( Figure 3b ). Again, these results are stratified by location. A general trend toward fewer incorrect interpretations with longer participation is evident for the US participants (P = 0.0031) but not for the international participants (P = 0.47). On average, the proportion of correct interpretations for the US laboratory has improved 0.7% per year of participation in the survey.

Figure 3
figure 3

Performance on the interpretation of the survey results from 2003 to 2012. (a) Interpretation of results by survey year and (b) interpretation of results by the number of years that an individual laboratory has participated in the survey. In a, the pattern of incorrect responses is similar for the two groups and does not, on average, change over time. In b, the trend toward fewer incorrect interpretations with longer participation is evident for the US participants (P = 0.0031), but not for the international participants (P = 0.47).

Full size image

Discussion

BRCA1 is located on chromosome 17q21 and contains 24 coding exons, whereas BRCA2 is on chromosome 13q12.3 and contains 27 coding exons. Thousands of pathogenic sequence changes have been identified in both genes, most of which are frameshift deletions, insertions, or nonsense mutations, leading to premature truncation of protein transcription. Testing strategies are enumerated to confirm or establish the diagnosis in a proband of Ashkenazi Jewish ancestry or in a family with no known germ-line mutation as well as predictive testing for at-risk asymptomatic adult family members in a family known to have a germ-line mutation (GeneReviews).5 If a familial mutation exists and it is known, it is best to test for that particular mutation to facilitate test result interpretation. The interpretation of a negative test result is dependent on the methodology used, i.e., targeted mutation analysis, sequence analysis of coding regions, deletion/duplication, or multigene panels by massive parallel sequencing or NGS. It may be explained by the inherent technical limitation(s) of the specific test methodology employed. Hence, failure to detect any germ-line mutation does not eliminate the possibility of hereditary cancer susceptibility in a given family. Furthermore, breast cancer may also be secondary to a change in a different cancer susceptibility gene or a consequence of nonhereditary factors. With increasing use of NGS, variants of uncertain clinical significance in either BRCA1 or BRCA2 have been reported. In such instances, family studies may help determine whether the variant segregates with the cancer phenotype.

This report is based on PT data and provides evidence of the excellent analytical performance with rare analytical misses. Participants utilized a variety of techniques to perform BRCA1/2 mutation testing, but all are considered laboratory-developed tests. Of the six analytical errors identified, five were false negatives. False-negative results will incorrectly reduce the predicted cancer risk for the patient as well as for the relatives of the patient tested. Incorrect mutation in a mutation-positive proband may not pose immediate harm. However, if the wrong mutation is reported, family members typically undergo targeted testing based on the reported mutation, potentially affected individuals would be missed, and the resulting risk assessments will be inappropriate. It is possible that some of the analytical errors were due to postanalytical clerical error. The apparent sample mix-up that was associated with the one false-positive result may be this sort of error. Although the PT surveys are performed according to a laboratory’s standard testing practices alongside clinical samples, it is not possible to know whether specimen switches or misidentification is more or less likely to occur with PT surveys than in real clinical practice.

The majority of the incorrect interpretations were associated with correct genotyping results. These errors were identified from several laboratories, both US and international. The vast majority of the interpretation errors (81%) were in the case of a negative result, and the laboratories interpreted that result as being “lifetime risk of breast cancer is the same as that in the general population” instead of “lifetime risk of breast cancer cannot be determined without BRCA mutation testing of the affected relative.” This is an incorrect interpretation because the risk of breast cancer in patients who test negative for BRCA1/2 mutations is dependent on the family history and on the results of family testing. Despite reinforcements from the PT survey discussions, more active education on the clinical interpretation of negative test results is warranted.

For the 2012 survey, one participant used a laboratory-developed test based on NGS without any errors. Although it may still be cost effective in some situations to use a targeted BRCA1/2 mutation panel for the founder mutations in the Ashkenazi Jewish population, the costs of NGS are decreasing, and the accessibility of such testing in clinical laboratories is increasing. Soon, it may be equally or more cost effective to perform exome sequencing as opposed to targeted screening. NGS is described as a disruptive technology, and the rapid incorporation of this technology into clinical laboratories is changing the manner in which we view laboratory testing/genotyping and makes it necessary for PT to evolve. How this will happen is a topic of debate, but alternative approaches emerge. The CAP/ACMG Biochemical and Molecular Genetics Resource Committee already offers a method-based Sanger sequencing PT for clinical laboratories offering rare disease testing.6 It would be difficult to have individual PT for the hundreds of rare diseases tested in clinical laboratories, and this generic sequencing survey can provide a reliable source for external oversight. The CAP’s NGS working group is developing novel PT approaches to NGS and promulgating specific checklist questions for NGS laboratories. Before the June 2013 Association for Molecular Pathology v. Myriad Genetics Supreme Court ruling, BRCA1 and BRCA2 whole-gene sequencing in the United States could only be performed by one laboratory, Myriad Genetics, due to patent rights on these genes. A recent landmark decision ruled that DNA is naturally occurring and is not patent eligible, opening the door for laboratories to develop their own BRCA1/2 testing programs, likely based on NGS platforms.

PT for targeted mutations in other Ashkenazi Jewish diseases such as Tay–Sachs disease, Canavan disease, and familial dysautonomia is also offered by the CAP/ACMG Biochemical and Molecular Genetics Resource Committee. The PT experience for these three disorders has recently been published and also shows a high analytical sensitivity and specificity similar to our findings in this report (97.2/99.8% for Tay–Sachs disease, 98.9/99.7% for Canavan disease, and 99.2/99.7% for familial dysautonomia for analytical sensitivity and specificity, respectively).7 However, the analytical interpretation rate was better than our rate (99.3% for Tay–Sachs disease; 99.2% for Canavan disease; 99.7% for familial dysautonomia versus 92.5% for BRCA1/2). Overall, the results of the PT for BRCA1/2 are consistent with similar publications on CAP/ACMG PT results.7,8,9 These publications illustrate that US molecular genetic laboratories are providing high-quality testing services.

In conclusion, both US and international laboratories testing for targeted mutations in BRCA1/2 had a better analytical performance than interpretative performance on the CAP/ACMG PT survey during the past decade. Looking forward, we anticipate that more US-based laboratories will be testing for BRCA1/2 mutations and that the rapid advances in NGS technologies will require a shift from the traditional analyte-specific PT tests to more innovative method-based PT testing.

Disclosure

The authors declare no conflict of interest.