Introduction

Inherited breast cancer accounts for 5–10% of the cases. Mutations of the BRCA1 and BRCA2 genes are responsible for about 15–40% of the breast cancer with familial aggregation and for the majority of hereditary ovarian cancer families.1, 2 Although initial data from the Linkage Consortium Study indicated a probability of 85% to develop breast cancer in a woman with BRCA gene mutation,3, 4 more recent studies suggest that the penetrance of BRCA1/2 mutations might be lower in non-high-risk families (Antoniou et al5 and references therein). In the Italian population, the probability to develop a breast or an ovarian cancer at age 70 years was estimated to be 39 or 43%, respectively, for a BRCA1 mutation carrier and 44 and 15% for a BRCA2 mutation carrier.6 The considerably high costs of BRCA1/2 mutation screening, together with the expanding request for inherited breast and/or ovarian cancer counselling and genetic testing, prompted the development of tools that might help in the selection of the appropriate candidates for the analysis. Two different approaches have been used in developing such models: the empirical and the Mendelian ones. The empirical models take into account the reported frequencies of BRCA1/2 mutations in large populations stratified on the bases of particular features describing their family history. The Penn model, the first empirical predictive tool developed after the discovery of the BRCA1 gene, is a logistic regression model applicable to the BRCA1 gene only.7 A number of other platforms subsequently developed, also considered the contribution of several factors (including proband status, family history, age at onset, breast and ovarian cancer association, presence of a male breast cancer and ethnicity) for the prediction of the carrier mutation probability for BRCA1 and, in some instances, BRCA2.8, 9, 10, 11 In contrast, Mendelian models make use of penetrance and allelic frequencies of characters inherited according to the Mendelian genetics. One of the most widely used prediction tool, the Brcapro software,12 is based on the mathematical model originally developed by Parmigiani et al.13 Brcapro is currently distributed as part of counselling package CaGene that also includes other models for predicting the carrier status as well as the risk of developing breast and ovarian cancer. A number of alternative models have also been proposed.14, 15 Despite the fact that empirical models predict the probability of identifying mutations, while Mendelian models indicate the probability of being a gene carrier,16 both are useful instruments for cancer genetic counselling.

The rate of BRCA1/2 mutation detected in different population might be influenced by local genetic factors (such as allelic frequencies, penetrance and founder effect). As an example, most studies reported an higher frequency of BRCA2 mutation in breast cancer family in Italy.17, 18, 19 Taking this into account, the Italian Consortium for Hereditary Breast and Ovarian Cancer recently developed a country-customized version of the Brcapro software16 that is distributed as part of a counselling package named Italian CaGene (IC software).

The aims of the present study were two-fold: first, to evaluate the accuracy of BRCA1/2 mutation carrier prediction by means of the novel IC model in an independent data set; second, to compare the performance of this model to other widely used methods such as the Brcapro and Myriad mutation prevalence tables. We report here on the results of the analysis of 70 families, stratified in four different profiles, selected and screened in our institution.

Patients and methods

Families

From March 2002 to June 2004, 108 breast/ovarian cancer families were referred to the genetic and oncology counselling centre of the Policlinico Umberto I, University La Sapienza. According to our minimal criteria (Table 1), 70 unrelated families were selected for BRCA1 and BRCA2 mutation screening. A total of 36 individuals did not reach our minimal criteria for inclusion because they did not display significant familial aggregation. In two cases, we did not obtain the consent for the genetic test. For each family, blood sample for DNA preparation and an informed consent were obtained from the most informative affected individual, identified on the base of the highest mutation probability (proband). The entire coding sequences and each intron/exon boundary of BRCA1 and BRCA2 were screened by protein truncation test (PTT, limited to BRCA1 exon 11) and/or direct sequencing. Only protein-truncating mutations reported on the BIC database as pathogenic were considered.

Table 1 Selection criteria

Models

For each proband, the a priori probability of carrying a pathologic BRCA1/2 germline mutation was calculated by either Brcapro (CA Gene) or IC softwares, or by the Myriad prevalence mutation tables. Brcapro is a mathematical model developed by statisticians at the Duke University, USA, and distributed as part of the counselling package CaGene.12 This model uses Bayes’ theorem to calculate the probability that a proband inherit a mutated gene on the basis of the proband characteristics (including breast and/or ovarian cancer; age at onset; current age) and family history of breast and/or ovarian cancer (first- and second-degree relatives). By incorporating the autosomal dominant Mendelian features of BRCA1 and BRCA2 genes and the prevalence and penetrance data from recent literature, this model is claimed to allocate detailed mutation probabilities for each of the two genes.

The IC model is based on the parameter values of Brcapro software and implemented in a country-customized versions by the Italian Consortium for Hereditary Breast and Ovarian Cancer.16 According to the authors, the main difference between the Brcapro and a novel model (IC) is with respect to calculation of penetrance for patients with multiple tumours. In our work, we used the Italian CaGene version 3.4.

Myriad mutation prevalence table is an empirical model that stratifies the probands in different carrier mutation risk categories according to variables describing their personal or familial history. This model is an extension of the Shattuck–Eidens model further described by Frank et al11 (Shattuck-Eidens et al8). The proportion of BRCA1 and BRCA2 deleterious mutations reported at the Myriad laboratories for each category might be used for calculating the mutation expectance. In this work, we made use of the last available version of the table for manual use (update spring 2004, http://www.myriadtests.com/provider/mutprev.htm).

Data analysis

The receiving operating characteristics (ROC) curve is constructed by plotting sensitivity on the y-axis and 1-specificity on the x-axis, for all possible cutoff values of a clinical test. Sensitivity is the probability that a subject with a score greater than the tested one have the condition of interest (BRCA gene mutation), whereas specificity is the probability that a subject with a score smaller than the tested one have not the condition of interest. According to ASCO directions,20, 21 we used the 10% threshold value for the evaluation of Brcapro, IC and Myriad tables and we classified our index cases as ‘carrier-positive’ when their carrier probability exceeds 10%, and ‘carrier-negative’ when their probability was <10%. The area under the ROC curve gives the global assessment of the performance of the test. In our case, the area under the ROC curve represents the fraction of all probands with identified mutations that have detection probabilities higher than probands with no mutations. Areas under the ROC curves were compared according to DeLong et al.22 Statistical analysis took advantage of the CIA 2.0 (T Bryant, 2000, University of Southampton) and SPSS (version 10; SPSS, Chicago, IL, USA) softwares.

Results

Description of the sample

From an initial sample of 108 unrelated breast and/or ovarian cancer families, we selected 70 probands for mutation screening of BRCA1 and BRCA2, according to our minimal criteria reported in Table 1. We observed 17 deleterious germline mutations with an overall detection rate of 24.3%. Nine of them were in the BRCA1 gene (53%) and eight in the BRCA2 gene (47%). Stratifying our sample in four different family profiles, we observed pathogenic protein truncating mutations in eight out of the 48 probands belonging to families with multiple cases of female breast cancer (HBC). The detection rate in this subset was 16.6%. Index cases showed a deleterious mutation in six out of the 15 families with hereditary breast and ovarian cancers cases (HBOC; detection rate 40%). A total of 33% of the male breast cancer (MBC) cases (with or without female breast cancer cases in the family) reported germline mutation in BRCA2 gene and only one family in the hereditary ovarian cancer (HOC) subset showed a BRCA1 pathogenic mutation (Table 2).

Table 2 Stratification in distinct cancer family profiles and mutation detection rates

Model comparison and IC validation

To validate the performance of the IC model and compare it to the capabilities of the Brcapro software and 2004 version of the Myriad mutation tables in predicting the presence of BRCA1 and BRCA2 mutations, we applied these models to our selected series. Brcapro identified 41 carrier-positive probands (see Patients and Methods for the definition) and pathogenic BRCA1/2 mutations were present in 12 of them (Table 3), performing a sensitivity of 71% (Table 4). According to IC prediction, 52 index cases were carrier-positive (Table 3). In this group, we observed 16 mutated individuals (Table 3), accounting for the very high sensitivity (94%) of this model (Table 4). Furthermore in the 12 cases correctly indicated as carrier-positive by both softwares, Brcapro predicted the mutation on the correct gene in 83% of the cases compared to the 91% rate of IC. Myriad tables identified 46 carrier-positive probands, 15 of which bearing BRCA1/2 mutations (Table 3), thus also displaying a good sensitivity (88%, Table 4). The probands identified as carrier-positive by the three models were largely overlapping but not identical. In contrast, both IC software and Myriad Tables recognized the 12 mutated carrier-positive probands identified by Brcapro, plus three additional cases. In one case, the IC software defined as carrier-positive a mutated proband missed by the other two models (Table 3).

Table 3 Carrier-positive families (carrier probability>10%) and mutation rates
Table 4 Performances of the three models

The negative predictive value (NPV, that is a measure of the prediction reliability of not being mutated) was 83% for Brcapro, but considerably higher for IC and Myriad tables (94 and 92%, respectively) (Table 4). In contrast, Brcapro showed the highest specificity (47%), compared to IC and Myriad tables (32 and 42%, respectively) (Table 4).

The overall accuracy in predicting BRCA1/2 mutations is often defined by the ROC curve, which takes into account both sensitivity and specificity values. The area under the curve (AUC) confirmed the presence of different performances among the models (Figure 1 and Table 4). We observed the highest values for Myriad tables (74%) and IC (72%) compared to Brcapro (67%). However, those values did not reach statistical significance.

Figure 1
figure 1

ROC curves representing the performances of the three models. Threshold value of mutation detection probability set to 10%.

Pure HBC families account for the large majority of our sample (48 out of 70 families, Table 2). Thus, we thought to assess the performances of three models in this subset, too. Interestingly, while we observed the presence of seven mutated probands among the 30 carrier-positive identified by IC, there were only four mutated probands in the 23 carrier-positive identified by Brcapro. Thus, the gap in sensitivity between the IC and the Brcapro software (88 and 50%, respectively) is further amplified in this subset. Myriad tables displayed an intermediate level of sensitivity (75%), identifying 24 probands in the HBC subgroup, six of which are indeed mutation carriers. Also in this subgroup, Brcapro showed the lowest NPV (84%) compared to IC and Myriad tables (94 and 92%, respectively). Analysis of the ROC curves indicated that only Myriad tables reached a value (75%) significantly different than chance, while the predictive efficiency of Brcapro and IC was not significant, probably due to the reduced size of the sample.

Application of the three models to our family sample allowed the identification of 18 (25.7%), 29 (41.4%) and 24 (34.3%) carrier-negative probands (mutation probability ≤10%) by IC, Brcapro and Myriad Tables, respectively (Table 5). According to this, those families would have been discounted from mutation screening. Interestingly, five out of 29 carrier-negative probands identified by the Brcapro (17.2%) had a mutation. Four probands were female subjects belonging to HBC families and one was a male with multicentric late onset breast cancer belonging to a family with two more late onset cases. In all of them, we found BRCA2 mutations. Conversely, only one (5.6%) of the 18 carrier-negative probands identified by IC showed a mutation that again was on the BRCA2 gene. Rather interestingly, this unique carrier-negative proband belonging to an HBC family was scored 9.7% by the IC software, thus extremely close to the 10% threshold level. Myriad tables defined 24 carrier-negative probands, two (11.8%) of which were mutated on the BRCA2 gene. Interestingly, the five probands misclassified by the Brcapro software as carrier-negative also included those misclassified by the other models.

Table 5 Carrier-negative families (carrier probability <10%) and mutation rates

Discussion

In this report, we validated the IC software on an independent Italian series of breast/ovarian cancer families and compared its accuracy with the most recent version of the Brcapro software as well as with the 2004 edition of the Myriad tables.

In most of the studies examining large series, they were obtained by pooling families selected in different cancer centers, according to different criteria and analyzed by different methodologies. In contrast, our study benefits of a consistent set of selection criteria and methodologies applied to all included families and has been strictly limited to affected probands. Indeed our detection rate for BRCA1/2 mutations was largely within the range of the previously published reports in the Italian population (8–37%,17, 18, 19) and extremely close to that provided by the largest study reporting 23.4% detection rate in 568 Italian families collected by five different cancer genetics clinics.16 Also confirming that our sample is representative of the Italian population at risk for hereditary breast/ovarian cancer, we detected the same mutation rate (16.6%) in the subset of HBC families, as reported by Marroni et al.

According to Euhus et al,23 the best method for estimating pretest mutation will ideally have a high sensitivity, a high specificity and a high NPV. When applied to our sample, Brcapro software provided relatively scarce performances, providing the lowest sensitivity and NPV values. In particular, it failed to identify as carrier-positive five probands that were mutated in the BRCA2 gene. The reduced ability of Brcapro in identifying BRCA2 carrier-positive families and the relatively high frequency of BRCA2 mutations in the Italian population we confirmed in our sample might help in explaining the suboptimal performances of Brcapro. Indeed, failure of the Brcapro software in detecting BRCA2-mutated carrier-positive probands had already been reported.15, 16 Interestingly, we noted that three of the four HBC-mutated families, misclassified by Brcapro as carrier-negative, were characterized by the presence of at least three cases of breast cancer, but not early onset. In all of them, we found the mutation in the BRCA2 gene. In contrast, the country-customized version, the IC software, performed very well and assigned to the carrier-positive group all effectively mutated probands with the exception of one case. Again this proband belongs to same HBC subgroup with at least three cases of breast cancer, but not early onset. However, the score assigned to this proband (9.7%) is extremely close to the 10% probability threshold, suggesting that IC is very close to identify as carrier-positive all families with those features. Furthermore, we noted that IC was more accurate in the selection of male breast cancer families compared to Brcapro. Thus, we confirm that the implementation operated by Marroni et al on the Brcapro consistently improved its performances, when applied to an Italian population. This software appeared very reliable for reducing the risk of discounting false carrier-negative families from genetic testing and more efficiently predicted the preferred candidate gene for mutation. However, IC showed the lowest specificity. This translates into the selection of 11 more probands (about 15%) to be screened for BRCA1/2 mutations as compared to Brcapro. Only four of them truly were mutation carriers. This is also likely to contribute, at least partially, to the lack of statistical significance in the observed differences between the IC and the parental Brcapro software at the analysis of the ROC curves. Considering the economic aspects of BRCA1/2 genetic testing, this might be a big limitation to the use of the software in the clinical practice, unless this aspect will be corrected in future versions.

ROC curve analysis provided the best AUC value for the Myriad tables. Indeed, Myriad tables also performed good sensitivity and NPV values and an intermediate specificity. Thus, they might also represent a valid help in the genetic counselling. Contrasting with our evidence, Myriad tables were reported to be slightly less accurate than Brcapro.16 Whether this depends on the different version of the tables we used or on additional factors is presently unknown.

In conclusion, our analysis indicates that all three models might be efficient tools in selecting possible BRCA1/2 mutations carriers to be subjected to mutation analysis. Each of them, however, still displays praises and faults. Our work also confirmed that implementation of Brcapro based on the country-customized evaluation of BRCA1/2 mutation penetrances in Italian families improved the accuracy of the mutation prediction. Therefore, our data are in support of the hypothesis that a similar strategy might be efficiently used worldwide for the development of more accurate prediction models, although, at the present, we cannot rule out the possibility that the IC model itself might more accurately predict BRCA1/2 mutation also in non-Italian families.