Main

Although multiple factors have been associated with increased risk for breast and ovarian cancer, family history remains the most significant and consistent predictor of disease development.1,2 It is estimated that 2% to 7% of breast cancers and 10% to 15% of ovarian cancers are the result of an inherited mutation in one of two major hereditary breast/ovarian cancer susceptibility genes, BRCA1 and BRCA2 (BRCA1/2).3–5 The lifetime risk for breast cancer ranges from 45% to 90% for women who carry BRCA1 mutations and 36% to 75% in those carrying BRCA2 mutations.6–9 Mutation carriers also face a lifetime risk for ovarian cancer from 24% to 66% and 8% to 32% for BRCA1 and BRCA2, respectively.5–9

The potential benefits of identifying individuals at hereditary risk for breast and ovarian cancer are well documented. Management options for BRCA1/2 mutation carriers—such as prophylactic mastectomy, bilateral salpingoophorectomy, and breast magnetic resonance imaging—have shown substantial potential to reduce cancer incidence and improve the likelihood of early detection.10–15 Furthermore, data regarding reduction in breast/ovarian cancer related mortality in mutation carriers are now emerging.16

Despite the advantages of cancer genetic risk assessment and testing, the majority of individuals in the general population who would benefit from such services are not currently receiving them.17–19 Narod and Foulkes20 estimated in 2004 that of the approximate 250,000 individuals in the United States who carry BRCA1 or BRCA2 mutations, only 10,000 (4%) had been identified. Furthermore, a recent study suggested that a substantial majority of family physicians would refer low-risk women for cancer genetics services if requested, a concern as demand for specialists in this area continues to grow.21

Although multiple sophisticated risk prediction models have been developed to assess the likelihood of carrying a BRCA1/2 mutation,22–27 their complexity limits their usefulness in a primary care setting. Applying these models requires some if not all of the following: expertise regarding hereditary breast/ovarian cancer, collection of a detailed three- or four-generation cancer pedigree, accurate data entry into a computer program, some understanding of expected outcome to ensure use of relevant data, and interpretation of the results in light of other clinically relevant information.28,29 As such, these existing models are primarily used by genetics professionals in clinical settings that specialize in cancer risk assessment and are used in circumstances in which some level of risk is already suspected. If the advantages of identifying BRCA1/2 carriers are to be realized on a broader population level, simple and accurate screening tools are needed.

In 2005, the United States Preventive Services Task Force recommended that “women whose family history is associated with an increased risk for deleterious mutations in BRCA1 or BRCA2 genes be referred for genetic counseling and evaluation for BRCA testing.”30 The United States Preventive Services Task Force further identified the need to “develop and validate tools feasible for use in primary care practice that would help clinicians make appropriate referrals for genetic counseling.”30

A referral screening tool (RST) intended to address this goal was designed and administered to a large cohort of unselected women undergoing routine mammography within a private Midwestern Health Care System. The accuracy of the RST in identifying individuals at potential hereditary risk for breast/ovarian cancer was evaluated in a random subset of these women by collection of detailed cancer pedigrees by telephone interview. Risk classification by the RST was compared with analysis of individual pedigrees using established hereditary risk models.

MATERIALS AND METHODS

The RST consists of a single page (Fig. 1) used to record both patient responses to family history questions and the assessment. The RST is scored as positive (two or more checks in the table) or negative (no family history or less than two checks in the table). The selection of two of the RST items as the cutoff for a positive screen is consistent with previously published risk criteria used to define “high risk.”31,32

Fig. 1
figure 1

Referral screening tool.

Full size image

The study was conducted in two phases. The intent of the first or “screening” phase was to evaluate the feasibility of applying the RST to a large number of women in a busy clinic setting; to assess the RSTs reliability using a random subset of individuals; to determine the percentage of women in this clinic population scored as positive or negative; and to provide a sizeable pool of subjects with a wide range of family histories from which to draw individuals for the second or “validation” phase. All individuals presenting for a scheduled mammogram at one of three clinics associated with a private health care system in Madison, WI were invited to participate during the period of October 6 through December 15, 2006. The study was approved by the University of Wisconsin-Madison Health Sciences Institutional Review Board and the Dean Foundation, and informed consent was obtained from all subjects. The RST was administered and immediately scored by the technologist performing the mammogram. A random 6% of participants underwent repeat administration by a second technologist to assess reliability.

To select interview subjects for the second phase of the study, a three-level RST scoring system was used: positive (two or more checks in the table), intermediate (one check in the table), and negative (zero checks in the table). The rationale behind creating this intermediate group was to maximize the number of subjects most likely to be misscategorized by a two-level (positive or negative) screening tool. Computer randomization was used to select subjects from these three categories, with a target recruitment of 100, 125, and 75, respectively. The positive and intermediate groups were intentionally oversampled to provide a sufficient number of potentially high-risk pedigrees to adequately address sensitivity. For all concordance and validation analyses presented in this article, the two-level RST score (positive or negative) was used, with the intermediate subjects from the three-level score included in the screen-negative group. A sample size of 300 subjects provided a power of 99% to detect a false-negative rate 10% or more greater than the false-negative rates of the hereditary risk prediction models being used for comparison.

Selected subjects were mailed an information and consent packet and initially contacted to review the study and schedule the interview time. The interviews were conducted by telephone by trained genetic counselors and included collection of demographic data as well as a detailed four-generation cancer pedigree consistent with the recommendations of the National Society of Genetic Counselors.28 To evaluate the potential validity of the RST under optimal circumstances of accurate scoring and reporting of family history at the time of administration, the original RST score was corrected based on the analogous information obtained by the pedigree interview. Concordance between the original and corrected RST score was 0.89.

Each completed pedigree was analyzed using four established and validated risk assessment models: the breast and ovarian analysis of disease incidence and carrier estimation algorithm (BOADICEA),26,33 BRCAPRO,25,34 Myriad II,35,36 and the family history assessment tool (FHAT).37 These models provide the best possible measure of risk—short of genetic testing itself—and thus were used as the standard against which the RST was compared. Using more than one model was considered desirable as each has relative strengths and weaknesses,38–42 and none is considered the single “gold standard.” The first three models provide estimates of BRCA1/2 mutation probability. The FHAT model is most analogous to the RST because it was intended as a screen to identify patients for cancer genetics referral. All four models have been previously validated by BRCA1/2 mutation analysis. BOADICEA and BRCAPRO incorporate detailed pedigree information, whereas FHAT and Myriad II models address only selected family history features. The model properties and accuracy measures are summarized in Table 1.

Table 1 Hereditary models used in pedigree analysis
Full size table

Consistent with the original recommendations of The American Society of Clinical Oncology,43 a BRCA1/2 mutation probability of 10% or greater was selected to designate “high risk” for the BOADICEA, BRCAPRO, and Myriad II models. Based on its design, a FHAT value of 10 defined the cutoff for “high risk”. To obtain the BRCAPRO, Myriad II, and FHAT values, the pedigrees were analyzed using CaGene 4.3.2 software (The University of Texas Southwestern Medical Center, Dallas, TX; http://www8.utsouthwestern.edu/utsw/cda/dept47829/files/65844.html). The BOADICEA model was accessed by participation in a beta trial of their online version (now available at http://www.srl.cam.ac.uk/genepi/boadicea/boadicea_home.html).

The validity of the RST was explored by analysis of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) comparing the two-level corrected RST score obtained on the subject (positive or negative) with their detailed pedigree analysis (high or low risk by the models). These measures were calculated using each model classification independently as the comparative standard and by considering an “actual positive” a pedigree that was high risk by any of the four models (high risk by any model). The area under the receiver operator characteristic (ROC) curve was evaluated as a measure of the discriminatory accuracy of the RST. In this situation, it represents the RSTs capacity to delineate between individuals at high risk and individuals at low risk as determined by the models.

To address the question of whether a 10% BRCA1/2 mutation probability is the appropriate cutoff to prompt a cancer genetics referral, the actual numerical values produced by the models were also examined. The skewed distribution of these data required comparison of median and percentile model values for the RST-positive and -negative subjects.

RESULTS

During the screening phase recruitment period, 2464 of the 3096 mammography patients approached (79.6%) agreed to participate in completing the RST. Of these individuals, 2056 (83.2%) also agreed to provide their name and contact information for possible participation in the telephone interview. Consistent with previous publications,19,31 the RST classified 6.2% of subjects as positive (high risk). Approximately 50% of subjects reported no family history of breast or ovarian cancer in first- or second-degree relatives. When divided according to the three-level RST score, 23.3% of subjects fell into the intermediate category. Successful administration of the RST by the technologists did not add undue time to the mammogram appointment or result in a decrease in the number of procedures performed—approximately 30 per day per clinic.

A random selection of 156 of 2464 subjects (6.3%) underwent repeat RST administration. Test-retest and inter-rater reliability were assessed by comparison of the RST score (two level) obtained by the first technologist (Tech 1) with that of the second technologist (Tech 2), as well as the Tech 1 score against the RST score assigned by the research assistant reviewing the screens. Concordance was >95% for both comparisons with kappa values in the good and excellent range for the Tech 1/Tech 2 and Tech 1/research assistant comparisons, respectively (Table 2).

Table 2 Agreement of RST scores
Full size table

Interviews were successfully completed on 296 of the 348 (66.1%) subjects recruited, with no significant difference in completion rate by the three RST risk levels (χ2(2) = 0.44, P = 0.8). The mean age of the interviewed subjects was 54.6 years (range, 26–83), 99% were white/non-Hispanic, 82% had some education beyond high school, and the majority was married (73%), with private health insurance (90%), and incomes of $50,000/year or greater (67%).

The validation measures of the RST are given in Table 3. For these analyses, the two-level RST score was used, with the intermediate subjects considered screen negative. Sensitivity (the proportion of subjects classified as high risk by the model that also scored positive on the RST) when compared with the four individual models ranged from 0.81 to 0.91, with an overall (high risk by any model) value of 0.81. Specificity (the proportion of subjects classified as low risk by the model that also scored negative on the RST) of the RST ranged from 0.77 to 0.90 with an overall value of 0.92. The discriminatory accuracy of the RST (area under the ROC curve) ranged from 0.83 to 0.86, with an overall value of 0.87.

Table 3 RST validation measures
Full size table

The PPV (proportion of subjects scored as positive on the RST that were high risk by the model) ranged from 0.28 to 0.34 when compared with the BRCA1/2 probability models, but reached 0.76 when compared with FHAT, and 0.80 overall. In contrast, the NPV (proportion of subjects scored as negative on the RST that were low risk by the model) was consistently high ranging from 0.93 to 0.99 and 0.92 overall. It should be noted that these predictive values are not representative of those that would be obtained in a general mammography population, as moderate- and high-risk subjects were intentionally oversampled. Using the prevalence of 6% RST screen-positive individuals in this study, and the overall sensitivity and specificity obtained, the PPV and NPV values expected in a general mammography population would be 0.39 and 0.78, respectively.

The distribution of the actual model values of subjects scored as positive or negative by the RST is given in Table 4. There was no overlap of the interquartile range (25th–75th percentile) between subjects scored as negative versus positive on the RST, for any of the model values. This suggests that the RST can provide a quick estimate of a subject's BRCA1/2 probability or FHAT score, clearly separating those at very low probabilities from those with moderate or high (>10%) likelihoods.

Table 4 Percentiles of hereditary model numerical values by RST score
Full size table

DISCUSSION

Because of their complexity, existing risk prediction models developed to assess the likelihood of carrying a BRCA1/2 mutation are not suitable for routine primary care application. If the advantages of identifying BRCA1/2 carriers are to be realized on a broader population level, simple and accurate screening tools are needed. The intent of such tools should be to identify the population subset that would benefit from more in-depth cancer risk assessment, genetic counseling, and genetic testing services for breast/ovarian cancer.

The RST was designed as a screening tool applicable for use in primary care settings. The RST does not require collection of a full pedigree, computer entry, calculations, or knowledge of hereditary breast/ovarian cancer; and it yields an instantaneous answer regarding the appropriateness of a cancer genetics referral. Within this study population, the RST was easily administered and reliable. Furthermore, it provided a valid representation of the likelihood a subject may be at hereditary risk for breast and ovarian cancer when compared with the gold standard of detailed pedigree analysis with established risk models. The overall ability of the RST to discriminate individuals at high risk from individuals at low risk (ROC area under the curve = 0.87) exceeded the highest values reported for the models themselves (range, 0.68–0.82).38–42

As the purpose of the RST is not to determine who should be offered BRCA1/2 genetic testing, but to assist in ascertaining which patients may benefit from referral for formal cancer risk assessment, maximizing sensitivity over specificity is desirable. Failure to ascertain a possible BRCA1/2 carrier may be considered a more adverse outcome than a patient undergoing an unnecessary cancer genetics appointment. Of the subjects who were high risk by all four models, 100% also scored positive on the RST, whereas 98% of those high risk by any three models were RST screen positive. It should also be noted that with rare exceptions (i.e., inaccurate reporting of family history at the time of administration), individuals who turn out to be “false positives” on the RST (i.e., BRCA1/2 mutation probability of <10%) will have some family history of breast and ovarian cancer and thus may still benefit from clarification of their risk. Burke et al.44 demonstrated that genetic counseling for women with a moderate family history of breast cancer significantly improved self-perception of risk, lowered cancer worry, and decreased the number who considered themselves candidates for genetic testing. Furthermore, although these individuals may not be appropriate candidates for genetic testing, they may qualify for screening with breast magnetic resonance imaging45 or tamoxifen chemoprevention.46 Although the RST is not intended to identify women in this category, it does suggest that a reasonable balance of “benefits to harms” might be expected for many RST false-positive individuals referred for cancer genetics services.

Review of the literature revealed only two previous publications addressing the development and validation of screening tools comparable with the RST. These include the FHAT used in this study37 and the pedigree assessment tool (PAT) of Hoskins et al.47 The FHAT was validated by actual genetic testing in a selected cohort of 184 subjects referred to a high-risk clinic based on their personal or family history of breast or ovarian cancer. The PAT was applied in a large cohort of women undergoing screening mammography and validated against a 10% or greater BRCA1/2 mutation probability as determined by the Myriad II model. Neither a full cancer pedigree nor genetic testing was used to assess the accuracy of the information obtained on the PAT. Both tools are more complex to administer and score than the RST because they require either complete pedigree entry into CaGene (FHAT) or assigning point values based on specific family history characteristics, adding scores manually for maternal and paternal histories, and determining if the assigned threshold is met. Ease of application is an important consideration when comparing tools intended as a screen for the general population.

Maintaining the simplicity of the RST is acknowledged to have resulted in some limitations in its accuracy. The manual scoring system does not differentiate between cancers on the maternal and paternal side. This is primarily an issue when two affected second-degree relatives are on opposite sides of the family (i.e., a paternal aunt with breast cancer younger than 50 years and a maternal aunt with ovarian cancer). Adjusting for this on a manual form would likely complicate the assessment to a point where it could no longer be rapidly scored. However, a simple computerized version of the tool has been developed, which still allows direct entry to a one-page form but assesses the screen as positive only if at least two checks involve the same side of the family.

The RST as designed does not account for two factors known to be associated with an increased likelihood for a BRCA1/2 mutation—specifically bilateral breast cancer and the occurrence of breast and ovarian cancer in the same individual. There is clear evidence that women with BRCA1/2 mutations and previous cancer of one breast have a substantially increased risk for contralateral breast cancer.48 However, the degree to which bilateral breast cancer predicts a BRCA1/2 mutation is less certain, especially for later onset disease.49,50 Although it is possible that sensitivity would be improved by adding this item to the RST, the specificity would likely suffer as would the ease of application and assessment. In situations where a family member had both breast and ovarian cancer related to a BRCA1/2 mutation, it is likely there would be other features of the family history that would lead to a positive RST score.

Finally, breast and/or ovarian cancer in nieces and granddaughters is not recorded on the RST, nor is cancer history in first cousins (third-degree relatives) included. However, cancer history in these individuals was collected for the pedigree analyses, and thus the validity measures account for this limitation. It should be noted that cousins and granddaughters are not included in the CaGene program used to derive the FHAT, Myriad II, and BRCAPRO values. Only the BOADICEA model allows for their inclusion. Examining the three subjects that were false negatives on the RST according to this model revealed no circumstances where affected cousins/granddaughters were the cause of the false-negative result. For the purpose of accurate risk assessment and genetic counseling regarding BRCA1/2 testing, consideration of third-degree relatives, nieces, and granddaughters is essential. However, collecting this information in a screening format is likely to be associated with negative tradeoffs in terms of time and simplicity of assessment and may increase the false-positive rate.

Several limitations of the study design deserve consideration. First, the subjects did not undergo BRCA1/2 genetic testing, which would be the ultimate “gold standard” in determining the tool's validity. Because of this, the RST must not be considered a tool equal to use of detailed pedigree analysis by the models, but rather a proxy thereof.

Another concern is the accuracy of the self-reported family history. No attempts were made to obtain medical record confirmation of the cancers reported in relatives. It has been documented, however, that recall of cancer diagnoses in at least first- and second-degree relatives is quite accurate.51

The most significant limitation of this study stems from the use of subjects exclusively from mammography clinics within a single Midwestern Health Care System. In addition, the results are skewed toward a white, educated, upper middle class cohort and therefore evaluation in alternate populations and settings will be required to address generalizability. However, it is the clinical application of the tool rather than its validity that will most likely be influenced by demographic characteristics—as they are for the most part independent of genetic risk for breast/ovarian cancer.

The RST is specifically designed to identify individuals at potential risk for breast/ovarian cancer because of BRCA1 or 2 mutations, which account for the majority of the hereditary cause of these malignancies. However, other hereditary cancer syndromes are associated with significant increased risk for breast and/or ovarian cancer including Cowden, Peutz-Jegher, Lynch, and Li-Fraumeni.28,29,31,32 The RST is expected to miss many individuals at risk for these syndromes because they also involve unique clusters of other neoplasms. Examination of the 296 full pedigrees revealed 12 subjects (4%) with family histories suggestive of possible Lynch, Li-Fraumeni, or Cowden syndromes. Of these, five (42%) were screen positive on the RST. A cancer genetics consultation, prompted either by a positive RST screen or an unusual family history, may reveal that testing for syndromes associated with genes other than BRCA1/2 is warranted. This issue is in part why clinicians should not view the RST as a decision tool for ordering BRCA1/2 testing but rather a means of identifying individuals appropriate for a cancer genetics referral to evaluate in more depth both family history and considerations for genetic testing. Efforts at dissemination of the RST will include a disclaimer to this effect, in addition to information regarding other family history features that should prompt a genetics referral.

The strengths of this study include the large number of subjects involved, high participation rates not influenced by risk level, assessment of reliability, collection of a four-generation cancer pedigrees by trained genetics professionals, and use of multiple and well-validated risk models for comparison. The demand for a simple screening tool such as the RST is expected to increase as Myriad Genetic Laboratories, Inc.—the laboratory that currently holds a patent on BRCA1/2 testing—is actively launching direct-to-consumer marketing campaigns. Analysis of the impact of a pilot direct-to-consumer campaign conducted by Myriad in 2002 demonstrated a substantial increase in the overall number of cancer genetics referrals, with a preponderance of individuals at low risk for a BRCA1/2 mutation.52 Patients inquiring about BRCA1/2 genetic testing could be reassured by their primary care providers that if negative on the RST, it is very unlikely they would test positive for a BRCA1/2 mutation.

In conclusion, this preliminary validation study seems to justify future research directed at applying the RST in other populations in primary care settings. Such analyses will be necessary to demonstrate the RSTs potential to improve not only the identification of women at hereditary risk for breast and ovarian cancer, but also the appropriateness of referrals to cancer genetics services.