Main

The Human Genome Project has accelerated the development of genetic tests to detect mutations that may predispose to adult onset diseases, such as cancer. As a result, genetic testing is expected to assume a larger role in preventive medicine in the near future.1 It has been predicted that the burden of providing genetic testing and counseling to patients will fall on primary care practitioners (PCPs), due to a shortage of trained medical genetics specialists and genetics counselors.2,3 In order for PCPs to understand the clinical significance of a genetic test, they must have a working knowledge of basic principles of genetic inheritance and reasonable facility with the interpretation of probabilistic data. Some studies suggest that many PCPs are not adequately prepared to interpret genetic tests for disease susceptibility.4,5 Others have shown that physicians often have difficulty interpreting probabilistic data related to the clinical utility of diagnostic tests, such as the positive or negative predictive value of a laboratory test.6–8 The current standards for offering genetic testing for cancer risk assessment require (a) provision of genetic counseling, (b) discussion of the risks and benefits of testing, and (c) proper test interpretation, including factors that contribute to the predictive value of the test, such as genetic heterogeneity and incomplete penetrance.9 In fact, some genetic tests have been introduced into clinical practice before their clinical validity has been clearly established, further complicating test interpretation.2,10–14

At the present time, genetic testing for mutations in cancer susceptibility genes is performed in a small number of highly specialized laboratories. The clinical reports issued by these laboratories may assume a certain level of sophistication with the testing process and familiarity with the scientific terminology used to describe the test result. Some genetic testing laboratories attempt to address the needs of generalist clinicians by providing educational materials that explain the indications for testing and the interpretation of the test results; however, this information may not be complete.15 Furthermore, physicians may not fully understand the information and may not refer to the educational materials when reviewing the patient's results. Consequently, the laboratory report may complicate the testing process and impede the generalists' ability to understand and correctly apply the information in clinical practice.

To better prepare PCPs to provide competent genetic testing services, laboratories must design effective reports that facilitate correct interpretation of this complex information. Improvements in the communication of genetic test results should theoretically lead to reduction in the misinterpretation of genetic test results, and improve the delivery of preventive health care services related to genetic testing. Professional societies, such as the College of American Pathologists (CAP) and the American College of Medical Genetics (ACMG), have published standards or guidelines for laboratories that provide genetic testing, which require that reports containing genetic information include interpretive content that is understandable by generalist physicians.16,17 It is not yet clear how laboratories will interpret or comply with this guideline.

This study evaluates PCPs' comprehension of the cancer risk information contained in two experimental laboratory reports that were designed to provide test results for comprehensive BRCA1 and BRCA2 (BRCA1/2) gene analyses. Mutations in these genes have been associated with increased risk of breast and ovarian cancers, as well as some other malignancies.18–23 These experimental reports contained risk data for mutation carriers and the general population that was presented in either a tabular or a graphic format. The main objective of the study was to determine if physician comprehension of the probabilistic risk information provided in the two reports was influenced by the format in which the information was presented. Secondary objectives of the study were to assess PCPs' knowledge of basic inheritance, and to identify physician characteristics that might be associated with correct interpretation of genetic risk information.

MATERIALS AND METHODS

Study population

Family practitioners, general internists, and gynecologists in the Cleveland metropolitan area were recruited from a target population that was not currently ordering BRCA1/2 testing. Physicians were directly solicited at hospital or professional conferences. The physician subjects were obtained from the Department of Family Medicine of the Case Western Reserve University School of Medicine, the Departments of Internal Medicine and Obstetric/Gynecology of Ohio-Permanente Medical Group (OPMG), and the Department of General Internal Medicine at MetroHealth Medical Center. Additional subjects were recruited at the annual meeting of the Cleveland Academy of Family Practitioners. Resident physicians were not excluded from the sample; however, medical students were excluded. All responses were anonymous.

Survey design

Case vignette

A brief case vignette presented a 30-year-old woman who underwent BRCA1/2 gene analysis for appropriate clinical indications. Her mother had premenopausal breast cancer, and a maternal aunt had both breast and ovarian cancer before the age of 60. The history also stated that the mother had been previously tested for a mutation and had the same result as the patient. This information was provided to emphasize the importance of performing gene analysis on an affected member of the pedigree before testing other family members. Survey participants were instructed to read the attached laboratory report and then answer several multiple choice questions, using the report as their only source of information.

Development of the reports

Two experimental reports were designed, each containing the same BRCA1/2 comprehensive gene sequence analysis result for the hypothetical patient. The experimental reports were modeled on an actual report issued by a commercial genetics laboratory. The experimental reports adopted a similar page layout and extracted terminology and phrases directly from the interpretive portion of the model report. The report indicated the specific genetic variant of the BRCA1 gene that was present, and stated that “this mutation has been defined as deleterious in linkage studies of high risk families.” The major difference between the model report and the experimental report was the addition of tables or graphs to present the age-related breast cancer risk information. There were two tables or two graphs, which contained the identical information on the cumulative risk of breast cancer to age 80 and the incidence of breast cancer for mutation carriers and the general population by decade from age 30 to age 80 (Fig. 1). The cumulative risk and incidence rates of breast cancer for BRCA1/2 mutations carriers were calculated from data published by Struewing et al.22 The breast cancer incidence data for the general population was from Feuer et al.24

Fig 1
figure 1

Report format A contained tables. Report format B contained graphs. Cumulative risk and incidence data for BRCA1/2 mutation carrier were derived from Struewing et al.22 Breast cancer risk data were taken from Feuer et al.24

Full size image

There were other differences between the model report and the experimental reports. The experimental reports did not provide information on the risk of ovarian cancer associated with the BRCA1/2 mutations. Also, the risk statistics presented in the reports were based on a single population-based study, and the reference was not stated in the report. These differences were intended to simplify the experimental report, thereby reducing the sources of variance.

Creation of variables

Three domains of comprehension were measured as distinct outcome variables: (1) comprehension of the mutation status; (2) comprehension of the cancer risk information; and (3) comprehension of Mendelian inheritance. The survey questions are provided in the Appendix. Q1 asked the patient's mutation status, which was clearly stated in the report. Correct responses to the four questions that focused on risk assessment (Q2–Q5) required interpretation of the information contained in the tables or graphs. Q2 and Q3 addressed the relative risk of breast cancer for BRCA1/2 mutation carriers compared to the general population. The information to answer these questions was most easily derived from the first table or first graph in each report (“Cumulative Risk of Breast Cancer”). Q4 and Q5 required interpretation of the information contained in the second table and graph (“Incidence of Breast Cancer”), and concerned the attributable risk of BRCA1/2 mutations (Q4) and the incidence rates of breast cancer among mutation carriers by decades of life (Q5). We reasoned that physicians who properly interpreted the risk information that was presented in the report should be able to answer all of the risk assessment items correctly. Therefore, we dichotomized the risk comprehension outcome variable based on whether participants correctly answered all 4 questions (Q[2–5]).

The report stated that first degree relatives had a “one-in-two chance of having the same mutation as the patient.” Correct responses to the inheritance items (Q6 and Q7) required the respondents to know that a brother and a daughter are both first degree relatives, and that males and females have equal probabilities of inheriting these autosomal mutations. The variable Total Score (TS) was created by summing the correct responses to Q1–Q7. A perfect score of 7 would be achieved by physicians who (1) read the mutation result correctly, (2) interpreted the cancer risk information content correctly, and (3) understood Mendelian inheritance among first degree relatives. We reasoned that demonstration of this level of understanding was essential for appropriate counseling of patients with BRCA1/2 mutations. Therefore, we dichotomized TS based on whether participants answered all of the comprehension questions correctly.

The independent variables were physician characteristics that might be associated with comprehension of the risk information presented in the reports. These included medical specialty, practice type, number of years since completing training, and hours of continuing medical education in genetics in the previous 2 years. Another variable (Q8) was created to assess understanding of the concept of cumulative risk. Participants were asked to select the correct definition of cumulative risk from several choices. The purpose of this variable was to identify a subset of physicians who might have prior knowledge of the epidemiologic concepts that were implicit in the risk information. Since physicians who were more familiar with epidemiologic concepts would be at an advantage, this variable (Q8) served to adjust for differential knowledge of epidemiology between the two report groups.

Pilot of the survey

The survey instrument was piloted on 10 OPMG physicians in a setting similar to that in which it would be administered to the study population. As a result of the pilot study, the wording of one question (Q4) was slightly modified for clarification.

Data analysis

All analyses were done using SPSS version 10. Descriptive statistics were done on the physician sample. Frequency distributions were used to collapse polychotomous independent variables into dichotomous variables for subsequent analyses, where appropriate. The distribution of physician characteristics by report type was analyzed by the chi-square test or Fisher's exact test. The responses to the individual comprehension items by report type were analyzed by Fisher's exact test. Student's t-test was used to compare the proportion of correct responses to the main outcome variable, Q[2–5], by report type. Fisher's exact test was used to examine associations between physician characteristics and the response patterns. Multivariate logistic regression was used to adjust for physician predictor variables in comparing the effectiveness of the two report formats.

RESULTS

Validation of the survey

The survey was mailed to a nationwide sample of 90 genetic counselors, all of whom were members of the Cancer Risk Counseling Special Interest Group of the National Society of Genetic Counselors. These genetic counselors were believed to represent an appropriate sample of experts for purposes of this study. Forty-two (47%) responded to the mailed survey; 17 of the respondents received the tabular format and 25 received the graphic format. Each of the questions was answered correctly by at least 98% of the respondents, with the exception of Q5 (81%). The lower correct response rate to this question probably reflected its greater difficulty.

Description of study participants

The study physicians consisted of 124 PCPs in the Cleveland Metropolitan area. There were 82 respondents for an overall response rate of 69%. The characteristics of the study sample and their distribution by report type are shown in Table 1. Ninety-one percent (91%) of the PCPs were general internists or family practitioners. The physicians were approximately evenly distributed among academic and nonacademic practices and number of years post-training. The majority (76%) reported that they had received no CME in genetics within the past 2 years. Forty-two (51%) respondents received the tabular format of the report and 40 (49%) received the graphic format. The report formats appeared to be randomly distributed by medical specialty, years post-training, and CME in genetics. However, there appeared to be nonrandom distribution of report formats by practice type and knowledge of cumulative risk (Q8). Sixty-seven percent (67%) of academic physician respondents received the tabular format, while 63% of nonacademic respondents received the graphic format (P = 0.009). Sixty-four percent of physicians who selected the correct definition of cumulative risk received the tabular format, while 70% of those who could not define cumulative risk received the graphic format (P = 0.005).

Table 1 Physician characteristics by report format
Full size table

Physician responses to the comprehension questions are shown in Table 2. All respondents correctly noted that a harmful mutation of the BRCA1 gene was detected in the hypothetical patient (Q1). In an unadjusted analysis, there were no significant differences between report formats in the number of correct responses to the individual questions addressing cancer risk. Forty-two (51%) of the respondents answered all 4 of these items correctly (Q[2–5]). Responses to the questions about the risk of inheritance of the mutation by first degree relatives (Q6 and Q7) indicated that 30% of respondents did not know that the patient's brother had a 50% chance of having inherited the same BRCA1 mutation, and 15% did not understand that the patient's daughter also had a 50% chance of inheriting the mutation. Overall, 32% of the PCPs did not demonstrate sufficient knowledge of Mendelian inheritance to answer both of these questions correctly.

Table 2 Responses to comprehension items by report format
Full size table

In the unadjusted analysis, there was no significant difference in comprehension of the cancer risk information (Q[2–5]) by report format (55% correct for tabular format vs. 48% correct for graphic format;P = 0.517). However, knowledge of cumulative risk was strongly associated with comprehension of the cancer risk information; 37/50 (74%) of respondents who selected the correct definition of cumulative risk also correctly answered all of the risk assessment questions, while only 4/30 (13%) of those who could not define cumulative risk responded correctly (P < 0.001). Using a logistic model that adjusted for this predictor variable, there was a nonsignificant trend toward better performance with the graphic format (OR = 3.1;P = 0.104).

The frequency distribution of TS ranged from 3 to 7, and was skewed toward the higher correct response rates. Thirty-nine percent had perfect scores of 7, and low scores were rare. Again, knowledge of cumulative risk emerged as a significant predictor of TS (OR = 69;P < 0.001). None of the other physician variables had a significant effect on comprehension of the cancer risk information or on TS.

Since knowledge of cumulative risk appeared to be an important predictor of overall performance, we explored the potential relationships between responses to Q8 and other physician characteristics. For academic physicians with more than 10 years of experience, the odds of knowing the definition of cumulative risk were 13 times greater (OR = 13;P = 0.02) than for their younger colleagues (academic or nonacademic). Younger PCPs in this sample had a 50/50 chance of correctly answering Q8, regardless of practice type. Physician specialty and CME in genetics were not associated with this variable.

DISCUSSION

The results of this study suggest that a graphic presentation of genetic risk information might be more helpful to PCPs than a tabular presentation (OR = 3.1; 95% confidence interval [CI] = 0.8–12.1). However, the small sample size limited the ability to detect a statistically significant difference between report types. The significant finding of this study is that prior understanding of the concept of cumulative risk appears to be an important predictor of physician comprehension of genetic risk information. The magnitude of the effect that understanding cumulative risk appears to have in predicting those physicians who are able to correctly interpret the genetic risk information is striking (OR = 31.9; 95% CI = 7.1–143.3). It is possible that knowledge of cumulative risk is a marker of physicians who are more familiar with basic principles of epidemiology and who are, therefore, better prepared to interpret genetic risk information. Alternatively, this variable may reflect a subset of physicians who keep current with the medical literature in general and may be more familiar with developments in medical genetics. On the other hand, failure to understand cumulative risk may be the more important marker of physicians who lack experience interpreting medical research results and who are, therefore, less prepared to interpret the clinical significance of genetic test results. Only 13% of PCPs who did not know the definition of cumulative risk answered all the risk questions correctly.

Although academic physicians demonstrated greater familiarity with the concept of cumulative risk, this knowledge did not necessarily translate into better overall performance on the interpretation of the cancer risk information contained in the reports. Some physicians who correctly defined cumulative risk misinterpreted the questions concerning the attributable risk of BRCA1/2 mutations for breast cancer among different age groups of women (Q4 and Q5). The majority of physicians who incorrectly answered Q4 chose the incorrect response stating that the attributable risk of a BRCA1/2 mutation was greater in older women than in younger women. Similarly, among those who answered Q5 incorrectly, the majority interpreted the 10-year risk of developing breast cancer to be greater for a 60-year-old BRCA1/2 mutation carrier than for a 40-year-old mutation carrier, despite the incidence data presented in the report. These erroneous interpretations may reflect an underlying cognitive bias. Most physicians are well acquainted with breast cancer statistics in the general population, which show that the incidence of breast cancer increases steadily with age. The fact that BRCA1/2 mutations increase the risk for early-onset breast cancer (i.e., women in their 30s and 40s) is contrary to the general population trend. Representativeness is the cognitive bias that uses resemblance as a quick way to assess risk.25,26 To the extent that older BRCA1/2 mutation carriers are representative of older women in general, physicians may have interpreted their 10-year risk of breast cancer as being greater than the risk for younger mutation carriers. This erroneous interpretation was made despite the incidence data, which showed that the probability that 60-year-old BRCA1/2-positive women will develop breast cancer during the next decade is less than the 10-year incidence of breast cancer among 40-year-old BRCA1/2-positive women. The reason for this age-related risk difference can be partially explained by the earlier onset of breast cancer in BRCA1/2 mutation carriers and incomplete penetrance of BRCA1/2 mutations, a concept that may not be familiar to many nongeneticist physicians.

The report stated that first-degree relatives had a “one in two chance of inheriting the same mutation.” The study results also demonstrated that many PCPs are deficient in their knowledge of Mendelian inheritance, as demonstrated by the 32% of respondents who failed to correctly answer the two questions regarding risk to siblings and offspring. It is possible that not all PCPs know that first-degree relatives include children and siblings. It also appeared that some physicians were not aware that male first-degree relatives had the same probability of inheriting this mutation as females. These results are consistent with the research by Hofman and colleagues on physician knowledge of genetics.5

If PCPs will be offering genetic testing for cancer risk assessment to their patients in the future, laboratory reports that provide these results must be understood by generalist physicians. This desirable goal may be more difficult to achieve than previously anticipated. The CAP standards for reporting DNA test results for disease-associated genes that have multiple mutations require that the report include a discussion of the “limitations of the findings and the clinical implications of the detected mutation (or negative result) for the disease, with regard to recessive or dominant inheritance, recurrence risk, penetrance, severity, and other aspects of genotype-phenotype correlation.”17 The results of our study suggest that many generalists may not be prepared to interpret this complex information.

Laboratories that perform testing should not assume that physicians understand terms such as “first-degree relative” or that they can correctly interpret how summary statements of risk should be applied to an individual patient. Our findings suggest that it may be more useful to generate reports that are age-specific, so that short-term and long-term risk information presented in the report can be more easily applied to the actual patient, providing less opportunity for misinterpretation. Ideally, pedigree information would also be incorporated into the report. However, a practical problem with implementation of this strategy is that reference laboratories often do not receive the accurate family history that is needed to produce a patient-specific risk model, either because the physician did not provide the history, or the information provided is incomplete. Since it is routine practice to provide patient age on laboratory test requisitions, this information can generally be reliably obtained. The potential consequences of misinterpretation of the reports and lack of adequate understanding of basic inheritance include the possibility that physicians could misinterpret not only the patient's risk of breast cancer, but the risk to relatives as well. Overestimates of risk might lead to increased anxiety among older women mutation carriers, increased inappropriate surveillance or even radical prophylaxis. Underestimates of risk to potentially affected relatives may result in missed opportunities for intervention in at-risk women.

The results of this study also suggest that continuing education programs for physicians in medical genetics and genetic testing should include some general epidemiology to prepare physicians for more accurately interpreting genetic risk information. Future efforts directed toward learning how to present the results of genetic tests and other complex laboratory tests to physicians who may not have expertise in a particular field would benefit laboratory professionals, clinicians, and their patients.