Personal genome testing allows the identification of single-nucleotide polymorphisms associated with an increased risk for common complex disorders. An area of concern in the use of personal genome testing is how risk estimates generated differ from traditional measures of risk (e.g., family history analysis). We sought to analyze the concordance of risk estimates generated by family history analysis and by personal genome testing.
Risk categorizations for 20 complex conditions included in Navigenics personal genome testing were compared with risk categorization estimates derived from family history assessment using the kappa (κ) statistic.
The only conditions showing slight agreement between risk assessment methods were Alzheimer disease (κ = 0.131), breast cancer (κ = 0.154), and deep vein thrombosis (κ = 0.201) in females, and colon cancer (κ = 0.124) in males. Eighty-six individuals (11.4%) were found to have additional genetic risks not assessed by personal genome testing after family and medical history assessment, including 38 individuals with family histories suggestive of hereditary cancer syndromes.
Discordance between personal genome testing and family history risk estimates suggests that these methods may provide independent information that could be used in a complementary manner. Results also support that eliciting family history adds value to overall risk assessment for individuals undergoing personal genome testing.
Genet Med 2014:16(3):231–237
complex diseases; family history; Navigenics; personal genome testing; risk comparison; risk estimates
At a glance
In the past decade, there have been rapid advancements in genome-based screening technologies and their accessibility to the general public. Genome-wide association studies have enabled the identification of hundreds of single-nucleotide polymorphisms (SNPs) associated with common multifactorial conditions, including type 2 diabetes, heart disease, and many types of cancer. These findings have spurred the proliferation of private companies offering personal genome testing (PGT) for common complex disorders—by 2011, there were over 30 companies offering PGT worldwide.1 Many concerns have been raised regarding the clinical validity and utility of PGT.2,3,4,5,6 Measuring these constructs has presented considerable challenges,7 given the novelty of this type of testing, limited access to largely proprietary consumer data,2 and the length of time required to assess outcomes following testing.
One approach that has been used to assess the validity of PGT (i.e., the accuracy with which PGT can predict a clinical outcome)8 is to measure the predictive value of SNPs associated with common disease, in comparison to other clinical information including family history of disease. Predictive value studies address one area of concern about PGT—does genotyping SNPs from genome-wide association studies improve risk prediction for common complex diseases beyond risks generated by analysis of family history and clinical assessment? A recent study of SNPs associated with prostate cancer risk describes the development of a prediction model for absolute risk of prostate cancer, incorporating SNP markers and family history, to effectively identify a subset of men who should undergo screening at an earlier age.9 This study illustrates how a risk assessment model for complex disorders, inclusive of both PGT and family history information, can be used to refine risk prediction for complex diseases. With the growing number of SNPs determined to be associated with complex disorders, the development of predictive models incorporating multiple factors will be essential to guide appropriate clinical usage of PGT.
Family history assessment for common complex disorders has long been viewed as the gold standard for assessing individual risk of developing these disorders, with demonstrable value in providing an accurate method of risk stratification.10,11,12 If family history is deemed to be a clinically valid method of risk assessment for complex disease, then comparing it to PGT risk assessment may provide a means of measuring the clinical validity of SNP-based testing for complex disease risk. In turn, analysis of the concordance between the two methods may reflect the utility of PGT, by determining whether they are interchangeable methods or complementary tools for risk assessment.
To date, few studies have examined the concordance between PGT risk estimates and those based on family history. In a recent study, the correlation between self-reported family history/personal medical history and PGT results was measured for 15 conditions in over 3,000 individuals.13 For five of the conditions studied (those with higher heritability estimates and prevalence), PGT risk estimates were found to be significantly associated with family history and personal medical history, suggesting that PGT may not provide much additional value above that of traditional risk assessment measures.13 Conversely, a recent prospective study by Heald et al.14 looked at concordance between PGT and family history risk assessments for breast, prostate, and colon cancers and found minimal agreement for all three cancers, suggesting that PGT and family history could be used as complementary tools for risk assessments.
One of the most widely known personal genomics companies was Navigenics,15 which offered susceptibility testing for 28 common multifactorial conditions (see Supplementary Figure S1 online). On the basis of an individual’s SNP genotyping, Navigenics PGT provided estimates for the lifetime risk of developing each condition tested. In this study, we sought to compare risk estimates for a cohort of clients who underwent Navigenics PGT with risk estimates generated by family history analysis, in order to determine the concordance between these two risk assessment methods. An additional goal was to explore additional information or potential genetic risks gleaned from family history assessments. The results of this comparative study have implications for the clinical utility of PGT in combination with family history assessments.
Materials and Methods
A retrospective cohort of all individuals who purchased Navigenics PGT through the Medcan Clinic from July 2009 to May 2011 was established. The Medcan Clinic is a private comprehensive health-care clinic based in Toronto, Canada, which offers private-pay PGT to all its clients in conjunction with in-person genetic counseling sessions (including a detailed family history assessment) from a certified genetic counselor.16 Navigenics PGT included SNP analysis for 28 common multifactorial conditions (see Supplementary Figure S1 online). Navigenics used a multiplicative model to combine odds ratios for each SNP marker (converted into relative risk) with the lifetime risk of each condition and the prevalence of the marker in a reference population (i.e., the HapMap database) to provide estimates for the lifetime risk of developing each condition tested.17
Data, including demographic information such as sex and age at time of testing, PGT results, family history, and personal medical history for each client, were collected using chart extraction. Individuals who did not have a standard, three-generation pedigree taken18 or who could not provide information regarding both paternal and maternal family history were excluded from this analysis. The study was approved by the research ethics boards at the Hospital for Sick Children (Toronto, Canada) and the University of Toronto (Toronto, Canada).
Of the 28 complex conditions included in the Navigenics Health Compass panel (see Supplementary Figure S1 online), rare conditions were excluded from the analysis (i.e., those with <2% estimated lifetime risk in either males or females according to Navigenics—brain aneurysm, celiac disease, Crohn disease, lupus, multiple sclerosis, sarcoidosis). Diffuse stomach cancer was also excluded from analysis because the SNP used by Navigenics for risk estimation has been validated only in the Asian population.19,20 Information regarding ethnicity was not collected in this study; however, anecdotally Medcan clientele is predominantly of northern European (Caucasian) descent. Finally, obesity was excluded because information regarding family history of obesity was not regularly collected during family history assessments. Therefore, a total of 20 conditions were examined in this study (see Supplementary Figure S1 online).
The medical history of each client, as collected by the genetic counselor, was reviewed, and the total number of individuals affected by any of the 20 conditions was recorded. The maternal and paternal family history for each client was reviewed, and any family members affected with any of the 20 conditions were noted.
On the basis of family history assessment, individual lifetime risk for each condition was classified into two categories: “greater than general population” risk, or “general population” risk. “Greater than general population” risk categorization was defined according to the presence of a positive family history: either one affected first-degree relative or multiple affected relatives (two or more first-degree relatives, and/or two or more second-degree relatives on the same side of the family). Lack of a positive family history of any condition resulted in classification of individuals as at “general population” risk.
The estimated numerical lifetime risks for each condition, as calculated by Navigenics PGT, are grouped into three categories: “less than general population” risk, “general population” risk, or “greater than general population” risk. Navigenics categorized risks into the “less than” or “greater than” categories if the risk was ±20% as compared with the general population lifetime risk. For the purposes of directly calculating agreement between PGT and family history risk categorizations in this study, the “less than general population” PGT risk group was collapsed into the “general population” risk category.
The agreement of risk categorizations (“general population risk” and “greater than general population risk”) between PGT and family history assessment was compared using kappa (κ) statistic. The kappa statistic was used for this study because it provides a measure of the magnitude of agreement between two methods, after adjusting for agreement that could occur based on chance alone. Statistical analysis was performed using SAS version 9.3 (SAS Institute, Cary, NC). Qualitative categorization of agreement based on κ was as per Landis and Koch (Table 1).21 Because disease frequencies, lifetime risks, and SNP markers used to estimate lifetime risk differ between the sexes, analyses of concordance between risk categorizations were conducted separately for men and women.
Information was collected regarding additional genetic risks identified during family and medical history assessments, as well as pursuit of genetic testing outside of PGT as recommended based on these risks, if known. Such genetic risks were defined by a reported family history of a specific genetic syndrome or a family/medical history suggestive of a genetic syndrome (e.g., hereditary cancer syndromes).
A total of 757 individuals who purchased Navigenics PGT were included in this study; 472 (62.4%) were male and 285 (37.6%) were female. This sex distribution approximates the client demographics of the Medcan Clinic (~70% male to 30% female clients). The mean age for male and female clients was 49.1 years (SD = 9.9; range: 19–77 years) and 48.8 years (SD = 10.4; range: 18–72 years), respectively. A breakdown of the total clients grouped into each PGT risk category is provided in Supplementary Table S1a,b online along with the corresponding total number of clients in each lifetime risk category by family history assessment. Notably, significantly more individuals were grouped into higher risk categories by PGT than by family history for the majority of conditions. There was no significant difference between the average age of individuals who reported a family history of each of the conditions and those who did not for either males or females.
Agreement between risk assessment methods
The agreement between family history and Navigenics PGT risk assessment is shown in Figure 1a,b and Table 2. Overall, the two methods of risk assessment had poor concordance, as shown by the calculated kappa statistic (κ) values. There were no conditions for either males or females that showed more than slight agreement between risk assessments (i.e., had a κ value >0.20; Table 1), aside from deep vein thrombosis in females (κ = 0.201). Similarly, when combining males and females into a single sample for the 18 conditions that overlapped between the sexes, there were no conditions that showed significant agreement between risk assessment methods (see Supplementary Table S2 online).
For males, 18 of 19 conditions analyzed had κ values whose 95% confidence intervals (CIs) crossed zero (Table 2), indicating poor agreement between risk assessment methods, similar to what would be expected by chance. Colon cancer was the sole condition for males for which there was slight agreement. PGT and family history assessment agreed on risk categorization for colon cancer 79.7% of the time; however, the agreement between the measures was only modest (κ = 0.124; 95% CI = 0.0165–0.232). Fifty-three men (11.2% of total men) had a family history of colon cancer; only 15 of these individuals (28.0%), however, were categorized as having higher risk by PGT.
Similarly, for females, 16 of 19 conditions had κ values whose 95% CI crossed zero (Table 2), suggesting poor agreement between PGT and family history. The three conditions for which there was a slight agreement above that expected by chance, in ascending order, were Alzheimer disease (κ = 0.131; 95% CI = 0.0206–0.241), breast cancer (κ = 0.154; 95% CI = 0.0185–0.289), and deep vein thrombosis (κ = 0.201; 95% CI = 0.00380–0.399). Overall, PGT and family history assessment agreed on risk categorization for these conditions 75.4, 76.5, and 91.2% of the time, respectively. Nineteen women had a family history of Alzheimer disease, and 10 of these women (52.6%) were categorized as having high risk by PGT. Of the 49 women with a family history of breast cancer, 14 (28.6%) were categorized as having high risk by PGT. Only 11 women in this study had a family history of deep vein thrombosis, and 7 of these women (63.6%) were classified as either having general population risk/less than general population risk by PGT (data not shown).
The number of individuals with a personal medical history of the conditions included was not large enough to analyze the concordance of personal medical history and PGT risks. For 12 of the 19 conditions in each sex, there were fewer than 10 individuals with a personal medical history of these conditions (data not shown).
Family and medical history assessments: other potential genetic risks
Eighty-six (11.4%) individuals in this study were found to have additional genetic risks on the basis of review of their family and/or personal medical histories (Figure 2a). Similar proportions of males and females in the study (9.8 and 14.0%, respectively) had additional genetic risks identified after family history assessment (χ2 = 2.835; P = 0.092; d.f. = 1). Notably, 5.1% (n = 39) of all individuals in the study had family histories suggestive of hereditary cancer syndromes (Figure 2b). This subgroup included 24 individuals who had a family history suggestive of hereditary breast and ovarian cancer as determined by genetic counselors. Detailed information regarding additional health condition risks identified can be found in Supplementary Table S3 online.
Sixteen of these 86 individuals (18.6%) pursued additional genetic testing through the Medcan Clinic outside of PGT (Figure 2a). Many individuals in this subgroup were not themselves eligible for clinical genetic testing. That is, for at least 20 individuals whose family histories were suggestive of hereditary cancer syndromes, affected family members were more likely to receive informative genetic-testing results; therefore, genetic testing for the unaffected probands was not recommended or pursued (see Supplementary Table S3 online). Additional genetic testing results were available at the time of the chart review for 13 of 16 clients. Three of these genetic tests yielded informative positive results.
Of females whose family histories were suggestive of hereditary breast and ovarian cancer (with a lifetime risk to develop breast cancer likely to be higher than the general population) and males and females whose family histories were suggestive of Lynch syndrome (with a potentially higher lifetime risk of developing colon cancer), the majority were categorized as general population or lower risk of developing breast and colon cancers, respectively, by PGT (Table 3).
Much of the concern regarding PGT lies in a lack of proven clinical utility, validity, and predictive value.2,3 This is, in part, related to the longitudinal nature of the data needed to determine PGT validity and the subsequent translational research necessary to provide evidence of clinical utility. Our study represents one of the first and largest studies evaluating the concordance between PGT and family history risk assessments for complex disorders and provides novel data with implications for the validity and usage of PGT.
It has been suggested that differences between PGT lifetime risk estimates and those provided by traditional measures of risk such as family history and clinical information could lead to false reassurance for PGT consumers (e.g., if genomic testing were to indicate a lower than average lifetime risk for a certain condition, with family history indicating a much higher risk).22 This phenomenon, known as misattributed equivalence, refers to the differences in the reliability and usefulness of different test methods or risk assessments.23 If the two methods of risk assessment were found to have perfect agreement, this might have suggested that PGT and family history risk assessment were equivalent assessment methodologies and could be used independently. However, the lack of concordance for the majority of conditions analyzed suggests that family history and PGT provide different, independent information regarding risk for complex disorders.
The two methods of risk assessment are inherently different, in that PGT does not take into account environmental factors, which can be shared by members of the same family and thus are partially encompassed in empiric recurrence risks generated for multifactorial disorders. This significant contributor to the development of complex diseases could account for some of the differences in risk estimate categorizations.
The results of our study are consistent with those found by Heald et al.,14 which reported poor concordance between risk assessment methods for three conditions tested in Navigenics PGT (breast cancer, prostate cancer, and colon cancer). We expand on their work by analyzing more conditions included in the PGT panel and in a larger sample of individuals. The finding that overall these risk estimates do not agree for most conditions tested implies that these two risk assessment methods may not provide equivalent risk information for most individuals.
In contrast to our findings, Bloss et al.13 reported association between Navigenics PGT risk estimates and family history for a subset of five conditions. Two of these five conditions (type 2 diabetes and Graves disease) were included in our analysis and were found to show poor concordance between PGT and family history risk estimates. Differences between our findings are likely explained by a significantly lower prevalence of family history reports in our cohort and a difference in sample size, as well as different statistical methodologies used to assess association and concordance.
This study also exhibits the value of family history assessments in identifying possible hereditary risks for single-gene conditions, as demonstrated by the number of individuals for whom additional risks required discussion with genetic counselors (Figure 2a,b). This finding highlights the possibility that consumers could be falsely reassured by PGT results classified as low risk or general population risk (Table 3). In the absence of explicit family history interpretation from a health-care provider (HCP), such individuals could misconstrue the PGT results to mean that they are not at an increased risk for developing cancer, for example, and thus do not require increased surveillance. This potential for consumer misinterpretation demonstrates the value of integrating family history assessment and genetic counseling services into PGT. It also emphasizes the importance of having a genetic counselor or other qualified HCP available to contextualize the results of PGT for consumers.
Most PGT companies do recommend consultation with an HCP; for example, Navigenics test reports stated that women with strong family histories of breast cancer discuss this history with their family physician regardless of PGT results.17 However, many individuals may not fully appreciate the implications of such a recommendation. Given the value of family history assessment for consumers of PGT, it is reasonable to suggest that PGT not be offered in a direct-to-consumer manner without the available guidance of a knowledgeable HCP. Involvement of an HCP such as a genetic counselor is particularly important in cases where family history assessments identify risks for hereditary conditions for which screening or preventative recommendations exist. More studies assessing how PGT results and family history analysis influence consumer risk perception24 are essential to address the potential for false reassurance and misattributed equivalence, as well as the need for genetic counseling services. Such studies would shed light on the true drawbacks and benefits of PGT and guide recommendations for the appropriate use of PGT.
One of the main limitations in comparing self-reported family history to PGT is the potential for inaccurate or incomplete reporting of family history information. Numerous studies have shown significant reporting differences and patient errors in the reporting of family history,25,26,27 suggesting that it is unlikely that all individuals in this study provided entirely complete and accurate information. For the majority of client-reported family history, medical documentation regarding affected family members was not sought or obtained. For an unbiased, accurate family history assessment, one would require access to medical records for each individual in a family. An additional limitation in accurately assessing risk is that information regarding affected family members was collected at a single point in time. Barring changes in interpretation and risk-calculation models in PGT, a person’s risk based on PGT would remain the same over a lifetime because an individual’s SNP markers do not change. Yet as this cohort ages, they or their family members will likely develop some of the conditions analyzed, thereby altering the concordance between risk estimates. This is reflected by our finding that for the majority of the conditions, significantly more individuals were grouped into higher risk categories by PGT risk assessment (see Supplementary Table S1a,b online). For a more accurate comparison of risks between PGT and family history, further long-term studies should be undertaken looking at a large cohort and their families to capture information regarding new diagnoses of adult-onset complex disorders.
In summary, our data along with other studies13,14 suggest that these two methods of risk assessment for common complex disorders provide different and perhaps complementary information. Our study provides one of the most extensive analyses to date of concordance between risk estimates provided by PGT and family history assessments and finds little to no concordance for 20 of the multifactorial conditions included in Navigenics PGT. This lack of concordance and the number of individuals in this study for whom additional potential genetic risks were identified suggest a crucial role for the inclusion of a family history assessment for individuals undergoing PGT. In the future, risk prediction models should be developed, incorporating information gleaned from clinical presentation, family history, and PGT results. Such models would combine these data into a single, more highly accurate risk estimate, which could then support a personalized medicine strategy.
The authors declare no conflict of interest.
- Tables of direct-to-consumer genetic testing companies and conditions tested. 2011. http://www.dnapolicy.org/resources/DTCTableAug2011Alphabydisease.pdf. Accessed 1 October 2012. , .
- Using lifetime risk estimates in personal genomic profiles: estimation of uncertainty. Am J Hum Genet 2009;85:786–800. , , , , , .
- An agenda for personalized medicine. Nature 2009;461:724–726. , , , .
- Multigenic condition risk assessment in direct-to-consumer genomic services. Genet Med 2010;12:279–288. .
- Direct-to-consumer genetic testing: good, bad or benign? Clin Genet 2010;77:101–105. , , , , .
- A formal risk-benefit framework for genomic tests: facilitating the appropriate translation of genomics into clinical practice. Genet Med 2010;12:686–693. , , , , .
- Extending the reach of public health genomics: what should be the agenda for public health in an era of genome-based and “personalized” medicine? Genet Med 2010;12:785–791. , , , et al.; .
- Clinical validity and clinical utility of genetic tests. Curr Protoc Hum Genet 2009;Chapter 9:Unit 9.15. .
- Inherited genetic markers discovered to date are able to identify a significant number of men at considerably elevated risk for prostate cancer. Prostate 2011;71:421–430. , , , et al.
- The family history–more important than ever. N Engl J Med 2004;351:2333–2336. , , .
- Family history: a comprehensive genetic risk assessment method for the chronic conditions of adulthood. Am J Med Genet 1997;71:315–324. , , , , .
- Family history in public health practice: a genomic tool for disease prevention and health promotion. Annu Rev Public Health 2010;31:69–87 1 p following 87. , , , , .
- Association of direct-to-consumer genome-wide disease risk estimates and self-reported disease. Genet Epidemiol 2012;36:66–70. , , .
- Prospective comparison of family medical history with personal genome screening for risk assessment of common cancers. Eur J Hum Genet 2012;20:547–551. , , .
- www.navigenics.com. Accessed 23 January 2011. .
- www.medcan.com/services/genetics/personal_genome_testing. Accessed 23 January 2011. .
- The science behind the Navigenics service. 1(866):1–13. http://www.navigenics.com/static/pdf/Navigenics-TheScience.pdf. Accessed 20 January 2011. .
- Standardized human pedigree nomenclature: update and assessment of the recommendations of the National Society of Genetic Counselors. J Genet Couns 2008;17:424–433. , , , .
- Genetic variations of prostate stem cell antigen (PSCA) contribute to the risk of gastric cancer for Eastern Asians: a meta-analysis based on 16792 individuals. Gene 2012;493:83–91. , .
- SNP Guide. Navigenics Practice Enhancement Kit. 2011. .
- The measurement of observer agreement for categorical data. Biometrics 1977;33:159–174. , .
- The current landscape for direct-to-consumer genetic testing: legal, ethical, and policy issues. Annu Rev Genomics Hum Genet 2008;9:161–182. , , .
- Bioethical and clinical dilemmas of direct-to-consumer personal genomic testing: the problem of misattributed equivalence. Sci Transl Med 2010;2(17):175. , .
- Genetic susceptibility testing versus family history-based risk assessment: Impact on perceived risk of Alzheimer disease. Genet Med 2005;7:48–53. , , , et al.
- Bias in the reporting of family history: implications for clinical care. J Genet Couns 2012;21:547–556. , , , et al.
- Accuracy of reporting of family history of colorectal cancer. Gut 2004;53:291–295. , , , et al.
- Accuracy of self-reported family history is strongly influenced by the accuracy of self-reported personal health status of relatives. J Clin Epidemiol 2012;65:82–89. , , , et al.
This study was supported by the MSc in Genetic Counselling Education Fund of the University of Toronto.
L.A. acknowledges that she had full access to all the data in the study and takes responsibility for the integrity of the data, the accuracy of the data analysis, and the decision to submit for publication.