Main

The accelerating rate of genomic discoveries is rapidly increasing our understanding of the genetic basis of common diseases. Recent genome-wide association studies have identified novel susceptibility variants for type 2 diabetes, age-related macular degeneration, cancer, and many other common diseases.1 These discoveries have fueled expectations about applications of predictive genetic tests in preventive and clinical health care.2,3 It is envisioned that genetic tests will personalize medicine through targeted treatment for patients with common diseases and individualized lifestyle and dietary recommendations for high-risk individuals.4,5

Although genome-based clinical and public health applications still await empirical evidence, several companies already offer online genetic tests to predict an individual's risk of common diseases.6 These tests are based on single susceptibility genes (e.g., DNA direct7); based on genetic profiles using a limited number of variants (e.g., Sciona8 and Genovations9), or genome-wide scans (e.g., 23andMe,10 Navigenics,11 and de-CODEme12); and based on whole genome sequencing (e.g., Knome13). It is widely acknowledged that testing single susceptibility genes is uninformative for predicting common diseases as, on their own, they only minimally affect disease risk14–16 and most currently offered profiles based on a few selected variants are uninformative as they lack a firm scientific basis for the polymorphisms included.6

Companies that offer genome-wide scans take a more rigorous approach in the selection of the variants, but the clinical validity and utility of their results may also be limited at present, as susceptibility genes for common diseases are still being discovered. Because of this, risk predictions from genome-wide scans frequently become outdated when scientific knowledge progresses. Therefore, commercial companies offer updates of the risk predictions when new susceptibility genes are discovered. Given that single new variants only have a minor contribution to disease risk, we might expect that risk predictions change minimally at each update. However, as many individuals will have disease risks that are only slightly higher or lower than average,17 even minor updates may reclassify people from below to above average disease risk or vice versa, and lifestyle and nutrition recommendations may vary accordingly.

We investigated the extent to which updating of risk predictions leads to reclassification of individuals from below to above average disease risk or vice versa. Taking type 2 diabetes as an example, we compared risk predictions based on a single gene, on multiple polymorphisms and on multiple polymorphisms combined with age, sex, and body mass index (BMI). Analyses were performed using data from the Rotterdam Study, a population-based cohort of individuals aged 55 years and older.

MATERIALS AND METHODS

Subjects

The design and data collection of the Rotterdam Study was been described elsewhere.18 In short, the Rotterdam Study is a prospective, population-based, cohort study among 7983 inhabitants of a Rotterdam suburb, designed to investigate determinants of chronic diseases. Participants were aged 55 years and older. Baseline examinations took place from 1990 until 1993, and follow-up examinations were performed in 1993–1994, 1997–1999, and 2002–2004. Among these examinations, continuous surveillance on major disease outcomes was conducted. The medical ethics committee of the Erasmus Medical Center approved the study protocol, and all participants gave their written informed consent.

Data collection

The following polymorphisms19 were genotyped: TCF7L2 rs7903146 (MIM 602228), CDKAL1 rs7754840 (MIM 611259), CDKN2A/B rs10811661 (MIM 600160, MIM 600431), FTO rs8050136 (MIM 610966), HHEX rs1111875 (MIM 604420), IGF2BP2 rs4402960 (MIM 608289), KCNJ11 rs5219 (MIM 600937), PPARG rs1801282 (MIM 601487), SLC30A8 rs13266634 (MIM 611145), ADAMTS9 rs4411878 (MIM 605421), CDC123-CAMK1D rs11257622 (MIM 607957), CDKN2A/B rs1412829, JAZF1 rs1635852 (MIM 606246), NOTCH2 rs1493694 (MIM 600275), TCF2 rs4430796 (MIM 189907), THADA rs7578597 (MIM 611800), TSPAN8-LGR5 rs1353362 (MIM 600769, MIM 606667), and WFS1 rs10012946 (MIM 606201).20–24 Details on genotyping techniques, genotype success and odds ratios for the genotyped variants have been published elsewhere.25

At baseline, diagnostic criteria for prevalent cases of diabetes were a nonfasting or a postload glucose level (after oral glucose tolerance testing) ≥11.1 mmol/L and/or treatment with antidiabetic medication (oral medication or insulin) with a diagnosis of diabetes recorded by a general practitioner. During follow-up, incident cases of diabetes were diagnosed at fasting plasma glucose levels ≥7.0 mmol/L, and/or nonfasting plasma glucose levels ≥11.1 mmol/L, and/or treatment with antidiabetic medication (oral medication or insulin26,27), with a diagnosis of diabetes recorded by a general practitioner. Patients with a recorded diagnosis of type 1 diabetes were excluded from the present analyses (n = 15). BMI was calculated as weight (kg) divided by height (m) squared. Age and BMI were obtained from the baseline assessment.

Statistical analyses

Predicted risks were obtained using logistic regression analyses with type 2 diabetes (prevalent and incident cases) as the dependent variable. All polymorphisms were entered as categorical variables in the analyses, allowing effect sizes to differ between heterozygous and homozygous carriers of the risk alleles. To evaluate how risk predictions change after adding more information, we compared first risk predictions based on the strongest genetic predictor of type 2 diabetes, TCF7L2, alone, 18 polymorphisms including TCF7L2, and 18 polymorphisms plus age, sex, and BMI. Second, we compared risk predictions based on clinical factors, clinical factors and TCF7L2, and clinical factors plus all 18 polymorphisms. Predicted risks from the three models were evaluated by comparing risk distributions and discriminative accuracy and by examining reclassification. To evaluate how risk predictions change when each polymorphism is added individually, we simulated 1000 random permutations of all possible orderings of the added polymorphisms. Discriminative accuracy, measured as the area under the receiver operating characteristic curve (AUC), indicates the degree to which the predicted risks can discriminate between individuals who will develop the disease and those who will not. AUC can range from 0.50 (equal to tossing a coin) to 1.00 (perfect discrimination). Reclassification was calculated as the percentage of individuals who switched from being at increased to being at decreased risk, when compared with the average risk in the population or vice versa.28 Reclassification was assessed in individuals with complete genotype and clinical information. Analyses were performed using the SPSS software version 15.0.1 and R programming language version 2.8.0.

RESULTS

General characteristics

A total of 6544 participants were successfully genotyped for at least one polymorphism. Complete genotype information on all polymorphisms was available from 5297 participants, of whom 490 were incident and 545 were prevalent cases of type 2 diabetes (i.e., 20% had type 2 diabetes). Of those with complete genotype information, 41% were men, mean age was 69.5 years (standard deviation 9.1 years), and mean BMI was 26.3 kg/m2 (standard deviation 3.7 kg/m2). Complete information on genotype, age, sex, and BMI was available from 5111 participants. The average risk of type 2 diabetes in the population was defined as the actual prevalence (i.e., 20%).

Improving risk prediction at population level

Prediction based on the 18 polymorphisms and clinical characteristics yielded more differentiation in predicted risks than prediction based on one or multiple polymorphisms alone (Fig. 1), which means that adding more risk factors yielded more extreme risk predictions. For example, the 5% of the population indicated to be at highest risk had a predicted risk of 28.0% based on TCF7L2 but predicted risks of at least 29.7% based on the 18 polymorphisms and at least 36.8% based on the polymorphisms plus clinical factors. This increased differentiation is also reflected in higher AUCs. The AUC was 0.55 (95% CI: 0.53–0.57) for prediction based on TCF7L2, 0.60 (95% CI: 0.58–0.62) for prediction based on 18 polymorphisms, and 0.66 (95% CI: 0.64–0.68) for prediction based on 18 polymorphisms plus age, sex, and BMI. At the average risk, the main improvement in model performance was reflected in the increase in specificity. The specificity was 51.8% for prediction based on TCF7L2, 62.7% for prediction based on 18 polymorphisms, and 64.5% for prediction based on 18 polymorphisms plus age, sex, BMI (Table 1).

Fig 1
figure 1

Predictiveness curves42 for TCF7L2 alone, 18 polymorphisms alone, and 18 polymorphisms plus age, sex and body mass index. Predicted risks were obtained using logistic regression analyses. Cumulative percentage indicates the percentage of the population that has a predicted disease risk equal or lower than the risk value. For example, based on genetic testing of 18 polymorphisms, 90% (x-axis) of the individuals have a predicted risk lower than 26.8% (y-axis). a, the predictiveness curves for all participants, (b) for participants without type 2 diabetes, and (c) for participants with type 2 diabetes. BMI, body mass index (calculated as weight [kg] divided by height [m] squared).

Full size image
Table 1 Measures of model performance for dichotomizing predicted risk at average risk
Full size table

Reclassification

Predicted risks were lower than average for carriers of the TCF7L2 CC genotype (predicted risk 17.6%) and higher than average for the CT and TT genotypes (20.8% and 28.0%, respectively; Fig. 2). As indicated by the larger standard deviations predicted risks diverged after adding novel risk factors, leading to reclassification. Based on testing the 18 polymorphisms, 33.5% of the participants were reclassified: 23.6% of noncarriers switched to the increased risk category and 43.6% of the heterozygous and homozygous carriers (51.6% and 5.6%, respectively) switched to the decreased risk category (Table 2). Based on all polymorphisms, age, sex, and BMI, 28.5% of participants switched their risk category compared with prediction based on the 18 polymorphisms alone; the proportion of switchers was 26.5%, 31.4%, and 19.1% for carriers of the CC, CT and TT genotypes, respectively (data not shown). Overall, predicted risks changed from above to below average or vice versa in 50% of all individuals: 39% switched once and 11% switched twice, i.e., back to their initial risk category (Fig. 3).

Fig 2
figure 2

Predicted risk of type 2 diabetes based on TCF7L2 alone, 18 polymorphisms alone, and 18 polymorphisms plus age, sex, and body mass index. Predicted risks were obtained using logistic regression analyses. The bold line shows the median, the boxes indicate the interquartile ranges (25–75% range), and the whiskers present 1.5 times the interquartile range. The points represent outliers, and the asterisks represent extreme outliers that have values more than three times the interquartile range. BMI, body mass index (calculated as weight [kg] divided by height [m] squared).

Full size image
Table 2 Risk stratification table for the pairwise comparison of two consecutively updated prediction models
Full size table
Fig 3
figure 3

Patterns of reclassification that result from updating risk predictions. The number of reclassifications represents how many times a person switched between risk categories based on the three prediction models. For example, a person did not reclassify (reclassification is 0) if they had above average or below average risks according to all three models. A person reclassified once (reclassification is 1) if they switched risk categories from the model based on TCF7L2 to the model based on 18 polymorphisms or from the model based on 18 polymorphisms to the model including clinical factors. The table explains what percentage of people reclassifies 0, 1, or 2 times, overall and by TCF7L2 genotype. BMI, body mass index (calculated as weight [kg] divided by height [m] squared).

Full size image

Updating a model starting from age, sex, and BMI

The AUC was 0.63 (95% CI: 0.61–0.65) for prediction based on age, sex, and BMI, 0.64 (95% CI: 0.62–0.66) for prediction based on age, sex, BMI, and TCF7L2, and 0.66 (95% CI: 0.64–0.68) for prediction based on age, sex, and BMI plus 18 polymorphisms. Starting from predictive testing based on age, sex, and BMI, 13.2% of participants were reclassified after updating by TCF7L2. Based on age, sex, BMI, and all polymorphisms, 16.3% of participants switched their risk category compared with prediction based on age, sex, BMI, and TCF7L2. Overall, predicted risks changed from above to below average or vice versa in 25.6% of all individuals: 21.7% switched once and 3.9% switched twice, i.e., back to their initial risk category.

Updating by adding each polymorphism individually

Finally, we considered risk updating by each additional polymorphism individually to the model that was based on testing TCF7L2 alone, up to the model based on all 18 polymorphisms. Using 1000 random orderings in which the 17 polymorphisms can be added to the profile, we calculated that on average 47% (standard deviation 1.2%) of the participants ultimately switched at least once when risks were updated after every single polymorphism (Fig. 4). Seventeen percent switched once, and 30% switched multiple times (range 2–15) from below to above average disease risk or vice versa. When TCF7L2 was also added in a random order, on an average, 71% (standard deviation 8.3%) of the participants switched at least once (data not shown).

Fig 4
figure 4

Median percentage of reclassification in step-by-step update of prediction based on TCF7L2 to prediction based on 18 polymorphisms. The plot is obtained from a simulation of 1000 permutations of single polymorphism updates. Numbers on the x-axis represent no reclassification (i.e., number of reclassification is 0) up to reclassification at each step (i.e., number of reclassification is 17). The bold line shows the median, the boxes indicate the interquartile ranges (25–75% range), and the whiskers present 1.5 times the interquartile range.

Full size image

DISCUSSION

Using type 2 diabetes as an example, we showed that updating risk predictions by including more polymorphisms, age, sex, and BMI improved risk prediction at the population level as reflected in the higher AUC values. However, at the individual level, we found that 34% of the participants switched between risk categories when risks were updated from 1 to 18 polymorphisms and that 29% switched when age, sex, and BMI were taken into consideration. In total, 39% of the participants switched risk categories once and 11% switched twice.

Before interpreting the public health relevance of these results, two methodological issues of our study, that may affect the degree of reclassification, should be pointed out. First, although we investigated 18 established type 2 diabetes polymorphisms, only about half were statistically significantly associated with type 2 diabetes risk in our population.25 This is in line with other studies that investigated the combined predictive value of the 18 polymorphisms, which also found that not all polymorphisms were statistically significantly associated with the disease.29–32 If the effect sizes of all polymorphisms in our study had been the same as in the original studies that had identified their associations, we would have observed a larger variation in predicted risks (Fig. 2) and likely also more reclassification.

Second, we focused on changes in risk prediction based on TCF7L2 with that based on 18 polymorphisms, age, sex, and BMI, which reflects the practice of commercial companies. However, in clinical settings, it is more logical to update a model based on recognized clinical risk factors. We showed that when risks were updated from age, sex, and BMI to age, sex, BMI, and TCF7L2, and further to age, sex, BMI, and all 18 polymorphisms, 22% of individuals switched risk categories once, and 4% twice.

We compared risk prediction based on TCF7L2 with that based on 18 polymorphisms, age, sex, and BMI, but we considered only two risk updates: one based on adding 17 polymorphisms and one on adding age, sex, and BMI. The percentage of reclassification was even higher when we considered risk updating by each additional polymorphism individually, as is done by the companies. The exact percentage of reclassification then varies with the order in which polymorphisms are added, and for 1000 random orderings of the 17 polymorphisms, we calculated that 47% of the participants would have switched at least once when risks were updated after every single polymorphism, when compared with 34% of the participants when the 17 polymorphisms were added in a single update.

The reason why people switch between risk categories is that the added polymorphisms may have different effects on disease risk compared with the polymorphisms already considered. Figure 2 showed that individuals who are at increased risk according to their TCF7L2 genotype may be at decreased risk of type 2 diabetes, when all 18 polymorphisms are considered if they inherited protective genotypes on many other polymorphisms. In our analyses, the risk increase conferred by TCF7L2 risk alleles was counterbalanced by protective effects of other alleles in 6% of the individuals and was counterbalanced by young age, male sex, and normal BMI in 19% of the individuals.

Our final prediction model included 18 polymorphisms, age, sex, and BMI, but it is important to realize that risk predictions can be further improved. Even with our current understanding of genomic factors, prediction of type 2 diabetes risk can be improved by also considering family history and fasting plasma glucose levels,32 factors that are currently not considered by the companies that offer genome-wide scans. In the future, risk prediction may be improved by the addition of novel genetic factors, novel biomarkers, and with gene-gene and gene-environment interactions if these are demonstrated in future genetic epidemiologic studies.33,34 Thus, the risk predictions presented in this article are not final and individuals may be subject to further reclassification as science advances.

Commercial companies assert that genome-wide scans will help consumers to learn their likelihood of developing a disease, but it is widely agreed that risk predictions and results from genetic tests are difficult for the lay public to understand.35,36 To facilitate the interpretation of risk estimates, companies present the predicted risks together with the average risk of the disease for the total population or for a sex- and age-matched population. Individuals can thereby learn whether they are at higher or lower risk than others and, based on this information, may decide to make lifestyle and dietary changes. However, individuals differ in the way they value the information gained from genetic testing. Some may find a slight increase in predicted risk sufficiently motivating to adopt or maintain healthy behaviors, whereas others may not even change their behavior when they learn that their risk is markedly increased. A systematic review of the psychological and behavioral impact of genetic testing for hereditary nonpolyposis carcinoma, hereditary breast, and ovarian cancer, and Alzheimer disease reported that, generally, 12 months after testing, perceived risk in carriers decreased to the pretest level or even below it.37 A study on the harms and benefits of APOE genotyping in first-degree relatives of patients with Alzheimer disease reported that disclosure of genotype status increased the motivation for risk reduction activities.38 Note that previous studies mainly addressed psychological and behavioral impact of genetic testing for monogenic and major gene disorders, and these findings cannot be directly translated to the impact of low-risk susceptibility genetic testing.

If individuals are informed that they have switched categories from above to below average risk of disease, or vice versa, their perceptions about the need for health behavior changes may vary accordingly. In current commercial genome scans, risks are updated on every new gene discovery and individuals may frequently reclassify over time. To date, it is unknown how individuals respond to variations in risk predictions over time and how it affects their perceptions about the threat of being at increased risk.39–41 Because health behavior changes are difficult to achieve, we might expect that individuals will become insensitive to risk information if they learn that their risk status may change over time, even without any lifestyle changes. Also, reclassification primarily focuses on changes in risk compared with the average risk and less on the absolute risks of disease. The absolute risk should be important as well in decision making about healthy behavior, and it is of interest to find out whether absolute or comparative risk information influences health behavior change. Such potentially adverse consequences of updating risk predictions warrant further investigation.

The companies that offer genome-wide scans or whole genome sequencing for the prediction of multiple diseases take a higher scientific standard for the selection of susceptibility variants than those previously reviewed.6 They include only variants that have been consistently associated in multiple studies, and transparently present the polymorphisms that constitute genetic profiles for each disease, including references to scientific studies demonstrating their impact on disease risk. Nevertheless, with scientific advance their risk predictions may further improve, as causation of disease is better understood, and the benefit of these updates at the individual level are unclear. This does not imply that the introduction of genome-based applications in health care should wait until we completely understand the etiology of diseases, but we need to recognize that a premature introduction may have adverse effects.