Retinopathy Signs Improved Prediction and Reclassification of Cardiovascular Disease Risk in Diabetes: A prospective cohort study

CVD risk prediction in diabetics is imperfect, as risk models are derived mainly from the general population. We investigate whether the addition of retinopathy and retinal vascular caliber improve CVD prediction beyond established risk factors in persons with diabetes. We recruited participants from the Singapore Malay Eye Study (SiMES, 2004–2006) and Singapore Prospective Study Program (SP2, 2004–2007), diagnosed with diabetes but no known history of CVD at baseline. Retinopathy and retinal vascular (arteriolar and venular) caliber measurements were added to risk prediction models derived from Cox regression model that included established CVD risk factors and serum biomarkers in SiMES, and validated this internally and externally in SP2. We found that the addition of retinal parameters improved discrimination compared to the addition of biochemical markers of estimated glomerular filtration rate (eGFR) and high-sensitivity C-reactive protein (hsCRP). This was even better when the retinal parameters and biomarkers were used in combination (C statistic 0.721 to 0.774, p = 0.013), showing improved discrimination, and overall reclassification (NRI = 17.0%, p = 0.004). External validation was consistent (C-statistics from 0.763 to 0.813, p = 0.045; NRI = 19.11%, p = 0.036). Our findings show that in persons with diabetes, retinopathy and retinal microvascular parameters add significant incremental value in reclassifying CVD risk, beyond established risk factors.


Results
The baseline demographic characteristics of SiMES and SP2 cohorts are shown in Table 1. The mean follow-up was 6.1 years (range: 0.2-7.1, median: 6.3) for SiMES, and 5.78 years (0.29-7.18, 6.06) for SP2. Participants in the SiMES cohort were older, had more women and hypertensive subjects compared with SP2. Participants in SiMES also had higher BMI, systolic blood pressure, total cholesterol and LDL, creatinine level, HbA1c level, and lower eGFR. More participants had retinopathy in the SiMES cohort. However, there was no significant difference in CRAE and CRVE or hsCRP levels between the two cohorts.
During follow-up, 86 cases (12.1%) of CVD occurred in SiMES, of which 29 were MI, 19 were CVD death, 22 were strokes and 16 had mixed etiologies. Of the 31 cases (7.3%) of incident CVD in SP2, 13 were MI, 4 were CVD death, 7 were strokes and 7 had mixed etiologies. Between the two cohorts, there was no difference in incident MI (p = 0.298) or incident stroke (p = 0.187). However, there were significantly more CVD events in SiMES compared with SP2 (p = 0.01). Table 2 shows that in Model 1, SiMES participants were older, with higher systolic blood pressure and higher HbA1c level were more likely to develop incident CVD. The predictive discrimination for incidence of CVD event of this model was modest, with a C statistic of 0.721. Both a lower GFR and higher hsCRP were significantly associated with increased risk of incident CVD in Model 2. Serum biomarkers improved predictive discrimination of incident CVD, with an increase in the C statistic to 0.747 (p = 0.056). When retinal measures were included in Model 3, discrimination was even better, with an increase in C statistics to 0.751 (p = 0.011). In both Models 2 and 3, the difference compared to Model 1 was not statistically significant (p = 0.0556 and 0.0542, respectively). The combination of hsCRP, eGFR and the presence of retinopathy in Model 4, significantly improved risk discrimination, with a C statistic of 0.762 (p = 0.011). Model 5 similarly revealed that lower eGFR), higher hsCRP, lower CRAE and the presence of retinopathy are predictors of incident CVD. Model 5 showed a gain in indices of model fit and discrimination, with a statistically significant increase in the C-statistic to 0.774 compared with Model 1 (5.3%, p = 0.003) (ESM Fig. 3).
The Hosmer-Lemeshow χ 2 p values were p = 0.91 for the Model 1, p = 0.217 for Model 2, p = 0.633 for Model 3, p = 0.677 for Model 4, and p = 0.402 for Model 5, indicating that none of the models had a significant lack of fit. Table 3 shows the reclassification of CVD risk of Model 1 compared to Model 5. Among individuals with incident CVD event, 12 (14%) participants were reclassified to a higher risk category and 9 (10.5%) participants to a lower risk category (NRI = 3.49%; p = 0.512). In individuals with no incident CVD, the newer model reclassified 62 (10%) participants to a higher risk category and 146 (23.4%) to a lower risk category (NRI = 13.5%, p < 0.001). The overall improvement in net reclassification with the new model was statistically significant (NRI = 17%, p = 0.004).

Internal Validation. The reclassification table by cross validation within the SiMES cohort is shown in
Supplementary Table S1. In subjects with incident CVD, 14 (16.3%) subjects were reclassified to a higher risk category and 11 (12.8%) to a lower risk category. In subjects without incident CVD event, 67 (10.8%) were reclassified to a higher risk, and 148 (23.8%) to a lower risk. The new model performed consistently, with a downward reclassification of subjects with no incident CVD event (NRI 13%; p < 0.001). As a whole, the new model yielded an NRI of 16.5% (p = 0.009), which indicates a better risk stratification of the new model.  Table S2). Hazard ratios were largely in the same direction and of similar magnitudes in both cohorts for the Model 1. The new model showed that the presence of retinopathy (HR 0.98, 95% CI, 0.42-2.28) was not associated with incident CVD in SP2. Surprisingly, an increase in venular caliber instead, was significantly associated with CVD risk (HR 2.15, 95% CI, 1.38-3.32) unlike the SiMES cohort. Nevertheless, the C statistics for the new model increased from 0.763 to 0.813 (p = 0.045), representing a significant improvement in predictive discrimination compared to Model 1 in SP2. The Hosmer-Lemeshow χ 2 p values were p = 0.80 and p = 0.30 for Models 1 and 5, respectively), demonstrating no evidence of poor fit.
For individuals who remained free of CVD, the NRI was statistically significant at 22.3% (p < 0.001), demonstrating an improved identification of truly low-risk patients with additional of retinal parameters. Serum biomarkers and retinal microvascular measures significantly improved CVD risk stratification beyond established risk factors in the SP2 cohort (NRI 19.1%; p = 0.036). This finding was similar to our initial analysis in SiMES (Table 4). Ethnicity was not significantly associated with incident CVD in SP2.

Discussion
In this prospective population-based study, we characterized CVD risk models with the addition of biochemical markers of hsCRP and eGFR, as well as retinal markers of retinopathy and retinal vascular caliber measured from retinal photographs, then assessed the performance of the models with measures of discrimination, calibration and reclassification. The addition of retinal and serum biomarkers markers significantly improved the predictive value of models compared to traditional risk factors alone, with the addition of retinal parameters proving to be stronger predictors of CVD compared with serum biomarkers of hsCRP and eGFR. When used in combination, both predictive discrimination and reclassification of CVD event were significantly better. Our risk model showed good robustness in both internal and external validation.
To our knowledge, our study is the first to assess the incremental value of retinal parameters to established risk factors for CVD risk prediction and stratification in patients with diabetes. Large epidemiology studies have shown that retinal measures such as arteriolar narrowing and venular widening, are associated with all-cause mortality and incident CVD [16][17][18][20][21][22][23] , therefore changes in retinal vasculature may reflect similar changes in the systemic peripheral and cerebrovascular circulation, providing additional insights into the structure and function of small vessels that are important in the development of CVD 24 . Microcirculation changes in diabetes occur as a result of impaired hemodynamic autoregulation 25 , endothelial dysfunction and defects of metabolism 26 , which incites a inflammatory response 27 . This suggests that retinopathy and retinal microvascular changes are sensitive markers of microvascular dysfunction related to CVD in patients with diabetes. Our current study thus underscored the relationship between retinal measures and incident CVD not only in general population but also in persons with diabetes. We further extend this knowledge by showing that the inclusion of these signs in a CVD risk model demonstrates significant improvement in predictive discrimination and reclassification.
We also demonstrated that a "multiple marker" approach might be better than traditional risk scores for CVD prediction in patients with diabetes. Previous studies that incorporated novel markers (e.g. B-type natriuretic peptide, CRP) into conventional risk models 11,12,28 , used alone or in tandem with other markers, showed modest change in the C statistic, as they were already strongly correlated with established risk factors, and therefore could not substantially improve risk prediction 11,29 . Moreover, previous studies that reviewed the performance of CVD risk scores in persons with diabetes lack complete measures of predictive incremental performance, consisting of discrimination, calibration and model fit 30 . The unique advantage of retinal parameters over other biomarkers in risk prediction is that the retinal vasculature directly reflects microvascular damage and are therefore discrimanatory, whereas other factors are not as specific 31 . While the addition of novel serum biomarkers to established risk factors led to minimal change in the reclassification of cases, our model successfully moved individuals with no incident CVD event to a lower risk category. Better discrimination of CVD risk will empower physicians with the improved ability to administer appropriate treatment, and therefore reduce unnecessary intensification of therapy in patients who are at lower risk of CVD. The utility of our model with retinal parameters in clinical practice is indeed highly feasible, and merits consideration. Many patients with diabetes undergo retinal photography as part of routine yearly eye screening. Data from photographs can be incorporated into risk prediction models without much additional cost. Future studies to assess the acceptability to patients in clinical settings would be insightful.
Strengths of our study include two independent population-based samples, and masked standardized retinal assessment. External validation was performed in an independent population, which proved the accuracy and  generalizability of our model 29,31 . We used C statistic and NRI, both shown to be more clinically relevant than traditional odds ratio in risk prediction. Moreover, we used NRI to access the effects of the inclusion of the retinal microvascular measures into the models in this study. However, NRI has its limitations; for example, NRI does not provide information about the performances of the models and it weighs reclassifications indiscriminately for three or more risk categories 32 . The incremental and clinical value of the retinal microvascular measures should be interpreted within these limitations and further confirmed in other studies. Limitations include relatively small increase in C statistics, indicating minimal change in overall discrimination, and possibly insufficient clinically meaningful improvement for our model to be adopted as a screening tool. However, the ease of obtaining retinal data in a quick and noninvasive manner may still warrant consideration for future application of these novel markers. Our study had a relatively short follow-up period, and as we only included subjects with diabetes in the current analysis, the sample size is relative small for a population based study. After refitting in SP2, most of the predicting factors from SiMES were insignificant, likely due to the sample size and event rates being insufficient to detect significance in the SP2 cohort. Furthermore, the differences in the strength of associations between risk factors were not taken into account when formulating the prediction models in SP2. Indeed, refitting and recalibration of a model is especially difficult in a heterogenous population. Overfitting in the models of SP2 is also a limitation in out study. Further validation of our model in other cohorts with larger sample sizes will be required. Despite having multiple ethnic groups within SP2, generalization of our findings across ethnic groups may be limited due to small numbers of incident CVD, and further studies are required to recalibrate these functions to fit other populations.
In conclusion, we demonstrated that an assessment of retinopathy and retinal microvascular measures captured from retinal photographs provides prognostic information on risk of CVD and significantly improves CVD risk stratification when incorporated into established risk models in persons with diabetes. The translation of retinal photography as a useful tool to improve the management of diabetes in clinical practice is promising but requires further evaluation.

Methods
Study populations. Data from two independent population-based studies was used. Individual data was obtained from the Singapore Malay Eye Study (SiMES, 2004(SiMES, -2006 as the primary cohort to develop the CVD prediction model, which was then validated in the Singapore Prospective Study Program (SP2, 2004-2007). Study populations and methodology for both cohorts have been previously reported 33,34 . Of the 3280 eligible individuals from SiMES 33 selected by an age-stratified random sampling method, 709 persons with diabetes were included in Supplementary Figure S1. 425 persons with diabetes were included from 7744 randomly recruited participants across four cross-sectional studies in SP2 34 (see Supplementary Figure S2).
The methodology for SiMES and SP2 were similar except that non-fasting blood samples were collected in SiMES and fasting samples were collected in SP2. Written, informed consent was obtained from each participant and both studies adhered to the Declaration of Helsinki. Ethical approval was obtained from the Institutional Review Board at the Singapore Eye Research Institute (SiMES) and the National University of Singapore and Singapore General Hospital (SP2).  diabetes or use of diabetic medication, if random glucose was ≥ 11.1 mmol/L or glycosylated hemoglobin (HbA1c) level ≥ 6.5%. In SP2, diabetes was diagnosed if participants reported physician diagnosed diabetes or use of diabetes medication, or if fasting plasma glucose was ≥ 7 mmol/L, and HbA1C> = 6.5%.

Assessment of established risk factors.
Biochemical testing of serum total cholesterol, high-density lipoprotein cholesterol (HDL), low-density lipoprotein cholesterol (LDL), HbA1c, hsCRP and eGFR were obtained from each participant. Current smokers were defined as smoking 1 cigarette or more per day (ie, current versus past/never). Measurements of body mass index (BMI) and blood pressure have previously been reported in detail 33,34 . eGFR was estimated from serum creatinine using the recently developed Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation 35,36 .
Retinal Photography. Digital fundus photography was taken with a 45° digital retinal camera (Canon CR-DGi with a 10D SLR digital camera back; Canon, Tokyo, Japan) after pupil dilation using tropicamide 1% and phenylephrine hydrochloride 2.5%. Two retinal images of each eye were obtained, one centered on the optic disc and one centered at the fovea. The spatial resolution of each image was 3072 by 2048 pixels and the images were stored without compression before analysis. Statistics. Descriptive data were presented as mean (standard deviation [SD]) for continuous variables or number (percentages) of participants for categorical variables. Cox proportional hazard regression was used to compute hazard ratios (HRs) with 95% confidence interval (CI) for the associations between covariates and CVD outcomes. Proportional hazard assumption was confirmed for all predictors with Schoenfeld's residuals. Potential predictors of CVD were evaluated with four models, retaining significant variables (p < 0.05) at each stage. Variables were considered as candidates for the model if they were found to be significant at alpha < 0.10 by univariate analysis. Variables that remained significant at alpha < 0.05 were retained, as well as those that were identified as confounders. Although some confounders were not initially selected due to nonsignificance (such as smoking and cholesterol), as important factors in the Framingham risk calculations, they were included as they may be important contributors to the risk prediction in the presence of the other significant variables. Our basic model (Model 1) included traditional CVD risk factors 39 . Serum CVD biomarkers (eGFR and hsCRP) were added to Model 1 (Model 2). Retinal signs were added in Model 3, and separately in Model 4 (Model 2 and retinopathy) and Model 5 (Model 4 and retinal vascular calibers). The variables were retained at each stage so as to build up the nested model structure eventually.

Retinopathy and Retinal
Prognostic comparisons between models were done using Harrell's concordance C-statistic 40 . Effects of the inclusion of the retinal microvascular measures into the models were assessed using the Net Reclassification Improvement (NRI) 41 .
Because there are no recommended risk thresholds for diabetes in clinical practice, the predicted probabilities of CVD event were treated as a continuous variable and the approximate tertile cutoffs were used as threshold points. Based on these 2 cut-off points, subjects were stratified into low, intermediate, and high-risk categories. The predicted probability for the incident CVD outcome were regenerated using both retinal measures and established risk factors, and the subjects were re-assigned to low, intermediate or high risk categories.
Hosmer-Lemeshow chi-square statistic was calculated for the models to test goodness-of-fit. P < 0.05 represents a significant difference between the expected and observed event rates, indicating that the model is not well calibrated.
Validation of the prediction model. The prediction models were validated using internal validation and external validation.
In order to avoid over-fitting and optimism in model performance, the final model for CVD events among diabetic individuals was internally cross validated. Internal validation is a measure of how well a model is able to predict the outcome for new observations that were not used in developing the model. Ten-fold cross validation was implemented by randomly generating 10 test samples using 10-fold cross-validation within the SiMES cohort. The data was randomly spilt into 10 mutually exclusive subsets (S1-S10) of equal sizes containing approximately the same number of events. From this, we generated 10 test samples leaving out subset S1 and so on. NRI was then calculated using the statistics from all 10 subsets.
To externally validate the models in the SP2, the strength of associations between risk factors identified and CVD were first evaluated for similarities between the SP2 and SiMES cohorts. The coefficients (hazard ratios) for all variables selected in the SiMES model were refitted to the SP2 cohort, even though some of the risk factors in SiMES did not show significance in SP2. As our aim was to validate Model 5 from SiMES in another cohort, we kept all the variables to remain consistent with the SiMES model. Hosmer-Lemeshow χ 2 p values were calculated to calibrate Model 1 and Model 5 in the SP2 cohort. The coefficients from the SiMES models were then recalibrated to the baseline survival of SP2, in order to perform model test similar to those in SiMES. As we expected there to be ethnic differences between groups, ethnicity first reviewed as an independent variable in univariate analysis. However, it was found not to be significant in SP2, so we chose to omit ethnicity to remain consistent with the SiMES model.
We regarded P values of < 0.05 from 2-sided tests to indicate statistical significance. All statistical analyses were performed using the Stata Statistical computer package (STATA Statistical Software, Version 12, Statacorp, College Station, Texas, USA).