Introduction

Cardiovascular Diseases (CVDs) are one of the leading causes of death globally1. The annual mortality of CVDs has been predicted to increase from 17.5 million in 2012 to 22.2 million by 2030. Then, CVDs would be the number one cause of death and disability worldwide2. In Iran, the share of non-communicable diseases in mortality increased from 57% in 1990 to 76% in 20103,4 mostly due to ischemic heart disease, stroke, and other vascular diseases, which are generally considered CVDs5.

Most of the deaths related to CVDs are premature and preventable and can be improved using health management, effective diets, lifestyle interventions, and medication interventions6. A useful method to prevent CVDs is to assess the risk of CVDs regularly and to modify the lifestyle or the clinical treatment accordingly. Since the treatment of these diseases imposes heavy burdens and high costs on healthcare systems7, their early diagnosis and appropriate treatment have significant health benefits for patients with the highest absolute risk of CVDs8. Hence, the early prevention of CVDs can be considered the most useful and cost-effective intervention strategy. To effectively implement prevention strategies, reliable tools are required to identify individuals without overt CVDs who are at high risk for CVDs consequences in future. One of the tools utilized for predicting CVDs risk is the Framingham risk score, which can estimate the 10-year risk of fatal and non-fatal cardiovascular consequences.

In 2008, the Framingham risk scores developed by D’Agostino et al. that are calculated using either a laboratory-based or a non-laboratory-based algorithm. The laboratory-based model is based on age, sex, SBP and treatment status, current smoking, diabetes, cholesterol, and HDL. However, in developing countries, laboratory markers may not always be available at primary healthcare centers or people may not be able to pay for the cost of laboratory tests. In such cases, the non-laboratory-based algorithm can be used. The non-laboratory-based model is based on age, sex, SBP and treatment status, current smoking, diabetes and, BMI9.

The Framingham model, on the other hand, is a complex mathematical equation. Thus, the risk calculation formula has been simplified to a points-based system to make it easier to be used in places where there is no computer or calculator10. Determining the risk score using the points-based risk scoring method is easier than the equation-based method and is more applicable in places where primary healthcare is provided.

Given that CVDs are the most important cause of death in the Iranian population and their burden has been expected to increase in the coming years11, it seems necessary to estimate the 10-year risk of CVDs based on the risk algorithms. There is currently no specific CVD risk prediction tool developed in Iran. Therefore, we use created tools in other countries such as Framingham risk score and Systematic Coronary Risk Evaluation (SCORE) models. Some studies have compared different risk prediction models. In studies carried out in Sri Lanka, Rwanda, and Canada, the agreement between BMI-based and cholesterol-based models was examined12,13,14. In Iran, Bavarsad et al. compared Framingham and World health organization/International society of hypertension (WHO/ISH), and SCORE models15. Mirzaei et al. assessed Framingham and WHO/ISH models16. But no study has been done to examine the agreement between laboratory-based and non-laboratory-based Framingham risk scores.

The important issue to be examined is whether the two laboratory-based and non-laboratory-based risk models provide a similar estimate of the CVDs risk in an individual according to the points-based risk scoring system. Therefore, the present study aims to evaluate the agreement between laboratory-based and non-laboratory-based risk scores in a large population.

Methods

Pars cohort study design

This cross-sectional study was conducted using the baseline data of Pars cohort study. Pars cohort study is a part of the Persian Prospective Epidemiological Research Studies in Iranian Adults (PERSIAN) program. PERSIAN cohort study was designed and implemented in 2014 and included 18 different geographical, ethnic, and climatic groups in 18 provinces of Iran17.

Details of the Pars cohort study have already been published18. Briefly, it was conducted on 9,264 individuals aged 40–75 years living in Valashahr and neighboring villages (southern Iran). In Pars cohort study, trained interviewers collected information about demographic characteristics, lifestyle, and health history of the participants using structured questionnaires. They also measured their height, weight, and blood pressure. Blood samples were also taken for biochemical tests. In the present study, people with a history of CVDs or strokes were excluded. The sample contained 8138 cases, and all methods were performed in accordance with PERSIAN instructions.

CVDs risk

In this study, the 10-year risk of CVDs was calculated using the Framingham risk scores of laboratory-based and non-laboratory-based models. For the laboratory-based model, scores were calculated using age, sex (male/female), systolic blood pressure (mmHg) and treatment status, current smoking status (yes/no), diabetes (yes/no), and total cholesterol and HDL (mg/dL). For the non-laboratory-based model, scores were calculated using age, sex (male/female), systolic blood pressure (mmHg) and treatment status, current smoking status (yes/no), diabetes (yes/no), and BMI (kg/m2)9.

This study considered a smoker as someone who had smoked ≥ 100 cigarettes, and a current smoker as someone who smoked regularly. Diabetes status was assessed by the previous history of the disease. Blood pressure was measured by a trained individual using a mercury sphygmomanometer after five minutes of sitting. Blood pressure was measured twice with a 10-min interval from each arm and the mean blood pressure was recorded. Cholesterol and HDL were tested in the laboratory. Finally, BMI was evaluated by dividing weight by height squared (kg/m2).

Statistical analysis

Percentage was reported for grouped data and mean and standard deviation for quantitative data. Chi-square and t-test were used for categorical and continuous variables, respectively. The risk of CVDs was calculated for laboratory-based and non-laboratory-based models using the points‐based risk‐scoring system. Two methods were used to determine the agreement between the two models. In the first method, the risk score of CVDs was considered a quantitative variable and the agreement between the two models was examined using Bland Altman plots. It means that the risk difference between the non-laboratory-based and laboratory-based models was calculated after determining the risk14. The difference between the mean risk scores was also calculated by gender and age groups (< 60 and ≥ 60 years). In Bland Altman plots, the difference between the two scores was shown on the vertical axis and the mean of the two scores on the horizontal axis. Since the true risk of CVDs is uncertain for each individual, the mean of laboratory-based and non-laboratory-based scores is the best estimate available14. The mean difference of the scores + /− two standard deviations represent 95% of the limit of agreement. The interval made by 95% of the limit of agreement indicates that 95% of the difference between the two scores is not real19.

In the second method, the risk of CVDs was categorized. In both models, individuals were divided into low-risk (< 10%), moderate-risk (10–20%), and high-risk (> 20%) groups. Then, the agreement between the laboratory-based and non-laboratory-based risk models was determined using kappa statistics. Kappa statistics < 0 indicated the agreement less than odds and values 0.01–0.20, 0.21–0.40, 0.41–0.60, 0.61–0.80, and 0.81–0.99 represented slight, fair, moderate, substantial, and almost complete agreement, respectively20.

Ethical considerations

This study was approved by the Ethics Committee of Shiraz University of Medical Sciences (IR.SUMS.REC.1398.860). The data were collected anonymously and informed consent forms were obtained from the participants.

Results

Among the 8138 participants in this study, 3789 (46.56%) were male. The mean age of the participants was 51.65 ± 9.06 years. In addition, 14.41% were smokers. The prevalence of smoking was significantly higher in males (30.22% vs. 0.64% in females). The prevalence of hypertension was 12.79%. Hypertension was significantly more prevalent in females (17.68%vs. 7.18% in males). Besides, the prevalence of diabetes was 8.38%. Diabetes was significantly more prevalent in females (10.99% vs. 5.38% in males). Abdominal obesity was significantly higher in females (89.33%vs. 70.60% in males).

The mean diastolic blood pressure was significantly higher in males than in females (73.44 ± 11.61 vs. 72.92 ± 11.91). However, the mean systolic blood pressure was higher in females than in males (111.54 ± 19.64 vs. 110.78 ± 17.50). The means of HDL and cholesterol were also significantly higher in females compared to males. BMI was also significantly higher in females than in males (27.05 ± 4.73 vs. 24.31 ± 4.08).

The mean of the 10-year risk of CVDs in the general research population was higher in the non-laboratory-based model than in the laboratory-based model (9.38 ± 7.38 vs. 6.68 ± 6.15). In addition, the mean of the 10-year risk of CVDs was significantly higher in males compared to females in both models (Table 1).

Table 1 Reporting of the participants’ characteristics.
Full size table

Mean differences in risk scores

Among all participants, the mean difference between the laboratory-based and non-laboratory-based scores was 2.69% (95% CI: 2.63% to 2.76%), which was 3.97% (95% CI: 3.86% to 4.08%) for males and 1.58% (95% CI: 1.51% to 1.64%) for females. The mean difference in scores was 2.23% (95% CI: 2.16% to 2.29%) among the participants < 60 years old and 4.48% (95% CI: 4.28% to 4.68%) among those ≥ 60 years old.

The mean difference of the scores was also calculated based on the age groups. Accordingly, the mean difference of the scores was 3.43% (95% CI: 3.33% to 3.52%) in the males < 60 years old, 6.11% (95% CI: 5.79% to 6.44%) among the males ≥ 60 years old, 1.17% (95% CI: 1.11% to 1.23%) in the females < 60 years old, and 3.11% (95% CI: 2.90% to 3.32%) among the females ≥ 60 years old.

Bland–Altman plots/limits of agreement

Bland Altman plots of the agreement between the two risk scores for males and females < 60 and ≥ 60 years old have been presented in Fig. 1. Accordingly, the limit of agreement was better for females < 60 years old compared to females ≥ 60 years old, males < 60 years old, and males ≥ 60 years old. The limit of agreement was −2.27% to 4.61% for females < 60 years old, 9.67% to -3.45% for females ≥ 60 years old, -2.5% to 8.91% for males < 60 years old, and -3.01% to 15.23% for males ≥ 60 years old. The limit of agreement was wider for males ≥ 60 years old in comparison to other groups.

Figure 1
figure 1

Bland–Altman plots showing agreement between the laboratory-based and non-laboratory-based CVDs risk scores.

Full size image

Categorical agreement

In the total research population, the agreement of the two risk scores was 78.79%. The agreement between the two risk scores according to the risk score categories in males and females has been presented in Tables 2 and 3, respectively. For males, the agreement between the risk scores was equal to 66.43% (kappa = 0.42, Standard Error (SE) = 0.01). It should be noted that there was a larger number of participants in the high-risk group for the non-laboratory-based model than in the laboratory-based model (733 vs. 320). Disagreement was observed in both low-risk and high-risk groups. Non-laboratory-based risk groups tended to show a higher risk, such a way that 1238 males (32.67%) were in the risk groups higher than the laboratory-based risk, but only 12 (0.32%) were in the lower risk group.

Table 2 Agreement between the laboratory-based and non-laboratory-based risk scores according to the grouped risk in males.
Full size table
Table 3 Agreement between the laboratory-based and non-laboratory-based risk scores according to the grouped risk in females.
Full size table

For males < 60 years old, the agreement was 71.95% (kappa = 0.39, SE = 0.01). For this age group, the disagreement was observed in both directions. In the non-laboratory-based model, there was a larger number of participants in the high-risk group compared to the laboratory-based model (249 vs. 89). In the non-laboratory-risk categories, there was a larger number of participants in the higher risk groups than in the laboratory-based risk categories [829 (27.42%)], but only 12 males (0.40%) were in the lower risk groups.

For males ≥ 60 years old, the agreement was 44.65% (kappa = 0.14, SE = 0.02). In this age group, there were more participants in the high-risk group in the non-laboratory-based model than in the laboratory-based model (484 vs. 231). In the non-laboratory-based risk model, there was no male participant in the low-risk group, while there were 186 males (24.28%) in the low-risk group in the laboratory-based model. In the non-laboratory-based risk group, there was a larger number of participants in the higher risk groups compared to the laboratory-based risk categories [409 (53.39%)], but there were no males in the lower risk groups.

For all females, the agreement was 89.56% (kappa = 0.59, SE = 0.01). In the non-laboratory-based risk model, there was a larger number of females in the high-risk group in comparison to the laboratory-based risk model (207 vs. 94). The disagreement was observed in both low-risk and high-risk groups. Yet, non-laboratory-based risk categories were more likely to show a higher risk compared to the laboratory-based risk categories. Accordingly, there were 425 females (9.77%) in the higher risk groups, but only 20 (0.46%) in the lower risk groups.

Considering the females < 60 years old, the agreement was 94.26% (kappa = 0.57, SE = 0.02). For this age group, the disagreement was observed in both low-risk and high-risk groups. In the non-laboratory-based risk model, there was a larger number of participants in the high-risk group compared to the laboratory-based risk model (45 vs. 17). In the non-laboratory-risk categories, there were more females in the higher risk groups (181, 5.27%) in comparison to the laboratory-based risk categories, but there were only 14 females (0.41%) in the lower risk groups.

For females ≥ 60 years old, the agreement was 71.91% (kappa = 0.51, SE = 0.02). In the non-laboratory-based risk model, there was a larger number of females in the high-risk group (162 vs. 77) in comparison to the laboratory-based risk model. In the non-laboratory-based risk categories, there were more females in the higher risk groups (244, 26.67%) compared to the laboratory-based risk categories, but there were only six females (0.65%) in the lower risk groups.

Discussion

In the present study, the agreement between Framingham 10-year general CVDs risk was measured by two laboratory-based and non-laboratory-based models using Bland Altman plots and kappa statistics in a large population using the points-based risk scoring system. Since no calculator or computer program may be available to calculate the Framingham risk according to the risk equation in the centers where primary healthcare is provided, the staff of these centers calculate the risk of CVDs using the points-based risk scoring system. On the other hand, due to the high cost and lack of resources, it is not always possible for patients to perform laboratory tests and in some settings, the non-laboratory-based method is used to determine the risk of CVDs.

Up to now, few studies have measured the agreement between laboratory-based and non-laboratory-based CVDs risk scores using the risk scoring equation-based system that uses Cox proportional hazards regression. In the current study, Bland–Altman plots showed that the agreement between the two risk scores was better among the females aged < 60 years and was less among the males ≥ 60 years old. Moreover, the limit of agreement was appropriate for the females < 60 years old (95% CI: -2.27% to 4.61%), but not for other groups (95% CI: -3.45% to 9.67% for females ≥ 60 years old and 95% CI: -2.05% to 8.91% for males < 60 years old). For males ≥ 60 years old, the limit of agreement was wider in comparison to other age groups (95% CI: -3.01% to 15.23%). In a study carried out on South Asian Canadians, the agreement between BMI-based and cholesterol-based models was examined with an equation-based risk scoring system. The results were similar to those of the present study and showed that the difference between the two scores and the change in the differences increased by increase in the mean 10-year risk of CVDs. In terms of clinical significance, the limit of agreement was appropriate for the females < 60 years old14. In this regard, the current study found that the limit of agreement was appropriate for males and females < 60 years old, but not for the other participants.

In this study, the mean difference between the risk scores of laboratory-based and non-laboratory-based models in the total population was 2.69%, which was higher in males than in females (3.97% vs. 1.58%). However, according to grouping, the mean difference in scores was higher in older females and males (3.43% in males < 60 years old and 6.11% in males ≥ 60 years old). Moreover, the mean difference in scores was less in females than in males (1.17% in females < 60 years old and 3.11% in those ≥ 60 years old). In another study, the mean difference in the scores of the two models was lower compared to the present study (0.5% among males and 0.6% among females)14.

In the present study, the kappa statistics for the risk categories in the two models indicated that the agreement between the two risk scores was better in females than in males. The agreement was moderate in all females although the kappa statistic was higher in the females < 60 years old compared to those aged ≥ 60 years, but both age groups had a moderate agreement. The agreement was also moderate for all males. Nonetheless, when kappa statistics were measured by age groups, the agreement was fair for the males < 60 years old and slight for those aged ≥ 60 years. Pandya et al. revealed a high agreement between the laboratory-based and non-laboratory-based risk scores, and stated that the non-laboratory-based score could be a good proxy of Framingham risk scores in case resources were limited21.

Gaziano et al. also showed that the non-laboratory-based model correctly predicted the fatal and non-fatal consequences of CVDs in the laboratory-based model22. In these studies, the limit of agreement was not examined at the individual level. However, the agreement between the two models in each individual was assessed using Bland Altman plots in the present study. The results of a longitudinal study in Iran demonstrated that the Framingham risk of the non-laboratory-based model was comparable to the laboratory-based model. However, that study used the Cox proportional hazards model to determine the risk23, while the present study used the points-based risk scoring system. The results of another study in Iran showed that the agreement between the CVDs risk groupings was negligible with different risk prediction models16.

The results of the present study showed that the non-laboratory-based risk categories tended to show a higher risk compared to the laboratory-based risk model. Accordingly, in the non-laboratory-based model, the number of males and females in the high-risk group were slightly more than twice as the laboratory-based model. Probably, the method of measuring BMI could underestimate or overestimate the risk of CVDs, which might increase or decrease the laboratory-based and non-laboratory-based risk scores14. Even if we assume that there was a BMI measurement error, this error had randomly occurred in the whole research population. However, the Pars cohort study used accurate measurement tools, which could guarantee the accuracy of the data. Yet, it should be considered that obese people have other risk factors, because obesity has an important role in hypertension, hyperlipidemia, and hyperglycemia and is independently associated with a higher cardiovascular risk24,25,26. Therefore, the non-laboratory-based model showed a higher risk of CVDs in comparison to the laboratory-based model. Similarly, the results of the study by Jones et al. indicated that the BMI-based risk model tended to show higher risks compared to the cholesterol-based risk model. Nonetheless, in that study, the agreement was moderate for all groups, except for males aged 60–74 years, which showed a fair agreement14. In the RODAM study, the agreement between laboratory-based and non-laboratory-based Framingham risk scores was 74.8%, with a Kappa statistic of 0.6327, which was almost similar to the general agreement in the present study. The results of the research by Gray et al. indicated that the non-laboratory algorithm predicted a higher absolute risk in comparison to the laboratory algorithm28. However, due to ethnic, racial, and geographical differences, risk instruments created in one population might not accurately predict the real risk in other populations. Therefore, it seems necessary to validate risk scores in different populations.

The present study results demonstrated that the non-laboratory-based model could not be used instead of the laboratory-based model. However, CVDs risk assessment using the non-laboratory-based model has been shown to have the potential to improve the usefulness of risk scores and CVDs prevention efforts worldwide29,30. Therefore, especially in settings with limited resources where extensive laboratories are not available and it is not economically possible to perform laboratory tests31, the non-laboratory-based model is recommended because it tends to show a higher risk of CVDs and is more conservative compared to the laboratory-based model.

Study strengths and limitations

The present study is the first population-based study to examine the agreement between Framingham laboratory-based and non-laboratory-based CVDs risk scoring methods in a large Iranian population. Due to the large sample size, the findings of this study can be generalized. The accuracy of the data can also be guaranteed because data collection was performed using accurate and reliable tools. However, the present study followed a cross-sectional design based on the baseline data of a cohort study. Thus, a longitudinal study with an adequate follow-up period is required to validate the laboratory-based and non-laboratory-based risk models in the study population. Also, because there is no specific CVD prediction risk tool in Iran. So, we used the Framingham risk score that was developed for the Caucasian population. It seems that modification of the Framingham risk score for the Iranian population is essential.

Conclusion and recommendations

In the general population, the agreement between the two risk scores was moderate according to risk grouping. The non-laboratory-based model is measured without a blood test. Risk factors information of this model can be obtained easily and quickly. Therefore, the non-laboratory-based risk model can be used in resource-limited settings where individuals cannot afford laboratory tests. Future longitudinal cohort studies are suggested to measure the discrimination of these two risk scores in the covered population with an adequate follow-up period. Also, future studies are recommended to modify the Framingham risk score for the Iranian population.