A scoring system to predict the occurrence of very late stent thrombosis following percutaneous coronary intervention for acute coronary syndrome

We aimed to derive and validate an effective risk score to identify high-risk patients of very late stent thrombosis (VLST), following percutaneous coronary intervention (PCI) for acute coronary syndrome (ACS). Stepwise multivariable Cox regression was used to build the risk model using data from 5,185 consecutive ACS patients treated with PCI (derivation cohort) and 2,058 patients from the external validation cohort. Eight variables were independently associated with the development of VLST: history of diabetes mellitus, previous PCI, acute myocardial infarction as admitting diagnosis, estimated glomerular filtration rate <90 ml/min/1.73 m2, three-vessel disease, number of stents per lesion, sirolimus-eluting stent, and no post-dilation. Based on the derived score, patients were classified into low- (≤7), intermediate- (8–9), and high- (≥10) risk categories. Observed VLST rates were 0.5%, 2.2%, and 8.7% and 0.45%, 2.3%, and 9.3% across the 3 risk categories in the derivation and validation cohorts, respectively. High discrimination (c-statistic = 0.80 and 0.82 in the derivation and validation cohorts, respectively) and excellent calibration were observed in both cohorts. VLST risk score, a readily useable and efficient tool to identify high-risk patients of VLST after PCI for ACS, may aid in risk-stratification and pre-emptive decision-making.

In the present study, we aimed to develop a risk score for VLST based on clinical variables from a large cohort of consecutive patients with ACS and independently validate the system in another cohort with the aim of identifying patients at high risk of VLST.

Results patient characteristics.
A total of 6,038 patients were eligible for the derivation cohort; of which, 236 were excluded owing to death prior to discharge, 413 subjects refused to participate in follow-up interviews, and 204 were lost to follow-up. Finally, 5,185 patients were enrolled (average age 59.85 ± 9.95 years, 68.60% male). The median length of follow-up was 1,502 days.
For the validation cohort, of the 2,438 patients identified as eligible, we excluded 88 patients who died prior to discharge, 186 who did not agree to participate in follow-up interviews, and 106 who were lost to follow-up. Finally, 2,058 patients were enrolled (average age 60.00 ± 10.22 years, 68.71% male). The median length of follow-up was 1,422 days. Supplementary Table S1 provides a comparison of the baseline and procedural characteristics as well as laboratory data between patients within the two cohorts.
The median time from index PCI to the occurrence of VLST was 697 and 803 days for the derivation and validation cohorts, respectively. The incidence of VLST was not significantly different between the derivation and validation cohorts at 1.77% and 1.70%, respectively (P = 0.829). Table 1 provides a comparison of the baseline and procedural characteristics as well as laboratory data between patients with and without VLST in the derivation cohort. A significantly larger number of patients were admitted with an acute MI among those with VLST (P < 0.001), and the peak troponin I level was higher among these patients; although, the difference was not statistically significant (P = 0.051). Patients with VLST experienced significantly more Thrombolysis in Myocardial Infarction (TIMI) flow grade 0 before PCI (P = 0.017), stent overlap was significantly higher (P = 0.013), and stent length significantly lon ger (P = 0.001) during PCI than that among subjects without VLST. There was no difference between the occurrence of slow flow (P = 0.619) and no reflow (P = 0.547) after PCI between patients with VLST and those without. More patients without VLST received dual-antiplatelet therapy (DAPT) during follow-up compar ed to those that received DAPT with VLST; although, this was not significantly different (44.37% vs. 38.04%, respectively, P = 0.226).
Development of the very late stent thrombosis risk score. In total, 12 predictors were identified for the development of VLST from the univariable analysis in the derivation cohort, and 4 independent risk factors were identified in the literature (Supplementary Table S2). Eight predictors were independently associated with the occurrence of VLST by multivariable Cox regression analysis as follows: history of diabetes mellitus (DM), previous PCI, acute MI as the admitting diagnosis, estimated glomerular filtration rate (eGFR) of <90 ml/ min/1.73 m 2 , three-vessel disease, number of stents per lesion, sirolimus-eluting stent (SES), and no post-dilation. The variance-inflation factor was <2, indicating the absence of multicollinearity among variables in the model. The p-value from the Hosmer-Lemeshow goodness-of-fit test of the final model was 0.082 for the derivation cohort. The c-statistic of the final model was 0.81 (95% confidence interval [CI]: 0.76-0.86), and cross-validation predicted a slight decrease in discriminative ability (c-statistic 0.75, 95% CI: 0.70-0.80).
The points assigned to each variable of the risk-scoring system are shown in Table 2. The relationship between score value and the predicted incidence of VLST is illustrated in Supplementary Fig. S1 (P = 0.345 from the Hosmer-Lemeshow test of the score). Supplementary Fig. S2 presents the calibration plot of predicted vs. observed incidence of VLST development among the derivation cohort across deciles of the risk score, which shows excellent calibration. The c-statistic and hazard ratio (HR) of the score was 0.80 (95% CI: 0.75-0.85) and 1.69 (95% CI: 1.54-1.85), respectively, in the derivation cohort. Levels of risk were defined according to the predicted incidence of VLST: low risk = score of ≤7, corresponding to a predicted incidence of <1% (3,135 patients, 60.5% of the derivation cohort); intermediate risk = 8 or 9 points, corresponding to a predicted incidence of 1%-3% (1,837 patients, 35.4% of the derivation cohort); high-risk was ≥10 points, corresponding to a predicted incidence of >3% (473 patients, 9.1% of the derivation cohort). The observed incidences of VLST according to these cut-offs were 0.5%, 2.2%, and 8.7%, respectively, in the derivation cohort (Fig. 1).
Risk-score validation. The c-statistic of the final model was 0.83 (95% CI: 0.78-0.89) in the validation cohort, with a p-value of 0.66 from the Hosmer-Lemeshow test. The c-statistic of the VLST risk score was 0.82 (95% CI: 0.76-0.90), with a p-value of 0.97 from the Hosmer-Lemeshow test. The HR of the risk score was 1.74 for the validation cohort (95% CI: 1.53-1.99). The predicted and observed incidences of VLST across deciles of the risk score in the validation cohort are illustrated in Supplementary Fig. S2. According to predicted risk, 64.67% (n = 1,331), 26.97% (n = 555), and 8.36% (n = 172) of the validation cohort were identified as low, intermediate, and high risk, respectively. The observed incidences of VLST in each risk group were 0.45%, 2.34%, and 9.3%, respectively, in the validation cohort (Fig. 1). The risk scores and the occurrence of VLST in the derivation and validation cohorts are depicted in Supplementary Fig. S3.
The prediction model was further evaluated by decision-curve analysis (Fig. 2). The results demonstrated that, for relevant decision thresholds, the VLST risk-score model provided a substantial net clinical benefit compared with that of the LST risk-score model, which included only three variables (current smoking, history of PCI, and ST-elevated MI [STEMI] as the admitting diagnosis). For a decision threshold of 2% of VLST risk, compared with the LS risk score, the VLST risk score should identify 4 additional cases, without any false-positive results, from a population of 1,000 patients with a 1.7% incidence of VLST.

Discussion
In this study, development of VLST was independently associated with the following eight variables: history of DM, previous PCI, acute myocardial infarction as admitting diagnosis, estimated glomerular filtration rate (eGFR)<90 ml/min/1.73 m 2 , three-vessel disease, number of stents per lesion, SES, and no post-dilation. The observed VLST rates were 0.5%, 2.2%, and 8.7% in the derivation cohort and 0.45%, 2.3%, and 9.3% in the validation cohort across the low-, intermediate-, and high-risk categories, respectively. High discrimination (c-statistic of 0.80 and 0.82 in the derivation and validation cohorts, respectively) and excellent calibration were noted in both cohorts. To the best of our knowledge, this is the first study to provide a simple but useable scoring system for identifying impending VLST after PCI from a large cohort of consecutive patients with ACS. The VLST risk score ( Fig. 3) showed good predictive ability and calibration across the two cohorts. Being based on simple  www.nature.com/scientificreports www.nature.com/scientificreports/ categorical variables, the score could be used to identify high-risk patients, with particular utility in the decision of treatment plans for patients more than 1 year after PCI.
Patient-and lesion-related variables included a history of DM, previous PCI, acute MI as the admitting diagnosis, eGFR <90 ml/min/1.73 m 2 , and three-vessel disease. Previous studies have suggested that DM plays an important role in promoting atherosclerosis, and in-stent neoatherosclerosis has been found to be an important underlying mechanism of VLST 15,16 . Additionally, a history of DM has been described as a powerful predictor of subacute, late, and very late stent thrombosis 17 . Previous PCI might indicate prior stent placement and has been included as a variable in the LST risk score 3 . Previous studies have reported higher rates of stent thrombosis (ST) among patients with a history of MI, which might be related to the different plaque characteristics underlying the MI and angina pectoris [18][19][20] . Chronic kidney disease could promote the progression of arteriosclerosis and has been found to be an independent risk factor for ST 21,22 . The three-vessel disease is an indicator of the severity and diffusion of lesions and is correlated with the development of ST 6 .
Regarding the procedure-and stent-related variables, a greater number of stents indicates longer lesions and stents; thus, more stent overlap, all of which could contribute to the development of ST 6 . Furthermore, no post-dilation was reported as an independent risk factor for VLST in the case of bioresorbable stents 23 , and might lead to stent malposition, which is an important mechanism underlying VLST development 7,24,25 . The representativeantiproliferation-drug-eluting stents of the first-generation drug-eluting stents (DESs) are SESs; local toxicity and inflammatory reactions after implantation of these stents could lead to delayed endothelialisation or even positive remodelling of the vessel walls. This is also an important mechanism involved in the development of VLST 16,26-30 .

Figure 2.
Decision-curve analysis. Net benefit of using a model to predict VLST compared with strategies of 'assuming all' or 'assuming none' patients would be at high-risk for different decision thresholds is shown. The VLST risk score (blue) shows improved benefit compared with the LST risk score (red). LST, late stent thrombosis; VLST, very late stent thrombosis. www.nature.com/scientificreports www.nature.com/scientificreports/ In clinical practice, VLST often occurs more than 1 year after the initial stent placement; therefore, identifying patients at high risk for VLST is extremely important as it may enable the implementation of intensive follow-up and preventative interventions. The LST risk score was established in a cohort of selected patients with ACS and has proven to be equally useful in predicting the occurrence of VLST. The system includes three variables as follows: current smoker, history of previous PCI, and STEMI as the admission diagnosis, but the predictive ability of the score is limited (c-statistic = 0.66) 3 . A recent study 31 presents a new risk score developed from a small sample case-control study; however, no validation data were reported for this system, and some variables, such as suboptimal stent result, that were included in the score cannot be assessed objectively. Compared with the above study 3,31 , the strength of our study is the inclusion of a cohort of consecutive patients with ACS, which can be expected to reflect real-world clinical practice, and improved discrimination was shown in both the derivation and validation cohorts.
The VLST risk score provides a simple but efficient tool to identify patients at high risk for VLST after PCI. Intensive follow-up should be carried out for these patients, and more aggressive therapy might be required. The independent risk factors included in the present risk model will be significant in guiding the choice of therapeutic approach. For example, selecting fewer stents, using new-generation DESs, and performing effective post-dilation could reduce the incidence of ST. However, the fact that the VLST risk score presented here was developed and validated in patients with ACS means that its value in patients with stable angina pectoris still needs to be proven.
Part of the inclusion criteria for this study was definite ST (assessed by coronary angiography [CAG]) to ensure consistency of events. This might have resulted in some events being missed, though we tried to reduce this situation. In addition, although a single-centre cohort was used for validation, the reliability of the score warrants validation using multi-centre, large-scale prospective cohorts.
The VLST risk score presented here provides a readily useable and efficient tool to identify patients with ACS who are at high risk of developing VLST after PCI. The system may enable further risk stratification and thus, facilitate pre-emptive clinical decision-making.

Methods
Study population. This study is registered at ClinicalTrials.gov: NCT03491891. We recruited consecutive patients ≥18 years who were admitted to The First Hospital of Jilin University between January 1, 2014 and June 1, 2015 and underwent PCI for treatment of ACS and who had signed informed consent. The exclusion criteria were as follows: expected survival time of less than 1 year, severe organ failure, malignancy, a surgical plan for important organs within 1 year after PCI and antiplatelet drugs should be discontinued; death prior to discharge; and refusal to sign informed consent or take part in follow-up interviews or patients who were lost to follow-up after discharge.
For the external validation cohort, we recruited all consecutive ACS patients who were admitted to the first affiliated hospital of Shantou University for PCI between January 1, 2014 and June 1, 2015, and the same exclusion criteria were applied.
All patients enrolled were treated with newer or second-generation DESs (Supplementary Table S3) and DAPT, lasting for at least one year. Antiplatelet therapy continued beyond this time was based on the decision of patients' personal doctors considering their specific conditions (Supplementary Table S4).
In order to improve the diagnosis of VLST, we set up a 24-hour online consultation to ensure the patients enrolled would receive professional medical evaluation and medical advice immediately when they have cardiac symptoms. For highly suspicious patients, they will be immediately transferred to the nearby Chest Pain Center, and CAG images of confirmed patients will be uploaded to the core laboratory for analysis. ethical information. Ethics committee approval (NO. 2013-256) was obtained from the appropriate committee of both institutes (Medical Ethics Committee of The First Hospital of Jilin University, First Affiliated Hospital of Shantou University Medical College Ethics Committee), and the study was performed in accordance with the ethical standards laid out in the 1964 Declaration of Helsinki and its later amendments. All patients provided written informed consent.

Definitions and endpoints. We defined VLST according to the 2007 criteria of the Academic Research
Consortium (ARC) for ST; specifically, ST that occurred more than one year after coronary stent implantation and was confirmed by CAG. All documented cases of VLST were determined by CAG, and the procedures were independently reviewed by two experienced interventional cardiologists. If a consensus could not be reached between the two reviewers, a third reviewer was consulted. The primary endpoint of this study was definite ST.
Statistical analysis. Categorical data are presented as numbers (percentage). Continuous data are presented as mean ± standard deviation. Categorical data were compared using the chi-square test or Fisher's exact test, while quantitative data were compared using the Mann-Whitney U test. All statistical tests were 2-sided with a 0.05 significance level.
The derivation cohort was used to identify predictors for the occurrence of VLST and to establish a risk-scoring system, which was verified using the validation cohort. Significant variables (P <0.05) in univariable Cox regression analysis and independent risk factors reported in previous literature were included in the multivariate Cox regression analysis and the optimal predictive variables of VLST were identified using the backward stepwise method. Some variables were excluded, such as leukocyte count (for which there was insufficient evidence to definitively conclude that the increased count was not owing to infection or other factors). Procedural variables, such as stent diameter, length, type, and post-dilation, were based on the operator's subjective assessment during PCI.