Comparison of heart, grace and TIMI scores to predict major adverse cardiac events from chest pain in a Spanish health care region

Acute non-traumatic chest pain (ANTCP) is the second cause of consultation in the Emergency department (ED). About 70% of all Acute Myocardial Infarctions present as non persistent ST-elevation acute coronary syndrome (NSTE-ACS) in the electrocardiogram. Our aim was to compare whether the HEART risk score is more effective than the GRACE and TIMI scores for the diagnosis and prognosis of Major Adverse Cardiac Events (MACE) at six weeks in patients with ANTCP and NSTE-ACS. A prospective cohort study was conducted with patients with ANTCP that attended an ED and a Primary Care Emergency Center (PCEC) from April 2018 to December 2020. The primary outcome was MACE at six weeks. Diagnostic performance was calculated for each scale as the Area under the Receiver Operating Characteristic (ROC) curve (AUC), sensitivity (SE), specificity (SP), and predictive values (PV). Qualitative variables were compared using the Chi-square test, and continuous variables were compared using the nonparametric Kruskal–Wallis test. We adjusted a logistic regression for risk groups, age, and gender to determine the effect of the HEART, GRACE, and TIMI scores on MACE. The degree of agreement (kappa index) was calculated in the categorical classification of patients according to the three risk scales. Cox proportional hazards regressions were performed for each scale and were compared using partial likelihood ratio tests for non-nested models. From a sample of 317 patients with ANTCP, 14.82% had MACE at six weeks. The AUC was 0.743 (95% CI 0.67–0.81) for the HEART score, 0.717 (95% CI 0.64–0.79) for the TIMI score, and 0.649 (95% CI 0.561–0.738) for the GRACE score. The HEART scale identified low-risk patients with a higher SE and negative PV than the GRACE and TIMI scores. The HEART scale was better than the GRACE and TIMI scores at diagnosing and predicting MACE at six weeks in patients with ANTCP and probable NSTE-ACS. It was also a reliable tool for risk stratification in low-risk patients. Its application is feasible in EDs and PCECs, avoiding the need for complementary tests and their associated costs without compromising patient health.

Acute non-traumatic chest pain (ANTCP) is the second cause of consultation in emergency departments (ED) in industrialized countries.About 70% of patients with ANTCP who develop major adverse cardiac events (MACE) present as a non persistent ST-elevation acute coronary syndrome (NSTE-ACS) in the electrocardiogram (ECG).Managing ANTCP suspected of NSTE-ACS correctly in EDs and primary care emergency centers (PCECs) is essential to avoid inappropriate discharges and costs related to complementary tests and unnecessary admissions [1][2][3] .
On the one hand, inadequate discharges lead to new consultations for ANTCP, and some patients end up developing MACE within six weeks.In a study conducted in the US in 2010 4 , 5-10% of patients were discharged from the ED because of non-coronary ANTCP, and in subsequent visits they were diagnosed with an acute myocardial infarction (AMI).
On the other hand, Owens et al. 4 observed that 20% of the consultations for ANTCP and probable NSTE-ACS result in acute coronary syndrome (ACS).Half of the patients admitted for suspected ACS who underwent diagnostic cardiac tests were ultimately not diagnosed with ACS.These tests amount to 10 billion dollars/year,or 3,000-6,000 dollars/patient,butonly 10% of these indicate a cardiac issue.
In Spain 2 , ACS is one of the main causes of morbidity, mortality, and costs 2 .There are approximately 120,000 cases of ACS per year, 41.67% of which are hospitalized with NSTE-ACS.
Finally, because diagnosing MACE in the context of an NSTE-ACS is complex, it is important to have a rapid and reproducible tool for both high-intermediate-risk patients, who require additional tests, and low-risk patients, who can be discharged quickly and safely.
In an attempt to improve theaccuracy in the diagnosis of MACE in patients with probable NSTE-ACS, the European Society of Cardiology (ESC) 3 and the National Institute for Health and Care Excellence (NICE) 5 recommend the use of the Global Registry of Acute Coronary Events (GRACE 2.0) 6 and the thrombolysis in myocardial infarction (TIMI) risk scores 8 .However, according to validation studies, these recommendations are more useful in patients with a high probability of presenting ACS 10,11,14 .
To help identify low-risk patients, Six AJ et al. (2008) 7 developed the medical history, ECG, age, risk factors, and troponin (HEART) score,a risk stratification tool for MACE in the context of NSTE-ACS.In 2013 Backus et al. 9 externally validated the HEART score in a prospective multicenter study, in which the c-statistic of the

Variables
Data on HEART risk score variables, including medical history, ECG, age, risk factors, and troponin, were collected (Table 1).The clinical history of chest pain with suspicion of ACS followed the ESC 3 and NICE 5 guidelines.The TnI test used in the HUAV was scored as follows: 0 points = normal, limit < 0.01 ng/mL; 1 point = 1-3 times the normal limit, 0.01-0.03ng/mL; 2 points = more than 3 times the normal limit, ≥ 0.04 ng/mL.The cTnT test used in the PCEC was scored as follows: 0 points ≤ 40 ng/L; 1 point ≥ 40 and < 120 ng/L; and 2 points ≥ 120 ng/L.Both troponins are the upper limit of the reference range and correspond to the 99th percentile of the population studied by our method, without correction for hemolysis 3,10,17,24 .According to these items, patients were stratified into three risk levels: low (≤ 3 points), moderate [4][5][6] points, and high (≥ 7 points).
The variables for the GRACE 2.0 (age, heart rate, blood pressure, creatinine, Killip class, cardiorespiratory arrest on admission, elevated cardiac enzymes, ST segment elevation) and TIMI (age > 65 years, cardiovascular risk factors, coronary stenosis > 50%, two episodes of Angina in < 24 h, taking acetylsalicylic acid in the last 7 days, elevated creatine kinase isoenzyme MB (CK-MB) or troponin) scales were collected (Table 1).For both scales, for the "elevation of markers of myocardial damage, " we used the values TnI > 0.01 ng/mL and cTnT > 40 ng/L, respectively 3,10,11,17,24 18 , cardiac tests performed, blood count and coagulation used in the diagnostic process, type of domicile (rural or urban), level of education (higher degree/bachelor's degree, intermediate degree/technician/auxiliary/operator, and unskilled worker/laborer), employment status (if the patient works or not), family status (if the patient lives alone or with a partner, and/or with children, and/or with parents, and/ or with siblings), number of hours of hospital stay under observation in the ED or PCEC, numberof days of admission/hospital stay for suspected ACS, number of new visits for ANTCP in the following six weeks.
The dependent variable was MACE, which includes death due to any cause (DDTAC); AMI, defined according to the third universal definition of myocardial infarction 17 ; PCI; CABG; and significant coronary stenosis but conservative treatment (SCSBCT) because of other concurrent pathologies that contraindicate any interventional coronary revascularization 9,25,27 .

Data sources
We obtained the data for this study from the computerized registry of the SAPof the HUAV and ECAP, ICS-Lleida.The collected data were entered into the RED Cap (Research Electronic Data Capture) web platform.

Statistical analysis
The diagnostic accuracy and Rand balanced accuracy of the HEART, GRACE, and TIMI scores in predicting MACE were calculated at the initial visit and six weeks later.This endpoint was chosen because the HEART score predicts the risk of MACE at 6 weeks 9 .
Continuous variables are expressed as the mean and 95% confidence interval (CI) and categorical variables as the absolute and relative frequency.Qualitative variables were compared using the Chi-square test, and continuous variables were compared using the nonparametric Kruskal-Wallis test.To determine the effect of the HEART, GRACE, and TIMI scores on MACE, we adjusted a logistic regression for risk groups, age, and gender.The degree of agreement (kappa index) was calculated in the categorical classification of patients according to the HEART, GRACE, and TIMI scores.The AUC was used as a discrimination model and precision index.The Hosmer-Lemeshowc-test was applied to assess model calibration by grouping patients by similar model results.The sensitivity (SE), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), false positive rate (FPR), and false negative rate (FNR) were studied simultaneously.DeLong's test was used for testing two correlated ROC curves.Cox proportional hazards regressions were performed for each scale and were compared using partial likelihood ratio tests for non-nested models.A significance level of 0.05 (α = 0.05) was considered.All statistical tests were performed using R software version R-4.1.1.
During the study, patients that could not be followed up for six weeks, because of lack of data or impossibility of contact, were excluded from the analysis.To minimize this possible bias, a comparative analysis was made between the patients who were followed up and those who were not.When we detected clinically relevant www.nature.com/scientificreports/differences, weights were assigned based on these differences.The weights were developed using the Inverse probability weighting algorithms 24 that have been validated and applied in different observational studies.

Ethics approval and consent to participate
According to Law 14/2007 concerning the Biomedical Research Regulations in Spain, Article 3-m, this study is an «Observational Study»: a study carried out on individuals in which the treatment or intervention to which they may be subjected is not modified nor is any other guideline prescribed that could affect their personal integrity.
The patients received the optimal treatment according to the current guidelines of the HUAV and the CUAP of Lleida, Spain.All methods were performed in accordance with the relevant guidelines and regulations.The research was carried out in accordance with the Declaration of Helsinki, revised in 2013.The study was approved by the local ethics committee of the HUAV (minutes: 6/2017) and by the ComitèÈticd'InvestigacióClínica de l'IDIAP Jordi Gol (code: P17 / 219).All patients signed an informed consent form before participating.

Results
Figure 1 shows the flow of patients in the study.A total of 3112 patients attended the HUAV ED and PCEC for ANTCP.Using consecutive systematic sampling and after applying the inclusion, exclusion, and follow-up loss criteria, we obtained a representative sample of 317 patients.The mean age was 58 years (Table 1), and 67.5% of them were male.
The most frequent MACE were those that required PCI (10.7%;).The most frequent type of MACE in highrisk patients differed according to the risk stratification score: MACE requiring PCI, according to the HEART score (25.6% and DDTAC, according to the GRACE (17.1%) and TIMI (16.7%) scores ( The balanced accuracy in the risk stratification of MACE in the low-and moderate-high risk patients was 0.67 (95% CI: 0.58-0.69)between the HEART and GRACE scores, 0.68 (95% CI: 0.59-0.69)between the HEART and TIMI scores, and 0.67 (95% CI: 0.67-0.77)between the TIMI and GRACE scores.
Regarding the likelihood of MACE occurring, the HEART score performed significantly better than the TIMI and GRACE scores: the risk of MACE in moderate-risk patients was 5 times higher than in low-risk patients (OR 5.32[95% CI: 2.07-15.79]);Table 4).
The kappa index in the categorical classification of patients as low-and moderate-high-risk was 0.32 (95% CI: 0.58-0.69)according to the HEART and GRACE scores, according to the HEART and TIMI scores, and 0.31 (95% CI: 0.67-0.77)according to the GRACE and TIMI scores.

Discussion
In this study, we compared the effectiveness of the HEART, GRACE, and TIMI scales for the diagnosis and prognosis of MACE at six weeks in patients with ANTCP and suspected NSTE-ACS.The HEART scale was superior to TIMI and GRACE, with results similar to previous studies 10,11,21 .Specifically, the HEART scale is more effective in stratifying low-risk patients, with an excellent level of safety.In this group, 2.18% of patients (95% CI 1.2-3.2) had MACE at six weeks, similar to the 2.5% observed by Six et al. 7 , and our own previous observations from a retrospective study 15 .
The HEART score is currently validated for use in some medical emergency services, and it is considered superior to TIMI and GRACE (GRACE 2.0) for determining MACE risk at six weeks, with highSE and a NPV 9,11,26 .Poldervaart et al. 10 observed that the HEART score identified low-risk patients better than the TIMI and GRACE scores, with only a 0.8% incidence of MACE in this group of patients.Stopyra et al. 14    the HEART score with the clinical history of patients and observedthat 97.5% of low-risk patients did not die of AMI within one year, with a 7% reduction in the yearly hospitalization rate.
The superior diagnostic and predictive ability for MACE at six weeks of the HEART score compared to the TIMI and GRACE scores was expected.Indeed, HEART was designed to predict MACE at six weeks in patients with ANTCP 7,9 , while TIMI and GRACE were created to determine the prognosis of patients with unstable angina and NSTE-ACS and are not effective for patients with a medium-low probability of MACE 19 .Therefore, the efficacyof the TIMI and GRACE scores in predicting MACE is questioned and subjected to constant validity studies 14,19,21,26 .
The higher the SE and NPV, the better the clinical scoring system for risk stratification 22 .In our study, the HEART score stratified low-risk patients for MACE at six weeks better than the TIMI and GRACE scores, with the best SE and NPV.Similar results were obtained in previous validation studies, in which the SE ranged from 99% (95% CI 97%-100%) to 99.5% (95% CI 97.1%-99.9%)14,19 .
Kline JA et al. 21evaluated the risk score for MACE at 30 days for chest pain, suggesting arange of0.5%-3.0%for a valid FNR, with an average estimationof 2.0% for low-risk patients.Wamala et al. 19 compared nine coronary risk scores and alsoobtained an FNR of 2% for the HEART score in low risk patients, while Stark et al. 22 found an FNR of 0.09% to 3.2% for low-riskMACE patients according to the HEART scale.We obtained similar results, with an FNR of 0 to 2.13% for low-risk MACE patients according to the HEART scale.Additionally, the OR and the Kapplan Meir we obtained highlight the better risk stratification by the HEART score and are similar to that of Six et al. 7 .
Scales that do not include chest pain would be expected to have better reproducibility and reliability and less interobserver bias y 12 .On the contrary, we observed that chest pain is a relevant component and does not reduce the effectiveness of the HEART score, with each of its variables being statistically significant (p < 0.001).
The kappa index between the HEART, GRACE, and TIMI scores was acceptable in the categorical classification of patients as low-and moderate-high-risk, which reinforces the high degree of reliability of the HEART scale.
The effective risk stratification of ANTCP patients in EDs and PCECs is always a challenge.Recent attempts to further improve the accuracy of predictive models include more specific cardiac markers and the combination of risk scores and other complementary tests that are not present in some hospital EDs and most PCECs.In this sense, the HEART score is easier to calculate compared to other risk scales because its elements are more affordable.From the GRACE score, the Killip class component requires the physician's judgment for the diagnosis of heart failure, generating inter-rater variation.Very few patients undergo additional tests for heart failure, such as ECG and natriuretic peptide level, because they are not available in PCECs and some hospital EDs.In our study, only 1.2% of the patients assisted in the HUAV ED had their natriuretic peptide level measured, and 31.9%underwent an ECG.In the CUAP, none of the patients underwent these tests because the resources were not available.

Figure 2 .
Figure 2. ROC curves for the HEART, TIMI, and GRACE risk scales.

integrated Table 3 .
Performance characteristics of the TIMI, GRACE, and HEART risk scales.CI: confidence interval; PPV: positive predictive value; NPV: negative predictive value; TIMI score: low risk determined by score = < 2, moderate risk determined by score = < 4, and high risk determined by score > = 5; GRACE score: low risk determined by score < = 108, moderate risk determined by score > = 109, and high risk determined by score > = 141; HEART score: low risk determined by score = < 3, moderate risk determined by score > = 4, and high risk determined by score > = 7.

Figure 3 .
Figure 3. Survival analysis: global Kaplan Meier curve.a:major acute cardiac event; b:acute non-traumatic chest pain.

Table 1 .
Variables of the risk scales and MACE at 6 weeks.
a: major adverse cardiovascular events; b: history of ischemic heart disease (M < 55 y.W < 65 y.): Family history of ischemic heart disease in men under 55 years of age and in women under 65 years of age; c: coronary stenosis greater than 50%.Risk factors: cardiovascular risk factors; d: Killip-Kimball classification; e: unspecific alteration of repolarization on the electrocardiogram.

Table 2 .
MACE according to the risk stratification of the HEART, GRACE, and TIMI scales.a: death due to any cause; b: acute myocardial infarction; c: coronary artery bypass grafting; d: percutaneous coronary intervention; e: significant coronary stenosis but conservative treatment; f: major adverse cardiovascular events.

Table 4 .
Comparison of the Odds Ratio-MACE according to the risk stratification of the HEART, GRACE, and TIMI scales.CI: confidence interval.