External assessment of the EUROMACS right-sided heart failure risk score

The EUROMACS Right-Sided Heart Failure Risk Score was developed to predict right ventricular failure (RVF) after left ventricular assist device (LVAD) placement. The predictive ability of the EUROMACS score has not been tested in other cohorts. We performed a single center analysis of a continuous-flow (CF) LVAD cohort (n = 254) where we calculated EUROMACS risk scores and assessed for right ventricular heart failure after LVAD implantation. Thirty-nine percent of patients (100/254) had post-operative RVF, of which 9% (23/254) required prolonged inotropic support and 5% (12/254) required RVAD placement. For patients who developed RVF after LVAD implantation, there was a 45% increase in the hazards of death on LVAD support (HR 1.45, 95% CI 0.98–2.2, p = 0.066). Two variables in the EUROMACS score (Hemoglobin and Right Atrial Pressure to Pulmonary Capillary Wedge Pressure ratio) were not predictive of RVF in our cohort. Overall, the EUROMACS score had poor external discrimination in our cohort with area under the curve of 58% (95% CI 52–66%). Further work is necessary to enhance our ability to predict RVF after LVAD implantation.

Center approved this study. The requirement of informed consent for the study is waived by the institutional review board of the University of Minnesota. All methods were carried out in accordance with relevant guidelines and regulations. At the time of this analysis, the larger continuous-flow LVAD cohort consisted of 451 patients implanted between 2007 and 2017. Of these, 254 patients met the following inclusion criteria 1) first time, continuous-flow LVAD implantation 2) complete pre-operative variables to complete EUORMACS score calculation, and 3) complete post-operative data to determine date of inotrope wean, pulmonary vasodilator wean and RVAD use.
The following demographic and clinical covariate data are available in the University of Minnesota LVAD database, which is updated through data extraction and manual chart review: age, gender, body mass index (BMI), serum creatinine, albumin, Interagency Registry for Mechanical Circulatory Support (INTERMACS) profile, pre-operative hemodynamics, bridge to transplant status, cardiomyopathy type, presence of diabetes, and NT pro b-type natriuretic peptide (NT-proBNP). Vital status was obtained from chart review. The date of the last clinic visit was recorded for patients who were still alive at the end of follow up. For cardiac transplantation, the date of cardiac transplant was obtained from the electronic medical record and confirmed with an operative report.
Primary predictor and outcome. The primary predictor for this analysis was follow up EUROMACS RV score, which was calculated by totaling points for the following pre-LVAD clinical variables. RA/PCWP > 0.54: 2 points, hemoglobin ≤ 10 g/dL:1 point, multiple inotropes: 2.5 points, INTERMACS 1-3: 2 points, and severe RV dysfunction: 2 points. The primary outcome was early severe post-operative right ventricular failure defined as need for short/long-term mechanical right-sided circulatory support within 30 days of LVAD implantation, continuous inotropic support greater than or equal to 14 days, or need for pulmonary vasodilators for greater than 48 h. Statistical methods. All statistical analyses were performed using STATA 16 (College Station, Texas). A p value of < 0.05 was considered statistically significant. Baseline characteristics between cohort subjects who developed early RVF and those who did not develop early RVF were compared with normally distributed continuous variables were compared with t-tests and Wilcoxon rank sum tests for non-normally distributed continuous variables. Categorical variables were compared with Pearson chi-squared tests or Fishers exact test, where appropriate. In order to determine the relationship between baseline co-variates and follow up pulmonary capillary wedge pressure, multivariable linear regression analyses were performed.
In order to assess EUROMACS model discrimination, a receiver operator curve was generated with EUROMACS score as the predictor and post LVAD RVF as the outcome. This was repeated using the RVF definition of RVAD or prolonged inotropes use only. The EUROMACS score was also assessed as a predictor of RVF using logistic regression. In order to determine the relationship between other pre-operative variables and RVF, logistic regression was performed. The relationship between RVF (or RVAD use alone) and subsequent LVAD mortality was assessed with cox regression.

Results
The mean age of the cohort was 59 years of age. Eighty two percent were male, 80% were Caucasian, and 51% were designated as bridge to transplantation. The majority of patients (52%) were INTERMACS profile 2-3. A total of 100 patients (39%) were diagnosed with early right ventricular failure and 154 patients (61%) did not have post-operative right heart failure. There were no significant differences in age, sex, race, baseline atrial fibrillation, cardiomyopathy type, diabetes, chronic obstructive pulmonary disease, LVAD surgical strategy, INTERMACS profile, pulmonary artery systolic pressure or cardiac index between both groups ( Table 1). The 60 day mortality of the cohort was 10.6% (27/254). Of those, 15/27 (56%) were deaths attributable to right heart failure.
Outcomes of right heart failure. Of the 100 patients who met RV failure criteria, 12 patients required RVAD and 23 patients required inotropes > 14 days. Patients who required pulmonary vasodilators > 48 h comprised the vast majority (n = 65) of patients who were defined as having post-operative RVF. Early right-sided heart failure was associated with increased mortality (unadjusted HR 1.45; 95% CI 0.98-2.16, p = 0.066). Use of RVAD after LVAD surgery was the primary driver of increased mortality (HR 4.56; 95% CI 2.28-9.11, p < 0.001, Fig. 1).
Risk factor for early postoperative right heart failure. We tested 66 distinct risk factors for early post-operative right heart failure. An additional nine factors were calculated from these 66 variables (Tables 2,  3, 4). Of the individual variables tested, a higher pulmonary arterial pressure was protective against RV failure as was a higher pulmonary arterial pulse pressure. Higher creatinine, lower albumin, and higher total bilirubin were all associated with increased risk of RV failure after LVAD. Pre-operative ventilator support, multiple inotropes, extra corporal membrane oxygenator support, and non-elective IABP were also associated with higher risk of RVF. Of note, hemodynamic parameters such as right atrial pressure, right atrial/pulmonary capillary wedge pressure, pulmonary artery pulsatility index, pulmonary artery compliance, and pulmonary artery elastance were not associated with increased risk of right sided heart failure in this dataset.  (Fig. 2). The performance of the EUROMACS score to predict RVF as defined by RVAD use or prolonged inotrope use was 67% (95% CI 54-79).

Discussion
In this external validation of the EUROMACs score to predict right ventricular failure after CF-LVAD, we found we found that the EUROMACS score had relatively poor discrimination in predicting RV failure. Right ventricular failure was significantly associated with mortality after CF-LVAD in this dataset, which was mainly driven by the need for RVAD and/or prolonged inotropes. Many of the variables used in the EUROMACs score were not associated with RV failure in our cohort including hemoglobin and RA/PCWP ratio. www.nature.com/scientificreports/ www.nature.com/scientificreports/ Right heart failure is one of the most common causes of early morbidity and mortality after CF-LVAD implantation. Predicting RV failure is important as planned institution of RVAD during LVAD surgery has been associated with improved outcomes compared to delayed RVAD implantation 11,12 , and a direct to transplant strategy can be employed in eligible patients to avoid this severe complication. The present study again demonstrates the difficulty in predicting RV failure after LVAD, even with the most contemporary risk scores. Our results mirror external validations of the EUROMACS right sided heart failure risk score where ROCs were in the 0.65 range [8][9][10] .
There are many reasons why our results may have differed from the original EUROMACs analysis. In our cohort, the incidence of the RV failure occurred in 39% patients undergoing CF-LVAD implantation. The majority of the patients (65%) met this definition due to prolonged pulmonary vasodilator use, while the EUROMACs data only had 1% of patients with prolonged pulmonary vasodilator use. RV failure defined by prolonged vasodilator use was not associated with increased mortality in our cohort, which questions the clinical significance of this part of the RV failure definition. The use of pulmonary vasodilators and time course of weaning these medications appears to be different between the University of Minnesota and EUROMACS derivation cohorts, which reflects significant variability in practice. When restricting the RVF definition to inotropes ≥ 14 days and/or need for RVAD, the performance of the EUROMACS score improved. Another difference between the cohorts was the lower numbers of destination therapy CF-LVAD implantations in the EUROMACs cohort (14% vs. 49%). The larger number of patients with comorbid conditions in the University of Minnesota dataset may have explained some of the lower performance observed in the model, as these populations have real differences.
There have been several previous studies designed to predict those patients at high likelihood of developing RV failure following LVAD implantation. Many of these include hemodynamic variables such as right atrial pressure, pulmonary artery pulsatility index, and right atrial pressure: pulmonary capillary wedge pressure ratio 13,14 . Other studies have shown that certain echocardiographic features of RV dysfunction such as TAPSE and semi-quantitative RV function are predictive of RV failure after LVAD 15,16 . In our cohort, none of the previously mentioned hemodynamic variables were predictive of RV failure. Similar to previous analyses, higher creatinine, lower albumin, and higher total bilirubin were all associated with increased risk of RV failure after LVAD. Pre-operative ventilator support, multiple inotropes, extra corporal membrane oxygenation, and nonelective IABP were also associated with higher risk of RVF in this cohort. All of the associated variables are direct or indirect markers of patient acuity, and may suggest longer standing heart failure is a risk factor for post-operative RV failure.
There are many reasons why published risk scores may perform poorly in validation cohorts. First, RV failure after LVAD does not have a universal definition 17 . For example, the widely used INTERMACS criteria for RV failure does not include pulmonary vasodilators as definition for RVF. With regard to the hemodynamic variables, these can change dramatically over a 24 h period as patient are managed with diuretics, inotropes and temporary support. Lastly, insults that occur to the RV in the operating room, such as RV ischemia from hypotension, prolonged cardiopulmonary bypass time, surgical positioning of the inflow cannula, fluid resuscitation and blood transfusions are not predictable ahead of time 18 . These intraoperative conditions can "unmask" underlying RV failure that might not have been predicted with traditional risk scores using clinical, hemodynamic, and echocardiographic data.
This study has several limitations. First, this was a single-center observational analysis. Second, our sample size is smaller than the EUROMACs cohort, which could limit the power of our study to determine variables that are associated with severe RV failure.

Conclusion
The EUROMACS Right-Sided Heart Failure Risk Score had poor external discrimination on external validation. In the present cohort, variables associated with long-standing heart failure (creatinine, bilirubin, albumin) were more predictive of right ventricular heart failure. As RV failure post LVAD is a complex syndrome influenced by pre-operative, intra-operative, and post-operative factors, it remains difficult to predict. Further work will be required to enhance our understanding of the post-LVAD right ventricular failure syndrome.