This study aimed to explore the validity of the use of the net clinical benefit (NCB), i.e. the sum of major bleeding and thrombotic events, as a potential surrogate for all-cause mortality in clinical trials assessing antithrombotics. Published randomized controlled trials testing anticoagulants in the prevention or treatment of venous thromboembolism (VTE) and non-valvular atrial fibrillation (NVAF) were systematically reviewed. The validity of NCB as a surrogate endpoint was estimated by calculating the strength of correlation of determination (R2) and its 95% confidence interval (CI) between the relative risks of NCB and all-cause mortality. Amongst the 125 trials retrieved, the highest R2trial values were estimated for NVAF (R2trial = 0.41, 95% CI [0.03; 0.48]), and acute VTE (R2trial = 0.30, 95% CI [0.04; 0.84]). Conversely, the NCB did not correlate with all-cause mortality in prevention studies with medical (R2trial = 0.12, 95% CI [0.00; 0.36]), surgical (R2trial = 0.05, 95% CI [0.00; 0.23]), and cancer patients (R2trial = 0.006, 95% CI [0.00; 1.00]). A weak correlation between NCB and all cause-mortality was found in NVAF and acute VTE, whereas no correlation was observed in clinical situations where the mortality rate was low. Consequently, NCB should not be considered a surrogate outcome for all cause-mortality in anticoagulation trials.
Non-valvular atrial fibrillation (NVAF) and venous thromboembolism (VTE), which includes deep vein thrombosis (DVT) and pulmonary embolism (PE), are two common diseases associated with significant morbidity and mortality1. Furthermore, patients hospitalized for medical illness, undergoing surgery, and cancer patients have a higher risk of venous thromboembolism or bleeding events than the general population2,3,4. As they reduce the risk of death and injury related to thrombotic events, anticoagulants are the cornerstone of the management of these cardiovascular diseases5.
Since their discovery, the use of vitamin K antagonists (VKA) has been shown to reduce the mortality related to cardiovascular diseases6. However, due to their narrow therapeutic index, bleeding events are the most important complications related to the use of VKA, and the incidence of major bleeding ranges from 1.4 to 3.4% in NVAF patients7,8 and from 0 to 4.3% in VTE patients9. More recently, direct oral anticoagulants (DOAC) have been shown to be non-inferior to VKA in preventing thrombotic events while being associated with a reduction of major bleedings, and thus became the standard therapy and preventive treatment for VTE and NVAF10,11.
The positive impact of anticoagulants on cardiovascular mortality is presumably a result of the reduced amount of thrombotic events. However, major bleeding is also associated with significant morbidity and mortality12,13. To take into account the balance between potential benefits (i.e. reduced risk of thromboembolism) and harm (i.e. increased risk of major bleeding) in randomized trials, the concept of net clinical benefit (NCB), which is the sum of major bleeding and thrombotic events, appeared recently as a potentially relevant outcome in phase III clinical trials evaluating antithrombotics14. These measurable events lie within the pathophysiological spectrum of NVAF and VTE, allowing the summarization of treatment effects and increasing the number of total events, thus increasing the study power. Technically, to evaluate the value of NCB as a surrogate endpoint in clinical trials, a linear correlation between the treatment effects on the surrogate and on the final outcome needs to be established for each indication separately, and its strength has to be checked in advance with several statistics analytic methods15. However, direct evidence of the association between the treatment effects of anticoagulation on the NCB and cardiovascular mortality in the setting of clinical trials is lacking.
The present study aimed to explore the validity of the NCB as a potential surrogate for all-cause mortality in trials testing antithrombotics for the prevention and treatment of VTE and NVAF.
Materials and methods
The present study was conducted according to the PRISMA statement16.
Search strategy and study identification
First, all published randomized controlled trials registered in the META-EMBOL database (Silvy Laporte, University of Saint-Etienne, PHRC 2008) were investigated. This database collected the results of trials assessing the efficacy of antithrombotics in the prevention or treatment of VTE and NVAF17.
Additional studies were searched for on electronic databases such as MEDLINE, the Cochrane Library databases, and EMBASE from 1990 to December 2020, in English and non-English language by using sensitive keywords to detect all the studies (see online supplement).
Hand searching through medical journals, reviews, and bibliography of each selected article was carried out to identify additional studies that were not reported in those electronic databases and META-EMBOL.
The database was screened by two authors (R.K and R.A) independently to identify studies that potentially met the eligibility criteria. These were: randomized controlled trials, parallel groups, open or double-blind design evaluating antithrombotics compared to placebo or control treatment for (1) VTE, (2) thromboprophylaxis in hospitalized patients for medical conditions, major orthopedic, and/or abdominal surgery, (3) thromboprophylaxis in cancer patients, and/or (4) NVAF. Also, for inclusion studies needed to report the three outcomes of interest: thrombosis, major bleeding, and all-cause mortality. Disagreements about inclusion were resolved by consensus or by consulting a third author (J.-C.L.).
Definition of outcomes
The net clinical benefit (NCB) was computed in each arm by adding major bleeding events and thrombosis events retrospectively in each study. The major bleeding in non‐surgical patients was defined according to the International Society on Thrombosis and Haemostasis (ISTH) criteria18, i.e. fatal bleeding, bleeding manifest in a critical organ such as intracranial bleeding, and/or explicit bleeding correlated with a decrease of hemoglobin level of 20 g/L or more, or necessitating a transfusion of at least two units of red cells or whole blood. Major bleeding for surgical patients was defined according to the European Agency for the Evaluation of Medical Products definition, i.e. bleeding detected at the surgical site and conducting to re-operation or any special medical intervention, in addition to the criteria mentioned above.
The effects of the assessed treatments on thrombotic events were evaluated from the main pre-specified efficacy outcome of each trial. All-cause mortality was used irrespective of its relationship with the cardiovascular event.
When a trial met the eligibility criteria, two authors separately extracted the following data in addition to thrombotic and bleeding events: name of the first author and year(s) of publication, study acronym, study design, disease, treatment regimens, class of comparison, and study size.
For each condition (acute treatment of VTE, treatment of NVAF, and VTE prevention in medical, surgical, and cancer patients), a meta-analysis was conducted and forest plots were generated and computed using the random-effects model to estimate the relative risk (RR) for all-cause mortality (standard outcome) and NCB (surrogate outcome), as well as the corresponding 95% confidence intervals (CI) in patients treated (in the experimental anticoagulation group) compared to patients in control groups. Additional meta-analysis and forest plots were generated for each class of medicine separately (Antiplatelet, VKA, DOAC, and LMWH) and calculated the RR for all-cause mortality and NCB. Adjusted continuity corrections of 0.5 were used for any study with no event19.
To validate NCB as a surrogate of all-cause mortality, the method reported by Buyse et al.20 was used. A linear regression model was therefore used to assess the association between the RR for NCB and the RR for all-cause mortality by calculating the coefficient of determination (R2trial). The percentile bootstrapping method (resampling 1000 times) was applied in order to obtain a high accuracy to compute the 95% confidence interval for R2trial in addition to the prediction interval. The validity of the surrogate endpoints depends on the strength of the association with the ultimate endpoint. Indeed, the coefficient of determination R2trial should be more than 0.65 and close to 121. In practice, an R2trial with an upper limit of the 95% confidence interval (95% CI) ≤ 0.7 (i.e. limited correlation) confirms the lack of validity of the surrogate endpoint, whereas an R2trial with a lower limit of the 95% confidence interval ≥ 0.85 supports the validity of the surrogate. In case of intermediate correlation (0.7 < Rtrial < 0.85), the validation of the surrogate endpoints remains unclear15.
The surrogate threshold effect (STE) can also be assessed to estimate the minimal treatment effect on the surrogate endpoint predicting a significantly nonzero effect on the true endpoint22,23. To compute the STE, the linear regression model was calculated and the 95% prediction intervals were plotted. The value of the STE is the value on the x-axis (log RR of NCB) for which the lower limit of the prediction interval meets a point corresponding to 1 (zero effect on the true endpoint) on the y-axis (log RR of all-cause mortality).
Furthermore, two sensitivity analyses were performed to assess the robustness of the results. The first analysis included only studies with a double-blind design, whereas the second one was applied to studies that included only the new direct oral anticoagulants (except ximelagatran) in the experimental arm.
The association between the mortality rate and the relative risk reduction was explored using linear regression for all indications together.
Search results and characteristics of studies
The primary META-EMBOL database and the additional literature search identified 264 trials for review. Among them, there were 25 duplicated studies, 41 studies were excluded as they did not meet the year of publication criterion, and 73 articles were excluded after full-text review. Finally, a total of 125 eligible studies were included for data extraction (Fig. 1).
Among those included studies, 27 studies were conducted in the field of NVAF (114,689 patients), 27 focused on acute treatment of VTE (55,216 patients), 25 on thromboprophylaxis in patients hospitalized for medical conditions (69,022 patients), and 38 on major orthopedic and abdominal surgery (70,982 patients). Additionally, 8 studies (6372 patients) assessed thromboprophylaxis in cancer patients. A total of 79 studies were double-blind randomized controlled trials, 28 were open-label with blind adjudication, and 18 had an open-label design.
Treatments assessed in experimental arms were apixaban (k = 8), acenocoumarol (k = 2), betrixaban (k = 2), dabigatran (k = 8), rivaroxaban (k = 12), dalteparin (k = 4), edoxaban (k = 2), AZD0837 (k = 1), idrabiotaparinux (k = 2), fondaparinux (k = 10), aspirin (k = 5), dipyridamole (k = 2), nadroparin (k = 11), ximelagatran (k = 4), enoxaparin (k = 19), reviparin (k = 2), bemiparin (k = 1), semuloparin (k = 6), indobufen (k = 1), UFH (k = 2), tinzaparin (k = 1), other LMWH (k = 2), certoparin (k = 5), gemcitabine + dalteparin (k = 1), fraxiparin (k = 1), ardeparin (k = 2), and warfarin with/without aspirin (k = 9; Table 1).
Surrogacy evaluation by clinical indication
Summary of the meta-analysis and forest plot for all indications are shown in Fig. 2, while the results of meta-analysis and forest plot for each disease separately are presented in Appendix A. The results are presented according to drug classes in Appendix C.
The coefficient of determination of the treatment effects was the highest for NVAF studies (R2trial = 0.41, 95% CI [0.03; 0.48]; Fig. 3). For acute VTE studies, the coefficient of determination was R2trial = 0.30 (95% CI [0.04; 0.84]; Fig. 4). Thus, in both NVAF and acute VTE studies, the correlation between NCB and all-cause mortality was weak.
Regarding the coefficient of determination for studies investigating the prevention of VTE, there was no correlation between NCB and all-cause mortality for medical patients (R2trial = 0.12, 95% CI [0.00; 0.36]; Fig. 5), neither for surgical patients (R2trial = 0.05, 95% CI [0.00; 0.23]; Fig. 6), nor for cancer patients (R2trial = 0.006, 95% CI [0.00; 1.00]; Fig. 7).
Surrogate threshold effect (STE)
Considering the lower limit of the prediction interval of the treatment effect on the surrogate endpoint in the prevention of VTE, treatment of VTE, and treatment of NVAF, the STE could not be determined and calculated for all the indications previously mentioned.
Sensitivity and post-hoc analyses
The coefficients of determination for double-blind clinical trials only and for those evaluating DOAC for the acute treatment of VTE only were higher than the primary analysis, but their wide confidence intervals did not support significant differences (Results of primary and sensitivity analysis are shown in details in Table 2). Similarly, the linear regression found no significant correlation between the overall death rate and the relative risk reduction of all-cause mortality (Figures of forest plot and GGPLOT are shown in Appendix B).
The objective of this study was to describe the relation between NCB and all-cause mortality to validate this outcome as a surrogate endpoint in NVAF and VTE trials using meta-regression. While the coefficient of determination R2trial was low for acute VTE and NVAF studies, the correlation between the NCB and all cause-mortality was very weak. Additionally, no correlation was observed in prevention studies for which the R2trial were negligible. These results were also consistent irrespective of experimental treatments and study design. Taken together, these results do not support the use of NCB as a surrogate endpoint for all-cause mortality in NVAF and VTE trials.
The limited association between NCB and all-cause mortality reduction may be explained by several factors. First, major bleeding and thrombosis may not lead systematically to death even though they are morbid outcomes. Indeed, in a post-hoc analysis of the ROCKET-AF trial comparing rivaroxaban and warfarin in NVAF26, only 1 in 10 deaths has been related to MB and ischemic stroke. In addition, a meta-analysis that analyzed the causes of death in patients receiving DOAC or warfarin in NVAF has reported that ischemic strokes and fatal bleedings were responsible for a minority (6%) of all death, while the main cause of death in NVAF appeared to be related to sudden cardiac death, heart failure, and myocardial infarction rather than outcomes targeted by study protocols27. Thus, anticoagulants have limited impacts on events ultimately leading to deaths among NVAF patients. Indeed, the low incidence of VTE and bleeding might be due to the improvement in patient management strategy28. Of note, the results herein did not show a relationship between the crude mortality rate and mortality risk reduction related to anticoagulant exposure. Thus, a lack of power related to low event incidence is unlikely.
Similarly, cancer has been found to be the most common cause of death (42%) in a meta-analysis of seven randomized trials evaluating DOAC for the treatment of VTE29, whereas recurrent VTE and fatal bleeding have been estimated to be responsible for only 20% and 6% of deaths, respectively.
Furthermore, a study combining the results of ACTIVE and RE-LY trials30,31 has calculated the NCB according to the relative weights of different events, and has reported that the clinical importance of major bleeding events, except hemorrhagic stroke, was less than that of ischemic stroke. Indeed, the adjusted hazard ratio of death after a hemorrhagic stroke, ischemic stroke, subdural hemorrhage, and major extracranial bleeding were highly different one from another (26.92, 8.33, 6.89, and 5.23, respectively). Hemorrhagic stroke has been reported to increase the risk of death by 3.29-fold per 100 patient-years compared to ischemic stroke30. Consequently, the relative importance and the clinical impact of major bleeding and thrombotic events are not similar and do not have the same weight and incidence32.
The European network for Health Technology Assessment does not recommend the use of a composite endpoint as a principal outcome measure when a suitable single primary endpoint is available, especially when the combined primary outcomes have different weights33. Therefore, the limited correlation between NCB and all-cause mortality found herein and the absence of recommendation from the regulatory agencies regarding the use of NCB in NVAF and VTE studies argue against the use of NCB as a primary outcome in randomized control trials.
Some limitations of the present study need to be noted. First, one trial identified in the database META EMBOL was not published and therefore not included in the present study, but it is unlikely that the inclusion of this trial would change the correlation between NCB and overall mortality found herein. Second, the number of patients, the experimental designs, the experimental and control groups, and the definitions of outcomes were variable between studies, which might have affected the statistical results. Additionally, only studies that measured the three outcomes (MB, recurrent ischemic/thrombotic event, and all-cause mortality) were included, without the composite or other outcomes, and so, the number of studies included was reduced. Finally, NCB was estimated in the present study by the sum of MB and thrombosis for all patients within each study, and not for each patient individually. This alternative calculation may have led to different results.
A weak correlation between NCB and all cause-mortality was found in studies investigating NVAF and acute VTE, whereas no correlation was observed in clinical situations where the mortality rate was low. Therefore, using the NCB should not be considered as a validated surrogate outcome of all-cause mortality in NVAF, acute VTE, and VTE prevention trials.
Shariff, N., Aleem, A., Singh, M., Li, Y. Z. & Smith, S. J. AF and venous thromboembolism—Pathophysiology, risk assessment and CHADS-VASc score. J Atr Fibrillation 5(3), 649 (2012).
Cohen, A. T. et al. Rivaroxaban for thromboprophylaxis in acutely Ill medical patients. N. Engl. J. Med. 368(6), 513–523 (2013).
Fisher, W. D. et al. Extended venous thromboembolism prophylaxis in patients undergoing hip fracture surgery—The SAVE-HIP3 study. Bone Jt J. 95-B(4), 459–466 (2013).
Haas, S. K. et al. Low-molecular-weight heparin versus placebo for the prevention of venous thromboembolism in metastatic breast cancer or stage III/IV lung cancer. Clin. Appl. Thromb. Off. J. Int. Acad. Clin. Appl. Thromb. 18(2), 159–165 (2012).
Julia, S. & James, U. Direct oral anticoagulants: A quick guide. Eur. Cardiol. Rev. 12(1), 40–45 (2017).
Pamela, J. B. et al. Warfarin use and mortality, stroke, and bleeding outcomes in a cohort of elderly patients with non-valvular atrial fibrillation. J Atr Fibrillation 12(1), 2155 (2019).
Agarwal, S., Hachamovitch, R. & Menon, V. Current trial-associated outcomes with warfarin in prevention of stroke in patients with nonvalvular atrial fibrillation: A meta-analysis. Arch. Intern Med. 172(8), 623–631 (2012) (discussion 631–633).
Lopes, L. C. et al. Bleeding risk in atrial fibrillation patients taking vitamin K antagonists: Systematic review and meta-analysis. Clin.. Pharmacol. Ther. 94(3), 367–375 (2013).
Wells, P. S. et al. The outpatient bleeding risk index: validation of a tool for predicting bleeding rates in patients treated for deep venous thrombosis and pulmonary embolism. Arch. Intern. Med. 163(8), 917–920 (2003).
Hsu, J. C., Hsieh, C.-Y., Yang, Y.-H.K. & Lu, C. Y. Net clinical benefit of oral anticoagulants: A multiple criteria decision analysis. PLoS ONE 10(4), e0124806 (2015).
Gonsalves, W. I., Pruthi, R. K. & Patnaik, M. M. The new oral anticoagulants in clinical practice. Mayo Clin. Proc. 88(5), 495–511 (2013).
Eikelboom John, W., Quinlan Daniel, J. & O’Donnell, M. Major bleeding, mortality, and efficacy of fondaparinux in venous thromboembolism prevention trials. Circulation 120(20), 2006–2011 (2009).
Gómez-Outes, A. et al. Case fatality rates of recurrent thromboembolism and bleeding in patients receiving direct oral anticoagulants for the initial and extended treatment of venous thromboembolism: A systematic review. J. Cardiovasc. Pharmacol. Ther. 20(5), 490–500 (2015).
Renda, G., di Nicola, M. & De Caterina, R. Net clinical benefit of non-vitamin K antagonist oral anticoagulants versus warfarin in phase III atrial fibrillation trials. Am. J. Med. 128(9), 1007-1014.e2 (2015).
Prasad, V., Kim, C., Burotto, M. & Vandross, A. The strength of association between surrogate end points and survival in oncology: A systematic review of trial-level meta-analyses. JAMA Intern. Med. 175(8), 1389–1398 (2015).
Liberati, A. et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 6(7), e1000100 (2009).
Laporte, S. et al. Assessment of clinically relevant bleeding as a surrogate outcome for major bleeding: Validation by meta-analysis of randomized controlled trials. J. Thromb. Haemost. 15(8), 1547–1558 (2017).
Schulman, S. & Kearon, C. Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients. J. Thromb. Haemost. 3(4), 692–694 (2005).
Sweeting, M. J., Sutton, A. J. & Lambert, P. C. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat. Med. 23(9), 1351–1375 (2004).
Buyse, M., Sargent, D. J., Grothey, A., Matheson, A. & de Gramont, A. Biomarkers and surrogate end points—The challenge of statistical validation. Nat. Rev. Clin. Oncol. 7(6), 309–317 (2010).
Ciani, O. et al. Use of surrogate end points in healthcare policy: a proposal for adoption of a validation framework. Nat. Rev. Drug Discov. 15(7), 516 (2016).
Burzykowski, T. & Buyse, M. Surrogate threshold effect: an alternative measure for meta-analytic surrogate endpoint validation. Pharm. Stat. 5(3), 173–186 (2006).
Validity of surrogate endpoints in oncology Executive summary of rapid report A10–05, Version 1.1. In: Institute for Quality and Efficiency in Health Care: Executive Summaries [Internet]. Cologne, Germany: Institute for Quality and Efficiency in Health Care (IQWiG); 2005 [cited 2019 Mar 11]. http://www.ncbi.nlm.nih.gov/books/NBK198799/.
R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing [Internet]. 2018. https://www.R-project.org.
Hadley Wickham. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York [Internet]. 2016. https://ggplot2.tidyverse.org.
Pokorney Sean, D. et al. Cause of death and predictors of all-cause mortality in anticoagulated patients with nonvalvular atrial fibrillation: Data from ROCKET AF. J. Am. Heart Assoc. 5(3), e002197 (2016).
Gómez-Outes, A. et al. Causes of death in anticoagulated patients with atrial fibrillation. J. Am. Coll. Cardiol. 68(23), 2508–2521 (2016).
Jiménez, D. et al. Trends in the management and outcomes of acute pulmonary embolism: Analysis from the RIETE registry. J. Am. Coll. Cardiol. 67(2), 162–170 (2016).
Gómez-Outes, A. et al. Causes of death in patients with venous thromboembolism anticoagulated with direct oral anticoagulants: A systematic review and meta-analysis. Semin. Thromb. Hemost. 44(4), 377–387 (2018).
Eikelboom, J. W. et al. Balancing the benefits and risks of 2 doses of dabigatran compared with warfarin in atrial fibrillation. J. Am. Coll. Cardiol. 62(10), 900–908 (2013).
Connolly, S. J. et al. Net clinical benefit of adding clopidogrel to aspirin therapy in patients with atrial fibrillation for whom vitamin K antagonists are unsuitable. Ann. Intern. Med. 155(9), 579–586 (2011).
Potpara, T. S. et al. Decision-making in clinical practice: oral anticoagulant therapy in patients with non-valvular atrial fibrillation and a single additional stroke risk factor. Adv. Ther. 34(2), 357–377 (2017).
Composite-endpoints.pdf [Internet]. [cited 2020 Nov 11]. https://www.eunethta.eu/wp-content/uploads/2018/01/Composite-endpoints.pdf.
We are grateful to the META-EMBOL group, which is supported by the Programme Hospitalier de Recherche Clinique 2008, Ministère de la Santé, France for kindly providing the data. The authors thank the Direction de la Recherche Clinique et de l’Innovation—Hospices Civil de Lyon for participation in the manuscript preparation.
This work was supported by the Programme national PAUSE, Collège de France. The sponsor financed the thesis of which this work is a part, and has no role in the design or the conduct of this study.
SL reports grants from LeoPharma and Bayer HealthCare for research projects, and personal fees from Bayer HealthCare (advisory board) and Pfizer (lectures). SP has received speaker fees from Janssen, and research grants from AstraZeneca (in-kind only), Janssen and Resverlogix. LB reports personal fees and others from Bayer, personal fees and other from BMS, personal fees and other from Pfizer, personal fees and other from Léo-Pharma, non-financial support from Sanofi, outside the submitted work. MC reports grants from BMS for research projects and personal consulting fees from Bayer HealthCare. RK, LV, GG, RA, SM, and JCL have no conflict of interest to declare.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kilo, R., Laporte, S., Arab, R. et al. Meta-regression of randomized control trials with antithrombotics: weak correlation between net clinical benefit and all cause-mortality. Sci Rep 11, 14728 (2021). https://doi.org/10.1038/s41598-021-94160-1