Introduction

Non-valvular atrial fibrillation (NVAF) and venous thromboembolism (VTE), which includes deep vein thrombosis (DVT) and pulmonary embolism (PE), are two common diseases associated with significant morbidity and mortality1. Furthermore, patients hospitalized for medical illness, undergoing surgery, and cancer patients have a higher risk of venous thromboembolism or bleeding events than the general population2,3,4. As they reduce the risk of death and injury related to thrombotic events, anticoagulants are the cornerstone of the management of these cardiovascular diseases5.

Since their discovery, the use of vitamin K antagonists (VKA) has been shown to reduce the mortality related to cardiovascular diseases6. However, due to their narrow therapeutic index, bleeding events are the most important complications related to the use of VKA, and the incidence of major bleeding ranges from 1.4 to 3.4% in NVAF patients7,8 and from 0 to 4.3% in VTE patients9. More recently, direct oral anticoagulants (DOAC) have been shown to be non-inferior to VKA in preventing thrombotic events while being associated with a reduction of major bleedings, and thus became the standard therapy and preventive treatment for VTE and NVAF10,11.

The positive impact of anticoagulants on cardiovascular mortality is presumably a result of the reduced amount of thrombotic events. However, major bleeding is also associated with significant morbidity and mortality12,13. To take into account the balance between potential benefits (i.e. reduced risk of thromboembolism) and harm (i.e. increased risk of major bleeding) in randomized trials, the concept of net clinical benefit (NCB), which is the sum of major bleeding and thrombotic events, appeared recently as a potentially relevant outcome in phase III clinical trials evaluating antithrombotics14. These measurable events lie within the pathophysiological spectrum of NVAF and VTE, allowing the summarization of treatment effects and increasing the number of total events, thus increasing the study power. Technically, to evaluate the value of NCB as a surrogate endpoint in clinical trials, a linear correlation between the treatment effects on the surrogate and on the final outcome needs to be established for each indication separately, and its strength has to be checked in advance with several statistics analytic methods15. However, direct evidence of the association between the treatment effects of anticoagulation on the NCB and cardiovascular mortality in the setting of clinical trials is lacking.

The present study aimed to explore the validity of the NCB as a potential surrogate for all-cause mortality in trials testing antithrombotics for the prevention and treatment of VTE and NVAF.

Materials and methods

The present study was conducted according to the PRISMA statement16.

Search strategy and study identification

First, all published randomized controlled trials registered in the META-EMBOL database (Silvy Laporte, University of Saint-Etienne, PHRC 2008) were investigated. This database collected the results of trials assessing the efficacy of antithrombotics in the prevention or treatment of VTE and NVAF17.

Additional studies were searched for on electronic databases such as MEDLINE, the Cochrane Library databases, and EMBASE from 1990 to December 2020, in English and non-English language by using sensitive keywords to detect all the studies (see online supplement).

Hand searching through medical journals, reviews, and bibliography of each selected article was carried out to identify additional studies that were not reported in those electronic databases and META-EMBOL.

Study selection

The database was screened by two authors (R.K and R.A) independently to identify studies that potentially met the eligibility criteria. These were: randomized controlled trials, parallel groups, open or double-blind design evaluating antithrombotics compared to placebo or control treatment for (1) VTE, (2) thromboprophylaxis in hospitalized patients for medical conditions, major orthopedic, and/or abdominal surgery, (3) thromboprophylaxis in cancer patients, and/or (4) NVAF. Also, for inclusion studies needed to report the three outcomes of interest: thrombosis, major bleeding, and all-cause mortality. Disagreements about inclusion were resolved by consensus or by consulting a third author (J.-C.L.).

Definition of outcomes

The net clinical benefit (NCB) was computed in each arm by adding major bleeding events and thrombosis events retrospectively in each study. The major bleeding in non‐surgical patients was defined according to the International Society on Thrombosis and Haemostasis (ISTH) criteria18, i.e. fatal bleeding, bleeding manifest in a critical organ such as intracranial bleeding, and/or explicit bleeding correlated with a decrease of hemoglobin level of 20 g/L or more, or necessitating a transfusion of at least two units of red cells or whole blood. Major bleeding for surgical patients was defined according to the European Agency for the Evaluation of Medical Products definition, i.e. bleeding detected at the surgical site and conducting to re-operation or any special medical intervention, in addition to the criteria mentioned above.

The effects of the assessed treatments on thrombotic events were evaluated from the main pre-specified efficacy outcome of each trial. All-cause mortality was used irrespective of its relationship with the cardiovascular event.

Data extraction

When a trial met the eligibility criteria, two authors separately extracted the following data in addition to thrombotic and bleeding events: name of the first author and year(s) of publication, study acronym, study design, disease, treatment regimens, class of comparison, and study size.

Statistical analysis

For each condition (acute treatment of VTE, treatment of NVAF, and VTE prevention in medical, surgical, and cancer patients), a meta-analysis was conducted and forest plots were generated and computed using the random-effects model to estimate the relative risk (RR) for all-cause mortality (standard outcome) and NCB (surrogate outcome), as well as the corresponding 95% confidence intervals (CI) in patients treated (in the experimental anticoagulation group) compared to patients in control groups. Additional meta-analysis and forest plots were generated for each class of medicine separately (Antiplatelet, VKA, DOAC, and LMWH) and calculated the RR for all-cause mortality and NCB. Adjusted continuity corrections of 0.5 were used for any study with no event19.

To validate NCB as a surrogate of all-cause mortality, the method reported by Buyse et al.20 was used. A linear regression model was therefore used to assess the association between the RR for NCB and the RR for all-cause mortality by calculating the coefficient of determination (R2trial). The percentile bootstrapping method (resampling 1000 times) was applied in order to obtain a high accuracy to compute the 95% confidence interval for R2trial in addition to the prediction interval. The validity of the surrogate endpoints depends on the strength of the association with the ultimate endpoint. Indeed, the coefficient of determination R2trial should be more than 0.65 and close to 121. In practice, an R2trial with an upper limit of the 95% confidence interval (95% CI) ≤ 0.7 (i.e. limited correlation) confirms the lack of validity of the surrogate endpoint, whereas an R2trial with a lower limit of the 95% confidence interval ≥ 0.85 supports the validity of the surrogate. In case of intermediate correlation (0.7 < Rtrial < 0.85), the validation of the surrogate endpoints remains unclear15.

The surrogate threshold effect (STE) can also be assessed to estimate the minimal treatment effect on the surrogate endpoint predicting a significantly nonzero effect on the true endpoint22,23. To compute the STE, the linear regression model was calculated and the 95% prediction intervals were plotted. The value of the STE is the value on the x-axis (log RR of NCB) for which the lower limit of the prediction interval meets a point corresponding to 1 (zero effect on the true endpoint) on the y-axis (log RR of all-cause mortality).

Furthermore, two sensitivity analyses were performed to assess the robustness of the results. The first analysis included only studies with a double-blind design, whereas the second one was applied to studies that included only the new direct oral anticoagulants (except ximelagatran) in the experimental arm.

The association between the mortality rate and the relative risk reduction was explored using linear regression for all indications together.

The linear regression models were performed using the statistical software R, version 3.5.224 with the META, METAFOR, and GGPLOT25 packages.

Results

Search results and characteristics of studies

The primary META-EMBOL database and the additional literature search identified 264 trials for review. Among them, there were 25 duplicated studies, 41 studies were excluded as they did not meet the year of publication criterion, and 73 articles were excluded after full-text review. Finally, a total of 125 eligible studies were included for data extraction (Fig. 1).

Figure 1
figure 1

PRISMA flowchart. NCB, net clinical benefit.

Full size image

Among those included studies, 27 studies were conducted in the field of NVAF (114,689 patients), 27 focused on acute treatment of VTE (55,216 patients), 25 on thromboprophylaxis in patients hospitalized for medical conditions (69,022 patients), and 38 on major orthopedic and abdominal surgery (70,982 patients). Additionally, 8 studies (6372 patients) assessed thromboprophylaxis in cancer patients. A total of 79 studies were double-blind randomized controlled trials, 28 were open-label with blind adjudication, and 18 had an open-label design.

Treatments assessed in experimental arms were apixaban (k = 8), acenocoumarol (k = 2), betrixaban (k = 2), dabigatran (k = 8), rivaroxaban (k = 12), dalteparin (k = 4), edoxaban (k = 2), AZD0837 (k = 1), idrabiotaparinux (k = 2), fondaparinux (k = 10), aspirin (k = 5), dipyridamole (k = 2), nadroparin (k = 11), ximelagatran (k = 4), enoxaparin (k = 19), reviparin (k = 2), bemiparin (k = 1), semuloparin (k = 6), indobufen (k = 1), UFH (k = 2), tinzaparin (k = 1), other LMWH (k = 2), certoparin (k = 5), gemcitabine + dalteparin (k = 1), fraxiparin (k = 1), ardeparin (k = 2), and warfarin with/without aspirin (k = 9; Table 1).

Table 1 Studies characteristics and main antithrombotics for all the studied diseases.
Full size table

Surrogacy evaluation by clinical indication

Summary of the meta-analysis and forest plot for all indications are shown in Fig. 2, while the results of meta-analysis and forest plot for each disease separately are presented in Appendix A. The results are presented according to drug classes in Appendix C.

Figure 2
figure 2

Summary of the meta-analysis and forest plot for all indications. Forest plot of treatment effects on all-cause mortality and net clinical benefit (NCB). The horizontal error bars show the 95% confidence interval (CI) of each relative risk (RR) based on the random-effect model. The square represents the RR. An RR of < 1 favors the experimental group, an RR = 1 indicates no difference in treatment effects, and an RR of > 1 indicates a harmful effect of the control group. AF = Atrial fibrillation, K = number of studies for each indication, MB = Major bleeding, N = Total number of the included patients, VTE = Venous thromboembolism.

Full size image

The coefficient of determination of the treatment effects was the highest for NVAF studies (R2trial = 0.41, 95% CI [0.03; 0.48]; Fig. 3). For acute VTE studies, the coefficient of determination was R2trial = 0.30 (95% CI [0.04; 0.84]; Fig. 4). Thus, in both NVAF and acute VTE studies, the correlation between NCB and all-cause mortality was weak.

Figure 3
figure 3

Trial-level association between treatment effects on net clinical benefit (NCB) and all-cause mortality in the treatment of non-valvular atrial fibrillation. The correlation was (Cor) = 0.62 with the linear regression model: "Log RRDeath = 0.44 × Log RRNCB − 0.11”. Each study is represented by a circle. A log scale was used for the x-axis and y-axis. The solid blue line represents the regression line and the grey area represents the 95% confidence interval. The red dashed lines represent the upper and lower limits of the 95% prediction interval. RR, relative risk.

Full size image
Figure 4
figure 4

Trial-level association between treatment effects on net clinical benefit (NCB) and all-cause mortality in the treatment of acute venous thromboembolism. Cor = 0.24. The corresponding linear regression model was "Log RRDeath = 0.6 × Log RRNCB + 0.15”. Each study is represented by a circle. A log scale was used for the x-axis and y-axis. The solid blue line represents the regression line and the grey area represents the 95% confidence interval. The red dashed lines represent the upper and lower limits of the 95% prediction interval. RR, relative risk.

Full size image

Regarding the coefficient of determination for studies investigating the prevention of VTE, there was no correlation between NCB and all-cause mortality for medical patients (R2trial = 0.12, 95% CI [0.00; 0.36]; Fig. 5), neither for surgical patients (R2trial = 0.05, 95% CI [0.00; 0.23]; Fig. 6), nor for cancer patients (R2trial = 0.006, 95% CI [0.00; 1.00]; Fig. 7).

Figure 5
figure 5

Trial-level association between treatment effects on net clinical benefit (NCB) and all-cause mortality in the treatment of the prevention of VTE in medical patients. Cor = 0.32 and the linear regression model was "Log RRDeath = 0.26 × Log RRNCB + 0.05”. Each study is represented by a circle. A log scale was used for the x-axis and y-axis. The solid blue line represents the regression line and the grey area represents the 95% confidence interval. The red dashed lines represent the upper and lower limits of the 95% prediction interval. RR, relative risk.

Full size image
Figure 6
figure 6

Trial-level association between treatment effects on net clinical benefit (NCB) and all-cause mortality in the treatment of the major orthopedic and abdominal surgery. Cor = 0.24 with the linear regression model "Log RRDeath = 0.49 × Log RRNCB − 0.05". Each study is represented by a circle. A log scale was used for the x-axis and y-axis. The solid blue line represents the regression line and the grey area represents the 95% confidence interval. The red dashed lines represent the upper and lower limits of the 95% prediction interval. RR, relative risk.

Full size image
Figure 7
figure 7

Trial-level association between treatment effects on net clinical benefit (NCB) and all-cause mortality in the treatment of the other prevention in cancer patients. Cor = 0.41 and the linear regression model: "Log RRDeath = -0.05 × Log RRNCB + 0.01". Each study is represented by a circle. A log scale was used for the x-axis and y-axis. The solid blue line represents the regression line and the grey area represents the 95% confidence interval. The red dashed lines represent the upper and lower limits of the 95% prediction interval. RR, relative risk.

Full size image

Surrogate threshold effect (STE)

Considering the lower limit of the prediction interval of the treatment effect on the surrogate endpoint in the prevention of VTE, treatment of VTE, and treatment of NVAF, the STE could not be determined and calculated for all the indications previously mentioned.

Sensitivity and post-hoc analyses

The coefficients of determination for double-blind clinical trials only and for those evaluating DOAC for the acute treatment of VTE only were higher than the primary analysis, but their wide confidence intervals did not support significant differences (Results of primary and sensitivity analysis are shown in details in Table 2). Similarly, the linear regression found no significant correlation between the overall death rate and the relative risk reduction of all-cause mortality (Figures of forest plot and GGPLOT are shown in Appendix B).

Table 2 Primary and sensitivity analysis for all the studied diseases.
Full size table

Discussion

The objective of this study was to describe the relation between NCB and all-cause mortality to validate this outcome as a surrogate endpoint in NVAF and VTE trials using meta-regression. While the coefficient of determination R2trial was low for acute VTE and NVAF studies, the correlation between the NCB and all cause-mortality was very weak. Additionally, no correlation was observed in prevention studies for which the R2trial were negligible. These results were also consistent irrespective of experimental treatments and study design. Taken together, these results do not support the use of NCB as a surrogate endpoint for all-cause mortality in NVAF and VTE trials.

The limited association between NCB and all-cause mortality reduction may be explained by several factors. First, major bleeding and thrombosis may not lead systematically to death even though they are morbid outcomes. Indeed, in a post-hoc analysis of the ROCKET-AF trial comparing rivaroxaban and warfarin in NVAF26, only 1 in 10 deaths has been related to MB and ischemic stroke. In addition, a meta-analysis that analyzed the causes of death in patients receiving DOAC or warfarin in NVAF has reported that ischemic strokes and fatal bleedings were responsible for a minority (6%) of all death, while the main cause of death in NVAF appeared to be related to sudden cardiac death, heart failure, and myocardial infarction rather than outcomes targeted by study protocols27. Thus, anticoagulants have limited impacts on events ultimately leading to deaths among NVAF patients. Indeed, the low incidence of VTE and bleeding might be due to the improvement in patient management strategy28. Of note, the results herein did not show a relationship between the crude mortality rate and mortality risk reduction related to anticoagulant exposure. Thus, a lack of power related to low event incidence is unlikely.

Similarly, cancer has been found to be the most common cause of death (42%) in a meta-analysis of seven randomized trials evaluating DOAC for the treatment of VTE29, whereas recurrent VTE and fatal bleeding have been estimated to be responsible for only 20% and 6% of deaths, respectively.

Furthermore, a study combining the results of ACTIVE and RE-LY trials30,31 has calculated the NCB according to the relative weights of different events, and has reported that the clinical importance of major bleeding events, except hemorrhagic stroke, was less than that of ischemic stroke. Indeed, the adjusted hazard ratio of death after a hemorrhagic stroke, ischemic stroke, subdural hemorrhage, and major extracranial bleeding were highly different one from another (26.92, 8.33, 6.89, and 5.23, respectively). Hemorrhagic stroke has been reported to increase the risk of death by 3.29-fold per 100 patient-years compared to ischemic stroke30. Consequently, the relative importance and the clinical impact of major bleeding and thrombotic events are not similar and do not have the same weight and incidence32.

The European network for Health Technology Assessment does not recommend the use of a composite endpoint as a principal outcome measure when a suitable single primary endpoint is available, especially when the combined primary outcomes have different weights33. Therefore, the limited correlation between NCB and all-cause mortality found herein and the absence of recommendation from the regulatory agencies regarding the use of NCB in NVAF and VTE studies argue against the use of NCB as a primary outcome in randomized control trials.

Some limitations of the present study need to be noted. First, one trial identified in the database META EMBOL was not published and therefore not included in the present study, but it is unlikely that the inclusion of this trial would change the correlation between NCB and overall mortality found herein. Second, the number of patients, the experimental designs, the experimental and control groups, and the definitions of outcomes were variable between studies, which might have affected the statistical results. Additionally, only studies that measured the three outcomes (MB, recurrent ischemic/thrombotic event, and all-cause mortality) were included, without the composite or other outcomes, and so, the number of studies included was reduced. Finally, NCB was estimated in the present study by the sum of MB and thrombosis for all patients within each study, and not for each patient individually. This alternative calculation may have led to different results.

Conclusion

A weak correlation between NCB and all cause-mortality was found in studies investigating NVAF and acute VTE, whereas no correlation was observed in clinical situations where the mortality rate was low. Therefore, using the NCB should not be considered as a validated surrogate outcome of all-cause mortality in NVAF, acute VTE, and VTE prevention trials.