Trial sequential analysis of efficacy and safety of direct oral anticoagulants and vitamin K antagonists against left ventricular thrombus

Meta-analysis may increase the risk of random errors. Trial sequential analysis (TSA) has been developed to adjust for these random errors. We conducted TSA on the efficacy and safety of direct oral anticoagulants (DOACs) and vitamin K antagonists (VKAs) in left ventricular thrombus (LVT) patients in order to estimate how many additional patients should be required to draw definite conclusions. PubMed, Scopus, and Cochrane Library databases were searched for articles directly comparing DOACs and VKAs for LVT in LV thrombus resolution, stroke, any thromboembolism, major bleeding, any bleeding, and all-cause death. TSA was conducted with a cumulative Z-curve, monitoring boundaries, and required sample size. A simulated trial was run and TSA estimated the sample sizes of trials needed to draw definite conclusions. Of 4749 articles, 25 studies were used for the analysis. TSA revealed the current sample size already demonstrated superiority of DOACs in LV thrombus resolution and stroke, and futility in any thromboembolism and all-cause death. Two other outcomes did not achieve the required sample size. The sample size of new trials needed to demonstrate the superiority of DOACs over VKAs was estimated 400 for any bleeding. Corresponding trials needed to demonstrate no significant differences could be estimated for major bleeding and any bleeding (n = 200 and n = 2000, respectively). Current results show that the sample size required to draw definite conclusions was not reached for two outcomes, and there was a risk of random error. Further randomized controlled trials with sample sizes estimated by TSA will work effectively to obtain valid conclusions.

Direct oral anticoagulants (DOACs) are indicated for prevention of ischemic stroke and systemic embolism caused by non-valvular atrial fibrillation (AF), and for treatment and prevention of recurrent venous thromboembolisms (VTE); however, they have not been approved to treat left ventricular thrombus (LVT).Recently, several small, randomized controlled trials (RCTs) [1][2][3] and many observational studies  on off-label use of DOACs for LVT have been published, attracting attention to the potential use of DOACs in treatment of LVT. Howeer, results regarding efficacy and safety of DOACs vary among reports.Although many meta-analyses comparing DOACs and vitamin K antagonists (VKAs) for LVT have been published in the past few years [25][26][27][28][29][30][31][32][33][34] , no definitive consensus has been reached yet, and some controversy remains.Most of these meta-analyses concluded that the efficacy and safety of DOACs should be evaluated in large RCTs, but implementation is a major challenge because the low incidence of LVT requires significant effort in patient recruitment and large RCTs require significant funding.Therefore, evidence from small RCTs and observational studies that have accumulated so far should be carefully assessed, and minimal RCTs should be conducted to complement evidence that is currently lacking.
Trial sequential analysis (TSA) is conceptually similar to sequential interim analysis of RCTs, in which trial results are validated at regular intervals to determine whether certain differences, or lack thereof, due to interventions, have been conclusively demonstrated 35 .TSA can provide a required sample size and can adjust boundaries for trial sequential monitoring for benefit, harm, and futility.Conclusions drawn using TSA demonstrate greater reliability than those relying upon traditional meta-analytical techniques 36 .Furthermore, TSA can estimate how TSA.TSA is a statistical approach that can determine sufficient sample sizes and reduce false inferences from meta-analyses due to type 1 or type 2 errors.This approach combines traditional meta-analysis with a method of calculating sample sizes and adjusting boundaries of sequential monitoring of trials for benefit, harm, and futility.In TSA, monitoring boundaries are applied and effect sizes are tested with each trial added to the analysis until the boundary of significance (benefit or harm) or lack of significance (futility) is achieved.
If cumulative data (Z-curve) cross the traditional boundary (Z = ± 1.96, p = 0.05), but do not cross the superiority boundary, an effect is considered false and the trial should be continued to detect or reject the effect of the intervention (false positive) (Fig. 1A).If the cumulative Z-curve crosses the superiority boundary before the required sample size is reached, it is considered firm evidence and no further trials are needed (true positive) (Fig. 1B).If the cumulative Z-curve does not cross the traditional boundary and does not cross the futility boundary, evidence is considered insufficient and the trial should be continued to detect or reject the effect of the intervention (false negative) (Fig. 1C).If the required sample size is not reached but the cumulative Z-curve crosses the futility boundary, it indicates no intervention effect and further trials are meaningless (true negative) (Fig. 1D).
To estimate the number of trials needed to draw definitive conclusions about outcomes in the future, two simulated trials were set up and TSA was conducted.For superiority analysis, it is necessary for the cumulative Z-curve to cross the superiority boundary to draw a definitive conclusion.A simulated trial was set up in which the Z-curve crossed the superiority boundary, assuming that there were predefined risk differences between DOACs and VKAs.The risk difference between DOACs and VKAs for each outcome was assumed to be the aforementioned values, and the minimum sample size was calculated to cross the superiority boundary.For futility analysis, a simulated trial was set up in which the Z-curve crossed the futility boundary, assuming that there were no risk differences between DOACs and VKAs.The risks of LV thrombus resolution, stroke, any thromboembolism, major bleeding, any bleeding, and all-cause death were assumed to be 70%, 8%, 20%, 3%, 10%, and 8% for both DOACs and VKAs, respectively, and the minimum sample size was measured to cross the futility boundary.Quality assessment of the articles was conducted using the Newcastle-Ottawa Assessment Scale for observational studies and the Cochrane risk of bias tool for RCTs in meta-analyses 39,40 .

Results
Articles directly comparing VKAs and DOACs for at least one of the outcomes in patients with LVT were selected based on PRISMA guidelines (Supplementary Fig. S2).Of the 4749 articles retrieved, 1129 were excluded due to duplication and 1117 due to publication before DOACs were commercially available.2432 were excluded because their titles or abstracts did not meet inclusion criteria, and 71 were read in full text.46 articles that met one or more of the exclusion criteria were excluded after reading the full text.Finally, 25 articles that directly compared DOACs and VKAs were included in this meta-analysis.Of 3490 patients, 1047 treated with DOACs and 2443 treated with VKAs.These publications included three RCTs, 20 retrospective observational studies, and two prospective observational studies.Tables 1 and 2 show a summary of included articles, the main clinical characteristics, and comparison between DOACs and VKAs.Qualitative assessment of the articles is presented in Supplementary Table S2 and S3.
Left ventricular thrombus resolution.Twenty articles (2 RCTs and 18 observational studies) reported LV thrombus resolution.Conventional meta-analysis showed that the LV thrombus resolution rate was significantly higher with DOACs than with VKAs (pooled OR: 1.28, 95% CI 1.05-1.57,p = 0.02, Fig. 2A).Using TSA, the required sample size for significant or non-significant differences in LV thrombus resolution based on an event rate of 74% with DOACs and 68% with VKAs was 1796 patients (Fig. 3).The number of patients included in the present 20 trials was 1616, which indicated that 90% of the required number of patients have been included so far.Ninety percent of the required sample size was accumulated and the cumulative Z-curve crossed both the traditional boundary and the superiority boundary.Thus, the evidence from the pooled meta-analysis is considered conclusive and truly positive.Based on these results, it is concluded that DOACs have a significantly higher incidence of LV thrombus resolution than VKAs and the sample size is sufficient to demonstrate significance, indicating that further accumulation of evidence is unnecessary.

Stroke.
Eighteen articles (3 RCTs and 15 observational studies) reported the stroke rate.Conventional metaanalysis showed that the incidence of stroke was significantly lower for DOACs than for VKAs (pooled OR 0.68, 95% CI 0.54-0.86,p < 0.01, Fig. 2B).Using TSA, the required sample size was 2714 patients (the number of patients in 18 articles was 2638) to demonstrate or reject the difference in incidence of stroke of 7% for DOACs and 10% for VKAs (Fig. 4).Ninety-seven percent of the required sample size was accumulated and the cumulative Z-curve crossed both the traditional boundary and the superiority boundary.

Any thromboembolism.
Eighteen articles (2 RCTs and 16 observational studies) provided rates of any thromboembolism.A conventional meta-analysis showed that the incidence of any thromboembolism was comparable between DOACs and VKAs (pooled OR 0.80, 95% CI 0.53-1.22,p = 0.29, Fig. 2C).With TSA, the sample size required to demonstrate or reject the difference in the event rate of any thromboembolism of 16% for DOACs and 21% for VKAs was 4955 patients (Fig. 5).Currently, 3081 patients (62% of required sample size) are included and the cumulative Z-curve crossed the futility boundary.Therefore, the pooled meta-analysis was considered definitive with sufficient evidence (true negative).These results indicated that the incidence of any thromboembolism with DOACs does not differ significantly from that with VKAs and the sample size is sufficient to demonstrate futility, suggesting that further evidence accumulation is not warranted.

Major bleeding. Eleven publications (2 RCTs and 9 observational studies) reported rates of major bleeding.
A meta-analysis showed no significant difference between DOACs and VKAs in incidence of major bleeding (pooled OR 0.67, 95% CI 0.27-1.66,p = 0.34, Fig. 2D).To demonstrate or reject the difference in the event rate of major bleeding of 2% for DOACs and 4% for VKAs, a sample size of 3749 patients (versus only 1869 patients in the 11 studies) was estimated by TSA (Fig. 6).Fifty percent of the required sample size is available and the cumulative Z-curve did not cross either the traditional or futility boundary, suggesting a lack of evidence (false negative).
Assuming an event rate of 2% for DOACs and 4% for VKAs, the cumulative Z-curve could not cross the superiority boundary no matter how many patients were added.When 24,000 patients were added, the boundary of futility was crossed before the superiority or traditional boundaries were crossed (Supplementary Fig S3 Any bleeding.There were 16 publications (1 RCT and 15 observational studies) that reported rates of any bleeding.The incidence of any bleeding was significantly lower for DOACs than for VKAs (pooled OR 0.67, 95% CI 0.54-0.83,p < 0.01, Fig. 2E).According to TSA, the sample size required to demonstrate or reject the difference in the incidence of any bleeding with an event rate of 8% for DOACs and 10% for VKAs was 6429 patients (Fig. 7).Only 2922 patients (45% of the required sample size) have been evaluated and the cumulative Z-curve crossed the traditional boundary, but did not cross the superiority boundary.Accordingly, the pooled metaanalysis evidence is not conclusive and is considered a spurious effect (false positive).With an 8% event rate for DOACs and 10% event rate for VKAs, an additional 400 patients will need to be included to cross the superiority boundary (Supplementary Fig. S5).With an event rate of 10% for both DOACs and VKAs, 2000 patients would be required to cross the futility boundary (Supplementary Fig. S6).
All-cause death.Thirteen articles (2 RCTs and 11 observational studies) reported rates of all-cause death.
The incidence of all-cause death did not differ significantly between DOACs and VKAs in a pooled analysis (pooled OR 0.67, 95% CI 0.45-1.02,p = 0.06, Fig. 2F).For TSA, the sample size required to determine whether there is a significant difference in all-cause death was 2124 patients, based on an event rate of 6% for DOACs and 10% for VKAs (Fig. 8).Eighty percent of the required sample size (1696 patients) was included and the cumulative Z-curve crossed the futility boundary.Accordingly, the pooled meta-analysis evidence was considered sufficient and definitive (true negative).These results supported the conclusion that the incidence of all-cause death with DOACs was not significantly different from that with VKAs, and the sample size was sufficient to prove futility, suggesting that no further accumulation of evidence is needed.

Discussion
To the best of our knowledge, this is the first study to employ TSA regarding the efficacy and safety of DOACs and VKAs in LVT, and to determine sample sizes required to reach definitive conclusions for each outcome.The main findings of this study are summarized as follows: (1) DOACs and VKAs have been directly compared in three RCTs and 22 prospective or retrospective observational studies; (2) a conventional meta-analysis found that DOACs were favorable over VKAs for LV thrombus resolution, stroke and any bleeding, while any thromboembolism, major bleeding, and all-cause death were not significantly different in the two treatments; (3) TSA results showed that the superiority or futility boundary was reached for LV thrombus resolution and stroke, and any thromboembolism and all-cause death, whereas cumulative numbers of patients did not reach the required sample size for major bleeding and any bleeding, resulting in either type 1 or type 2 errors; (4) sample sizes (patient numbers) in future trials needed to demonstrate the superiority of DOACs could be estimated for any bleeding; (5) it is possible to estimate patient numbers needed in future trials to demonstrate that there are no significant differences between DOACs and VKAs for major bleeding and any bleeding.
Previous studies.DOACs are convenient for daily clinical use because they do not require blood international normalized ratio monitoring or dietary restrictions and have a rapid onset of efficacy 41 .Furthermore, DOACs have proven useful in a variety of situations, such as cancer-related VTE 42 and AF with bioprosthetic mitral valves 43 .Although LVT has traditionally been treated with VKAs, off-label use of DOACs has been increasingly reported during the last decade.Several meta-analyses compared efficacy and safety of DOACs and VKAs for LVT, but conclusions conflicted among publications [25][26][27][28][29][30][31][32][33][34] .
Although meta-analyses are considered the highest level of evidence for assessing benefits and harms of interventions, and while they permit strong inferences about the quality of available evidence, they may also increase the risk of random errors (type 1 error [false positive] or type 2 error [false negative]) when statistical tests are performed repetitively on accumulated data, or when sample sizes or numbers of events are insufficient [44][45][46][47] .Meta-analyses that do not reach the required sample size increase the risk of over-or underestimating effects of treatment, leading to spurious inferences 48 .To overcome these problems, trial sequential analysis (TSA) has been developed to adjust for these random errors and to address false conclusions 38 .
Current study.In the current study, which included three RCTs and 22 observational studies, a randomeffects model showed no significant differences between DOACs and VKAs for any thromboembolism, major bleeding, and all-cause death, while the incidence of LV thrombus resolution, stroke, and any bleeding differed significantly.However, when performing TSA, the sample size required to draw definitive conclusions was not reached for two of the six outcomes (major bleeding and any bleeding).Although major bleeding showed no significant difference between DOACs and VKAs in the conventional meta-analysis, TSA showed false-negative and no definitive conclusions were reached.The same was true for any bleeding showed significantly lower inci-  dence among patients with taking DOACs in the conventional meta-analysis, whereas TSA showed a false-positive, which did not lead to a definitive conclusion.TSA revealed that the results of conventional meta-analyses for these two outcomes lack sufficient evidence and that it would be misleading to draw conclusions based only upon conventional meta-analysis results.On the other hand, regarding LV thrombus resolution, stroke, any    thromboembolism, and all-cause death, the cumulative Z-curve crossed the superiority or futility boundary, and sufficient sample size for a definitive conclusion was reached.
For major bleeding and any bleeding, simulation trials were used to determine additional sample sizes required.Assuming that event rates are the same for DOACs and VKAs in simulated trials, we determined the additional sample sizes necessary to draw conclusions that there are no significant differences between the two groups.In contrast, when assuming that the DOACs is preferable to VKAs, it was possible to determine the number of additional samples needed to conclude the superiority of DOACs, except for major bleeding.For major bleeding, no matter how many trials were added to show the superiority of the DOACs, the required number of samples did not cross the superiority boundary before crossing the futility boundary.
As far as we know, several RCTs comparing DOACs and VKAs for LVT are currently ongoing (NCT04970576; NCT05705089).A trial evaluated LV thrombus resolution, stroke or systemic embolism, and major bleeding in LVT with acute coronary syndromes treated with Rivaroxaban or Warfarin (NCT04970576).Given the results of our TSA, an additional 200 patients were needed to demonstrate that there was no difference between DOACs and VKAs in major bleeding.

Study limitations.
The limitations of this study include the following.First, almost all limitations of metaanalysis also apply to TSA, because TSA uses data from meta-analyses.Publication bias and risk of bias in individual studies may affect the results of TSA.Second, since there are only three RCTs comparing DOACs and VKAs for LVT, the present TSA included observational studies in addition to RCTs.Although TSA is generally used to analyze RCTs, our study also included observational studies.The quality of each observational study was scored using the Newcastle-Ottawa Assessment Scale, and the quality of most studies was high; however, results need to be interpreted with caution.Third, the degree of bleeding and the proportion of time in the therapeutic range for VKAs have not been standardized among studies, and some patients also received concomitant antiplatelet agents, which may have affected the results.Fourth, follow-up duration in each event varied among the included studies, which may have affected the results.Fifth, in TSA and in estimating future needed sample sizes, predefined event rates for DOACs and VKAs were used for each calculation, and some of these assumptions may not be correct.

Conclusions
TSA revealed that the current sample sizes have still not been reached to draw definite conclusions for major bleeding and any bleeding, and there is a risk of random error.TSA is a useful method to estimate sample sizes of further RCTs to draw definitive conclusions.

Figure 1 .
Figure 1.Schema of four examples of trial sequential analysis (TSA).(A) False positive: the cumulative Z-curve crosses the traditional boundary (Z = ± 1.96, p = 0.05), but does not cross the superiority boundary.(B) True positive: the cumulative Z-curve crosses the superiority boundary before the required sample size is reached.(C) False negative: the cumulative Z-curve crosses neither the traditional boundary nor the futility boundary.(D) True negative: the required sample size is not reached but the cumulative Z-curve crosses the futility boundary.

Figure 4 .
Figure 4. TSA of DOACs and VKAs in stroke using power of 80%, 5% significance, to detect a 3% absolute difference (DOACs: 7%, VKAs: 10%).The required sample size was 2714 (vertical red line).The cumulative Z-curve (blue line with small black squares representing each trial) crossed both the traditional boundary (horizontal red line) and the superiority boundary (trial sequential monitoring boundary) (concave red line), indicating evidence is sufficient (true positive).

Figure 6 .
Figure 6.TSA of DOACs and VKAs in major bleeding using power of 80%, and 5% significance, to detect a 2% absolute difference (DOACs:2%, VKAs: 4%).The required sample size was 3749 (vertical red line).The cumulative Z-curve (blue line with small black squares representing each trial) did not cross the traditional boundary (horizontal red line), the superiority boundary (trial sequential monitoring boundary) (concave red line), or the futility boundary (convex red line), indicating a lack of evidence (false negative).

Figure 7 .
Figure 7. TSA of DOACs and VKAs in any bleeding using power of 80% and 5% significance, to detect a 2% absolute difference (DOACs: 8%, VKAs: 10%).The required sample size was 6429 (vertical red line).The cumulative Z-curve (blue line with small black squares representing each trial) crossed the traditional boundary (horizontal red line), but did not cross the superiority boundary (trial sequential monitoring boundary) (concave red line), indicating a lack of evidence (false positive).

Figure 8 .
Figure 8. TSA of DOACs and VKAs in all-cause death using power of 80% and 5% significance, to detect a 4% absolute difference (DOACs:6%, VKAs: 10%).The required sample size was 2124 (vertical red line).The cumulative Z-curve (blue line with small black squares representing each trial) crossed both the traditional boundary (horizontal red line) and the superiority boundary (trial sequential monitoring boundary) (concave red lines), indicating evidence is sufficient (true positive).

Table 2 .
Clinical characteristics and the comparison between treatment groups.Data are expressed as number (%) or means ± standard deviations.AFib atrial fibrillation, CAD coronary artery disease, CKD chronic kidney disease, DL dyslipidemia, DM diabetes mellitus, DOAC direct oral anticoagulant, HT hypertension, VKA vitamin k antagonist.