Second look at the potential use of ginseng berry extract for treating erectile dysfunction

Choi et al.¹ conducted a Phase II double-blind, randomized, placebo-controlled clinical trial of Korean ginseng berry extract (KGBE) for the treatment of mild-to-moderate male erectile dysfunction. Their initial univariate analysis indicates that KGBE outperformed placebo at 8 weeks only with respect to the premature ejaculation diagnostic tool (PEDT) score. Furthermore, the study’s authors performed no post-hoc test on these results to keep the family-wise error rate of their results below 5%. Despite these results, the authors have come to the conclusion that KGBE is beneficial for the sexual functioning of males with erectile dysfunction.

Ginsenosides, present in Panax ginseng extract, comprise an example of naturally occurring, pharmacologically active substances, which have been shown to increase nitric oxide production in cultured endothelial cells.² Given that augmented nitric oxide activity also results from the action of phosphodiesterase inhibitors (such as sildenafil) used to treat male erectile dysfunction,³ ginsenosides are being assessed for the same indication. To this end, Choi et al. conducted a well-designed, randomized, double-blind, placebo-controlled, Phase II trial of Korean ginseng berry extract (KGBE) in a cohort of 119 men with mild-to-moderate erectile dysfunction.¹ Outcome was measured through the overall and subsection scores of the IIEF-15 (International Index of Erectile function) and through the PEDT score (premature ejaculation diagnostic tool), at 4 and 8 weeks post treatment. Of these, the observed effect of KGBE reached the researchers’ pre-defined, univariate statistical significance cutoff (P<0.05) vs placebo with regard to: intercourse satisfaction (week 4), overall IIEF-15 score (week 4), and PEDT score (weeks 4 and 8) (Table 1).

Table 1 Subject parameters reaching researcher-defined, univariate statistical significance (P<0.05)

In their discussion of these results, Choi et al. came to the conclusion that the ‘…KGBE-treated group showed improvement in all domains of sexual function,’ and that, ‘the 8-week treatment… has a positive effect on their sexual life’.¹ This conclusion was most likely reached due to the apparent improvement of sexual function scores within the treatment group, over the course of eight weeks, an improvement which in many cases had an intra-group P-value of <0.05 in univariate analysis. However, KGBE did no better than placebo during this trial, with the exception of the three measurements described above. In fact, even adhering to the study’s definition of statistical significance, only the secondary endpoint of PEDT score had a sustained, superior effect vs placebo at 8 weeks. Thus, the validity of the authors’ conclusion is highly questionable, given the results of the particular study.

Besides the limited parameters in which KGBE appears to be superior to placebo, the very definition of statistical significance used in this study is problematic. Specifically, it is well known that the cumulative alpha-error increases significantly when comparative analysis of multiple variables/parameters is carried out within the same data set, a statistical property termed family-wise error rate.⁴ The result is that a cutoff of P<0.05 used in multiple comparisons leads to a chance, misidentification of statistical significance with a probability considerably greater than the commonly acceptable 5%. In this study, seven variables were measured at each of the two time points.

There are several acceptable post-hoc tests to correct for family-wise error rate, each with pros and cons, especially with regard to how conservative they are (that is, having the effect of decreasing study power).⁵ A simple post-hoc test, with an intermediate level of conservativeness is the sequentially rejective Bonferroni–Holm (BH) correction.⁵ In this method, variables within each data set are ordered by increasing P-value; beginning from the variable with the smallest P-value, the cutoff for statistical significance is equal to:

All comparisons following the first non-significant P-value using this method are considered non-significant. Applying this post-hoc test to the results of Choi et al. (separately, for each of the two time points) would require the smallest P-value to be <0.007 in order to be statistically significant, while maintaining an alpha-error 5%. As none of the univariate comparisons had such a small P-value, none of the observed changes in sexual function scores reach statistical significance when using the BH correction.

In conclusion, the evidence to support the claim of Choi et al. that KGBE significantly improves all measured aspects of male sexual function is far from convincing, with only the PEDT score surpassing the effect of placebo at 8 weeks, in the initial statistical analysis. Furthermore, although the choice of post-hoc family-wise error rate correction is debatable, ignoring this issue altogether results in a considerable chance of false-positive results. Regardless of whether a clinical trial assesses a naturally occurring supplement or a synthetic drug, it is important for study conclusions to reflect the actual results rather than being unrealistically optimistic, as the latter simply leads to confusion and an undermining of clinical research in general.