Comment on ‘Bevacizumab vs ranibizumab for age-related macular degeneration: 1-year outcomes of a prospective, double-masked randomised clinical trial’

Sir,

Subramanian et al.¹ carried out a trial comparing bevacizumab and ranibizumab for treatment of age-related macular degeneration. They concluded that 1-year outcomes failed to show a difference.

They designed their study to detect a difference, although they do not specify a hypothesis that one treatment is superior; arguably, it should have been designed as an equivalence trial or a non-inferiority trial, which would require an appreciably larger sample size. Regardless, they calculated that they would require 135 patients to detect a ‘real, moderate effect size difference’. Although not specified, ‘moderate’ is presumably 0.5 standard deviations, but they fail to discuss whether such a difference between the two groups might plausibly exist.

The authors do not justify using an unequal (2 : 1) allocation to the two groups. Statistically, unequal allocation is suboptimal and leads to a larger sample-size requirement, so one would expect to see the reasons fully explained.

Only 22 patients completed the trial. This is far too few to detect any plausible difference that might exist. With this sample size, the power to detect a moderate difference of half a standard deviation is about 15%, a totally unacceptable level. The type II (false negative) error probability is 0.85, and thus a negative result is far more likely than not to be reported. Inevitably, any realistic difference would fail to be detected. There is not even sufficient power to reliably detect an implausibly large difference of 1.5 standard deviations. The statement that a ‘Two-tailed t-test failed to showed statistical significance between the two groups (P=0.5)’ is worthless.

The best way to convey the results of this small study is by using a confidence interval. The authors write ‘The difference in visual acuity change from baseline to 1-year follow-up between the two groups was 1.3±14.9 (95% confidence interval 0.64–15.5)’. This is implausible, as a confidence interval corresponding to a t-test will be symmetrical about the mean. Presumably the true interval is approximately −13 to +15. All we can conclude is that bevacizumab may well result in a benefit as large as about 15 letters, or in a disadvantage almost equally as large of about 13 letters.

Although RCTs are to be encouraged, the conclusions stated in this paper are very misleading, yet could influence the opinions of a casual reader in this important area within ophthalmology.