Introduction

Anti-vascular endothelial growth factor (VEGF) agents have become an important treatment option for neovascular age-related macular degeneration (AMD) since their introduction over a decade ago. These agents inhibit VEGF, a key factor in the development of underlying cell proliferation and neovascularization.1 Improvements in visual and anatomical outcomes following monthly injections of the anti-VEGF antibody fragment ranibizumab have been demonstrated in two key studies in neovascular AMD.2, 3 However, delivery of monthly dosing in clinical practice is challenging, and alternative dosing regimens of intravitreal ranibizumab are often used, including as-needed, quarterly, or treat-and-extend, although outcomes can be more variable.4–10 Not surprisingly, observational studies have shown that ranibizumab may be underused in routine practices, resulting in poor long-term outcomes.11–13

AURA (a retrospective noninterventional study to assess the effectiveness of existing Anti-vascUlar endothelial growth factor treatment Regimens in patients with wet Age-related macular degeneration) monitored 2-year outcomes in patients with neovascular AMD who started treatment with ranibizumab between January 2009 and August 2009.11 This observational study showed that visual acuity gains were not maintained over 2 years, and the mean number of injections was low. Initially, we performed logistical regression analysis of AURA, and found that inadequate monitoring, injections, and use of diagnostic tools compromised treatment outcomes.12

To further determine the relationship between ranibizumab injections and outcome in a nonrandomized setting, such as AURA, we also performed an instrumental variable analysis of this data set. This approach is validated and well established, and is used to identify causal relationships in a noncontrolled situation that is often subject to bias and confounding from both measured and nonmeasured variables. Notably, the instrumental variable method has the potential to adjust for these confounders, making it ideal for evaluating observational data.14, 15, 16, 17, 18, 19, 20 In this case, we identified the (instrumental) variable from the AURA data set (ie, baseline characteristics and resource use) that had the strongest causal link with number of ranibizumab injections and outcome (mean change in visual acuity (letters) at year 1 and year 2). Any confounding between the instrumental variable identified and the outcome was tested through application of the Wald estimator (which statistically tests the true value of the parameter based on the sample estimate). To test whether the selection was random, the F-statistic was also applied; this test identifies the level of bias in the sample. The outcomes from the instrumental variable analysis of the AURA data are reported in this paper.

Materials and methods

AURA was a retrospective, observational, multicenter study conducted in eight countries (Canada, France, Germany, Ireland, Italy, the Netherlands, the United Kingdom, and Venezuela). The design has been described in detail elsewhere.11, 12 In brief, its primary aim was to evaluate changes in visual acuity in patients who started ranibizumab therapy between January 2009 and August 2009. The overall (exposed) population consisted of those who received at least one dose of anti-VEGF treatment, whereas the effectiveness analysis set consisted of patients who additionally had at least one post-baseline assessment of visual acuity for the treated eye. The first-year and second-year completer analysis sets included those in the effectiveness analysis set for whom follow-up data for ≥1 and ≥2 years after first injection, respectively, were documented. Because of the exploratory nature of the study, the statistical analysis was descriptive. To account for missing data, mean change in visual acuity was assessed using a last-observation-carried-forward analysis.

Objectives

The aim of this paper is to apply an instrumental variable analysis to the AURA data set in order to identify the strongest instrumental variable(s) linked with the number of ranibizumab injections and outcomes (mean change in visual acuity (letters) at year 1 and year 2), and to identify ways to improve visual acuity outcomes using this association.

Analyses

The analyses were performed using StataCorp LP 2007 (Stata Statistical Software: Release 10, College Station, TX, USA). The AURA data set and definitions used in this paper have been described elsewhere.12 The baseline characteristics, health insurance, reimbursement, and resource utilization (in year 1 and over 2 years) by country are summarized in this paper. Resource utilization included number of ophthalmoscopies, optical coherence tomography (OCT) images, fluorescein angiographies, and indocyanine green angiographies. Monitoring (diagnostic assessments only) and clinic (scheduled treatment) visits were also reported. Any differences between country data were analyzed using either ANOVA (for continuous variables that were normally distributed) or χ2 test (for categorical variables).

Given the volume of variables included in AURA, Pearson’s correlation coefficient (95% confidence interval (CI)) was initially used to test the strength of correlation between variables and number of ranibizumab injections. The instrumental variable analysis (summarized in Supplementary Figure 1) that was applied to the candidate variables identified has been described in numerous publications.14–20 A standard approach was used; that is, we applied a series of linear equations (listed below). Using the F-statistic (to test for random assignment), the variable with the strongest correlation to the number of ranibizumab injections was selected as the instrumental variable. The Wald estimator was also used (to evaluate the validity of the instrumental variable in relation to the number of injections and outcomes). The instrumental variable approach was then applied to test the relationship between the number of ranibizumab injections and outcomes using regression analysis. The analysis was developed using known assumptions (described below).

Testing random assignment

The relationship between the instrumental variable (ie, a variable with high correlation with the exposure but low correlation with the outcome) and exposure (the number of ranibizumab injections) should not be confounded by other variables. The F-statistic of the regression (equation (1)) estimates the magnitude of bias in terms of confounding by other variables.17 In equation (1) ρ̂Z,X2 refers to the square of the estimated correlation coefficient between the instrumental variable (Z; 1 × n) and the exposure (X; 1 × n) and n is the sample size. An F-value not far from 1 indicates a high risk of small sample bias, whereas a value of 10 seems to be sufficient for the bias to be negligible:18

Evaluating the validity of the instrumental variable in relation to the number of injections and outcomes

The Wald estimator was used to evaluate the overall validity of the instrumental variable.14, 17 The Wald estimator statistically tests the true value of the parameter based on the sample estimate. In the case of a dichotomous instrument and exposure, the Wald estimator is given by equation (2). The numerator of this estimator is an intention-to-treat estimator (ie, the effect of the instrument on the outcome measured as a risk difference). The denominator is the difference in treatment rates between levels of the instruments (eg, treatment arms of the randomized controlled trial) and is a measure of compliance. As the noncompliance increases, the denominator shrinks and the instrumental variable estimator increases relative to the intention-to-treat estimator.

where X=exposure, Y=outcome, and Z=instrumental variable.

Regression models using the validated instrumental variable were developed; this method has been applied in other studies.21, 22 In brief, a two-stage least-squares approach was used to calculate the instrumental variable estimates. In the first stage, each explanatory variable that is an endogenous covariate in the equation of interest is regressed on all of the exogenous variables in the model, including both exogenous covariates in the equation of interest and the excluded instrumental variable. The predicted values from these regressions are obtained using the following equations (equations (3) and (4)), where Z (column vector with n objects) and X (matrix with all covariates for n objects) are the instrumental variable and the covariates, respectively; δ̂ includes the estimated regression coefficients, T is the letter indicating the transpose of the vector, and X̂ contains the predicted X values.

Stage 1: regress each column of X on Z:

and save the predicted values:

In the second stage, the regression of interest is estimated as usual, except that in this stage each endogenous covariate is replaced with the predicted values from the first stage based on the following equations (equations (5) and (6)), where Y is the outcome, X̂ is the predicted X, β are the regression coefficients, and ϕ is the error.

Stage 2: regress Y on the predicted values from the first stage:

which gives the two-stage least-square estimator β2SLS,where Pz = Z(ZTZ)−1ZT:

Results

Participants

In the AURA study, 2227 patients were included in the effectiveness analysis set, and 1695 patients completed year 1 and 1184 patients completed year 2. Baseline characteristics for the effectiveness analysis set are shown in Table 1a. There were statistically significant differences between countries in potential confounding variables such as mean age at start of therapy that ranged from 73 years (Ireland and Venezuela) to 80 years (Canada); health insurance (public health insurance ranged from 99% in the Netherlands to 2% in Ireland); reimbursement (national standard was achieved by 96% in the Netherlands and Ireland, and by 15% in Venezuela); and mean baseline visual acuity score (VAS) that ranged from 47 letters in Canada to 66 letters in Italy.

Table 1 Baseline characteristics (a) and resource utilization and outcomes over 2 years (b) by country
Full size table

Resource utilization by country is summarized in Table 1b. There were statistically significant differences between countries in the number of ranibizumab injections, diagnostic tests, and visits. In year 1, the mean number of ranibizumab injections ranged from 3.4 (Venezuela) to 7.4 (Ireland), and clinic visits ranged from 3.8 (Venezuela) to 8.1 (Ireland). The mean number of OCT examinations ranged from 2.5 (Venezuela) to 9.3 (United Kingdom) in year 1, and from 5.2 (Venezuela) to 17.5 (United Kingdom) over 2 years. Spectral-domain OCT was used in 64.7% of first-year completers and 63.3% of second-year completers. Time domain OCT was used in 33.1% of first-year completers and 34.2% of second-year completers.

Identification of candidate instrumental variables

The Pearson’s correlation coefficient was used as the primary criterion to test the correlation between the number of ranibizumab injections and candidate instrumental variables (Table 2). Treatment duration was the variable most strongly correlated with number of injections, with a correlation coefficient of 0.6; however, as treatment duration can be related to the outcome, it was not chosen as the instrumental variable. The number of OCT and ophthalmoscopy examinations performed was also correlated with the number of injections; the correlation coefficients were 0.4 and 0.3, respectively.

Table 2 Correlation between number of ranibizumab injections and candidate variables (all countries; over 2 years)
Full size table

According to clinical guidelines and published literature, OCT is recommended for screening the macula before performing more invasive imaging such as fluorescein angiography.23, 24, 25 OCT alone may be able to provide sufficient information to facilitate follow-up decisions, and is considered the essential diagnostic and monitoring tool in the treatment of neovascular AMD. The number of OCTs performed is, therefore, the most clinically relevant instrumental variable. At the time of this study, OCT was the most frequently used diagnostic and monitoring tool in France, the United Kingdom, Ireland, Italy, the Netherlands, and Venezuela, and ophthalmoscopy was the dominant diagnostic and monitoring tool in Germany and Canada (Table 1b). Therefore, a combination of OCT and ophthalmoscopy was used as the most appropriate instrumental variable across countries, resulting in a correlation coefficient of 0.4 (Table 2).

The suitability of using OCT and ophthalmoscopy as an instrumental variable was further evaluated using the Wald estimator. As the Wald estimator commonly uses a binominal variable for the instrumental variable in the equation (equation (2)), the number of OCT and ophthalmoscopy examinations was dichotomously grouped by the median number of OCT and ophthalmoscopy examinations performed at year 1, and over the 2-year period. The variables 0 and 1 were used to represent <8 and ≥8 OCT and ophthalmoscopy examinations in year 1, and <19 and ≥19 OCT and ophthalmoscopy examinations over the 2-year period. In this equation, the exposure was defined as the number of ranibizumab injections and the outcome was the number of letters gained. This analysis showed that a higher number of OCT and ophthalmoscopy examinations affected the number of injections and resulted in an increase from baseline of 17.6 letters in year 1 and 2.5 letters in year 2. When the Wald estimator was used in relation to the number of ranibizumab injections and letters gained from baseline, the effect of the number of OCT and ophthalmoscopy examinations was 13.9 letters in year 1 and 2.4 letters in year 2. Given that the clinical significance for letters gained in visual acuity is ±15 letters, the Wald estimates suggest that the effect of the instrumental variable on letters gained in visual acuity is significant in year 1 but not in year 2.

The regression models using the instrumental variable (OCT and ophthalmoscopy combined) are presented in Table 3a and b. For both analyses, the values for the F-statistics were >10 in the first-stage model, indicating that the regression model was stable. The regression model also indicates that the average patient aged 76.9 years with a baseline VAS of 56.9 letters would lose 27.8 letters from baseline to year 1 without any ranibizumab injections (ie, –3.76 + (–0.279 × 56.9) + (–0.106 × 76.9)). At least 5.1 (95% CI: 3.3–11.4) ranibizumab injections would be needed to maintain visual acuity from baseline to year 1. Similarly, the average patient aged 77.0 years with a baseline VAS of 57.2 letters would lose 16.0 letters from year 1 to year 2 without any ranibizumab injections (ie, 19.953 + (–0.391 × 57.2) + (–0.177 × 77)). At least 8.3 (95% CI: 5.3–18.8) ranibizumab injections would be needed to maintain visual acuity from year 1 to year 2. To gain ≥15 letters, ≥7.9 (95% CI: 5.1–17.5) ranibizumab injections would be needed in year 1 and ≥16.1 (95% CI: 10.3–36.4) ranibizumab injections would be needed over 2 years.

Table 3 Relationship between number of ranibizumab injections and letter gains in year 1 (a) and over 2 years (b) tested by regression analysis using optical coherence tomography and ophthalmoscopy as an instrumental variable (all countries)
Full size table

Discussion

AURA monitored the routine use of ranibizumab injections for the treatment of neovascular AMD in clinical practices from eight countries. Overall, patients gained +2.4 letters in year 1 that declined to +0.6 letters in year 2. The mean number of injections was low, decreasing from 5.0 in year 1 to 2.2 in year 2. The mean number of OCTs performed also declined from 4.5 in year 1 to 3.2 in year 2.11 These findings indicate that, at the time the AURA study was conducted, outcomes achieved in clinical studies were not translated into clinical practice. In the Comparison of Age-Related Macular Degeneration Treatments Trials (CATT), patients treated with ranibizumab on an as-needed basis maintained gains in visual acuity (+6.7 letters at year 2) and received a mean of 12.6 injections over the 2-year period.9 However, it should be noted that CATT was a clinical study with strict inclusion criteria and mandated monthly follow-up; OCT imaging was performed at each visit, and the study protocol described the retreatment criteria. AURA was an observational, retrospective study with no limit to visual acuity and other parameters at baseline.

The current analysis used an instrumental variable to explore potential confounding parameters in this association, and identified that the number of OCT and ophthalmoscopy examinations performed was a strong instrumental variable when testing the relationship between number of injections and letters gained given the acceptable results from F-statistics and Wald estimates. By using the instrumental variable, regression analysis showed that the number of injections is significantly associated with letters gained in visual acuity in year 1 and year 2.

Based on the AURA outcomes, this analysis revealed that the average patient would need ≥5.1 ranibizumab injections to maintain VAS from baseline to year 1 and ≥8.3 ranibizumab injections to maintain VAS from year 1 to year 2. For the average patient to gain ≥15 letters at year 1, ≥7.9 ranibizumab injections would be needed. For the average patient to gain ≥15 letters at year 2, ≥16.1 ranibizumab injections would be needed. It should be noted that these results are only appropriate when either OCT or ophthalmoscopy is assessable during the course of treatment for patients with neovascular AMD.

The analysis has a number of strengths and limitations. An instrumental variable is a useful tool to identify causality where patients are not randomized26 and providing that the data set is sufficiently large. Results are not reliable when the instrumental variable does not fulfil one of the required assumptions.18 The AURA study, however, fulfills these requirements. If the instrumental variable is correlated with the outcome, generalizability of the overall results is weakened as the instrumental variable is supposed to provide ‘randomized’ effects. In our analysis, we removed treatment duration to avoid this, and showed that the number of OCT and ophthalmoscopy examinations combined was the most suitable candidate for an instrumental variable, and that the correlation with the number of injections was strong.

An instrumental variable should be neither directly related to the outcome nor indirectly related via pathways through unmeasured variables. OCT and ophthalmoscopy are diagnostic tools and not treatments, and therefore there should be no direct relationship between the number of OCT and ophthalmoscopy examinations performed and letters gained in visual acuity. However, the use of OCT and ophthalmoscopy examinations as a diagnostic tool may indirectly influence treatment decisions and, thus, treatment outcomes. Indirect relation via pathways through unmeasured variables cannot be tested within the AURA data set. Another limitation is that AURA data on OCT and ophthalmoscopy use were collected during 2009, with a 2-year follow-up. The use of diagnostic tools may have changed in recent years; in Germany, OCT, and not ophthalmoscopy, is now the most frequent diagnostic tool. It must also be noted that treat-and-extend approach has recently become a common approach, but this is unlikely to have been used at the time of the study. The data suggest that a low number of examinations with ophthalmoscopy or OCT imaging or both was used, resulting in a low number of injections and a worse visual acuity outcome. This implies that an as-needed approach was being employed at the time. Given that there has been a shift toward treat-and-extend or extended fixed dosing with increased access to OCT, it is possible that the situation may have improved since the results of AURA were published. However, these results are still applicable when an as-needed approach is preferred. There was also a lack of adjustment for any country-specific differences, including further exploration of reimbursement or health insurance on access to resource. Although access to public health insurance or private health insurance (specifically for Ireland and Venezuela) was evident in all countries, there was wide variation in the use of diagnostic tools and injections. There may be a number of reasons for this, including limitations in insurance coverage. In the United Kingdom, public insurance (National Health Service) is free to all permanent residents, but there are still charges associated with eye tests under some circumstances; this may have resulted in some patients recording ‘no insurance’ rather than free public insurance. Finally, only the overall baseline VAS was included as a confounding parameter, and this may have excluded any potential (and clinically relevant) association between higher baseline VAS, fewer injections, and less letter gains.

In conclusion, the instrumental variable method supports the association between the number of injections and letters gained in visual acuity, and also the role that regular monitoring plays in guiding therapy. The regression model showed that the number of ranibizumab injections needed to maintain or gain visual acuity is greater than the mean number of injections administered in the AURA study. This likely explains, at least in part, why the initial visual improvements achieved by patients in AURA were not sustained at year 2. These findings will be of benefit to health-care physicians with an interest in improving treatment decisions for patients with neovascular AMD.