Determinants of visual acuity outcomes in eyes with neovascular AMD treated with anti-VEGF agents: an instrumental variable analysis of the AURA study

Purpose To identify the strongest variable(s) linked with the number of ranibizumab injections and outcomes in AURA, and to identify ways to improve outcomes using this association. Methods AURA was a large observational study that monitored visual acuity over a 2-year period in patients with neovascular age-related macular degeneration (AMD) who received ranibizumab injections. Baseline characteristics, resource use, and outcomes were analyzed using an instrumental variable approach and regression analysis. Results Data were analyzed from 2227 patients enrolled in AURA. Optical coherence tomography (OCT) and ophthalmoscopy were the most common diagnostic tests used, and this combination was the strongest instrumental variable. Use of OCT and ophthalmoscopy affected the number of injections given and resulted in an increase in visual acuity gains from baseline of 17.6 letters in year 1 and 2.5 letters in year 2. Regression models using the instrumental variable (OCT and ophthalmoscopy combined) showed that ≥5.1 (95% CI: 3.3–11.4) ranibizumab injections were needed to maintain visual acuity from baseline to year 1 and ≥8.3 (95% CI: 5.3–18.8) injections were needed to maintain visual acuity from year 1 to year 2. To gain ≥15 letters, ≥7.9 (95% CI: 5.1–17.5) ranibizumab injections would be needed in year 1 and ≥16.1 (95% CI: 10.3–36.4) injections would be needed over 2 years. Conclusions These findings highlight the role that regular monitoring plays in guiding neovascular AMD therapy and they showed that the number of ranibizumab injections needed to maintain visual acuity is higher than that administered in AURA.


Introduction
Anti-vascular endothelial growth factor (VEGF) agents have become an important treatment option for neovascular age-related macular degeneration (AMD) since their introduction over a decade ago. These agents inhibit VEGF, a key factor in the development of underlying cell proliferation and neovascularization. 1 Improvements in visual and anatomical outcomes following monthly injections of the anti-VEGF antibody fragment ranibizumab have been demonstrated in two key studies in neovascular AMD. 2,3 However, delivery of monthly dosing in clinical practice is challenging, and alternative dosing regimens of intravitreal ranibizumab are often used, including as-needed, quarterly, or treat-andextend, although outcomes can be more variable. [4][5][6][7][8][9][10] Not surprisingly, observational studies have shown that ranibizumab may be underused in routine practices, resulting in poor long-term outcomes. [11][12][13] AURA (a retrospective noninterventional study to assess the effectiveness of existing Anti-vascUlar endothelial growth factor treatment Regimens in patients with wet Age-related macular degeneration) monitored 2-year outcomes in patients with neovascular AMD who started treatment with ranibizumab between January 2009 and August 2009. 11 This observational study showed that visual acuity gains were not maintained over 2 years, and the mean number of injections was low. Initially, we performed logistical regression analysis of AURA, and found that inadequate monitoring, injections, and use of diagnostic tools compromised treatment outcomes. 12 To further determine the relationship between ranibizumab injections and outcome in a nonrandomized setting, such as AURA, we also performed an instrumental variable analysis of this data set. This approach is validated and well established, and is used to identify causal relationships in a noncontrolled situation that is often subject to bias and confounding from both measured and nonmeasured variables. Notably, the instrumental variable method has the potential to adjust for these confounders, making it ideal for evaluating observational data. [14][15][16][17][18][19][20] In this case, we identified the (instrumental) variable from the AURA data set (ie, baseline characteristics and resource use) that had the strongest causal link with number of ranibizumab injections and outcome (mean change in visual acuity (letters) at year 1 and year 2). Any confounding between the instrumental variable identified and the outcome was tested through application of the Wald estimator (which statistically tests the true value of the parameter based on the sample estimate). To test whether the selection was random, the F-statistic was also applied; this test identifies the level of bias in the sample. The outcomes from the instrumental variable analysis of the AURA data are reported in this paper.

Materials and methods
AURA was a retrospective, observational, multicenter study conducted in eight countries (Canada, France, Germany, Ireland, Italy, the Netherlands, the United Kingdom, and Venezuela). The design has been described in detail elsewhere. 11,12 In brief, its primary aim was to evaluate changes in visual acuity in patients who started ranibizumab therapy between January 2009 and August 2009. The overall (exposed) population consisted of those who received at least one dose of anti-VEGF treatment, whereas the effectiveness analysis set consisted of patients who additionally had at least one post-baseline assessment of visual acuity for the treated eye. The firstyear and second-year completer analysis sets included those in the effectiveness analysis set for whom follow-up data for Z 1 and Z 2 years after first injection, respectively, were documented. Because of the exploratory nature of the study, the statistical analysis was descriptive. To account for missing data, mean change in visual acuity was assessed using a lastobservation-carried-forward analysis.

Objectives
The aim of this paper is to apply an instrumental variable analysis to the AURA data set in order to identify the strongest instrumental variable(s) linked with the number of ranibizumab injections and outcomes (mean change in visual acuity (letters) at year 1 and year 2), and to identify ways to improve visual acuity outcomes using this association.

Analyses
The analyses were performed using StataCorp LP 2007 (Stata Statistical Software: Release 10, College Station, TX, USA). The AURA data set and definitions used in this paper have been described elsewhere. 12 The baseline characteristics, health insurance, reimbursement, and resource utilization (in year 1 and over 2 years) by country are summarized in this paper. Resource utilization included number of ophthalmoscopies, optical coherence tomography (OCT) images, fluorescein angiographies, and indocyanine green angiographies. Monitoring (diagnostic assessments only) and clinic (scheduled treatment) visits were also reported. Any differences between country data were analyzed using either ANOVA (for continuous variables that were normally distributed) or χ 2 test (for categorical variables).
Given the volume of variables included in AURA, Pearson's correlation coefficient (95% confidence interval (CI)) was initially used to test the strength of correlation between variables and number of ranibizumab injections. The instrumental variable analysis (summarized in Supplementary Figure 1) that was applied to the candidate variables identified has been described in numerous publications. 14-20 A standard approach was used; that is, we applied a series of linear equations (listed below). Using the F-statistic (to test for random assignment), the variable with the strongest correlation to the number of ranibizumab injections was selected as the instrumental variable. The Wald estimator was also used (to evaluate the validity of the instrumental variable in relation to the number of injections and outcomes). The instrumental variable approach was then applied to test the relationship between the number of ranibizumab injections and outcomes using regression analysis. The analysis was developed using known assumptions (described below).
Testing random assignment The relationship between the instrumental variable (ie, a variable with high correlation with the exposure but low correlation with the outcome) and exposure (the number of ranibizumab injections) should not be confounded by other variables. The F-statistic of the regression (equation (1)) estimates the magnitude of bias in terms of confounding by other variables. 17 In equation (1) r̂2 Z;X refers to the square of the estimated correlation coefficient between the instrumental variable (Z; 1 × n) and the exposure (X; 1 × n) and n is the sample size. An F-value not far from 1 indicates a high risk of small sample bias, whereas a value of 10 seems to be sufficient for the bias to be negligible: 18 Evaluating the validity of the instrumental variable in relation to the number of injections and outcomes The Wald estimator was used to evaluate the overall validity of the instrumental variable. 14, 17 The Wald estimator statistically tests the true value of the parameter based on the sample estimate. In the case of a dichotomous instrument and exposure, the Wald estimator is given by equation (2). The numerator of this estimator is an intention-to-treat estimator (ie, the effect of the instrument on the outcome measured as a risk difference). The denominator is the difference in treatment rates between levels of the instruments (eg, treatment arms of the randomized controlled trial) and is a measure of compliance. As the noncompliance increases, the denominator shrinks and the instrumental variable estimator increases relative to the intention-to-treat estimator.
where X = exposure, Y = outcome, and Z = instrumental variable. Regression models using the validated instrumental variable were developed; this method has been applied in other studies. 21,22 In brief, a two-stage least-squares approach was used to calculate the instrumental variable estimates. In the first stage, each explanatory variable that is an endogenous covariate in the equation of interest is regressed on all of the exogenous variables in the model, including both exogenous covariates in the equation of interest and the excluded instrumental variable. The predicted values from these regressions are obtained using the following equations (equations (3) and (4)), where Z (column vector with n objects) and X (matrix with all covariates for n objects) are the instrumental variable and the covariates, respectively; dincludes the estimated regression coefficients, T is the letter indicating the transpose of the vector, and X̂contains the predicted X values.
Stage 1: regress each column of X on Z: and save the predicted values: In the second stage, the regression of interest is estimated as usual, except that in this stage each endogenous covariate is replaced with the predicted values from the first stage based on the following equations (equations (5) and (6)), where Y is the outcome, X̂is the predicted X, β are the regression coefficients, and φ is the error. Stage 2: regress Y on the predicted values from the first stage: which gives the two-stage least-square estimator β 2SLS , Results

Participants
In the AURA study, 2227 patients were included in the effectiveness analysis set, and 1695 patients completed year 1 and 1184 patients completed year 2. Baseline characteristics for the effectiveness analysis set are shown in Table 1a. There were statistically significant differences between countries in potential confounding variables such as mean age at start of therapy that ranged from 73 years (Ireland and Venezuela) to 80 years (Canada); health insurance (public health insurance ranged from 99% in the Netherlands to 2% in Ireland); reimbursement (national standard was achieved by 96% in the Netherlands and Ireland, and by 15% in Venezuela); and mean baseline visual acuity score (VAS) that ranged from 47 letters in Canada to 66 letters in Italy. Resource utilization by country is summarized in Table 1b. There were statistically significant differences between countries in the number of ranibizumab injections, diagnostic tests, and visits. In year 1, the mean number of ranibizumab injections ranged from 3.4 (Venezuela) to 7.4 (Ireland), and clinic visits ranged from 3.8 (Venezuela) to 8.1 (Ireland). The mean number of OCT examinations ranged from 2.5 (Venezuela) to 9.3 (United Kingdom) in year 1, and from 5.  Abbreviations: ICGA, indocyanine green angiography; VAS, visual acuity score. Mean unless stated. a P-value was derived from ANOVA test for continuous variables and χ 2 test for categorical variables. b Individual: reimbursement granted individually for this patient and treatment occasion; national standards: reimbursement according to national standards; independently: reimbursement granted for patient independently of how often treatment was provided. c Monitoring (diagnostic assessments only) and clinic (scheduled treatment) visits. As this was an observational study, the actual number of treatments may differ to visits that were scheduled as clinic (treatment) visits, and the use of diagnostic tests, such as ophthalmoscopies, may have also occurred during clinic or monitoring visits, accounting for differences in numbers. d Vision gainer (defined as patients who gained ⩾ 15 letters) and vision maintained (defined as no decline in visual acuity from baseline).

Identification of candidate instrumental variables
The Pearson's correlation coefficient was used as the primary criterion to test the correlation between the number of ranibizumab injections and candidate instrumental variables (Table 2). Treatment duration was the variable most strongly correlated with number of injections, with a correlation coefficient of 0.6; however, as treatment duration can be related to the outcome, it was not chosen as the instrumental variable. The number of OCT and ophthalmoscopy examinations performed was also correlated with the number of injections; the correlation coefficients were 0.4 and 0.3, respectively. According to clinical guidelines and published literature, OCT is recommended for screening the macula before performing more invasive imaging such as fluorescein angiography. [23][24][25] OCT alone may be able to provide sufficient information to facilitate follow-up decisions, and is considered the essential diagnostic and monitoring tool in the treatment of neovascular AMD. The number of OCTs performed is, therefore, the most clinically relevant instrumental variable. At the time of this study, OCT was the most frequently used diagnostic and monitoring tool in France, the United Kingdom, Ireland, Italy, the Netherlands, and Venezuela, and ophthalmoscopy was the dominant diagnostic and monitoring tool in Germany and Canada (Table 1b). Therefore, a combination of OCT and ophthalmoscopy was used as the most appropriate instrumental variable across countries, resulting in a correlation coefficient of 0.4 ( Table 2).
The suitability of using OCT and ophthalmoscopy as an instrumental variable was further evaluated using the Wald estimator. As the Wald estimator commonly uses a binominal variable for the instrumental variable in the equation (equation (2)), the number of OCT and ophthalmoscopy examinations was dichotomously grouped by the median number of OCT and ophthalmoscopy examinations performed at year 1, and over the 2-year period. The variables 0 and 1 were used to represent o8 and Z 8 OCT and ophthalmoscopy examinations in year 1, and o19 and Z 19 OCT and ophthalmoscopy examinations over the 2-year period. In this equation, the exposure was defined as the number of ranibizumab injections and the outcome was the number of letters gained. This analysis showed that a higher number of OCT and ophthalmoscopy examinations affected the number of injections and resulted in an increase from baseline of 17.6 letters in year 1 and 2.5 letters in year 2. When the Wald estimator was used in relation to the number of ranibizumab injections and letters gained from baseline, the effect of the number of OCT and ophthalmoscopy examinations was 13.9 letters in year 1 and 2.4 letters in year 2. Given that the clinical significance for letters gained in visual acuity is ± 15 letters, the Wald estimates suggest that the effect of the instrumental variable on letters gained in visual acuity is significant in year 1 but not in year 2.
The regression models using the instrumental variable (OCT and ophthalmoscopy combined) are presented in Table 3a and b. For both analyses, the values for the F-statistics were 410 in the first-stage model, indicating that the regression model was stable. The regression model also indicates that the average patient aged 76.9 years with a baseline VAS of 56.9 letters would lose 27.8  11 These findings indicate that, at the time the AURA study was conducted, outcomes achieved in clinical studies were not translated into clinical practice.
In the Comparison of Age-Related Macular Degeneration Treatments Trials (CATT), patients treated with ranibizumab on an as-needed basis maintained gains in visual acuity (+6.7 letters at year 2) and received a mean of 12.6 injections over the 2-year period. 9 However, it should be noted that CATT was a clinical study with strict inclusion criteria and mandated monthly follow-up; OCT imaging was performed at each visit, and the study protocol described the retreatment criteria. AURA was an observational, retrospective study with no limit to visual acuity and other parameters at baseline. The current analysis used an instrumental variable to explore potential confounding parameters in this association, and identified that the number of OCT and ophthalmoscopy examinations performed was a strong instrumental variable when testing the relationship between number of injections and letters gained given the acceptable results from F-statistics and Wald estimates. By using the instrumental variable, regression analysis showed that the number of injections is significantly associated with letters gained in visual acuity in year 1 and year 2.
Based on the AURA outcomes, this analysis revealed that the average patient would need Z 5.1 ranibizumab injections to maintain VAS from baseline to year 1 and Z 8.3 ranibizumab injections to maintain VAS from year 1 to year 2. For the average patient to gain Z 15 letters at year 1, Z 7.9 ranibizumab injections would be needed. For the average patient to gain Z 15 letters at year 2, Z 16.1 ranibizumab injections would be needed. It should be noted that these results are only appropriate when either OCT or ophthalmoscopy is assessable during the course of treatment for patients with neovascular AMD.
The analysis has a number of strengths and limitations. An instrumental variable is a useful tool to identify causality where patients are not randomized 26 and providing that the data set is sufficiently large. Results are not reliable when the instrumental variable does not fulfil one of the required assumptions. 18 The AURA study, however, fulfills these requirements. If the instrumental variable is correlated with the outcome, generalizability of the overall results is weakened as the instrumental variable is supposed to provide 'randomized' effects. In our analysis, we removed treatment duration to avoid this, and showed that the number of OCT and Table 3 Relationship between number of ranibizumab injections and letter gains in year 1 (a) and over 2 years (b) tested by regression analysis using optical coherence tomography and ophthalmoscopy as an instrumental variable (all countries) An instrumental variable should be neither directly related to the outcome nor indirectly related via pathways through unmeasured variables. OCT and ophthalmoscopy are diagnostic tools and not treatments, and therefore there should be no direct relationship between the number of OCT and ophthalmoscopy examinations performed and letters gained in visual acuity. However, the use of OCT and ophthalmoscopy examinations as a diagnostic tool may indirectly influence treatment decisions and, thus, treatment outcomes. Indirect relation via pathways through unmeasured variables cannot be tested within the AURA data set. Another limitation is that AURA data on OCT and ophthalmoscopy use were collected during 2009, with a 2year follow-up. The use of diagnostic tools may have changed in recent years; in Germany, OCT, and not ophthalmoscopy, is now the most frequent diagnostic tool. It must also be noted that treat-and-extend approach has recently become a common approach, but this is unlikely to have been used at the time of the study. The data suggest that a low number of examinations with ophthalmoscopy or OCT imaging or both was used, resulting in a low number of injections and a worse visual acuity outcome. This implies that an as-needed approach was being employed at the time. Given that there has been a shift toward treat-and-extend or extended fixed dosing with increased access to OCT, it is possible that the situation may have improved since the results of AURA were published. However, these results are still applicable when an as-needed approach is preferred. There was also a lack of adjustment for any country-specific differences, including further exploration of reimbursement or health insurance on access to resource. Although access to public health insurance or private health insurance (specifically for Ireland and Venezuela) was evident in all countries, there was wide variation in the use of diagnostic tools and injections. There may be a number of reasons for this, including limitations in insurance coverage. In the United Kingdom, public insurance (National Health Service) is free to all permanent residents, but there are still charges associated with eye tests under some circumstances; this may have resulted in some patients recording 'no insurance' rather than free public insurance. Finally, only the overall baseline VAS was included as a confounding parameter, and this may have excluded any potential (and clinically relevant) association between higher baseline VAS, fewer injections, and less letter gains.
In conclusion, the instrumental variable method supports the association between the number of injections and letters gained in visual acuity, and also the role that regular monitoring plays in guiding therapy. The regression model showed that the number of ranibizumab injections needed to maintain or gain visual acuity is greater than the mean number of injections administered in the AURA study. This likely explains, at least in part, why the initial visual improvements achieved by patients in AURA were not sustained at year 2. These findings will be of benefit to health-care physicians with an interest in improving treatment decisions for patients with neovascular AMD.

Summary
What was known before K Improvements in visual and anatomical outcomes following the use of monthly injections of anti-VEGF agents have been shown in randomized studies.
K However, use of a monthly treatment regimen is logistically challenging to health-care providers for reasons mainly relating to resource implications.
K Observational studies, such as AURA, have shown that outcomes achieved in randomized studies are not translated into clinical practice, visual acuity gains are not maintained, and resource utilization (visits and injections) is low, with wide variations across countries.
What this study adds K This analysis identified determinants of visual acuity outcomes in an observational setting, using data from the AURA study.
K OCT and ophthalmoscopy were found to be the most common diagnostic tests used in AURA, and this combination was the strongest independent variable for testing the relationship between number of injections and letters gained.
K Regression analysis using the instrumental variable (OCT and ophthalmoscopy combined) showed that Z 7.9 ranibizumab injections would be needed to gain Z 15 letters in year 1, and Z 8.3 injections would be required to maintain visual acuity in year 2.
Stephen Beatty is a consultant and board member for Bayer Pharmaceuticals, and has received support for travel to meetings and fees for participation in review activities from Bayer Pharmaceuticals. Alan R Berger has received honoraria from Bayer Pharmaceuticals, Novartis, Alcon, and Allergan; institutional research funding support from Bayer Pharmaceuticals, Novartis, and Alcon; and fellowship support funding from Alcon, Bayer Pharmaceuticals, Novartis, and Woodgreen Pharmacy. He has received travel fees and speaker fees and participated in advisory boards for Bayer Pharmaceuticals and Novartis. Matteo G Cereda is a consultant to Bayer Pharmaceuticals and has received support for travel to meetings from Bayer Pharmaceuticals and Alcon. Philip Hykin has received grants from Bayer Pharmaceuticals, Allergan, and Novartis; and acts as a consultant to Bayer Pharmaceuticals, Allergan, and Novartis. He has also received support for travel to meetings, participation in review activities, and provision of writing assistance, medicines, equipment, or administrative support from Bayer Pharmaceuticals. He has received payment for lectures from Allergan and Novartis. Giovanni Staurenghi acts as a consultant to Novartis, Bayer Pharmaceuticals, Allergan, Genentech, Roche, Heidelberg Engineering, and Alcon. He has also received support for travel to meetings from Bayer Pharmaceuticals, Centervue, Heidelberg Engineering, and Novartis. He has received payment for lectures from Zeiss, and is a patent holder in conjunction with Ocular Instruments, Inc. He has received payment for development of educational presentations for Roche. Sobha Sivaprasad has received research grants from Bayer Pharmaceuticals, Novartis, and Allergan that were paid to her institution. She has received travel fees and speaker fees, and has participated in advisory boards for Bayer Pharmaceuticals, Novartis, and Allergan, and has participated in an advisory board meeting for Roche. Kim Wittrup-Jensen is an employee of Bayer Pharmaceuticals. Jonas Nilsson and Kun Kim are employees/former employees of Mapi Group who were contracted by Bayer Pharmaceuticals to perform the analyses.