Introduction

Geographic atrophy (GA) is an advanced manifestation of age-related macular degeneration (AMD)—the leading cause of severe vision loss for people over 55 years old in the industrialised world.1 Geographic atrophy accounts for ~20% of legal blindness registration in North America.1 It is associated with formation of large, confluent drusen, degeneration of retinal epithelium and loss of photoreceptors eventually leading to irreversible sight loss.1 Although some progress has been made to understand the pathophysiology of GA and there is no treatment to prevent progression of GA,1 a number of randomised controlled trials (RCTs) testing potential interventions have taken place.2

RCTs are considered the most powerful tools when determining the efficacy of interventions.3 However, poor planning of RCTs may result in substandard studies. To enable readers to access methodological rigour of an RCT, reports of trials should include transparent descriptions of key information.4 One of the critical aspects for planning an RCT is the selection of appropriate primary and secondary outcomes.5 It has been recognised that there is a large variability in outcome selection in RCTs for different diseases.6 This variability results in difficulties in synthesising medical evidence7 and a higher risk of outcome reporting bias.8 Standardisation of outcomes solves this problem by creating a core outcome set (COS) for a specific disease. This group of outcomes, which should be reported in each interventional trial within a particular area, would reduce inconsistencies in outcome reporting between studies and improve design of RCTs.9 This concept has been adapted in several fields of medicine such as rheumatology, gastroenterology, and paediatrics.10

The selection of outcomes in an RCT has also critical implications for the sample size calculation. Sample size estimation is typically based upon the primary outcome. The sample size should be suitable to answer with enough power the research question regarding efficacy of an intervention. It also ought to constitute a guarantee that a study is informative.11 However, insufficient attention has been paid to sample size estimations in some areas of vision research.12

Despite dry AMD being the most prevalent form of AMD, such major breakthrough as anti-VEGF (vascular endothelial growth factor) drugs for neovascular AMD has not been yet introduced.2 Potential GA therapeutic options have just stepped in the phase of clinical randomised trials.2 Thus, it gives a good opportunity to reflect on what were the main outcomes selected and how the sample size was determined in recent GA trials.

The aim of this paper is to identify the variety of outcomes and reporting of sample size calculations within GA trials.

Materials and methods

In our systematic review, we adhered to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines (Transparent Reporting in Systematic Reviews and Meta-analyses, www.prisma-statement.org). We registered the systematic review in PROSPERO (International prospective register of systematic reviews, www.crd.york.ac.uk/PROSPERO/) with registration number: CRD42014010040.

Search strategy

A systematic review was conducted because of recent developments in AMD area of research. We limited our search to RCTs published after January 2010. The last search date was November 2013. The following databases were searched: Ovid MEDLINE Full, Ovid EMBASE, SCOPUS, and Cochrane Library. We ran a modified sensitive search strategy with subject subheadings and text terms. In order to find eligible abstracts, we examined two types of terms: AMD or AMD synonyms (for instance: ARMD, maculae senilis) and RCTs and words associated with RCTs (eg, randomly, single-blind, double-blind study). An ‘AND’ operator was used to merge the results from each area. The search strategies from the different electronic databases are presented in Supplementary Appendix 1. We exported and combined the yielded abstracts in Refworks. In addition, the clinical trials’ registries (www.controlled-trials.com and www.clinicaltrials.gov) were searched. A keyword ‘macular degeneration’ limited to the same time period was used to attain the required trials’ records.

Inclusion and exclusion criteria

Two investigators independently screened all abstracts. We accepted only RCTs on AMD published in English. RCTs from references and appendices of Cochrane Systematic Reviews were also searched and identified. All records were grouped as ‘include’, ‘uncertain’, or ‘exclude’. We discussed any inconsistencies between the reviewers in categorising records to reach an agreement. Full articles of those classified as ‘Uncertain’ and ‘include’ were obtained and then a final decision of inclusion was made.

Data extraction strategy

Data were captured using online forms (www.maculaesenilis.com), specifically designed for this project. The information on the type of AMD, outcome (clinical, patient reported outcome, safety, or other) and the category of outcome (primary or secondary) was collected by one investigator. Information on sample size calculation and population of identified RCTs was also congregated. One tenth of included RCTs was selected at random and evaluated by a masked second investigator to evaluate agreement of data extracted.

On-going RCTs and those from appendices of Cochrane Reviews did not have information on the sample size calculation.

Results

Our search identified 3816 abstracts. From the abstracts, 103 RCTs were included. The additional trials’ registries search found 493 records. Of these, 74 RCTs were included. A combination of both searches resulted in a total number of 177 RCTs of all AMD types. The number of identified GA trials was 23. (Figure 1, PRISMA flow chart). Of these, eight had published results, including study design, while information on 15 trials was obtained from the trial registries and appendices of Cochrane Systematic Reviews. Masked investigators agreed on identification of the primary outcomes measures in 100% of articles assessed. However, there was one RCT, where the first reviewer recognised outcome information as one primary outcome, while the second one assessed it as two outcomes—one primary and ‘other primary outcome’. This discrepancy was resolved after discussion when reviewers agreed to classify retrieved information as one primary outcome.

Figure 1
figure 1

Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram illustrating the process of randomised controlled trials (RCTs) selection. Asterisk denotes additional potential RCTs from references of the articles and appendixes of the Cochrane Systematic Reviews.

Full size image

Primary outcomes were identified in 21 GA RCTs (91.3%), whereas in two RCTs (from appendices of Cochrane Systematic Reviews), they were not clearly stated. Three studies reported more than one primary outcome. In two trials (from the trial registries), two primary outcomes were identified—‘original primary’ and ‘current primary’—and in one trial, different biomarkers were used as main outcomes.

In six RCTs, lesion area and visual acuity (VA) were used as a primary outcome (26.1%). It was followed by fundus autofluorescence (five RCTs, 21.74%). There was variability in the way that the primary outcomes were measured (Table 1).

Table 1 Primary outcomes in geographic atrophy randomised controlled trials
Full size table

Overall, a total of 81 clinical outcomes (including primary and secondary) were reported (71.7% of all 113 GA outcomes). VA was the most common outcome (reported 24 times, 29.6% of all clinical outcomes). It was used in 18 GA RCTs (78.3%). There was heterogeneity in the methods used to analyse and report visual acuity in GA trials (some examples are shown in Table 1).

The second most common type of outcome was lesion area and this was reported in eight RCTs (34.8%). Anatomical outcomes were also described in a variety of ways depending on the imaging modality or method of assessment used. In general, GA RCTs focused on drusen and geographic atrophy lesion size (Table 2). Autofluorescence, fundus photography, and OCT measures were the most frequently used methods of anatomical changes assessment.

Table 2 Descriptions of lesion area in geographic atrophy randomised controlled trials
Full size table

Detailed information on the most common types of clinical outcomes and their frequency in GA trials is presented in Table 3.

Table 3 Types and frequency of clinical outcomes (primary and secondary) in geographic atrophy randomised controlled trials
Full size table

Patient reported outcomes were used in two GA trials. The questionnaires used were the Visual Functioning Questionnaire-14 (VFQ-14) and the National Eye Institute VFQ-25.

Regarding sample size calculation, of the eight GA RCTs from the database search with details of study design, none of the trials included relevant information according to the CONSORT statement. In one trial, information on sample size was limited to the statement that Lehr formula was used to determine a sample size. Another trial described that target number of participants (covered by the grant) was reached.

Discussion

A selection of appropriate outcomes is a critical step in the design of an RCT. To facilitate interpretation of results and comparisons between studies, interventional trials on the same condition should share a standard component known as COS.7 This systematic review shows that there is currently no standardisation in the selection of outcomes in GA RCTs.

Although effective treatments for GA remain elusive, various promising therapeutics for GA are currently in the pipeline and in the years to come more and more trials for GA are likely to be instigated.2 Visual function outcomes other than visual acuity, such as contrast sensitivity or dark adaptation,13 have been considered for use in GA RCTs, as VA measures in isolation may be insensitive to slow progression of GA.13 Progression, anatomic features, size of atrophy were among the most commonly used outcomes for GA.

Other scientists and clinicians have acknowledged this issue of optimal outcome selection and trial design.14 Our study also demonstrates that a broad diversity exists in the methods of reporting outcomes in GA RCTs.15, 16 The most frequently used outcome in GA RCTs was VA, yet it differed widely in what metric was used as the focus of the data analysis. Some studies used the change in number of letters read, others determined the proportion of participants achieving a particular criterion such as a specific number of lines gained or lost. This diversity in how VA data are analysed is likely to make comparisons between different trials problematic. To help in planning evidence synthesis and analysis of visual acuity measures, attention should be drawn to the association between different ways of reporting VA measures.17

Despite the fact that AMD is known as a disease that causes both, visual impairment, and decrease in quality of life associated with limitations in everyday living and depression,18 we have observed that only two GA RCTs have used patient-reported outcome measures (PROMs). Ideally outcomes relevant to patients and PROMs should be used in all AMD trials.19

The unambiguous selection of relevant outcomes during the design of interventional trials is essential, because it allows health care providers to compare scientific evidence and then make well-informed decisions about treatment,20 which in turn enables the efficient development of healthcare systems.9 A useful aid for researchers to select appropriate outcomes for their RCTs is creating a COS. The COMET initiative has popularised this idea since 2010.7 This must-include minimum set of outcomes for RCTs on a particular condition would reduce heterogeneity in outcome reporting, facilitate comparisons between studies, and decrease the level of outcome reporting bias.7, 9 Previous studies have confirmed the need for COS in such chronic ophthalmologic condition as glaucoma,21 whereas our review verifies the need for consensus in outcome selection and type of measurement for GA AMD trials.

The need for accurate sample size calculations has been increasingly acknowledged in recent years12 as it allows achieving power sufficient to answer the main research question of a study22 and promotes meaningful research that does not waste resources or expose participants to potentially harmful interventions without advancing knowledge.23 Yet, our review has shown that information on sample size calculation in GA RCTs is limited. This deficiency in sample size calculation may impair interpretation of study results. Failure in reaching planned sample size makes results of a study less reliable and delays implementation of results in clinical practice. Determination of statistical significance that a study is designed to detect is another key consideration of sample size calculation, as incorrectly specified target difference may impair decision making.11 To avoid that, a method tailored to the aim, context, and statistical framework of a study should be implemented to define a target difference.24 Taking into account the importance of sample size calculation and the relevant information collected on GA RCTs, we confirm the observation that some vision research has struggled with this area.12 However, we are aware that some trials are designed to generate the pilot data, which may be used to inform sample size calculation in larger and definitive RCTs.

Overall, reporting of ophthalmological RCTs has improved since the CONSORT (Consolidated Standards for Reporting Trials) statement was introduced.25 The CONSORT statement includes items regarding a title, an introduction, methods, results, discussion, and others to guide reporting of RCTs.4 Our interest in primary/secondary outcome selection and sample size are important elements of the methods section of the CONSORT statement. In this review, we have observed that primary outcomes for dry AMD trials were clearly stated and there was a range of secondary outcomes described. However, the observed variability of time, metrics, and the way of assessment of outcomes would require further attention. Regarding sample size calculation, improvements of reporting of this aspect are needed to improve standard of publications and research.

Our review was prepared according to PRISMA recommendations (www.prisma-statement.org) in a systematic way and it includes RCTs on GA population. A potential limitation is that we took into consideration only published and on-going RCTs and those within a limited time frame. It is also plausible that some eligible RCTs were not included as we did not seek the ‘grey literature’ and we limited our search to English language only. However, our approach was sufficient to notice deficiencies and variability of current GA trials’ design. The development of a COS for the area of geographic atrophy is a great opportunity to improve standards of RCTs planning and reporting, and facilitate direct comparisons between interventions.

In summary, our systematic review clearly demonstrated outcomes and ways of measuring them used in the reporting of GA RCTs vary widely. We recommend that the ophthalmic community should focus on developing a COS for future AMD RCTs with potentially separate sets developed for neovascular AMD, GA, and early AMD. Through our review, we also wish to draw researchers’ attention to the need for explicitly stating the a priori sample size calculation in the reporting of GA RCTs.