Introduction

Wet age-related macular degeneration (neovascular AMD, nAMD) is the leading cause of vision loss in people aged ≥50 years [1]. Current treatment for nAMD is intravitreal injections of anti-angiogenic drugs that inhibit vascular endothelial growth factor (anti-VEGF antibodies). Treatment starts with a loading phase of 3 injections at intervals of 4–6 weeks; patients then enter a maintenance phase and are reassessed at each subsequent hospital monitoring visit to determine lesion activity. The two main treatment regimens are pro re nata (prn; treatment as required) and treat-and-extend (prophylactic injections and increasing the time interval between visits up to 3 months). The frequency of monitoring depends on the drug prescribed and the treatment regimen. Lesion status can become inactive in a proportion of eyes during the maintenance phase, which are then treated prn or by treat-and-extend. Relapse is common with prn treatment and can still occur with a treat-and-extend regimen.

Monitoring visits use a combination of best-corrected visual acuity (BCVA), clinical biomicroscopic examination and optical coherence tomography (OCT) or other retinal imaging modalities to determine if a lesion is active (wet) or inactive (dry). Patients may have many months without requiring treatment and regular monitoring visits place a significant burden on NHS Hospital Eye Service (HES) out-patient clinics [2]. Self-monitoring at home (“home-monitoring”) for patients with inactive disease would be more convenient and less costly for both patients and the NHS. If self-monitoring were to identify reactivation accurately, HES clinic appointments could be given only given to patients likely to require treatment or to newly referred patients, enabling prompt initiation of treatment.

Home-monitoring with internet-enabled devices is possible due to development of applications for self-monitoring visual function in nAMD [3]. Such applications allow automatic transmission of test results for review without the need for patients to interpret test results [4, 5]. However, reactivation of nAMD is difficult to detect because inactive lesions nevertheless cause some degree of visual disability. Therefore, any test must detect an increase or change in visual disability, not just the presence of disability. Technologies such as visual and memory stimulating grids [6], preferential hyperacuity perimetry [4, 7] and shape discrimination tests [5, 8,9,10,11] have been reported to quantify distortion more accurately than either the Amsler grid or visual acuity in clinical settings [3].

The MONARCH study is evaluating three home-monitoring tests. All have some supporting peer-reviewed literature and usability data from AMD populations [5, 6, 8, 9, 12,13,14,15]. Two of these tests are electronic applications that run on an iPod touch and one is a paper journal of reading puzzles.

Objectives

The objectives of the MONARCH study are:

  1. (a)

    To estimate the test accuracy of three tests to self-monitor reactivation of nAMD compared to the reference standard of detection during usual care monitoring in HES clinics.

  2. (b)

    To determine the acceptability of the tests to patients and carers, explore reasons for non-adherence and the influence of carers on the acceptability of the tests and adherence.

  3. (c)

    To explore whether inequalities (by age, sex, social-economic status and visual acuity) exist in recruitment to the study, and are associated with participants’ ability to do the tests and to adhere to weekly testing.

  4. (d)

    To provide pilot data for using the same home-monitoring tests to detect conversion to nAMD in fellow eyes of patients with unilateral disease, compared to the reference standard of detecting conversion in usual care monitoring.

Subjects and methods

Study design

The MONARCH study is a multicentre, diagnostic test-accuracy cohort study. Participants are recruited in secondary care hospital eye service (HES) clinics. Eligible patients are asked to use three home-monitoring vision tests (the index tests) to test their vision weekly at home for a period of 12–30 months. Participants continue to attend HES monitoring clinics with retinal imaging carried out as required to inform usual care management decisions. See Fig. 1 for the study schema and Supplementary Information for details of retinal image collection.

Fig. 1: Study schema.
figure 1

The study schema shows the recruitment pathway, predicted numbers and follow-up schedule.

Full size image

There are no safety reporting procedures for this study; the study is regarded as low risk as it does not require participants to undergo any additional investigations and clinical adverse events cannot be attributed to study specific procedures.

All participating sites are NHS Trusts based in the United Kingdom (UK). The study has a favourable opinion from the Northern Ireland Health and Social Care Research Ethics Committee A (ref 17/NI/0235), is approved by the Health Research Authority, England and is being conducted in accordance with established principles of Good Clinical Practice [16].

Integrated qualitative study

Three sites are also taking part in an integrated qualitative study (objective B). Person-centred interviews are capturing difficulties, concerns, fears and perceived benefits about the index tests. Information about factors that influence successful implementation for specific participant subgroups is also being collected. A qualitative researcher is conducting (i) one-to-one home-based interviews and observations of participants regularly performing the index tests (n = ~75), (ii) telephone or e-interviews with other subgroups (see Study Population below), and (iii) interviews with health-care professionals involved in the recruitment and training of participants at each site.

Study population

Patients with at least one potential study eye are eligible. A study eye must have a diagnosis of nAMD meeting the eligibility criteria. Participants with bilateral nAMD may have two study eyes if both meet the criteria. A fellow eye is an eye without nAMD, but which meets all other eligibility criteria. An excluded eye is an eye which is not eligible to be a study eye or a fellow eye.

Inclusion criteria

A participant aged ≥ 50 years old with at least one potential study eye is eligible if:

  1. (1)

    Eye was first treated for active nAMD ≥6 months ago and ≤42 months ago,

  2. (2)

    Eye is currently being monitored for nAMD by the NHS.

Exclusion criteria

A potential study eye is excluded if ANY of the following apply:

  1. (1)

    Vision is worse than Snellen score 6/60, LogMAR 1.04 or 33 letters;

  2. (2)

    Vision is limited by a condition other than nAMD;

  3. (3)

    Surgery in the potential study eye in the previous 6 months;

  4. (4)

    Refractive error in the potential study eye > −6D;

  5. (5)

    Retinal or choroidal neovascularisation in the potential study eye not due to nAMD.

    In addition, a participant is excluded if ANY of the following apply:

  6. (6)

    Unable to do one or more of the proposed tests during the training session;

  7. (7)

    Unable to understand English;

  8. (8)

    Home or personal circumstances unsuitable for home testing.

Participants complete the index tests for both study and fellow eyes. Eye-specific outcomes, such as visual acuity, are collected at usual care monitoring visits. An excluded eye can convert to a study eye if time since first treatment and/or time since surgery reaches 6 months and all other eligibility criteria are met.

Participants are provided with all equipment required for the study. Details of all patients approached and reason(s) for non-participation are documented. A minimum dataset comprising age, sex, visual acuity, index of multiple deprivation and ethnicity is collected for all patients approached to monitor inequalities for objective C. Patient approach is detailed in Supplementary Information.

Study population: integrated qualitative study

Eligibility is not separately assessed for the integrated qualitative study. Participants and carers who consent to be contacted are approached directly by the qualitative research team. Health care professionals willing to be interviewed are invited for face-to-face, telephone or e-interview to explore acceptability of study processes and reasons why patients decline the study. Maximum-variation and purposive sampling will capture the range of participant and related-factors that may be potentially relevant to assessing acceptability (e.g. age, gender and eye health history) and the role of carers. Patients will be asked to ‘nominate’ a carer or person who offers personal support to them. The qualitative sample comprises a number of subgroups, described in Supplementary Information.

Stratification

Participants are being recruited into 3 strata according to time since first treatment for nAMD in a study eye (first-treated study eye for participants with two study eyes): (a) 6–17 months; (b) 18–29 months; (c) 30–41 months. This will allow the performance of index tests to be evaluated across a range of nAMD durations. Approximately equal recruitment into each stratum is planned.

Home-monitoring (index) tests

The following tests are being evaluated: (a) KeepSight Journal (KSJ) [6] adapted by the study team for UK use, (b) MyVisionTrack® (mVT) electronic application [5, 8, 9, 12, 15] (Genentech Inc.) and (c) MultiBit (MBT) electronic application [13, 14], developed by Visumetrics, licensed by Novartis International AG.

The paper-based KSJ was developed by the International Macular and Retinal Foundation (New Gloucester, Maine, US). It comprises three tests, viewed one eye at a time: (a) near visual acuity formatted as a puzzle with varying font sizes (Fig. 2a), (b) an instruction to assess distortions by viewing objects with straight lines (Fig. 2b), (c) a modified Amsler chart to record areas of distortion or scotoma (Fig. 2b).

Fig. 2: KeepSight journal puzzles for measuring visual acuity.
figure 2

Participants complete the paper KeepSight journal, containing (a) a reading vision test, (b) a daily home object test, and (c) a grid test, weekly.

Full size image

mVT is a shape discrimination threshold test displayed on an iPod Touch. It displays four circles, one of which is deformed. The participant identifies the odd-one-out (Fig. 3).

Fig. 3: MyVisionTrack® (mVT) electronic software application.
figure 3

Participants select the odd circle out, i.e. irregularly shaped circle; completing multiple rounds which increase in difficulty.

Full size image

MBT is a near acuity threshold test, which displays numbers made up of receptive field size dots or ‘rarebits’. These stimuli provide limited information to the visual system compared to conventional targets. Patients are presented with pairs of numbers (Fig. 4a) and required to state aloud the numbers they can see. The numbers are then presented in high contrast with a recording of the participant’s responses and the participant marks their own performance (Fig. 4b). We are validating self-marking in a sample of patients.

Fig. 4: MultiBit (MBT) electronic software application.
figure 4

a Pairs of numbers are displayed on the screen in quick succession. Participants must say the numbers they see aloud whilst the iPod audio-records. b At the end of the test, the numbers which were presented are displayed clearly and the audio-recording is played back to the participant. Participants have to mark their performance; if they said the correct number, they select the ‘Hit’ button below the corresponding number or if they said an incorrect number, they select the ‘Miss’ button.

Full size image

Reference standard

The reference standard is the status of the lesion in study eyes during usual care monitoring in HES clinics, classified as: definitely active, definitely inactive, or uncertain. Lesion status is decided by the ophthalmologist at the clinic visit and not defined by tests results, although optical coherence tomography (OCT) imaging and visual acuity will typically be available to complement clinical examination. Local site teams collect data for study and fellow eyes at usual care monitoring visits when the status of an nAMD lesion and/or the treatment plan is reviewed, i.e. a “management decision” is made. See Supplementary Information for the data collection schedule. Data are also collected on anti-VEGF injections administered between usual care monitoring visits.

Participant self-reported data

A participant’s willingness to continue in the study, wish for retraining and self-reported data (e.g. a participant’s own assessment of their vision) are confirmed by phone or in person by the local research team before each management decision in usual care.

Equipment and technical support

Participants are provided with an equipment pack containing: an eye patch, an iPod with mVT and MBT installed and activated, a mobile broadband device (MiFi device), a lens cloth, a stylus pen, a paper KSJ, a pen, instruction sheets, a weekly checklist and a phone number for technical support.

The Clinical Trials and Evaluation Unit (CTEU) study management team receives calls to the technical support phone line during standard office hours Mon-Fri (except holidays). Calls received to the support line are documented and reasons reviewed.

Adherence to home eye testing

Adherence is monitored from test data transmitted to the CTEU Bristol. Data summaries are reported to the study management group and study steering committee. Non-adherent participants who have provided consent for telephone contact are called by CTEU Bristol to ‘trouble-shoot’. Reasons for non-adherence are being documented.

Outcomes

Primary outcome

The primary outcome is classification of lesion status in a study eye.

Secondary outcomes

For Objective C we are measuring outcomes of uptake of index tests:

(a) Participation in the study, defined as consent (yes/no) among eligible patients approached to take part.

(b) Ability of participants to do the tests during follow-up, defined as the proportion of HES monitoring visits for which some data for an index test are available since the last HES monitoring visit.

(c) Adherence of participants to weekly testing, defined as the proportion of weeks for which data for an index test are available, aggregated across intervals between monitoring visits.

For Objective D, the outcome is onset of active nAMD in fellow eyes, defined by initiation of treatment.

Masking

All personnel carrying out usual care NHS monitoring are masked to data from index tests. Site teams are instructed to report any instance when a participant’s index test results are seen by a member of the site team, for example when assisting a patient who is having difficulties with the equipment/tests, as a protocol deviation. Reasons for deviations are recorded and remedial actions implemented. Participants may gain an impression that their vision has changed from home-monitoring tests.

Features to minimise bias

Risk of bias is considered with respect to bias domains previously described for diagnostic accuracy studies [17]. Features of the study to protect against bias in each domain are described in Supplementary Information.

Withdrawal and discharge

Each participant can withdraw from the study at any time. An investigator may also withdraw a participant at any time. If a participant is discharged from usual NHS monitoring, the participant is withdrawn from the study. All withdrawals are recorded. Data collected up to withdrawal will be used in analyses, unless consent is withdrawn.

Participants can choose to stop completing one or two of the index tests and continue in the study. The numbers of participants withdrawn, lost to follow-up or deceased will be reported.

Sample size

Objectives A and C

The target sample size is 400 participants with matched data from home eye testing. The number of visits is the key parameter and the target is based on assumptions that: participants will have about 2300 monitoring visits (average 6 visits/participant, with 5% attrition); nAMD will be ‘active’ for 30% of monitoring visits; and correlations between tests and reference standard will be 0.6 for both active and inactive lesions [18]. Multiple visits per participant are not independent and measurement error will dilute power to discriminate test performance; therefore, we have assumed an effective sample size of 1200 visits. This number of visits gives 90% power to detect a difference of 0.06, or 80% power to detect a difference of 0.05, in the area under the receiver operating characteristic curves (AUROC) for two tests if the AUROC is 0.75 [18].

Objective D

Estimates of the rate of conversion to nAMD per year among fellow eyes vary, ranging from 4 to 16% [19,20,21,22]. Assuming the risk in unselected patients is 5–6% per year, about 50 patients will have nAMD in both eyes at the time of recruitment. Among the remaining 350 patients, we expect fellow eyes of about 25–30 patients to convert to nAMD.

Plan for statistical analysis

For the index texts, alternative threshold criteria for classification will be explored to maximise test performance.

Objective A

The study will be analysed and reported in line with reporting guidelines for studies of diagnostic accuracy [23] and will follow a statistical analysis plan that will be written in advance of the analyses being carried out. We are unable to prespecify the analyses in detail because of the early stage of evaluation of the index tests.

Sensitivity, specificity, positive and negative predictive values of each test will be reported with 95% confidence intervals. The overall performance of the tests will be quantified by the AUROC. AUROCs for the tests will be compared to determine if one or more tests is superior to one or more of the others. Analyses will take account of the structure within the data, i.e. the nesting of visits (and eyes) within patients.

Other analyses may be explored to investigate whether the performance of home-monitoring overall can be improved by, for example, combining information: (a) from multiple index tests; (b) from adjacent home-monitoring periods preceding a monitoring visit (to see if there is evidence that index tests provide ‘advance warning’ of nAMD becoming active); (c) for the study eye and an unaffected fellow eye (to see if differences in scores between affected and unaffected eyes contribute to test accuracy). See Supplementary Information for further details.

Objective B

Interviews will be audio-recorded and transcribed for analysis and reporting. The research team will review results iteratively and data will be managed and analysed using NVivo software and content and thematic analytical strategies [24, 25]. The focus of the analysis will be on, for example, the acceptability of the tests, the factors that facilitate or impede such acceptability, ease/difficulty of using each test and the perceived benefits as well as the role of carers and family members [26]. The transcripts of interviews will be analysed to produce an integrated and synthesised account and interpretation of the acceptance of the new tests. The qualitative researchers and wider research team will meet to discuss iteratively and early on the results of the analysis including the generation of codes and categories from the content of the transcripts [27]. Overall, the rigour, transparency and sensitivity of the methodology [28] will be enhanced by following the consolidated criteria for reporting qualitative research (COREQ) such as respondent validation, reflexivity and discussion of analytical codes and categories [29].

Objective C

Regression models will be fitted to explore the influences of age, sex, social-economic status and visual acuity at recruitment on the outcomes of: consent to take part (among all patients approached); ability of a participant to complete a test, analysed separately for each index test (among all participants); and adherence to the study protocol (among all participants). The influence of these factors will be reported as odds ratios with 95% confidence intervals. Analyses will take account of nesting of visits within participants.

Objective D

All analyses for objective D will be descriptive only. We will explore how test accuracy for detecting conversion changes as a function of index test scores and report the test accuracy statistics for each test for detecting conversion, with 95% confidence intervals.

Frequency of analyses

For study objectives A, C and D, the primary analysis will take place when follow-up is complete for all recruited participants. No formal interim analysis is planned.

Data for objective B will be analysed iteratively during the study and there are no planned subgroup analyses. Study findings will be reported descriptively by strata.

Study management and monitoring

See supplementary information for details.

Protocol amendments

Version 3.0, in use since 30/05/2019, superseded version 2.0 which was used when recruitment started. The changes included lowering the minimum visual acuity threshold from Snellen 6/24, LogMAR 0.64 or 53 letters to Snellen 6/60, LogMAR 1.04 or 33 letters, the addition of interviews with healthcare professionals to address objective B and postal newsletters for participants.

Discussion

Recruitment started on 31/07/2018. Recruitment and index test data collection has been challenging, primarily due to technical difficulties.

  1. (a)

    During set-up, sites queried insurance, warranties and guarantees for liability of high-value equipment.

  2. (b)

    Study staff were inexperienced in setting-up multiple device management software to maintain a large number of iPod devices.

  3. (c)

    Distributing study equipment has been affected by restrictions on transporting devices containing lithium ion batteries by air.

  4. (d)

    Mobile phone signal coverage at participant’s homes and at NHS sites is limited. The mVT and MBT applications require internet connectivity by the MiFi device. Difficulties with internet connectivity is a common reason for home eye test data not being transmitted.

  5. (e)

    Faults and updates to software applications and Apple operating systems have caused interruptions to the availability of the index tests at participants’ homes.

Monitoring adherence to weekly home-monitoring is an important consideration. Following up participants who are not testing is time-consuming.

The results of this study will fill a gap in knowledge regarding the test-accuracy of home-monitoring for diagnosis of nAMD reactivation and the practicality of using home-monitoring with electronic devices in this patient group. After publication of the study results, the anonymised data will be made available upon reasonable request to the Sponsor institution, The Queen’s University of Belfast. The protocol includes a statement on data sharing [30].

Summary

What was known before

  • Wet age-related macular degeneration (nAMD) is the leading cause of vision loss in people aged ≥50 years. Whilst nAMD lesions can become inactive following treatment, relapse is common.

  • Patients may have many months without requiring treatment and regular monitoring visits place a significant burden on NHS Hospital Eye Service (HES) out-patient clinics.

  • Self-monitoring at home for patients with inactive disease would be more convenient and less costly for both patients and the NHS. HES clinic appointments could be given only to patients likely to require treatment or to newly referred patients.

What this study adds

  • The MONARCH study is a multi-centre diagnostic test-accuracy cohort study evaluating three home-monitoring tests: two electronic tests using an iPod and one paper journal.

  • The objectives of the MONARCH study are to estimate the test accuracy of the tests to self-monitor reactivation of nAMD, to determine the acceptability of the tests to patients and carers, to explore whether inequalities exist in recruitment, and to provide pilot data for using the same home-monitoring tests to detect conversion to nAMD in fellow eyes.