Introduction

RetmarkerAMD® (Retmarker SA, Coimbra, Portugal) is a semiautomatic grading software developed specifically for age-related macular degeneration (AMD). Pilot studies demonstrated higher sensitivity and specificity than manual grading and attested its capacity to decrease grading time and identify more AMD features, thus reducing human error [1]. The aim of this study was to validate RetmarkerAMD® as an AMD grading tool, by comparing it with an already validated and widely used platform—Topcon IMAGEnet2000®.

Methods

Multicentre, cross-sectional study. A set of 202 colour fundus photographs (CFPs) randomly selected from a pool of eyes with and without AMD were used. All images had previously been graded by a senior retina specialist (gold standard) using Topcon IMAGEnet2000®. Two certified graders with different experience independently classified all CFPs using both platforms (Fig. 1; Supplemental Fig. 1) after brightness, contrast and colour balance standardization [2]. Conversion of AMD staging from the Rotterdam classification to the AREDS [3] classification (Supplemental Tables 1 and 2) was achieved to allow a comparison between platforms.

Fig. 1
figure 1

Real-time demonstration of the grading process using RetmarkerAMD®. a First, it is necessary to manually identify the fovea and establish the optic disc diameter (blue dot and arrows, respectively) to achieve the calibration. The software then generates an automated grid b according to International Grading System for AMD. c Fundus abnormalities including drusen, pigmentary changes, geographic atrophy and neovascular AMD can be quantified using free-forms or pre-defined circles (63, 125, 175, 250 and 500 μm). d Grading results (including automatic AMD staging according to the Rotterdam classification—Supplementary Table 1) are depicted on screen and can be exported to a Microsoft Excel® file for posterior statistical analysis. RetmarkerAMD® also allows to categorically quantify the number (0, 1–9, 10–19, ≥20) and area (<1%, <10%, <25%, <50%, ≥50%) of drusen, and additionally to determine the real number and area of drusen (μm2), total or by semi-field

Full size image

Intra- and inter-grader agreement was evaluated by the percentage of agreement and the weighted Kappa coefficient considering linear weights [4].

Results

The inter-grader analysis for all features analysed with RetmarkerAMD® is shown in Table 1. For AMD staging alone, an almost perfect agreement (93.0%; Kappa = 0.95, p < 0.001) was observed. The same was true for AMD staging using Topcon IMAGEnet2000® (90.1%; Kappa = 0.87, p < 0.001). Both graders showed a high agreement with the gold standard (90.1%; Kappa = 0.88, p < 0.001 and 87.1%; Kappa = 0.86, p < 0.001 for graders 1 and 2, respectively). Regarding the inter-modality analysis (Supplemental Table 3), a 76.8% agreement (Kappa = 0.73, p < 0.001) and a 70.8% agreement (Kappa = 0.67, p < 0.001) was observed for graders 1 and 2, respectively.

Table 1 Inter-grader (grader 1 vs. grader 2) agreement analysis using the RetmarkerAMD® software
Full size table

Discussion

This study aimed to validate RetmarkerAMD® as a semiautomatic grading software for the fundoscopic changes associated with AMD. We used CFP because of its reproducibility and loyalty to clinical fundoscopy, making it easy to extrapolate results to clinical practice.

Regarding AMD staging, a considerably high agreement was found in both platforms. Subtle differences between graders may have been influenced by a higher level of experience of grader 1. In fact, grader experience is a factor used across studies to justify discrepancies on agreement analyses [5].

The comparison of intra-grader analyses using the two software systems revealed a substantial agreement. The clear overlap between early and intermediate stages where subtle and somewhat arbitrary differences are seen may have influenced the results. Classification bias may also have been introduced due to the conversion of the Rotterdam classification into the AREDS classification.

Despite several advantages, RetmarkerAMD® presents some limitations. First, it demands lesion identification by a human grader, a rate limiting and fatigable process. Second, this platform is not prepared to grade optical coherence tomography, a rapidly growing technology with promising outcomes in automatic and semiautomatic AMD grading.

By conducting a carefully planned study, we were able to demonstrate that RetmarkerAMD® is a reliable and consistent semiautomated grading tool for AMD. The validation of RetmarkerAMD® may prompt its use both in clinical studies and in clinical trials.