Introduction

Amblyopia, or ‘lazy eye’, is associated with a lack of visual stimulation in the early years of life, resulting in cortical visual impairment [1]. This could be due to amblyogenic risk factors including uncorrected refractive error, astigmatism, media opacities, ptosis or other congenital pathologies that cause stimulus deprivation, and abnormal binocular interaction from strabismus [2,3,4]. This condition represents a significant public health concern, with population prevalence estimated between 2–5% [5,6,7].

Even though amblyopia is initially largely asymptomatic, untreated amblyopia resulting in vision loss could lead to problems at school, bullying, reduced quality of life, lifelong consequences on future occupation choices, and mental health issues [8, 9]. Contrary to the traditional notion that amblyopia treatment may be ineffective for children above 7 years old [10], the Paediatric Eye Disease Investigator Group (PEDIG) studies demonstrated that treatment of amblyopia may still be effective in children aged 7 to 17 years [11, 12], with the effectiveness of treatment becoming significantly reduced with time [13]. Hence, it is better to detect amblyopia early via screening.

Traditionally, vision screening for amblyopia was performed in the healthcare setting by experienced or trained healthcare professionals, including orthoptists, optometrists, and ophthalmologists, or by non-trained professionals in schools. Many programmes have not been successful due to inconsistencies in screening modalities utilised and lack of systematic assessment of their impact [14]. Overcoming barriers to these traditional amblyopia screening methods such as cost, limited access to healthcare and a limited number of qualified screeners is an ongoing issue [15]. With the COVID-19 pandemic and its related restrictions and lockdowns, traditional vision screening has become more difficult and the use of home-based screening tools for amblyopia are increasingly advocated, so that children do not miss out on opportunities for amblyopia screening [16, 17]. However, the role of home-based screening tools for amblyopia has not yet been rigorously assessed by a systematic review.

Here, we performed a systematic review to evaluate the accuracy and reliability of home-based amblyopia screening tools compared with the existing gold standard.

Methods

This systematic review was conducted in accordance with Preferred Reporting Items for Systematic Reviews (PRISMA) guidelines [18]. The study was registered on PROSPERO (CRD42021233511) and the protocol is published in BMJ Open [19].

Eligibility criteria for studies in this review

Eligibility criteria were established prior to the conduct of this systematic review. All studies reporting diagnostic accuracies of home-based amblyopia screening tools among subjects less than 18 years old were included. Home-based screening tools included web or internet-based screening tools, mobile applications and other low-cost instruments such as digital cameras which could be used from home. All studies evaluating orthoptist or ophthalmologist-led screening, commercial photoscreeners, instruments incorporating artificial intelligence, autorefractors, and of adult subjects were excluded. Only Oxford Centre of Evidence Based Medicine (CEBM) level 4 evidence and above were included [20]. This includes case-series, cross-sectional studies, case-control studies, cohort studies, randomised controlled trials, and systematic reviews. All studies which did not report outcomes pertinent to the diagnostic accuracy of home-based amblyopia screening tools, such as studies reporting only validity or repeatability, epidemiological studies, case reports, expert reviews, opinion pieces, and conference abstracts without full publications, were excluded.

Information sources and search strategy

Electronic searches were performed through Ovid MEDLINE (1946 to present), PubMed, The Cochrane Library, Embase (1974 to present), Web of Science Core Collection (1970 to present), and Clinicaltrials.gov. Searches were conducted from inception until 31st August 2021. References of relevant studies were also searched and included if they met the inclusion criteria.

Medical subject heading (MeSH) terms such as ‘amblyopia’, ‘visual acuity’, ‘vision screening’ and terms to capture home-based screening tools such as ‘home’, ‘web’, ‘internet’ ‘app’, ‘smartphone’, and ‘mobile’ were used for the search where applicable. The full search strategy can be obtained via Online Supplementary Appendix 1. EndNote V.X9 (Thomson Reuters, New York, New York, USA) was used for data management. No date or language restrictions were stipulated.

Selection process

All studies went through a three-stage screening process involving title, abstract and full texts by two independent reviewers (SS, CS) according to the screening criteria available via Online Supplementary Appendix 2. Any disagreements were resolved with a third arbitrator (HJK). If there were any ambiguity on the screening tool, an email was sent to the first author of the paper to ask for more clarification before these studies were included. If there were missing data on specificity or sensitivity values, an email was also sent to the first author of the paper to acquire them. A reminder email was sent again if they did not respond after two weeks. If there was still no response after four weeks in total, those data were excluded from our analysis.

Data Collection

The main outcome measure reported was the diagnostic accuracy of home-based screening tools for amblyopia detection, expressed as sensitivity and specificity values.

Data was extracted from eligible studies using a tool adapted from the Cochrane Collaboration in the form of a table (Online Supplemental Appendix 3). Data collected included study design, number of included patients, duration of study, method of intervention used, index test, and reference standard where applicable. Data pertinent to the quality of diagnostic studies including investigators conducting test, subjects receiving test, method of interpretation of test, blinding of participants or investigators, and withdrawal rate of participants were also collected.

Risk of bias assessment

Risk of bias assessment was assessed using the QUADAS-2 tool for diagnostic accuracy studies (available via Online Supplementary Appendix 4) [21]. These judgments were made independently by two review authors (SS, CSC) and any disagreements resolved by the third arbitrator (HJK). Risk of bias and applicability concern were graded as low, unclear, or high.

Risk of bias and applicability concerns were graded for the following domains:

  1. i.

    Patient selection

  2. ii.

    Index test

  3. iii.

    Reference test

  4. iv.

    Flow and Timing

Summary measures

In addition to specificity and sensitivity values, confusion matrices (tables containing true positive, false positive, true negative, and false negative outcomes) were extracted from the included studies. Authors of papers that did not publish these values were contacted via email and given two weeks to respond. A reminder email was sent again if they did not respond after two weeks. If there was still no response after four weeks in total, those data were excluded from our analysis. If there were multiple thresholds used for amblyopia detection within the same study, or more than one sensitivity or specificity value were reported, the results will be reported based on the thresholds specified.

Patient and public involvement

There was no patient and public involvement during the conduct of this systematic review.

Results

Descriptive synthesis

The search was executed on 14 August 2021 and the screening was completed on 31 November 2021. Our search returned 3670 studies in total, of which 1021 were duplicates. Following title, abstract and full text screening, 28 studies were included in our systematic review (Fig. 1). The full list of excluded articles and reasons for exclusion is available on Online Supplementary Table 1. Among the home-based screening tools for amblyopia, there were seven internet-based tools measuring visual acuity [22,23,24,25,26,27,28], 16 mobile phone or iPad applications [29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44], three digital cameras; [45,46,47] whilst two studies used a combination of visual acuity charts and questionnaires [48, 49]. The included studies reported different forms of amblyogenic conditions including high refractive errors, astigmatism, ocular misalignment, and leukocoria. For these conditions except leukocoria, there are different criteria used as a cut-off point for amblyopia detection. Age of patients included in studies ranged from 0 to 16 years old.

Fig. 1: PRISMA study inclusion flow diagram.
figure 1

*Reasons for exclusion were as follows: Wrong intervention (n = 28); conference abstracts, short article or pre-prints (n = 19); Tool does not qualify as home-based tool (n = 12); wrong population (n = 4); wrong outcomes (n = 4); no results available (n = 2). Online Supplementary Table 1 contains the full list of excluded studies with reasons.

Full size image

All 28 studies included a reference standard except one longitudinal study, which compared facial photographs of patients with and without leukocoria. 22 studies had full ophthalmological examination whilst 6 studies had validated visual acuity measurement tools as their reference standard.

Of the 27 studies included, the sample size ranged from 70 to 36,973 patients, with a mean of 2004 patients. A summary of the studies included in the systematic review is presented in Table 1.

Table 1 Summary of studies grouped by type of home-based screening tool.
Full size table

Sensitivity and specificity values were reported in 25 studies. Confusion matrices were included in 16 studies and can be assessed through Online Supplementary Table 2.

Risk of bias assessment

We used the QUADAS-2 tool in the systematic appraisal of all but one longitudinal study which did not include a reference standard or index test (Table 2) [39].

Table 2 Risk of bias assessment using QUADAS-2 tool [22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49].
Full size table

With regards to patient selection, 52% of included studies were shown to have high applicability concern. This was primarily due to patient selection from tertiary centres which were not representative of the normal population. A similar proportion showed unclear or high risk of bias in patient selection as the process was either unclear or non-systematic. Around a quarter of studies showed unclear or high risk of bias and applicability concern in index test used: 18.5% showed unclear or high risk of applicability concern in reference test used, whilst 33.3% showed unclear or high risk of bias in reference test used.

15% of studies were also shown to have a high risk of bias in flow and timing, as not all patients who went through the index test were subjected to the reference test or were lost to follow up.

Quantitative synthesis

Specificity and sensitivity varied across studies with regards to the type of diagnostic tests used, country of study, patient selection and age of children included. Sensitivity estimates for internet-based tests ranged from 50.0% [22] to 93.8% [25] while specificity estimates ranged from 70.0% [27] to 98.9% [22]. Sensitivity estimates for mobile applications were between 15.4% [42] to 94.9% [34], while specificity estimates were between 63.0% [36] to 93.0% [38]. Digital camera as index test showed sensitivity estimates between 89.0% [46] to 94.6% [47] and specificity estimates between 82.9% [46] to 98% [45].

A meta-analysis could not be carried out as the studies were heterogenous in terms of type of screening tool used, differing cut-off values for amblyogenic conditions detected, population, reference standard used, and lack of raw data from confusion matrices. The heterogeneity of studies included is demonstrated in Fig. 2.

Fig. 2: Summary of sensitivity and specificity estimates through a HSROC curve.
figure 2

A line of best fit, which is the HSROC curve, passes through these dots. The size of dots represents the size of study included. The region marked by the dotted lines represents the 95% confidence interval for the studies included.

Full size image

Discussion

Summary

To the best of our knowledge, this represents the first systematic review on the diagnostic accuracy of home-based amblyopia screening tools. Our systematic review revealed that there is a broad variety of home-based amblyopia screening tools available globally in the form of internet-based tests, mobile applications, digital cameras, and a combination of others. However, based on our rigorous assessment on the quality of studies included, there is an overall risk of bias in the existing studies. Lack of raw data pertinent for a pooled analysis, heterogeneity in the studies, and lack of standardisation on age-defining values for detecting amblyogenic conditions makes meaningful comparison of sensitivity and specificity estimates limited, which is crucial for the systematic appraisal of diagnostic accuracy studies [50].

Quality of evidence

Patient selection in around half of the studies demonstrated unclear or high risk of bias, either due to patient selection from tertiary centres, or non-systematic methods of recruitment. The paediatric clinic represents a population that could have a higher prevalence of amblyogenic risk factors. This may lead to overestimation of sensitivity values. Less than half of the studies showed unclear or high risk of bias in the index and reference test used. This is due to the lack of clarity on blinding during the conduct of the index test, or its interpretation which is not independent of the reference test, and the lack of uniformity in the reference test used across the studies. In studies that included full ophthalmological examination as their reference test, most included slit lamp and fundus examination by ophthalmologists, but not all specified whether retinoscopy, cover tests, or assessment of ocular motility were included. It was also difficult to standardise the visual acuity measurement tools used for studies that selected this as their reference standard, as the modality of tests would depend on the age of children recruited. In addition, there is lack of standardisation and international consensus on the age-defining values for detecting amblyogenic conditions such as high refractive errors, astigmatism, and ocular misalignment. Response bias was also present in studies that used a combination of screening tools, including questionnaires, as this depended on the response rate of the population involved [25, 48, 49].

Internet-based tests

Five studies published computer algorithm-generated standard visual acuity tests [22,23,24,25, 28], one study involved the use of video game (Eyespy) [26], and another study used computer generated checkboards of various frequencies for amblyopia detection [28]. Among these, only Longmuir et al. [23], Schlenker et al. [24], and Trivedi et al. [26] included lay screeners or parents in the conduct of the index test. The Jaeb Visual Acuity Screener (JVAS) [27], a free internet-based visual acuity screening test has been developed and validated for use among non-ophthalmic health professionals by the PEDIG group.

Mobile applications

The most common mobile application from the diagnostic accuracy studies included was GoCheck Kids (Gobiquity Mobile Health, Scottsdale, AZ, USA) [29,30,31, 35,36,37,38, 41, 44]. One of the advantages of mobile applications for screening is it can be used for younger children, such as the CRADLE app [39, 42] which is used on children as young as two months old to detect leukocoria. However, it demonstrated low sensitivity as it does not detect other amblyogenic risk factors with higher prevalence. In contrast, the GoCheck Kids app and Peek acuity app were used to detect high refractive errors and astigmatism, which were more common in the study population, hence reporting higher sensitivity values. The Eyeturn app [32] and Mhealth [38] were both used to detect ocular misalignment. Table 3 summarises the existing internet-based tests or mobile applications from our systematic review which has gone through validation studies and are free of charge.

Table 3 Validated tablet or mobile home screening tools.
Full size table

Peek Acuity is the only app to date which had been shown through a cluster randomized controlled trial to increase follow-up rates through an integrated system involving the app [51].

Digital camera

There were only three studies that used digital camera for amblyopia screening. One incorporated a DV-S20 [45], and the other two utilised modified digital camera [46, 47]. However, the interpretation of the images was either by professionals or based on computer-generated analysis, and they tend to be more expensive than internet or mobile applications, making them less ideal as home-based screening tools.

Research in context

This systematic review is the first to publish a descriptive summary on the diagnostic accuracy of the latest available home-based amblyopia screening tools. There are parallels on the use of telemedicine rapidly in the COVID-19 era for the diagnosis and management of diabetic retinopathy, glaucoma, age-related macular degeneration, and for triage in emergency eye care [52]. Amblyopia detection to date has been largely undertaken by trained ophthalmic professionals. Even though the advent of photoscreeners such as the Plusoptix™, SPOT™, and autorefractors have enabled the screening process to be easier and less time-consuming, there is a lack of strong evidence on its cost-effectiveness for use in the community [53]. Home-based tools for amblyopia screening satisfies the WHO criteria for a good screening test [54]. Amblyopia has a pre-clinical phase that enables early detection and is a condition that can be treated. Home-based screening tools are easily available, less costly, or free of change, and reduce the dependence on trained ophthalmic professionals. Our systematic review showed that the sensitivity and specificity of some home-based tools to be comparable to photoscreeners or autorefractors [55]. As there has been a lot of research into the effectiveness of home-based tools for detection of amblyopia, this systematic review is important to summarise the existing evidence from literature in this field.

Strengths and limitations

This systematic review has highlighted key findings which may help pave the way for further research using home-based tools in amblyopia screening. The home-based tools included in this review are not restricted to the digital age, as questionnaires, digital cameras, and visual acuity charts used in the home setting are also included. Even though this systematic review included a comprehensive range of home-based tools in diagnostic accuracy studies, we did not include any studies reporting validity or reliability of home-based amblyopia screening tools or studies evaluating the cost-effectiveness of these tools. In addition, there is a possibility that other validated home-based tools are not captured in this systematic review if they have not been evaluated in diagnostic accuracy studies. Moreover, studies examining the feasibility of these tools by lay screeners were not included. Some of these internet-based tools or mobile applications may require some training before use, which may exclude some users.

Conclusions

This systematic review highlighted the availability of home-based screening tools, which could aid in amblyopia screening. However, there is a need to improve the quality and reporting of diagnostic accuracy studies using these tools. Home-based screening tools could be advantageous especially due to the COVID-19 pandemic, where amblyopia screening has stalled. Such tools may be a suitable option for low- and middle-income countries. However, as the incentive is upon the parents or lay screeners to utilise these resources, there is a need to educate the public on the importance of amblyopia screening at home, given the various options available for this. Ideally, evidence-based amblyopia screening tools could made be widely available for home-use, but further work is needed to identify the most effective tools for this purpose.