Introduction

Developments in digitization and technology have revolutionized and advanced all aspects of ophthalmology in recent decades [1, 2]. However, even in the current digital era, the majority of findings derived from the entire strabismus evaluation process are documented by strabismologists using only words and numericals in the medical charts. Conventional examinations for ocular misalignment, including the alternate cover test (ACT), prism alternate cover test (PACT), and simultaneous prism cover test, which are performed manually by ophthalmologists or orthoptists, can be a complex procedure that requires ample experience [3]. Furthermore, the results of these examinations may be limited for direct comparisons, which may result from operator experience, patient cooperation, and measurement errors, as evidenced by documented interobserver variability [4, 5]. Thus, newer techniques of strabismus evaluation are needed to improve measurement efficacy and reduce errors.

Currently, multiple video-oculography systems that implement various technologies such as infrared imaging, smartphone apps, eye-tracking, virtual reality (VR), and artificial intelligence have been proposed and used for video recording and diagnosis of eye movement disorders [6,7,8,9,10,11,12,13,14,15,16,17]. The NeuroSpeed System (NSS; Ophthalmic version), designed and engineered by Neurobit (Neurobit Technologies Co., Ltd, Taiwan), is an automated analysis system with portable head-mounted eye-recording glasses. This device was designed to evaluate all patients with strabismus. In our current study, we demonstrated the application of this novel device in strabismic patients and the results of the angle of deviation measured in the primary position.

Subjects and methods

This study incorporated a method comparison design. The study period extended from January 1, 2020, through December 31, 2020. The study was conducted in accordance with tenets of the Declaration of Helsinki and its protocol and supporting documents (1-106-05-157) were reviewed and approved by the institutional review board of the Tri-Service General Hospital, Taipei, Taiwan. All patients who presented to the strabismus division of the ophthalmology department of Tri-Service Hospital with strabismic or neuro-ophthalmic problems involving eye movements were invited to participate in this study. After providing written informed consent, the patients enrolled in the study underwent standard ophthalmologic examination (similar to that during a routine clinical visit), and other ocular diagnostic examinations, such as a slit-lamp examination or nonmydriatic fundus photography, if needed. All participants were assessed using conventional procedures for strabismus evaluation (as described below) to obtain clinically relevant results that could serve as diagnostic standards for comparison. Thereafter, the entire NSS recording protocol was completed on the same day for each patient under the guidance of a trained examiner, as described below. The video recordings were further analysed using the NSS software.

Conventional strabismus examination procedure

All strabismus evaluations were conducted by one examiner (KHC). Conventional techniques for determining ocular alignment, such as the cover-uncover test and ACT, were performed using the standard protocol. The angle of deviation in the primary position was measured using the PACT with the patient fixating on a distant target located 4 m away and neutralizing the deviation with a prism bar. The examination results were documented in each patient’s medical chart and reviewed.

NSS recording protocol

After undergoing conventional strabismus evaluation in the routine clinical setting, the patient was referred to a trained examiner for the NSS recording protocol. First, the head-mounted video-oculography machine (smart glasses of the NSS) was adjusted for the patient, who stood in front of a target-point projector (Fig. 1A). A programmed automatic recording examination sequence was started with audio and target-point visual guidance by a computer installed with the software designed by Neurobit. The patient was instructed to follow the audio instructions to complete the recordings under the examiner’s guidance. The recording sequence could be modified to cater to different examination protocols in order to mimicking standard strabismus tests; in the current study, the recording sequence was a modified ACT. Subsequently, the video recording was stored and further analysed by customized software. The hardware and software design of the NSS device is described briefly in the following section.

Fig. 1: The NeuroSpeed system (NSS) recording protocol.
figure 1

A Representation of the environmental setup for the NSS exam protocol. The patient was situated 2.62 meters in front of a projector screen while wearing the head-mounted NSS smart goggles with both devices connected to a portable computer. B The hardware of the NSS, including the smart goggles (left) and the connected computer with customized software (right), is designed for recording and analysing simultaneous eye movements. The NSS smart goggles are equipped with two infrared reflecting mirrors and two camera sensors. C An example summary of the customized NSS software for analysing strabismus of its horizontal deviations. The video recorded from the head-mount goggles is analysed via an offline eye-tracking program to obtain accurate eye/pupil positions in an alternate-cover-test (ACT) recording sequence for qualitative and quantitative measurement in prism dioptre (PD). D An example of the NSS report of a 32-year-old patient with alternating exotropia. The report summarizes and displays the patient’s clinically relevant data (in the upper section of the report), waveform analysis of the recorded ACT recorded sequence (in the middle section of the report), sequence of still images of the ACT sequence (bottom right), and the actual video recording through a linked QR code (bottom left). The NSS-predicted results of the ACT sequence for the primary position are provided in the centre table.

Full size image

Hardware design of the NSS video-oculograph

In brief, the whole eye, including the eyeball, eyelid, and periocular area, was imaged using a wearable and head-mounted glasses frame, which was equipped with multiple low-power infrared light emitting diodes to illuminate the eyes, two narrow-band infrared reflecting mirrors designed to reflect upright eye images, and two wide-angle infrared image camera sensors to receive the video image (Fig. 1B). The video of both eyes was synchronized in every single time frame and stitched together directly on the goggle device, followed by recording, transmission, and stored in a connected computer. The participant was able to see through the mirrors with the naked eye (or wear contact lenses for refractive correction if the refractive status interfered with fixation) and view designated targets located 1–10 m away and within 0–50° in the cone angle of vision of each eye. In this study, the patient was placed 2.62 m in front of the external screen to provide a geometrically calculated 60° viewing angle of the projected screen.

Customized software system design

The customized software installed on the connected computer performed three main functions to assist in the entire recording sequence. The first function was to collect participants’ basic data and eye movement videos via a customized user interface; the second was to project one or several time-synchronized target points to an external screen (via a projector or large-sized TV screen) with a previously defined size, position, and sequence with audio instructions (refer to the “recording sequence parameter” section below); and the third function was to analyse the general eye position and angle data in a time series and provide a summary of the test. The recorded infrared video measured 1280 pixels in width and 480 pixels in height at 25 frames per second in resolution, with an aspect ratio of 1.0, and depth of 8 bits. Video analysis was performed using custom codes written in Python (briefly described in the “software analysis” section below). The customized software also performed an automatic quality check for each video recording. Whenever the software could not detect the centre of the pupil, the analysing program lost signal data of the eye position in the recorded video; these lost data were defined as missing points and were automatically analysed by the software for quality control. When the cumulative signal losses (missing points) reached a predefined threshold, the video was deemed to be of poor quality, which could possibly result in poor prediction of the angle of deviation and was thus excluded or interpreted with caution.

Recording sequence parameter

The recording sequence was designed to be synchronous with the audio instructions and target-point projections on the external screen in a predefined order. In the current study, the recording sequence was a modified ACT test. In the ACT recording sequence, a target dot was projected in front of the patient at the (0, 0) position approximately at the centre of the external screen. The patient was instructed to fixate on the target point with either eye, while the examiner covered the right or left eye with an occluder repeatedly every 3 s according to the audio instructions. With the synchronized audio/target-point instruction, a sequence-labelled video recording was generated, i.e., the time point of covering the right or left eyes was labelled.

Software analysis

The recorded sequence-labelled eye movement video was analysed automatically using customized software (Fig. 1C). First, the software automatically captured the eye position according to the centre of the pupil of each eye in each frame throughout the entire video. Second, an eye movement waveform was generated using the information of the eye position. Third, using the labelled sequence throughout the video, the specifically designed ACT waveform analysis algorithm calculated the pixel differences (eye position change) in the sequence of each eye’s movement and between both eyes. Fourth, the pixel changes were calculated via a mathematical model adjusted with a precalibrated algorithm to provide an actual prediction of the angle of deviation in prism dioptres (PD). Finally, using the eye position differences throughout the recording sequence, the software summarized the pattern and degree of strabismus in the final report immediately after the conclusion of the recording (Fig. 1D).

Outcome measurements and statistical analysis

For the method comparison study, each video was manually checked for quality to ensure that no confounding factors would interfere with the accuracy of the NSS-predicted results. Thereafter, the results of the NSS-based ACT recording sequence were compared to those of the conventional PACT, which served as the standard for comparison. The angle of deviation in the primary position predicted by the NSS software was compared with those of the conventional PACT for a far target (4 m). The data were analysed using SPSS software version 16.0 for Windows (SPSS Inc., Chicago, IL, USA). The Spearman rank correlation test was conducted for the ACT and NSS results data to ensure a positive correlation coefficient, followed by method comparison analysis. Passing‒Bablok regression analysis was performed for method comparison; the fitted values of the intercept (systematic bias) and slope (proportional bias) of the regression line were computed and tested to determine if the methods were comparable using confidence intervals. The linearity of data was also tested and graphs were generated for the regression line and residuals. Bland‒Altman plot analysis was used to demonstrate the comparison between the differences against the averages of the two methods, i.e., PACT and NSS.

Results

This study enrolled 70 individuals aged 4 to 80 years [mean age 39.71; standard deviation (SD) = 18.39], of which 38 were men and 32 were women. Thirty-five of the 70 participants had comitant horizontal strabismus, 8 had thyroid eye disease-related strabismus, 10 had orbital fracture-related strabismus, 11 had strabismus due to cranial nerve palsy or other tumour-related incomitant strabismus, and 6 individuals without strabismus were included as normal controls. Furthermore, 25 of the 70 participants underwent surgery for the underlying condition and were examined after surgery, which resulted in orthotropia (n = 15), residual exotropia (n = 5), or residual esotropia (n = 5) in the primary position. All the video recordings passed the automatic and manual quality check for the comparison study. For method comparison analysis, participants were divided into 3 groups based on the type of horizontal deviation: orthodeviation (n = 30), exodeviation (n = 20), and esodeviation (n = 20). Moreover, 11 participants with concurrent vertical deviations were also subjected to NSS predictions and analysis for comparison. All 6 normal controls were orthotropic and were also predicted to be orthotropic by the NSS. Subsequently, the NSS-predicted results of the ACT recording sequence were compared with the results obtained by the conventional method (PACT) documented in the medical chart. The overall results of horizontal and vertical deviations in the primary position of both methods for all 70 participants are enumerated in Supplementary Table S1.

We classified the results into horizontal and vertical deviations for statistical analysis of the method comparison. Overall analysis of horizontal deviations using the Passing‒Bablok regression revealed a correlation coefficient r of 0.969 with a systemic bias (intercept, I) of 0.00, proportional bias (slope, s) of 0.809 (Fig. 2B), and perpendicular residual SD of 4.134 (Fig. 2A). The performance was also evaluated for the individual deviation types: exodeviation, r = 0.971, I = −1.63, s = 0.912, and residual SD = 3.133 (Fig. 2D, E); and esodeviation, r = 0.920, I = 5.230, s = 1.319, and residual SD = 5.390 (Fig. 2G, H). Bland‒Altman plots were also generated to evaluate the agreement between the results derived from the NSS and PACT. The average overall bias was −0.685 PD (SD = 4.872) (Fig. 2C), while the overall bias for exodeviation alone was −3.203 PD (SD = 4.425 PD) (Fig. 2F), and that for esodeviation alone was 0.890 PD (SD = 7.520 PD) (Fig. 2I). In general, the NSS-based system could achieve good prediction with less than 5 PD of bias for patients with less than 40 PD of horizontal deviation. However, the NSS data showed slight inconsistencies with the PACT data in calculating esodeviation at larger angles. We reexamined the participants (n = 3) with a larger bias between the two methods and found that the horizontal deviation of two eyes was more than 10 PD caused by secondary deviations due to abducens nerve palsy, which could have increased the overall systemic bias for esodeviation results.

Fig. 2: Method comparison study between the prism alternate cover test (PACT) and the NeuroSpeed system (NSS)-based results.
figure 2

Residual plots (A, D, G, and J) and scatter plots with regression lines (B, E, H, and K) were derived from Passing‒Bablok regression analysis for comparing the deviation results in prism dioptre (PD) obtained by the PACT and the NSS. Bland‒Altman plots (C, F, I, and L) were also used to reveal the differences in comparing the results from the two methods. A, B, and C depict the comparison results of the horizontal deviations for all participants. D, E, and F depict the comparison results for patients with exodeviation (XT). G, H, and I depict the comparison results for patients with esodeviation (ET). J, K, and L depict the comparison results for the vertical deviations for all participants.

Full size image

Method comparison analysis was also performed for the vertical deviation results (Fig. 2J, K, L). However, due to the small sample size of vertical deviations in our study, the results of the statistical analysis were not significant.

Discussion

Accurate ocular alignment tests are crucial for the differential diagnosis of strabismus disorders, surgical decision-making, and implementation of quantitative treatment. In the current study, we adopted novel strabismus photography glasses with a video analysis system (hereinafter referred to as the new smart glasses, NSS). This new device employs a wearable design, including an infrared imaging system with customized software, for strabismus analysis. This current study evaluated the difference in eye movement analysis between the traditional PACT and the preliminary results of the NSS. In our study, the results of the NSS were comparable to those of the PACT for assessing ocular alignment in the primary position.

Different techniques have been used to measure the deviations qualitatively and quantitatively during the assessment of ocular alignment and motility. Subjective methods, such as the Maddox rod test [18], the Hess chart, and the Lancaster screen test [19,20,21,22,23,24], are most commonly used in the clinical setting to verify patients’ strabismus complaints. Although all these tests yield accurate results with good patient cooperation, these subjective measurement methods lack objectivity and require active cooperation of the patient. Moreover, these tests cannot be performed accurately in patients with visual suppression or abnormal retinal correspondence. Some of these tests, especially the Hess chart and the Lancaster screen test, require additional professionals and are very time-consuming, taking up to 20 min for completion. In terms of objective methods, the most commonly used technique in measuring ocular deviation, and clinically considered the gold standard, is the PACT [18, 25]. However, the manual PACT requires trained personnel, such as an experienced orthoptist or ophthalmologist specializing in paediatric ophthalmology, to achieve reliable results [4, 5, 25, 26]. This simple examination can be time-consuming and yield inaccurate results if the operator is unfamiliar with the examination techniques and protocol. A 2014 study that assessed the interexaminer variability between four trained examiners for the PACT measurements found variations ranging from 7.5–11.7 PD at distance fixation [3]. In that study, the Bland‒Altman plots revealed a mean difference of 0.62 PD (±4.3 PD) for distance measurement among four experienced orthoptists assessed at the same visit [3]. Therefore, although the manual PACT is an objective examination, it is not absolutely reliable. Furthermore, repeated measurements by the same inspector or several different operators may yield inconsistent results [4]. The examiner must observe eye movements while alternately covering and uncovering the eyes. Therefore, some tiny eye movements may be easily missed, which will affect the diagnosis of some diseases with subtle abnormalities in eye positions, such as partial cranial nerve palsy.

Therefore, a method or device for measuring ocular deviation and eye movement should be easy to execute for the examiner, promote patient cooperation, and allow repeatability under the same conditions. The results of successive examinations should be objectively comparable with each other. Studies on technology-assisted strabismus measurement reported that the difference between manual and automated measurement was 0.88 PD (±11.32 PD) in VR headset study [9] and −2.9 PD (±11.4 PD) in an eye-tracking system study [8]. Our system employed NSS smart glasses that can both track ocular deviation and provide automated measurements of ocular deviation with a refined software sequence, whose measurement accuracy is reflected in the overall mean difference of −0.685 PD (±4.872 PD).

Hence, the novel NSS device introduced in the current study can serve as an alternative to conventional examination methods for the diagnosis of strabismus in the primary position and calculating its deviations, which yield results comparable to those of the PACT test. This system also possesses several additional advantages over conventional methods. First, the head-mounted recording device was designed to allow the participant to look forward without blocking the line of sight of the patient and examiner. The examiner can assess the patient using any conventional method (e.g., the 3-step test, or any kind of cover test that entails placing prisms in front of the smart glasses) without any difference. The NSS can also simply serve as a recording system. Furthermore, the analysis software can be designed to analyse any conventional strabismus method with a customized sequence protocol. Second, the additional computing and prediction power provided by the accompanying software and recording protocol of the NSS further facilitate the evaluation of strabismus. Examiners do not need to shift the occluder and frequently change the prism (to recheck multiple times) in order to achieve an accurate result while applying the NSS recording protocol for strabismus evaluation, which could potentially shorten the total examination time. Finally, in the event of ambiguity in the predicted results, the examiner can simply rewind the video multiple times (which is stored in the NSS database) to cross-check the video quality or verify any eye movement disorder (e.g., a specific type of strabismus, nystagmus, or microdeviations that are difficult to interpret by the naked eye) recorded during the examination, which could significantly lessen the examination burden for the patient and examiner.

Accommodation is one of the parameters that should be considered in strabismus measurements, since there is usually a difference in measurement between far and near instances [25]. Automated measurements provided by modern instruments such as VR-based head-mounted displays and smartphones, reportedly demonstrate an esoshift in patients with esotropia and exotropia [9]. An esoshift is assumed to be the result of accommodation reactions to near reflected monitors and oculomotor conflict, which increase the accommodation convergence/accommodation ratio and phoria [9, 27]. Accordingly, some modifications must be made to compensate for the disagreement to produce precise results in the smartphone software design [7]. Esoshift reportedly results in overestimation of the measurements in patients with esotropia and underestimation in those with exotropia [9]. We modified the hardware and employed see-through glasses instead of a closed VR system to avoid the esoshift effect associated with these devices and retain the flexibility to measure misalignments at both far and near distances. However, a slight convergence should be anticipated with the current preset distance (2.62 m) between the projector screen and the patient. Hence, we also observed a slight esoshift trend between manual and automated measurements, resulting in a shifted predicted angle of deviation in both esotropic (0.890 PD more) and exotropic (3.203 PD less) participants.

The current study had some limitations. First, akin to all video-oculography-based techniques, poor video acquisition quality could have resulted in poor prediction. Patients with conditions that would hinder the automated algorithm from capturing the pupil, such as small palpebral fissures, long eyelashes, severe ptosis, excessive blinking, or other pathologies that do not respond to voice commands, such as complete nerve palsy with an immobilized eye, poor visual acuity, and nystagmus with no null point, can encumber the acquisition of good-quality video for analysis. The algorithm in the current version of the customized program for automated strabismus analysis program is designed to analyse the pupil for eye movement tracking, on the basis of the eye being able to follow the voice commands to achieve good video quality. However, since these above-mentioned patient population are of considerable clinical research interest, the next iteration of the algorithm will be more robust for these specialized conditions. Second, the esoshift encountered in our study was mainly due to the preset distance of the monitor, resulting in a slight convergence of the eye. This issue will be addressed in the next version of the algorithm via autocorrection of this convergence issue. Third, some large angle deviations with inconsistent results were observed in patients with cranial nerve palsies that had significant secondary deviations. The current version of the analysis software and recording sequence were not designed for differentiating the difference between the primary and secondary deviations; however, we included these cases deliberately to assess the performance of the current algorithm, which, unsurprisingly, provided poor results. We endeavour to solve this problem in the next iteration of the analysis algorithm (which is in development). Fourth, the current study presented data of only the primary position with only the ACT sequence with a primary focus on horizontal deviations (due to the small number of participants with vertical deviations), which would probably result in not showing any statistical or clinical significance; thus, the application of the proposed system may appear limited. However, in follow-up experiments, we performed measurements for incomitant strabismus and other strabismic disorders with our novel NSS under various recording protocols, such as the 9-gaze motility test. The results are promising and we endeavour to provide more data and detailed information in our future reports.

In conclusion, our study demonstrated that the NSS, a novel automated system with head-mounted video-oculography and analysis software, can serve as a novel diagnostic aid for the clinical evaluation of strabismus. This innovative system provides insights into developing machine-based strabismus examination methods for reducing personnel load in clinical practice.

Summary

What was known before

  • Currently, the conventional ocular alignment tests are usually documented in numbers and need the experience to master which can be time-consuming and unreliable for a novice.

What this study adds

  • In this study, we demonstrated the novel smart glasses NeuroSpeed system, a video-oculography machine with automated analysis software, which could efficiently provide accurate and reliable results comparable to the prism alternate cover test in the primary position for strabismus measurement.