Introduction

Dry eye disease (DED) is the most commonly occurring ocular surface disease, affecting 5–50% of the population globally1,2. Its prevalence is expected to increase with the ongoing digitalization and aging of the society2,3. Patients with DED present with a wide range of symptoms, such as ocular pain, discomfort, and decreased visual acuity caused by decreased tear film breakup time (TFBUT) and kerato-conjunctival epithelial defects4,5. Hence, DED has a negative impact on productivity and quality of vision, thereby impacting quality of life and resulting in economic loss6,7. A significant proportion of patients with DED may be undiagnosed and do not seek treatment despite experiencing symptoms8, indicating the need for a novel approach that can expand the reach of DED screening, promote early diagnosis and intervention for the prompt management of symptoms, prevent decreased quality of life, and reduce the societal costs of DED management5.

The demand for non-invasive and non-contact examinations and the incorporation of telemedicine in routine practice have rapidly increased with the novel coronavirus disease (COVID-19) pandemic9,10. DED is diagnosed by evaluating subjective symptoms and objective findings on examinations, such as TFBUT and ocular surface staining11,12. Dry eye examinations require specialized equipment, such as slit-lamp microscopes and fluorescein dye; moreover, the invasive nature of the examination disrupts the true in vivo tear composition12. Therefore, performing a comprehensive DED evaluation in a telehealth setting is impractical, warranting the formal appraisal of various telehealth strategies to remotely diagnose DED and manage its symptoms5,13,14.

The maximum blink interval (MBI), which is defined as the duration that participants can keep their eyes open before blinking during each trial, correlates positively with TFBUT15. MBI can be measured non-invasively, without contact, under observation with a slit-lamp microscope. The combined use of a slit-lamp microscope and DED-specific symptom questionnaire has shown a sensitivity and specificity of 75.4% and 92.9%, respectively, for DED diagnosis16. By eliminating the requirement for slit-lamp-based MBI measurements, MBI could replace TFBUT in remote settings to enable non-invasive and non-contact DED diagnosis and monitoring.

In November 2016, we developed and released an in-house smartphone application (app), DryEyeRhythm, which is capable of measuring MBI and administering DED-specific symptom questionnaires14,17,18 with positive and negative predictive values, sensitivity, and specificity of 91.3%, 69.1%, 50.0%, and 95.0%, respectively5. DryEyeRhythm could measure MBI by biosensing blinking using smartphone-attached cameras. Additionally, recent data on various DED subtypes suggest that MBI monitoring is useful in determining the disease mechanism for stratified and personalized treatment approaches18. The administration of DED-specific questionnaires through DryEyeRhythm provides reliable patient-reported data14,17; however, MBI measurement through the app must be assessed for reliability and validity.

Therefore, in this study, MBI data that were collected through the DryEyeRhythm app (app-based MBI) were formally evaluated for their validity, reliability, and equivalence to visually confirmed MBI.

Results

Participants’ characteristics

Figure 1 shows the enrollment process. This study initially included 94 participants. One patient was excluded due to refusal to participate after the MBI measurement. Among the 93 remaining participants, 10 (10.8%) were excluded owing to their inability to obtain app-based MBI measurements. MBI was unobtainable in 10 participants—on both platforms in 4 participants, on the iPhone operating system (iOS) in 2 participants, and on Android in 4 participants. Table 1 presents the characteristics of the 83 included participants. Supplementary Table S1 shows a comparison between the included and excluded individuals.

Figure 1
figure 1

Flowchart of the study indicating the number of participants at each. CFS, corneal fluorescein staining; TFBUT, tear film breakup time; MBI, maximum blink interval.

Full size image
Table 1 Participants’ characteristics.
Full size table

Internal consistency and agreement of app-based MBI

Table 2 shows the internal consistency of app-based MBIs, with Cronbach’s alpha coefficients and intraclass correlation coefficients (ICCs). The Cronbach’s alpha coefficients of the correlation between app-based and visually confirmed MBIs were above 0.7 in the iOS (0.999) and Android (1.000) versions. The ICCs (95% confidence intervals [CIs]) of the app-based (0.996 [0.994–0.998], iOS; and 0.998 [0.997–0.999], Android) and visually confirmed MBIs were above 0.7.

Table 2 Reliability of app-based MBIs using Cronbach’s alpha and intraclass correlation coefficient values.
Full size table

Discriminant validity of app-based and slit-lamp-based MBIs

Table 3 shows the discriminant validity of app-based and slit-lamp-based MBIs. All MBIs were significantly shorter in the DED than in the non-DED groups (app-based MBI [iOS], P = 0.021; visually confirmed MBI [iOS], P = 0.018; app-based MBI [Android], P = 0.028; and visually confirmed MBI [Android], P = 0.031).

Table 3 Discriminant validity of app-based and visually confirmed MBIs.
Full size table

Concurrent validity among app-based, visually confirmed, and slit-lamp-based MBIs

The concurrent validity among the app-based, visually confirmed, and slit-lamp-based MBIs was assessed using Pearson’s correlation analysis (Table 4). Significant positive correlations were identified between the app-based (iOS) and visually confirmed MBIs (iOS) (r = 0.999, P < 0.001), app-based (Android) and visually confirmed MBIs (Android) (r = 0.999, P < 0.001), and app-based (iOS) and app-based MBIs (Android) (r = 0.824, P < 0.001).

Table 4 Correlations among app-based, visually confirmed, and slit-lamp-based MBIs.
Full size table

Bland–Altman analysis

Bland–Altman analysis for agreement showed differences (biases) of − 0.08 s (Fig. 2a; 95% limits of agreement [LoA], − 0.76 to 0.60) between app-based (iOS) and visually confirmed MBIs (iOS), − 0.09 s (Fig. 2b; 95% LoA, − 0.63 to 0.45) between app-based (Android) and visually confirmed MBIs (Android), and -0.43 s (Fig. 2c; 95% LoA, − 7.98 to 7.13) between app-based (iOS) and app-based MBIs (Android) [− 0.88 s (Fig. 2d; 95% LoA, − 10.3 to 8.58) and -0.88 s (Fig. 2e; 95% LoA, − 9.70 to 8.80), respectively].

Figure 2
figure 2

Bland–Altman plot for the app-based and visually confirmed maximum blink interval (MBI). The x-axis of the Bland–Altman plot represents average MBI values, and the y-axis represents differences between two of the different MBI measurement methods. The central line indicates the mean difference (bias) between the MBI values, whereas the superior and inferior lines indicate the upper and lower 95% limits of agreement, respectively. Differences between app-based and visually confirmed MBIs are displayed for (a) iOS and (b) Android. Differences in app-based MBI based on the operating systems are shown in (c). Differences between app-based and slit-lamp-based MBIs are displayed for (d) iOS and (e) Android. MBI, maximum blink interval; iOS, iPhone Operating System.

Full size image

Sensitivity, specificity, and cut-off values of app-based MBIs for detecting DED and TFBUT ≤ 5 s

Table 5 shows the sensitivity, specificity, and cut-off values for detecting DED determined by app-based and slit-lamp-based MBIs. For app-based MBI (iOS), the sensitivity, specificity, and area under the curve (AUC) for detecting DED were 80.4%, 50.0%, and 0.651, respectively, with an optimum cut-off value of 10.5 s. For the app-based MBI (Android), the sensitivity, specificity, and AUC for detecting DED were 49.0%, 78.1%, and 0.644, respectively, with an optimum cut-off value of 7.0 s. For slit-lamp-based MBI, the sensitivity, specificity, and AUC for detecting DED were 80.4%, 43.8%, and 0.638, respectively, with an optimum cut-off value of 11.7 s. Supplementary Tables S2–S4 show the cut-off values, sensitivity, specificity, and Youden indices of the app-based (iOS), app-based (Android), and slit-lamp-based MBIs for detecting DED.

Table 5 Sensitivity, specificity, cut-off value, and AUC for detecting DED.
Full size table

Table 6 shows the sensitivity, specificity, and cut-off values for detecting a TFBUT of ≤ 5 s determined by app-based and slit-lamp-based MBIs. For the app-based MBI (iOS), the sensitivity, specificity, and AUC for detecting DED were 44.3%, 76.9%, and 0.519, respectively, with an optimum cut-off value of 6.8 s. For the app-based MBI (Android), the sensitivity, specificity, and AUC for detecting DED were 85.7%, 30.7%, and 0.543, respectively, with an optimum cut-off value of 16.8 s. For the slit-lamp-based MBI, the sensitivity, specificity, and AUC for detecting DED were 72.9%, 38.5%, and 0.540, respectively, with an optimum cut-off value of 11.7 s. Supplementary Tables S5–S7 show the cut-off values, sensitivity, specificity, and Youden indices of the app-based (iOS), app-based (Android), and slit-lamp-based MBIs for detecting a TFBUT of ≤ 5 s.

Table 6 Sensitivity, specificity, cut-off value, and AUC for detecting TFBUT of ≤ 5 s.
Full size table

Discussion

The unmet medical need for non-invasive, non-contact DED evaluation has drastically increased with the increase in DED prevalence due to aging and a digitalized society, as well as the COVID-19 pandemic. In this study, the performance of app-based MBI collected through the DryEyeRhythm app was compared with that of the slit-lamp-based MBI to determine its validity and reliability compared to the traditional technique. The results reflected the feasibility of the DryEyeRhythm app-based MBI in DED diagnosis and may help lay the foundation to implement digital phenotyping strategies and biometric data collection through mobile health (mHealth) apps. With a smartphone app for DED screening and management, DED care may become possible in a longitudinal, day-to-day manner with minimal invasiveness and no requirement to visit a specialized facility.

The synergy between mHealth and biometric data collection has been gaining attention owing to its potential for creating a comprehensive dataset on patient pathophysiology and elucidating new digital phenotypes with minimal intrusion18,19,20. Additionally, the push toward telemedicine has expanded substantially owing to the requirement of non-contact and non-invasive examinations during the COVID-19 pandemic9. However, robust evidence to support the implementation of mHealth- and app-based biometrics for clinical use is lacking21. In this study, we evaluated MBI biosensing using an image recognition app programming interface as part of a smartphone app to assist in DED diagnosis. Numerous approaches in incorporating mHealth and biosensing techniques have been used in ophthalmology, including visual acuity testing22, allergic conjunctivitis management23, pupillary reflex testing for amblyopia and strabismus detection24, and leukocoria recognition apps25. Their use is expected to expand with the rapidly increasing capabilities of commonplace smart devices and attached sensors. The unique advantage of mHealth can be attributed to its alignment with the paradigm shift from traditional facility-oriented medicine to non-intrusive, longitudinal care in a patient-centered manner18.

The results of this study demonstrate the validity, reliability, and equivalence of app-based MBI determination to its visually confirmed and slit-lamp-based counterparts. Good reliability values using Cronbach’s alpha coefficient and ICC were shown by both app-based and visually confirmed MBIs for iOS and Android platforms, reflecting sufficient internal consistency. App-based MBI also showed satisfactory discriminant validity and concurrent validity. Minimal biases were present between visually confirmed and app-based MBIs for both operating systems on Bland–Altman analysis. The discrepancy between the iOS and Android MBI measurements was minimal. Notably, the AUC of the app-based MBI for detecting decreased TFBUT was 0.519 and 0.543 for iOS and Android, respectively, possibly due to the temporal gap between app-based MBI and TFBUT measurements, which may be sufficient to affect the consistency of measurements. The demonstrated equivalence of app-based MBI with manually measured MBI and its reliability and validity suggest that app-based MBI measurements may be useful in obtaining an objective finding to support the diagnosis of DED in a telemedicine setting.

Two major criteria must be met to confirm a diagnosis of DED: subjective symptoms and objective clinical findings6,12. The Asia Dry Eye Society characterizes the pathophysiology of DED as a disease of tear film instability, which leads to visual decline12. TFBUT is a crucial component in assessing the tear film status26, and subjective symptoms quantified through disease-specific questionnaires alone are insufficient to make the diagnosis. However, specialized equipment and procedures (i.e., slit-lamp microscopy and fluorescein dye administration) are required to obtain TFBUT measurements, thus hindering remote assessment of DED status. Our previous efforts to find an appropriate substitute for TFBUT posited MBI as a possible candidate based on the positive correlation between the two measurements15. Additionally, the diagnostic performance of concomitant Japanese version of Ocular Surface Disease Index (J-OSDI) and MBI was satisfactory, with a sensitivity, specificity, and AUC of 75.4%, 92.9%, and 0.938, respectively16. The validity and reliability of the app-based J-OSDI were satisfactory, and its performance was comparable with its paper-based counterpart5,14. Therefore, accurate measurement of MBI using a smartphone app can enable comprehensive assessment of tear film status in a remote setting for DED diagnosis and progression monitoring. The results of this study demonstrate the validity, reliability, and equivalence of app-based MBI compared with the traditional measurement methods. Assisting DED diagnosis in a telemedicine environment may be possible by administering J-OSDI and measuring MBI using a smartphone app.

Our results indicate that the optimal cutoff values for app-based MBI were shorter, compared with the visually confirmed MBI. This discrepancy in MBI cutoff likely stems from the difference in visual tasks during various MBI measurements, with strong evidence supporting a significant decrease in blink rate and amplitude when using electronic monitors, such as smartphones and computers27. The observed decrease in optimal MBI cutoff for app-based measurements, compared with visually confirmed measurements, is thought to be affected by the specific visual task of focusing on handheld screens, which may ultimately elongate the physiological blink interval and subsequently the MBI when obtaining an app-based measurement. MBI cutoff values from prior studies were entirely derived under slit lamp-based measurements15,16. To enhance the assessment of different diagnostic capabilities of MBI and encourage its utilization on mobile platforms, future research should employ methodologies that effectively explore the optimal cutoff value for app-based MBI as a primary outcome10.

Previous efforts to screen for DED using web-based administered questionnaires lacked the objective component of DED diagnosis, such as TFBUT28,29. One notable strategy was to utilize an external infrared thermography device for smartphones, which showed satisfactory sensitivity, specificity, and AUC values of 96%, 91%, and 0.79, respectively30. However, specialized external devices are not ideal for screening purposes. DryEyeRhythm is an easily accessible software that can be executed by most commonplace smartphones to assess DED intermittently and longitudinally without the use of special devices or intrusive procedures. By administering the J-OSDI and measuring MBI through a single mHealth app, DryEyeRhythm, a comprehensive assessment of DED using remotely collected subjective and objective data on a patient’s tear film status may be possible.

This study has few limitations. First, it may have been affected by selection bias owing to its single-center design. The average TFBUT of the non-DED cohort participating in this study was 4.4 ± 2.5 s, lower than normal (range of normal TFBUT values: 7.6 ± 10.4 s to 9.1 ± 3.5 s)11,12,31,32. This may likely indicate that specialized university facilities are frequented by patients with various underlying ocular diseases that may affect tear film stability, and our sample may have included participants who may not accurately represent the larger population. Therefore, an ongoing multicenter, open-label, prospective, cross-sectional study is underway to determine the diagnostic ability of the smartphone app for DED and a cutoff value for app-based MBI10. Second, this study did not evaluate several factors associated with DED, such as socioeconomic status, education level, cultural background, lifestyle patterns, and systemic medications1. Third, as this study aimed to assess the reliability, validity, and equivalence of app-based MBI compared with traditional measurements, several objective findings were not evaluated, including Rose Bengal staining scores, tear osmolality, meibomian gland dysfunction, and corneal sensations. Lastly, this study excluded participants with ptosis or other palpebral dysfunctions that may physically disrupt normal blinking physiology. Therefore, the app may not accurately measure MBI in older patients with dermatochalasis. Additionally, as the blink recognition function of DryEyeRhythm can be hindered for users wearing a mask, the app-based MBI was measured with masks removed. Another factor that may affect the blink recognition function of DryEyeRhythm is the narrow palpebral fissure width of the participants33, due to which, 10 participants were unable to undergo app-based MBI measurements in this study. Future studies and updates of the app should focus on enhancing the recognition algorithm, aiming to eliminate the necessity for users to remove masks and adjusting for narrow palpebral fissure width.

In summary, MBI measured through DryEyeRhythm, an app available on iOS and Android platforms, showed good reliability, validity, and equivalence compared with slit-lamp-based MBI measurements, suggesting that app-based MBI could be a substitute for TFBUT in an mHealth environment. The results of this study indicate that DryEyeRhythm may serve as a novel tool for assisting in DED diagnosis and progression monitoring in a remote setting.

Methods

DryEyeRhythm smartphone application

The DryEyeRhythm smartphone app was initially developed using the open-source framework ResearchKit of Apple Inc. (Cupertino, CA, USA)17. The app was released in November 2016 for iOS and September 2020 for Android under a consignment contract with the Juntendo University Graduate School of Medicine, Tokyo, Japan, and InnoJin Inc., Tokyo, Japan. It is freely available on Apple’s App Store and Google Play.

The DryEyeRhythm app collects data regarding user demographics, medical history, lifestyle history, daily subjective symptoms, J-OSDI score, blink monitoring including blink frequency and MBI biosensor data, depression data (Zung Self-Rating Depression Scale), and work productivity4,5,8,17,18,34,35,36.

Study design and participants

This single-center, prospective, cross-sectional, observational study was conducted at the Juntendo University Hospital, Department of Ophthalmology, Tokyo, Japan37. Patients aged ≥ 20 years were recruited between February 16, 2022, and August 3, 2022. Written informed consent was obtained from all participants. This study was approved by the Independent Ethics Committee of the Juntendo University Faculty of Medicine (approval number: 20-092) in accordance with the Declaration of Helsinki.

Participants with a history of eyelid disorders, ptosis, psychiatric disease, Parkinson’s disease, or any other disease affecting blinking were excluded5,15. Those with any missing data and whose MBI measurements could not be obtained with the DryEyeRhythm app were also excluded.

Dry eye disease diagnosis

According to the 2016 Asia Dry Eye Society criteria12, participants with a TFBUT ≤ 5 s and J-OSDI ≥ 13 points were diagnosed with DED. The TFBUT was considered positive if the average was ≤ 5 s in a severely affected eye.

MBI

MBI was defined as the time that patients could keep their eyes open before blinking15. It was measured in three ways: using a slit-lamp microscope (slit-lamp-based MBI), DryEyeRhythm (app-based MBI [iOS and Android]), and a stopwatch (visually confirmed MBI). All MBIs were measured thrice. Slit-lamp-based MBI was calculated using a stopwatch under light microscopy. App-based MBIs were measured using the iOS and Android versions of the DryEyeRhythm smartphone app installed on an iPhone 12 Pro MAX (Apple Inc., Cupertino, CA, USA) and an Xperia 5 II (Sony Corporation, Tokyo, Japan) and their embedded cameras, with a face recognition technology called ARCore for the iOS and Android interface38. During the measurement of app-based MBIs, visually confirmed MBI was measured by the examiner by observing the user’s eyes with a stopwatch. The mean MBI was used in the analysis. Figure 3 shows a representative illustration (Fig. 3a) and screenshots of MBI measurement (Fig. 3b–e) using the DryEyeRhythm app.

Figure 3
figure 3

Illustration showing the process of DryEyeRhythm application-based maximum blink interval (MBI) measurement. (a) Representative illustration of MBI measurement using the DryEyeRhythm app. (b) Prior to the app-based MBI measurement, instructional information on the procedure is displayed to the user. (c) The user must align their face with the displayed guideline during the 3-s countdown before MBI measurement. (d) MBI measurement screen: measurements are obtained with the face aligned to the guideline. (e) Measurement completion screen: once the app detects a blink, the MBI is displayed to the user and the measurement is complete.

Full size image

Study procedures

Figure 1 shows the flowchart of this study. All participants underwent visual acuity measurement; intraocular pressure measurement; and several DED evaluations, including TFBUT, corneal fluorescein staining (CFS), and slit-lamp-based MBI. After completing these tests, the app-based and visually confirmed MBIs were measured simultaneously. A Schirmer test I was performed after the MBI measurement.

Clinical assessment of DED

TFBUT was measured using fluorescein sodium staining (Ayumi Pharmaceutical Co., Tokyo, Japan) according to standard methodology12. The mean values of three measurements in the right eye were used for the analysis. CFS was graded according to the van Bijsterveld grading score39, with a maximum potential score of 9. Schirmer’s test I was performed without topical anesthesia after completing all examinations.

Reliability

The internal consistency of the app-based MBIs was assessed using Cronbach’s alpha coefficient; an alpha value > 0.70 was considered acceptable40. ICC was used to evaluate the agreement among the slit-lamp-based, app-based, and visually confirmed MBIs. An ICC value ≥ 0.70 was considered acceptable41.

Validity

The discriminant validity of each MBI was evaluated by comparing the non-DED and DED groups. Concurrent validity was assessed by calculating correlations (Pearson coefficients) between each app-based MBI. Bland–Altman analysis42 was conducted to indicate systematic random error, heteroscedasticity of the data, and 95% LoA of each MBI.

Statistical analysis

The sample size was predetermined using the formula derived from hypothesis testing43. Using the following settings, the required sample size was determined to be 79 cases: power, 80%; significance level, 5%; minimal acceptable ICC score, 0.5; expected ICC score, 0.7; and number of raters, 2. Considering a dropout rate of 15% possibly owing to missing data, unmeasurable app-based MBIs, and withdrawal of consent, 94 participants were recruited.

The characteristics of study participants were compared using an unpaired t test for continuous variables and an χ2 test for categorical variables. Data were presented as mean ± standard deviation or percentage. Receiver operating characteristic curve analysis was performed to examine the diagnostic efficacy of MBIs in detecting DED or TFBUT ≤ 5 s. The AUC was estimated using the trapezoidal rule4. The cut-off values of MBIs for detecting DED and TFBUT ≤ 5 s were determined using the Youden index44. Statistical analyses were performed using STATA version 15 (StataCorp, College Station, TX, USA), and statistical significance was set at P < 0.05.