Use of eye tracking to improve the identification of attention-deficit/hyperactivity disorder in children

Attention-deficit/hyperactivity disorder (ADHD) is the most common neurodevelopmental disorder of childhood. Although it requires timely detection and intervention, existing continuous performance tests (CPTs) have limited efficacy. Research suggests that eye movement could offer important diagnostic information for ADHD. This study aimed to compare the performance of eye-tracking with that of CPTs, both alone and in combination, and to evaluate the effect of medication on eye movement and CPT outcomes. We recruited participants into an ADHD group and a healthy control group between July 2021 and March 2022 from among children aged 6–10 years (n = 30 per group). The integration of eye-tracking with CPTs produced higher values for the area under the receiver operating characteristic (AUC, 0.889) compared with using CPTs only (AUC, 0.769) for identifying patients with ADHD. The use of eye-tracking alone showed higher performance compare with the use of CPTs alone (AUC of EYE: 0.856, AUC of CPT: 0.769, p = 0.029). Follow-up analysis revealed that most eye-tracking and CPT indicators improved significantly after taking an ADHD medication. The use of eye movement scales could be used to differentiate children with ADHD, with the possibility that integrating eye movement scales and CPTs could improve diagnostic precision.

Study procedure.Participants and caregivers completed a demographic/health questionnaire and Korean versions of the ADHD Rating Scale (ARS) and Child Behavior Checklist (CBCL).To minimise external distractions, participants were then moved to a separate room with a computer, where they underwent calibration for the eye tracker and comprehensive attention testing (CAT) while their eye movements were tracked.Given the potential effect of medication on ADHD symptoms, together with the possibility of fatigue, all testing took place in the morning or early afternoon.We repeated the CAT once during follow-up after patients had received medical treatment for ADHD.
Assessment tools.Attention and psychopathology.Attention was evaluated with the ARS, an 18-item scale developed by DuPaul (1991) for use with children 16 .Symptoms are rated on 4-point Likert-type scales, ranging from 0 (never) to 3 (very often).The Korean version of ARS has internal consistency ranging from 0.77 to 0.89 and test-retest reliability of 0.85 17 .
Psychopathology was assessed with the CBCL, which contains 120 behavioural items that parents rate on 3-point Likert-type scales from zero to two (Not True to Very True/Often True).Items are summed to yield a syndrome scale score across three dimensions (internalising, externalising, and total behavioural problems) and six DSM-oriented scale scores.The syndrome and DSM-oriented scales have been validated 18 , and the Korean version of the CBCL was standardised in 1997 19 .
Computerised CAT .Computerized CAT is the type of the computer-based CPT, and has been developed for ages 4-49 20 .The CAT is composed of six subtests: the simple selective attention (visual and auditory), continuous inhibition, interference selection, divided attention, and working memory tests.However, we excluded the auditory test for simple selective attention to allow comparison with the eye-tracking test, together with the divided attention and working memory tests that are only used from age 9 years.The CAT was performed using a computer, with participant understanding of text and voice guides presented at the start of each subtest checked by trained researchers.
Overall, the amended CAT took approximately 25 min to complete, including the assessments of selective visual attention (300 stimuli, 10 min), continuous inhibition (300 stimuli, 10 min), and interference selection (150 stimuli, 5 min).For the selective attention test, participants press the space bar button quickly when they see a circle figure at the center of a monitor.For the continuous inhibition test, they press the space bar when they see any figure except an X at the center of monitor.For the interference selection test, participants are instructed to pay attention to a central target while ignoring interference stimuli.Each subtest has five indicators: commission errors (CE), for the number of wrong responses; omission errors (OE), for the number of missed responses; mean reaction time (RT mean), for the average response time to the stimuli; standard deviation of reaction time (RT SD), for response time variability; and sensitivity coefficient (d′), for how successfully the target stimuli are differentiated from the non-target stimuli.Because only four indicators were calculated in eye-tracking, we excluded d′ from the comparison.
Eye-tracking apparatus and eye movement measures.Stimuli were presented on a Samsung Notebook (NT551XCJF-COM) with a 15.6-inch display, a screen resolution of 1920 × 1080 pixels, and an eye-to-screen viewing distance of approximately 50 cm.The eye-tracking apparatus (Happymind Inc.CAT test) included a host PC that tracked and computed the participant's gaze position, as well as a display PC to present the stimuli.After downloading and running the eye-tracking programme (SeeSo; https:// visual.camp/ demo-archi ve/), eye movements were recorded at a 30 Hz sampling rate with an approximate accuracy of 1.7° (VisualCamp Co., Ltd, Seoul, Korea).Calibration to each participant in SeeSo used a five-point procedure.Online Supplementary Fig. S1 shows the graphical user interface and gaze coordinate of the eye-tracking programme.
To compare the extent of visual attention directed to the task and irrelevant regions, the participants' field of view was divided into central and peripheral areas of interest (AOIs).As shown in online Supplementary Fig. S2, the central AOI represented the middle third of the width and length.Each subtest had four indicators: fixation ratio (FR), for the ratio of gaze fixation; mean fixation time (FT), for the average gaze fixation time to the screen; central gaze ratio (CR), for the central AOI gaze ratio; and standard deviation of gaze coordinates (gaze SD), for gaze variability.The equations used are presented in online Supplementary Fig. S3.

Sample size. NCSS PASS (version 14
) was used for the sample size calculation 21 .A recent study of eyetracking among patients with ADHD showed that the ratio of center gaze duration between patients with ADHD (80.48%) and a healthy control group (88.35%) differed significantly according to Welch's unequal variance t-test 14 .Therefore, allowing for a 5% probability of a type 1 error and a power of 80%, the minimum sample size was 29 participants in each group.Considering drop out, we decided a total sample size of 30 in each group.

Statistical analysis.
We compared baseline characteristics, ARS, CBCL, CAT indicators, and eye-tracking indicators between the ADHD group before medications and the control group by independent-sample t-tests and chi-square analyses for parametric and non-parametric variables, respectively.Welch's unequal variance t-test was used when data failed to meet the assumption of variance homogeneity.Group differences in gaze were visualised using the gaze coordinates in subtests.
Pearson's correlation between CAT and eye-tracking indicators was evaluated before performing the regression analyses.Using the correlation matrix, we considered that indicators with r-values of > 0.7 had multicollinearity 22 , which we evaluated further based on a variance inflation factor (VIF) of < 5 23 .Logistic regression then assessed the ability of the CAT indicators, eye-tracking indicators, or both indicators combined to identify group membership (control or ADHD).Sensitivity, specificity, and area under the curve (AUC) were compared against patients with ADHD by receiver operator characteristic (ROC) curve analysis.The method reported by DeLong et al. was used to compare AUC values 24 .
In the secondary analysis, we used paired t-tests to assess the change in ARS, CBCL, and indicators (CAT and eye-tracking) within the medication group from before to after taking medication.Differences in gaze from before to after taking medication were visualised by using gaze coordinates according to subtests.
Statistical significance was evaluated at the 5% significance level (p < 0.05), and all analyses were performed using R (version 4.1.0)and its open-source statistical packages.
Ethical approval.This study was approved by the Ajou University Hospital Institutional Review Board (AJIRB-MED-SUR-21-240), and All participants and caregivers provided written informed consent.All the experiment protocol for involving human data was in accordance with the guidelines of Declaration of Helsinki.

Results
Participants and baseline characteristics.In total, we included 30 children with a diagnosis of ADHD and 39 healthy controls, before excluding 9 participants from the control group (Fig. 1).Among patients with ADHD, 16 of the 30 (53%) were combined type, 11 (37%) were inattentive type, and 3 (10%) were hyperactive type.Of the 30 patients with ADHD, 21 (70%) reported use of ADHD medications and 9 (30%) reported no ADHD medications.9 patients also take ADHD medications for follow-up analysis, resulting in follow-up data for 30 patients [n = 26 with stimulant medication only (methylphenidate), n = 3 with non-stimulant medication only (atomoxetine), and n = 1 with combined medication (methylphenidate and clonidine)].Among 21 patients who stopped their medication for the experiment, 3 of them complained of irritability.The physician explained www.nature.com/scientificreports/ the symptoms to the patients and their caregivers, and the symptoms disappeared after the patients restarted their medication.
The ADHD and control groups did not differ significantly by age, sex, height, weight, or main caregiver (Table 1).However, the ADHD group had lower parental education (p = 0.002 in maternal education, p < 0.001 in paternal education, respectively).

Comparisons of assessments between ADHD patients and healthy controls.
Table 1 shows that patients with ADHD had increased scores on the ARS overall (p < 0.001) and both the inattentive (p < 0.001) and hyperactivity (p < 0.001) domains.Patients with ADHD also had significantly higher scores on the syndrome and DSM-5 domains (except for somatic problems) of the CBCL.Concerning the CAT indicators, patients with ADHD performed worse than healthy controls in simple selective attention OE (p = 0.005), simple selective attention CE (p = 0.022), simple selective attention RT SD (p < 0.001), continuous inhibition RT mean (p = 0.048), continuous inhibition RT SD (p = 0.010), and interference selection RT SD (p = 0.043).Although not included in the logistic regression, there were also differences in dʹ (online Supplementary Table S1).Patients with ADHD performed worse than healthy controls in simple selective attention dʹ (p = 0.006) and continuous inhibition dʹ (p = 0.012).
Significant group differences existed for most eye-tracking indicators.Compared with controls, the ADHD group showed less fixation in simple selective attention (ratio, p < 0.001; time p = 0.002), continuous inhibition (ratio, p < 0.001; time, p = 0.028), and interference selection (ratio, p < 0.001; time, p = 0.009).The ADHD group also showed less central gaze in simple selective attention (p < 0.001), continuous inhibition (p < 0.001), and interference selection (p = 0.001).Moreover, patients with ADHD had increased gaze variability in the interference selection test (p = 0.028).
Figure 2 presents the group differences in gaze and fixation time between the study groups for the simple selective attention test.Compared with the control group, patients with ADHD demonstrated less central gaze and shorter fixation times.The other subtests revealed similar patterns between these groups (online Supplementary Fig. S4 and S5).
Identification of patients with ADHD.Among the CAT indicators, five were excluded due to multicollinearity and the seven remaining indicators (i.e., simple selective attention OE, simple selective attention CE, simple selective attention RT mean, continuous inhibition CE, continuous inhibition RT sd, interference selection OE, and interference selection RT sd remained) had VIF values of < 5.Among the eye-tracking indicators, eight were excluded due to multicollinearity and the four remaining indicators (i.e., simple selective attention FR, simple selective attention gaze SD, continuous inhibition FT, and interference selection CR remained) had VIF values of < 5.These results are summarised in online Supplementary Table S2 and S3.
Logistic regression using the CAT indicators showed a high specificity (0.931) and AUC (0.769), but a low sensitivity (0.533) (Table 2, Fig. 3, and online Supplementary Table S4).By contrast, logistic regression with the eye-tracking indicators showed high sensitivity (0.733), specificity (0.861), and AUC (0.856) values.Finally, logistic regression using both indicators revealed a sensitivity of 0.833, with a specificity of 0.862 and an AUC of 0.889.Significant differences did exist between the CAT indicators alone and eye-tracking indicators alone (p = 0.029) and between the CAT indicators alone and both indicator sets combined (p < 0.001).
Comparisons of assessments within ADHD patients from before to after taking medication.Treatment with ADHD medications was associated with an overall improvement in the ARS and both the CAT and eye-tracking indicators (Table 3).The ARS total (p = 0.019) and hyperactivity (p = 0.025) scores decreased significantly within the medication group from before to after taking medication.Significant improvements were from before to after taking medication for all CAT indicators (except simple selective attention OE, continuous inhibition CE, and interference selection CE) and all eye-tracking indicators (except simple selective attention gaze SD and continuous inhibition gaze SD).However, we observed no significant change in CBCL scores within the medication group from before to after taking medication.
Similar to the results of the primary analysis, we also found differences in the simple selective attention test (gaze and fixation time) within the medication group from before to after taking medication.Other subtests showed similar patterns (Fig. 1, online Supplementary Fig. S4 and S5), but the differences between groups were less obvious than in the primary analyses.

Discussion
Integrating eye-tracking with CPTs improved task performance at identifying ADHD compared with the use of CPTs alone.The use of eye-tracking alone also showed higher performance compare with the use of CPTs alone.Moreover, most eye-tracking indicators (e.g., FR and time, gaze ratio at the center, and gaze variability) differed significantly between the ADHD and control groups.Follow-up analysis of the effect of medication revealed that most eye-tracking and CPT indicators improved significantly with treatment.
Despite the limited research on eye-tracking applications in patients with ADHD, existing studies have shown the potential for them to discriminate between patients with ADHD and healthy controls 14 .For example, Elbaum et al. reported that eye-tracking had performance comparable to that of CPTs 25 , consistent with our results that the AUCs for eye-tracking was significantly higher than that of CPTs.In addition, while CPTs had a low sensitivity and a high false negative rate (47%), eye-tracking had a relatively high sensitivity and a low false negative rate (26%).
Furthermore, our findings are consistent with those of a study with a similar methodology that demonstrated improved discriminatory performance after the integration of eye-tracking 14  www.nature.com/scientificreports/duration with and without a distractor, we also showed that patients with ADHD had a lower CR than healthy controls.Estimating central gaze in this way could show the distractibility that underpins higher-order deficits in ADHD 26 .Another study revealed that evaluating eye movement distractibility could be used as a diagnostic tool for ADHD 27 .We used different indexes to estimate distractibility (e.g., fixation time, FR, and gaze variability) and show that patients with ADHD often had lower fixation ratios and times.Elsewhere, Caldani et al. also demonstrated poor fixation in youths with ADHD compared with healthy controls 28 .Moreover, our finding of increased gaze variability in patients with ADHD is consistent with previous research showing that ADHD was associated with difficulties in suppressing exploratory saccades compared with healthy controls 28,29 .These findings are in line with the concept that children with ADHD exhibit poor inhibitory control.However, our approach produced superior differentiating performance (AUC, 0.889) than the previous study that used central gaze duration only (AUC, 0.826) 14 , possibly due the inclusion of additional distractibility variables.Overall, our findings confirmed that using eye movement indicators, alone or in combination, have the potential to improve ADHD case identification.
Analyzing the effect of medication during the follow-up among patients with ADHD produced less dramatic results than the comparisons with healthy controls.However, we did observe a significant difference in most eye-tracking variables by the presence of drug treatment.These results suggest that eye-tracking may be suitable for both diagnosing ADHD and identifying treatment response to medications.
A distinct advantage of our methods is the simplicity of the technical setup, which only required the downloading of eye-tracking software to a laptop computer.This could improve the translation of our findings to clinical practice and other settings.Eye-tracking systems could also be used in brief sessions to assess different tasks and treatment effects 15 .However, important limitations warrant further consideration.
First, we included participants aged 6-10 years old.Although research has indicated that ADHD symptoms differed significantly between these ages 30,31    group 32 .Complementing this, we found no age differences between the ADHD and control groups.Second, this study used the relatively small sample size.However, our study included an appropriate sample size considering the previous study.Third, we did not evaluate other methods that can be integrated with eye-tracking.For example, Stolicyn et al. combined measures of eye and face movement during cognitive performance to predict depression symptoms 33 , whereas Fernandez-Ruiz et al. performed the antisaccade task during an fMRI study 34 .
We focused on integrating eye-tracking with CPT because these tests are easy to implement in practice.Fourth, we did not evaluate the difference in effect according to the drug type.Atomoxetine can have an onset of action within 1-2 weeks of starting treatment, while methylphenidate starts working within hours 35 .However, of the 30 patients with ADHD, 26 were on methylphenidate and only 4 were on atomoxetine, so we could not analyze them separately.In addition, since the drug-taking patterns of ADHD patients in this study were similar to those of ADHD patients in the Korean sample data 36 , it can be seen as a reflection of actual patients.
In conclusion, the present study indicates that eye-tracking during CPTs can improve the identification and classification of children with ADHD by uncovering reductions in gaze fixation and central gaze, together with increases in gaze variability.These findings suggest that eye-tracking could be a feasible option for screening and testing patients with ADHD.Given this potential, prospective research should now include larger samples with equal sex distributions to compare different tasks.

Figure 1 .
Figure 1.Study flowchart of children aged 6-10 years with or without ADHD.

Figure 2 .
Figure 2. Comparison of gaze distribution in the selective attention test between the ADHD and control groups and within the ADHD group with or without medication.(A) Distribution of heat maps for gaze distribution.(B) Distribution of gaze fixation times.ADHD, attention-deficit/hyperactivity disorder.

Figure 3 .
Figure 3. ROC curves for models identifying patients with ADHD.The AUCs for computerised CAT, eyetracking, and both combined are compared to assess performance.ADHD attention-deficit/hyperactivity disorder; AUC, area under the receiver operating characteristic curve; CAT, comprehensive attention test; ROC, receiver operating characteristic.

Table 1 .
, other research has considered ages 6 to 10 years as the same age Descriptive statistics and statistical analyses for ADHD patients and healthy controls.ARS, attentiondeficit hyperactivity disorder (ADHD) rating scale; CBCL, Child Behavior Checklist; CAT, comprehensive attention test; OE, omission errors; CE, commission errors; RT mean, mean reaction time; RT sd, standard deviation of reaction time; FR, fixation ratio; FT, mean fixation time; CR, central gaze ratio; Gaze sd, standard deviation of gaze coordinates.

Table 2 .
Results of identification of patients with ADHD.AUC, area under the receiver operating characteristic curve.

Table 3 .
Comparisons of statistical analyses within the medication group from before to after taking medication.ARS, attention-deficit hyperactivity disorder (ADHD) rating scale; CBCL, Child Behavior Checklist; CAT, comprehensive attention test; OE, omission errors; CE, commission errors; RT mean, mean reaction time; RT sd, standard deviation of reaction time; FR, fixation ratio; FT, mean fixation time; CR, central gaze ratio; Gaze sd, standard deviation of gaze coordinates.