Face size biases emotion judgment through eye movement

Faces are the most commonly used stimuli to study emotions. Researchers often manipulate the emotion contents and facial features to study emotion judgment, but rarely manipulate low-level stimulus features such as face sizes. Here, I investigated whether a mere difference in face size would cause differences in emotion judgment. Subjects discriminated emotions in fear-happy morphed faces. When subjects viewed larger faces, they had an increased judgment of fear and showed a higher specificity in emotion judgment, compared to when they viewed smaller faces. Concurrent high-resolution eye tracking further provided mechanistic insights: subjects had more fixations onto the eyes when they viewed larger faces whereas they had a wider dispersion of fixations when they viewed smaller faces. The difference in eye movement was present across fixations in serial order but independent of morph level, ambiguity level, or behavioral judgment. Together, this study not only suggested a link between emotion judgment and eye movement, but also showed importance of equalizing stimulus sizes when comparing emotion judgments.

Most studies up to date focus on the diagnostic facial features for emotion judgment (e.g., 20 ) and investigate neural correlates of facial expressions by manipulating the emotion contents (e.g., 25 ). However, it remains unclear whether a simple low-level feature, face size, will influence emotion judgment. In this study, I investigated whether a mere difference in face size would cause different emotion judgments. I employed an emotion judgment task with fear-happy morphed faces. Indeed, when subjects viewed larger faces, they not only had a lower threshold to detect fear on the face, but also showed a higher specificity in emotion judgment. However, subjects showed a similar confidence judgment between face sizes. Concurrent eye tracking further provided insights into the underlying mechanism: more fixations were directed to the eyes when people viewed larger faces whereas there was a wider spatial dispersion of fixations when people viewed smaller faces. This difference was present across fixations in serial order, but independent of the morph level, ambiguity level, and behavioral judgment. Together, this study not only suggested a link between emotion judgment and eye movement, but also showed importance of choosing stimulus size to study emotion judgment.

Results
Emotion judgment. Subjects performed an emotion judgment task (Fig. 1A) with "anchor" (unambiguously happy or unambiguously fearful) and morphed faces (Fig. 1B). Psychometric curves were fitted for each subject (Eq. 1). The proportion of trials identified as fearful monotonically increased as a function of fearful level in the face (Fig. 1C). Two metrics from the fitted psychometric curves were used to compare emotion judgment. First, x half , the midpoint of the curve with equal proportions of fearful and happy judgment, shows emotion judgment bias. I found that subjects had a significantly smaller x half for large faces compared to small faces ( Fig. 1D; large: 48.2 ± 5.09 (mean ± SD), small: 51.3 ± 5.90; paired two-tailed t-test: t(22) = 3.65, P = 0.0014, effect size in Hedges' g (standardized mean difference): g = 0.56), suggesting that they were more likely to judge faces as fearful when viewing large faces. Second, α, the steepness of the psychometric curve, shows emotion judgment sensitivity and specificity. I found that subjects had a greater α for large faces ( Fig. 1E; large: 0.14 ± 0.061, small: 0.12 ± 0.045; t(22) = 2.29, P = 0.032, g = 0. 39), showing that they had steeper psychometric curves when viewing large faces, which in turn suggested that subjects were more sensitive and specific in their emotion judgment when they viewed large faces.
Together, my results show that different face sizes led to not only different thresholds for judging emotions, but also different sensitivity and specificity of emotion judgment.
Together, I found very similar patterns of confidence judgment between large vs. small faces, suggesting that face size did not influence confidence judgment. Different face sizes led to different spatial distributions of fixations. Could the difference in emotion judgment be attributed to difference in eye movement? To investigate this question, I next analyzed the spatial distribution of fixations. For each subject, I collapsed fixations from all trials during the 1 s stimulus period. I found that when subjects viewed both large and small faces, the fixation density distribution was symmetric along the horizontal dimension. However, I found a significant difference along the vertical dimension: the fixation density distribution for large faces shifted up towards the eyes, whereas the distribution for small faces remained around the center on the nose (Fig. 3A). Direct comparison at each pixel confirmed the difference along the vertical dimension but not the horizontal dimension (two-tailed paired t-test; green bar in Fig. 3A). Furthermore, fixation density maps (Fig. 3B) showed that large faces led to a narrower dispersion of fixations whereas small faces led to a wider dispersion. This was expected due to the smaller visual angle subtended by small faces-a fixation from the stimulus center with the same distance could travel farther on the stimulus. However, interestingly, when subjects viewed large faces, they not only had more fixations around the stimulus center, but also had more fixations towards the eyes (Fig. 3C). Direct comparisons with individual density maps (two-tailed paired t-test at each pixel, uncorrected with P < 0.05; Fig. 3D) further confirmed such upward shift of fixations. It is worth noting that this finding is consistent with the behavioral results (i.e., increased fearful judgment with large faces): previous literature has shown that people fixate on eyes 19 and utilize eye information 20 for fear judgment. It is worth noting that valid recording durations per trial were similar between large and small faces (large: 920 ± 64.7 ms, small: 934 ± 46.4 ms; two-tailed paired t-test: t(23) = 1.23, P = 0.23, g = 0.25), suggesting that my results could not be attributed to different qualities in eye tracking. Moreover, subjects had more fixations per trial for large faces (3.48 ± 0.70) compared to small faces (2.77 ± 0.73; t(23) = 6.64, P = 8.87 × 10 −7 , g = 0.98), suggesting that subjects had more eye movements when they viewed large faces.
Together, I found that different face sizes resulted in different patterns of eye movement, which in turn led to different emotion judgments.  In conclusion, ROI analysis showed that subjects had more fixations onto eyes when they viewed large faces. However, small faces featured more fixations outside ROIs as well as longer fixations.
ROI analysis by fixation serial order. The results shown above used all fixations from the trial. However, were there differences at particular individual fixations? To answer this question, I next analyzed fixation densities in serial order ( Fig. 5; three-way repeated-measure ANOVA of face size X ROI type X fixation order; main effect of ROI type: F(3,678) = 41.3, P = 1.56 × 10 −24 ; interaction between face size and ROI type: F(3,678) = 10.9, P = 5.36 × 10 −7 ; interaction between ROI type and fixation order: F(9,678) = 6.73, P = 2.86 × 10 −9 ; all other Ps > 0.05). For both large and small faces, subjects started from the face center ( Fig. 5C; due to the central cross preceding the face) and then looked at the eyes (Fig. 5A) and mouth (Fig. 5B). Therefore, the fixation density decreased for the center ROI ( Fig. 5C; comparing fixation 1 to fixation 2-4: two-tailed paired t-test: large: all Ps < 0.001; small: all Ps < 0.05), and increased for the eye ROI ( Fig. 5A; large: Ps < 0.05 between fixation 1 and fixation 2, 4; small: P < 0.05 between fixation 1 and fixation 4) and mouth ROI ( Fig. 5B; large: Ps < 0.05 between fixation 1 and fixation 2-4; small: P < 0.05 between fixation 1 and fixation 3). Notably, when comparing large vs. small faces, the difference in the eye ROI was not only at a particular fixation but it was across fixations. Similarly, across fixations, there were only a small proportion of fixations outside ROIs for large faces but the proportion was significantly larger for small faces (Fig. 5D), confirming the wider dispersion of fixations for small faces. Together, this temporal analysis of fixations showed that the differences between large and small faces were not restricted to a particular fixation in serial order but was across fixations.
Fixation density across morph levels, ambiguity levels, and emotion judgments. In this last section, I analyzed whether fixation density was modulated by stimulus level and/or emotion judgment. First, for both large and small faces, fixation density maps did not vary as a function of morph levels ( Fig. 6A; three-way repeated-measure ANOVA of face size X ROI type X morph level; main effect of ROI type: F(3,1222) = 99.5, P = 1.31 × 10 −57 ; interaction between face size and ROI type: F(3,1222) = 22.1, P = 6.07 × 10 −14 ; all other Ps > 0.05). Direct comparisons between large and small faces in each ROI (Fig. 6B) showed a significant difference in the eyes (all Ps < 0.005) but not in the mouth or center, with large faces having more fixations onto the eyes and small faces having more fixations outside ROIs. Furthermore, for both large and small faces, fixation density maps did not vary as a function of ambiguity levels ( Fig. 6C; three-way repeated-measure ANOVA of face size X ROI type X ambiguity level; main effect of ROI type: F(3,498) = 43.1, P = 9.25 × 10 −25 ; interaction between face size and ROI type: F(3,498) = 9.52, P = 4.02 × 10 −6 ; all other Ps > 0.05). I next analyzed fixation densities separately for each behavioral judgment ( Fig. 6D; three-way repeated-measure ANOVA of face size X ROI type X judgment; main effect of ROI type: F(3,318) = 28.4, P = 2.59 × 10 −16 ; interaction between face size and ROI type: F(3,318) = 6.14, P = 4.51 × 10 −4 ; all other Ps > 0.05). Again, similar patterns of fixation densities were found and large faces attracted more fixations to the eyes.
Together, I found that more fixations were directed to the eyes when people viewed larger faces whereas there was a wider spatial dispersion when people viewed smaller faces. However, this difference was independent of the morph level, ambiguity level, and behavioral judgment.

Discussion
In this study, I employed an emotion judgment task to study whether a difference in stimulus size could influence emotion judgment. I also conducted concurrent eye tracking to reveal the underlying mechanism. I found that face size influenced the threshold to report an emotion as well as specificity in emotion judgment, but not confidence judgment. Eye movement data showed that larger faces attracted more fixations to the eyes whereas smaller faces featured a wider spatial dispersion of fixations. This difference was present across fixations in serial order, and was present at all morph levels, ambiguity levels, and behavioral judgments. Therefore, I demonstrated that face size could bias emotion judgment through eye movement.
I found that large faces resulted in increased judgment of fear and attracted more fixations towards the eyes. Consistent with this finding, it has been shown that the eye region attracts more fixations when people view fearful faces whereas the mouth region attracts more fixations when people view happy faces 19 . Also, eyes contain more information for fearful faces but the mouth contains more information for happy faces 20 . The present study not only revealed an impact of stimulus size on emotion judgment, but also suggested a link between emotion judgment and eye movement, which might explain the underlying mechanism for such modulation by stimulus size. To explore a more direct relationship between emotion judgment and eye movement, I found that the difference in slope (α) was marginally significantly correlated with the difference in fixation density in the eye ROI (r = 0.36, P = 0.088), indicating that the more fixations onto eyes, the more sensitive the emotion judgment. However, this curious relationship was still primitive and limited by statistical power. Future studies with a larger sample size and/or a more sensitive task will be needed to show a direct relationship between emotion judgment and eye movement under the modulation of stimulus size.
It is worth noting that visual acuity was normal or corrected-to-normal for all subjects, and the small faces were still large enough (5.95° × 5.95°) so that the details of the faces were clearly visible to subjects. Therefore, the difference in emotion judgment and eye movement between large and small faces was not likely due to differences in low-level stimulus features and visual processing. This was further supported by comparable RTs and confidence ratings between face sizes (Fig. 2). Furthermore, fewer fixations attracted to the eye region for small faces might be due to that when viewing small faces, the eye region was within the foveal or parafoveal area so that upward saccades were not necessary. However, foveation could not fully explain my result, because (1) in contrary to the idea that fewer saccades were needed to sample facial ROIs, small faces did not have more fixations in the image center (note that faces were preceded by a central fixation) but had more distributed fixations compared to large faces ( Fig. 3 and Fig. 4A), even outside the ROIs (Fig. 4A), suggesting that subjects still made saccades (including upward saccades) to sample the faces. Notably, subjects had comparable fixations onto the mouth (Fig. 4A), again showing that subjects still saccaded to facial ROIs when viewing small faces. (2) Importantly, large faces had asymmetric fixation densities along the vertical axis, with more fixations directed upwards to the eyes (34.1 ± 20.8% (mean ± SD)) but fewer downwards to the mouth (21.2 ± 15.7%), whereas small faces had a rather symmetric vertical distribution, with similar fixation densities for the eyes (23.3 ± 18.5%) and mouth (24.7 ± 14.3%; Fig. 3 and Fig. 4A). The difference in fixation density between the eyes and mouth further confirmed this finding (large: 12.9 ± 34.8%, small: −1.47 ± 30.0%; two-tailed paired t-test: t(23) = 3.15, P = 0.0045, g = 0.44). Therefore, the difference in fixations onto eyes between small and large faces was primarily due to selective upward saccades when viewing large faces, a psychological process probably related to distance (see below), rather than a simple consequence of foveation. Notably, such upward saccades towards the eyes can be in turn attributed to amygdala activation 21 .
Stimulus size is an important factor in optical 26,27 , visual arts 28 , and psychophysiological 29,30 research. In one very early study, it has been showed that, in general, medium size circles are preferred over either larger or smaller circles 31 . Later studies using art works showed that the size-distance relationship influences judgments of preference and interestingness as well as the duration of looking time 28 . Stimulus size directly affects retinal projection and it is in turn affected by viewing distance. However, it has been shown that perceived distance and stimulus size are judged quite independently from the retinal and extra-retinal information, in the sense that no attempt is made to attain consistency 32 . Psychological studies have shown that stimulus size influences motion aftereffect 33 and affects attention 34 . Physiologically, the polarity and saturation profile of the visual evoked potential change as     a function of stimulus size 30 . Furthermore, the amplitude of the visual gamma-band response is diminished for small and peripheral stimuli 35 . In particular, although significant emotional modulation of event-related potentials is observed across stimulus sizes at both earlier and later stages of processing, the modulation of earlier processes is reduced in smaller compared to larger stimulus sizes 36 . Consistent with this result, when viewers respond to video images from television and film that display different emotions, the largest stimulus produces greater heart rate deceleration as well as greater skin conductance than the medium and small stimuli, suggesting a more pronounced emotional response for larger stimuli 37 .
Larger stimuli indicate a smaller distance between the observer and the encountered stimuli. Therefore, a plausible explanation of the present result is that larger faces indicate a smaller interpersonal distance. Consistent with the lower threshold to report fear with larger faces in the present study, it has been shown that perspective distortion from interpersonal distance is an implicit visual cue for social judgments of faces: photographs of faces taken from within personal space elicit lower investments in economic trust game and lower ratings of social traits (such as trustworthiness, competence, and attractiveness), compared to photographs taken from a greater distance 38 . On the other hand, the lower threshold to report fear with larger faces might indicate that for a given level of fear-happy morph, subjects perceived more fearful emotion with large faces, consistent with larger emotional responses elicited by larger (thus indicated nearer) stimuli 36 . Relatedly, snake-phobic people show a linear increase of autonomic responses and self-reported fear as a function of distance to snakes 39 . Because distance and retinal size are strictly related 40 , it can be expected that changes in stimulus size determine arousal modulations similarly to distance. This is thus consistent with the increased judgment of fear with larger faces in the present study.
Using movie shots, it has been shown that smaller faces take longer to categorize the valence of facial expressions than those that are larger, and more clutter creates crowding and impedes the interpretation of expressions for more distant faces but not proximal ones 41 . This indicates an attentional mechanism underlying emotion judgment related to stimulus size, consistent with the result from computational modeling showing that object-level saliency (including face size) contributes to attract more fixations to faces 5 . Neural processing of facial expressions of emotions appears to require attention 42 ; and therefore another plausible explanation of the size effect might be that different stimulus sizes elicited different attention. Although in the present study I found that RT was comparable between large and small faces (Fig. 2), this might be due to that my subjects were only allowed to respond after the offset of the stimulus, therefore, they might make their decisions well before executing the button press. The task was not fully speeded because it was designed to better dissociate perception and decision; however, a fully speeded version of the task can well replicate the emotion judgment result 43 .
In the past decades, there have been numerous studies investigating facial expressions of emotions. However, most studies investigating emotion judgment focus on the emotion contents as well as the facial features related to emotions, but rarely investigate whether a simple low-level feature, face size, can affect emotion judgment. In this study, I revealed not only a difference in emotion judgment, but also a possible attribution to eye movements. The present result implies that future studies investigating emotion judgment should consider stimulus size carefully, especially for comparisons between different subject groups and investigations across different research sites. A future direction is to test whether the present result can be extended to other facial emotions and more complex social traits such as trustworthiness, dominance, and guilt. It also remains to investigate the neural mechanisms underlying the modulation by stimulus size.

Methods
Subjects. There were 24 subjects (16 female, 22.3 ± 3.39 years). Subjects gave written informed consent according to protocols approved by the institutional review board of the South China Normal University, and all methods were carried out in accordance with the approved guidelines. Visual acuity was normal or corrected-to-normal for all subjects. One subject was excluded from emotion judgment analysis because the psychometric function could not be fitted.
Task and stimuli. I used an established task and stimuli to study emotion judgment 15,[43][44][45] . Subjects viewed a face for 1 second and were asked to report their judgment of the facial emotion (fearful or happy) as quickly as possible (Fig. 1A). Subjects had to respond within 2 seconds after stimulus offset. Following emotion judgment, subjects were also asked to report their confidence in their emotion judgment (3 levels: very sure, sure, and unsure). There was no correct answer to the emotion judgment or confidence judgment (purely subjective), and no feedback was thus provided to subjects. After confidence judgment, a central cross was displayed for 1 to 2 seconds before the next trial started.
SCientifiC REPORtS | (2018) 8:317 | DOI:10.1038/s41598-017-18741-9 Psychometric curve. I used a logistic function to fit smooth psychometric curves (Fig. 1C): x is the stimulus level, P is the proportion of trials of fearful judgment, P inf is the curve's maximum value when x approaches infinity, x half is the curve's midpoint (i.e., symmetric inflection point), and α is the slope (steepness) of the curve. The parameters P inf , x half , and α were derived from the observed data (P and x) for each subject, and x half and α were used to compare emotion judgment. Specifically, x half shows emotion judgment bias and α shows emotion judgment sensitivity and specificity.
Eye tracking. Two eye trackers were used in this study. Fourteen subjects were recorded using an EyeLink 1000 System (SR Research, Canada) and ten subjects were recorded using a Tobii T120 system. EyeLink tracked one of the eyes at 1000 Hz and Tobii tracked both eyes at 120 Hz. In both experiment setups, MATLAB with the Psychophysics Toolbox 46 was used and the viewing distance was approximately 60 cm. Calibration was performed at the beginning of each block. In experiments with EyeLink, fixations and saccades were extracted using the software supplied with EyeLink (deflection threshold = 0.1°, velocity threshold = 30°/s, and acceleration threshold = 8000°/s 2 ). In experiments with Tobii, fixations and saccades were extracted using Tobii Fixation Filter 47 implemented in Tobii Studio (velocity threshold = 35 [pixels/samples] and distance threshold = 35 [pixels]). Rectangular ROIs were drawn to encompass 3 facial features: eyes, mouth, and center ( Fig. 3B; note that eye ROI has the same size as mouth ROI). Fixation density maps were derived by smoothing fixation locations using a 2D Gaussian kernel (size = 40 pixels, SD = 10 pixels) and were then normalized within each subject. Fixation density maps represent the likelihood of fixating a particular location of the stimulus and are shown in arbitrary units.

Statistics.
Because each subject viewed both large faces and small faces, two-tailed paired t-tests and repeated-measure ANOVAs were used throughout the analyses. Specifically, paired t-tests were used to compare the indices of emotion judgment (Fig. 1), and one-way or two-way repeated-measure ANOVAs were used to compare confidence judgments (Fig. 2). To compare the spatial distribution of fixations (Fig. 3A), point-by-point two-tailed paired t-test was used and further corrected by false discovery rate (FDR) 48 . To compare fixation density maps (Fig. 3D), pixel-by-pixel two-tailed paired t-test was used, but it was uncorrected for multiple comparisons. In all ROI analyses (Figs 4-6), ANOVAs with multiple factors were first performed, followed by post-hoc two-tailed paired t-tests between face sizes for each ROI to further confirm the findings. MATLAB was used to compute statistics. Detailed statistical measures were also specified before each result.