Visual salience is affected in participants with schizophrenia during free-viewing

Abnormalities in visual exploration affect the daily lives of patients with schizophrenia. For example, scanpath length during free-viewing is shorter in schizophrenia. However, its origin and its relevance to symptoms are unknown. Here we investigate the possibility that abnormalities in eye movements result from abnormalities in visual or visuo-cognitive processing. More specifically, we examined whether such abnormalities reflect visual salience in schizophrenia. Eye movements of 82 patients and 252 healthy individuals viewing natural and/or complex images were examined using saliency maps for static images to determine the contributions of low-level visual features to salience-guided eye movements. The results showed that the mean value for orientation salience at the gazes of the participants with schizophrenia were higher than that of the healthy control subjects. Further analyses revealed that orientation salience defined by the L + M channel of the DKL color space is specifically affected in schizophrenia, suggesting abnormalities in the magnocellular visual pathway. By looking into the computational stages of the visual salience, we found that the difference between schizophrenia and healthy control emerges at the earlier stage, suggesting functional decline in early visual processing. These results suggest that visual salience is affected in schizophrenia, thereby expanding the concept of the aberrant salience hypothesis of psychosis to the visual domain.


Salience-guided eye movements are affected in examples of single images
To investigate the relationship between the gaze positions (defined as endpoints of regular saccades) and the visual salience of the test images, saliency maps were computed using the Itti-Koch saliency computational model (Fig. 1A and Fig. S1) 16,20 .
In representative examples (for the test image #8 and #38), saccades of healthy controls were distributed not only in highly salient positions (shown in yellow) but also in other locations in the image (Fig. 1B, left).In contrast, saccades of participants with schizophrenia remained at salient locations of the image during freeviewing (Fig. 1B, right).To quantify the time course of salience values during the 8-s viewing period, the salience values for each gaze were averaged across participants for the test images and were plotted across time (Fig. S2A, left).The mean salience values were higher throughout the viewing period in participants with schizophrenia (SZ; blue in Fig. S2A, left) than in healthy controls (HC; magenta in Fig. S2A, left).The difference between the participant groups becomes more evident when the salience values were averaged across all saccades during the viewing period (Fig. S2A, right).

Visual salience affects the gaze of participants with schizophrenia
To assess population averages, we plotted the salience values for each saccade averaged across all test images and all participants (Fig. S2B, top).Consistent with the single image results (Fig. S2A), the mean saliency values were consistently higher in SZs than in HCs.
For statistical analyses, a linear mixed model with random intercepts and slopes was used to test the main effect of the participant group (SZ vs. HC) and the interaction between the participant group and the saccade number.Using the log of saccade numbers as a factor, it is more reasonable to fit the data with a linear mixed model (Fig. S2B, bottom).Since the interaction term was not significant (F(1, 330.11) = 0.17 and P = 0.67), a model without interaction was selected.The main effect for the participant group was highly significant (F(1, 332.00) = 44.29 and P = 1.17 × 10 −10 ).These results suggest that the gaze of SZs is more likely to be directed toward visually salient locations than that of HCs.
When the data from age-matched resampled data for HC were compared with the data from SZ (Fig. 1C, leftmost), the results of the statistical analysis were not affected.The interaction term was not significant (F(1, 159.69) = 0.02 and P = 0.87), and a model without interaction was selected.The main effect for the participant group was highly significant (F(1, 162.00) = 25.49and P = 1.18 × 10 −6 ).The resampling procedure was repeated 100 times, one of which was used throughout the following analyses.See also the supplementary text, "Consideration of resampling scheme" for more details.

The gaze of participants with schizophrenia is affected by orientation salience
To assess the contribution of low-level visual features, single-feature saliency maps for color ("Col"), luminance ("Lum"), and orientation ("Ori") were used for analysis as in the full salience model (Fig. 1A and Fig. S1).In all three models, interaction terms were not statistically significant (F(1, 160.79) = 0.93 and P = 0.34 for color, F(1, 161.60) = 0.52 and P = 0.47 for luminance, and F(1, 160.14) = 1.79 and P = 0.18 for orientation, respectively).In the color (Fig. 1C, middle left) and the luminance (Fig. 1C, middle right) models, the main effects for the participant group were not significant (F(1, 162.01) = 0.16 and P = 0.69 for color and F(1, 161.97) = 0.12 and P = 0.72 for luminance, respectively).In contrast, the main effect of the participant group was highly significant (F(1, 161.99) = 44.80 and P = 3.39 × 10 −10 ) in the orientation model (Fig. 1C, rightmost).These results suggest that the main effect in the participant group in the full model (Fig. 1C, leftmost) is explained sorely by orientation salience.

The specific effect of orientation salience is also evident in time-averaged data
Since the liner mixed model showed no significant interaction effect between the participant group and saccade numbers in all four salience models (Fig. 1C), it is reasonable to summarize the data by averaging all saccades during the viewing period (Fig. 1D).The mean saliency value from the full model was higher in SZ than in HC (Z = 4.82, P = 5.75 × 10 −6 ; in Wilcoxon rank-sum test with Bonferroni correction).The effect size evaluated by Cliff 's delta was 0.44.The mean salience value from the orientation model was higher in SZ than in HC (Z = 6.41,P = 5.64 × 10 −10 ; in Wilcoxon rank-sum test with Bonferroni correction).The effect size evaluated by Cliff 's delta for luminance; in Wilcoxon rank-sum test with Bonferroni correction).Based on these findings, the following analyses were performed on the time-averaged data.All of these mean salience values were significantly higher than chance.See also the supplementary text, "Consideration of random sampling scheme" for more details.We also tested whether the saliency values for the first saccade were significantly different between SZ and HC (Fig. S5).See "Discussion" for details.

Image category does not affect the effect of orientation salience
To examine whether the effects of orientation salience depend on image category, the effect size (Cliff 's delta) that evaluates the difference between the time-averaged saliency values of HC and SZ for each test image was plotted for all four salience models (Fig. S3).Positive values in the effect size indicate that saliency values were higher in SZ than in HC.The results (Fig. S3) show that the effect sizes for the Full and the orientation models were consistently positive for all image categories except for the Face and Noise categories.This suggests that the effect of orientation salience is overall robust to image categories.

The L + M channel of the DKL color space is dominant in the effect of orientation salience
Since it is already known that the magnocellular pathway conveys relatively low spatial frequency signals 21 and that the magnocellular pathway is specifically impaired in schizophrenia 8 , this point is directly examined.In the Itti-Koch salience model (hereafter referred to as the "original model"), the orientation salience is calculated from grayscale images.Instead, it is possible to decompose the test images into images of three channels (L + M, L − M, and S-(L + M)), based on the Derrington-Krauskopf-Lennie (DKL) color space (Fig. 2A).In the DKL color space, the three channels (L + M, L − M, and S-(L + M)) separately convey the visual information for the magnocellular, parvocellular, and koniocellular pathways 22 .The decomposed images were then processed to obtain saliency maps with (orientation map) or without (intensity map) Gabor filtering (Fig. 2B).We refer to this as the "Extended six-channel model." Then the salience values for these six maps were compared between SZ and HC (Fig. 2C,D).In the intensity maps (Fig. 2C), all differences were not statistically significant (Z = 0.21, P > 1.0 for the L + M channel; Z = 1.09,P > 1.0 for the L − M channel; Z = − 0.51, P > 1.0 for the S-(L + M) channel; in Wilcoxon rank-sum test with Bonferroni correction).In the orientation maps (Fig. 2D), all of the differences were statistically significant (Z = 6.25,P = 2.51 × 10 −9 for the L + M channel; Z = 4.20, P = 1.59 × 10 −4 for the L − M channel; Z = 3.84, P = 7.49 × 10 −4 for the S-(L + M) channel; in Wilcoxon rank-sum test with Bonferroni correction).These results further support the finding of a specific effect of orientation salience and suggest that the L + M channel of the DKL color space is dominant in the effect of orientation salience.

Saliency maps for orientation are correlated with gaze maps
In Fig. 1C,D, saliency values were evaluated separately for each feature/channel.It is important to note that saliency maps for the different features/channels are spatially correlated (Fig. S4A).This correlation may affect statistical analyses in Fig. 1C,D.To address this issue, we calculated the partial correlation between the saliency maps and the "gaze maps" which quantifies gaze distribution as a density map summed across participants for each test image (Fig. S4B).We also created a gaze map for the difference between SZ and HC ("SZ-HC").The partial correlations of these maps were then calculated for each test image (n = 56).
For the original model, we plotted the partial correlations between the gaze maps and the saliency maps for the three features (color, luminance, and orientation) (Fig. S4C).For the three gaze maps (SZ, HC, and SZ-HC), the partial correlation with the saliency map for orientation was significantly larger than zero (P = 4.0 × 10 −8 for SZ, P = 1.7 × 10 −7 for HC, and P = 1.6 × 10 −7 for SZ-HC, respectively; Wilcoxon signed-rank test with Bonferroni correction).In contrast, the partial correlations with the saliency map for color and luminance were not significantly larger than zero (P > 0.05, Wilcoxon signed-rank test with Bonferroni correction) except for a negative correlation in luminance for SZ-HC (P = 2.7 × 10 −4 ).
For the extended six-channel model, three maps for orientation were shown (Fig. S4D).For two gaze maps (SZ, HC), the partial correlations with the saliency map for the L + M and the L − M channels were significantly larger than zero (P = 2.0 × 10 −5 and 0.002 for SZ, P = 6.0 × 10 −5 and 0.008 for HC; Wilcoxon signed-rank test with Bonferroni correction).On the other hand, for the SZ-HC gaze map, the partial correlation with the saliency map of the L + M channel was significantly larger than zero (P = 5.2 × 10 −4 , Wilcoxon signed-rank test with Bonferroni correction).These results suggest that the L + M channel contributes specifically to the effect of orientation salience, even after accounting for spatial correlations between saliency maps.
The key computational stage that produces differences in orientation salience is Gabor filtering Next, we examined which stage of the salience computation produces differences in orientation salience.In the Itti-Koch salience model, saliency maps for the orientation feature are generated through five processing stages (Fig. S1): (1) transforming the input image into a grayscale image with various spatial resolutions (i.e., Gaussian pyramids), (2) filtering the images with Gabor patches with four different orientations, (3) center-surround inhibition, (4) peak normalization, and finally, (5) adding all images together to obtain the saliency map of the orientation feature.These intermediate maps were calculated and used for analysis as was done in the analysis of Fig. 1D.Differences between participant groups were detected not only in the saliency map for the orientation feature (Fig. 3A, rightmost) but also in peak normalization, center-surround inhibition, and Gabor filter (Fig. 3A, middle).In contrast, no differences between participant groups were detected in intensity (Fig. 3A, leftmost).These differences became more evident when each comparison was evaluated by the effect size (Fig. 3B).These results suggest that the differences between participant groups emerge at the stage of Gabor filtering with four

Orientation salience is correlated with the scores of cognitive tests and visual oculomotor characteristics
Finally, we examined whether orientation salience is correlated with demographic data, cognitive ability, and eye movement characteristics collected from the same participants (see 7,23 for details).When the dependent variables (demographic data, cognitive tests, or eye movement characteristics) were fitted with orientation salience with the participant group as a covariate, age was not correlated with orientation salience but WAIS-3, processing speed (PS), social functioning scale (SFS), and scanpath length (SPL) were correlated with orientation salience (Fig. 4A).For a more systematic analysis, two regression models were constructed.In the first model, the dependent variables were fitted with saliency values from the original model, with the participant group as a covariate.The t-values obtained from the fitting were then plotted (Fig. 4B).Orientation salience is correlated with various scores of cognitive tests and eye movement characteristics (indicated by the gray bars in Fig. 4B).This is consistent with previous findings that scores of cognitive tests are correlated with visual oculomotor characteristics such as scanpath length 23 .Regression analysis also suggested that these correlations are specific to orientation.
In the second model in which the dependent variables were fitted with saliency values from the extended six-channel model, with the participant group as a covariate, orientation salience in the L + M channel was significantly correlated with a cognitive test score (SFS) and eye movement characteristics (saccade amplitude and SPL) (Fig. 4C).In contrast, orientation saliences in the other channels are not correlated with these values.These results again suggest that the L + M channel is dominant in the effect of orientation salience.Then saliency maps for intensity and orientation were obtained for each of the three channels (six maps in total).(C,D) As in Fig. 1D, the salience values averaged across test images and saccades were plotted, but for six saliency maps.Symbols are the same as in Fig. 1D.

Discussion
In this study, we examined whether abnormalities in eye movements result from aberrant processing of visual salience.For this purpose, we analyzed the eye movement data during free-viewing with Itti-Koch's salience model (Fig. 1A).We found that the saliency values at the gaze were reduced across saccades, which is consistent with previous findings 24,25 .We found that the saliency values at the gaze of SZs were persistently higher during the viewing period compared to the HCs (Fig. 1C).Further analysis using single-feature saliency maps revealed that this difference was due to orientation salience (Fig. 1C,D).We then confirmed that these results are robust for various image categories (Fig. S3).We also analyzed the gaze with an extended salience model that evaluates the channels for the DKL color space separately and found that the L + M cannel has a dominant role in the effects of orientation salience (Fig. 2).We also evaluated a spatial correlation between gaze maps and saliency maps and found that orientation salience in the L + M channel correlates specifically with the difference between the gaze maps of the SZs and those of HCs (Fig. S4).In addition, we delved into the stages of salience computation and found that differences between SZs and HCs were found in an early stage of salience computation, where grayscale images were filtered with Gabor patches with four orientations (Fig. 3).Finally, the saliency values at gazes were not correlated with symptom-related measures such as PANSS scores but were correlated with various measures of cognitive functions and saccade-related characteristics (Fig. 4).These results suggest that the difference between schizophrenia and healthy control emerges at the earlier stage, suggesting functional decline in early visual processing.Our findings also suggest that visual salience is affected in schizophrenia, thereby expanding the concept of the aberrant salience hypothesis of psychosis to the visual domain.

Relationship with eye movement abnormalities: visual-oculomotor properties
As described in the "Introduction", visual-oculomotor properties such as scanpath length, saccade number, and saccade amplitudes are affected in the gaze of SZs during free-viewing (see also Table S1).We found that these visual-oculomotor properties are correlated with saliency values (Fig. 4).Since the bottom-up salience of the images is related to target selection, the abnormalities in visual exploration that have been reported in various studies 8 arise, at least in part, from the affected salience-guided eye movement found in this study.

Relationship with eye movement abnormalities: inhibition-of-return
Recently, our research group reported the finding that inhibition-of-return is impaired in SZs 26 .This might raise the question of whether return saccades toward salient stimuli are more frequent in schizophrenia (Fig. S5A).
If this is the case, salience values at the gaze could be higher in SZ than in HC.However, it is unlikely because salience values at the first gaze for the viewing period, where it has no contribution to inhibition-of-return, were significantly higher in SZ than HC in the orientation model (Fig. S5C).Furthermore, if the SZs made more return saccades toward salient positions than in the HCs, the slope of the time course of saliency values would be shallower in the SZs than in the HCs (Fig. S5B, left).However, this is not the case because analyses using linear mixed models showed no interaction between the participant group and saccade numbers.Thus, abnormalities in visual salience found in this study have a different origin from abnormal inhibition-of-return.For further analysis and discussion, see also Okada et al. 26 .

Contribution of low-level visual features, early visual cortex, and the magnocellular pathway
We found that salience computation for the orientation feature is specifically affected in schizophrenia.Consistent with this finding, previous findings have also shown that orientation processing is affected in schizophrenia 27,28 .Other papers have shown that the early visual cortex is involved in changes in contextual modulation in schizophrenia [29][30][31] .The effect of orientation salience can explain impairment in contour integration in schizophrenia 28 because contour integration can be distracted by aberrant processing of orientation salience.
Based on the analysis of Fig. 3 and the accompanying text, we argue that the crucial difference between SZs and HCs stems from their response to Gabor-filtered images.Since the spatial frequencies of the Gabor filters were less than 5 cycles per degree (see also Fig. S1), they resemble responses in the early visual cortex.On the other hand, there was no difference in gazes toward luminance salience.Computation of luminance salience involves center-surround inhibition without Gabor filtering.Such processing resembles the response of the lateral geniculate nucleus (LGN).We propose that aberrant orientation salience reflects abnormalities at the level of the primary visual cortex but not at the level of the LGN.Post-mortem morphological studies of the schizophrenia brains support this possibility.The volume and number of neurons in the primary visual cortex are reduced in schizophrenia 32,33 but it is not the case for the LGN 34,35 .Thus, the idea that visual abnormalities in schizophrenia occur between the LGN and primary visual cortex is consistent with both previous studies and the present current study.
The analyses in Figs. 2 and 4 suggest that orientation salience in the L + M channel of the DKL color space is specifically affected in schizophrenia.Since the L + M channel carries achromatic information in the magnocellular pathway 22 , the present results are consistent with previous findings that visual information processing in the magnocellular pathway is specifically affected in schizophrenia [36][37][38] .

Relationship with clinical/cognitive tests
Figure 4 shows that symptom-related measures such as the PANSS total score, CPZ equivalent, and duration of illness were not significantly correlated with orientation salience.These results suggest that orientation salience is a trait marker rather than a state marker.This is consistent with the view that eye movement-related measures are trait markers of schizophrenia 39 .
Figure 4 also shows that scores on some of the cognitive tests are correlated with orientation salience.This is consistent with previous findings that scores of cognitive tests are correlated with visual oculomotor characteristics such as scanpath length 23 .It is unlikely that the abnormality in scanpath length causes abnormalities in orientation salience, as oculomotor properties of eye movements are less likely affected in schizophrenia than visual or visuo-cognitive processing.Rather, we propose that aberrant processing of orientation salience causes abnormalities in visual exploration, which can be assessed by visual oculomotor properties such as scanpath length, which in turn causes difficulty in cognitive abilities such as social functioning.

Relevance to the aberrant salience hypothesis of psychosis
The aberrant salience hypothesis of psychosis proposes that an aberrant assignment of salience to the elements of one's experience leads to delusion and hallucination 15 .The concept of salience in this hypothesis covers not only salience derived from emotion and motivation (motivational/incentive salience) 40 , but also salience due to novelty and sensory features (perceptual salience) 41 .The present study provides direct evidence that visual salience is affected in schizophrenia, thereby explicitly extending the concept of aberrant salience to the visual domain.In support of our findings, a recent study revealed that brain responses to images with various forms of salience such as novelty, negative emotion, targetness, and rarity/deviance are affected in schizophrenia 42 .

Future projects
In this study, we obtained clues about what is affected in the brain of schizophrenia, for example, Gabor filtering that is presumably performed in the magnocellular pathway of the early visual cortex.To understand exactly how the brain performs such computation, it is necessary to understand the processes at the neuronal level.To this end, neurophysiological studies using animal models of schizophrenia are needed.Since eye-tracking during freeviewing and analysis of visual salience is an experimental paradigm that has been successfully used in non-human primates such as macaque monkeys 17 and marmosets 19 , replicating the present results in a non-human primate model of schizophrenia 43,44 would open the door to understanding the precise brain mechanism of schizophrenia.

Participants
Eye movement data were sampled from 82 SZs (male, 42; female, 40) and 252 HCs (male, 144; female, 108) as part of a large-scale cohort recruited at Osaka University (Table S1) 7,11,23 .There was an overlap in data with a previous study on eye movement abnormalities in schizophrenia 7 .All participants were biologically unrelated, were of Japanese descent, and had no history of the ophthalmologic disease, or neurological/medical conditions that could influence the central nervous system.Specific exclusion criteria included atypical headaches, head trauma with loss of consciousness, chronic lung disease, kidney disease, chronic hepatic disease, thyroid disease, active cancer, cerebrovascular disease, epilepsy, seizures, substance-related disorders, or mental retardation 7,11 .
SZs were recruited from Osaka University Hospital and had been diagnosed by two or more trained psychiatrists according to criteria from the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) based on the Structured Clinical Interview for DSM-IV (SCID).Estimated cognitive decline was calculated by the methods described by Fujino 45 .The current symptoms of the SZs were assessed using the Positive and Negative Syndrome Scale (PANSS) 46 , and daily antipsychotic use was calculated using chlorpromazine (CPZ) equivalents (mg/day) 47  www.nature.com/scientificreports/HCs were assessed for psychiatric, medical, and neurological concerns using a non-patient version of the SCID to exclude individuals with current or past contact with psychiatric services or who had received psychiatric medication.
Informed consent was obtained from all subjects after a full explanation of the study procedures.Anonymity was preserved for all participants.The study was performed in accordance with the World Medical Association's Declaration of Helsinki and was approved by the Research Ethical Committee of Osaka University, the National Center of Neurology and Psychiatry, and Center for Experimental Research in Social Sciences, Hokkaido University.

Task and stimuli
In the free-viewing task, the participants faced a 19-inch liquid crystal display monitor (1280 × 1024 pixels) placed 70 cm from the observer's eyes.Presentation of the visual stimuli was done using the Psychophysics Toolbox extension 48 in MATLAB (The Mathworks, Natick, MA, USA).Each trial began with the presentation of a fixation point on the center of the display.Once the participant fixated on the fixation spot for a random time, a test image was presented for 8 s.The participant was instructed to view the image as they like.One task consisted of 56 images, the order of which was randomly shuffled for each participant.The images were chosen from eight categories: natural environments, buildings, everyday items, foods, faces, animals, fractal patterns, and noise (seven images for each).The images of natural environments and animals were selected from the International Affective Pictures System (IAPS) 49 , and the face images from Matsumoto and Ekman 50 .Since IAPS images are not allowed to be published in scientific journals, only the saliency maps for these images (#8 and #38) are shown in the figures.In cases when images needed to be shown as examples (Figs.1A, 2B, and Fig. S1 ), a photograph taken by one of the authors was used.

Recording and preprocessing of eye movement data
Recording and preprocessing of eye movement data were performed as follows 7 .Eye position and pupil area of the left eye were measured at 1 kHz using EyeLink1000 (SR Research, Ontario, Canada).Eye position data (in degrees) were smoothed with a digital finite impulse response (FIR) filter (− 3 dB at 30 Hz), and eye velocity and acceleration traces were derived from a two-point difference algorithm.Eye movement recordings were segmented into blink, saccade, and fixation periods.Detected saccades included both regular saccades and microsaccades.Here, following previous papers on microsaccades during free-viewing such as 51 , saccades with amplitudes greater than one degree were selected as regular saccades.
We examined the main sequence relationship of the saccades of individual subjects by fitting the function V = a × {1 − exp (− b × A)} + c to the amplitude (A) and peak eye velocity (V) of the saccades obtained from all trials, where a, b and c were optimized (Fig. 4B,C and Table S1).

Computational models and saliency analysis
To assess salience-guided eye movements, we used a validated computational model of visual attention and compared it with individual eye movements.The saliency maps for the test images were computed with the Itti-Koch saliency model for static images 16 implemented in the Graph-Based Visual Saliency (GBVS) toolbox for Matlab 20 .The Itti and Koch model is a neurobiologically inspired model which computes salient locations for low-level visual features (Fig. 1A and Fig. S1).Since we are interested in neurobiological mechanisms of saliency-guided eye movements, the Itti-Koch model was chosen.
In the full model, saliency maps of the three features were summed with equal weights.When evaluating the contribution of low-level visual features, single-feature saliency maps were used (Fig. 1A).For a test image of 640 × 512 pixels, we obtained a saliency map of 80 × 64 pixels.To treat the saliency maps as density maps, all maps were normalized so that the sum of saliency values of all pixels was one.
The details of the computational stages in the Itti-Koch saliency model are shown in Fig. S1.The original image is decomposed into the luminance channel, the red-green opponent channel ('R-G' in Fig. S1), and the blue-yellow opponent channel ('B-Y' in Fig. S1).Both color channels were calculated as either the L − M or S-(L + M) channels of the DKL color space.Then Gaussian pyramids at five scales were obtained ('2-6' in Fig. S1).For orientation salience, the luminance images were processed with Gabor filters with four different orientations (0°, 45°, 90°, and 135°; Fig. S1).The formula of the 2D Gabor filter with 0-degree orientation is as follows.
The size of the filter is 27 × 27 in pixels.The spatial frequency ('freq' in the formula) was 3.14 pixels.The standard deviation of the x and y axis ('stdx' and 'stdy') was 2 and 4 pixels, respectively.The filter was convolved with the gaussian pyramids with five scales to obtain five filtered images ('Gabor filter' at 0° in Fig. S1).Similarly, the Gabor filters with other three orientations (45°, 90°, and 135°) were applied to obtain the filtered images.As a result, this is equivalent to these images being processed by Gabor filters with five spatial frequencies (3.3, 1.6, 0.82, 0.41, and 0.20 cycles per degree).Then center-surround inhibition and peak normalization were done for these images.These intermediate files for salience computation were used for the analysis in Fig. 3 and the accompanying text.These files were also generated by the GBVS toolbox.

Extended six-channel model for visual salience
An "extended six-channel model" for visual salience was constructed for the analysis in Figs. 2 and 4 and the accompanying text.For this purpose, the test images were decomposed into three-channel images (L + M, L − M, and S-(L + M)) based on the DKL color space 22 .The decomposed images were then subjected to salience

Figure 1 .
Figure 1.The mean value of orientation salience at the gaze of participants with schizophrenia is higher than that of healthy control subjects.(A) The saliency map was calculated from the Itti-Koch model.Visual salience for low-level visual features (color, "Col"; luminance, "Lum"; orientation, "Ori") was also computed in this model.(B) Gaze positions of two control subjects (left) and two SZs (right) represented by numbers and superimposed on the saliency map of test images.Numbers indicate saccade order.(C) The saliency values averaged across test images and participants are plotted across saccade numbers on a log scale.(C) As in (B), but for single feature salience models.Age-matched resampled data are plotted for healthy controls.Magenta, the healthy controls (HC; n = 252); blue, the participants with schizophrenia (SZ; n = 82).Numbers on the plots denote P values for the main effect of the participant group.(D) Mean saliency values for all images and saccades are plotted for the healthy controls (HC) and the schizophrenia group (SZ).Data from four salience models are plotted.Symbols denote median values.Error bars denote the first and the third quartile.Numbers on the plots denote P values ("p") and the effect sizes ("Δ", Cliff 's delta) of the Wilcoxon rank-sum test.

Figure 2 .
Figure 2. The L + M channel of the DKL color space is dominant in the effect of orientation salience.(A) The Derrington-Krauskopf-Lennie (DKL) color space.See "Results" and "Methods" for details.(B) To construct the extended six-channel model, the original image was decomposed into three channels in the DKL color space: the magnocellular L + M channel, the parvocellular L − M channel, and the koniocellular S-(L + M) channel.Then saliency maps for intensity and orientation were obtained for each of the three channels (six maps in total).(C,D) As in Fig.1D, the salience values averaged across test images and saccades were plotted, but for six saliency maps.Symbols are the same as in Fig.1D.

Figure 3 .
Figure 3.The key computational stage that produces differences in orientation salience is Gabor filtering.(A) The Itti-Koch saliency model computes the saliency map of the orientation feature through five stages (Fig.S1).The map values were calculated for these intermediate images are plotted as in Fig.1C.(B) Effect sizes (Cliff 's delta) for the tests in (A).Numbers indicate the spatial scales of the Gaussian pyramids.In the three plots in the center, the mean effect size for four orientations (bar), as well as the effect size for four orientations (symbols) were plotted."o, " 0°; "x, " 45°; triangle, 90°, and square, 135°.The horizontal lines in the rightmost plot ("Saliency map (Orientation)") indicate the common guidelines for the effect size (Cliff 's delta).See "Methods" for details.

Figure 4 .
Figure 4. Orientation salience is correlated with the scores of cognitive tests and visual oculomotor characteristics.(A) Scattered plots for the mean orientation salience (averaged across images and saccades) and age, WAIS-3 processing speed (PS), social functioning scale (SFS), and scanpath length (SPL).Each dot represents the value for one participant.Magenta: healthy controls (HC); blue, participants with schizophrenia (SZ).Lines indicate regression lines for each participant group.(B) Absolute value of t-values from regression analysis 1, where the dependent variables were fitted individually with the saliency values of the original model.Gray bars indicate statistical significance (P < 0.05) after correction of multiple comparisons by FDR.(C) As in (B), but those from regression analysis 2, where the dependent variables were fitted individually with the saliency values of the extended six-channel model.