Reliable Attention Network Scores and Mutually Inhibited Inter-network Relationships Revealed by Mixed Design and Non-orthogonal Method

The attention system can be divided into alerting, orienting, and executive control networks. The efficiency and independence of attention networks have been widely tested with the attention network test (ANT) and its revised versions. However, many studies have failed to find effects of attention network scores (ANSs) and inter-network relationships (INRs). Moreover, the low reliability of ANSs can not meet the demands of theoretical and empirical investigations. Two methodological factors (the inter-trial influence in the event-related design and the inter-network interference in orthogonal contrast) may be responsible for the unreliability of ANT. In this study, we combined the mixed design and non-orthogonal method to explore ANSs and directional INRs. With a small number of trials, we obtained reliable and independent ANSs (split-half reliability of alerting: 0.684; orienting: 0.588; and executive control: 0.616), suggesting an individual and specific attention system. Furthermore, mutual inhibition was observed when two networks were operated simultaneously, indicating a differentiated but integrated attention system. Overall, the reliable and individual specific ANSs and mutually inhibited INRs provide novel insight into the understanding of the developmental, physiological and pathological mechanisms of attention networks, and can benefit future experimental and clinical investigations of attention using ANT.


Results
shows the RT and accuracy of each condition. Because the ceiling effect was evident, values of accuracy were not used for further analysis. Since RTs were not normally distributed, the median RT per condition was used for computing ANSs and directional INRs 21 . We replaced RTs with the median for trials with error responded (0.97%) or with the RT larger than 3 standard deviations (1.17%) in each cue-target condition in each subject. We replaced these data due to two reasons: 1) there are only 24 pairs of trial for each ANS which is much fewer than previous studies (e.g., 72 or 96 pairs of trial in the original version of ANT); 2) the analysis of split-half reliability requires equal numbers of trial for two halves. Table 2 and Fig. 1, all ANSs and directional INRs were remarkably different from zero. For ANSs, obvious decrease in RTs could be seen on central cue condition than on no cue condition; while RTs were significantly increased on spatial target and incongruent target conditions than on central target and congruent target conditions, respectively. As shown in Fig. 1B, the O and E impaired the efficiency of A, indicating a weakened alert on spatial target and incongruent target conditions. The A enhanced the efficiencies of O and E, indicating slowed orienting and weakened executive control capacity in cued trials. The effects of O and E mutually enhanced, indicating slowed orienting on the incongruent target condition and impaired executive control capacity on the spatial target condition. These directional INRs revealed remarkable inter-network inhibition in interaction blocks, suggesting that the non-orthogonal method and mixed design were necessary to avoid inter-trial and inter-network interferences in measuring ANSs. Table 3, both main effects and interactions for three pairs of network were extremely significant, indicating strong coordination between networks when the cognitive system operates more than one network. Visualized inter-network interactions could be seen in Fig. 2. Post hoc analysis was not performed because the directional relationship analysis had described INRs in detail.

Inter-network interactions. As shown in
Inter-network correlations. For ANSs, Pearson correlation analysis showed a weak negative correlation between A and O: r = -0.310, P = 0.040. However, this correlation could not pass Bonferroni adjustment 22 . The correlations between A and E (r = 0.174, P = 0.257) and between O and E (r = 0.045, P = 0.772) were far from significance. These results revealed that the three networks are mutually independent.
For directional INRs, striking negative correlations between A→O and O→A (r = -0.411, P = 0.006) and between A→E and E→A (r = -0.348, P = 0.020), and positive correlation between O→E and E→O (r = 0.605, P < 0.0001) were observed (Fig. 3). Of note, the negative correlation was due to the negative value of A. These significant correlations indicated that the networks were mutually inhibited when two networks were operated simultaneously.
Considering that the split-half reliabilities of A (0.14-0.27) and O (0.26-0.38) were much lower than 0.5 in most studies 10 , the high reliabilities of three networks in the current study, therefore, have positive theoretical and clinical values.

Discussion
The reliability of ANSs and independency of networks suffered a long-term debate since Posner and Peterson 2 put forward the three networks of attention system. Combined the non-orthogonal method with mixed design, we observed significant ANSs and INRs, indicating a differentiated but integrated attention system. Although with a few trials, we got very high split-half reliabilities of ANSs, suggesting that our method has a strong potential in measuring ANSs in developmental and clinical investigations.   The ANSs are significant and mutually independent. The lack of correlations among ANSs indicates that each person has a specific profile of A-O-E (Fig. 1A 23 ; the A gradually develops and matures during adolescence 24 . Essentially, the three networks are driven by different genes 9 and neurotransmitters 3,25 . These genetic, developmental, and physiological factors may cooperate to create a specific profile of A-O-E on each person.
The independence and between subjects variability of ANSs are also supported by selective deficit in a variety of diseases. For example, specific E deficits have been observed in individuals with borderline personality disorder 26 and posttraumatic stress disorder 27 . A specific O deficit has been reported in individuals with concussion 28 . A specific A deficit 29 or both O and E deficits 30 have been reported in different schizophrenia groups. The selective impairment of one or more networks indicates that the profile of A-O-E could serve as a psychological marker of a wide range of diseases 31 . However, these reports of selective impairments rely on the independence and reliability of ANSs 10 . The high reliability and remarkably independent ANSs in the present study suggested that combining non-orthogonal contrast with mixed design can provide a better measurement of the profile of A-O-E than previous studies and thus can benefit future theoretical and clinical studies.
The reliability of ANSs is particularly important given the widely application of ANT in developmental and clinical studies. Low reliability of A and O in previous studies not only fails to meet the demand of clinical investigations but also decreases statistical power and underestimates true inter-network correlations 10,11 . We suggest that the reliability of ANSs is constrained by at least three factors except for methodological factors announced in the Introduction section. First, the infra-slow fluctuations of RT are associated with significant intra-individual variation 32,33,34 , resulting in low reliability 11 . Second, more trials may increase reliability 11 but induce fatigue effect that prolongs and increases variability of RTs 35 . Third, the fluctuations of arousal may increase the variability of observed ANSs on the one hand, because the ANSs are obtained by contrasting different effects with the baseline state; and the true ANSs on the other hand, because the arousal interacts and endogenously contributes to all three networks 36 . Therefore, reducing inter-trial variability or bringing it into statistical analysis is necessary to improve the reliability of ANSs.  The interaction between A and O has been reported by some researchers 7,37 but not by others 5,38 . Those studies reporting a speeded-up orienting of attention under conditions of high alerting have proposed that the influence from A to O occurs only within a short (<500 ms) time course and peaks at about 150 ms 7,17 . They hypothesized that the A benefits the O 7, 37 , but provided no evidence and assumption on the influence from O to A. So far, there is not a theoretical framework about the interplay between A and O.
Beyond the unidirectional influence from A to O, we observed a larger effect of O when there was an alerting cue and a smaller effect of A when spatial target was presented. With a close inspection, we found that the alerting cue improved the respond to central target more than to spatial target, whereas the spatial target increased RTs to cued trials more than to un-cued trials. Therefore, the influence from A to O in the present study is opposite to previous findings. This discrepancy can be coordinated by a cue-target consistency effect. The consistent cue-target condition improves not only the phasic alerting but also the target processing at the cued location 39,40 , supporting the interaction between A and O. For the influence from A to O, when a spatial cue is presented as in previous studies, the processing of the target at that location is improved than at other locations. Reversely, if an alerting signal cues a central position, the processing of the target at this location is improved. For the influence from O to A, if the target appears at the same location as the cue, the processing is improved; if it appears at a different location from the cue, the processing is impaired. It's worth noting that, in previous studies, the cue validity and cue-target interval may also contribute to the interaction between A and O by modulating both A and O via expectancy and time variability 41 . As a promising theoretical framework about the relationship between A and O, the cue-target consistency effect should be verified by manipulating cue validity and cue-target interval in future studies with behavioral and neuroimaging techniques 25 .
There are several opinions on the interaction between alerting and executive control networks. First, Callejas et al. 7 proposed that alertness directly inhibits executive control by impeding the function of anterior cingulate cortex. Second, Böckler et al. 42 and Fischer et al. 43,44 suggested that alerting cues amplify automatic response activation related to both the relevant and irrelevant task dimensions, thus enhancing the conflict effect. Third, McConnell and Shore 13 suggested that alerting cues encourage diffusing attention between two possible target locations. The uncertainty of target location in the ANT disturbs the attention in a focused state, making it harder to ignore distracting stimuli. Fourth, Fan et al. 5 assumed that the alerting condition reduced RTs, thus providing less time to process conflict. Fifth, Weinbach and Henik 45,46 demonstrated that alerting cues expand the attention scope and create a bias to global processing, thus increasing the disturbance of distracting stimuli. These views highlighted the influence from A to E, whereas the resource competition explanation focused on the bidirectional relationship between A and E 6 . Accordingly, the A and E may compete resources from their shared brain regions, e.g., the anterior cingulate cortex and fronto-parietal network.
By eliminating spatial targets, we observed enlarged conflict effect in the alerting condition as well as reduced alerting effect in the incongruent condition. These results are consistent with the resource competition explanation 6 , and can be partially explained by other perspectives except for McConnell and Shore's 13 target uncertainty hypothesis. It's worth noting that the resource competition hypothesis can also explain the mutual inhibition between A and O, and between O and E. Given the complex and flexible interaction between A and E, more factors should be considered in future investigations such as the spatial pattern of targets 46 , dopamine level 47 , task difficulty 36 and arousal 48 . We suggest that investigations on bidirectional relationship between A and E are essential to illuminate the mechanisms of alerting and executive control systems.
There are different assumptions about the influence from O to E in previous studies 5,7,13 . Fan et al. 5 explained it from the perspective of attention resource that prior orienting of attention reduced RTs, resulting in insufficient time to process conflict. Callejas et al. 7 suggested that the prior orienting of attention to the target location could help subjects to concentrate on that area and ignore the incongruent flankers. McConnell and Shore 13 assumed that orienting to another location induced the global processing strategy, leading to a difficult in ignoring flankers. The confliction among these hypotheses lies in how subjects use attention resource and focusing strategy when orienting to a new position.
In this study, RTs of O were longer in incongruent condition than in congruent condition, while conflict effect was larger in spatial target condition compared with the central target condition. When O and E occur simultaneously, the resource competition hypothesis predicts bidirectional impairment between them, while the focusing strategy hypothesis anticipates an inhibition from O to E and a promotion from E to O because both E and O induce a global processing strategy. Therefore, the current results are consistent with the resource competition perspective rather than the focusing strategy hypothesis.
Convergent evidence showed that the attention system is a differentiated but integrated system. First, there is no consensus on the correlations among ANSs 5,10,13,49 , indicating a large inter-individual variability of the A-O-E profile. With high reliability of ANSs, the inter-network correlation is more properly estimated in the current study than in previous studies with low reliability of ANSs 10,11 . The lack of inter-network correlation we observed in a high homogeneous sample (college students), therefore, strongly propose a large inter-individual variability of the A-O-E profile. Many factors such as gene, development, physiology, and the state during test may be responsible for the particular A-O-E profile of each participant 9 Second, as an integrated system, attention resources would be allocated flexibly among networks. The resource competition hypothesis can explain the current findings as well as results in previous studies 5, 6 . Meanwhile, inter-network cooperation is essential to produce a more adaptive behavior 7 . This requires the attention system to balance the roles of three networks dynamically 37,41 and use various strategies such as global/local processing 13,45,46 and multisensory integration 7,40,49 .
The differentiated networks and inter-network integration of attention are supported by different neurotransmitters and partially overlapped brain regions 3,4,17 . We also show that the networks are independent when they are separately measured, but mutually impaired when they are operated simultaneously. The differentiation and bidirectional impact of attention networks are of importance for our understanding of various A-O-E profiles in developmental and clinical investigations. For instance, if the E is differentiated from O 23 , can we expect stronger influence from O to E than from E to O in children before three years old compared with elder children and juveniles? In addition, if individuals with borderline personality disorder have a specific E deficit 26 , can we anticipate alterations of directional influence between E and A and between E and O? Our method may provide an effective solution to these developmental and clinical mysteries. Furthermore, reliable measurements of ANSs and directional INRs in a simple context are vitally important in developmental and clinical studies.
Combining non-orthogonal method with mixed design, we obtained independent ANSs with high reliability and remarkable bidirectional INRs. However, there are several limitations. First, the attention system is too complex to be measured by a single task. For example, several dichotomies have been highlighted in recent studies such as exogenous/endogenous attention 17 , tonic/phasic alerting 46 , top-down/bottom-up control 14 , and spatial/objective (non-spatial) attention 46 . Although the ANT tests a basic model of attention, complex relationships among more subcomponents should be examined in detail with reasonable reliability. Second, the cue-target interval in the present study is constant, eliminating the interference from temporal expectancy 25,41 . However, the inter-network relationship has been demonstrated to be time and probability dependent 37,41 . Cue validity, inter-and intra-trial possibilities, and more cue-target intervals are warranted in future studies to explore complex INRs. Third, our subjects were all college students with high homogeneity. This may not well reflect between-subject variability and the development and aging of attention system. A large sample size involves children, adults and old people may help us to dissociate between-and within-subject variability. Last, variables (e.g., sleep hours, time of testing, and drug consumption) who influence attention state should be recorded in future studies.

Conclusions
Compared with previous studies, we used fewer trials to produce higher reliability of ANSs. Meanwhile, we demonstrated a differentiated but integrated attention system. Specifically, the three networks are independent but mutually inhibited when two networks are operated simultaneously. The mutual inhibition was in line with the resource competition hypothesis. The high reliability of ANSs and directional INRs may shed light on the developmental, degenerative, and pathological mechanisms of attention networks and benefit the application of ANT in developmental and clinical studies.

Material and methods
Subjects. Fifty subjects took part in the experiment at extracurricular time (29/21 males/females, reported right-handed, ages ranged from 18-29 years with mean age of 20.34 ± 2.86). All participants had regular routines and were asked to have enough sleep and avoid from any drugs and alcohol in 12 hours before the experiment. All participants reported normal or correct-to-normal vision, without any medication, and neurological or psychiatric disorders. Six subjects were excluded from subsequent analyses due to following reasons: one for misunderstanding the introduction, two for procedure erroneous, and three for low accuracy (<80%) in any condition.

Ethics Statement. This study was approved by the research ethical committee of School of Life
Science and Technology, University of Electronic Science and Technology of China. The methods were carried out complied with the tenets of the Declaration of Helsinki. All the participants gave their informed consents prior to their inclusion in the study.
Apparatus and Stimuli. The experiment was run on a Lenovo-PC computer with a 20-inch color screen monitor. E-Prime 2.0 software (http://www.pstnet.com; Psychology Software Tools, Inc) was used for programming, display of stimuli, and timing control. Responses were collected through the computer keyboard. All stimuli were black figures presented at the center of a screen on a gray background. Fixation was a plus sign and subtended 0.22° visual angle. The length of the whole target (five arrows and inter-arrow blank) was 3.1°. The length of each arrow was 0.53° visual angle. The spatial target was 2° from the fixation.
Procedure and design. All participants completed six blocks of the mixed design attention network test. The order of six blocks was randomized. Each effect block contained 4 practice trials and 48 experimental trials, while each interaction block included 4 practice trials and 96 experimental trials. Eighteen cue-target conditions were involved in the task. This design advanced our previous study 12 Scientific RepoRts | 5:10251 | DOi: 10.1038/srep10251 to all directional INRs. Participants were seated 70 cm from the monitor and were asked to focus on the fixation throughout the experiment and, during practice. As shown in Fig. 4, each trial began with a fixation lasting for 400-1000 ms. A 100 ms fixation (for O, E and half of the trials of A) or asterisk (for half of the trials of A) was followed. After a 300 ms delay, the target (central or spatial, congruent or incongruent) was presented for 1700 ms or lasted until the participant pressed a key. Lastly, another delay was presented to insure the overall time of the trial from cue to the end of the trial was 3000 ms.
Instead of the conventional subtraction measure 5,6 , we used ratio scores to define the efficiency of attention networks and INRs. The ratio scores could avoid the baseline difference and isolate the attention system from the overall reaction time (RT) 21,50 . The ANSs are listed in equations (1-3) using conditions in effect blocks. The INRs are calculated using conditions in interaction conditions with equations (4-9). This method allows us to test the direction of impact beyond interaction, promoting our understanding of the relationship among attention networks. , and executive control (E) networks were tested in separated blocks to avoid interactions among them. The inter-network relationships were also assessed in independent blocks. For A and E, central targets were used to exclude the orienting effect; for O and E, cues were canceled to exclude the alerting effect; for A and O, incongruent targets were eliminated to avoid the conflict effect.