Silent word reading leads to the activation of orthographic (spelling), semantic (meaning), as well as phonological (sound) information. For bilinguals, native language information can also be activated automatically when they read words in their second language. For example, when Chinese-English bilinguals read words in their second language (English), the phonology of the Chinese translations is automatically activated. Chinese phonology, however, consists of consonants and vowels (segmental) and tonal information. To what extent these two aspects of Chinese phonology are activated is yet unclear. Here, we used behavioural measures, event-related potentials and oscillatory EEG to investigate Chinese segmental and tonal activation during word recognition. Evidence of Chinese segmental activation was found when bilinguals read English words (faster responses, reduced N400, gamma-band power reduction) and when they read Chinese words (increased LPC, gamma-band power reduction). In contrast, evidence for Chinese tonal activation was only found when bilinguals read Chinese words (gamma-band power increase). Together, our converging behavioural and electrophysiological evidence indicates that Chinese segmental information is activated during English word reading, whereas both segmental and tonal information are activated during Chinese word reading. Importantly, gamma-band oscillations are modulated differently by tonal and segmental activation, suggesting independent processing of Chinese tones and segments.
Reading is a complex cognitive skill unique to humans. A fundamental component of this skill is the recognition of individual words. In current theories of visual word recognition, the word recognition process is affected not only by written characteristics (orthography) but also by the spoken characteristics (phonology) of target words1. This is supported by data obtained with alphabetical languages2,3,4,5,6 as well as non-alphabetic languages, such as Chinese7,8,9,10,11,12,13.
The activation of phonology during visual word recognition is not limited to the target language. Studies with Chinese-English bilinguals have demonstrated that the native language phonology (Chinese) is activated during second language (English) word recognition in a purely English task context14,15,16,17,18. For example, Wu and Thierry16 asked Chinese-English bilinguals to judge whether English word pairs were related in meaning or not (i.e., a semantic relatedness judgment task). Participants were not aware that the Chinese translations of some of the English word pairs contained phonological repetitions (e.g., experience - surprise, /Jing1Yan4/ - /Jing1Ya4/) or orthographic repetitions (e.g., accountant - conference, /Kuai4Ji4/ - /Hui4Yi4/). In their event-related potentials (ERPs) data, a reduced N400 was observed for English word pairs featuring a Chinese phonological repetition but not for pairs involving a Chinese orthographic repetition. This would indicate that Chinese phonology, rather than orthography, was activated during English visual word recognition. Phonological activation of Chinese translations as evidenced by an N400 reduction was also found in a study that manipulated the emotional valence of English word pairs18. Furthermore, activation of the phonology of Chinese translations was also observed during a visual shape searching task when critical English words could be entirely ignored by participants15,17. Taken together, these studies indicate that the phonology of Chinese translations is activated during English word recognition.
Chinese is, however, a tone language in which the phonological component of a Chinese character consists of both segments (consonants and vowels) and a lexical tone. For example, in Mandarin Chinese (henceforth Chinese), the segment Ji with a high level tone (tone 1), a rising tone (tone 2), a falling-rising tone (tone 3), and a falling tone (tone 4) can mean “chicken” (/Ji1/), “hurry” (/Ji2/), “squeeze” (/Ji3/), and “send” (/Ji4/) respectively. Note that tone is marked with numbers in Pinyin (romanisation system for Chinese phonology, indicated between slashes), whereas tone is not marked in Chinese characters. Because segment and tone are not explicitly marked in Chinese characters, grapheme-phoneme correspondences are opaque and therefore Chinese has a deep orthography. However, previous EEG (electroencephalogram) studies have found that phonology is still activated during Chinese visual word recognition. For example, in a Chinese semantic relatedness judgment experiment, Kong et al.7 found a larger P200 amplitude when a Chinese character was preceded by its homophone (e.g., – , /Qian4/ – /Qain4/). Such phonological repetition in Chinese has been found to be indexed by either a P2007,13 or an N400 effect16,18 and the phonological repetition stimuli used in the literature have so far always involved identical segments and tones. As far as we are aware, no studies have used EEG to investigate the activation of segment and tone separately in Chinese visual word recognition. However, a few behavioural investigations have been conducted with the colour-naming Stroop task12,19. For example, Li, et al.19 presented in the Stroop task colour words (e.g., /Hong2/, “red”) as well as homophones of colour words (+Segment +Tone, e.g., /Hong2/, “flood”), words sharing segments with colour words (+Segment -Tone, e.g., /Hong1/, “boom”), and words sharing tones with colour words (−Segment +Tone, e.g., /Ping2/, “bottle”). In the congruent conditions (e.g., , , , presented in red), the +Segment +Tone, +Segment −Tone, −Segment +Tone conditions all showed facilitation effects, suggesting independent activation of segment and tone. Furthermore, computational models of Chinese word processing assume de facto that segmental and tonal representations are independent, because they code segmental and tonal information separately20,21,22. Thus, consistent with the findings of Stroop experiments and models of Chinese word processing, we predict that segmental and tonal information play an independent role in visual word recognition in Chinese-English bilinguals.
To investigate the role of Chinese segmental and tonal activation in bilingual visual word recognition, we conducted two EEG experiments with Chinese-English bilinguals. In the first experiment, we investigated Chinese segmental and tonal activation during English word reading in a semantic relatedness judgment task with English word pairs (prime-target). The repetition of segmental and tonal information of the Chinese translations (i.e., the first character of Chinese translations for the English words) was systematically manipulated in semantically and orthographically unrelated English prime and target pairs, i.e., segmental and tonal repetition (+Segment +Tone): media - rose, /Mei2Ti3/ −/Mei2Gui4/; only segmental repetition (+Segment −Tone): mirror - police, /Jing4Zi3/ - /Jing3Cha2/; only tonal repetition (−Segment +Tone): milk - file, /Niu2Nai3/ -/Wen2Jian4/; no repetition (−Segment −Tone): card - frog, /Ka3Pian4/ - /Qing1Wa1/. In the second experiment, we investigated Chinese segmental and tonal activation during Chinese word reading in a Chinese semantic relatedness judgment task. Participants in this experiment read the Chinese translations of the English word pairs used in the first experiment. Together, these two experiments investigate indirect and direct activation of Chinese phonology during visual word recognition and aim to dissociate segmental and tonal aspects of Chinese phonology. It is important to note that Chinese phonological activation in English word reading is through translation, and that spelling-sound correspondences are opaque in Chinese. Therefore, if evidence of phonological activation is found in both experiments, it will strongly support the idea that native language phonology is an essential component of visual word recognition in the native language23,24,25 and second language26.
In addition to analysing ERPs, we also conducted time-frequency analyses on the EEG data to investigate event-related oscillations (EROs). An advantage of EROs is that they reveal non-phase-locked signals that conventional ERP data cannot capture27,28,29. Oscillations observed in the scalp EEG are assumed to reflect underlying neuronal synchronization30,31. When the power of oscillations relatively increases, it is referred to as an event-related synchronization, whereas a relative power decrease is termed an event-related desynchronization. Changes in EROs in response to linguistic manipulations have been reported in almost all major frequency bands (theta: 4–7 Hz, alpha: 8–12 Hz, lower beta: 15–20 Hz, higher beta: 20–30 Hz, gamma: above 30 Hz)30,31. The importance of analysing EROs in addition to ERPs is nicely illustrated in Hagoort, et al.32. This seminal study found that semantic violations and world knowledge violations modulated the N400 amplitude, whereas only world knowledge violations modulated gamma-band power. Thus, gamma activity was only modulated by world knowledge violations, but not semantic violations, which suggests that the cognitive processes involving world knowledge and semantics are separable. This finding cannot be clearly demonstrated with ERPs alone.
In sum, we investigated Chinese segmental and tonal activation during visual word recognition in Chinese-English bilinguals. If Chinese segmental and tonal information are activated independently, changes in ERPs and EROs are expected for both segmental and tonal repetition conditions. Alternatively, if only segmental or tonal information is activated, changes in ERPs and EROs would only be expected in one of these repetition conditions. Because an N400 reduction has been consistently reported when English word pairs contained a phonological repetition in the Chinese translations14,16,18, we focussed in our ERP analysis of the English experiment on the N400 component. In contrast, in Chinese visual word recognition the P2007,13 and N40016,18 have been found to be sensitive to phonological repetition. No previous similar word recognition studies have analysed EROs and therefore we did not have prior knowledge about which frequency bands, time windows, and electrodes may be sensitive to the effects of Chinese phonological repetition. Thus, our analyses of EROs are more exploratory and involve a data-driven approach. A behavioural impact of Chinese segmental and tonal repetition was not expected in English because no behavioural effects have been found previously14,16,18. For Chinese, the behavioural effects reported in the literature have so far been mixed14,16,18.
A summary of the mean error rates and reaction times for the four experimental conditions in the English and Chinese experiments is presented in Table S1 of the Supplementary Material.
Responses to English word pairs with repeated Chinese segments in their Chinese translations were 14 ms faster than to English words pairs without repeated Chinese segments, F (1, 18) = 8.968, p < 0.01 ηp2 = 0.333. No other effects were found in the response times (all ps > 0.30) and error rates (all ps > 0.12).
The ERP results were based on the cluster-based random permutation test implemented in FieldTrip (see below for details).
A reduced N400 was found in the time window from 403 to 457 ms for English word pairs with segmental repetition in their Chinese translations (cluster with p = 0.024, see Fig. 1). There was no effect of tonal repetition or an interaction between segmental and tonal repetition.
A significant effect of segmental repetition in the time window from 524 to 680 ms was found (cluster with p = 0.034) because Chinese words pairs with segmental repetition elicited a larger LPC (i.e., late positive component) than Chinese word pairs without segmental repetition. The difference in the LPC was found to be more pronounced over posterior electrodes (see Fig. 1). There was no effect of tonal repetition or an interaction between segmental and tonal repetition.
The time-frequency results were also based on the cluster-based random permutation test implemented in FieldTrip (see below for details).
Gamma power (30–60 Hz) was significantly weaker in the time window from 578 to 633 ms when segments of the Chinese translations of English word pairs were repeated than when there was no segmental repetition in the Chinese translations (cluster with p = 0.026). This effect was most pronounced over central electrodes (see Fig. 2). There were no gamma-band power modulations by tonal repetition, and no interaction. Additional analyses in the low frequency range (5–30 Hz) did not reveal any significant effects (see supplementary material).
For Chinese word pairs, segmental repetition effects were found in the gamma-band power (44–60 Hz), indicating weaker effects for word pairs with repeated segments than for word pairs without repeated segments (cluster with p = 0.040, 531 to 676 ms). As can be seen in Fig. 3, the decreased gamma-band power for segmental repetition was widely distributed. For Chinese word pairs with repeated tones, the gamma-power (31–60 Hz) was significantly stronger in the time window from 465 to 800 ms for word pairs with the same tone than for word pairs with a different tone (cluster with p = 0.004). The increased gamma-band power in tonal repetition was widely distributed. No interaction was found between segmental and tonal repetition. No significant clusters were found in the low frequency range (5–30 Hz).
Two experiments investigated the electrophysiological dynamics of Chinese segmental (consonants and vowels) and tonal activation in Chinese-English bilinguals. In the English semantic relatedness judgement task, a hidden segmental repetition effect was observed in the behavioural data (faster responses), the ERP data (a reduced N400), and in the time-frequency data (gamma-band power reduction). These converging behavioural, ERP, and time-frequency results indicate that Chinese segmental information (consonants and vowels) of Chinese translations is activated during English word reading. In the Chinese experiment, the segmental repetition effect was observed in the ERP data (an increased LPC) and the time-frequency data (gamma-band power reduction). In addition, we found a tonal repetition effect in the time-frequency data (gamma-band power increase). Taken together, the combined results of the two experiments indicate that when Chinese-English bilinguals read English words, the segmental information of Chinese translations is activated, whereas when they read Chinese words both segmental and tonal information are activated.
We did not expect to find behavioural effects of segmental repetition in the English experiment because previous studies using the same paradigm did not find significant behavioural effects14,16,18. One possible explanation for our behavioural finding is that our experiment included more trials with segmental repetition than previous studies (60 trials for + Segment +Tone plus 60 trials for +Segment −Tone vs. 45 trials for +Segment +Tone in Wu and Thierry16). Using more trials can reduce the noise in the data and therefore increase the chance to detect an effect.
The N400 reduction observed in the English experiment is consistent with previous studies revealing activation of the first language during second language reading14,15,16,17,18,33,34,35,36. These findings support the idea that visual word recognition in bilinguals involves non-selective lexical access26,37. Importantly, because the N400 reduction was found for hidden segmental repetition but not tonal repetition, it suggests that the hidden phonological repetition observed in previous studies14,16,18 is likely driven by segmental repetition in the Chinese translations of English word pairs. Furthermore, the N400 reduction elicited by the hidden segmental repetition supports previous suggestions that the N400 component is an index of phonological priming2,5. Given that the N400 component can also reflect semantic processing of English words in the semantic relatedness judgement task, the observed N400 reduction indicates that Chinese phonology may impact the on-going semantic processing of English words.
An unexpected finding of the present study is that the ERP effects of segmental repetition were different in English and Chinese. The N400 effect in English occurred in the time window that has been associated with lexical processing, whereas the LPC effect in Chinese occurred at the decision stage33,38. The LPC is assumed to reflect conscious conflict resolution33,38 that might specifically be needed when the Chinese word pairs had overt segmental repetition. This conscious conflict resolution may abolish any behavioural facilitation effects due to segmental repetition. Therefore, the presence of the LPC in the Chinese experiment and its absence in the English experiment may explain why behavioural effects of segmental repetition were only observed in English.
The ERP effects of segmental repetition in Chinese occurred later than in previous studies investigating phonological activation in Chinese word recognition. However, these studies have focussed mainly on the contrast between +Segment +Tone and −Segment −Tone, and reported early ERP effects (e.g., N400 effect16,18 or a P200 effect7,13). To compare our data with previous studies, a new analysis of the Chinese experiment was conducted to directly compare +Segment +Tone and −Segment −Tone. A significant P200 reduction was found for +Segment +Tone (see Figure S6 in the Supplementary Material), suggesting that Chinese phonological activation impacted ERP responses in an early time window. This P200 effect resembled the patterns reported in Thierry and Wu14, which showed a reduction of the P200 in Chinese word pairs containing orthographic and phonological repetitions (e.g., [Hou3Che1] –[Huo3Tui3]). However, it is unclear why the phonological repetition (+Segment +Tone) elicited an enhanced P200 in previous studies7,13, whereas the current data showed a reduced P200. Although the P200 component has been associated with lexical form overlap8,38, the exact mechanisms underlying the decrease and increase of the P200 remains unclear (for a discussion, see33). One possible explanation for the discrepancy of the P200 modulations is that two-character Chinese words were presented in the present study in contrast to single characters used in previous studies7,13. In other words, different P200 modulations may indicate that the phonological repetition in two-character words is different from that in single characters.
Time-frequency analyses have so far only been conducted in a few bilingual studies, such as studies involving sentence reading39,40, word production41, and language learning42,43,44. As far as we are aware, the current study is the first to use oscillatory EEG data to investigate bilingual visual word recognition. Our key finding is that gamma-band power is reduced by segmental repetition of Chinese translations during an English semantic relatedness task. There is unfortunately no one-to-one mapping between cognitive functions and gamma band power45. However, one way to interpret our findings is to look at the findings of previous studies investigating repetition. Various types of stimulus repetition (e.g., words, drawings, pictures) have been reported to cause a reduction of neural activity, which is often referred to as repetition suppression46,47,48,49,50,51. Repetition suppression commonly correlates with decreases in haemodynamic responses in fMRI/PET studies and in gamma-band power in EEG studies50. Although the exact neural mechanism underlying repetition suppression is unclear, it can generally be understood as greater neural efficiency in processing repeated stimuli50. For example, in a masked priming experiment with briefly presented primes (i.e., 30 ms)52, gamma-band power was reduced for homophone primes (e.g., pair - pear) compared to control primes. Matsumoto and Iidaka52 interpreted this decrease in gamma-band power as repetition suppression, suggesting that repetition of one stimulus feature (phonology) is able to reduce neural activity. The reduced gamma power elicited by segmental repetition in both the Chinese translations of English word pairs in the English experiment and the Chinese words in the Chinese experiment is consistent with this interpretation. The gamma power reduction was not associated with the modulation of specific ERP components. This is because conventional ERP data cannot capture non-phase-locked signals (unlike EROs), and frequencies above 15 Hz are usually non-phase-locked27,28,29. Furthermore, low-pass filters were applied in the ERP analyses as is common in many language studies (e.g., 30 Hz in the present study and 25 Hz in Wu and Thierry). Therefore, signals from the gamma frequency range (>30 Hz) were largely attenuated.
Another important finding of the current study is that tonal repetition increased gamma-band power in the Chinese experiment. Because tonal repetition effects were not found in the behavioural and ERP data, it turns out to be crucial to analyse both phase and non-phase locked EEG signals when studying the role of phonology in Chinese word processing. Our study provides, as far as we are aware, the first electrophysiological evidence of tonal activation when native Chinese speakers read two-character Chinese words. Previous investigations of tonal activation during Chinese word reading have mainly focussed on single Chinese characters19,53,54. For example, using a visual odd-ball paradigm, two ERP studies53,54 have reported a visual mismatch negativity elicited by characters with a deviant tone (e.g., /Yi1/) presented amongst characters with a standard tone (e.g., , , , , /Yi4/), suggesting that tonal information is quickly activated. However, single character recognition may not provide the complete picture of tonal activation during Chinese word recognition because there are much fewer one-character than two-character Chinese words (e.g., 5,321 vs. 45,871 in SUBTLEX-CH55). It is not yet clear why gamma-band power increased in response to tonal repetition but decreased in response to segmental repetition. The different direction of gamma-band modulation for tonal and segmental repetition may be a sign of functional dissociation between tonal and segmental information. However, this explanation seems at odds with the idea that a gamma-band decrease in response to segmental repetition is the neural consequence of repetition suppression. Alternatively, a gamma-band increase elicited by tonal repetition could be interpreted as repetition enhancement56. Stimuli-related repetition enhancement has in fact been observed in other studies46,49,56, and the contrast between repetition suppression and enhancement might be related to differences in the types of repeated stimuli. For example, gamma-band power has been found to increase when pseudowords were repeated, whereas gamma-band power reduction has been found for word repetition49.
The different patterns of gamma-band power modulation in segmental and tonal repetition could be related to the distribution of repeated segments and tones. There are in Chinese 404 different syllables57, whereas there are only four different tones. Thus, segmental repetition may be more salient than tonal repetition. Furthermore, native Chinese speakers often rely on Pinyin to enter Chinese characters on computers and mobile phones. Because only segmental information is required for typing Pinyin, segmental information may be activated more often than tonal information. Thus, native Chinese speakers may be more sensitive to segmental repetition than tonal repetition. However, these interpretations are speculative, and therefore further research is needed to examine why gamma band modulations are affected differently by segmental and tonal repetition.
In sum, we showed that brain oscillations are influenced by Chinese phonological activation during visual word recognition in Chinese-English bilinguals. We obtained converging behavioural and electrophysiological evidence that segmental information of Chinese translations is activated during English word reading, indicating that previous phonological repetition effects observed in the literature are likely due to specifically Chinese segmental repetition. In contrast, both segmental and tonal information are activated during Chinese word recognition. Importantly, tonal repetition effects were only found in the time-frequency data, illustrating that oscillatory EEG data are crucial in detecting the activation of tonal features of words. The present study advances our understanding of the role of Chinese segmental and tonal information in visual word recognition and highlights the importance of brain oscillations in studying language processing.
In total, 64 Chinese-English bilinguals participated (32 in each experiment) and they were all paid for their participation. Participants were all native Mandarin Chinese speakers who learnt Chinese from birth, and they were undergraduate or postgraduate university students studying in Nottingham. They all met the minimum English language entry requirements to study at the University of Nottingham (IELTS 6.0 for undergraduates and 6.5 for postgraduates). The details of participants’ language background are summarized in Table 1. All participants had normal or corrected-to-normal vision and had no history of neurological or psychiatric impairment. According to an adapted version of Coren’s handedness questionnaire58 (available at http://faculty.washington.edu/chudler/rltablen.html), all participants were right-handed. For the English experiment, one participant was excluded due to high error rates (>25%), and one participant was excluded due to excessive eye blinks. Because the data were recorded during a hot summer in a lab without air-conditioning, an additional 11 participants in the English experiment and nine participants in the Chinese experiment were excluded from the analyses due to slow drifts. A minimum of 40 epochs were required for each condition and the analyses were based on 19 participants in the English experiment and 23 participants in the Chinese experiment.
Materials and Design
Using the data from the English-Chinese translation norming study reported in Wen and van Heuven59, 240 experimental English words pairs with similar word frequency, length, and concreteness were selected to create four experimental conditions in the English experiment: +Segment +Tone, +Segment −Tone, −Segment +Tone and −Segment −Tone. For the Chinese experiment, the materials and design were identical to the English experiment except that participants read the Chinese translations of the English word pairs. Examples of stimuli are presented in Table S2 of the Supplementary Material. All critical 240 word pairs were unrelated in meaning, whereas 60 filler pairs were semantically related.
The experiments were approved by the Ethics Committee of the School of Psychology, University of Nottingham. The experiments were performed in accordance with approved guidelines and regulations. All participants gave their written informed consent before the experiment started. Participants were tested individually in a dimly-lit shielded experimental room. Each participant received a unique random presentation order of the stimuli. Stimuli were presented on a CRT monitor (17″ Liyama Vision Master 400, 1024 × 768 pixels, 85 Hz) controlled by PsychoPy60. Each trial began with a fixation cross, presented for 500 ms followed by a 300 ms black screen. Next, the prime word (in lowercase for the English experiment) was presented for 500 ms, followed by a variable interstimulus interval of 500, 600 or 700 ms14,16 after which the target word (in uppercase for the English experiment) with a fixed presentation time of 1000 ms was presented. Participants had to decide whether the presented word pairs (prime-target) were related in meaning or not by pressing buttons on a Cedrus response box attached to the computer. They were instructed to perform the task as quickly and as accurately as possible. If participants did not respond during the presentation of the target, a blank screen would appear for up to a maximum of 2000 ms or until the participants responded. Response sides were counterbalanced between participants. The inter-trial interval was set at 1000 ms.
Participants in the English experiment made semantic relatedness judgments to 300 English word pairs, and participants in the Chinese experiment performed the same task to Chinese translations of the 300 English word pairs. Prior to the experiment, a practice block of 15 trials was used to familiarize the participants with the procedure. Participants were instructed to minimize blinks, eye-movements, and body movements during stimuli presentation. After the semantic relatedness judgment task, participants completed an English vocabulary test61 and a language history questionnaire. The entire session for each experiment lasted about two hours.
EEG Recording and Preprocessing
The electroencephalogram (EEG) was recorded continuously at a sample rate of 1024 Hz using a Biosemi ActiveTwo system with 64 Ag/AgCl active electrodes placed according to the 10/20 system. Two additional electrodes placed close to Pz, namely, the Common Mode Sense [CMS] active electrode and the Driven Right Leg [DRL] passive electrode, were used for online referencing (for detailed explanations, see62,63). In addition to the 66 electrodes on the scalp, seven external electrodes were applied: two over left and right mastoids for off-line re-referencing, two at the outer canthus of each eye, two below each eye and one above the left eye for monitoring eye blinks and movements. For each measurement site, the voltage offset from the CMS was below 25 mV.
Data were preprocessed and analysed using FieldTrip64. The continuous EEG was filtered offline with a high-pass filter set at 0.1 Hz. The EEG data were then re-referenced off-line to the averaged mastoids and segmented in epochs of 3000 ms beginning 1000 ms prior to the onset of the second word (a long epoch is required in time-frequency analyses, because the value of the power spectrum at a given time point is a weighted average of values within a certain time window centred at that point, see27,28). Trials with incorrect responses or RTs less than 300 ms or more than 2,500 ms were discarded (2.8% of the data). Epochs contaminated by drifts and muscle activity were first manually dismissed (8.4% of the data, mostly due to drift), and eye-blinks and eye movements were corrected using an independent component analysis (ICA) algorithm65. The epochs with potentials greater than 100 μV were automatically rejected (16.0% of the data). For the data of each participant to be included in the analysis, a minimum of 40 epochs were required for each condition. The preprocessed data were then used for the ERP analysis and the time-frequency analysis.
The clean EEG data were re-segmented in epochs of 1100 ms beginning 100 ms prior to the onset of the second word. Baseline correction was applied, using the interval from −100 to 0 msec. The epochs were low-pass filtered at 30 Hz. Finally, the average of the ERPs was computed for each participant in each of the experimental conditions.
The preprocessed data were used for the time-frequency analyses. The power spectra were computed for all single trials using complex Morlet wavelets66. A wavelet of seven-cycle width was used, with frequencies increasing from 5 to 60 Hz, in 1-Hz steps. The individual trial power spectra were then averaged across trials for each condition and participant. The powers obtained were expressed as a percentage change relative to the power in a baseline interval from 300 to 100 ms before the target words.
For the behavioural data, a 2 (+Segment vs. −Segment) × 2 (+Tone vs. −Tone) repeated measures ANOVA was run for both reaction times and error rates.
The ERP data were analysed using the cluster-based random permutation test as implemented in FieldTrip (for details see67). The ERP data analysis of the English experiment focused on the N400 component14,16,18, and for the analyses nine electrodes were selected (FC1, FC2, FCz, C1, C2, Cz, CP1, CP2, CPz), which showed maximal effects in previous studies14,16,18. Thus, a cluster-based random permutation test was applied on an a-priori region of interest from 350 ms to 500 ms post-stimulus for the main effect of Segment and Tone and their interaction. The focus of the ERP analysis in the Chinese experiment was the P200 component7,13, the N400 component14,16,18 and the LPC component33,38 (see Supplementary Material for more discussion). To capture these three components, the cluster-based random permutation test was applied from 150 ms to 800 ms post-stimulus on all 64 electrodes.
For the time-frequency data, power in percentage change relative to baseline was tested statistically using the cluster-based random permutation test implemented in FieldTrip. We did not have prior knowledge about which frequency bands, time windows, and electrodes would be sensitive to the effect of Chinese phonological activation during English word reading. Therefore, the time-frequency results from the Chinese experiment were used to guide the time-frequency analysis of the English experiment. For the Chinese experiment, cluster-based random permutation tests were separately conducted for low- (5–30 Hz) and high-(30–60 Hz) frequency ranges68,69 from 200 ms to 800 ms post-stimulus on the main effects of Segment and Tone and their interaction. Therefore, cluster-based random permutation tests were conducted over each frequency, time and electrode. Because significant changes were found in the gamma-band power (30–60 Hz) approximately 400–800 ms post-stimulus, the gamma-band power (30–60 Hz) was first averaged for each electrode and each condition in the English experiment to increase the sensitivity of the permutation test67. For the main effects of Segment and Tone and their interaction, a cluster-based random permutation test was then conducted for the averaged gamma-band power from 400 ms to 800 ms post-stimulus. Therefore, cluster-based random permutation tests in the English experiment were conducted over time and electrode.
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Norris, D. Models of visual word recognition. Trends Cogn. Sci. 17, 517–524, https://doi.org/10.1016/j.tics.2013.08.003 (2013).
Carreiras, M., Perea, M., Vergara, M. & Pollatsek, A. The time course of orthography and phonology: ERP correlates of masked priming effects in Spanish. Psychophysiology 46, 1113–1122, https://doi.org/10.1111/j.1469-8986.2009.00844.x (2009).
Ferrand, L. & Grainger, J. Phonology and orthography in visual word recognition: Evidence from masked non-word priming. Q. J. Exp. Psychol. A 45, 353–372, https://doi.org/10.1080/02724989208250619 (1992).
Ferrand, L. & Grainger, J. The time course of orthographic and phonological code activation in the early phases of visual word recognition. Bull. Psychon. Soc. 31, 119–122, https://doi.org/10.3758/BF03334157 (1993).
Grainger, J., Kiyonaga, K. & Holcomb, P. J. The time course of orthographic and phonological code activation. Psychol. Sci. 17, 1021–1026, https://doi.org/10.1111/j.1467-9280.2006.01821.x (2006).
Rastle, K. & Brysbaert, M. Masked phonological priming effects in English: Are they real? Do they matter? Cogn. Psychol. 53, 97–145, https://doi.org/10.1016/j.cogpsych.2006.01.002 (2006).
Kong, L. et al. P200 and phonological processing in Chinese word recognition. Neurosci. Lett. 473, 37–41, https://doi.org/10.1016/j.neulet.2010.02.014 (2010).
Liu, Y., Perfetti, C. A. & Hart, L. ERP evidence for the time course of graphic, phonological, and semantic information in Chinese meaning and pronunciation decisions. J. Exp. Psychol. Learn. Mem. Cogn. 29, 1231, https://doi.org/10.1037/0278-73126.96.36.1991 (2003).
Perfetti, C. A. & Tan, L. H. The time course of graphic, phonological, and semantic activation in Chinese character identification. J. Exp. Psychol. Learn. Mem. Cogn. 24, 101–118, https://doi.org/10.1037/0278-73188.8.131.52 (1998).
Perfetti, C. A. & Zhang, S. Phonological processes in reading Chinese characters. J. Exp. Psychol. Learn. Mem. Cogn. 17, 633–643 (1991).
Perfetti, C. A. & Zhang, S. Very early phonological activation in Chinese reading. J. Exp. Psychol. Learn. Mem. Cogn. 21, 24–33, https://doi.org/10.1037/0278-73184.108.40.206 (1995).
Spinks, J. A., Liu, Y., Perfetti, C. A. & Tan, L. H. Reading Chinese characters for meaning: The role of phonological information. Cognition 76, B1–B11, https://doi.org/10.1016/S0010-0277(00)00072-X (2000).
Zhang, Q., Zhang, J. X. & Kong, L. An ERP study on the time course of phonological and semantic activation in Chinese word recognition. Int. J. Psychophysiol. 73, 235–245, https://doi.org/10.1016/j.ijpsycho.2009.04.001 (2009).
Thierry, G. & Wu, Y. J. Brain potentials reveal unconscious translation during foreign language comprehension. Proc. Natl. Acad. Sci. USA 104, 12530–12535, https://doi.org/10.1073/pnas.0609927104 (2007).
Wu, Y. J., Cristino, F., Leek, C. & Thierry, G. Non-selective lexical access in bilinguals is spontaneous and independent of input monitoring: Evidence from eye tracking. Cognition 129, 418–425, https://doi.org/10.1016/j.cognition.2013.08.005 (2013).
Wu, Y. J. & Thierry, G. Chinese-English bilinguals reading English hear Chinese. J. Neurosci. 30, 7646–7651, https://doi.org/10.1523/JNEUROSCI.1602-10.2010 (2010).
Wu, Y. J. & Thierry, G. Unconscious translation during incidental foreign language processing. NeuroImage 59, 3468–3473, https://doi.org/10.1016/j.neuroimage.2011.11.049 (2012).
Wu, Y. J. & Thierry, G. How reading in a second language protects your heart. J. Neurosci. 32, 6485–6489, https://doi.org/10.1523/JNEUROSCI.6119-11.2012 (2012).
Li, C., Lin, C.-Y., Wang, M. & Jiang, N. The activation of segmental and tonal information in visual word recognition. Psychon. Bull. Rev. 20, 773–779, https://doi.org/10.3758/s13423-013-0395-2 (2013).
Perfetti, C. A., Liu, Y. & Tan, L. H. The lexical constituency model some implications of research on Chinese for general theories of reading. Psychol. Rev. 112, 43–59, https://doi.org/10.1037/0033-295X.112.1.43 (2005).
Zhao, X. & Li, P. Simulating cross-language priming with a dynamic computational model of the lexicon. Biling. Lang. Cogn. 16, 288–303, https://doi.org/10.1017/S1366728912000624 (2013).
Zhao, X. & Li, P. An online database of phonological representations for Mandarin Chinese. Behav. Res. Meth. 41, 575–583, https://doi.org/10.3758/BRM.41.2.575 (2009).
Frost, R. Toward a strong phonological theory of visual word recognition: True issues and false trails. Psychol. Bull. 123, 71–99, https://doi.org/10.1037/0033-2909.123.1.71 (1998).
Frost, R. Towards a universal model of reading. Behav. Brain Sci. 35, 263–279, https://doi.org/10.1017/S0140525X11001841 (2012).
Grainger, J. & Ziegler, J. A dual-route approach to orthographic processing. Front. Psychol. 2 https://doi.org/10.3389/fpsyg.2011.00054 (2011).
Dijkstra, T. & van Heuven, W. J. B. The architecture of the bilingual word recognition system: From identification to decision. Biling. Lang. Cogn. 5, 175–197, https://doi.org/10.1017/S1366728902003012 (2002).
Cohen, M. X. Analyzing neural time series data: Theory and practice. (MIT Press 2014).
Herrmann, C. S., Rach, S., Vosskuhl, J. & Strüber, D. Time–frequency analysis of event-related potentials: A brief tutorial. Brain Topogr. 27, 438–450, https://doi.org/10.1007/s10548-013-0327-5 (2014).
Herrmann, C. S., Grigutsch, M. & Busch, N. A. In Event-related potentials: A methods handbook (ed T. Handy) 229–259 (MIT Press 2005).
Bastiaansen, M. C. M. & Hagoort, P. Oscillatory neuronal dynamics during language comprehension. Prog. Brain Res. 159, 179–196, https://doi.org/10.1016/S0079-6123(06)59012-0 (2006).
Bastiaansen, M. C. M., Mazaheri, A. & Jensen, O. In The Oxford handbook of event-related potential components (eds S. J. Luck & E. S. Kappenman) 31–49 (Oxford University Press 2011).
Hagoort, P., Hald, L., Bastiaansen, M. & Petersson, K. M. Integration of word meaning and world knowledge in language comprehension. Science 304, 438–441, https://doi.org/10.1126/science.1095455 (2004).
Guo, T., Misra, M., Tam, J. W. & Kroll, J. F. On the time course of accessing meaning in a second language: An electrophysiological investigation of translation recognition. J. Exp. Psychol. Learn. Mem. Cogn. 38, 1165–1186, https://doi.org/10.1037/a0028076 (2012).
Morford, J. P., Wilkinson, E., Villwock, A., Piñar, P. & Kroll, J. F. When deaf signers read English: Do written words activate their sign translations? Cognition 118, 286–292, https://doi.org/10.1016/j.cognition.2010.11.006 (2011).
Marian, V. & Spivey, M. Competing activation in bilingual language processing: Within-and between-language competition. Biling. Lang. Cogn. 6, 97–115, https://doi.org/10.1017/S1366728903001068 (2003).
Meade, G., Midgley, K. J., Sevcikova Sehyr, Z., Holcomb, P. J. & Emmorey, K. Implicit co-activation of American Sign Language in deaf readers: An ERP study. Brain Lang. 170, 50–61, https://doi.org/10.1016/j.bandl.2017.03.004 (2017).
van Heuven, W. J. B., Dijkstra, T. & Grainger, J. Orthographic neighborhood effects in bilingual word recognition. J. Mem. Lang. 39, 458–483, https://doi.org/10.1006/jmla.1998.2584 (1998).
Misra, M. & Holcomb, P. J. Event–related potential indices of masked repetition priming. Psychophysiology 40, 115–130, https://doi.org/10.1111/1469-8986.00012 (2003).
Kielar, A., Meltzer, J. A., Moreno, S., Alain, C. & Bialystok, E. Oscillatory responses to semantic and syntactic violations. J. Cognitive Neurosci. 26, 2840–2862, https://doi.org/10.1162/jocn_a_00670 (2014).
Litcofsky, K. A. & van Hell, J. G. Switching direction affects switching costs: Behavioral, ERP and time-frequency analyses of intra-sentential codeswitching. Neuropsychologia 97, 112–139, https://doi.org/10.1016/j.neuropsychologia.2017.02.002 (2017).
Grabner, R. H., Brunner, C., Leeb, R., Neuper, C. & Pfurtscheller, G. Event-related EEG theta and alpha band oscillatory responses during language translation. Brain Res. Bull. 72, 57–65, https://doi.org/10.1016/j.brainresbull.2007.01.001 (2007).
Bakker, I., Takashima, A., van Hell, J. G., Janzen, G. & McQueen, J. M. Changes in theta and beta oscillations as signatures of novel word consolidation. J. Cognitive Neurosci. https://doi.org/10.1162/jocn_a_00801 (2015).
Kaan, E., Wayland, R. & Keil, A. Changes in oscillatory brain networks after lexical tone training. Brain Sci. 3, 757, https://doi.org/10.3390/brainsci3020757 (2013).
Küssner, M. B., de Groot, A. M. B., Hofman, W. F. & Hillen, M. A. EEG beta power but not background music predicts the recall scores in a foreign-vocabulary learning task. Plos One 11, e0161387, https://doi.org/10.1371/journal.pone.0161387 (2016).
Herrmann, C. S., Fründ, I. & Lenz, D. Human gamma-band activity: A review on cognitive and behavioral correlates and network models. Neurosci. Biobehav. Rev. 34, 981–992, https://doi.org/10.1016/j.neubiorev.2009.09.001 (2010).
Gruber, T. & Müller, M. M. Oscillatory brain activity dissociates between associative stimulus content in a repetition priming task in the human EEG. Cereb. Cortex 15, 109–116, https://doi.org/10.1093/cercor/bhh113 (2005).
Gruber, T. & Müller, M. M. Effects of picture repetition on induced gamma band responses, evoked potentials, and phase synchrony in the human EEG. Cogn. Brain Res. 13, 377–392, https://doi.org/10.1016/S0926-6410(01)00130-6 (2002).
McDonald, C. R. et al. Multimodal imaging of repetition priming: Using fMRI, MEG, and intracranial EEG to reveal spatiotemporal profiles of word processing. NeuroImage 53, 707–717, https://doi.org/10.1016/j.neuroimage.2010.06.069 (2010).
Fiebach, C. J., Gruber, T. & Supp, G. G. Neuronal mechanisms of repetition priming in occipitotemporal cortex: Spatiotemporal evidence from functional magnetic resonance imaging and electroencephalography. J. Neurosci. 25, 3414–3422, https://doi.org/10.1523/jneurosci.4107-04.2005 (2005).
Grill-Spector, K., Henson, R. & Martin, A. Repetition and the brain: Neural models of stimulus-specific effects. Trends Cogn. Sci. 10, 14–23, https://doi.org/10.1016/j.tics.2005.11.006 (2006).
Schacter, D. L. & Buckner, R. L. Priming and the brain. Neuron 20, 185–195, https://doi.org/10.1016/S0896-6273(00)80448-1 (1998).
Matsumoto, A. & Iidaka, T. Gamma band synchronization and the formation of representations in visual word processing: Evidence from repetition and homophone priming. J. Cognitive Neurosci. 20, 2088–2096, https://doi.org/10.1162/jocn.2008.20136 (2008).
Wang, X., Liu, A., Wu, Y. & Wang, P. Rapid extraction of lexical tone phonology in Chinese characters: A visual mismatch negativity study. Plos One 8, e56778, https://doi.org/10.1371/journal.pone.0056778 (2013).
Wang, X., Wu, Y., Liu, A. & Wang, P. Spatio-temporal dynamics of automatic processing of phonological information in visual words. Sci. Rep. 3 https://doi.org/10.1038/srep03485 (2013).
Cai, Q. & Brysbaert, M. SUBTLEX-CH: Chinese word and character frequencies based on film subtitles. Plos One 5, e10729, https://doi.org/10.1371/journal.pone.0010729 (2010).
Segaert, K., Weber, K., de Lange, F. P., Petersson, K. M. & Hagoort, P. The suppression of repetition enhancement: A review of fMRI studies. Neuropsychologia 51, 59–66, https://doi.org/10.1016/j.neuropsychologia.2012.11.006 (2013).
Duanmu, S. The phonology of standard Chinese. (Oxford University Press 2007).
Coren, S. The Left-Hander Syndrome: The Causes and Consequences of Left-Handedness. (New York 1992).
Wen, Y. & van Heuven, W. J. B. Chinese translation norms for 1429 English words. Behav. Res. Meth., https://doi.org/10.3758/s13428-016-0761-x (2016).
Peirce, J. W. Generating stimuli for neuroscience using PsychoPy. Front. Neuroinform. 2, 10, https://doi.org/10.3389/neuro.11.010.2008 (2008).
Lemhöfer, K. & Broersma, M. Introducing LexTALE: A quick and valid Lexical Test for Advanced Learners of English. Behav. Res. Meth. 44, 325–343, https://doi.org/10.3758/s13428-011-0146-0 (2012).
Metting van Rijn, A. C., Peper, A. & Grimbergen, C. A. High-quality recording of bioelectric events. Med. Biol. Eng. Comput. 28, 389–397 (1990).
Schutter, D. J. L. G., Leitner, C., Kenemans, J. L. & van Honk, J. Electrophysiological correlates of cortico-subcortical interaction: A cross-frequency spectral EEG analysis. Clin. Neurophysiol. 117, 381–387, https://doi.org/10.1016/j.clinph.2005.09.021 (2006).
Oostenveld, R., Fries, P., Maris, E. & Schoffelen, J. M. FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell. Neurosci. 2011, 1–9, https://doi.org/10.1155/2011/156869 (2010).
Jung, T. P. et al. Removing electroencephalographic artifacts by blind source separation. Psychophysiology 37, 163–178, https://doi.org/10.1111/1469-8986.3720163 (2000).
Tallon-Baudry, C., Bertrand, O., Delpuech, C. & Pernier, J. Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. J. Neurosci. 16, 4240–4249 (1996).
Maris, E. & Oostenveld, R. Nonparametric statistical testing of EEG-and MEG-data. J. Neurosci. Meth. 164, 177–190, https://doi.org/10.1016/j.jneumeth.2007.03.024 (2007).
Bastiaansen, M. C. M. & Hagoort, P. Frequency-based Segregation of Syntactic and Semantic Unification during Online Sentence Level Language Comprehension. J. Cognitive Neurosci., 1–13, https://doi.org/10.1162/jocn_a_00829 (2015).
Bauer, M., Stenner, M.-P., Friston, K. J. & Dolan, R. J. Attentional modulation of alpha/beta and gamma oscillations reflect functionally distinct processes. J. Neurosci. 34, 16117–16125, https://doi.org/10.1523/JNEUROSCI.3474-13.2014 (2014).
We would like to thank Markus Bauer for his help with the time-frequency analyses. We are also grateful to Guillaume Thierry and Markus Bauer for their helpful comments on an earlier version of this manuscript. Y.W. was supported by a PhD scholarship from the University of Nottingham and the School of Psychology at the University of Nottingham.
The authors declare no competing interests.
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
About this article
Cite this article
Wen, Y., Filik, R. & van Heuven, W.J.B. Electrophysiological dynamics of Chinese phonology during visual word recognition in Chinese-English bilinguals. Sci Rep 8, 6869 (2018). https://doi.org/10.1038/s41598-018-25072-w
This article is cited by
Scientific Reports (2019)
Journal of Psycholinguistic Research (2019)