One may have experienced his or her footsteps unconsciously synchronize with the footsteps of a friend while walking together, or heard an audience's clapping hands naturally synchronize into a steady rhythm. However, the mechanisms of body movement synchrony and the role of this phenomenon in implicit interpersonal interactions remain unclear. We aimed to evaluate unconscious body movement synchrony changes as an index of implicit interpersonal interaction between the participants, and also to assess the underlying neural correlates and functional connectivity among and within the brain regions. We found that synchrony of both fingertip movement and neural activity between the two participants increased after cooperative interaction. These results suggest that the increase of interpersonal body movement synchrony via interpersonal interaction can be a measurable basis of implicit social interaction. The paradigm provides a tool for identifying the behavioral and the neural correlates of implicit social interaction.
Synchronization is, in a broad sense, coordination of rhythmic oscillators due to their interaction. Interpersonal body movement synchronization has been widely observed. A person's footsteps unconsciously synchronize with those of a partner when two people are walking together, even though their foot lengths, and thus their intrinsic cycles, are different1,2,3. The phenomenon has been thought of as social self-organizing process4. Previous studies found that the degree of interpersonal body movement synchrony, such as finger tapping and drumming, predicted subsequent social ratings5,6. The findings indicate a close relationship between social interface and body movement synchronization7. However, the mechanism of body movement synchrony and its relationship to implicit interpersonal interaction remain vague.
We aimed to evaluate unconscious body movement synchrony and implicit interpersonal interactions between two participants. We also aimed to assess the underlying neural correlates and functional connectivity within and among the brain regions of two participants. Thus we measured unconscious fingertip movements between the two participants while simultaneously recording EEG in a face-to-face setting (Fig. 1A and 1B). Participants were asked to straighten their arms, point and hold their index fingers toward each other, and look at the other participant's fingertip. Face-to-face interactions closely approximate real-life situations and reinforce the social nature of interpersonal interactions8.
We believe that our implicit fingertip synchrony task, as well as unconscious footstep synchrony, interpersonal finger tapping and drumming synchrony, are all forms of social synchronization and our task is the simplest form of such. Thus, we hypothesized that interpersonal interaction between two participants would increase body movement synchronization and the interaction would be correlated with social traits of personality. The traits in turn would possibly be reflected in within- and among-brain synchronizations. More specifically, we expected experience-based changes of synchrony in sensorimotor as well as in theory-of-mind related networks, including the precuneus, inferior parietal and posterior temporal cortex9,10. The parietal cortex is especially expected to be involved, given that implicit processing of emotional stimuli, as compared to explicit emotional processing, is associated with theta synchronization in the right parietal cortex11. Furthermore, the right parietal cortex is a part of the dorsal unconscious processing stream that contributes to vision-for-action12. Previous EEG simultaneous recording studies (i.e., hyperscanning EEG) also showed that the right parietal area played a key role in non-verbal social coordination and movement synchrony13,14.
Simultaneous functional magnetic resonance imaging (fMRI) of two participants, called hyperscanning, has been used to assess brain activity in human behavioral experiments in which participants can interact with each other15,16. However, there are limitations to utilizing fMRI to examine interpersonal interactions. First, neural synchronization, which may be critical for such interactions, usually occurs on a time-scale that is beyond the reach of the temporal resolution capabilities of fMRI14,17,18. In addition, fMRI requires each participant to be tested in an isolated magnetic shield facility, precluding a face-to-face experimental setup. Hyperscanning EEG of two participants may open new vistas on the neural mechanisms underlying social relationships and decision making13,14,19,20 by providing a tool for quantifying neural synchronization in face-to-face interactions with high temporal resolution14,21,22.
Local neural synchronization can be detected by measuring frequency-specific power changes of each electrode component of the EEG. However, local power changes alone cannot provide evidence of large-scale network formation because it depends on oscillatory interactions between spatially distant cortical regions23,24, which may be critical for understanding neural mechanisms during interpersonal interaction. To address this issue, we used phase synchrony to quantify long-range functional connectivity; this would allow us to detect not only intra-brain, but also inter-brain connectivity.
Behavioral results showed that in the post-training sessions, finger movement correlation between two participants was significantly higher in comparison to the pre-training sessions (two-tailed paired t-test, t(9) = 2.51, p < 0.03) (Fig. 1C and Fig. S1). The maximum correlation coefficients occurred at the zero time lag, indicating that the observed synchrony was not due to intentional time-lagged following. Rather, instantaneous, unintentional synchrony occurred between two fingertips. Furthermore, our crosscheck validation, done by random shuffling of participant pairs for the data from the pre-training session, revealed no significant correlations (two-tailed one-sample t-test, t(9) = 0.51, p = 0.62). Thus, the observed synchrony was specific for each pair that was physically facing each other at a close distance. Additional control experiments consistently suggest that non-social or non-responsive training cannot increase the fingertip synchrony (See Fig. S2 and Supplementary Methods for the details of the control conditions). These results are consistent with those of previous studies on unconscious mimicry, which have found that one's behavior passively and unintentionally changes to match that of others25,26.
To see if the increase in fingertip synchrony is related to known social traits of personality, we performed a correlation analysis between the finger movement synchrony changes (post – pre-training sessions) and scales for social anxiety using the Leary's scale27. We found significant negative correlations of the fingertip synchrony increase with each pair's averaged scores of ‘Fear of Negative Evaluation' and ‘Blushing Propensity' (Spearman's rho, p < 0.05, n = 22) (Table S1), indicating that the more the person has social anxiety, the less the fingertip synchrony increases.
In our EEG source localization results, we observed that theta (4~7.5 Hz) frequency activity in the precuneus (PrC) and beta (12~30 Hz) frequency activity in the right posterior middle temporal gyrus (MTG) increased significantly (nonparametric permutation test, p < 0.001, n = 20) after training (Fig. 2A and 2B). According to a quantitative meta-analysis of 70 functional neuroimaging studies, the right inferior parietal and posterior temporal cortex have been suggested to play a critical role in various aspects of social cognition, such as theory of mind and empathy9. These regions are also known to be engaged in the lower-level (bottom-up) computational processes associated with the sense of agency and in reorienting attention to salient stimuli9. The increase of the right posterior MTG activation in our study, as a domain-general social cognitive processes from lower to higher levels9, is consistent with our hypothesis that body movement synchrony is an implicit basis of interpersonal interaction.
We performed a regression analysis between the changes in fingertip synchrony and the changes in brain activity from pre- to post-training sessions. We found a significant positive correlation between the fingertip synchrony changes from post- to pre-training sessions and the ventromedial prefrontal cortex (VMPFC) theta frequency activity (regression with nonparametric permutation test, p < 0.05, n = 20) (Fig. 2C). It is consistent with a previous finding indicating VMPFC as a shared circuit for reflective representations of both self (i.e., introspection) and other (i.e., theory of mind)28.
The functional connectivity analysis found that the overall number of significant phase synchrony in inter-brain connections increased after training (chi-square test, theta: χ2(1) = 8.4, p < 0.005; beta: χ2(1) = 15.4, p < 0.0001), but not in intra (chi-square test, theta: χ2(1) = 1.3, p = 0.26; beta: χ2(1) = 1.3, p = 0.25) (Fig. 3A). Thus, the cooperative training increased synchrony not only between fingertip movements of the two participants but also between cortical regions across the two brains. This may be taken as additional evidence that such inter-brain synchrony is a neural correlate of implicit interpersonal interaction. Inter-brain connections were found mainly in the inferior frontal gyrus (IFG), anterior cingulate (AC), parahippocampal gyrus (PHG), and postcentral gyrus (PoCG) (phase randomization surrogate statistics, p < 0.000001) (Fig. 3B and 3C). Increased IFG synchronization is consistent with our hypothesis in that the training effect may be attributed to the increased coupling between the one's own introspective and the other's theory of mind networks. The PHG and medial frontal regions are known to be involved in detecting social contextual cues29,30 and PoCG is the primary somatosensory area. Furthermore, the fact that inter-brain synchrony was disrupted in the random shuffling cross-validation analysis confirmed that those connections were not caused by chance or artifact, but were due to the paired face-to-face interaction (Fig. S3A and S3B).
Interestingly, the inter-brain synchrony was not symmetric (Fig. S3C and S3D, phase randomization surrogate statistics, p < 0.000001). This may look peculiar at first glance because the tasks during the pre- and the post-trainings were identical for two participants. The only difference in the instructions for the two participants was that one participant (the leader) moved a finger spontaneously and randomly, whereas the other (the follower) followed the leader's finger in the training session. Furthermore, more influences were detected from the brains of participants who performed the following role rather than the leading role in the training session. One possibility was that the leader might be implicitly inclined to help the follower to match the finger. If so, involvement of the regions that had been noted in relation to social processing, including IFG, middle frontal gyrus (MFG), and PHG29,30 would be expected. Also, the follower was required to actively match the fingertip movement with the leader's. This could be the reason why, in addition to the social regions, the regions related to socio-motor and motor-visual interactions, including IFG, MFG, PoCG, precentral gyrus (PrCG), and lingual gyrus (LG), were activated in the follower's brain. We did not find inter-brain connections between the motor or visual cortices. Thus, the increased synchrony we found may be mainly due to triggering implicit interpersonal interaction, not to mere sensory inputs or top-down commands to move in synchrony. The increase of synchrony may occur without the direct synchrony between the motor or visual cortices.
We devised the novel combination of hyperscanning EEG and motion tracking with the implicit body movement synchronization paradigm31. The advantage of this experimental paradigm is twofold. First, we were able to detect an implicit-level interaction that is interpersonal and real time in nature. Previous studies have mainly concentrated on explicit social interactions, i.e., with mutual explicit intention to communicate, such as observing conversation10 and playing economic games16,19, where tasks are possibly complicated by both implicit and explicit aspects of the interaction. However, we aimed to identify an implicit process by minimizing explicit interaction. We achieved this by using simple tasks such as looking at the other participant's finger while holding one's own finger stationary (in the pre- and post-training sessions). Participants had to intentionally neglect the movement of the partner and stay stationary, as the task demanded. Nonetheless, the movement could not be neglected and the participants tended to unconsciously synchronize each other. In this specific sense, the task and the results are social, yet implicit. The fingertip movement synchronization could be potentially applied as a somatic measure of implicit interpersonal interaction. The second advantage of the paradigm rests on the fact that since the instructions in our experimental paradigm were to stay stationary, movement artifacts were minimized in the EEG data. Robustness to noise during face-to-face interaction makes our experimental paradigm optimally sensitive to the underlying EEG dynamics and the functional connectivity of implicit interpersonal interaction.
The increase of fingertip synchrony after the training session indicates that the large, voluntary, and intentional mimicry affects the small, involuntary, and unintentional body movement synchronization afterwards. Two participants seem to build their own rhythmic structure during the intentional mimicry training, resulting in increased unintentional synchronization. It is consistent with, and extending the previous studies showing that motor mimicry increased implicit social interaction between two interacting participants25, and that the spontaneous bi-directional improvisation (i.e. implicit synchronization) increased motor synchrony compared with the uni-directional imitation (i.e. explicit following)32. Correlations between the fingertip synchrony increase and the social anxiety scales further support that the increased fingertip synchrony could be a marker of implicit social interaction. In addition, the drastic decrease of variation of synchrony should be noted. We only suspect the possibility of the ceiling effect of correlation coefficient, but it was also small at larger absolute values of time delay. This might possibly be interpreted as a part of the effect of the cooperative training and further examinations may be needed.
One may still expect that the leader-follower asymmetry in the training would create a lag in the post-training test, even though the instructions for both participants remained the same as the pre-training test, just to stay stationary. According to previous studies as described in the introduction, however, unintentional body movement synchrony with zero-lag is widely observed, such as walking together and clapping. When two people are walking together and they do not pay attention, sometimes one person leads, and sometimes the other does alternately and irregularly, but overall, they are more or less in sync due to a natural tendency, the expected outcome would be as correlation peak at lag zero with broad skirts. What we found in our study is qualitatively the same. The only difference was that the correlation was observed without and against the participant's intention.
We found the positive correlation between the VMPFC activity and fingertip synchrony changes from post- to pre-training sessions (Fig. 2C). Meanwhile, the fingertip synchrony increase was negatively associated with individual social anxiety level. Taken together, the VMPFC may be involved in implicit social interaction. However, it remains to be a speculation, which is consistent with the previous findings that the VMPFC has been associated with not only explicit33,34, but also implicit social and emotional attitudes35, including implicit gender bias36, automatic activation of political attitudes37, and implicit preference38.
We also found that theta and beta phase synchronization occurred between two brains due to the training. Long-range theta phase synchronization has been known for its role in implicit social and emotional processing11,39. A previous hyperscanning-EEG study reported beta inter-brain synchronization between the central and right parieto-occipital regions, and interpreted it as a brain-to-brain top-down modulation in social interaction14. Furthermore, neural activities at beta frequency can synchronize over long distances, which makes it a likely candidate for neural correlates of higher-level interactions40. Thus, the synchronization between the parietal (PoCG) and central regions (AC) in theta and beta frequency range may be considered as such, for implicit social processing and its top-down modulation. The fact that the neuroelectric signals are synchronized at the millisecond timescale, which is much faster than the finger movements, also supports that the neural synchrony we found was not a mere consequence of motor synchrony, but more dynamically linked with the behavioral synchrony.
We found no alpha activity in our study. Alpha rhythm has been known to be involved in the mirror neuron system, integrating and processing motor and social information41,42. Previous hyperscanning-EEG studies showed inter-brain alpha synchronization between the right centro-parietal regions, indicating socially coordinated dynamics of intentional body movements13,14. However, the current experiment did not involve explicit and intentional motor movements, thus no significant activation should be expected in the motor regions. It could be a part of the reason why we found no significant alpha activity.
Our current findings suggest that IFG, AC, PHG, and PoCG may be the neurophysiological substrates of inter-brain synchronization. Frontal (IFG)-parietal (PoCG) interaction as a frontoparietal network has been implicated in social cognitive processes35,43,44, especially playing a key role in interpersonal awareness45. Anticipation of somatosensory events, which may be related to the current experimental setup (where two fingers of two participants stayed close though not touched), is associated with the central (AC and PHG) and parietal (PoCG) coupling46. This centroparietal coupling is also involved in top-down modulatory controls14. Along these lines, in our experiments, the frontoparietal and centroparietal networks may have been crucially involved in body and neural synchronization. It may appear peculiar to some that the inter-brain neurophysiological substrates were temporally synchronized, but spatially asymmetrical. As we mentioned earlier in the discussion, the neural synchrony was dynamically linked with the behavioral synchrony and so the behavior and brain do not have to be one-to-one correspondence. Also, note that the time scales for synchronization are different in the factor of at least 10; the neural synchrony being at 100 milliseconds whereas behavioral synchrony being extended to several seconds. We would rather argue that the two brains worked as one neurophysiological network to dynamically process implicit social functioning and modulatory sensory-motor control.
Our novel experimental paradigm serves as a simple, accurate, and efficient implicit measure of interpersonal interaction. While behavioral synchrony in the daily activity such as walking or clapping has been reported, the present effects are more implicit, and at microscopic temporal and spatial scales. The simplicity of the experiment can minimize the movement artifact in the EEG data so that the task is optimally sensitive for hyperscanning-EEG study to identify underlying EEG dynamics of implicit interpersonal interaction. The negative correlations of the fingertip synchrony changes with the scales of social anxiety verify that our task can be a marker of implicit social interaction. To the best of our knowledge, this is the first study to demonstrate that not only the fingertip movement but also the neural activity between the two participants synchronized after cooperative interaction. Moreover, inter- and intra-brain connectivity results in the EEG analysis may have both theoretical and methodological implications towards understanding the neural mechanisms underlying impaired social interaction.
Twenty healthy male right-handed subjects participated in this study (mean age: 24.5 ± 4.05 years). All participants provided written informed consent after receiving a detailed explanation of the experimental procedures. The Institutional Review Board of California Institute of Technology approved all experimental procedures for this study. Participants were excluded if they had a history of neurological disorder such as seizure, stroke, head injury, or a substance use disorder other than caffeine or nicotine. We only recruited male participants to control for the effect of gender in implicit interpersonal interaction during the experiment47.
The experiment consisted of 8 sessions: 2 sessions of pre-training; 4 sessions of training; and 2 sessions of post-training. Each session lasted 1 minute. In the pre- and post-training sessions, participants straightened their arms and pointed and held their index fingers stationary toward each other and looked at the other participant's fingertip (Fig. 1A). They were instructed to look at the other participant's finger while holding his own finger as stationary as possible. The task and the instruction were selected on purpose to minimize movement artifacts in the EEG recording. The distance between the two participants' fingertips was approximately 5 cm. In the training session, one participant (leader) randomly moved his finger (in the approximate area of 20×20 cm square) and the other (follower) followed. Both participants matched the direction of their hands to maintain mirror position, i.e., one participant used his right hand and the other used his left. They alternated their hands in each session. Additional control conditions include (1) eight consecutive non-training sessions (i.e., the same as the pre- and the post-training sessions), and (2) single subject interacting with a dot moving on the screen which was based on the recorded finger movement and (3) with a recorded video of another subject.
During the experiment, infrared reflection markers were attached to both participants' fingertips. All of the fingertip motion data along the X, Y, and Z-axes were captured by a Vicon motion tracking system (Vicon MX; Vicon Motion Systems) with four Vicon cameras at the rate of 100 Hz. X-axis lies from left to right, Y-axis lies from front to back, and Z-axis lies from up to down in the human anatomical position. The origin of the Vicon coordinate system lied on the tip of one participant's fingertip. The 3D reconstruction function in the Vicon software pipeline calculates and reconstructs the 2D optical data from each camera and constructs a 3D motion data. We calculated the cross-correlation coefficients along the −10~10 sec duration (time lags) of both the X and Z-axes movements of two participants' fingertips. Thus for instance, if one follows the other's movements with approximately 1 sec. delay, then we would expect the peak of synchrony at 1 sec. delay. Paired t-test was performed on the maximum correlation coefficients across the time lag.
Electrophysiological recording and analysis
We recorded simultaneous EEG activity from 128 scalp electrodes (EGI System 200; Electrical Geodesics) in the electromagnetically shielded room. Electrode impedance was kept under 40 kΩ for all recordings. A previous study found that there was no significant difference in EEG data quality as scalp-electrode impedance increased from under 10 kΩ (abraded skin) to 40 kΩ (intact skin)48. Vertical and horizontal ocular movements were also recorded. The EEG was continuously recorded (sampling frequency 500 Hz) and filtered (high pass 0.1 Hz, low pass 200 Hz, notch filter 60 Hz and 120 Hz). Two EEG recording systems were synchronized using a pulse signal from the control server computer delivered to both EEG recording systems. Epochs with eye movement or other movement artifacts were manually removed by inspection. Ocular artifact reduction was performed using ICA component rejection by using EEGLAB49. We examined 10~50 sec data and randomly selected 6 artifact free 5 sec epochs. We computed each epoch's source and phase synchrony values and averaged across within subject.
Standardized low resolution brain electromagnetic tomography (sLORETA)50 was used for source localization, and then the source-localized time series were used for functional connectivity analysis. Extracranial EEG activity consists of synchronized postsynaptic potentials (PSPs) of pyramidal cells generated in the cortex of the brain51. sLORETA makes it possible to achieve the three dimensional localization of these highly synchronized PSPs. We allocated individual voxels of cortical regions to the raw LORETA values. These voxels correspond to Brodmann areas (BA) on the basis of the coordinates of the digitized Talairach Atlas52. The subject-specific 3D coordinates of the 128 electrode positions were estimated (Geodesic Photogrammetry System; Electrical Geodesics) and applied to a digitized MRI version of the Talairach Atlas (McConnell Brain Imaging Centre, Montréal Neurological Institute, McGill University). These Talairach coordinates were then used to compute the sLORETA transformation matrix. Following the transformation to an average reference, the EEG activity of each time and frequency band was used to calculate cross spectra in sLORETA for each participant. Using the sLORETA transformation matrix, the cross spectra of each participant and for each time (6 randomly selected 5 sec epochs) and frequency band (delta, 2~3.5 Hz; theta, 4~7.5 Hz; alpha, 8~12 Hz; beta, 12~30 Hz) were then transformed into sLORETA files. These files included the 3D cortical distribution of the electrical neuronal generators for each participant. The computed sLORETA image displayed the cortical neuronal oscillators in 6239 voxels, with a spatial resolution of 5 mm53. We used a nonparametric permutation test with 5000 randomizations for comparisons between the pre- and post-training, corrected for multiple comparisons54. The threshold was set to p < 0.001.
Phase synchrony between the two participants' brain regions
The source localized data were analyzed using a fast Fourier transform. The amplitude and phase were computed for each time epoch and frequency bin. C(f)r and C(f)i can be defined as the real and imaginary Fourier coefficients obtained for frequency f in each time epoch. The amplitude of the signal at frequency f is given by the square root of the squared sum of each real and imaginary Fourier coefficient. This corresponds to the magnitude of the oscillatory neural activities: The phase of the signal at a given time and frequency is defined as the angle of the vector defined by the real and imaginary Fourier coefficients. This corresponds to the position within the oscillation cycle: The phase information was used to compute a time-varying phase-locking value (PLV) as an index of neural synchrony, as reported in the previous study55. This method computes the phase difference within a time window and a frequency band between all electrode pairs and then assesses the statistical significance of such phase differences across all trials. We defined ϕj and ϕi to be unitary vectors representing the phase of signals for electrodes i and j during time window t and for frequency bin f. The phase difference between these electrodes is equal to ϕij = ϕi−ϕj, and the phase-locking value is . This equation was calculated across all epochs, with N representing the total number of epochs.
In our phase synchrony analyses, we tested the H0 hypothesis that the two series of phase values, namely ϕi and ϕj, were independent. For this purpose, we generated 200 new series of variables, which exhibited the same characteristics as the original signal from electrode j, except that we designed them to be independent of the signals from electrode i55. These series were created by shuffling the trials within the measures of electrode j to create a new series . For each surrogate series , we measured the maximum differences between ϕi and in time. These 200 values were used to estimate the significance of PLV between the original signals ϕi and ϕj. The proportion of surrogate values that exceeded the original PLV (between ϕi and ϕj) is known as the phase-locking statistics (PLS)55. These data measure the probability of seeing false positives for a given level of significance. In this study, we controlled the statistical artifact of multiple comparisons (type II error) by choosing a very conservative significance threshold (p < 0.000001). The source localized 6239 voxels were averaged and selected for producing 84 cortical regions of interest (ROIs). We assumed that each synchrony connection between the regions of interest could occur by chance, and accordingly 168 ROIs and 14028 possible combinations [(168X167)/2 = 14028] were used to derive the statistical criteria. Chi-square test was used to compare between pre- and post-training number of functional connections of intra- and inter-brain.
This work was supported by Japan Science and Technology Agency (CREST) and Tamagawa-Caltech gCOE programs.