Recent advances in non-invasive brain-computer interface (BCI) technologies have shown the feasibility of neural decoding for both users’ gait intent and continuous kinematics. However, the dynamics of cortical involvement in human upright walking with a closed-loop BCI has not been investigated. This study aims to investigate the changes of cortical involvement in human treadmill walking with and without BCI control of a walking avatar. Source localization revealed significant differences in cortical network activity between walking with and without closed-loop BCI control. Our results showed sustained α/µ suppression in the Posterior Parietal Cortex and Inferior Parietal Lobe, indicating increases of cortical involvement during walking with BCI control. We also observed significant increased activity of the Anterior Cingulate Cortex (ACC) in the low frequency band suggesting the presence of a cortical network involved in error monitoring and motor learning. Additionally, the presence of low γ modulations in the ACC and Superior Temporal Gyrus may associate with increases of voluntary control of human gait. This work is a further step toward the development of a novel training paradigm for improving the efficacy of rehabilitation in a top-down approach.
Non-invasive scalp electroencephalogram (EEG) has recently been used to monitor cortical activities during human walking because of its portability and high time resolution1, 2. Prior studies suggested that EEG recording during walking display features that differ from those reported during standing, and more importantly, is coupled with the gait cycle1, 3,4,5. Electrocortical activities have also been used to distinguish between uphill versus level walking in humans6 and active versus passive walking in a robotic device7. A study with an active treadmill demonstrated that user-driven control increases cortical activity8.
These findings provide evidence that EEG has the capacity to monitor cortical activity in treadmill walking, enabling EEG-based Brain-Computer Interface (BCI) systems for walking. BCIs are systems that allow brain activity to directly control physical or virtual robots and machines. Non-invasive EEG-based BCIs have been proposed to control hand9, 10 and arm11 robotic devices. BCIs for the lower-limbs were proposed in offline analysis12,13,14,15, and validated in a discrete online control of an exoskeleton16 and a walking avatar in virtual reality (VR)17. With recent advances of VR devices, VR has been proposed as an alternative tool in stroke rehabilitation18. VR has been explored in a few walking-related BCI systems19,20,21. It provides real-time feedback to BCI users, while avoiding many technical challenges that lower-limb robotics face, such as high cost, risk of falls, and joint misalignment.
Recently, our group has successfully reconstructed human gait kinematics in real-time from EEG signals21, 22. In our previous study, we designed a real-time neural decoder to predict lower limb kinematics of human treadmill walking from EEG signals. The predicted joint angles (hip, knee, and ankle) were used to control a walking avatar in a virtual environment21. We demonstrated that BCI performance increased across sessions in multiple days21. However, to our best knowledge, the dynamics of cortical involvement in human upright walking with a closed-loop BCI has not been investigated. This study aims to investigate the changes of cortical involvement in human treadmill walking with and without BCI control of a walking avatar. Source localization using independent component analysis revealed significant differences in cortical network activity between walking with and without closed-loop BCI control. We hypothesized that the closed-loop EEG-based BCI-VR system enhances cortical involvement in human treadmill walking, triggers cortical networks involved in error monitoring and motor learning, and increases voluntary control of human gait. Once confirmed, it will have important implications in transferring BCI-based protocols to gait rehabilitation.
Materials and Methods
Experimental setup and procedure
Eight healthy individuals (three males, five females; aged from 19 to 29) with no history of neurological disease or lower limb pathology participated in this study. All experimental protocols and informed consent (signed by all participants) were approved by the Institutional Review Board (IRB) at the University of Houston. All experiments were performed in accordance with the 45 Code of Federal Regulations (CFR) part 46 (“The Common Rule”), specifically addressing the protection of human study subjects as promulgated by the U.S. Department of Health and Human Services (DHHS). Written informed consent to publish identifying images was also obtained. Each subject participated in two sessions with a total of three trials (two consecutive days, two trials in the first day). The subjects were instructed to have two minutes (mins) of standing still on a treadmill in the beginning and end of each trial. In the remaining period, the subjects walked on a treadmill at one mile per hour (mph) while looking at an avatar displayed eye-level on a 52-inch TV monitor placed in front of the treadmill. The movements of the avatar’s left leg in the sagittal plane followed the subjects’ movements precisely all the time by using the data from goniometer sensor placed at hip, knee, and ankle joint angle. However, the avatar’s right leg was driven either by goniometer sensors or predicted joint angles from BCI system depending on the protocol (Fig. 1). For safety purposes, all subjects were instructed to hold onto a front handle bar while walking on the treadmill.
Figure 1 shows the experimental protocol, EEG signals, and the subject’s and avatar’s joint angles. Each trial began and terminated with the subject standing still on the treadmill for two mins while looking at the avatar displayed on a monitor. In the rest phase, the avatar was standing and it was controlled by goniometer sensors to follow the subjects’ movements precisely. The rest phase was included because data in this phase were used as baseline for artifact subspace reconstruction (ASR) in EEG offline processing. Moreover, it provided continuous movements of the avatar from standing still, increasing the speed, and walking on a treadmill. Subjects were asked to minimize his/her eye blinks in the rest phase. After the first two mins, the treadmill speed was then increased to one mph. In the Gonio-ctrl phase (15 mins), the walking avatar was driven by signals from goniometer sensors and its movement followed the subject’s movement correctly. During this phase, the neural decoder (unscented Kalman filter, UKF) and its parameters were trained and updated in the background every minute by using a closed-loop decoder adaptation algorithm (CLDA). Details of CLDA-UKF decoder can be found in refs 17, 21. We found that the UKF’s parameters had to be updated about 15 times (15-min period) to yield the best decoding accuracy of lower limb joint angles. At the end of the Gonio-ctrl phase, the UKF’s parameters were fixed and the protocol switched to BCI-ctrl phase. In the BCI-ctrl phase (5 mins, operated at 10 ms cycle time), the avatar’s right leg was driven by the predicted joint angles from the UKF neural decoder. We developed the unilateral BCI-VR system to control only the right leg of the avatar for the following reasons: 1) The system will be used as a platform to examine the changes of neural representations for action and perception of bipedal locomotion during adaptation to virtual cortical lesions in one hemisphere, 2) Controlling both legs of the walking avatar from closed-loop BCI could be challenging to the subjects and they might also loose the interlimb phasing if the neural decoder was not perfect, and 3) A unilateral closed-loop BCI system could enhance cortical differences across conditions.
Data collection and signal processing for real-time BCI operations
The processing for both EEG and kinematic data for real-time closed-loop BCI was described in detail in our previous study21 and summarized here for completeness. All the real-time processing was run in our custom C++ software. First, we used data from EOG channels and run a robust adaptive filter to remove eye related artifacts (eye blinks and eye movements)23. Peripheral EEG channels were not used in the real-time decoding because they are most susceptible to artifacts from head movements and facial/cranical muscle activity. We used the fluctuations in the amplitude of slow cortical potential in the delta band (0.1–3 Hz) to decode lower limb joint angles. Lower limb joint angles were also low-passed filtered at 3 Hz using a Butterworth filter. This band was known to cover most power in the lower limb joint angle signals24. We used the fluctuations in the amplitude of slow cortical potential in the delta band (0.1–3 Hz) to decode lower limb joint angles. An Unscented Kalman filter (UKF) was implemented as neural decoder and its parameters were updated during BCI operations by using a closed-loop decoder adaptation (CLDA) to improve the performance17, 21.
Whole scalp 64-channel active EEG data were collected (ActiCap system, Brain Products GmbH, Germany) and labeled in accordance with the extended 10–20 international system. A wireless interface (MOVE system, Brain Products GmbH, Germany) was used to transmit data (sampled at 100 Hz) to the host PC. The channel layout was modified from the standard EEG cap setup. Ground (GND) and reference (REF) channels were placed on the left and right earlobe (A1 and A2), respectively. T7 and T8 channels were moved to FCz and AFz, respectively. FT9, FT10, TP9 and TP10 were used for electrooculography (EOG) to capture eye blinks and eye movements. The purpose of channel layout modification is to improve the real-time decoding accuracies based on our previous results. The two main reasons for the modification are: 1) GND and REF channels in the standard EEG cap layout are very close to the motor cortex which has been known to be important for neural decoding of human walking, and 2) EOG sensors were required in our real-time artifact removal algorithm21. Lower limb joint angles (hip, knee, and ankle) in the sagittal plane were recorded by goniometers (SG150 & SG110/A Gonio electrodes, Biometrics Ltd, UK) at 100 Hz, and in sync with EEG data using our customized C++ program. Details for the experimental setup can be found in the supplementary material and in our previous study21. Additionally, a 3D electrode localization system (BrainVision Captrak, Brain Products GmbH, Germany) was used to record electrode positions and consistently fit the cap to the subjects head across multiple sessions.
Offline EEG Signal processing
Data analysis and statistical analysis were performed using custom software written in Matlab R2016a (The MathWorks, Natick, MA.) and functions from EEGLAB v.1325. 64-ch EEG was recorded at 100 Hz and the EEG signal processing pipeline is illustrated in Fig. 2. Electrooculography (EOG) channels were first removed and the remaining EEG signals (60 channels) were high pass filtered at 0.1 Hz using a 4th order Butterworth filter. Bad EEG channels, indicated as standard deviation greater than 1000 µV or kurtosis of more than five standard deviations from the mean, were rejected3. The remaining EEG channels were then re-referenced to their common average. Next, artifact subspace reconstruction (ASR) was applied to remove high amplitude artifacts (e.g., eye blinks, muscle burst)26, 27. ASR applies principal component analysis to the EEG data in sliding windows and identifies channels that significantly deviate from the baseline data containing minimal movement artifact. Channels with variance above a predefined threshold compared to the baseline data are identified as corrupted and reconstructed using a mixing matrix that is computed from the baseline data. Here, we implemented the ASR algorithm, which is available as a plugin for EEGLAB toolbox25, 26. In this study, one min of EEG recorded during quite standing at the beginning of each trial was used as baseline (calibration) data for the ASR. A sliding window (length of 500 ms) and a variance threshold of three standard deviations were used to identify corrupted subspaces.
An equivalent current dipole matched to the scalp projection of each independent component (IC) source was computed by using a standard three-shell boundary element head model included in the DIPFIT toolbox in EEGLAB25. We used a Captrak system (Brain Products GmbH, Germany) for scanning EEG electrode positions and aligned them to a standard MNI brain model (Montreal Neurological Institute, Quebec, Canada). Only ICs in which the equivalent dipoles were located within the brain volume and explained >90% of the residual variance of the IC scalp projections were retained for further analysis. Next, we visually inspected each IC scalp projection, and its power spectra and removed ICs corresponding to eye blink/movement and neck muscle artifacts. The remaining ICs were then used to generate feature vectors, which include the information of power spectral density (<50 Hz), topographical scalp projections, and dipole locations, and clustered across subjects using k-means algorithm (k = 7). ICs that were further than three standard deviations from a cluster centroid were categorized into an outlier cluster and omitted from further analysis.
Power spectral density (PSD) for each IC was computed using multitaper (seven Selpian tapers) fast Fourier transform. Cluster grand mean PSDs were then computed by averaging across the ICs within each cluster. A within subject, repeated-measures one-way ANOVA with Tukey-Kramer correction for post-hoc multiple comparison was used to assess the effects of different walking conditions (Gonio-ctrl and BCI-ctrl) on the area under PSD curve across four frequency band of interest: Δ (0.1–3 Hz), θ: 4–7 Hz, α/µ (8–13 Hz), β (14–30 Hz), and low γ (30–49 Hz). Note that only the last 5 minute of data from the Gonio-ctrl phase were used for further comparison to the BCI-ctrl phase. We also analyze the differences in time-frequency spectrograms between walking with and without BCI control. First, the time-frequency decomposition (spectrogram) was performed using a short-time Fourier transform for each IC in the clusters (Hamming window length of 500 ms, maximum overlap) across the whole trial. The full-length spectrogram was then segmented into gait cycles, which were detected from kinematic data, and linearly time-warped to the average gait cycle length so that right heel-strikes occurred at the same median latency3. Individual epoch spectrograms were concatenated for each cluster in each walking conditions. Finally, we computed the difference spectrograms by subtracting the two walking conditions. The difference spectrograms were masked for significance (α = 0.05) using a non-parametric bootstrapping technique with random shuffling of 200 surrogate data25.
To illustrate intra-stride modulations of power for each IC, gait cycle mean log spectrum in each epoch spectrogram was computed and subtracted from log spectrogram at each time point. The mean subtracted log spectrograms were then averaged across all gait cycles, resulting in spectral power changes relative to the average power over the gait cycle3, 28. These event-related spectral perturbations (ERSPs) were averaged across all ICs in a cluster to create grand mean ERSPs for each walking condition. Significant ERSP values were identified using a bootstrapping technique within EEGLAB (α = 0.05)25.
Decoding performance of the closed-loop EEG-based brain computer interface (BCI)
The performance of the real-time closed-loop neural decoder was evaluated by Pearson’s r-value between the measured and the predicted lower limb joint angles (Fig. 3). Overall, the median and interquartile range (IQR) of the r-values for all the subjects across all trials for the hip are (median: 0.42; IQR: 0.53), knee (median: 0.45; IQR: 0.49), and ankle (median: 0.28; IQR: 0.42). We found the best decoding accuracy with the median r-value for hip, knee, and ankle being 0.85, 0.79, and 0.63, respectively. Figure 3C–E illustrates the mismatch between the subject’s and the avatar’s right legs in one gait cycle. The errors were significantly higher in the swing phase as compared to the stance phase.
Cortical IC source clusters and the features of their activations
Source localization using independent component analysis and k-mean clustering method revealed seven distinct cortical IC source clusters and one additional outlier cluster. Figure 4 shows the number of data sets and the number of IC sources contained in each cluster. Information of the cluster centroid such as the Talairach coordinates and Brodmann areas, which were identified from Talairach atlas29, is also illustrated. The Brodmann areas (BA) were searched within ±5 mm cube ranges around each cluster centroid. The searching resulted in 7 cortical areas such as: Anterior Cingulate Cortex (ACC; BA: 24, 32), Superior Temporal Gyrus (STG; BA: 13, 21, 22), Posterior Parietal Cortex (PPC; BA: 7, 31), Inferior Parietal Lobe (IPL; BA: 3, 40), Occipital Lobe (OL; BA: 17, 18), Inferior Frontal Gyrus (IFG, BA: 13, 22, 44), and Right Occipital Lobe (BA: -).
Figure 5 shows the average power spectra density (PSD) across ICs for each cluster in different experimental conditions (Gonio-ctrl and BCI-ctrl). The areas under the PSD curves in different frequency bands (Δ: 0.1–3 Hz; θ: 4–7 Hz; α/µ: 8–13 Hz; β: 14–30 Hz; and low γ: 30–49 Hz) were also computed. The results were averaged across clusters and statistical comparison across conditions was also performed and showed in Fig. 5. Post-hoc comparison reveals significant decreases of spectral power in the α/µ, β, and low γ bands in the BCI-ctrl as compared to the Gonio-ctrl condition. The differences occur in all clusters except for the Occipital Lobe.
We computed difference spectrograms between walking with and without BCI control of the walking avatar, BCI-ctrl and Gonio-ctrl, respectively. The significant differences in the spectrograms were obtained by using a non-parametric bootstrapping technique with random shuffling. Non-significant differences (α > 0.05) were set to 0 dB in Fig. 6. The spectral power in the α/µ band (8–13 Hz) substantially decreased (ERD) in the PPC and IPL regions during walking in the BCI-ctrl phase as compared to the Gonio-ctrl phase. The decreased power in the α/µ band was also sustained across the whole gait cycle. Significant decreases (ERD) were observed in the β band (14–30 Hz) band across all the clusters. However, the ERD in the β band was only observed intermittently throughout the gait cycle. The results from Fig. 6 also show significant increases (ERS) of spectral power in the low frequency bands (Δ: 0.1–3 Hz) in the ACC. Interestingly, this increased spectral power was more pronounced in the double support phase and early swing phase where the mismatch between the subjects’ gait patterns and the walking avatar was larger than in the stance phase (Fig. 6).
We observed amplitude modulations relative to the mean gait cycle in the α/µ bands in the ACC, STG, and IPL during the Gonio-ctrl phase (Fig. 7). The β modulations in the STG region were enhanced when the participants walking with BCI control of the avatar (BCI-ctrl phase). Additionally, gait phase modulation in the β and low γ bands of ACC, and in the low γ band of STG were present during the BCI-ctrl phase. Figure 7 also shows greater ERSP values in BCI-ctrl phase in the ACC and STG during double leg support period right before the swing period of the right leg.
To our best knowledge, this study is the first to examine cortical involvement during treadmill walking while controlling a virtual avatar using a closed-loop EEG-based BCI-VR. Source localization using independent component analysis and dipole fitting revealed significant differences in cortical activity between walking with and without closed-loop BCI control. Specifically, we found sustained α/µ (8–13 Hz) suppression in the PPC and IPL regions, while less sustained but still significant decreases (ERD) were observed in the β band (14–30 Hz) band across all the clusters, indicating increased cortical involvement in the walking task with real-time brain control of a virtual avatar. Our results also showed significant increases in cortical activity in the low frequency (Δ bands) in the ACC area suggesting the possible benefit of using a closed-loop BCI-VR that recruits cortical network involved in error monitoring and motor learning. Additionally, we observed β suppression in the ACC, PPC, and IPL and the presence of low γ modulation in the ACC and STG. Such results indicate that our closed-loop BCI-VR system promotes voluntary control of human gait.
Real-time closed-loop BCI-VR system to a walking avatar enhances cortical involvement in the gait task
Our results showed significant decreases in the α/µ and β bands of the ACC, STG, PPC, and IPL regions - as indicated by the area under the PSD curve (Fig. 5) and sustained desynchronization (ERD) in Fig. 6 - during the BCI-ctrl phase compared to the Gonio-ctrl phase. However, increased power in the low frequency (Δ bands) in the ACC region was observed in Fig. 6. Significant decreased power in the α/µ and β bands have been showed in recent studies when participants were actively involved in the walking tasks7, 27. Bulea et al. suggested that the decreases in power in the α/µ and β bands indicate increased cortical involvement when participants walked on an active treadmill or with increased gait speed. In the latter study, the decreased power in these bands were found in the sensorimotor areas when participants attempted to actively walk with a robotic assisted gait trainer as opposed to passively allowing the robotic system to move their legs. Furthermore, numerous studies have shown that decreased spectral power in these bands is correlated with higher neuronal activation due to increased task demand30, i.e. visual spatial attention and sensorimotor integration31, 32. Therefore, our results indicate that walking while controlling a virtual avatar using a real-time closed-loop BCI-VR system enhances cortical involvement in the gait task.
Closed-loop BCI control of a walking avatar promotes cortical network activity involved in error monitoring and possibly adaptation learning
While spectral power in the α/µ and β bands decreases (desynchronize) with increasing task demands, the low frequency band power is known to increase (synchronize) in response to increasing task demands33. Previous studies also showed θ oscillations of EEG signals correlated with task difficulty34, error monitoring and learning processes33, 35, and memory and decision making36. Interestingly, a study from Womelsdorf et al. reported that local field potential (LFP) theta-activity in the Anterior Cingulate Cortex (ACC) of macaque monkeys holds the core functions in cognitive control processes and in error monitoring37. fMRI studies with human individuals also showed evidence for a direct relation between ACC activity and error detection and behavioral adjustment38, and the online monitoring of performance39. Interestingly in our study we found significant increased power in the low frequency (Δ bands) in the ACC region during walking with closed-loop BCI control (Fig. 6). Moreover, the difference is more pronounced in the double leg support and early swing phases. Results from ERSP analysis in Fig. 7 also show greater ERSP values in BCI-ctrl phase during the double leg support period just before the swing period of the right leg. The increases of ERSP values were found in the ACC and STG. Interestingly, the mismatch between the subjects’ gait pattern and the walking avatar is more obvious during the swing phase (Fig. 3). The changes of neural activities right before the swing period of the right leg observed in Figs 6 and 7 could be error-related responses from cortical networks to visual-motor feedback errors40. This suggests that using a closed-loop BCI-VR promotes cortical network activity involved in error monitoring and possibly adaptation learning.
Action observation may differentially recruit the mirror neuron system (MNS) during BCI control of the walking avatar
We noted that during the closed-loop BCI control of the avatar, changes in cortical activity could reflect the conflict-solving between contrasting proprioceptive afference and visual afference during closed-loop BCI control of the avatar (i.e., a mismatch). Nevertheless, the ‘action observation network’ might be recruited during both Gonio- and BCI-ctrl conditions as avatar motion observation is critical for movement planning and adaptation. It has been postulated that the mirror neuron system (MNS) has evolved to understand other people’s actions, and during planning and performing self-made actions as in the present experiment41, where the MNS may also be recruited during closed-loop BCI control. Note that under the BCI-ctrl phase, the avatar’s motion represented the motor intentions inferred by the closed-loop BCI. Thus, the avatar performance, via the visual feedback, can then be compared with the motor intentions, which may lead to an error (error monitoring). This error can be used by the brain to improve BCI control. Indeed, our previous study showed that the subjects significantly improve their performance in closed-loop BCI control of the walking avatar using their brain signals across eight days of practices21.
The changes in spectral content of cortical networks in the BCI control task may correlate with voluntary control of human gait
Voluntary control of movements is crucial for motor learning and physical rehabilitation42, 43. Our results suggest the possible benefits of using a closed-loop EEG-based BCI-VR system in inducing voluntary control of human gait. First, we observed significant decreases in the β spectral power and significant decreased power in the β band of ACC, PPC, and IPL (Figs 5 and 6). It is commonly accepted that the oscillations in the β band promote the current sensorimotor state or motor set44 while compromising processing related to new or voluntary movements45,46,47. A recent study from Seeber et al. also showed EEG β-suppression across the whole gait cycle during walking compared to standing4. Thus, the presence of β-ERD may suppress an inhibitory network and enable voluntary control of movements. Second, our results demonstrate that gait phase modulation in the low γ band was present in the ACC and STG regions while participants controlled the walking avatar using the closed-loop BCI-VR system (Fig. 7). Recent studies have shown that low γ amplitudes are modulated related to the gait phase3, 4, 7. Furthermore, the modulation in the low γ oscillations has been related to voluntary walking when subjects attempted to actively walk with a robotic trainer compared to passively allowing it move their legs7, or when they performed the task of active gait adjustment in a virtual environment48. A recent study from Bulea et al. also demonstrated that low γ band synchronization is enhanced in sensorimotor area and PPC while participants voluntarily tracked a target speed on an active treadmill27. These results are in line with our findings in this study that provides additional evidence that associates cortical activity with voluntary control of human gait. Our results also support the hypothesis that the sensorimotor system shifts its oscillations to the low γ band to plan and execute more complex movements27 in response to increasing motor task demands and attentions49, 50. We therefore conclude that our closed-loop EEG-based BCI-VR system promotes voluntary control of human gait. We also hypothesize that such a system is more immersive compared to walking with visual feedback using a VR system alone.
Limitations and future work
Our current study demonstrated substantial changes in cortical activity when the subjects switched from open-loop (Gonio-ctrl phase) to closed-loop BCI-control of a walking avatar. The results further support previous studies suggesting that neural signals are highly dynamic51, 52 and neuroplasticity holds a critical role in developing robust neural decoder53. Neural decoders are typically trained offline by fitting neural signals against actual movements. This approach, however, ignores the neural dynamics from subjects when switching from open-loop to closed-loop BCI and typically results in decreased online performance. Based on previous studies54, 55 showing that closed-loop decoder adaptation (CLDA) yields performance improvement, we trained and updated the neural decoder at a fixed interval (one min). Although the decoding accuracy is promising, this study did not consider utilizing the information from cortical activity to optimize the neural decoder, i.e., optimal EEG channels for neural decoding of human gait have not been explored.
Future work has to examine if our closed-loop EEG-based BCI-VR could be deployed during observational therapy in post-stroke rehabilitation, which has been shown to be beneficial to the rehabilitation process56,57,58. In typical observational therapy, persons with stroke observe a video clip or other people demonstrating a movement before they try it themselves. Reports suggest that patients undergoing observational therapy show significant improvement in functional assessment before and after the treatment57. Based on the results from this study and our previous findings21 suggesting the feasibility of using a closed-loop BCI-VR with visuomotor perturbations to induce cortical adaptations, we hypothesize that our system could be clinically meaningful in a neurorehabilitation context.
Whereas previous studies have analyzed cortical activity during typical treadmill walking, walking with a robotic assisted gait trainer, or with varying gait speeds using an active treadmill, this study is the first to examine cortical involvement during treadmill walking while controlling a virtual avatar using a closed-loop EEG-based BCI-VR. Interestingly, we observed significant differences in cortical activity between walking with and without closed-loop BCI control. In particular, we found sustained α/µ suppression in the PPC and IPL regions and significant decreases (ERD) in the β band, indicating increased cortical involvement in the walking task with real-time control of a virtual avatar using a closed-loop BCI-VR system. Our results also showed significant increased cortical activity in the low frequency (Δ bands) in the ACC area suggesting the possible benefit of using a closed-loop BCI-VR that recruits cortical network involved in error monitoring and motor learning. Additionally, we observed β suppression in the ACC, PPC, and IPL and the presence of low γ modulation in the ACC and STG. Such results indicate that our closed-loop BCI-VR system promotes voluntary control of human gait. Thus, the current study demonstrated the feasibility of using a closed-loop EEG-based BCI for controlling a walking avatar, promoting cortical engagement, and monitoring cortical activity. Our closed-loop BCI-VR system may be relevant for neurological gait rehabilitation as a clinical tool for post-stroke physical training and clinical assessment. Finally, our system may also help to better understand cortical dynamics during walking with a closed-loop BCI system.
Wagner, J., Makeig, S., Gola, M., Neuper, C. & Müller-Putz, G. Distinct beta band oscillatory networks subserving motor and cognitive control during gait adaptation. Journal of Neuroscience 36, 2212–2226, doi:10.1523/JNEUROSCI.3543-15.2016 (2016).
Debener, S., Minow, F., Emkes, R., Gandras, K. & de Vos, M. How about taking a low-cost, small, and wireless EEG for a walk? Psychophysiology 49, 1617–1621, doi:10.1111/j.1469-8986.2012.01471.x (2012).
Gwin, J. T., Gramann, K., Makeig, S. & Ferris, D. P. Electrocortical activity is coupled to gait cycle phase during treadmill walking. NeuroImage 54, 1289–1296, doi:10.1016/j.neuroimage.2010.08.066 (2011).
Seeber, M. et al. EEG beta suppression and low gamma modulation are different elements of human upright walking. Frontiers in human neuroscience 9, 1–9, doi:10.3389/fnhum.2014.00485 (2015).
Cevallos, C. et al. Oscillations in the human brain during walking execution, imagination and observation. Neuropsychologia 79, 223–232 (2015).
Bradford, J. C., Lukos, J. R. & Ferris, D. P. Electrocortical activity distinguishes between uphill and level walking in humans. Journal of neurophysiology 115, 958–966, doi:10.1152/jn.00089.2015 (2015).
Wagner, J. et al. Level of participation in robotic-assisted treadmill walking modulates midline sensorimotor EEG rhythms in able-bodied subjects. NeuroImage 63, 1203–1211, doi:10.1016/j.neuroimage.2012.08.019 (2012).
Bulea, T. C., Kim, J., Damiano, D. L., Stanley, C. J. & Park, H.-S. User-driven control increases cortical activity during treadmill walking: An EEG study in Engineering in Medicine and Biology Society (EMBC), 36th Annual International Conference of the IEEE. 2111–2114 (IEEE, 2014).
Ramos-Murguialday, A. et al. Proprioceptive Feedback and Brain Computer Interface (BCI) Based Neuroprostheses. PLoS ONE 7, e47048, doi:10.1371/journal.pone.0047048 (2012).
Agashe, H. A. & Contreras-Vidal, J. L. Observation-based training for neuroprosthetic control of grasping by amputees in 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 3989–3992 (IEEE, 2014).
Sakurada, T., Kawase, T., Takano, K., Komatsu, T. & Kansaku, K. A BMI-based occupational therapy assist suit: asynchronous control by SSVEP. Frontiers in neuroscience 7, 1–10, doi:10.3389/fnins.2013.00172 (2013).
García-Cossio, E. et al. Decoding Sensorimotor Rhythms during Robotic-Assisted Treadmill Walking for Brain Computer Interface (BCI) Applications. PloS one 10, e0137910, doi:10.1371/journal.pone.0137910 (2015).
He, Y. et al. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society 3985–3988 (IEEE, 2014).
Presacco, A., Forrester, L. W. & Contreras-Vidal, J. L. Decoding intra-limb and inter-limb kinematics during treadmill walking from scalp electroencephalographic (EEG) signals. IEEE transactions on neural systems and rehabilitation engineering: a publication of the IEEE Engineering in Medicine and Biology Society 20, 212–219, doi:10.1109/TNSRE.2012.2188304 (2012).
Brantley, J. A. et al. Noninvasive EEG correlates of overground and stair walking in Engineering in Medicine and Biology Society (EMBC), IEEE 38th Annual International Conference of the. 5729–5732 (IEEE, 2016).
Kilicarslan, A., Prasad, S., Grossman, R. G. & Contreras-Vidal, J. L. High accuracy decoding of user intentions using EEG to control a lower-body exoskeleton in Engineering in Medicine and Biology Society (EMBC), 35th Annual International Conference of the IEEE. 5606–5609 (IEEE, 2013).
Luu, T. P., He, Y., Brown, S., Nakagome, S. & Contreras-Vidal, J. L. A closed-loop brain computer interface to a virtual reality avatar: Gait adaptation to visual kinematic perturbations in Virtual Rehabilitation Proceedings (ICVR), International Conference on. 30–37 (2015).
Henderson, A., Korner-Bitensky, N. & Levin, M. Virtual reality in stroke rehabilitation: A systematic review of its effectiveness for upper limb motor recovery. Topics in Stroke Rehabilitation 14, 52–61, doi:10.1310/tsr1402-52 (2007).
Wang, P. T., King, C. E., Chui, L. A., Do, A. H. & Nenadic, Z. Self-paced brain–computer interface control of ambulation in a virtual reality environment. Journal of Neural Engineering 9, 056016, doi:10.1088/1741-2560/9/5/056016 (2012).
King, C. E., Wang, P. T., Chui, L. A., Do, A. H. & Nenadic, Z. Operation of a brain-computer interface walking simulator for individuals with spinal cord injury. J Neuroeng Rehabil 10, 77, doi:10.1186/1743-0003-10-77 (2013).
Luu, T. P., He, Y., Brown, S., Nakagame, S. & Contreras-Vidal, J. L. Gait adaptation to visual kinematic perturbations using a real-time closed-loop brain–computer interface to a virtual reality avatar. Journal of neural engineering 13, 036006, doi:10.1088/1741-2560/13/3/036006 (2016).
Luu, T. P. et al. Unscented Kalman filter for neural decoding of human treadmill walking from non-invasive electroencephalography. 1548–1551, doi:10.1109/embc.2016.7591006 (2016).
Kilicarslan, A., Grossman, R. G. & Contreras-Vidal, J. L. A robust adaptive denoising framework for real-time artifact removal in scalp EEG measurements. Journal of neural engineering 13, 026013, doi:10.1088/1741-2560/13/2/026013 (2016).
Luu, T. P., Low, K. H., Qu, X., Lim, H. B. & Hoon, K. H. An individual-specific gait pattern prediction model based on generalized regression neural networks. Gait & Posture 39, 443–448, doi:10.1016/j.gaitpost.2013.08.028 (2014).
Delorme, A. & Makeig, S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of neuroscience methods 134, 9–21, doi:10.1016/j.jneumeth.2003.10.009 (2004).
Mullen, T. et al. Real-time modeling and 3D visualization of source dynamics and connectivity using wearable EEG in Engineering in Medicine and Biology Society (EMBC), 35th Annual International Conference of the IEEE. 2184–2187 (IEEE, 2013).
Bulea, T. C., Kim, J., Damiano, D. L., Stanley, C. J. & Park, H.-S. Prefrontal, posterior parietal and sensorimotor network activity underlying speed control during walking. Frontiers in human neuroscience 9, 247, doi:10.3389/fnhum.2015.00247 (2015).
Nathan, K. & Contreras-Vidal, J. L. Negligible motion artifacts in scalp electroencephalography (EEG) during treadmill walking. Frontiers in human neuroscience 9, 708, doi:10.3389/fnhum.2015.00708 (2016).
Lancaster, J. L. et al. Automated Talairach atlas labels for functional brain mapping. Human brain mapping 10, 120–131 (2000).
Pfurtscheller, G. & Da Silva, F. L. Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical neurophysiology 110, 1842–1857 (1999).
Capotosto, P., Babiloni, C., Romani, G. L. & Corbetta, M. Frontoparietal Cortex Controls Spatial Attention through Modulation of Anticipatory Alpha Rhythms. The Journal of Neuroscience 29, 5863–5872, doi:10.1523/jneurosci.0539-09.2009 (2009).
Hipp, J. F., Engel, A. K. & Siegel, M. Oscillatory synchronization in large-scale cortical networks predicts perception. Neuron 69, 387–396 (2011).
Klimesch, W., Doppelmayr, M., Russegger, H. & Pachinger, T. Theta band power in the human scalp EEG and the encoding of new information. Neuroreport 7, 1235–1240 (1996).
Gevins, A., Smith, M. E., McEvoy, L. & Yu, D. High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice. Cerebral cortex 7, 374–385 (1997).
Luu, P., Tucker, D. M. & Makeig, S. Frontal midline theta and the error-related negativity: neurophysiological mechanisms of action regulation. Clinical Neurophysiology 115, 1821–1835 (2004).
Jacobs, J., Hwang, G., Curran, T. & Kahana, M. J. EEG oscillations and recognition memory: theta correlates of memory retrieval and decision making. NeuroImage 32, 978–987 (2006).
Womelsdorf, T., Johnston, K., Vinck, M. & Everling, S. Theta-activity in anterior cingulate cortex predicts task rules and their adjustments following errors. Proceedings of the National Academy of Sciences 107, 5248–5253 (2010).
Kerns, J. G. et al. Anterior cingulate conflict monitoring and adjustments in control. Science 303, 1023–1026 (2004).
Carter, C. S. et al. Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280, 747–749 (1998).
Contreras-Vidal, J. L. & Kerick, S. E. Independent component analysis of dynamic brain responses during visuomotor adaptation. NeuroImage 21, 936–945 (2004).
Campbell, M. E. & Cunnington, R. More than an imitation game: Top-down modulation of the human mirror system. Neuroscience & Biobehavioral Reviews (2017).
Lotze, M., Braun, C., Birbaumer, N., Anders, S. & Cohen, L. G. Motor learning elicited by voluntary drive. Brain 126, 866–872 (2003).
Kaelin-Lang, A., Sawaki, L. & Cohen, L. G. Role of voluntary drive in encoding an elementary motor memory. Journal of neurophysiology 93, 1099–1103 (2005).
Engel, A. K. & Fries, P. Beta-band oscillations—signalling the status quo? Current opinion in neurobiology 20, 156–165 (2010).
Gilbertson, T. et al. Existing motor state is favored at the expense of new movement during 13–35 Hz oscillatory synchrony in the human corticospinal system. The Journal of neuroscience 25, 7771–7779 (2005).
Pogosyan, A., Gaynor, L. D., Eusebio, A. & Brown, P. Boosting cortical activity at beta-band frequencies slows movement in humans. Current Biology 19, 1637–1641 (2009).
Jenkinson, N. & Brown, P. New insights into the relationship between dopamine, beta oscillations and motor function. Trends in neurosciences 34, 611–618 (2011).
Wagner, J., Solis-Escalante, T., Scherer, R., Neuper, C. & Muller-Putz, G. It’s how you get there: walking down a virtual alley activates premotor and parietal areas. Frontiers in human neuroscience 8, 93, doi:10.3389/fnhum.2014.00093 (2014).
Mehrkanoon, S., Breakspear, M. & Boonstra, T. W. The reorganization of corticomuscular coherence during a transition between sensorimotor states. NeuroImage 100, 692–702 (2014).
Rouhinen, S., Panula, J., Palva, J. M. & Palva, S. Load dependence of β and γ oscillations predicts individual capacity of visual attention. The Journal of Neuroscience 33, 19023–19033 (2013).
Paluš, M. Nonlinearity in normal human EEG: cycles, temporal asymmetry, nonstationarity and randomness, not chaos. Biological Cybernetics 75, 389–396 (1996).
Stam, C. J. Nonlinear dynamical analysis of EEG and MEG: review of an emerging field. Clinical Neurophysiology 116, 2266–2301 (2005).
Orsborn, A. L. et al. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82, 1380–1393, doi:10.1016/j.neuron.2014.04.048 (2014).
Orsborn, A. L., Dangi, S., Moorman, H. G. & Carmena, J. M. Closed-loop decoder adaptation on intermediate time-scales facilitates rapid BMI performance improvements independent of decoder initialization conditions. IEEE transactions on neural systems and rehabilitation engineering: a publication of the IEEE Engineering in Medicine and Biology Society 20, 468–477, doi:10.1109/TNSRE.2012.2185066 (2012).
Ganguly, K. & Carmena, J. M. Neural correlates of skill acquisition with a cortical brain–machine interface. Journal of Motor Behavior 42, 355–360 (2010).
Celnik, P., Webster, B., Glasser, D. M. & Cohen, L. G. Effects of action observation on physical training after stroke. Stroke 39, 1814–1820, doi:10.1161/STROKEAHA.107.508184 (2008).
Ertelt, D. et al. Action observation has a positive impact on rehabilitation of motor deficits after stroke. NeuroImage 36, T164–T173 (2007).
Franceschini, M. et al. Clinical relevance of action observation in upper-limb stroke rehabilitation a possible role in recovery of functional dexterity. A randomized clinical trial. Neurorehabilitation and neural repair 26, 456–462 (2012).
This research was supported by award R01NS075889 from the National Institute of Neurological Disorders And Stroke (NINDS). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NINDS or the National Institutes of Health (NIH).
The authors declare that they have no competing interests.
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
About this article
Cite this article
Luu, T.P., Nakagome, S., He, Y. et al. Real-time EEG-based brain-computer interface to a virtual avatar enhances cortical involvement in human treadmill walking. Sci Rep 7, 8895 (2017). https://doi.org/10.1038/s41598-017-09187-0
This article is cited by
Portable deep-learning decoder for motor imaginary EEG signals based on a novel compact convolutional neural network incorporating spatial-attention mechanism
Medical & Biological Engineering & Computing (2023)
Halftone spatial frequency domain imaging enables kilohertz high-speed label-free non-contact quantitative mapping of optical properties for strongly turbid media
Light: Science & Applications (2021)
Scientific Data (2021)
Mobile BCI dataset of scalp- and ear-EEGs with ERP and SSVEP paradigms while standing, walking, and running
Scientific Data (2021)
Scientific Reports (2020)