## Introduction

A brain machine interface (BMI) is designed to translate raw neural signals into motor commands so that it reproduces body movements with a neuroprosthetic device1. For people with a severe neuromuscular disability, a BMI might replace or even restore lost natural outputs2. In particular, one of the major goals for a BMI is to decode a command signal from the motor cortex, which serves as a new functional output3. Then the question is what is the command signal that disabled people can manipulate. The answer would be the neural activity while imagining a movement. Neural activity occurs even while a person imagines moving body parts. We call it motor imagery (MI), which is a particular type of mental imagery defined as the mental simulation of a specific action without any corresponding motor output4. The neural activity by MI is known to originate from the motor and somatosensory areas5. Thus, most of the movement BMIs use sensorimotor activities during MI. However, the network underlying a MI, or some kind of imagery can be different depending on the imagery tactics (e.g., object-related imagery activates object areas, whereas the imagery of faces activates face-related areas in the fusiform gyrus)6.

In the EEG-based BMI, about 15–30% of users do not gain control of the BMI system. This phenomenon in these subjects is called "BMI illiteracy"7, which may result from the individual variability of the EEG signals. The EEG environment may not be identical for each recording. Contact impedance, placement of the electrodes, and environmental noise slightly vary from time to time. Additionally, the EEG activity may also change according to the alertness, attention, and inherent fluctuation of a subject. Furthermore, this variability could be larger in an imagery task rather than in an actual movement task because the individual strategy of imagination may differ among the subjects. In addition, BMI illiteracy may not only be due to the user but also partly to the training protocol and the feedback8. Mastering a MI resembles the acquisition of a new motor skill9. As appropriate practice improves a motor skill, imagery practice by appropriate feedback is required for robust BMI control8,10.

Recently, several approaches have been tried such as somatosensory11, tactile12, and visual feedbacks13 to give a more natural feedback to the subject. These feedback approaches relied on accurate classification. However, there is a lack of accurate classification for the type of MI that the subject used.

There are two main types of motor imageries: kinesthetic motor imagery (KMI)14 and visual motor imagery (VMI)14. KMI is described as the ability to imagine performing a movement by means of having an impression of the muscle contraction and sensation during an actual movement. On the other hand, VMI is the ability to imagining seeing myself performing the movement. Because most of the decoders for motor prediction are designed based on an actual movement, KMI could be more effective than VMI. Thus, the use of a KMI would be better to improve the BMI performance. However, without specific instructions, subjects may be confused in the imagery tactics. Furthermore, people with motor disabilities had more difficulties with kinesthetic imagination compared to healthy participants15. It would be helpful if we could classify the imagery types and give feedback to the subject.

Previous EEG studies that recognized different mental states such as motor execution (ME), KMI, VMI, and visual observation (VO) showed that mental states could be classified with accuracies in KMI (67%), VMI (56%), ME (80%) and VO (80%)14. The classification features were extracted based on the strength of the spatial cortical activity (e.g., frequency bands and electrode sites). However, several studies reported that the brain activation patterns of KMI and VMI showed similar activation patterns with only a temporal difference. KMI showed motor-related activation first, while VMI showed visual-occipital activations at the early stage16. In a practical condition, it is hard to detect the temporal changes. Furthermore, the local activity alone may have a high probability of a false positive because motor areas also show spontaneous activity even not in imagery or movement tasks (e.g., at rest)17. Therefore, a more robust approach rather than an activation-based method is needed. Motor-related brain regions apart from the motor cortex are involved in MI tasks18,19. The connectivity within motor-related brain regions would be an alternative feature to classify MI strategies. A previous study showed that connectivity in the dorsolateral prefrontal cortex increased during movement planning20. The connectivity refers to the statistical dependency of signals from spatially separated brain areas during a task performance21. The connectivity between brain areas can provide more direct evidence about information exchange even if the activity of a brain region does not increase compared to the baseline22. In addition, the connectivity could possibly reflect the underlying structural organization of anatomical connections23 by utilizing patterns of connectivity among brain regions that represent functional brain networks. Accordingly, many studies on memory22, resting-state17,24, and cognitive impairments25 used connectivity measures to analyze cognitive activities.

Despite the importance of MI in BMI, there is a lack of accurate methods for classifying MI types and providing feedback. In order to address this gap, the present study classified two types of MI with a connectivity measure. The study hypothesized that KMI would show more similar connectivity patterns with ME than that of VMI. Moreover, the global and local characteristics of these network patterns will be different between KMI and VMI.

## Methods

### Participants

Eleven healthy subjects (7 females; aged 25.5 years old; STD = 2.71 years) were recruited for this study. All subjects were novices in BMI. We used the Edinburgh Handedness Inventory to collect data on the handedness of the subjects. All the subjects were right-handed, and the score on the Edinburgh Handedness Inventory was above 80 for all subjects (87.73 ± 6.17)26. The intrinsic abilities of two motor imagery types in all subjects were tested using the Kinesthetic and Visual Imagery Questionnaire (KVIQ-10)27. No subjects were excluded from the analysis because the subjects' KVIQ-10 scores were higher than the exclusion criteria of 2 (KMI = 3.24 ± 0.60; VMI = 3.42 ± 0.70). Prior to the experiment, the experimenter explained KMI and VMI and instructed subjects to use a specific imagery strategy according to the instructions. All subjects provided written informed consent to participate in the experiment. This study was performed in accordance with the Declaration of Helsinki and was approved by the Institutional Review Board (IRB) of Seoul National University Hospital (1605-136-765).

### Electrophysiological recordings and preprocessing

We acquired neural activities using a 64-channel EEG cap (Quich-CapEEG, Compumedics, Abbotsford, Victoria, Australia) and an amplifier (Neuroscan synamps2, Compumedics, Abbotsford, Victoria, Australia) with an additional electrode for a reference. The reference electrode was attached to the mastoid opposite the dominant hand of the subject. Movement of the arm was tracked by a motion capture system with an active optical marker (3D investigator, Northern Digital Inc., Waterloo, Ontario, Canada) during the neural signal recording. The sampling frequency of the signals was 2000 Hz for the EEG signal and 1000 Hz for the motion tracker. Both were resampled to the same frequency of 500 Hz. In order to synchronize the EEG amplifier and motion tracker, we delivered a 5 V input simultaneously to all devices before starting the sessions (e.g., through a transistor-transistor logic (TTL) port and the external trigger port of the 3D investigator amplifier).

### Source analysis and regions of interest (ROIs)

We selected 14 areas related to cognitive mechanisms of arm movement processes, which were expected to be involved in the ME, KMI, VMI, and VO networks31,32,33,34 in the bilateral hemispheres as follows: the dorsolateral prefrontal cortex (DLPFC), premotor cortex (PM), supplementary motor area (SMA), primary motor cortex (M1), primary somatosensory cortex (S1), posterior parietal cortex (PPC), and primary visual cortex (V1). The locations are illustrated in Fig. 2c. We performed source analysis according to the determined region of interest (ROI) using the built-in function of the discrete model probing in BESA research 6.0 (GmbH, GrmbH, Gru, Bavaria, Germany). The EEG signals, which are illustrated in Fig. 2b, were band-pass filtered at a range of 0.1–100 Hz with a 60 Hz notch filter (Fig. 2d). Table 1 lists the coordinates used35. Each source was identically analyzed for all subjects by using equal masks.

Before the source analysis, the common average reference (CAR) was used to remove the global background activity36. We separated the ROI signal into theta (3–7 Hz), alpha (8–12 Hz), beta (13–30 Hz), and gamma (32–100 Hz) frequency bands.

### Power spectral density analysis

The source-localized and filtered data were epoched with a time window from 4 s before the onset to 13.2 s after the onset. To calculate the power for each condition, we used the Morlet wavelet transform with a frequency range of 0.1–100 Hz. The power spectra from the task time (0–8.2 s) were normalized by the power of each frequency from the baseline periods (− 1.5 to − 1 s). Then, the data were averaged for all ROIs making one power spectra for each trial. To compare the difference between conditions, we made one grand averaged power spectra plot for each condition by averaging the normalized power spectra of all the trials and subjects at each condition.

### Connectivity analysis

To compare the patterns of two MIs, we analyzed brain connectivity using a mutual information algorithm (Fig. 2e). Mutual information measures both linear and nonlinear interdependencies between EEG signals. We used cross mutual information, which can be applied to two different signals37. In this study, we utilized mutual information between the signals at the paired cortical ROIs. The connectivity values from the task time (0–8.2 s) were normalized by the connectivity values of each session from the baseline periods (− 1.5 to − 1.4 s). We had a total of 91 edges (14 ROIs), and each condition (i.e., ME, KMI, VMI, and VO) had 48 trials per individual. Connectivity patterns were grand averaged across subjects for the four conditions with the alpha and beta frequency bands separately.

### Network analysis with graph theory

Graph theory was used to reconstruct the brain network. We used undirected weighted values from the connectivity with mutual information as the network edges. We tried to identify the critical nodes that have strong relationships with other nodes in the network. We estimated the degree centrality (DC) in the weighted graph with the GRETNA toolbox38. The calculated DC was represented for each frequency band.

### Degree centrality

The DC of a node is can be easily computed by summing the weight of edges incident on the node. To analyze DC, we used gretna_node_degree_weight (GRETNA). If A is the N × N weighted matrix for a graph, the DC of the i-th node is quantitatively defined as follows:

$${\text{DC}}(i) = \mathop \sum \limits_{j = 1}^{i - 1} A[i,\;j] + \mathop \sum \limits_{j = i + 1}^{n} A[i,\;j]$$

where N is the number of nodes.

### Feature extraction and classification

Support vector machine (SVM) analysis was used to classify the KMI and VMI in a single trial. We used network measures and normalized power as features to classify the KMI and VMI to compare the classification accuracy of each measurement. The features were used by the edges, which consisted of the highest 20%39 (Fig. 2f), and normalized power, respectively. To classify with the SVM algorithm, we used the MATLAB function of fitcsvm (Fig. 2g). The classification performance was evaluated by a fivefold cross-validation with 100 iterations. The overall procedure is illustrated in Fig. 2a.

### Statistical analysis

The significant difference between the degree centralities at the nodes for KMI and VMI was calculated for each frequency band using a non-parametric permutation test. The significance was evaluated by the permutation distribution with 10,000 iterations40. Pearson's correlation coefficient (R) was calculated to measure the similarity between the connectivity matrixes of the different conditions. All analyses were conducted with MATLAB (R2018a, Math-Works, Natick, MA, USA) and the EEGLAB toolbox41.

## Results

### Connectivity analysis using mutual information

We analyzed the connectivity patterns for each condition in the four frequency bands, but the theta and gamma frequency band signals did not produce statistical differences between the ROI signals. Therefore, the results focused on the alpha and beta frequency signals, and the remaining two frequency results are not given. There was a total of 14 ROIs, and the time series of one trial was 8.2 s for each subject. The window size was 100 ms, which was the same as in our previous study42, with no overlap. Eighty-two windows were created per trial. Each ROI signal was analyzed with the mutual information algorithm, which yielded one 14-by-14 matrix for each time point of the windows. These 82 connectivity matrices generated at each time point were averaged to create one 14-by-14 matrix per trial. The connectivity matrix of all the trials was finally averaged for each condition, resulting in one 14-by-14 connectivity matrix.

### Connectivity classification using SVM

We used the SVM algorithm to classify the KMI and VMI based on the two different feature groups. One is the power spectrum, which is normalized by the baseline power, and the other is the connectivity. We compared the classification performances. Figure 3a is an example of the features for classifying the KMI and VMI. Each line represents the normalized power spectra grand averaged across all subjects for each condition. We used the normalized power of the alpha and beta frequency as the features for the SVM. The normalized power of the KMI and VMI conditions was similar between the alpha and beta frequency bands (R = 0.99), which could explain the low classification performance when using the normalized power shown in Fig. 4. Figure 3b depicts the highest 20% of the grand averaged connectivity at the alpha and beta frequency bands under each condition. The edge was calculated by mutual information algorithms, and all trials were averaged for all subjects. The size of a node represents the node degree centrality. The edges shown in Fig. 3b were selected for the features to classify the KMI and VMI conditions. Note that the networks of the KMI and ME had more connections to the ROI of the sensorimotor cortex and showed similar network patterns. On the other hand, the network patterns of the VMI and VO were similar, showing many networks at the DLPFC and PM (Table 3; Fig. 5). The classification was performed based on individual trials using the extracted features. The classification accuracies between the KMI and VMI are shown in Fig. 4a. In Fig. 4a, the accuracy was higher in the connectivity feature than in the power feature. In addition, the individual's accuracy was higher than the chance level (50%). The averaged accuracy was 98.5% in the alpha and 99.29% in the beta band with the connectivity features (Fig. 4b). On the other hand, the classification accuracies with the features of the power spectrum were significantly lower (p < 0.01, Wilcoxon paired rank test) than the accuracy with the connectivity features (alpha: 54.95%; beta: 55.1%).

To strengthen the results of the present study, we classified the ME and KMI and VO and VMI by using connectivity measures as a classifying feature (Table 2). The classification accuracy for each participant was slightly higher than the chance level (50%). The accuracy of mean and standard deviation of the ME and KMI was (62.9 ± 19.7) and (62 ± 15) for the alpha and beta frequency bands, respectively. The mean accuracy of the VO and VMI was (57 ± 14) and (54 ± 16.8) for the alpha and beta frequency bands, respectively. This is an important result of supporting our hypothesis because we hypothesized that motor imageries share a similar sensory modality to the equivalent actual movements.

### Correlation of connectivity patterns between KMI and VMI

The Pearson’s correlation of connectivity patterns between conditions was analyzed to identify global similarity. Table 3 shows the results of the correlation coefficient between the conditions. Correlation coefficients were calculated from the grand average of the connectivity matrix. The correlation coefficients between the KMI and ME and between the VMI and VO were significantly higher (p < 0.01) in the alpha and beta frequency bands, respectively. The similarity between the KMI and ME verifies our hypothesis that the brain network of the KMI is close to that of the ME. This is also the reason why the ME and KMI or VO and VMI cannot be separated by using connectivity measures as a classifying feature, respectively, as shown in Table 2. On the other hand, the other pairs were not significant. In particular, the insignificant correlation between the KMI and VMI was quite reasonable because the two connectivity patterns were able to be discriminated with high accuracy.

### Degree centrality of the KMI and VMI

We further measured the DC with the connectivity values in the brain network shown in Fig. 5. The top 20% of the connectivity values were selected for the network measure. In graph theory, the DC on a node is measured by adding the connectivity values from the node to the other nodes in the network. Here, we used DC to find out the local difference between the KMI and VMI in the alpha and beta frequency bands. Figure 6 depicts the overall DC in the alpha and beta frequency bands, which shows significant differences (p < 0.05) between the KMI and VMI. The result shows that the alpha frequency band showed significant differences (p < 0.05) in the left S1, and the beta frequency band showed significant differences (p < 0.05) in the right PM.

## Discussion

This study investigated the characteristics of two major MI types, KMI and VMI, by using connectivity patterns and a network measure. The results show that the KMI and VMI could be classified in the SVM method on single trials with high accuracy. To the best of our knowledge, the classification with the connectivity patterns is the first applied in discriminating the type of MI. The reason for the excellent classification performance is that the cognitive brain networks of the KMI and VMI would be different.

The results of this study showed similar network patterns between the ME and KMI and between the VMI and VO, respectively. These large network similarities show the underlying neural networks and neural mechanisms of different cognitive conditions because mental imagery uses almost identical neural substrates as perception in the same sensory modality43. In this case, we used an experimental design from the perspective that motor imageries maintain the same temporal characteristics as the equivalent actual movement44 and facilitate the receptive imagery task14,29. Therefore, these facilitations of the cognitive system at the cortical level would be the reason for the global network similarities between the KMI and ME and between the VMI and VO, respectively. In particular, the similarity between the KMI and ME implies the effectiveness of utilizing a decoder trained by an actual movement.

The local DC at a ROI level reflects the relationship of the ROI with other ROIs. Thus, key ROIs in each condition would have high values in the local DC. In the comparison of DC between the KMI and VMI, the left (contralateral side) S1 was significantly higher in the KMI than in VMI at the alpha band. The S1 is known as the primary region for somatosensory perception including proprioception. According to some reports, subjects improved their ability for MI by giving sensory stimulation during the task. This MI improvement suggests that through the processing of information by the neural circuitry associated with MI, sensory signals trigger a motoric memory of a given movement and support body limb representation45,46. Thus, S1 in KMI should be more activated and centralized for kinesthetic sensation than in VMI. In addition, in the beta frequency band, the DC of the right (ipsilateral) PM in VMI was significantly higher than in KMI. It is known that the premotor cortex could be activated in movement imagery, which involves planning and preparation for actual movements47. Another fMRI study showed that just observing a movement could facilitate motor learning by mirror neurons in the premotor cortex48. Thus, the PM might be associated with an action observation and would have an important impact on the VMI similar to the VO.

A previous study classified four MI conditions by the power amplitude14. That study found that event-related spectral perturbation (ERSP) patterns between the ME and KMI and between the VMI and VO were also similar. In particular, KMI is similar to ME because KMI increases the corticospinal excitability and is related to sensorimotor systems49. Additionally, the ERD pattern of the KMI was similar to that of actual executions50, while the KMI and VMI were mediated through separate neural systems16. However, a meta-analysis found no significant contrast from the triggered anatomical regions of the KMI and VMI in sports experts33. Moreover, some studies reported that the activity of VO was similar to ME because the activated patterns and sequences of the brain were similar to the real executions30,51. Due to this discrepancy, the classification of cognitive conditions by brain activation like ERSP has limitations. In our study, classification with band powers resulted in a worse performance than that with the connectivity features.

Instead of the activation feature, the neural pathway would be an alternative feature. Our previous studies demonstrated that the connectivity increased during movement execution independent of the strength of the brain activity20,42. The present results also show that connectivity measure is more powerful in classifying the type of MI than by the power measure. The complex cognitive process involves multiple brain regions. Thus, we used the brain source localization to extract representative source activity for the determined ROIs. Because neighbor channels in the EEG show a similar activity, features of the connectivity based on the activity of the channel level might not be effective. Relatively sparse source ROIs could result in a high classification accuracy. The neuronal activity does not occur alone. It has a relationship with other neurons. The connectivity evaluates the relation between the ROIs. This relationship is reflected in the connectivity-based feature. Therefore, these edges provide valuable features to classify KMI and VMI.

It is still difficult to train KMI and VMI performance. In particular, the people who cannot move their hands because of paralysis would have difficulty to incorporate an accurate MI. However, the training of a MI similar to an actual execution is essential. A MI is a covert action, which is in the category of exercise, and giving feedback is essential to improving the imagery ability. Our study shows that the strategy of MI, such as KMI and VMI, can be classified with very high performance. The network pattern of KMI is similar to that of real execution. The BMI should give sensory feedback to improve the performance of future BMIs. Combining kinesthetic motor imagery with somatosensory feedback would make a more vivid motor imagery, which is similar to the networks of movement execution.

Although MI training is important, there is no promising training protocol yet. Considering our results, it is possible to provide KMI feedback similar to actual movement through brain connectivity analysis. In addition, each individual can be given feedback to see if the imagination is similar to the natural signal of the actual movement. Future research will have to be done on how the network patterns of ME and KMI are so similar. In addition, future motor imagery training should not only provide movement information but also provide sensory feedback and proprioception information when moving so that it feels like a real move.

There are some limitations to the study. One of the current limitations is the small number of subjects in our study. However, the accuracy of classifying the KMI and VMI in a single trial showed that the possibility of giving accurate feedback is high. And the trial number of each individual was enough to permit statistical analysis showing differences between the KMI and VMI. Another limitation is that it is difficult to verify whether the subjects were performing KMI and VMI in the correct manner. In order to address this problem, KVIQ-10 was used to access the individual imagery ability to perform MI prior to the experiment, and it was confirmed that no subjects excluded from the criteria. Additionally, we used a block design to facilitate each imagery. This facilitation involves reproducing previously experienced movements from memory52, which helps guide subjects to use each imagery correctly.

## Conclusion

Classification of two motor imageries was significantly better with the features of connectivity than with normalized powers. Consequently, the high classification could be used as feedback for MI training. Because the KMI network characteristics were similar to ME, KMI would be more suitable for BMI. S1 in the alpha frequency band and right PM in the beta frequency band are the key nodes in KMI and VMI, respectively. These results open up new perspectives for the underlying neural networks of the cognitive conditions and toward designing MI training methods.