Using Brain-Computer Interface to Control a Virtual Drone Using Non-Invasive Motor Imagery and Machine Learning

Objective. We proposed a novel simultaneous hybrid brain–computer interface (BCI) by incorporating electrical stimulation into a motor imagery (MI) based BCI system. The goal of this study was to enhance the overall performance of an MI-based BCI. In addition, the brain oscillatory pattern in the hybrid task was also investigated. Approach. 64-channel electroencephalographic (EEG) data were recorded during MI, selective attention (SA) and hybrid tasks in fourteen healthy subjects. In the hybrid task, subjects performed MI with electrical stimulation which was applied to bilateral median nerve on wrists simultaneously. Main results. The hybrid task clearly presented additional steady-state somatosensory evoked potential (SSSEP) induced by electrical stimulation with MI-induced event-related desynchronization (ERD). By combining ERD and SSSEP features, the performance in the hybrid task was significantly better than in both MI and SA tasks, achieving a ~14% improvement in total relative to the MI task alone and reaching ~89% in mean classification accuracy. On the contrary, there was no significant enhancement obtained in performance while separate ERD feature was utilized in the hybrid task. In terms of the hybrid task, the performance using combined feature was significantly better than using separate ERD or SSSEP feature. Significance. The results in this work validate the feasibility of our proposed approach to form a novel MI-SSSEP hybrid BCI outperforming a conventional MI-based BCI through combing MI with electrical stimulation.

The aim of this paper is to classify both (left and right) hand motor imagery (MI) tasks using event-related (de) synchronization (ERD/ERS) patterns based on the inter trial variance (IV). In this work, publicly available online BCI-2003 competition data set was used which contains MI based Electroencephalography (EEG) data of a single subject. The raw MI based EEG signals were extracted from C3 and C4 channels and band pass filtered to mu (8–12 Hz) frequency band. The mean, inter trial variance and ERD/ERS patterns were calculated in both the channels for both MI tasks of the subject. The patterns were used as feature vector and input to the different machine learning classifiers. The logistic regression and Naive Bayesian classifiers were used for the classification of both MI tasks of the subject. For logistic regression classifier, the accuracy, precision and sensitivity were found to be 92.40%, 92.00% and 92.00%, respectively. Meanwhile, 92.19% accuracy, 92.00% precision and 92.00% sensitivity were obtained for the Naive Bayesian classifier. The logistic regression classifier provides maximum classification accuracy (92.40%) in comparison with the existing methods (87.90–90.20%). Result shows that, the proposed method can quantify ERD/ERS patterns effectively in both the channels and also classifies both MI tasks of the subject successfully.

Brain-computer interface (BCI) based on electroencephalogram (EEG) is a promising technology, allowing computers to estimate human intentions. Intention recognition tool such as motor imagery (MI) with high reliability is one of the major challenges in the BCI field. Recently, researchers have attempted to use transfer learning for various BCI datasets, but the studies showed low classification accuracy. This study aimed to increase the classification accuracy of the MI through sequential transfer learning for a single dataset. EEG-MI data with 9 subjects from the dataset 2a of BCI competition IV were used. EEGNet was used for MI classification. The pre-trained model was constructed by first learning whether the data were MI or not. The model was then sequentially fine-tuned through transfer learning for four MI tasks (i.e., left hand, right hand, both feet and tongue). The model was able to classify MI with 91.34% accuracy. In the meantime, the baseline model without transfer learning showed an accuracy of 61.62%, whereas the fine-tuned model presented an improved accuracy of 63.82%. Consequently, the sequential transfer learning was able to improve the performance of MI-BCI.

Brain computer interfaces (BCIs) based on motor imagery (MI) play an important role in helping to improve and restore the loss of physical function. However, traditional MI-BCIs are limited to the motion intention of gross limb, which places many restrictions on their applications. This study proposes a hybrid paradigm based on MI and the steady-state somatosensory evoked potential, with the aim of improving the spatial resolution of MI recognition. Twelve subjects participated in this study. They performed MI tasks under MI and hybrid conditions. In the MI condition, subjects only performed MI tasks, whereas, in the hybrid condition, they received an electrical stimulus while performing the same tasks. Under the hybrid condition, subjects were required not to pay extra attention to the electrical stimulation. The MI task included imagining clenching the right hand and lifting the right forearm. The classifier was built using the filter bank common spatial pattern algorithm and a support vector machine, and online experiments were used to verify the recognition of two MI tasks. During the online experiments, all subjects were able to output different commands at a recognition accuracy far higher than the random level. The average classification accuracy of the hybrid condition reached 76.39%, with a maximum value of 88.34%, which was about 11% higher than that of the MI condition. Moreover, based on offline data, the classification performance using the event-related desynchronization (ERD) feature under the hybrid condition did not differ significantly from that under the MI condition, indicating that the introduction of electrical stimulation did not interfere in the separability of ERD. The proposed paradigm improved the efficiency of decoding multiple MI locations within a single limb. Despite the introduction of external stimuli, users could still drive the new system in the same way as MI in traditional MI-BCI.

A novel Morse code-inspired method for multiclass motor imagery brain–computer interface (BCI) design

This study underscores the reactivity of sensorimotor rhythms (SMR) in EEG such as mu rhythm in connection with the imagination and voluntary movement of left and right knee extension with able-bodied subjects using the standard Graz brain-computer interface (BCI) paradigm. As the knee motor execution (ME) and motor imagery (MI) reflect the user's voluntary physical and imagination state of knee extension respectively, they have been analyzed in order to be used as control signals to restore motor function with neuro-plasticity for BCI applications. This reactivity has been evaluated using the quantification of event-related desynchronization (ERD) and event-related synchronization (ERS) patterns. Investigations on the cortical lateralization of ERD/ERS during the left and right knee MI and ME for the extension task was done. During left and right knee MI, the foot area mu rhythm was desynchronized in all subjects, whereas no enhancement of the hand area mu rhythm was observed during any task. The frequency of the most reactive components for mu rhythm was 8.8±0.5 Hz. Interestingly a contralateral dominance was visible at a central electrode position, during left knee extension MI task. This lead to the establishment of an understanding of the left-right knee MI and its associated cognitive behavior.

In the aging society, the number of people suffering from vascular disorders is rapidly increasing and has become a social problem. The death rate due to stroke, which is the second leading cause of global mortality, has increased by 40% in the last two decades. Stroke can also cause paralysis. Of late, brain‐computer interfaces (BCIs) have been garnering attention in the rehabilitation field as assistive technology. A BCI for the motor rehabilitation of patients with paralysis promotes neural plasticity, when subjects perform motor imagery (MI). Feedback, such as visual and proprioceptive, influences brain rhythm modulation to contribute to MI learning and motor function restoration. Also, virtual reality (VR) can provide powerful graphical options to enhance feedback visualization. This work aimed to improve immersive VR‐BCI based on hand MI, using visual‐electrotactile stimulation feedback instead of visual feedback. The MI tasks include grasping, flexion/extension, and their random combination. Moreover, the subjects answered a system perception questionnaire after the experiments. The proposed system was evaluated with twenty able‐bodied subjects. Visual‐electrotactile feedback improved the mean classification accuracy for the grasping (93.00% ± 3.50%) and flexion/extension (95.00% ± 5.27%) MI tasks. Additionally, the subjects achieved an acceptable mean classification accuracy (maximum of 86.5% ± 5.80%) for the random MI task, which required more concentration. The proprioceptive feedback maintained lower mean power spectral density in all channels and higher attention levels than those of visual feedback during the test trials for the grasping and flexion/extension MI tasks. Also, this feedback generated greater relative power in the μ‐band for the premotor cortex, which indicated better MI preparation. Thus, electrotactile stimulation along with visual feedback enhanced the immersive VR‐BCI classification accuracy by 5.5% and 4.5% for the grasping and flexion/extension MI tasks, respectively, retained the subject’s attention, and eased MI better than visual feedback alone.

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

Motor Imagery (MI)-based Brain Computer Interface (BCI) system is a potential technology for active neurorehabilitation of stroke patients by complementing the conventional passive rehabilitation methods. Research to date mainly focused on classifying left vs. right hand/foot MI of stroke patients. Though a very few studies have reported decoding imagined hand movement directions using electroencephalogram (EEG)-based BCI, the experiments were conducted on healthy subjects. Our work analyzes MI-based brain cortical activity from EEG signals and decodes the imagined hand movement directions in stroke patients. The decoded direction (left vs. right) of hand movement imagination is used to provide control commands to a motorized arm support on which patient's affected (paralyzed) arm is placed. This enables the patient to move his/her stroke-affected hand towards the intended (imagined) direction that aids neuroplasticity in the brain. The synchronization measure called Phase Locking Value (PLV), extracted from EEG, is the neuronal signature used to decode the directional movement of the MI task. Event-related desynchronization/synchronization (ERD/ERS) analysis on Mu and Beta frequency bands of EEG is done to select the time bin corresponding to the MI task. The dissimilarities between the two directions of MI tasks are identified by selecting the most significant channel pairs that provided maximum difference in PLV features. The training protocol has an initial calibration session followed by a feedback session with 50 trials of MI task in each session. The feedback session extracts PLV features corresponding to most significant channel pairs which are identified in the calibration session and is used to predict the direction of MI task in left/right direction. An average MI direction classification accuracy of 74.44% is obtained in performing the training protocol and 68.63% from the prediction protocol during feedback session on 16 stroke patients.

In the field of brain-computer interface (BCI) based on motor imagery (MI), multi-channel electroencephalography (EEG) data is commonly utilized for MI task recognition to achieve sensory compensation or precise human-computer interaction. However, individual physiological differences, environmental variations, or redundant information and noise in certain channels can pose challenges and impact the performance of BCI systems. In this study, we introduce a channel selection method utilizing Hybrid-Recursive Feature Elimination (H-RFE) combined with residual graph neural networks for MI recognition. This channel selection method employs a recursive feature elimination strategy and integrates three classification methods, namely random forest, gradient boosting, and logistic regression, as evaluators for adaptive channel selection tailored to specific subjects. To fully exploit the spatiotemporal information of multi-channel EEG, this study employed a graph neural network embedded with residual blocks to achieve precise recognition of motor imagery. We conducted algorithm testing using the SHU dataset and the PhysioNet dataset. Experimental results show that on the SHU dataset, utilizing 73.44% of the total channels, the cross-session MI recognition accuracy is 90.03%. Similarly, in the PhysioNet dataset, using 72.5% of the channel data, the classification result also reaches 93.99%. Compared to traditional strategies such as selecting three specific channels, correlation-based channel selection, mutual information-based channel selection, and adaptive channel selection based on Pearson coefficients and spatial positions, the proposed method improved classification accuracy by 34.64%, 10.8%, 3.25% and 2.88% on the SHU dataset, and by 46.96%, 5.04%, 5.81% and 2.32% on the PhysioNet dataset, respectively.

The right and left-hand motor imagery (MI) analysis based on the electroencephalogram (EEG) signal can directly link the central nervous system to a computer or a device. This study aims to identify a set of robust and nonlinear effective brain connectivity features quantified by transfer entropy (TE) to characterize the relationship between brain regions from EEG signals and create a hierarchical feature selection and classification for discrimination of right and lefthand MI tasks. TE is calculated among EEG channels as the distinctive, effective connectivity features. TE is a model-free method that can measure nonlinear effective connectivity and analyze multivariate dependent directed information flow among neural EEG channels. Then four feature subset selection methods namely relief-F, Fisher, Laplacian, and local learningbased clustering (LLCFS) algorithms are used to choose the most significant effective connectivity features and reduce redundant information. Finally, support vector machine (SVM) and linear discriminant analysis (LDA) methods are used for classification. Results show that the best performance in 29 healthy subjects and 60 trials is achieved using the TE method via the Relief-F algorithm as feature selection and support vector machine (SVM) classification with 91.02% accuracy. The TE index and a hierarchical feature selection and classification can be useful for the discrimination of right- and left-hand MI tasks from multichannel EEG signals. Effective connectivity features were extracted from electroencephalogram (EEG) to analyze relationships between regions.Four feature selection methods used to select most significant effective features.Support vector machine (SVM) used for discrimination of right and left hand motor imagery (MI) task. In this study, we investigated brain activity using effective connectivity during MI task based on EEG signals. The motor imagery task can accomplish the same goal as motor execution, since they are both activated by the same brain area. Transfer entropy, coherence, and Granger casualty were employed to extract the features. Differential patterns of activity between the left vs. right MI task showed activity around the motor area rather than other areas. In order to reduce redundant information and select the most significant effective connectivity features, four feature subset selection algorithms are used: Relief-F, Fisher, Laplacian, and learning-based clustering feature selection (LLCFS). Then, support vector machine (SVM) and linear discriminant analysis (LDA) are used to classify left and right hand MI task. Comparison of three different connectivity methods showed that TE index had the highest classification accuracy, and could be useful for the discrimination of right and left hand MI tasks from multichannel EEG signals.

In recent years, deep-learning-based motor imagery (MI) electroencephalography (EEG) decoding methods have shown great potential in the field of the brain–computer interface (BCI). The existing literature is relatively mature in decoding methods for two classes of MI tasks. However, with the increase in MI task classes, decoding studies for four classes of MI tasks need to be further explored. In addition, it is difficult to obtain large-scale EEG datasets. When the training data are limited, deep-learning-based decoding models are prone to problems such as overfitting and poor robustness. In this study, we design a data augmentation method for MI-EEG. The original EEG is slid along the time axis and reconstructed to expand the size of the dataset. Second, we combine the gated recurrent unit (GRU) and convolutional neural network (CNN) to construct a parallel-structured feature fusion network to decode four classes of MI tasks. The parallel structure can avoid temporal, frequency and spatial features interfering with each other. Experimenting on the well-known four-class MI dataset BCI Competition IV 2a shows a global average classification accuracy of 80.7% and a kappa value of 0.74. The proposed method improves the robustness of deep learning to decode small-scale EEG datasets and alleviates the overfitting phenomenon caused by insufficient data. The method can be applied to BCI systems with a small amount of daily recorded data.

Existing research methods for recognizing EEG (Electroencephalogram) signals in motor imagery (MI) often overlook the dynamic changes of brain networks over time, resulting in insufficient classification accuracy for MI tasks. This article addresses the recognition problem of dynamic changes in brain networks during MI tasks and applies an EEG signal classification method based on multi-layer time-varying functional brain networks. This article uses the BCI (Brain-Computer Interface) Competition IV 2a dataset to preprocess the raw EEG signals through bandpass filtering and CSP (Common Spatial Pattern) algorithm. The EEG signals of the MI task are divided into 7 1-second time windows with a step size of 0.5 seconds. Within each time window, Pearson correlation coefficients between EEG channels can be calculated to generate corresponding brain networks, and multiple time-varying functional brain networks can be constructed by stacking the brain networks from multiple time windows. The network topology features, node degree, clustering coefficient, network efficiency, and multi-layer network features of each window can be extracted, including Multiplex Clustering Coefficient (MCC), Multiplex Participation Coefficient (MPC), and inter layer correlation coefficient. By dividing the dataset through 10 fold cross validation, the random forest algorithm can be used to classify and recognize four types of motion imagination tasks. The experimental results show that the average recognition rate of the article’s method in four types of MI tasks reached 89.19%. This method can improve the classification accuracy of MI tasks and enhance a comprehensive understanding of the dynamic changes in brain networks during the process of MI.

Motor imagery electroencephalography (EEG) has been successfully used in the brain-computer interface (BCI) systems. Broad learning (BL) is an effective and efficient incremental learning algorithm with simple neural network structure. In this work, a novel EEG multi-classification method is proposed by combining with BL and common spatial pattern (CSP). Firstly, the CSP algorithm with the one-versus-the-test scheme is exploited to extract the discriminative multiclass brain patterns from raw EEG data, and then the BL algorithm is applied to the extracted features to discriminate the classes of EEG signals during different motor imagery tasks. Finally, the effectiveness of the proposed method has been verified on four-class motor imagery EEG data from BCI Competition IV Dataset 2a. Compare with other methods including ELM, HELM, DBN and SAE, the proposed method has yielded higher average classification test accuracy with less training time-consuming. The proposed method is meaningful and may have potential to apply into BCI field.

Multi-channel Electroencephalograph (EEG) signal is an important source of neural information for motor imagery (MI) limb movement intent decoding. The decoded MI movement intent often serve as potential control input for brain-computer interface (BCI) based rehabilitation robots. However, the presence of multiple dynamic artifacts in EEG signal leads to serious processing challenge that affects the BCI system in practical settings. Hence, this study propose a hybrid approach based on Low-rank spatiotemporal filtering technique for concurrent elimination of multiple EEG artifacts. Afterwards, a convolutional neural network based deep learning model (ConvNet-DL) that extracts neural information from the cleaned EEG signal for MI tasks decoding was built. The proposed method was studied in comparison with existing artifact removal methods using EEG signals of transhumeral amputees who performed five different MI tasks. Remarkably, the proposed method led to significant improvements in MI task decoding accuracy for the ConvNet-DL model in the range of 8.00~13.98%, while up to 14.38% increment was recorded in terms of the MCC: Mathew correlation coefficients at p<0.05. Also, a signal to error ratio of more than 11 dB was recorded by the proposed method.Clinical Relevance- This study showed that a combination of the proposed hybrid EEG artifact removal method and ConvNet-DL can significantly improve the decoding accuracy of MI upper limb movement tasks. Our findings may provide potential control input for BCI rehabilitation robotic systems.