EEG-based motor imagery classification using convolutional neural networks with local reparameterization trick

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

Objective. Deep transfer learning has been widely used to address the nonstationarity of electroencephalogram (EEG) data during motor imagery (MI) classification. However, previous deep learning approaches suffer from limited classification accuracy because the temporal and spatial features cannot be effectively extracted. Approach. Here, we propose a novel end-to-end deep subject adaptation convolutional neural network (SACNN) to handle the problem of EEG-based MI classification. Our proposed model jointly optimizes three modules, i.e. a feature extractor, a classifier, and a subject adapter. Specifically, the feature extractor simultaneously extracts the temporal and spatial features from the raw EEG data using a parallel multiscale convolution network. In addition, we design a subject adapter to reduce the feature distribution shift between the source and target subjects by using the maximum mean discrepancy. By minimizing the classification loss and the distribution discrepancy, the model is able to extract the temporal-spatial features to the prediction of a new subject. Main results. Extensive experiments are carried out on three EEG-based MI datasets, i.e. brain–computer interface (BCI) competition IV dataset IIb, BCI competition III dataset IVa, and BCI competition IV dataset I, and the average accuracy reaches to 86.42%, 81.71% and 79.35% on the three datasets respectively. Furthermore, the statistical analysis also indicates the significant performance improvement of SACNN. Significance. This paper reveals the importance of the temporal-spatial features on EEG-based MI classification task. Our proposed SACNN model can make fully use of the temporal-spatial information to achieve the purpose.

Objective. Classification of electroencephalography (EEG)-based motor imagery (MI) is a crucial non-invasive application in brain–computer interface (BCI) research. This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based MI classification that outperforms the state-of-the-art methods. Approach. The proposed CNN model, namely EEG-inception, is built on the backbone of the inception-time network, which has showed to be highly efficient and accurate for time-series classification. Also, the proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing. Furthermore, this paper proposes a novel data augmentation method for EEG signals to enhance the accuracy, at least by 3%, and reduce overfitting with limited BCI datasets. Main results. The proposed model outperforms all state-of-the-art methods by achieving the average accuracy of 88.4% and 88.6% on the 2008 BCI Competition IV 2a (four-classes) and 2b datasets (binary-classes), respectively. Furthermore, it takes less than 0.025 s to test a sample suitable for real-time processing. Moreover, the classification standard deviation for nine different subjects achieves the lowest value of 5.5 for the 2b dataset and 7.1 for the 2a dataset, which validates that the proposed method is highly robust. Significance. From the experiment results, it can be inferred that the EEG-inception network exhibits a strong potential as a subject-independent classifier for EEG-based MI tasks.

The unilateral upper limb classification from fMRI-weighted EEG signals using convolutional neural network

Objective. Motor imagery (MI)-based brain-computer interfaces (BCIs) provide an additional control pathway for people by decoding the intention of action imagination. The way people imagine greatly affects MI-BCI performance. Action itself is one of the factors that influence the way people imagine. Whether the different actions cause a difference in the MI performance is unknown. What is more important is how to manifest this action difference in the process of imagery, which has the potential to guide people to use their individualized actions to imagine more effectively. Approach. To explore action differences, this study proposes a novel paradigm named as action observation based delayed matching posture task. Ten subjects are required to observe, memorize, match, and imagine three types of actions (cutting, grasping and writing) given by visual images or videos, to accomplish the phases of encoding, retrieval and reinforcement of MI. Event-related potential (ERP), MI features, and classification accuracy of the left or the right hand are used to evaluate the effect of the action difference on the MI difference. Main results. Action differences cause different feature distributions, resulting in that the accuracy with high event-related (de)synchronization (ERD/ERS) is 27.75% higher than the ones with low ERD/ERS (p < 0.05), which indicates that the action difference has impact on the MI difference and the BCI performance. In addition, significant differences in the ERP amplitudes exists among the three actions: the amplitude of P300-N200 potential reaches 9.28 μV of grasping, 5.64 μV and 5.25 μV higher than the cutting and the writing, respectively (p < 0.05). Significance. The ERP amplitudes derived from the supplementary motor area shows positive correlation to the MI classification accuracy, implying that the ERP might be an index of the MI performance when the people is faced with action selection. This study demonstrates that the MI difference is related to the action difference, and can be manifested by the ERP, which is important for improving MI training by selecting suitable action; the relationship between the ERP and the MI provides a novel index to find the suitable action to set up an individualized BCI and improve the performance further.

As one of the most important research fields in the brain-computer interface (BCI) field, electroencephalogram (EEG) classification has a wide range of application values. However, for the EEG signal, it is difficult for the traditional neural networks to capture the characteristics of the EEG signal more comprehensively from the time and space dimensions, which has a certain effect on the accuracy of EEG classification. To solve this problem, we can improve the accuracy of classification via end-to-end learning of the time and space dimensions of EEG. In this paper, a new type of EEG classification network, the separable EEGNet (S-EEGNet), is proposed based on Hilbert-Huang transform (HHT) and a separable convolutional neural network (CNN) with bilinear interpolation. The EEG signal is transformed into time-frequency representation by HHT, which allows the EEG signal to be better described in the frequency domain. Then, the depthwise and pointwise elements of the network are combined to extract the feature map. The displacement variable is added by the bilinear interpolation method to the convolution layer of the separable CNN, allowing the free deformation of the sampling grid. The deformation depends on the local, dense, and adaptive input characteristics of the EEG data. The network can learn from the time and space dimensions of EEG signals end to end to extract features to improve the accuracy of EEG classification. To show the effectiveness of S-EEGNet, the team used this method to test two different types of EEG public datasets (motor imagery classification and emotion classification). The accuracy of motor imagery classification is 77.9%, and the accuracy of emotion classification is 89.91%, and 88.31%, respectively. The experimental results showed that the classification accuracy of S-EEGNet improved by 3.6%, 1.15%, and 1.33%, respectively.

Motor imagery (MI)-based brain-computer interface (BCI) systems are being increasingly employed to provide alternative means of communication and control for people suffering from neuro-motor impairments, with a special effort to bring these systems out of the controlled lab environments. Hence, accurately classifying MI from brain signals, e.g., from electroencephalography (EEG), is essential to obtain reliable BCI systems. However, MI classification is still a challenging task, because the signals are characterized by poor SNR, high intra-subject and cross-subject variability. Deep learning approaches have started to emerge as valid alternatives to standard machine learning techniques, e.g., filter bank common spatial pattern (FBCSP), to extract subject-independent features and to increase the cross-subject classification performance of MI BCI systems. In this paper, we first present a review of the most recent studies using deep learning for MI classification, with particular attention to their cross-subject performance. Second, we propose DynamicNet, a Python-based tool for quick and flexible implementations of deep learning models based on convolutional neural networks. We show-case the potentiality of DynamicNet by implementing EEGNet, a well-established architecture for effective EEG classification. Finally, we compare its performance with FBCSP in a 4-class MI classification over public datasets. To explore its cross-subject classification ability, we applied three different cross-validation schemes. From our results, we demonstrate that DynamicNet-implemented EEGNet outperforms FBCSP by about 25%, with a statistically significant difference when cross-subject validation schemes are applied.

Motor imagery (MI) is an important brain-computer interface (BCI) paradigm. The traditional MI paradigm (imagining different limbs) limits the intuitive control of the outer devices, while fine MI paradigm (imagining different joint movements from the same limb) can control the mechanical arm without cognitive disconnection. However, the decoding performance of fine MI limits its application. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are widely used in BCI systems because of their portability and easy operation. In this study, a fine MI paradigm including four classes (hand, wrist, shoulder and rest) was designed, and the data of EEG-fNIRS bimodal brain activity was collected from 12 subjects. Event-related desynchronization (ERD) from EEG signals shows a contralateral dominant phenomenon, and there is difference between the ERD of the four classes. For fNIRS signal in the time dimension, the time periods with significant difference can be observed in the activation patterns of four MI tasks. Spatially, the signal peak based brain topographic map also shows difference of these four MI tasks. The EEG signal and fNIRS signal of these four classes are distinguishable. In this study, a bimodal fusion network is proposed to improve the fine MI tasks decoding performance. The features of these two modalities are extracted separately by two feature extractors based on convolutional neural networks (CNN). The recognition performance was significantly improved by the bimodal method proposed in this study, compared with the performance of the single-modal network. The proposed method outperformed all comparison methods, and achieved a four-class accuracy of 58.96%. This paper demonstrates the feasibility of EEG and fNIRS bimodal BCI systems for fine MI, and shows the effectiveness of the proposed bimodal fusion method. This research is supposed to support fine MI-based BCI systems with theories and techniques.

To increase the performance Motor imagery (MI) EEG classification, a multi-scale window length algorithm was proposed in this paper. Under this algorithm, eight different time windows with the length of 600–2000 ms were selected to classify the MI data. The confidence level of the classification results from the eight windows were calculated simultaneously, to determine the weights for the outputs of each pre-classifier. The final classification results of MI tasks were generated by the weighted results from the eight windows. Using the proposed algorithm, the response time of MI classification was decreased by 11.4%, comparing to the single window method. On the same level of response time, the classification accuracy was increased by 2.8% by the multi-scale window algorithm. These results show that the proposed algorithm can speed up the detection of MI tasks with reliable classification accuracy.

In this study, we used two machine learning algorithms, namely, linear support vector machine (SVM) and convolutional neural network (CNN), to classify the BCI (Brain Computer interface) competition IV-2a 2-class MI (motor imagery) data set which consists of EEG data from 9 subjects. For each subject, 5 sessions of signals from three electrodes (C3, Cz, and C4) were recorded with sampling rate 250Hz. The training data, which consisted of the first 3 sessions, included 400 trials. The evaluation data, which consisted of the last 2 sessions, included 320 trials. Each trial started with gazing at fix cross on screen for 3 seconds followed by a one-second visual cue pointing either to the left or right to instruct the subject for left or right motor imagery over a period of 4 seconds, and then followed by a short break of at least 1.5 seconds. Features were extracted from the 0.5 to 2.5 second signals after the cue for each trial from C3 and C4. Each EEG trial was band pass filtered into different frequency bands, namely, delta (0.5-3Hz), theta (4-8Hz), alpha (8-12Hz), beta bands (13-30Hz), gamma bands (31-60Hz). Those filtered signals were then used as the input data for training the linear SVM. In addition, we generated a 2 by 500 matrix by down sampling the training data from each trial. There are 5760 such matrices in total generated from all subjects and serve as the input data for training CNN and the trained model was evaluated by another 340 matrices from each subject. Our CNN architecture consisted of 2 convolution layer and 2 fully connect layers, and there was a batch normalization layer before the activated layer and a dropout layer with a probability of 50% after the activated layer. The classification accuracies evaluated by averaged kappa values obtained from linear SVM and CNN are 0.5 and 0.621, respectively, suggesting the deep learning CNN method is superior to the classical linear SVM on the EEG classification.

MSAttNet: Multi-scale attention convolutional neural network for motor imagery classification.

One of the essential challenges in brain–computer interface is to classify motor imagery (MI) signals. In this paper, an ensemble SVM-based voting system is proposed. In each line of this system, the EEG signal is transformed into different representations based on discrete cosine transform, Fourier transform, common spatial pattern, and empirical mode decomposition, and then these representations are combined in a triple-frame matrix. These frames are fed into a pre-trained deep convolutional neural network as a feature extractor. For each line, an SVM is employed to classify the extracted feature vectors. Finally, a decision is made based on voting between these SVMs. Performance of the proposed method is examined on the BCI Competition III dataset Iva to separate right hand and foot movement imagery. The simple proposed method achieves the average accuracy of 96.34% for all of the subjects, and 99.70% for the best situation that is an improvement in MI classification. In addition, it can be seen that right side of the brain is more effective than the other side in EEG-based MI classification.

A cross-session motor imagery classification method based on Riemannian geometry and deep domain adaptation

The electroencephalogram (EEG) signal from motor imagery (MI) is used to drive brain–computer interaction (BCI). However, users usually are not adept at performing MI, which leads to low-quality EEG signals and decreases the performance of BCI applications. The humanoid robot stimulation approach can guide users in performing MI more proficiently by increasing the cortico-spinal excitability and improving the discrimination of event-related desynchronization patterns during MI tasks. Compared to the traditional stimulation modes, our proposed humanoid robot stimulation mode can activate higher quality MI EEG signals. We use convolutional neural network and long short-term memory algorithm for the extraction of EEG features and classification. The results showed that the CNN-LSTM can achieve the highest classification accuracy (93.7% ±1.7%) in the humanoid robot stimulation mode, and it outperformed all other classifier-stimulation mode combinations. This demonstrates the effectiveness and feasibility of using a humanoid robot in real-scene MI-BCI applications, such as service robots or rehabilitation system for person with motor disabilities.

Brain-computer interfaces (BCIs) based on electroencephalography (EEG) enable neural activity interpretation for device control, with motor imagery (MI) serving as a key paradigm for decoding imagined movements. Efficient feature extraction from raw EEG signals is essential to improve classification accuracy while minimizing reliance on extensive preprocessing. In this study, we introduce new hybrid architectures to enhance MI classification using data augmentation and a limited number of EEG channels. The first model combines a shallow convolutional neural network and a gated recurrent unit (CNN-GRU), while the second incorporates a convolutional neural network with a bidirectional gated recurrent unit (CNN-Bi-GRU). Evaluated using the publicly available PhysioNet dataset, the CNN-GRU classifier achieved peak mean accuracy rates of 99.71%, 99.73%, 99.61%, and 99.86% for tasks involving left fist (LF), right fist (RF), both fists (LRF), and both feet (BF), respectively. The experimental results provide compelling evidence that our proposed models outperform current state-of-the-art methods, underscoring their efficiency on small-scale EEG datasets. The CNN-GRU and CNN-Bi-GRU architectures exhibit superior predictive reliability, offering a faster, cost-effective solution for user-adaptable MI-BCI applications.