CTNet: a convolutional transformer network for EEG-based motor imagery classification

Objective. Deep transfer learning has been widely used to address the nonstationarity of electroencephalogram (EEG) data during motor imagery (MI) classification. However, previous deep learning approaches suffer from limited classification accuracy because the temporal and spatial features cannot be effectively extracted. Approach. Here, we propose a novel end-to-end deep subject adaptation convolutional neural network (SACNN) to handle the problem of EEG-based MI classification. Our proposed model jointly optimizes three modules, i.e. a feature extractor, a classifier, and a subject adapter. Specifically, the feature extractor simultaneously extracts the temporal and spatial features from the raw EEG data using a parallel multiscale convolution network. In addition, we design a subject adapter to reduce the feature distribution shift between the source and target subjects by using the maximum mean discrepancy. By minimizing the classification loss and the distribution discrepancy, the model is able to extract the temporal-spatial features to the prediction of a new subject. Main results. Extensive experiments are carried out on three EEG-based MI datasets, i.e. brain–computer interface (BCI) competition IV dataset IIb, BCI competition III dataset IVa, and BCI competition IV dataset I, and the average accuracy reaches to 86.42%, 81.71% and 79.35% on the three datasets respectively. Furthermore, the statistical analysis also indicates the significant performance improvement of SACNN. Significance. This paper reveals the importance of the temporal-spatial features on EEG-based MI classification task. Our proposed SACNN model can make fully use of the temporal-spatial information to achieve the purpose.

Objective. Classification of electroencephalography (EEG)-based motor imagery (MI) is a crucial non-invasive application in brain–computer interface (BCI) research. This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based MI classification that outperforms the state-of-the-art methods. Approach. The proposed CNN model, namely EEG-inception, is built on the backbone of the inception-time network, which has showed to be highly efficient and accurate for time-series classification. Also, the proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing. Furthermore, this paper proposes a novel data augmentation method for EEG signals to enhance the accuracy, at least by 3%, and reduce overfitting with limited BCI datasets. Main results. The proposed model outperforms all state-of-the-art methods by achieving the average accuracy of 88.4% and 88.6% on the 2008 BCI Competition IV 2a (four-classes) and 2b datasets (binary-classes), respectively. Furthermore, it takes less than 0.025 s to test a sample suitable for real-time processing. Moreover, the classification standard deviation for nine different subjects achieves the lowest value of 5.5 for the 2b dataset and 7.1 for the 2a dataset, which validates that the proposed method is highly robust. Significance. From the experiment results, it can be inferred that the EEG-inception network exhibits a strong potential as a subject-independent classifier for EEG-based MI tasks.

A hybrid approach for EEG motor imagery classification using adaptive margin disparity and knowledge transfer in convolutional neural networks.

One of the essential challenges in brain–computer interface is to classify motor imagery (MI) signals. In this paper, an ensemble SVM-based voting system is proposed. In each line of this system, the EEG signal is transformed into different representations based on discrete cosine transform, Fourier transform, common spatial pattern, and empirical mode decomposition, and then these representations are combined in a triple-frame matrix. These frames are fed into a pre-trained deep convolutional neural network as a feature extractor. For each line, an SVM is employed to classify the extracted feature vectors. Finally, a decision is made based on voting between these SVMs. Performance of the proposed method is examined on the BCI Competition III dataset Iva to separate right hand and foot movement imagery. The simple proposed method achieves the average accuracy of 96.34% for all of the subjects, and 99.70% for the best situation that is an improvement in MI classification. In addition, it can be seen that right side of the brain is more effective than the other side in EEG-based MI classification.

Graph vertex and spectral features for EEG-based motor imagery classification.

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

Training an accurate classifier for EEG-based brain-computer interface (BCI) requires EEG data from a large number of users, whereas protecting their data privacy is a critical consideration. Federated learning (FL) is a promising solution to this challenge. This paper proposes Federated classification with local Batch-specific batch normalization and Sharpness-aware minimization (FedBS) for privacy protection in EEG-based motor imagery (MI) classification. FedBS utilizes local batch-specific batch normalization to reduce data discrepancies among different clients, and sharpness-aware minimization optimizer in local training to improve model generalization. Experiments on three public MI datasets using three popular deep learning models demonstrated that FedBS outperformed six state-of-the-art FL approaches. Remarkably, it also outperformed centralized training, which does not consider privacy protection at all. In summary, FedBS protects user EEG data privacy, enabling multiple BCI users to participate in large-scale machine learning model training, which in turn improves the BCI decoding accuracy.

Brain Computer Interface (BCI) has become one of the most interesting alternatives to support automatic systems able to interpret brain functions. Recently, the Motor Imagery (MI) paradigm is a widely topic of interest as a tool to develop BCI-based systems. Here, we present a relevant feature extraction methodology, termed MI discrimination using kernel relevance analysis (MIDKRA), to support MI classification in BCI systems. For such a purpose, a similarity criterion to rank the contribution of EEG features for classifying an MI paradigm is employed. The introduced approach includes the supervised information regarding the MI paradigm to find out a relevant set of features encoding discriminative information. We model the EEG recordings by considering both time and time-frequency representations. Moreover, a k nearest-neighbor classifier is carried out to validate the proposed feature relevance approach. Experimental results carried out on two different BCI databases, a well-known public MI data and an Emotiv-based dataset built by us, demonstrate that MIDKRA outperforms state of the art methods and it is a suitable alternative to support straightforward BCI systems.

In this study, we used two machine learning algorithms, namely, linear support vector machine (SVM) and convolutional neural network (CNN), to classify the BCI (Brain Computer interface) competition IV-2a 2-class MI (motor imagery) data set which consists of EEG data from 9 subjects. For each subject, 5 sessions of signals from three electrodes (C3, Cz, and C4) were recorded with sampling rate 250Hz. The training data, which consisted of the first 3 sessions, included 400 trials. The evaluation data, which consisted of the last 2 sessions, included 320 trials. Each trial started with gazing at fix cross on screen for 3 seconds followed by a one-second visual cue pointing either to the left or right to instruct the subject for left or right motor imagery over a period of 4 seconds, and then followed by a short break of at least 1.5 seconds. Features were extracted from the 0.5 to 2.5 second signals after the cue for each trial from C3 and C4. Each EEG trial was band pass filtered into different frequency bands, namely, delta (0.5-3Hz), theta (4-8Hz), alpha (8-12Hz), beta bands (13-30Hz), gamma bands (31-60Hz). Those filtered signals were then used as the input data for training the linear SVM. In addition, we generated a 2 by 500 matrix by down sampling the training data from each trial. There are 5760 such matrices in total generated from all subjects and serve as the input data for training CNN and the trained model was evaluated by another 340 matrices from each subject. Our CNN architecture consisted of 2 convolution layer and 2 fully connect layers, and there was a batch normalization layer before the activated layer and a dropout layer with a probability of 50% after the activated layer. The classification accuracies evaluated by averaged kappa values obtained from linear SVM and CNN are 0.5 and 0.621, respectively, suggesting the deep learning CNN method is superior to the classical linear SVM on the EEG classification.

BackgroundThe aging of the world population poses a major health challenge, and brain–computer interface (BCI) technology has the potential to provide assistance and rehabilitation for the elderly.ObjectivesThis study aimed to investigate the electroencephalogram (EEG) characteristics during motor imagery by comparing young and elderly, and study Convolutional Neural Networks (CNNs) classification for the elderly population in terms of fatigue analysis in both frontal and parietal regions.MethodsA total of 20 healthy individuals participated in the study, including 10 young and 10 older adults. All participants completed the left- and right-hand motor imagery experiment. The energy changes in the motor imagery process were analyzed using time–frequency graphs and quantified event-related desynchronization (ERD) values. The fatigue level of the motor imagery was assessed by two indicators: (θ + α)/β and θ/β, and fatigue-sensitive channels were distinguished from the parietal region of the brain. Then, rhythm entropy was introduced to analyze the complexity of the cognitive activity. The phase-lock values related to the parietal and frontal lobes were calculated, and their temporal synchronization was discussed. Finally, the motor imagery EEG data was classified by CNNs, and the accuracy was discussed based on the analysis results.ResultFor the young and elderly, ERD was observed in C3 and C4 channels, and their fatigue-sensitive channels in the parietal region were slightly different. During the experiment, the rhythm entropy of the frontal lobe showed a decreasing trend with time for most of the young subjects, while there was an increasing trend for most of the older ones. Using the CNN classification method, the elderly achieved around 70% of the average classification accuracy, which is almost the same for the young adults.ConclusionCompared with the young adults, the elderly are less affected by the level of cognitive fatigue during motor imagery, but the classification accuracy of motor imagery data in the elderly may be slightly lower than that in young persons. At the same time, the deep learning method also provides a potentially feasible option for the application of motor-imagery BCI (MI-BCI) in the elderly by considering the ERD and fatigue phenomenon together.

EEG-based motor imagery classification using convolutional neural networks with local reparameterization trick

Motor imagery (MI) is a mental process widely utilized as the experimental paradigm for brain-computer interfaces (BCIs) across a broad range of basic science and clinical studies. However, decoding intentions from MI remains challenging due to the inherent complexity of brain patterns relative to the small sample size available for machine learning. This paper proposes an end-to-end Filter-Bank Multiscale Convolutional Neural Network (FBMSNet) for MI classification. A filter bank is first employed to derive a multiview spectral representation of the EEG data. Mixed depthwise convolution is then applied to extract temporal features at multiple scales, followed by spatial filtering to mitigate volume conduction. Finally, with the joint supervision of cross-entropy and center loss, FBMSNet obtains features that maximize interclass dispersion and intraclass compactness. We compare FBMSNet with several state-of-the-art EEG decoding methods on two MI datasets: the BCI Competition IV 2a dataset and the OpenBMI dataset. FBMSNet significantly outperforms the benchmark methods by achieving 79.17% and 70.05% for four-class and two-class hold-out classification accuracy, respectively. These results demonstrate the efficacy of FBMSNet in improving EEG decoding performance toward more robust BCI applications. The FBMSNet source code is available at https://github.com/Want2Vanish/FBMSNet.

Assessing robustness to adversarial attacks in attention-based networks: Case of EEG-based motor imagery classification

The brain-computer interface (BCI) work is to provide humans an alternative channel that allows direct transmission of messages from the brain by analyzing the brain’s mental activities [1–7]. The brain activity is recorded by means of multi-electrode electroencephalographic (EEG) signals that are either invasive or noninvasive. Noninvasive recording is convenient and popular in BCI applications so it is commonly used. According to the definition suggested at the first international meeting for BCI technology, the term BCI is reserved for a system that must not depend on the brain’s normal output pathways of peripheral nerves and muscles [2]. It has become popular for BCI systems on motor imagery (MI) EEG signals in the last decade [8]. It reveals that there are special characteristics of event-related desynchronization (ERD) and synchronization (ERS) in mu and beta rhythms over the sensorimotor cortex during MI tasks by discriminating EEG signals between left and right MIs [9, 10]. ERD/ERS is the task-related or event-related change in the amplitude of the oscillatory behavior of specific cortical areas within various frequency bands. An amplitude (or power) increase is defined as event-related synchronization while an amplitude (or power) decrease is defined as event-related desynchronization. As other event-related potentials, ERD/ERS patterns are associated with sensory processing and motor behavior [2]. The principal objective of this study is to propose a BCI system, which combines neuro-fuzzy prediction and multiresolution fractal feature vectors (MFFVs) with support vector machine, for MI classification.

A motor imagery (MI) brain-computer interface (BCI) plays an important role in the neurological rehabilitation training for stroke patients. Electroencephalogram (EEG)-based MI BCI has high temporal resolution, which is convenient for real-time BCI control. Therefore, we focus on EEG-based MI BCI in this paper. The identification of MI EEG signals is always quite challenging. Due to high inter-session/subject variability, each subject should spend long and tedious calibration time in collecting amounts of labeled samples for a subject-specific model. To cope with this problem, we present a supervised selective cross-subject transfer learning (sSCSTL) approach which simultaneously makes use of the labeled samples from target and source subjects based on Riemannian tangent space. Since the covariance matrices representing the multi-channel EEG signals belong to the smooth Riemannian manifold, we perform the Riemannian alignment to make the covariance matrices from different subjects close to each other. Then, all aligned covariance matrices are converted into the Riemannian tangent space features to train a classifier in the Euclidean space. To investigate the role of unlabeled samples, we further propose semi-supervised and unsupervised versions which utilize the total samples and unlabeled samples from target subject, respectively. Sequential forward floating search (SFFS) method is executed for source selection. All our proposed algorithms transfer the labeled samples from most suitable source subjects into the feature space of target subject. Experimental results on two publicly available MI datasets demonstrated that our algorithms outperformed several state-of-the-art algorithms using small number of the labeled samples from target subject, especially for good target subjects.