A cross-session motor imagery classification method based on Riemannian geometry and deep domain adaptation

EEG-based motor imagery classification using convolutional neural networks with local reparameterization trick

The existing Riemannian geometry-based approaches for brain computer interface (BCI) employ fixed time windows. However, the inherent variability and dynamic changes among subjects necessitate robust and adaptive solutions for time window optimization. Recognizing the current limitations of Riemannian classifiers, we propose a time window selection confidence metric (TWSCM) based on Riemannian geometry. This metric operates on the manifold of symmetric positive definite (SPD) matrices, providing a theoretically grounded and computationally efficient approach for time window optimization. The optimization process is unsupervised, which is able to deal with the online scenario without training labels. Experimental results on the BCI competition IV dataset IIa demonstrate that the classification performance is significantly improved for most subjects. The average performance over six subjects improved by 7.52%. The simulated online experiment shows enhanced performance in comparison to baseline experiments without time window optimization. Additionally, an in-depth analysis of TWSCM provides insights into performance variations among subjects. Overall, this paper introduces the first time window optimization method within the Riemannian geometric framework, presenting an effective and interpretable approach for optimizing time windows in motor imagery classification, providing a novel and promising perspective in EEG signal analysis.

In recent years, Brain Computer Interface (BCI) based on motor imagery has been widely used in the fields of medicine, active safe systems for automobiles, entertainment, and so on. Motor imagery relevant electroencephalogram (EEG) signals are weak, nonlinear, and susceptible to interference. As a feature extraction method for motor imagery, Common Spatial Pattern (CSP) has been proven to be very effective. However, its effectiveness depends heavily on the choice of frequency bands, and Euclidean space cannot effectively describe the inner relationship. To solve these problems, a classification approach for motor imagery using the Geodesic Filtering Common Spatial Pattern (GFCSP) and filter-bank Feature Weighted Support Vector Machine (FWSVM) is presented. First, GFCSP based on the Riemannian manifold is proposed, in which the extracted covariance features are spatially filtered in Riemannian tangent space, and the average covariance matrix is replaced by Riemannian mean in CSP. Second, filter-bank FWSVM with a feature weighted matrix is proposed. EEG signals are filtered into 8-12 Hz, 12-16 Hz, 18-22 Hz, 22-26 Hz, and a wide band of 8-24 Hz, and GFCSP features of these filtered signals are extracted. A feature weighted matrix is calculated using mutual information and the Pearson correlation coefficient from these features and class information. Then, the Support Vector Machine (SVM) is used for classification with the feature weighted matrix. Finally, the proposed method is validated on the dataset IVa in BCI competition III. Classification accuracies of the five subjects are 92.31%, 99.03%, 80.36%, 96.30%, and 97.67%, which demonstrate the effectiveness of our proposed method.

The common spatial pattern (CSP) is a popular method in feature extraction for motor imagery (MI) electroencephalogram (EEG) classification in brain-computer interface (BCI) systems. However, combining temporal and spectral information in the CSP-based spatial features is still a challenging issue, which greatly affects the performance of MI-based BCI systems. Here, we propose a novel circulant singular spectrum analysis embedded CSP (CiSSA-CSP) method for learning the optimal time-frequency-spatial features to improve the MI classification accuracy. Specifically, raw EEG data are first segmented into multiple time segments and spectrum-specific sub-bands are further derived by CiSSA from each time segment in a set of non-overlapping filter bands. CSP features extracted from all time-frequency segments contain more sufficient time-frequency-spatial information. An experimental study was implemented on the publicly available EEG dataset (BCI Competition III dataset IVa) and a self-collected experimental EEG dataset to validate the effectiveness of the CiSSA-CSP method. Experimental results demonstrate that discriminative and robust features are extracted effectively. Compared with several state-of-the-art methods, the proposed method exhibited optimal accuracies of 96.6% and 95.2% on the public and experimental datasets, respectively, which confirms that it is a promising method for improving the performance of MI-based BCIs.

Existing research methods for recognizing EEG (Electroencephalogram) signals in motor imagery (MI) often overlook the dynamic changes of brain networks over time, resulting in insufficient classification accuracy for MI tasks. This article addresses the recognition problem of dynamic changes in brain networks during MI tasks and applies an EEG signal classification method based on multi-layer time-varying functional brain networks. This article uses the BCI (Brain-Computer Interface) Competition IV 2a dataset to preprocess the raw EEG signals through bandpass filtering and CSP (Common Spatial Pattern) algorithm. The EEG signals of the MI task are divided into 7 1-second time windows with a step size of 0.5 seconds. Within each time window, Pearson correlation coefficients between EEG channels can be calculated to generate corresponding brain networks, and multiple time-varying functional brain networks can be constructed by stacking the brain networks from multiple time windows. The network topology features, node degree, clustering coefficient, network efficiency, and multi-layer network features of each window can be extracted, including Multiplex Clustering Coefficient (MCC), Multiplex Participation Coefficient (MPC), and inter layer correlation coefficient. By dividing the dataset through 10 fold cross validation, the random forest algorithm can be used to classify and recognize four types of motion imagination tasks. The experimental results show that the average recognition rate of the article’s method in four types of MI tasks reached 89.19%. This method can improve the classification accuracy of MI tasks and enhance a comprehensive understanding of the dynamic changes in brain networks during the process of MI.

As one of the most important research fields in the brain-computer interface (BCI) field, electroencephalogram (EEG) classification has a wide range of application values. However, for the EEG signal, it is difficult for the traditional neural networks to capture the characteristics of the EEG signal more comprehensively from the time and space dimensions, which has a certain effect on the accuracy of EEG classification. To solve this problem, we can improve the accuracy of classification via end-to-end learning of the time and space dimensions of EEG. In this paper, a new type of EEG classification network, the separable EEGNet (S-EEGNet), is proposed based on Hilbert-Huang transform (HHT) and a separable convolutional neural network (CNN) with bilinear interpolation. The EEG signal is transformed into time-frequency representation by HHT, which allows the EEG signal to be better described in the frequency domain. Then, the depthwise and pointwise elements of the network are combined to extract the feature map. The displacement variable is added by the bilinear interpolation method to the convolution layer of the separable CNN, allowing the free deformation of the sampling grid. The deformation depends on the local, dense, and adaptive input characteristics of the EEG data. The network can learn from the time and space dimensions of EEG signals end to end to extract features to improve the accuracy of EEG classification. To show the effectiveness of S-EEGNet, the team used this method to test two different types of EEG public datasets (motor imagery classification and emotion classification). The accuracy of motor imagery classification is 77.9%, and the accuracy of emotion classification is 89.91%, and 88.31%, respectively. The experimental results showed that the classification accuracy of S-EEGNet improved by 3.6%, 1.15%, and 1.33%, respectively.

Research on brain–computer interfaces (BCIs) advances the way scientists understand how the human brain functions. The BCI system, which is based on the use of electroencephalography (EEG) signals to detect motor imagery (MI) tasks, enables opportunities for various applications in stroke rehabilitation, neuroprosthetic devices, and communication tools. BCIs can also be used in emotion recognition (ER) research to depict the sophistication of human emotions by improving mental health monitoring, human–computer interactions, and neuromarketing. To address the low accuracy of MI-BCI, which is a key issue faced by researchers, this study employs a new approach that has been proven to have the potential to enhance motor imagery classification accuracy. The basic idea behind the approach is to apply feature extraction methods from the field of emotion recognition to the field of motor imagery. Six feature sets and four classifiers were explored using four MI classes (left and right hands, both feet, and tongue) from the BCI Competition IV 2a dataset. Statistical, wavelet analysis, Hjorth parameters, higher-order spectra, fractal dimensions (Katz, Higuchi, and Petrosian), and a five-dimensional combination of all five feature sets were implemented. GSVM, CART, LinearSVM, and SVM with polynomial kernel classifiers were considered. Our findings show that 3D fractal dimensions predominantly outperform all other feature sets, specifically during LinearSVM classification, accomplishing nearly 79.1% mean accuracy, superior to the state-of-the-art results obtained from the referenced MI paper, where CSP reached 73.7% and Riemannian methods reached 75.5%. It even performs as well as the latest TWSB method, which also reached approximately 79.1%. These outcomes emphasize that the new hybrid approach in the motor imagery/emotion recognition field improves classification accuracy when applied to motor imagery EEG signals, thus enhancing MI-BCI performance.

Objective. Motor imagery (MI)-based brain-computer interfaces (BCIs) provide an additional control pathway for people by decoding the intention of action imagination. The way people imagine greatly affects MI-BCI performance. Action itself is one of the factors that influence the way people imagine. Whether the different actions cause a difference in the MI performance is unknown. What is more important is how to manifest this action difference in the process of imagery, which has the potential to guide people to use their individualized actions to imagine more effectively. Approach. To explore action differences, this study proposes a novel paradigm named as action observation based delayed matching posture task. Ten subjects are required to observe, memorize, match, and imagine three types of actions (cutting, grasping and writing) given by visual images or videos, to accomplish the phases of encoding, retrieval and reinforcement of MI. Event-related potential (ERP), MI features, and classification accuracy of the left or the right hand are used to evaluate the effect of the action difference on the MI difference. Main results. Action differences cause different feature distributions, resulting in that the accuracy with high event-related (de)synchronization (ERD/ERS) is 27.75% higher than the ones with low ERD/ERS (p < 0.05), which indicates that the action difference has impact on the MI difference and the BCI performance. In addition, significant differences in the ERP amplitudes exists among the three actions: the amplitude of P300-N200 potential reaches 9.28 μV of grasping, 5.64 μV and 5.25 μV higher than the cutting and the writing, respectively (p < 0.05). Significance. The ERP amplitudes derived from the supplementary motor area shows positive correlation to the MI classification accuracy, implying that the ERP might be an index of the MI performance when the people is faced with action selection. This study demonstrates that the MI difference is related to the action difference, and can be manifested by the ERP, which is important for improving MI training by selecting suitable action; the relationship between the ERP and the MI provides a novel index to find the suitable action to set up an individualized BCI and improve the performance further.

Assessing robustness to adversarial attacks in attention-based networks: Case of EEG-based motor imagery classification

Objective. Deep transfer learning has been widely used to address the nonstationarity of electroencephalogram (EEG) data during motor imagery (MI) classification. However, previous deep learning approaches suffer from limited classification accuracy because the temporal and spatial features cannot be effectively extracted. Approach. Here, we propose a novel end-to-end deep subject adaptation convolutional neural network (SACNN) to handle the problem of EEG-based MI classification. Our proposed model jointly optimizes three modules, i.e. a feature extractor, a classifier, and a subject adapter. Specifically, the feature extractor simultaneously extracts the temporal and spatial features from the raw EEG data using a parallel multiscale convolution network. In addition, we design a subject adapter to reduce the feature distribution shift between the source and target subjects by using the maximum mean discrepancy. By minimizing the classification loss and the distribution discrepancy, the model is able to extract the temporal-spatial features to the prediction of a new subject. Main results. Extensive experiments are carried out on three EEG-based MI datasets, i.e. brain–computer interface (BCI) competition IV dataset IIb, BCI competition III dataset IVa, and BCI competition IV dataset I, and the average accuracy reaches to 86.42%, 81.71% and 79.35% on the three datasets respectively. Furthermore, the statistical analysis also indicates the significant performance improvement of SACNN. Significance. This paper reveals the importance of the temporal-spatial features on EEG-based MI classification task. Our proposed SACNN model can make fully use of the temporal-spatial information to achieve the purpose.

One of the essential challenges in brain–computer interface is to classify motor imagery (MI) signals. In this paper, an ensemble SVM-based voting system is proposed. In each line of this system, the EEG signal is transformed into different representations based on discrete cosine transform, Fourier transform, common spatial pattern, and empirical mode decomposition, and then these representations are combined in a triple-frame matrix. These frames are fed into a pre-trained deep convolutional neural network as a feature extractor. For each line, an SVM is employed to classify the extracted feature vectors. Finally, a decision is made based on voting between these SVMs. Performance of the proposed method is examined on the BCI Competition III dataset Iva to separate right hand and foot movement imagery. The simple proposed method achieves the average accuracy of 96.34% for all of the subjects, and 99.70% for the best situation that is an improvement in MI classification. In addition, it can be seen that right side of the brain is more effective than the other side in EEG-based MI classification.

A motor imagery (MI) brain-computer interface (BCI) plays an important role in the neurological rehabilitation training for stroke patients. Electroencephalogram (EEG)-based MI BCI has high temporal resolution, which is convenient for real-time BCI control. Therefore, we focus on EEG-based MI BCI in this paper. The identification of MI EEG signals is always quite challenging. Due to high inter-session/subject variability, each subject should spend long and tedious calibration time in collecting amounts of labeled samples for a subject-specific model. To cope with this problem, we present a supervised selective cross-subject transfer learning (sSCSTL) approach which simultaneously makes use of the labeled samples from target and source subjects based on Riemannian tangent space. Since the covariance matrices representing the multi-channel EEG signals belong to the smooth Riemannian manifold, we perform the Riemannian alignment to make the covariance matrices from different subjects close to each other. Then, all aligned covariance matrices are converted into the Riemannian tangent space features to train a classifier in the Euclidean space. To investigate the role of unlabeled samples, we further propose semi-supervised and unsupervised versions which utilize the total samples and unlabeled samples from target subject, respectively. Sequential forward floating search (SFFS) method is executed for source selection. All our proposed algorithms transfer the labeled samples from most suitable source subjects into the feature space of target subject. Experimental results on two publicly available MI datasets demonstrated that our algorithms outperformed several state-of-the-art algorithms using small number of the labeled samples from target subject, especially for good target subjects.

The brain-computer interface (BCI) work is to provide humans an alternative channel that allows direct transmission of messages from the brain by analyzing the brain’s mental activities [1–7]. The brain activity is recorded by means of multi-electrode electroencephalographic (EEG) signals that are either invasive or noninvasive. Noninvasive recording is convenient and popular in BCI applications so it is commonly used. According to the definition suggested at the first international meeting for BCI technology, the term BCI is reserved for a system that must not depend on the brain’s normal output pathways of peripheral nerves and muscles [2]. It has become popular for BCI systems on motor imagery (MI) EEG signals in the last decade [8]. It reveals that there are special characteristics of event-related desynchronization (ERD) and synchronization (ERS) in mu and beta rhythms over the sensorimotor cortex during MI tasks by discriminating EEG signals between left and right MIs [9, 10]. ERD/ERS is the task-related or event-related change in the amplitude of the oscillatory behavior of specific cortical areas within various frequency bands. An amplitude (or power) increase is defined as event-related synchronization while an amplitude (or power) decrease is defined as event-related desynchronization. As other event-related potentials, ERD/ERS patterns are associated with sensory processing and motor behavior [2]. The principal objective of this study is to propose a BCI system, which combines neuro-fuzzy prediction and multiresolution fractal feature vectors (MFFVs) with support vector machine, for MI classification.

关键词 运动想象 运动观察 ERD/ERD Gabor 滤波器 神经生理 运动观察(movement/action

Classification of Motor Imagery (MI) signals is the heart of Brain-Computer Interface (BCI) based applications. Spatial filtering is an important step in this process that produce new set of signals for better discrimination of two classes of EEG signals. In this work, a new approach of spatial filtering called Space-Frequency Localized Spatial Filtering (SFLSF) is proposed to enhance the performances of MI classification. The SFLSF method initially divides the scalp-EEG channels into local overlapping spatial windows. Then a filter bank is used to divide the signals into local frequency bands. The group of channels, localized in space and frequency, are then processed with spatial filter, and features are subsequently extracted for classification task. Experimental results corroborate that the proposed space localization helps to increase the classification accuracy when compared to the existing methods using spatial filters. The classification performance is further improved when frequency localization is incorporated. Thus, the proposed space-frequency localized approach of spatial filtering helps to deliver better classification result which is consistently 3-5% higher than traditional methods.