A classification method for EEG motor imagery signals based on parallel convolutional neural network

In recent years, the utilization of motor imagery (MI) signals derived from electroencephalography (EEG) has shown promising applications in controlling various devices such as wheelchairs, assistive technologies, and driverless vehicles. However, decoding EEG signals poses significant challenges due to their complexity, dynamic nature, and low signal-to-noise ratio (SNR). Traditional EEG pattern recognition algorithms typically involve two key steps: feature extraction and feature classification, both crucial for accurate operation. In this work, we propose a novel method that addresses these challenges by employing empirical mode decomposition (EMD) for feature extraction and a parallel convolutional neural network (PCNN) for feature classification. This approach aims to mitigate non-stationary issues, improve performance speed, and enhance classification accuracy. We validate the effectiveness of our proposed method using datasets from the BCI competition IV, specifically datasets 2a and 2b, which contain motor imagery EEG signals. Our method focuses on identifying two- and four-class motor imagery EEG signal classifications. Additionally, we introduce a transfer learning technique to fine-tune the model for individual subjects, leveraging important features extracted from a group dataset. Our results demonstrate that the proposed EMD-PCNN method outperforms existing approaches in terms of classification accuracy. We conduct both qualitative and quantitative analyses to evaluate our method. Qualitatively, we employ confusion matrices and various performance metrics such as specificity, sensitivity, precision, accuracy, recall, and f1-score. Quantitatively, we compare the classification accuracies of our method with those of existing approaches. Our findings highlight the superiority of the proposed EMD-PCNN method in accurately classifying motor imagery EEG signals. The enhanced performance and robustness of our method underscore its potential for broader applicability in real-world scenarios.

Advances in the field of Brain-Computer Interfaces (BCIs) aim, among other applications, to improve the movement capacities of people suffering from the loss of motor skills. The main challenge in this area is to achieve real-time and accurate bio-signal processing for pattern recognition, especially in Motor Imagery (MI). The significant interaction between brain signals and controllable machines requires instantaneous brain data decoding. In this study, an embedded BCI system based on fist MI signals is developed. It uses an Emotiv EPOC+ Brainwear®, an Altera SoCKit® development board, and a hexapod robot for testing locomotion imagery commands. The system is tested to detect the imagined movements of closing and opening the left and right hand to control the robot locomotion. Electroencephalogram (EEG) signals associated with the motion tasks are sensed on the human sensorimotor cortex. Next, the SoCKit processes the data to identify the commands allowing the controlled robot locomotion. The classification of MI-EEG signals from the F3, F4, FC5, and FC6 sensors is performed using a hybrid architecture of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. This method takes advantage of the deep learning recognition model to develop a real-time embedded BCI system, where signal processing must be seamless and precise. The proposed method is evaluated using k-fold cross-validation on both created and public Scientific-Data datasets. Our dataset is comprised of 2400 trials obtained from four test subjects, lasting three seconds of closing and opening fist movement imagination. The recognition tasks reach 84.69% and 79.2% accuracy using our data and a state-of-the-art dataset, respectively. Numerical results support that the motor imagery EEG signals can be successfully applied in BCI systems to control mobile robots and related applications such as intelligent vehicles.

A convolutional spiking neural network with adaptive coding for motor imagery classification

Objective. Deep transfer learning has been widely used to address the nonstationarity of electroencephalogram (EEG) data during motor imagery (MI) classification. However, previous deep learning approaches suffer from limited classification accuracy because the temporal and spatial features cannot be effectively extracted. Approach. Here, we propose a novel end-to-end deep subject adaptation convolutional neural network (SACNN) to handle the problem of EEG-based MI classification. Our proposed model jointly optimizes three modules, i.e. a feature extractor, a classifier, and a subject adapter. Specifically, the feature extractor simultaneously extracts the temporal and spatial features from the raw EEG data using a parallel multiscale convolution network. In addition, we design a subject adapter to reduce the feature distribution shift between the source and target subjects by using the maximum mean discrepancy. By minimizing the classification loss and the distribution discrepancy, the model is able to extract the temporal-spatial features to the prediction of a new subject. Main results. Extensive experiments are carried out on three EEG-based MI datasets, i.e. brain–computer interface (BCI) competition IV dataset IIb, BCI competition III dataset IVa, and BCI competition IV dataset I, and the average accuracy reaches to 86.42%, 81.71% and 79.35% on the three datasets respectively. Furthermore, the statistical analysis also indicates the significant performance improvement of SACNN. Significance. This paper reveals the importance of the temporal-spatial features on EEG-based MI classification task. Our proposed SACNN model can make fully use of the temporal-spatial information to achieve the purpose.

By 2021, WHO projects over a billion incapacitated people, with 20% facing daily functional impairments.Brain-Computer Interface (BCI) offers effortless machine control via direct brain-computer interaction, with Motor Imagery (MI) Electroencephalogram (EEG) as the key BCI foundation.The MI EEG signals were collected from the BNCI horizon2020 database for nine participants.The MI EEG data includes four tasks: imaginative movement of the left hand, right hand, feet, and tongue.There are 22 EEG channels with a sampling rate of 250Hz in the data.The MI EEG signals were band-pass filtered with a lower cut-off frequency of 0.5 Hz and an upper cut-off frequency of 100 Hz.Based on the energy count threshold approach (ECTA) nine channels were identified as dominating channels from the filtered MI EEG signals.Energy values for each channel were extracted in the ECTA method for the 3-sec window.And if the energy value of a channel for a particular window is greater than 60% of the maximum channel's energy, then the energy count value will be incremented by one.Finally whichever channels had larger energy counts were identified as a dominant channel.On these nine dominant MI EEG signals discrete wavelet transform using Daubechies 4 mother wavelet a four-level decomposition is applied, and Mu and beta rhythms were extracted.For a 3-sec window from the nine dominant channels, the energy and entropy feature values from the Mu and Beta rhythms were extracted.From the extracted features, 80% of the data is used for training Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM) and 20% for testing classifier models.Both the models performed well on the test data and the results obtained had a highest accuracy 91.72.7 for subject-9 and these results were compared with the existing methods.The obtained results of MI EEG signals classification using BCI holds potential for revolutionizing assistive technology, stroke rehabilitation, virtual reality gaming,

One of the essential challenges in brain–computer interface is to classify motor imagery (MI) signals. In this paper, an ensemble SVM-based voting system is proposed. In each line of this system, the EEG signal is transformed into different representations based on discrete cosine transform, Fourier transform, common spatial pattern, and empirical mode decomposition, and then these representations are combined in a triple-frame matrix. These frames are fed into a pre-trained deep convolutional neural network as a feature extractor. For each line, an SVM is employed to classify the extracted feature vectors. Finally, a decision is made based on voting between these SVMs. Performance of the proposed method is examined on the BCI Competition III dataset Iva to separate right hand and foot movement imagery. The simple proposed method achieves the average accuracy of 96.34% for all of the subjects, and 99.70% for the best situation that is an improvement in MI classification. In addition, it can be seen that right side of the brain is more effective than the other side in EEG-based MI classification.

Motor imagery (MI)-based brain-computer interface (BCI) systems are being increasingly employed to provide alternative means of communication and control for people suffering from neuro-motor impairments, with a special effort to bring these systems out of the controlled lab environments. Hence, accurately classifying MI from brain signals, e.g., from electroencephalography (EEG), is essential to obtain reliable BCI systems. However, MI classification is still a challenging task, because the signals are characterized by poor SNR, high intra-subject and cross-subject variability. Deep learning approaches have started to emerge as valid alternatives to standard machine learning techniques, e.g., filter bank common spatial pattern (FBCSP), to extract subject-independent features and to increase the cross-subject classification performance of MI BCI systems. In this paper, we first present a review of the most recent studies using deep learning for MI classification, with particular attention to their cross-subject performance. Second, we propose DynamicNet, a Python-based tool for quick and flexible implementations of deep learning models based on convolutional neural networks. We show-case the potentiality of DynamicNet by implementing EEGNet, a well-established architecture for effective EEG classification. Finally, we compare its performance with FBCSP in a 4-class MI classification over public datasets. To explore its cross-subject classification ability, we applied three different cross-validation schemes. From our results, we demonstrate that DynamicNet-implemented EEGNet outperforms FBCSP by about 25%, with a statistically significant difference when cross-subject validation schemes are applied.

Objective. Classification of electroencephalography (EEG)-based motor imagery (MI) is a crucial non-invasive application in brain–computer interface (BCI) research. This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based MI classification that outperforms the state-of-the-art methods. Approach. The proposed CNN model, namely EEG-inception, is built on the backbone of the inception-time network, which has showed to be highly efficient and accurate for time-series classification. Also, the proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing. Furthermore, this paper proposes a novel data augmentation method for EEG signals to enhance the accuracy, at least by 3%, and reduce overfitting with limited BCI datasets. Main results. The proposed model outperforms all state-of-the-art methods by achieving the average accuracy of 88.4% and 88.6% on the 2008 BCI Competition IV 2a (four-classes) and 2b datasets (binary-classes), respectively. Furthermore, it takes less than 0.025 s to test a sample suitable for real-time processing. Moreover, the classification standard deviation for nine different subjects achieves the lowest value of 5.5 for the 2b dataset and 7.1 for the 2a dataset, which validates that the proposed method is highly robust. Significance. From the experiment results, it can be inferred that the EEG-inception network exhibits a strong potential as a subject-independent classifier for EEG-based MI tasks.

Segment alignment based cross-subject motor imagery classification under fading data

Electroencephalography (EEG) signals have been using for brain-computer interface applications for the last two decades. Motor imagery (MI) signals are one of the EEG signal types formed by imagining a limb's movement. Recently with the help of deep neural networks (DNN) for classifying MI signals using time-frequency (TF) features, considerable performance improvement has been reported. This paper proposes using a well-known TF representation technique called Constant-Q Transform (CQT) for the MI signal classification. Experiments conducted on BCI IV 2b dataset with DNN classifier using CQT spectrogram show that CQT outperforms traditional short-time Fourier transform (STFT) representation.

The brain-computer interface (BCI) work is to provide humans an alternative channel that allows direct transmission of messages from the brain by analyzing the brain’s mental activities [1–7]. The brain activity is recorded by means of multi-electrode electroencephalographic (EEG) signals that are either invasive or noninvasive. Noninvasive recording is convenient and popular in BCI applications so it is commonly used. According to the definition suggested at the first international meeting for BCI technology, the term BCI is reserved for a system that must not depend on the brain’s normal output pathways of peripheral nerves and muscles [2]. It has become popular for BCI systems on motor imagery (MI) EEG signals in the last decade [8]. It reveals that there are special characteristics of event-related desynchronization (ERD) and synchronization (ERS) in mu and beta rhythms over the sensorimotor cortex during MI tasks by discriminating EEG signals between left and right MIs [9, 10]. ERD/ERS is the task-related or event-related change in the amplitude of the oscillatory behavior of specific cortical areas within various frequency bands. An amplitude (or power) increase is defined as event-related synchronization while an amplitude (or power) decrease is defined as event-related desynchronization. As other event-related potentials, ERD/ERS patterns are associated with sensory processing and motor behavior [2]. The principal objective of this study is to propose a BCI system, which combines neuro-fuzzy prediction and multiresolution fractal feature vectors (MFFVs) with support vector machine, for MI classification.

Brain-Computer Interfaces (BCI) based on Electroencephalography (EEG) signals, in particular motor imagery (MI) data have received a lot of attention and show the potential towards the design of key technologies both in healthcare and other industries. MI data is generated when a subject imagines movement of limbs and can be used to aid rehabilitation as well as in autonomous driving scenarios. Thus, classification of MI signals is vital for EEG-based BCI systems. Recently, MI EEG classification techniques using deep learning have shown improved performance over conventional techniques. However, due to inter-subject variability, the scarcity of unseen subject data, and low signal-to-noise ratio, extracting robust features and improving accuracy is still challenging. In this context, we propose a novel two-way few shot network that is able to efficiently learn how to learn representative features of unseen subject categories and how to classify them with limited MI EEG data. The pipeline includes an embedding module that learns feature representations from a set of samples, an attention mechanism for key signal feature discovery, and a relation module for final classification based on relation scores between a support set and a query signal. In addition to the unified learning of feature similarity and a few shot classifier, our method leads to emphasize informative features in support data relevant to the query data, which generalizes better on unseen subjects. For evaluation, we used the BCI competition IV 2b dataset and achieved an 9.3% accuracy improvement in the 20-shot classification task with state-of-the-art performance. Experimental results demonstrate the effectiveness of employing attention and the overall generality of our method.

The accuracy of vibration-based structural damage identification is affected by uncertainties in vibration measurements and finite element modeling. Moreover, a sufficient number of sensor measurements are not practical in large-scale structures. In this regard, the frequency domain [frequencies and mode shapes (FMS)] and time domain [acceleration cross-correlation function (ACCF)] indexes are put into a parallel convolutional neural network (P-CNN, a new convolutional neural network architecture with dual-channel) to locate and quantify structural damage. First, this approach is verified by a numerical model of a simply-supported beam, and the performance is evaluated by comparing it with methods using FMS and ACCF indexes input to the conventional 2D-CNN, respectively. The comparative results demonstrate that the proposed method can identify the damage with the lowest error, and the errors are less than 5%. In addition, three sparse measurement conditions are adopted to investigate the effect of the number of measuring points on damage identification. It is found that under the influence of uncertainties, even if only four sensors are used, this approach can identify damage accurately. Second, an engineering example of a continuous rigid frame bridge is adopted to validate the feasibility of the proposed method. The results show that when the number of sensors accounts for 65% and 47% of test sensors, respectively, it performs well in locating and quantifying structural damage. Therefore, this method can not only reduce the number of sensors and eliminate uncertainties effects, but also improve the performance of the 2D-CNN through information fusion and complementarity, which is also beneficial to minimize the cost of sensor layout in actual structures.

In the domain of chemometrics, multiblock data analysis is widely performed for exploring or fusing data from multiple sources. Commonly used methods for multiblock predictive analysis are the extensions of latent space modelling approaches. However, recently, deep learning (DL) approaches such as convolutional neural networks (CNNs) have outperformed the single block traditional latent space modelling chemometric approaches such as partial least-square (PLS) regression. The CNNs based DL modelling can also be performed to simultaneously deal with the multiblock data but was never explored until this study. Hence, this study for the first time presents the concept of parallel input CNNs based DL modelling for multiblock predictive chemometric analysis. The parallel input CNNs based DL modelling utilizes individual convolutional layers for each data block to extract key features that are later combined and passed to a regression module composed of fully connected layers. The method was tested on a real visible and near-infrared (Vis-NIR) large data set related to dry matter prediction in mango fruit. To have the multiblock data, the visible (Vis) and near-infrared (NIR) parts were treated as two separate blocks. The performance of the parallel input CNN was compared with the traditional single block CNNs based DL modelling, as well as with a commonly used multiblock chemometric approach called sequentially orthogonalized partial least-square (SO-PLS) regression. The results showed that the proposed parallel input CNNs based deep multiblock analysis outperformed the single block CNNs based DL modelling and the SO-PLS regression analysis. The root means squared errors of prediction obtained with deep multiblock analysis was 0.818%, relatively lower by 4 and 20% than single block CNNs and SO-PLS regression, respectively. Furthermore, the deep multiblock approach attained ∼3% lower RMSE compared to the best known on the mango data set used for this study. The deep multiblock analysis approach based on parallel input CNNs could be considered as a useful tool for fusing data from multiple sources.

Motor Imagery-based Brain-computer interface (MI-BCI) enables communications between the human brain and external machines, which is considered a valuable frontier technology. Motor imagery (MI) identification is a fundamental problem for these applications. However, because of the non-stationary and nonlinear properties of EEG motor imagery signals, it is very hard to get a high accuracy multi-class method. In this study, a multi-class classification method of MI EEG signals using wavelet packets and a one-dimensional convolutional neural network is presented. Data normalization and wavelet threshold denoising were performed for signal preprocessing. Then EEG signals were decomposed and reconstructed by wavelet packets transform. According to the features between different channels of EEG signal, a multilayer one-dimensional convolutional neural network model was adopted to get a multi-class classification model for MI. This model was validated on the public datasets BCI competition IV 2a. The result shows that the mean accuracy of this model is 93.05%. This study is beneficial to MI-BCI research applications in the future.