4-Class MI-EEG Signal Generation and Recognition with CVAE-GAN

Evolutionary conditional GANs for supervised data augmentation: The case of assessing berry number per cluster in grapevine

Recent advancements in signal processing techniques have enabled non-invasive Brain-Computer Interfaces (BCIs) to control assistive devices, like robotic arms, directly with users' EEG signals. However, the applications of these systems are currently limited by the low signal-to-noise ratio and spatial resolution of EEG from which brain intention is decoded. In this study, we propose a motor-imagery (MI) paradigm, inspired by the mechanisms of a computer mouse, that adds an additional "click" signal to an established 2D movement BCI paradigm. The additional output signal increases the degrees of freedom of the BCI system and may enable more complex tasks. We evaluated this paradigm using deep learning (DL) based signal processing on both healthy subjects and stroke-survivors in online BCI tasks derived from two potential applications: clicking on virtual targets and moving physical objects with a robotic arm in a continuous reach-and-grasp task. The results show that subjects were able to control both movement and clicking simultaneously to grab, move, and place up to an average of 7 cups in a 5-minute run using the robotic arm. The proposed paradigm provides an additional degree of freedom to EEG BCIs, and improves upon existing systems by enabling continuous control of reach-and-grasp tasks instead of selecting from a discrete list of predetermined actions. The tasks studied in these experiments show BCIs may be used to control computer cursors or robotic arms for complex real-world or clinical applications in the near future, potentially improving the lives of both healthy individuals and motor-impaired patients.

BackgroundA novel artefact removal algorithm is proposed for a self-paced hybrid brain-computer interface (BCI) system. This hybrid system combines a self-paced BCI with an eye-tracker to operate a virtual keyboard. To select a letter, the user must gaze at the target for at least a specific period of time (dwell time) and then activate the BCI by performing a mental task. Unfortunately, electroencephalogram (EEG) signals are often contaminated with artefacts. Artefacts change the quality of EEG signals and subsequently degrade the BCI’s performance.MethodsTo remove artefacts in EEG signals, the proposed algorithm uses the stationary wavelet transform combined with a new adaptive thresholding mechanism. To evaluate the performance of the proposed algorithm and other artefact handling/removal methods, semi-simulated EEG signals (i.e., real EEG signals mixed with simulated artefacts) and real EEG signals obtained from seven participants are used. For real EEG signals, the hybrid BCI system’s performance is evaluated in an online-like manner, i.e., using the continuous data from the last session as in a real-time environment.ResultsWith semi-simulated EEG signals, we show that the proposed algorithm achieves lower signal distortion in both time and frequency domains. With real EEG signals, we demonstrate that for dwell time of 0.0s, the number of false-positives/minute is 2 and the true positive rate (TPR) achieved by the proposed algorithm is 44.7%, which is more than 15.0% higher compared to other state-of-the-art artefact handling methods. As dwell time increases to 1.0s, the TPR increases to 73.1%.ConclusionsThe proposed artefact removal algorithm greatly improves the BCI’s performance. It also has the following advantages: a) it does not require additional electrooculogram/electromyogram channels, long data segments or a large number of EEG channels, b) it allows real-time processing, and c) it reduces signal distortion.

34 - Electroencephalogram-based brain–computer interface systems for controlling rehabilitative devices

We present a novel integration of a brain-computer interface (BCI) with a multi-touch surface. BCIs based on the P300 paradigm often use a visual stimulus of a flashing character to elicit an event related potential in the brain's EEG signal. Traditionally, P300-based BCI paradigms use a grid layout of visual targets, commonly an alphabet, and allow users to select targets using their thoughts. In our new system a multi-touch table senses objects placed upon its surface and the system can highlight the objects on the table by flashing an area of light around them. This allows us to construct a P300-based BCI that uses a user-assembled collection of objects as targets, rather than a pre-determined grid layout. An experiment shows that our new paradigm works just as well as the traditional paradigms, thus highlighting the potential for BCIs to be integrated in a broader range of situations.

EEG signal classification is an important task to build an accurate Brain Computer Interface (BCI) system. Many machine learning and deep learning approaches have been used to classify EEG signals. Besides, many studies have involved the time and frequency domain features to classify EEG signals. On the other hand, a very limited number of studies combine the spatial and temporal dimensions of the EEG signal. Brain dynamics are very complex across different mental tasks, thus it is difficult to design efficient algorithms with features based on prior knowledge. Therefore, in this study, we utilized the 2D AlexNet Convolutional Neural Network (CNN) to learn EEG features across different mental tasks without prior knowledge. First, this study adds spatial and temporal dimensions of EEG signals to a 2D EEG topographic map. Second, topographic maps at different time indices were cascaded to populate a 2D image for a given time window. Finally, the topographic maps enabled the AlexNet to learn features from the spatial and temporal dimensions of the brain signals. The classification performance was obtained by the proposed method on a multiclass dataset from BCI Competition IV dataset 2a. The proposed system obtained an average classification accuracy of 81.09%, outperforming the previous state-of-the-art methods by a margin of 4% for the same dataset. The results showed that converting the EEG classification problem from a (1D) time series to a (2D) image classification problem improves the classification accuracy for BCI systems. Also, our EEG topographic maps enabled CNN to learn subtle features from spatial and temporal dimensions, which better represent mental tasks than individual time or frequency domain features.

Brain-Computer Interfaces (BCIs) offer transformative potential in rehabilitation, enabling patients to drive their recovery through their own thoughts and efforts rather than relying on pre-programmed machine outputs. Electroencephalography (EEG) signals, due to their non-invasive nature and high temporal resolution, are the most widely used in BCIs. Recent advancements in deep learning have significantly improved BCI decoding performance for broader movements such as arm and leg motions. However, to achieve comprehensive rehabilitation outcomes, a greater emphasis on decoding finer movements, such as individual finger motions, is essential for restoring intricate motor functions. Current BCI systems face significant barriers to mainstream adoption in healthcare rehabilitation. A major challenge lies in overcoming the high noise-to-signal ratio inherent in EEG signals, which complicates the development of robust models capable of accurately decoding user intentions. Additionally, decoding finer movement intentions remains underexplored, presenting further difficulties due to the subtle and overlapping differences in brain signal features associated with these movements. The lack of standardized protocols and the variability of EEG signals across different sessions and individuals further complicate this challenge. To address these challenges, various methodologies such as transfer learning and representation learning have been proposed. However, these methods often rely on the assumption of a single distribution to represent latent features, which may not generalize well across different subjects due to temporal variations in EEG signals, leading to information loss. Furthermore, transfer learning requires a substantial amount of labeled target data, which can be difficult to obtain in EEG-based rehabilitation contexts. This issue is compounded by the need for extensive calibration sessions to tailor the BCI system to each individual user. In this research, we demonstrate that techniques from other deep learning domains, such as image and natural language classification, can be adapted to enhance EEG decoding. For example, employing meta-learning principles in training subject-independent EEG classifiers enables deep learning models to achieve superior decoding performance with fewer target subject data. Meta-learning allows the model to quickly adapt to new subjects by leveraging prior knowledge, thus reducing the need for large amounts of subject-specific data. Eventually, we also show that utilizing contrastive learning techniques allows for improved performance, removing the need for labeled adaptation data and facilitating easier implementation in real-world rehabilitation settings. Finally, we also explore the application of self-supervised transformer-attention networks to enhance decoding performance in smaller datasets and extend this approach to decode finer motor attempts, such as individual finger movements, thereby advancing the precision of BCIs in restoring detailed motor functions. The applications of this research are extensive and impactful. By advancing the precision and practicality of EEG-based BCI systems, we can significantly enhance personalized rehabilitation programs, making them more effective and accessible. Focusing on patient-driven rehabilitation, patients are able to have greater ownership of their own recovery process and the positive feedback loop of their own efforts in improving motor function would lead to greater recovery outcomes. This technology can be applied across various therapeutic settings, including stroke recovery, neurodegenerative disease management, and motor function rehabilitation for patients with spinal cord injuries or limb amputations. Furthermore, the principles and methodologies developed here have broader implications for other domains requiring precise brain-signal decoding, such as neurofeedback therapy, cognitive enhancement, and brain-controlled prosthetics. This research paves the way for BCIs to play a crucial role in improving quality of life and healthcare outcomes, heralding a new era of patient-centered therapeutic interventions.

Surface defect recognition (SDC) is essential in intelligent manufacturing. Deep learning (DL) is a research hotspot in SDC. Limited defective samples are available in most real-world cases, which poses challenges for DL methods. Given such circumstances, generating defective samples by generative adversarial networks (GANs) is applied. However, insufficient samples and high-frequency texture details in defects make GANs very hard to train, yield mode collapse, and poor image quality, which can further impact SDC. To solve these problems, this article proposes a new GAN called contrastive GAN, which can be trained to generate diverse defects with only extremely limited samples. Specifically, a shared data augmentation (SDA) module is proposed for avoiding overfitting. Then, a feature attention matching (FAM) module is proposed to align features for improving the quality of generated images. Finally, a contrastive loss based on hypersphere is employed to constrain GANs to generate images that differ from the traditional transform. Experiments show that the proposed GAN generates defective images with higher quality and lower variance between real defects compared to other GANs. Synthetic images contribute to pretrained DL networks with accuracies of up to 95.00%–99.56% for Northeastern University (NEU) datasets of different sizes and 91.84% for printed circuit board (PCB) cases, which proves the effectiveness of the proposed method.

Emotion classification using EEG (Electroencephalogram) brain signals has recently garnered significant interest due to its potential applications in fields such as mental health, human-computer interaction, and affective computing. The three prominent recurrent neural network (RNN) architectures GRU (Gated Recurrent Unit), LSTM (Long Short-Term Memory), and BiLSTM (Bidirectional Long Short-Term Memory) are thoroughly compared in this study with a focus on classifying emotions. The study examines how well these RNN variations capture temporal dependencies and patterns within EEG data to precisely discriminate between various emotional states. Based on classification accuracy and training effectiveness, these model's performance will be assessed. The results will reveal the most suitable architecture in terms of accuracy and robustness for EEG-based emotion classification. The findings provide valuable insights into choosing the appropriate RNN architecture for emotion classification tasks, advancing the understanding of AI's potential in interpreting human emotions through EEG signals.

Speech impairments, resulting from brain injuries, mental disorders, or vocal abuse, substantially affect an individual’s quality of life and can lead to social isolation. Brain–Computer Interfaces (BCIs), particularly those based on EEG, offer a promising support mechanism by harnessing brain signals. Owing to their clinical efficacy, cost-effective EEG devices, and expanding applications in the medical and social spheres, their usage has surged. This study introduces an ensemble-based feature engineering mechanism to pinpoint the optimal brain rhythm, channel subset, and feature set for accurately predicting imagined words from EEG signals via machine learning models. Leveraging the 2020 International BCI competition dataset, we employed bandpass filtering, channel wrapping, and ranking methods were applied to discern suitable brain rhythms and features associated with imagined speech. Subsequent application of kernel-based principal component analysis enabled us to compress the feature space dimensionality. We then trained various machine learning models, among which the kNN model excelled, achieving an average accuracy of 73% in a 10-fold cross-validation scheme ,surpassing 18% higher than the existing literature. The Gamma rhythm was identified as the most predictive of imagined speech from EEG brain signals. These advancements herald a new era of more precise and effective BCIs, poised to significantly improve the lives of those with speech impairments.

Deep Learning algorithms have almost become a key standard for majority of vision and machine learning problems. Despite its common usage and high performance for many applications, they have certain disadvantages. One major problem with deep learning methods is the size of the dataset to be used for training. The methods require a large dataset for an adequate training. However, a large dataset may not be available for all problems. In such a case, researchers apply data augmentation methods to obtain a larger dataset from a given dataset. For the image classification problem, conventional method for data augmentation is the application of transformation based methods; such as flipping, rotation, blurring etc. Recently, generative models which apply deep learning methods are also commonly used for data augmentation. On the other hand, in case of a too large dataset the classifiers may overfit and end up with a lack of generalization. In this study, we explore the usage of generative adversarial networks for data augmentation in the image classification problem. We evaluate the classification performance with three types of augmentation methods. Dataset is first augmented by two conventional methods; Gaussian blurring and dropout of regions, then by generative adversarial networks. Meanwhile, we observe the behavior of the classifier for various sized datasets with and without data augmentation. We observe that especially in datasets of certain sizes generative adversarial networks can be effectively used for data augmentation.

The modern commercial industry is increasingly dependent on the analysis of vast data records. Analysing big electroencephalogram (EEG) signal data (called brain signal data) plays an important role in a wide variety of applications such as healthcare practices, brain computer interface (BCI) systems, innovative education, privacy and security, and biometrics. The key objective of this paper is to establish a methodological framework for identifying communicative intentions of motor disabled people from EEG data for application in BCI systems. The proposed framework is designed based on common spatial pattern (CSP) data method and optimized ensemble (OE) machine learning method for the application of BCI technologies. The CSP method is used for discovering important features from EEG data and finally the extracted features are fed as an input to optimized ensemble (OE) method. The proposed method was tested on BCI Competition III dataset IVa, which contains motor imagery-based EEG signal data. The experimental results show that our proposed method can handle brain signal big data for identifying communicative intentions for an advanced BCI system. We compared the performance of our proposed method with several other existing methods. In comparison with other established methods, our method achieves higher classification accuracy performance. This research assists the experts in processing and analysing EEG signals for BCI applications. It also supports technologists to create a new EEG data analyser for BCI systems.KeywordsElectroencephalographyBrain computer interfaceCommon spatial patternMachine learning

Recent advances in deep learning have had a methodological and practical impact on brain-computer interface (BCI) research. Among the various deep network architectures, convolutional neural networks (CNNs) have been well suited for spatio-spectral-temporal electroencephalogram (EEG) signal representation learning. Most of the existing CNN-based methods described in the literature extract features at a sequential level of abstraction with repetitive nonlinear operations and involve densely connected layers for classification. However, studies in neurophysiology have revealed that EEG signals carry information in different ranges of frequency components. To better reflect these multi-frequency properties in EEGs, we propose a novel deep multi-scale neural network that discovers feature representations in multiple frequency/time ranges and extracts relationships among electrodes, i.e., spatial representations, for subject intention/condition identification. Furthermore, by completely representing EEG signals with spatio-spectral-temporal information, the proposed method can be utilized for diverse paradigms in both active and passive BCIs, contrary to existing methods that are primarily focused on single-paradigm BCIs. To demonstrate the validity of our proposed method, we conducted experiments on various paradigms of active/passive BCI datasets. Our experimental results demonstrated that the proposed method achieved performance improvements when judged against comparable state-of-the-art methods. Additionally, we analyzed the proposed method using different techniques, such as PSD curves and relevance score inspection to validate the multi-scale EEG signal information capturing ability, activation pattern maps for investigating the learned spatial filters, and t-SNE plotting for visualizing represented features. Finally, we also demonstrated our method's application to real-world problems. Based on our experimental results and analyses, we believe that the proposed multi-scale neural network can be useful for various BCI paradigms, as a starting model or as a backbone network in any new BCI experiments.

One of the main problems in Brain Computer Interface (BCI) systems is the non-stationary behavior of the electroencephalography (EEG) signals causing problems in real time applications. Another common problem in BCI systems is the situation where the labeled data are scarce. In this study, we take a semi-supervised learning perspective and propose solving both types of problems by updating the BCI system with labels obtained from the outputs of the classifier. To test the approach, data from motor imagery BCI system are used. Attributes extracted from EEG signals are classified with Linear Discriminant Analysis (LDA) and Support Vector Machines (SVM). With respect to the static classifiers, accuracy was improved approximately 4% using the proposed adaptation approach in the case of a training dataset. Even though the difference between the performance of static and adaptive classifiers decreases as the size of training data increases, the accuracy of our proposed adaptive classifier remains higher. The proposed approach has also improved the performance of a BCI system around 4% in the case of non-stationary signals as well.

For developing brain computer interface (BCI) applications, electroencephalography (EEG) is the most widely used measurement method due to its noninvasiveness, high temporal resolution, and portability. EEG signal contains sufficient neural information about each human task, which makes the extracting, and decoding of each task-related information is still challenging, especially to improve the existing BCI performances. In this paper, we present a comparison analysis to find the most relevant features and the most suitable classification method for decoding motor imagery for EEG-based BCI. Therefore, some signal processing and machine learning techniques have applied for features extraction and classification phases. For the decomposition of EEG signal, we used three type of features [EEG signal mean, root mean square (RMS) and Relative of band power (RBP)]. In addition, we investigated an analytical comparison between three methods of classification [Support Vector Machine (SVM), Linear Discriminant Analysis and K-Nearest Neighbors]. The methods were validated using a publicly available dataset (BCI Competition IV-III-a) to discriminate between two mental states (right and left hand movements) using 10-fold cross-validation. SVM method gave better classification accuracy of 76.4% using relative band powers as potential EEG features.