Flexible Patched Brain Transformer model for EEG decoding

In recent years, deep-learning-based motor imagery (MI) electroencephalography (EEG) decoding methods have shown great potential in the field of the brain–computer interface (BCI). The existing literature is relatively mature in decoding methods for two classes of MI tasks. However, with the increase in MI task classes, decoding studies for four classes of MI tasks need to be further explored. In addition, it is difficult to obtain large-scale EEG datasets. When the training data are limited, deep-learning-based decoding models are prone to problems such as overfitting and poor robustness. In this study, we design a data augmentation method for MI-EEG. The original EEG is slid along the time axis and reconstructed to expand the size of the dataset. Second, we combine the gated recurrent unit (GRU) and convolutional neural network (CNN) to construct a parallel-structured feature fusion network to decode four classes of MI tasks. The parallel structure can avoid temporal, frequency and spatial features interfering with each other. Experimenting on the well-known four-class MI dataset BCI Competition IV 2a shows a global average classification accuracy of 80.7% and a kappa value of 0.74. The proposed method improves the robustness of deep learning to decode small-scale EEG datasets and alleviates the overfitting phenomenon caused by insufficient data. The method can be applied to BCI systems with a small amount of daily recorded data.

A robust decoding model that can efficiently deal with the subject and period variation is urgently needed to apply the brain-computer interface (BCI) system. The performance of most electroencephalogram (EEG) decoding models depends on the characteristics of specific subjects and periods, which require calibration and training with annotated data prior to application. However, this situation will become unacceptable as it would be difficult for subjects to collect data for an extended period, especially in the rehabilitation process of disability based on motor imagery (MI). To address this issue, we propose an unsupervised domain adaptation framework called iterative self-training multisubject domain adaptation (ISMDA) that focuses on the offline MI task. First, the feature extractor is purposefully designed to map the EEG to a latent space of discriminative representations. Second, the attention module based on dynamic transfer matches the source domain and target domain samples with a higher coincidence degree in latent space. Then, an independent classifier oriented to the target domain is employed in the first stage of the iterative training process to cluster the samples of the target domain through similarity. Finally, a pseudolabel algorithm based on certainty and confidence is employed in the second stage of the iterative training process to adequately calibrate the error between prediction and empirical probabilities. To evaluate the effectiveness of the model, extensive testing has been performed on three publicly available MI datasets, the BCI IV IIa, the High gamma dataset, and Kwon et al. datasets. The proposed method achieved 69.51%, 82.38%, and 90.98% cross-subject classification accuracy on the three datasets, which outperforms the current state-of-the-art offline algorithms. Meanwhile, all results demonstrated that the proposed method could address the main challenges of the offline MI paradigm.

Data augmentation (DA) increases the diversity of training data to improve the model generalization ability. Most of the DA methods focus on image classification or object detection. Directly using the existing DA methods on change detection tasks not only falls short of fully exploring the specificity of the change image pairs but also leads to longer training times. In this article, we first propose a mask-guided mixing (MGM) DA for change detection, which mixes the change regions of the current training sample based on prediction results and labels to generate high-quality samples with more positive samples. We then propose a new reinforcement learning (RL)-based Adaptive DA method, AdaAug+, to adaptively select the optimal DA policy for the training samples. An actor selects the best augmentation operation from the operation set according to the image pair. The augmented image pairs make it easier for the change detector to learn the optimal parameters and improve the final detection performance. To reduce the training time, we identify and remove the redundant training samples during the training process by our redundancy searching policy. We have conducted various experiments on four remote sensing change detection datasets with different change detectors. The experimental results demonstrate that AdaAug+ achieves promising performance compared to the state-of-the-art DA methods and requires less training time.

Data augmentation (DA) is widely employed to improve the generalization performance of deep models. However, most existing DA methods employ augmentation operations with fixed or random magnitudes throughout the training process. While this fosters data diversity, it can also inevitably introduce uncontrolled variability in augmented data, which could potentially cause misalignment with the evolving training status of the target models. Both theoretical and empirical findings suggest that this misalignment increases the risks of both underfitting and overfitting. To address these limitations, we propose AdaAugment, an innovative and tuning-free adaptive augmentation method that leverages reinforcement learning to dynamically and adaptively adjust augmentation magnitudes for individual training samples based on real-time feedback from the target network. Specifically, AdaAugment features a dual-model architecture consisting of a policy network and a target network, which are jointly optimized to adapt augmentation magnitudes in accordance with the model's training progress effectively. The policy network optimizes the variability within the augmented data, while the target network utilizes the adaptively augmented samples for training. These two networks are jointly optimized and mutually reinforce each other. Extensive experiments across benchmark datasets and deep architectures demonstrate that AdaAugment consistently outperforms other state-of-the-art DA methods in effectiveness while maintaining remarkable efficiency. Code is available at https://github.com/Jackbrocp/AdaAugment.

- The Plant Disease Detection and Classification project is designed to help farmers identify crop diseases at an early stage using images of plant leaves. By applying advanced machine learning (ML) and deep learning (DL) techniques, this system aims to automatically detect and classify various plant diseases with high accuracy. Early identification of plant diseases plays a vital role in improving crop yield, reducing unnecessary pesticide usage, and encouraging sustainable farming practices. Once a disease is detected, the system not only identifies it but also provides a brief description and recommends appropriate prevention and treatment methods. To enhance performance, the project uses data augmentation and normalization techniques—helping the model handle variations in lighting, angles, and environmental conditions. The system is trained on a diverse dataset containing images of both healthy and diseased leaves from different plant species, categorized into 38 different classes. The project incorporates various machine learning algorithms such as Support Vector Machines (SVM), K-Nearest Neighbors (KNN), Decision Trees, and Random Forests, as well as a Convolutional Neural Network (CNN) for deep learning. Among these, CNNs have shown outstanding performance due to their ability to learn spatial features from images effectively. While existing research has shown promising results, real-world challenges such as varying image quality, lighting conditions, and background clutter still affect accuracy. This project aims to overcome these issues by training on large, diverse datasets and fine-tuning model parameters. Ultimately, this project hopes to empower farmers with an intelligent tool to monitor crop health, make informed decisions, and contribute to food security through precision agriculture. Keywords: Plant Disease Detection, Automated Disease Detection System, Machine Learning, Deep Learning, Data Augmentation, Data Normalization, Support Vector Machine, KNN Classifier, Convolutional Neural Network, Decision Trees, Random Forest

In recent years, machine learning algorithms have been developing rapidly, becoming increasingly powerful tools in decoding physiological and neural signals. The aim of this dissertation is to develop computational tools, and especially machine learning techniques, to identify the most effective methods for feature extraction and classification of these signals. This is particularly challenging due to the highly non-linear, non-stationery, and artifact- and noise-prone nature of these signals. Among basic human-control tasks, reaching and grasping are ubiquitous in everyday life. I investigated different linear and non-linear dimensionality reduction techniques for feature extraction and classification of electromyography (EMG) during a reach-grasp-lift task. The results highlighted the advantages of completely automated feature-learning by Laplacian Eigenmaps over manual feature engineering, especially when combined with classification, to achieve high accuracy with few training samples. The ability to decode and reduce the complexity of EMG could enable new practical applications for EMG in basic science and in the clinic. It could also help design humanoid and other robots. Beyond EMG, a key objective of my dissertation was to decode brain activity during the decision-making processes that lead to voluntary action. This was based on electroencephalography (EEG) and holds the promise for improved brain-computer interfaces (BCIs), particularly related to motor imagery (MI). We developed an end-to-end convolutional neural network with attentional mechanism together with different data augmentation techniques on two benchmark MI datasets. I also collected a new dataset, recorded using high-density EEG, and containing both MI and motor execution (ME) tasks. This enabled us to directly compare the decodability of MI and ME, investigate optimal channel configurations, and much more. In particular, this facilitates the analysis and decoding of MI on the fly—online and in in real time. Another potential use of EEG is measuring brain activity of people who are floating in body-temperature water in a sensory-deprivation-tank float pod. We compared lying in the float pod versus lying in bed (a control condition). And we found differences between the two, especially in the gamma band. More research is required to understand what these findings mean for levels of stress in the float pod.

The early and accurate detection of autism spectrum disorder (ASD) is crucial for timely interventions that can significantly improve the quality of life for individuals on the spectrum. Despite the importance of early diagnosis, current ASD diagnostic methods face several challenges, including being time-consuming, subjective, and requiring specialized expertise, which limits their accessibility and scalability. Addressing these limitations, automated ASD detection through facial image analysis offers a non-invasive, efficient, and scalable alternative. However, existing machine learning and deep learning techniques frequently face challenges such as limited generalizability, inadequate interpretability, and insufficient performance on diverse datasets. This study introduces an effective deep learning framework for automated ASD detection that leverages pre-trained convolutional neural networks (CNNs), including VGG16, VGG19, InceptionV3, VGGFace, and MobileNet. The proposed framework integrates advanced preprocessing techniques, data augmentation, and Explainable AI (XAI) methods, such as Local Interpretable Model-agnostic Explanations (LIME), to enhance both accuracy and interpretability. The experimental results demonstrate the effectiveness of the proposed framework, with the VGG19 model achieving an accuracy of 98.2%, outperforming many state-of-the-art methods. This work represents a significant step forward in automated ASD diagnostics, offering a reliable, efficient, and interpretable solution that can aid clinicians in making timely and accurate diagnoses.

Online semi-supervised learning for motor imagery EEG classification

Common spatial patterns (CSP) is a successful algorithm in motor imagery (MI) based brain-computer interfaces (BCI). However, the performance of the algorithm in electroencephalography (EEG) classification depends largely on the subject's specific frequency band. In this paper, we proposed a new wavelet packet coefficients weighted CSP (WPW-CSP) algorithm to optimize the frequency band of the specific subject for motor imagery (MI) classification. The algorithm is applied to 5 MI data sets of two classes. The results suggest that the proposed algorithm achieves a 4% increase of classification accuracy in off-line analysis, compared to the original CSP algorithm using 8–30 Hz wide band.

The problem of fault classification in industry has been studied extensively. Most classification algorithms are modeled on the premise of data balance. However, the difficulty of collecting industrial data in different modes is quite different. This inevitably leads to data imbalance, which will adversely affect the fault classification performance. This article proposes a novel data augmentation classifier (DAC) for imbalanced fault classification. Data augmentation based on generative adversarial networks (GANs) is an effective way to solve the problem of unbalanced classification. However, the randomness of the GAN generation process restricts the effect of data enhancement. DAC proposes a data selection strategy based on data filtering and data purification in model training to solve this problem. In addition, DAC combines supervised learning and data generation processes to obtain an end-to-end model. Meanwhile, multigenerator structure of DAC (MDAC) is proposed to solve the problem of incomplete learning of a single generator when data imbalances get complicated. The proposed DAC and MDAC are applied in two fault classification cases of the Tennessee Eastman (TE) benchmark process, results of which show superiority of DAC and MDAC compared to existing methods. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Data imbalances are common in fault classification and affect the effectiveness of modeling in industry. As a generative model, generative adversarial networks (GANs) provide new ideas for small-class data augmentation. However, the instability of its training process and the randomness of data generation affect the results of data augmentation. In this article, the GAN generation process is analyzed in detail. The results of the visualization indicate that no data generation was perfect at any one time. Based on the rules of GAN data generation, we propose a data selection strategy during training. High-quality data are selected for data augmentation through data filtering and data purification. Apart from this, we combine the training process of GAN and classification model for imbalanced data to reduce modeling time. Through industrial examples, we have evaluated the effectiveness of this method.

Brain-Computer Interface (BCI) is a system that measures and processes the activity of the human brain to improve or replace the function of the human body. In the future, this system can be a solution for people with disabilities, especially in locomotor organs such as hands and feet. The purpose of this research is to classify four classes of electroencephalogram (EEG) signals that represent four human motor imagery. The four motor imageries are left-hand, right-hand, left-foot, and right-foot that originated from motor imagery dataset. The proposed method in this research consists of filtering, feature extraction, and classification. The proposed method employed the Finite Impulse Response (FIR) in the filtering process to pass the required EEG signals such as delta, theta, alpha, beta, and gamma channels. The features are the Root Mean Square (RMS) values from the time domain filtered signal. Our system design used these features as input classification method that used the Artificial Neural Network (ANN). The training and testing data separation used 10-fold cross-validation. To analyze the testing performance used a confusion matrix. Based on the results, the proposed method brings the highest system accuracy as 61.2% on the beta channel.

A motor imagery (MI) brain-computer interface (BCI) plays an important role in the neurological rehabilitation training for stroke patients. Electroencephalogram (EEG)-based MI BCI has high temporal resolution, which is convenient for real-time BCI control. Therefore, we focus on EEG-based MI BCI in this paper. The identification of MI EEG signals is always quite challenging. Due to high inter-session/subject variability, each subject should spend long and tedious calibration time in collecting amounts of labeled samples for a subject-specific model. To cope with this problem, we present a supervised selective cross-subject transfer learning (sSCSTL) approach which simultaneously makes use of the labeled samples from target and source subjects based on Riemannian tangent space. Since the covariance matrices representing the multi-channel EEG signals belong to the smooth Riemannian manifold, we perform the Riemannian alignment to make the covariance matrices from different subjects close to each other. Then, all aligned covariance matrices are converted into the Riemannian tangent space features to train a classifier in the Euclidean space. To investigate the role of unlabeled samples, we further propose semi-supervised and unsupervised versions which utilize the total samples and unlabeled samples from target subject, respectively. Sequential forward floating search (SFFS) method is executed for source selection. All our proposed algorithms transfer the labeled samples from most suitable source subjects into the feature space of target subject. Experimental results on two publicly available MI datasets demonstrated that our algorithms outperformed several state-of-the-art algorithms using small number of the labeled samples from target subject, especially for good target subjects.

Classification algorithm for EEG-based motor imagery using hybrid neural network with spatio-temporal convolution and multi-head attention mechanism.

Recent deep neural network models trained on smaller and less diverse datasets use data augmentation to alleviate limitations such as overfitting, reduced robustness, and lower generalization. Methods using 3D datasets are among the most common to use data augmentation techniques such as random point drop, scaling, translation, rotations, and jittering. However, these data augmentation techniques are fixed and are often applied to the entire object, ignoring the object’s local geometry. Different local neighborhoods on the object surface hold a different amount of geometric complexity. Applying the same data augmentation techniques at the object level is less effective in augmenting local neighborhoods with complex structures. This paper presents PatchAugment, a data augmentation framework to apply different augmentation techniques to the local neighborhoods. Our experimental studies on PointNet++ and DGCNN models demonstrate the effectiveness of PatchAugment on the task of 3D Point Cloud Classification. We evaluated our technique against these models using four benchmark datasets, ModelNet40 (synthetic), ModelNetlO (synthetic), SHREC’16 (synthetic) and ScanObjectNN (real-world).

Motor imagery (MI) is a mental process widely utilized as the experimental paradigm for brain-computer interfaces (BCIs) across a broad range of basic science and clinical studies. However, decoding intentions from MI remains challenging due to the inherent complexity of brain patterns relative to the small sample size available for machine learning. This paper proposes an end-to-end Filter-Bank Multiscale Convolutional Neural Network (FBMSNet) for MI classification. A filter bank is first employed to derive a multiview spectral representation of the EEG data. Mixed depthwise convolution is then applied to extract temporal features at multiple scales, followed by spatial filtering to mitigate volume conduction. Finally, with the joint supervision of cross-entropy and center loss, FBMSNet obtains features that maximize interclass dispersion and intraclass compactness. We compare FBMSNet with several state-of-the-art EEG decoding methods on two MI datasets: the BCI Competition IV 2a dataset and the OpenBMI dataset. FBMSNet significantly outperforms the benchmark methods by achieving 79.17% and 70.05% for four-class and two-class hold-out classification accuracy, respectively. These results demonstrate the efficacy of FBMSNet in improving EEG decoding performance toward more robust BCI applications. The FBMSNet source code is available at https://github.com/Want2Vanish/FBMSNet.