Machine-Learning-Based Approach to Decoding Physiological and Neural Signals

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

Due to the volume conduct effect, the spatial resolution of electroencephalography (EEG) is low, leading to the limited motor imagery (MI) decoding accuracy based on the scalp EEG signals. In this work, we propose a new MI decoding method based on the EEG source imaging(ESI). Specifically, we first divide the cortical motor areas into several regions of interest (ROIs) using the multi-source pre-registration method. Subsequently, we employ the ESI algorithms to reconstruct the cortical sources within the motor areas, obtaining much higher spatial resolution information for MI decoding. The common spatial pattern (CSP) is then applied to each ROI. Using cross-validation, twenty ROIs contributing to the MI decoding the most are selected. Finally, the decoding decision is obtained based on the ensemble decision of the selected ROIs. Experiments results based on two public datasets show the superior MI decoding performance in the source space than that in the sensor space.

Studies on the influence of motor imagery (MI) on brain structure and function are limited to traditional imaging techniques and the mechanism for MI therapy is not clear. By observing the brain activation mode during MI and motor execution (ME) in healthy adults, this study aims to use near-infrared brain imaging technology to provide theoretical basis for the treatment of MI. A total of 30 healthy adults recruited to the public from June 2021 to August 2021. The MI and ME of the right knee movement served as the task mode. Block design was repeated 5 times alternately in a 20 s task period and a 30 s resting period. The activation patterns of brain regions were compared between the 2 tasks, and the regression coefficient was calculated to reflect the activation intensity of each brain region by Nirspark and SPSS 23.0 softwares. Lane 2, 3, 4, 5, 7, 9, 19, 20, 21, 24, 25, 26, 27, 32, 33, and 34 were significantly activated during the ME task (P<0.05, corrected by FDR) and lane 2, 5, 9, 16, 27, 29, 33, 34, and 35 were significantly activated during the MI task (P<0.05, corrected by FDR). According to the channel brain region registration information, the brain region activation pattern was similar during both MI and ME tasks in healthy adults, including left primary motor cortex (LM1), left primary sensory cortex (LS1), prefrontal pole, Broca area, and right supramarginal gyrus. Both LM1 and left pre-motor cortex (LPMC) were activated during MI in healthy adults, whereas dorsolateral prefrontal cortex (DLPFC) and only LM1 of the motor region were activated during ME. Compared to MI, the activation intensity of left sensory and left motor cortex was significantly enhanced in ME, and that of left and right prefrontal cortex especially left and right pars triangularis Broca's area (P<0.001, corrected by FDR) were significantly enhanced. The rationality of MI therapy is proved by functional near-infrared spectroscopy. The involvement of DLPFC in motor decision-making may regulate the two-way feedback of premoter cortex-M1 during ME; and Broca area, closely related to the motor program understanding, participates in MI and ME.

Motor imagery (M.I.) is a mental state in which real movements are evoked without overt actions. There is some behavioural evidence that M.I. declines with ageing. The neurofunctional correlates of these changes have been investigated only in two studies, but none of the these studies has measured explicit correlations between behavioural variables and the brain response, nor the correlation of M.I. and motor execution (M.E.) of the same acts in ageing. In this paper, we report a behavioural and functional magnetic resonance imaging (fMRI) experiment that aimed to address this issue. Twenty-four young subjects (27 ± 5.6 years) and twenty-four elderly subjects (60 ± 4.6 years) performed two block-design fMRI tasks requiring actual movement (M.E.) or the mental rehearsal (M.I.) of finger movements. Participants also underwent a behavioural mental chronometry test in which the temporal correlations between M.I. and M.E. were measured. We found significant neurofunctional and behavioural differences between the elderly subjects and the young subjects during the M.E. and the M.I. tasks: for the M.E. task, the elderly subjects showed increased activation in frontal and prefrontal (pre-SMA) cortices as if M.E. had become more cognitively demanding; during the M.I. task, the elderly over-recruited occipito-temporo-parietal areas, suggesting that they may also use a visual imagery strategy. We also found between-group behavioural differences in the mental chronometry task: M.I. and M.E. were highly correlated in the young participants but not in the elderly participants. The temporal discrepancy between M.I. and M.E. in the elderly subjects correlated with the brain regions that showed increased activation in the occipital lobe in the fMRI. The same index was correlated with the premotor regions in the younger subjects. These observations show that healthy elderly individuals have decreased or qualitatively different M.I. compared to younger subjects.

In this study, we used two machine learning algorithms, namely, linear support vector machine (SVM) and convolutional neural network (CNN), to classify the BCI (Brain Computer interface) competition IV-2a 2-class MI (motor imagery) data set which consists of EEG data from 9 subjects. For each subject, 5 sessions of signals from three electrodes (C3, Cz, and C4) were recorded with sampling rate 250Hz. The training data, which consisted of the first 3 sessions, included 400 trials. The evaluation data, which consisted of the last 2 sessions, included 320 trials. Each trial started with gazing at fix cross on screen for 3 seconds followed by a one-second visual cue pointing either to the left or right to instruct the subject for left or right motor imagery over a period of 4 seconds, and then followed by a short break of at least 1.5 seconds. Features were extracted from the 0.5 to 2.5 second signals after the cue for each trial from C3 and C4. Each EEG trial was band pass filtered into different frequency bands, namely, delta (0.5-3Hz), theta (4-8Hz), alpha (8-12Hz), beta bands (13-30Hz), gamma bands (31-60Hz). Those filtered signals were then used as the input data for training the linear SVM. In addition, we generated a 2 by 500 matrix by down sampling the training data from each trial. There are 5760 such matrices in total generated from all subjects and serve as the input data for training CNN and the trained model was evaluated by another 340 matrices from each subject. Our CNN architecture consisted of 2 convolution layer and 2 fully connect layers, and there was a batch normalization layer before the activated layer and a dropout layer with a probability of 50% after the activated layer. The classification accuracies evaluated by averaged kappa values obtained from linear SVM and CNN are 0.5 and 0.621, respectively, suggesting the deep learning CNN method is superior to the classical linear SVM on the EEG classification.

Brain-computer interfaces can be used to operate devices by detecting a person's intention from their brain activity. Decoding motor imagery (MI) from electroencephalogram (EEG) signals is a commonly used approach for this purpose. To reliably identify MI from EEG signals, a sufficient number of sensors is usually required. However, a large number of sensors increases the computational cost of discriminating MI classes. Furthermore, consumer-grade devices that measure EEG signals often employ a reduced number of sensors compared to medical- or research-grade devices. In this experimental study, we investigate the tradeoff between accuracy and complexity when decoding MI from a restricted number of EEG sensors. For this purpose, several decoding pipelines were trained on EEG data using different subsets of electrode locations employing well-established decoding methods. We found that there is no significant difference <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{(\mathrm{p}=[0.18-0.91])}$</tex> in average decoding accuracy when using fewer sensors. The largest loss in performance for a single individual was a reduction in mean decoding accuracy of 0.1 when using 8 out of 64 available sensors. Decoding MI from a limited number of sensors is therefore feasible, highlighting the potential of using commercial sensor devices for this purpose to reduce both monetary and computational costs.

Brain–Computer Interfaces (BCIs) offer a non-invasive pathway for restoring motor function, particularly for individuals with limb loss. This review explored the effectiveness of Electroencephalography (EEG) and function Near-Infrared Spectroscopy (fNIRS) in decoding Motor Imagery (MI) movements for both offline and online BCI systems. EEG has been the dominant non-invasive neuroimaging modality due to its high temporal resolution and accessibility; however, it is limited by high susceptibility to electrical noise and motion artifacts, particularly in real-world settings. fNIRS offers improved robustness to electrical and motion noise, making it increasingly viable in prosthetic control tasks; however, it has an inherent physiological delay. The review categorizes experimental approaches based on modality, paradigm, and study type, highlighting the methods used for signal acquisition, feature extraction, and classification. Results show that while offline studies achieve higher classification accuracy due to fewer time constraints and richer data processing, recent advancements in machine learning—particularly deep learning—have improved the feasibility of online MI decoding. Hybrid EEG–fNIRS systems further enhance performance by combining the temporal precision of EEG with the spatial specificity of fNIRS. Overall, the review finds that predicting online imagined movement is feasible, though still less reliable than motor execution, and continued improvements in neuroimaging integration and classification methods are essential for real-world BCI applications. Broader dissemination of recent advancements in MI-based BCI research is expected to stimulate further interdisciplinary collaboration among roboticists, neuroscientists, and clinicians, accelerating progress toward practical and transformative neuroprosthetic technologies.

ObjectiveThe objective of this study was to analyze the changes in connectivity between motor imagery (MI) and motor execution (ME) in the premotor area (PMA) and primary motor cortex (MA) of the brain, aiming to explore suitable forms of treatment and potential therapeutic targets.MethodsTwenty-three inpatients with stroke were selected, and 21 right-handed healthy individuals were recruited. EEG signal during hand MI and ME (synergy and isolated movements) was recorded. Correlations between functional brain areas during MI and ME were compared.ResultsPMA and MA were significantly and positively correlated during hand MI in all participants. The power spectral density (PSD) values of PMA EEG signals were greater than those of MA during MI and ME in both groups. The functional connectivity correlation was higher in the stroke group than in healthy people during MI, especially during left-handed MI. During ME, functional connectivity correlation in the brain was more enhanced during synergy movements than during isolated movements. The regions with abnormal functional connectivity were in the 18th lead of the left PMA area.ConclusionLeft-handed MI may be crucial in MI therapy, and the 18th lead may serve as a target for non-invasive neuromodulation to promote further recovery of limb function in patients with stroke. This may provide support for the EEG theory of neuromodulation therapy for hemiplegic patients.

In recent years, deep-learning-based motor imagery (MI) electroencephalography (EEG) decoding methods have shown great potential in the field of the brain–computer interface (BCI). The existing literature is relatively mature in decoding methods for two classes of MI tasks. However, with the increase in MI task classes, decoding studies for four classes of MI tasks need to be further explored. In addition, it is difficult to obtain large-scale EEG datasets. When the training data are limited, deep-learning-based decoding models are prone to problems such as overfitting and poor robustness. In this study, we design a data augmentation method for MI-EEG. The original EEG is slid along the time axis and reconstructed to expand the size of the dataset. Second, we combine the gated recurrent unit (GRU) and convolutional neural network (CNN) to construct a parallel-structured feature fusion network to decode four classes of MI tasks. The parallel structure can avoid temporal, frequency and spatial features interfering with each other. Experimenting on the well-known four-class MI dataset BCI Competition IV 2a shows a global average classification accuracy of 80.7% and a kappa value of 0.74. The proposed method improves the robustness of deep learning to decode small-scale EEG datasets and alleviates the overfitting phenomenon caused by insufficient data. The method can be applied to BCI systems with a small amount of daily recorded data.

Currently, there is no effective treatment for pediatric patients with complete spinal cord injury. Motor imagery has been proposed as an alternative to physical training for patients who are unable to move voluntarily. Our aim was to reveal the potential mechanism of motor imagery in the rehabilitation of pediatric complete spinal cord injury. Twenty-six pediatric patients with complete spinal cord injury and 26 age- and sex-matched healthy children as healthy controls were recruited. All participants underwent the motor imagery task-related fMRI scans, and additional motor execution scans were performed only on healthy controls. First, we compared the brain-activation patterns between motor imagery and motor execution in healthy controls. Then, we compared the brain activation of motor imagery between the 2 groups and compared the brain activation of motor imagery in pediatric patients with complete spinal cord injury and that of motor execution in healthy controls. In healthy controls, compared with motor execution, motor imagery showed increased activation in the left inferior parietal lobule and decreased activation in the left supplementary motor area, paracentral lobule, middle cingulate cortex, and right insula. In addition, our results revealed that the 2 groups both activated the bilateral supplementary motor area, middle cingulate cortex and left inferior parietal lobule, and supramarginal gyrus during motor imagery. Compared with healthy controls, higher activation in the bilateral paracentral lobule, supplementary motor area, putamen, and cerebellar lobules III-V was detected in pediatric complete spinal cord injury during motor imagery, and the activation of these regions was even higher than that of healthy controls during motor execution. Our study demonstrated that part of the motor imagery network was functionally preserved in pediatric complete spinal cord injury and could be activated through motor imagery. In addition, higher-level activation in sensorimotor-related regions was also found in pediatric complete spinal cord injury during motor imagery. Our findings may provide a theoretic basis for the application of motor imagery training in pediatric complete spinal cord injury.

Motor imagery (MI) based brain-computer interfaces (BCIs) enable the direct control of external devices through the imagined movements of various body parts. Unlike previous systems that used fixed-length EEG trials for MI decoding, asynchronous BCIs aim to detect the user's MI without explicit triggers. They are challenging to implement, because the algorithm needs to first distinguish between resting-states and MI trials, and then classify the MI trials into the correct task, all without any triggers. This paper proposes a sliding window prescreening and classification (SWPC) approach for MI-based asynchronous BCIs, which consists of two modules: a prescreening module to screen MI trials out of the resting-state, and a classification module for MI classification. Both modules are trained with supervised learning followed by self-supervised learning, which refines the feature extractors. Within-subject and cross-subject asynchronous MI classifications on four different EEG datasets validated the effectiveness of SWPC, i.e., it always achieved the highest average classification accuracy, and outperformed the best state-of-the-art baseline on each dataset by about 2%.

Background/Objectives: Motor imagery (MI) is the mental practice of motor actions with temporal dynamics and neural features in common with motor execution (ME). Although MI can improve motor performance, it remains unclear how closely performance-related signatures of MI resemble those of ME during learning, particularly in tasks with intrinsic variability. This study investigated similarities and differences between MI and ME during a probabilistic sequence-learning task. Methods: Participants performed a finger-tapping serial reaction time task in either a motor execution (ME; n = 10) or motor imagery (MI; n = 10) condition. The task consisted of 750 auditory stimuli mapped to right-hand finger movements and generated by a probabilistic sequence with deterministic and variable events. Reaction times were analyzed using ANOVA designs to assess the effects of Group, Block, Event Type, and the Last Variable event. Results: The MI group showed a significant reduction in reaction times across blocks (p < 0.001), indicating learning-related performance improvement, whereas no block-wise improvement was observed in the ME group. Both groups were sensitive to the probabilistic structure of the sequence, with reaction times differing across event types. A significant Group × Event interaction (p < 0.01) indicated distinct performance signatures for MI and ME. In both groups, reaction times were modulated by the last variable event. Conclusions: Motor imagery supported learning in a probabilistic sequence task but was influenced by factors distinct from those governing motor execution, suggesting partially different underlying mechanisms.

Results of several neuroimaging studies support the functional equivalence model, which states that motor imagery (MI) and motor execution (ME) involve the same processes, except for the final execution component. In contrast, the motor-cognitive model implies that MI additionally involves frontal executive control processes. However, according to some authors MI may actually be more comparable to motor preparation (MP). In the current electroencephalographic study, a version of the discrete sequence production paradigm was employed in which human participants initially had to prepare a sequence of five finger movements that subsequently had to be executed, imagined, or withheld. MI, ME, and MP were compared by computing event-related (de)-synchronization in the theta, alpha/mu, and beta bands. Results revealed a major increase in frontal theta power during MI as compared to ME and MP. At the end of the examined intervals, a posterior reduction in alpha power was present during ME and MP, but not during MI. Finally, above sensorimotor areas a decrease in beta power was observed that was most pronounced in the case of ME. The increase of frontal theta activity during MI may reflect increased effort, while the absence of a reduction in posterior alpha power suggests no major involvement of visuospatial attention and/or visual imagery. The present findings favor the motor-cognitive model, as it predicts extra involvement of frontal executive processes during MI.

Objective. Deep transfer learning has been widely used to address the nonstationarity of electroencephalogram (EEG) data during motor imagery (MI) classification. However, previous deep learning approaches suffer from limited classification accuracy because the temporal and spatial features cannot be effectively extracted. Approach. Here, we propose a novel end-to-end deep subject adaptation convolutional neural network (SACNN) to handle the problem of EEG-based MI classification. Our proposed model jointly optimizes three modules, i.e. a feature extractor, a classifier, and a subject adapter. Specifically, the feature extractor simultaneously extracts the temporal and spatial features from the raw EEG data using a parallel multiscale convolution network. In addition, we design a subject adapter to reduce the feature distribution shift between the source and target subjects by using the maximum mean discrepancy. By minimizing the classification loss and the distribution discrepancy, the model is able to extract the temporal-spatial features to the prediction of a new subject. Main results. Extensive experiments are carried out on three EEG-based MI datasets, i.e. brain–computer interface (BCI) competition IV dataset IIb, BCI competition III dataset IVa, and BCI competition IV dataset I, and the average accuracy reaches to 86.42%, 81.71% and 79.35% on the three datasets respectively. Furthermore, the statistical analysis also indicates the significant performance improvement of SACNN. Significance. This paper reveals the importance of the temporal-spatial features on EEG-based MI classification task. Our proposed SACNN model can make fully use of the temporal-spatial information to achieve the purpose.

A brain-computer interface (BCI) based on motor imagery (MI) from the same limb can provide an intuitive control pathway but has received limited attention. It is still a challenge to classify multiple MI tasks from the same limb. The goal of this study is to propose a novel decoding method to classify the MI tasks of four joints of the same upper limb and the resting state. EEG signals were collected from 20 participants. A time-distributed attention network (TD-Atten) was proposed to adaptively assign different weights to different classes and frequency bands of the input multiband Common Spatial Pattern (CSP) features. The long short-term memory (LSTM) and dense layers were then used to learn sequential information from the reweight features and perform the classification. Our proposed method outperformed other baseline and deep learning-based methods and obtained the accuracies of 46.8% in the 5-class scenario and 53.4% in the 4-class scenario. The visualization results of attention weights indicated that the proposed framework can adaptively pay attention to alpha-band related features in MI tasks, which was consistent with the analysis of brain activation patterns. These results demonstrated the feasibility and interpretability of the attention mechanism in MI decoding and the potential of this fine MI paradigm to be applied for the control of a robotic arm or a neural prosthesis.