An Integrated Machine Learning-Based Brain Computer Interface to Classify Diverse Limb Motor Tasks: Explainable Model

One of the main problems in Brain Computer Interface (BCI) systems is the non-stationary behavior of the electroencephalography (EEG) signals causing problems in real time applications. Another common problem in BCI systems is the situation where the labeled data are scarce. In this study, we take a semi-supervised learning perspective and propose solving both types of problems by updating the BCI system with labels obtained from the outputs of the classifier. To test the approach, data from motor imagery BCI system are used. Attributes extracted from EEG signals are classified with Linear Discriminant Analysis (LDA) and Support Vector Machines (SVM). With respect to the static classifiers, accuracy was improved approximately 4% using the proposed adaptation approach in the case of a training dataset. Even though the difference between the performance of static and adaptive classifiers decreases as the size of training data increases, the accuracy of our proposed adaptive classifier remains higher. The proposed approach has also improved the performance of a BCI system around 4% in the case of non-stationary signals as well.

BackgroundA novel artefact removal algorithm is proposed for a self-paced hybrid brain-computer interface (BCI) system. This hybrid system combines a self-paced BCI with an eye-tracker to operate a virtual keyboard. To select a letter, the user must gaze at the target for at least a specific period of time (dwell time) and then activate the BCI by performing a mental task. Unfortunately, electroencephalogram (EEG) signals are often contaminated with artefacts. Artefacts change the quality of EEG signals and subsequently degrade the BCI’s performance.MethodsTo remove artefacts in EEG signals, the proposed algorithm uses the stationary wavelet transform combined with a new adaptive thresholding mechanism. To evaluate the performance of the proposed algorithm and other artefact handling/removal methods, semi-simulated EEG signals (i.e., real EEG signals mixed with simulated artefacts) and real EEG signals obtained from seven participants are used. For real EEG signals, the hybrid BCI system’s performance is evaluated in an online-like manner, i.e., using the continuous data from the last session as in a real-time environment.ResultsWith semi-simulated EEG signals, we show that the proposed algorithm achieves lower signal distortion in both time and frequency domains. With real EEG signals, we demonstrate that for dwell time of 0.0s, the number of false-positives/minute is 2 and the true positive rate (TPR) achieved by the proposed algorithm is 44.7%, which is more than 15.0% higher compared to other state-of-the-art artefact handling methods. As dwell time increases to 1.0s, the TPR increases to 73.1%.ConclusionsThe proposed artefact removal algorithm greatly improves the BCI’s performance. It also has the following advantages: a) it does not require additional electrooculogram/electromyogram channels, long data segments or a large number of EEG channels, b) it allows real-time processing, and c) it reduces signal distortion.

The modern commercial industry is increasingly dependent on the analysis of vast data records. Analysing big electroencephalogram (EEG) signal data (called brain signal data) plays an important role in a wide variety of applications such as healthcare practices, brain computer interface (BCI) systems, innovative education, privacy and security, and biometrics. The key objective of this paper is to establish a methodological framework for identifying communicative intentions of motor disabled people from EEG data for application in BCI systems. The proposed framework is designed based on common spatial pattern (CSP) data method and optimized ensemble (OE) machine learning method for the application of BCI technologies. The CSP method is used for discovering important features from EEG data and finally the extracted features are fed as an input to optimized ensemble (OE) method. The proposed method was tested on BCI Competition III dataset IVa, which contains motor imagery-based EEG signal data. The experimental results show that our proposed method can handle brain signal big data for identifying communicative intentions for an advanced BCI system. We compared the performance of our proposed method with several other existing methods. In comparison with other established methods, our method achieves higher classification accuracy performance. This research assists the experts in processing and analysing EEG signals for BCI applications. It also supports technologists to create a new EEG data analyser for BCI systems.KeywordsElectroencephalographyBrain computer interfaceCommon spatial patternMachine learning

Event Abstract Back to Event Analysis and classification of EEG signals for brain computer interfaces Ugur Halici1*, E. Agi1, C. Özgen1 and I. Ulusoy1 1 Computer Vision and Intelligent Systems Research Laboratory, Middle East Technical University, Turkey Brain Computer Interfaces (BCIs) are systems developed in order to control devices by using only brain signals. Depending on the application, the device to be controlled might be a neuroprothesis, a wheel-chair or a computer. In BCI systems, different mental activities to be performed by the users are associated with different actions on the device to be controlled. The neurophysiologic brain signals produced as results of these mental activities are collected by a data acquisition device and converted to digital form. Electroencephalogram (EEG) is the most common technique employed for data acquisition in BCI systems today, due to its time-space resolution, its price and its easiness in use when compared to the other acquisition techniques. The digital brain signals collected are processed for feature extraction and then classified in order to decide on the associated action. A control signal produced according to the decision is used to change the state of the device to be controlled. Providing a feedback signal informing the user about the new state of the controlled device completes the BCI cycle. The early BCI systems were designed to work on pre-defined neurophysiologic features, where a feedback produced by a fixed set-up was provided to the system users. In these early BCI systems, the users should learn by themselves how to produce neurophysiologic signals in order to control the system. In several studies, it is reported that weeks, even months may be needed in order the user to learn how to produce these signals by adaptation of their brain. Recently, BCI systems in which machine learning algorithms employed for adapting the BCI system by using user specific signals are started to emerge. In several studies, it is shown that the adaptation of the BCI systems in this way reduces drastically the time required the user to able to produce appropriate signals and adaptation time in terms of days, even minutes are reported [1-4] The aim of this study is to analyze EEG signals collected for BCI purposes by using techniques Principal Component Analysis, Fisher Linear Discriminant Analysis, and Independent Component Analysis in order to reduce the dimension of the feature vectors obtained by sampling EEG signals in an representative manner and then classify the low dimension feature vectors obtained in this way by using Neural Networks and Support Vector Machines, which are able to learn through samples. The experimental studies are still going on for feature extraction and classification. The experimental results obtained will be presented in the final paper for comparison purposes.

Visual pursuit assessment is extensively applied in the behavioral scale-based clinical examination of patients with disorders of consciousness (DOC). However, this assessment is challenging because it relies on behavioral markers, and these patients severely lack behavioral responses. Brain-computer interfaces (BCIs) may provide a potential solution to detect brain responses to external stimuli without requiring behavioral expressions. A BCI system was designed to simulate visual pursuit detection in the coma recovery scale-revised (CRS-R). The graphical user interface included four buttons, one that moved on the screen and three that did not. These buttons flashed in a random order. The patients were prompted to follow the moving button. Based on the collected electroencephalography data, the algorithm determined whether the patient focused on the moving target. Among the 14 DOC patients who participated in the assessments based on the BCI system and the CRS-R, four patients exhibited visual pursuit, and three were nonresponsive in both assessments. More importantly, seven patients who did not exhibit visual pursuit in CRS-R were detected to be responsive to the moving target stimuli in the BCI assessment. Furthermore, five out of seven recovered consciousness to some degree and showed visual pursuit in the second CRS-R assessment. The proposed BCI system is better able to detect visual pursuit than the behavioral scale-based assessment and thus can assist in clinically evaluating the challenging population of DOC patients.

Motor imagery EEG recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network

Brain Computer Interface (BCI) systems have gained great visibility in the last years as they represent a quite innovative way of communication and a new instrument aimed at exploring brain functions. A lot of research labs are developing their own BCI system, everyone being involved in some particular aspects of them. At the "Tor Vergata" University our purpose is to develop tools for the evaluation and the optimization of the performances of BCI systems and to delineate some criteria for the analysis and implementation of different BCI systems; also we have defined file formats for BCI data in order to allow the sharing of tools among groups and to create models for the generalization and therefore the unification of the resources. All the tools and routines mentioned are part of the Body Language Framework++ 2.0.

Dealing with subjects who are unable to attain a proper level of performance, that is, those with brain–computer interface (BCI) illiteracy or BCI inefficients, is still a major issue in human electroencephalography (EEG) BCI systems. The most suitable approach to address this issue is to analyze the EEG signals of individual subjects independently recorded before the main BCI tasks, to evaluate their performance on these tasks. This study mainly focused on non-linear analyses and deep learning techniques to investigate the significant relationship between the intrinsic characteristics of a prior idle resting state and the subsequent BCI performance. To achieve this main objective, a public EEG motor/movement imagery dataset that constituted two individual EEG signals recorded from an idle resting state and a motor imagery BCI task was used in this study. For the EEG processing in the prior resting state, spectral analysis but also non-linear analyses, such as sample entropy, permutation entropy, and recurrent quantification analyses (RQA), were performed to obtain individual groups of EEG features to represent intrinsic EEG characteristics in the subject. For the EEG signals in the BCI tasks, four individual decoding methods, as a filter-bank common spatial pattern-based classifier and three types of convolution neural network-based classifiers, quantified the subsequent BCI performance in the subject. Statistical linear regression and ANOVA with post hoc analyses verified the significant relationship between non-linear EEG features in the prior resting state and three types of BCI performance as low-, intermediate-, and high-performance groups that were statistically discriminated by the subsequent BCI performance. As a result, we found that the frontal theta rhythm ranging from 4 to 8 Hz during the eyes open condition was highly associated with the subsequent BCI performance. The RQA findings that higher determinism and lower mean recurrent time were mainly observed in higher-performance groups indicate that more regular and stable properties in the EEG signals over the frontal regions during the prior resting state would provide a critical clue to assess an individual BCI ability in the following motor imagery task.

Non-invasive brain-computer interfaces (BCIs) have been widely used for neural decoding, linking neural signals to control devices. Hybrid BCI systems using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have received significant attention for overcoming the limitations of EEG- and fNIRS-standalone BCI systems. However, most hybrid EEG-fNIRS BCI studies have focused on late fusion because of discrepancies in their temporal resolutions and recording locations. Despite the enhanced performance of hybrid BCIs, late fusion methods have difficulty in extracting correlated features in both EEG and fNIRS signals. Therefore, in this study, we proposed a deep learning-based early fusion structure, which combines two signals before the fully-connected layer, called the fNIRS-guided attention network (FGANet). First, 1D EEG and fNIRS signals were converted into 3D EEG and fNIRS tensors to spatially align EEG and fNIRS signals at the same time point. The proposed fNIRS-guided attention layer extracted a joint representation of EEG and fNIRS tensors based on neurovascular coupling, in which the spatially important regions were identified from fNIRS signals, and detailed neural patterns were extracted from EEG signals. Finally, the final prediction was obtained by weighting the sum of the prediction scores of the EEG and fNIRS-guided attention features to alleviate performance degradation owing to delayed fNIRS response. In the experimental results, the FGANet significantly outperformed the EEG-standalone network. Furthermore, the FGANet has 4.0% and 2.7% higher accuracy than the state-of-the-art algorithms in mental arithmetic and motor imagery tasks, respectively.

Brain-computer interface (BCI) systems establish a direct communication channel from the brain to an output device. As the basis of BCIs, recognizing motor imagery activities poses a considerable challenge to signal processing due to the complex and non-stationary characteristics. This paper introduces an optimal and intelligent method for motor imagery BCIs. Because of the robustness to noise, wavelet packet decomposition and common spatial pattern (CSP) methods were implemented to reduce the dimensions of preprocessed signals. And a novel and efficient classifier projection extreme learning machine (PELM) was employed to recognize the labels of electroencephalogram signals. Experiments have been performed on the BCI Competition Dataset to demonstrate the superiority of wavelet-CSP in BCI and the outperformance of the PELM-based method. Results show that the average recognition rate of PELM approaches approximately 70%, while the optimal rate of other methods is 72%, whose training time and classification time are relatively longer as 11.00 ms and 11.66 ms, respectively, compared with 4.75 ms and 4.87 ms obtained by using the proposed BCI system.

Brain-Computer Interfaces (BCIs) offer transformative potential in rehabilitation, enabling patients to drive their recovery through their own thoughts and efforts rather than relying on pre-programmed machine outputs. Electroencephalography (EEG) signals, due to their non-invasive nature and high temporal resolution, are the most widely used in BCIs. Recent advancements in deep learning have significantly improved BCI decoding performance for broader movements such as arm and leg motions. However, to achieve comprehensive rehabilitation outcomes, a greater emphasis on decoding finer movements, such as individual finger motions, is essential for restoring intricate motor functions. Current BCI systems face significant barriers to mainstream adoption in healthcare rehabilitation. A major challenge lies in overcoming the high noise-to-signal ratio inherent in EEG signals, which complicates the development of robust models capable of accurately decoding user intentions. Additionally, decoding finer movement intentions remains underexplored, presenting further difficulties due to the subtle and overlapping differences in brain signal features associated with these movements. The lack of standardized protocols and the variability of EEG signals across different sessions and individuals further complicate this challenge. To address these challenges, various methodologies such as transfer learning and representation learning have been proposed. However, these methods often rely on the assumption of a single distribution to represent latent features, which may not generalize well across different subjects due to temporal variations in EEG signals, leading to information loss. Furthermore, transfer learning requires a substantial amount of labeled target data, which can be difficult to obtain in EEG-based rehabilitation contexts. This issue is compounded by the need for extensive calibration sessions to tailor the BCI system to each individual user. In this research, we demonstrate that techniques from other deep learning domains, such as image and natural language classification, can be adapted to enhance EEG decoding. For example, employing meta-learning principles in training subject-independent EEG classifiers enables deep learning models to achieve superior decoding performance with fewer target subject data. Meta-learning allows the model to quickly adapt to new subjects by leveraging prior knowledge, thus reducing the need for large amounts of subject-specific data. Eventually, we also show that utilizing contrastive learning techniques allows for improved performance, removing the need for labeled adaptation data and facilitating easier implementation in real-world rehabilitation settings. Finally, we also explore the application of self-supervised transformer-attention networks to enhance decoding performance in smaller datasets and extend this approach to decode finer motor attempts, such as individual finger movements, thereby advancing the precision of BCIs in restoring detailed motor functions. The applications of this research are extensive and impactful. By advancing the precision and practicality of EEG-based BCI systems, we can significantly enhance personalized rehabilitation programs, making them more effective and accessible. Focusing on patient-driven rehabilitation, patients are able to have greater ownership of their own recovery process and the positive feedback loop of their own efforts in improving motor function would lead to greater recovery outcomes. This technology can be applied across various therapeutic settings, including stroke recovery, neurodegenerative disease management, and motor function rehabilitation for patients with spinal cord injuries or limb amputations. Furthermore, the principles and methodologies developed here have broader implications for other domains requiring precise brain-signal decoding, such as neurofeedback therapy, cognitive enhancement, and brain-controlled prosthetics. This research paves the way for BCIs to play a crucial role in improving quality of life and healthcare outcomes, heralding a new era of patient-centered therapeutic interventions.

The brain computer interface (BCI) is an alternative communication method for those living with physically disabling conditions such as neuromuscular disorders, traumatic brain injuries, and stroke that uses a cap with embedded electrodes to read electroencephalography (EEC) signals and uses them to control a computer, Specifically, the P300, a naive response to target stimuli, is used to select letters from a matrix. The system provides those with no neuromuscular control with a novel channel for communication. To date, little to no work has investigated the effects of mental fatigue arising from short-term BCI use on the signals that are used to control the BCI itself. Methods typically used to characterize fatigue in EEG signals employ spectral analysis techniques that are confounded by periodic stimulus presentation in the BCI protocol. The characterization of the effects of fatigue could enable developers to create BCI systems that compensate for or prevent fatigue, which could improve overall usability. Eleven healthy subjects from the Philadelphia region participated in trials using the P300-based BCI alongside cognitive and self-reported measures of fatigue under informed consent. The changes in EEG signals related to increased mental fatigue occurred in a predictable manner, forming a "checkmark" pattern in P300 amplitude across BCI trials, mirroring the results of a 2006 study investigating fatigues effects on the P300 signal specifically. Future work may incorporate this pattern into adaptive BCI systems capable of detecting and compensating for mental fatigue during longterm BCI use. Additionally, a software package was designed and distributed to help accelerate and standardize research investigating event-related potentials (ERPs) such as the P300 wave. The package, created in MATLAB, allows the user to rapidly assess ERP quality and perform in-depth analysis of large quantities of data.

Brain Computer Interfacing (BCI) systems, which are a new communicating channel between humans and the computers are growing rapidly. One such a method is based on the Steady State Visual Evoked Potentials (SSVEP), which can be recorded during visual stimulating of the subject by a twinkling light source with a fixed frequency. An important parameter to be considered is the effect of the inter-sources distance on the accuracy of such BCI systems. In particular inter-sources (LEDs) distances of 4, 14, 24, 44 and 64 cm when the sources plane is located 60 cm away from the subject's eyes (producing inter-sources visual angles of 3.8°, 13.4°, 22.6°, 40.2° and 56° respectively) were examined. In addition, four different sweep lengths of 0.5, 1, 2 and 3 seconds are considered. In addition, due to the usage of the AR models for feature extraction from the SSVEP signals, selection of the best AR model together with the best classifier among the LDA, the SVM and the Naive Bayes are studied. It is showed that the BCI system with D=44 cm, AR order of 13 and either the LDA or the SVM classifiers could produce the best results compared to the other cases.

Mostly in natural disaster situations, for example an earthquake, we will face a situation where people get trapped and unable to be found as soon as possible due to the unpredictable terrain. Hence, robot is being seen as scouts that able to adapt and perceive, and usually snake type robot is able to move throughout rubble and small confines. Then, the idea of using modular self-reconfigurable (MSR) robots based on CEBOT in 1988 spark an interest to researchers to fit the search and rescue purpose. Besides that, the idea of controlling robots without manual control but by using “thinking” have fascinated researchers and lead to the development of interfaces between human and machines. In 1970, a Brain Computer Interface (BCI) system has been establish and the BCI system usually is based on the Electroencephalogram (EEG) which is the electrical potentials generated in which the information generated by user and independent of any muscular activities. The implementations of both technologies to each other will frontier for a new alternatives which improves self-reconfigurable modular robot in term of control strategy. As a preliminary framework, a simulation is needed to determine both systems capabilities, and it can be achieved by interfacing the robot simulator, OpenVIBE and Virtual Experimentation Platform (V-REP) with the Python programming language. This study presentation will focus on how to control the modular robot in V-REP environment by using Python language which can be used for interfacing with BCI system created in OpenVIBE software and the simulated robot configuration. Each joint handles of modular robot based on V-REP hierarchy being streamed by Python language. There are several methods for establishment of communication between OpenVIBE and other applications. The method purpose in this paper will utilizing Lab Streaming Layer (LSL) and Python script as mediator. The outcome that is expected to be obtained for this study will be the success of communication establish to control virtual modular robot based on data from OpenVIBE. The future work based on this study will involving the BCI system designed using OpenVIBE and implementation to real modular robot might be applied.

A brain computer interface (BCI) is a real-time communication system designed to allow users to voluntarily send messages or commands without sending them through the brain's normal output pathways. We designed a real-time BCI platform based on Android. A equipment that uses the human to control a portable device through some electroencephalographic (EEG) rhythms has also been presented. The neural signals obtained by EEG recording equipment are sent through a Web Service technology connection to a PC that processes it in real time and sends it wirelessly to Android. Web Service has been used as a wireless technology for sending EEG signals to a PC and vice versa, the Android gets the command from the PC though the Web Service technology, which has been proposed and furthers it showing that the system is reliable and robust enough to work in BCI systems. In this study we have investigated the human’s ability to application by manipulating neuronal motor cortex activity in the presence of a visual feedback environment. This paper presents one of these solutions, which control application by BCI based on motor imagery. This type of BCI solution is translating the motor imagery EEG into three control signals: left, right or transmutation. And then use it as an input device of application to realize the function of operating. The Android digitizes and stores the EEG signal, and exports the results of Signal Analysis on the Monitors to achieve the basic function which is application through the human’s ideas. COM technology allows the BCI platform has strong scalability to meet the needs of various types of brain computer interface solution for the application, and the result is cheering.