A novel brain-computer interface using a multi-touch surface

Emotion classification using EEG (Electroencephalogram) brain signals has recently garnered significant interest due to its potential applications in fields such as mental health, human-computer interaction, and affective computing. The three prominent recurrent neural network (RNN) architectures GRU (Gated Recurrent Unit), LSTM (Long Short-Term Memory), and BiLSTM (Bidirectional Long Short-Term Memory) are thoroughly compared in this study with a focus on classifying emotions. The study examines how well these RNN variations capture temporal dependencies and patterns within EEG data to precisely discriminate between various emotional states. Based on classification accuracy and training effectiveness, these model's performance will be assessed. The results will reveal the most suitable architecture in terms of accuracy and robustness for EEG-based emotion classification. The findings provide valuable insights into choosing the appropriate RNN architecture for emotion classification tasks, advancing the understanding of AI's potential in interpreting human emotions through EEG signals.

A brain computer interface (BCI) uses electrophysiological activities of the brain such as natural rhythms and evoked potentials to communicate with some external devices. P300 is a positive evoked potential (EP), elicited approximately 300 ms after an attended external stimulus. A P300-based BCI uses this evoked potential as a means of communication with the external devices. Until now this P300-based BCI has been rather slow, as it is difficult to detect a P300 response without averaging over a number of trials. Previously, independent component analysis (ICA) has been used in the extraction of P300. However, the drawback of ICA is that it extracts not only P300 but also non-P300 related components requiring a proper selection of P300 ICs by the system. In this study we propose an algorithm based on constrained independent component analysis (cICA) for P300 extraction which can extract only the relevant component by incorporating a priori information. A reference signal is generated as this a priori information of P300 and cICA is applied to extract the P300 related component. Then the extracted P300 IC is segmented, averaged, and classified into target and non-target events by means of a linear classifier. The method is fast, reliable, computationally inexpensive as compared to ICA and achieves an accuracy of 98.3% in the detection of P300.

Speech impairments, resulting from brain injuries, mental disorders, or vocal abuse, substantially affect an individual’s quality of life and can lead to social isolation. Brain–Computer Interfaces (BCIs), particularly those based on EEG, offer a promising support mechanism by harnessing brain signals. Owing to their clinical efficacy, cost-effective EEG devices, and expanding applications in the medical and social spheres, their usage has surged. This study introduces an ensemble-based feature engineering mechanism to pinpoint the optimal brain rhythm, channel subset, and feature set for accurately predicting imagined words from EEG signals via machine learning models. Leveraging the 2020 International BCI competition dataset, we employed bandpass filtering, channel wrapping, and ranking methods were applied to discern suitable brain rhythms and features associated with imagined speech. Subsequent application of kernel-based principal component analysis enabled us to compress the feature space dimensionality. We then trained various machine learning models, among which the kNN model excelled, achieving an average accuracy of 73% in a 10-fold cross-validation scheme ,surpassing 18% higher than the existing literature. The Gamma rhythm was identified as the most predictive of imagined speech from EEG brain signals. These advancements herald a new era of more precise and effective BCIs, poised to significantly improve the lives of those with speech impairments.

The brain computer interface (BCI) is an alternative communication method for those living with physically disabling conditions such as neuromuscular disorders, traumatic brain injuries, and stroke that uses a cap with embedded electrodes to read electroencephalography (EEC) signals and uses them to control a computer, Specifically, the P300, a naive response to target stimuli, is used to select letters from a matrix. The system provides those with no neuromuscular control with a novel channel for communication. To date, little to no work has investigated the effects of mental fatigue arising from short-term BCI use on the signals that are used to control the BCI itself. Methods typically used to characterize fatigue in EEG signals employ spectral analysis techniques that are confounded by periodic stimulus presentation in the BCI protocol. The characterization of the effects of fatigue could enable developers to create BCI systems that compensate for or prevent fatigue, which could improve overall usability. Eleven healthy subjects from the Philadelphia region participated in trials using the P300-based BCI alongside cognitive and self-reported measures of fatigue under informed consent. The changes in EEG signals related to increased mental fatigue occurred in a predictable manner, forming a "checkmark" pattern in P300 amplitude across BCI trials, mirroring the results of a 2006 study investigating fatigues effects on the P300 signal specifically. Future work may incorporate this pattern into adaptive BCI systems capable of detecting and compensating for mental fatigue during longterm BCI use. Additionally, a software package was designed and distributed to help accelerate and standardize research investigating event-related potentials (ERPs) such as the P300 wave. The package, created in MATLAB, allows the user to rapidly assess ERP quality and perform in-depth analysis of large quantities of data.

As the capability of an electroencephalogram’s (EEG) measurement of the real-time electrodynamics of the human brain is known to all, signal processing techniques, particularly deep learning, could either provide a novel solution for learning but also optimize robust representations from EEG signals. Considering the limited data collection and inadequate concentration of during subjects testing, it becomes essential to obtain sufficient training data and useful features with a potential end-user of a brain–computer interface (BCI) system. In this paper, we combined a conditional variational auto-encoder network (CVAE) with a generative adversarial network (GAN) for learning latent representations from EEG brain signals. By updating the fine-tuned parameter fed into the resulting generative model, we could synthetize the EEG signal under a specific category. We employed an encoder network to obtain the distributed samples of the EEG signal, and applied an adversarial learning mechanism to continuous optimization of the parameters of the generator, discriminator and classifier. The CVAE was adopted to adjust the synthetics more approximately to the real sample class. Finally, we demonstrated our approach take advantages of both statistic and feature matching to make the training process converge faster and more stable and address the problem of small-scale datasets in deep learning applications for motor imagery tasks through data augmentation. The augmented training datasets produced by our proposed CVAE-GAN method significantly enhance the performance of MI-EEG recognition.

The rapid serial visual presentation (RSVP) paradigm is a method that can be used to extend the P300 based brain computer interface (BCI) approach to enable high throughput target image recognition applications. The method requires high temporal resolution and hence, generating reliable and accurate stimulus triggers is critical for high performance execution. The traditional RSVP paradigm is normally deployed on two computers where software triggers generated at runtime by the image presentation software on a presentation computer are acquired along with the raw electroencephalography (EEG) signals by a dedicated data acquisition system connected to a second computer. It is often assumed that the stimulus presentation timing as acquired via events arising in the stimulus presentation code is an accurate reflection of the physical stimulus presentation. This is not necessarily the case due to various and variable latencies that may arise in the overall system. This paper describes a study to investigate in a representative RSVP implementation whether or not software-derived stimulus timing can be considered an accurate reflection of the physical stimuli timing. To investigate this, we designed a simple circuit consisting of a light diode resistor comparator circuit (LDRCC) for recording the physical presentation of stimuli and which in turn generates what we refer to as hardware triggered events. These hardware-triggered events constitute a measure of ground truth and are captured along with the corresponding stimulus presentation command timing events for comparison. Our experimental results show that using software-derived timing only may introduce uncertainty as to the true presentation times of the stimuli and this uncertainty itself is highly variable at least in the representative implementation described here. For BCI protocols such as those utilizing RSVP, the uncertainly introduced will cause impairment of performance and we recommend the use of additional circuitry to capture the physical presentation of stimuli and that these hardware-derived triggers should instead constitute the event markers to be used for subsequent analysis of the EEG.

One of the key components of brain–computer interface (BCI) design is the interfacing paradigm. Progress in BCI research and applications significantly depends on a successful paradigm implementation. BCI paradigm requires simple interpretation and ease of use for end users so that electroencephalogram (EEG) data can be mapped to an application, for example, to drive a prosthetic arm or to send a command to operate an external device. Another remarkable use of the BCI system is to set up a communication network with exterior environment. Such actions necessitate the user to disburse voluntary or involuntary mental or control task without making the subject fatigued or tired. To ensure these criteria, the paradigm design should be leveraged to deliver optimal performance. P300 and SSVEP are two paradigms that have gathered large attentions from BCI communities. Over the last decade, advancement of BCI research has experienced successful design of various hybrid BCI systems. This chapter focuses on the issues and challenges involved during the development of BCI paradigm to harness the advantages associated with recently developed BCI paradigms. In addition, importance of paradigms will be highlighted to open up novel BCI applications for future use, which, in turn, might have a great potential to increase the BCI users to a large extent.

Mental States are a function of brain activity; with advancements in Brain Computer Interface (BCI) tools, they can be effectively predicted. Generally, BCI researches are sophisticated requiring multi-channel electrodes, and often carried out in controlled lab environment. This paper illustrates that a simple BCI research, targeting specific region of brain, can be conducted with an elementary setup. A method is demonstrated to predict the level of attentiveness using Electroencephalography (EEG) signal obtained from single-channel, non-invasive, dry-electrode placed on prefrontal cortex region. The acquired raw signal was processed to obtain features which were used to classify the attention state into three classes. As Support Vector Machine (SVM) is more effective in training and classification of high dimensional data, it was implemented and its results were studied.

Brain-computer interfaces (BCIs) enable their users to interact with their surrounding environment using the activity of their brain only, without activating any muscle. This technology provides severely disabled people with an alternative mean to communicate or control any electric device. On the other hand, BCI applications are more and more dedicated to healthier people, with the aim of giving them access to augmented reality or new rehabilitation tools. As it is noninvasive, light and relatively cheap, electroencephalography (EEG) is the most used acquisition technique to record cerebral activity of the BCI users. However, when using such type of BCI, user movements are likely to provoke motions of the measuring electrodes which can severely damage the EEG quality. Thus, current BCI technology requires that the user sits and performs as little movements as possible. This is of course a strong limitation of BCI for use in ordinary life. Very recently, preliminary studies have been published in the literature and suggest that BCI applications can be realized even in the physically moving context. In this paper, we thoroughly investigate the possibility to develop a P300-based BCI system in ambulatory condition. The study is based on experimental data recorded with seven subjects executing a visual P300 speller-like discrimination task while simultaneously walking at different speeds on a treadmill. It is demonstrated that a P300-based BCI is definitely feasible in such conditions. Different artifact correction methods are described and discussed in detail. To conclude, a recommended approach is given for the development of a real-time application. © 2011 IEEE.

Recent advances in deep learning have had a methodological and practical impact on brain-computer interface (BCI) research. Among the various deep network architectures, convolutional neural networks (CNNs) have been well suited for spatio-spectral-temporal electroencephalogram (EEG) signal representation learning. Most of the existing CNN-based methods described in the literature extract features at a sequential level of abstraction with repetitive nonlinear operations and involve densely connected layers for classification. However, studies in neurophysiology have revealed that EEG signals carry information in different ranges of frequency components. To better reflect these multi-frequency properties in EEGs, we propose a novel deep multi-scale neural network that discovers feature representations in multiple frequency/time ranges and extracts relationships among electrodes, i.e., spatial representations, for subject intention/condition identification. Furthermore, by completely representing EEG signals with spatio-spectral-temporal information, the proposed method can be utilized for diverse paradigms in both active and passive BCIs, contrary to existing methods that are primarily focused on single-paradigm BCIs. To demonstrate the validity of our proposed method, we conducted experiments on various paradigms of active/passive BCI datasets. Our experimental results demonstrated that the proposed method achieved performance improvements when judged against comparable state-of-the-art methods. Additionally, we analyzed the proposed method using different techniques, such as PSD curves and relevance score inspection to validate the multi-scale EEG signal information capturing ability, activation pattern maps for investigating the learned spatial filters, and t-SNE plotting for visualizing represented features. Finally, we also demonstrated our method's application to real-world problems. Based on our experimental results and analyses, we believe that the proposed multi-scale neural network can be useful for various BCI paradigms, as a starting model or as a backbone network in any new BCI experiments.

Dealing with subjects who are unable to attain a proper level of performance, that is, those with brain–computer interface (BCI) illiteracy or BCI inefficients, is still a major issue in human electroencephalography (EEG) BCI systems. The most suitable approach to address this issue is to analyze the EEG signals of individual subjects independently recorded before the main BCI tasks, to evaluate their performance on these tasks. This study mainly focused on non-linear analyses and deep learning techniques to investigate the significant relationship between the intrinsic characteristics of a prior idle resting state and the subsequent BCI performance. To achieve this main objective, a public EEG motor/movement imagery dataset that constituted two individual EEG signals recorded from an idle resting state and a motor imagery BCI task was used in this study. For the EEG processing in the prior resting state, spectral analysis but also non-linear analyses, such as sample entropy, permutation entropy, and recurrent quantification analyses (RQA), were performed to obtain individual groups of EEG features to represent intrinsic EEG characteristics in the subject. For the EEG signals in the BCI tasks, four individual decoding methods, as a filter-bank common spatial pattern-based classifier and three types of convolution neural network-based classifiers, quantified the subsequent BCI performance in the subject. Statistical linear regression and ANOVA with post hoc analyses verified the significant relationship between non-linear EEG features in the prior resting state and three types of BCI performance as low-, intermediate-, and high-performance groups that were statistically discriminated by the subsequent BCI performance. As a result, we found that the frontal theta rhythm ranging from 4 to 8 Hz during the eyes open condition was highly associated with the subsequent BCI performance. The RQA findings that higher determinism and lower mean recurrent time were mainly observed in higher-performance groups indicate that more regular and stable properties in the EEG signals over the frontal regions during the prior resting state would provide a critical clue to assess an individual BCI ability in the following motor imagery task.

The electroencephalogram (EEG) signal is the key signal carrier of the brain-computer interface (BCI) system. The EEG data collected by the whole-brain electrode arrangement is conducive to obtaining higher information representation. Personalized electrode layout, while ensuring the accuracy of EEG signal decoding, can also shorten the calibration time of BCI and has become an important research direction. This paper reviews the EEG signal channel selection methods in recent years, conducts a comparative analysis of the combined effects of different channel selection methods and different classification algorithms, obtains the commonly used channel combinations in motor imagery, P300 and other paradigms in BCI, and explains the application scenarios of the channel selection method in different paradigms are discussed, in order to provide stronger support for a more accurate and portable BCI system.

Motor imagery EEG recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network

Non-invasive brain-computer interfaces (BCIs) have been widely used for neural decoding, linking neural signals to control devices. Hybrid BCI systems using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have received significant attention for overcoming the limitations of EEG- and fNIRS-standalone BCI systems. However, most hybrid EEG-fNIRS BCI studies have focused on late fusion because of discrepancies in their temporal resolutions and recording locations. Despite the enhanced performance of hybrid BCIs, late fusion methods have difficulty in extracting correlated features in both EEG and fNIRS signals. Therefore, in this study, we proposed a deep learning-based early fusion structure, which combines two signals before the fully-connected layer, called the fNIRS-guided attention network (FGANet). First, 1D EEG and fNIRS signals were converted into 3D EEG and fNIRS tensors to spatially align EEG and fNIRS signals at the same time point. The proposed fNIRS-guided attention layer extracted a joint representation of EEG and fNIRS tensors based on neurovascular coupling, in which the spatially important regions were identified from fNIRS signals, and detailed neural patterns were extracted from EEG signals. Finally, the final prediction was obtained by weighting the sum of the prediction scores of the EEG and fNIRS-guided attention features to alleviate performance degradation owing to delayed fNIRS response. In the experimental results, the FGANet significantly outperformed the EEG-standalone network. Furthermore, the FGANet has 4.0% and 2.7% higher accuracy than the state-of-the-art algorithms in mental arithmetic and motor imagery tasks, respectively.

BackgroundA novel artefact removal algorithm is proposed for a self-paced hybrid brain-computer interface (BCI) system. This hybrid system combines a self-paced BCI with an eye-tracker to operate a virtual keyboard. To select a letter, the user must gaze at the target for at least a specific period of time (dwell time) and then activate the BCI by performing a mental task. Unfortunately, electroencephalogram (EEG) signals are often contaminated with artefacts. Artefacts change the quality of EEG signals and subsequently degrade the BCI’s performance.MethodsTo remove artefacts in EEG signals, the proposed algorithm uses the stationary wavelet transform combined with a new adaptive thresholding mechanism. To evaluate the performance of the proposed algorithm and other artefact handling/removal methods, semi-simulated EEG signals (i.e., real EEG signals mixed with simulated artefacts) and real EEG signals obtained from seven participants are used. For real EEG signals, the hybrid BCI system’s performance is evaluated in an online-like manner, i.e., using the continuous data from the last session as in a real-time environment.ResultsWith semi-simulated EEG signals, we show that the proposed algorithm achieves lower signal distortion in both time and frequency domains. With real EEG signals, we demonstrate that for dwell time of 0.0s, the number of false-positives/minute is 2 and the true positive rate (TPR) achieved by the proposed algorithm is 44.7%, which is more than 15.0% higher compared to other state-of-the-art artefact handling methods. As dwell time increases to 1.0s, the TPR increases to 73.1%.ConclusionsThe proposed artefact removal algorithm greatly improves the BCI’s performance. It also has the following advantages: a) it does not require additional electrooculogram/electromyogram channels, long data segments or a large number of EEG channels, b) it allows real-time processing, and c) it reduces signal distortion.