Affective Brain–Computer Interfaces: A Tutorial

FBN-TCN: Temporal convolutional neural network based on spatial domain fusion brain networks for affective brain–computer interfaces

EDITORIAL article Front. Comput. Neurosci., 11 July 2023 Volume 17 - 2023 | https://doi.org/10.3389/fncom.2023.1237252

Research on human emotions and underlying brain processes is mostly performed open-loop, e.g. by presenting emotional stimuli and measuring subject's brain responses. Investigating human emotions in interaction with emotional stimuli (closed-loop) significantly complicates experimental setups and has so far rarely been proposed. We present concept and technical realization of an electroencephalography (EEG)-based affective Brain-Computer Interface (BCI) to study emotional brain processes in continuous closed-loop interaction. Our BCI consists of an algorithm generating continuous patterns of synthesized affective music, embedded in an online BCI architecture. An initial calibration is employed to obtain user-specific models associating EEG patterns with affective content in musical patterns. These models are then used in online application to translate the user's affect into a continuous musical representation; playback to the user results in closed-loop affective brain-interactions. The proposed BCI provides a platform to stimulate the brain in a closed-loop fashion, offering novel approaches to study human sensorimotor integration and emotions.

Traditional electroencephalograph (EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject, which restricts the application of the affective brain computer interface (BCI) in practice. We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples. To solve this problem, we propose a multi-modal domain adaptive variational autoencoder (MMDA-VAE) method, which learns shared cross-domain latent representations of the multi-modal data. Our method builds a multi-modal variational autoencoder (MVAE) to project the data of multiple modalities into a common space. Through adversarial learning and cycle-consistency regularization, our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge. Extensive experiments are conducted on two public datasets, SEED and SEED-IV, and the results show the superiority of our proposed method. Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.

Brain–computer interfaces (BCIs), also known as brain–machine interfaces (BMIs), translate brain activity into new outputs that replace, restore, enhance, supplement or improve natural brain outputs. BCI research and development has grown rapidly for the past two decades. It is beginning to provide useful communication and control capacities to people with severe neuromuscular disabilities; and it is expanding into new areas such as neurorehabilitation that may greatly increase its clinical impact. At the same time, significant challenges remain, particularly in regard to translating laboratory advances into clinical use.

Functional near-infrared spectroscopy (fNIRS) holds increasing potential for Brain-Computer Interfaces (BCI) due to its portability, ease of application, robustness to movement artifacts, and relatively low cost. The use of fNIRS to support the development of affective BCI has received comparatively less attention, despite the role played by the prefrontal cortex in affective control, and the appropriateness of fNIRS to measure prefrontal activity. We present an active, fNIRS-based neurofeedback (NF) interface, which uses differential changes in oxygenation between the left and right sides of the dorsolateral prefrontal cortex to operationalize BCI input. The system is activated by users generating a state of anger, which has been previously linked to increased left prefrontal asymmetry. We have incorporated this NF interface into an experimental platform adapted from a virtual 3D narrative, in which users can express anger at a virtual character perceived as evil, causing the character to disappear progressively. Eleven subjects used the system and were able to successfully perform NF despite minimal training. Extensive analysis confirms that success was associated with the intent to express anger. This has positive implications for the design of affective BCI based on prefrontal asymmetry.

Inspired by the neuroscience research results that the human brain can produce dynamic responses to different emotions, a new electroencephalogram (EEG)-based human emotion classification model was proposed, named R2G-ST-BiLSTM, which uses a hierarchical neural network model to learn more discriminative spatiotemporal EEG features from local to global brain regions. First, the bidirectional long- and short-term memory (BiLSTM) network is used to obtain the internal spatial relationship of EEG signals on different channels within and between regions of the brain. Considering the different effects of various cerebral regions on emotions, the regional attention mechanism is introduced in the R2G-ST-BiLSTM model to determine the weight of different brain regions, which could enhance or weaken the contribution of each brain area to emotion recognition. Then a hierarchical BiLSTM network is again used to learn the spatiotemporal EEG features from regional to global brain areas, which are then input into an emotion classifier. Especially, we introduce a domain discriminator to work together with the classifier to reduce the domain offset between the training and testing data. Finally, we make experiments on the EEG data of the DEAP and SEED datasets to test and compare the performance of the models. It is proven that our method achieves higher accuracy than those of the state-of-the-art methods. Our method provides a good way to develop affective brain–computer interface applications.

Brain-Computer Interface (BCI)-based affect recognition in humans utilizing physiological signals is typically guided by a supervised method that relies heavily on the presence of appropriate affect representations. In this paper, we assess the effectiveness of contemporary affect representation methodologies for establishing ground truth for reliable affective BCI systems. On the DEAP dataset, two experiments were conducted to compare the observed emotion clusters based on subjective ratings with the clustering of corresponding physiological responses based on EEG characteristics. The outcomes of the two investigations are inconsistent with one another. The findings suggest the subjective use of emotion categories as opposed to their general application in establishing ground truth for supervised approaches and must encourage researchers to examine affect representation techniques critically before considering them as concrete representations of a person’s affective state.

Limb repositioning is necessary for individuals with severe physical disabilities to sustain muscle strength and prevent pressure sores. As robotic technologies become ubiquitous, these tools offer promise to support the repositioning process. However, research has yet to focus on ways in which individuals with severe physical disabilities can control robots for these tasks. This paper presents a study that examines the needs and attitudes of potential users with physical disabilities to control a robotic aid for limb repositioning. Subjects expressed interest in using brain---computer interface (BCI) and speech recognition technologies for purposes of executing robotic tasks. The performance of four subjects controlling arm movements on an avatar through the keyboard, mouse, BCI, and Dragon NaturallySpeaking speech recognition was evaluated. Although BCI and speech technologies may limit physical fatigue, more challenges were faced using BCI and speech conditions compared to the keyboard and mouse. This research promotes accessibility into mainstream robotic technologies and represents the first step in the development of a robotic prototype using a BCI and speech recognition technologies for limb repositioning.

Brain-computer interfaces (BCIs) are defined as the science and technology of devices and systems responding to neural processes in the brain that generate motor movements and to cognitive processes (e.g., memory) that modify motor movements. Advances in neuroscience, computational technology, component miniaturization, the biocompatibility of materials, and sensor technology have led to the much-improved feasibility of useful BCIs. Brain-Computer Interface can be developed by engineers, neuroscientists, physical scientists, and behavioral and social scientists as a team effort. A study on brain computers (BCI) discusses how the brain and external systems interact. In intrusive systems, electrodes are implanted in the cortex; in non-invasive systems, they are mounted on the scalp and use electroencephalography or electrocorticography to monitor neuronal activity. The BCI systems can be generally ranked based on the location of the electrodes used for detecting and measuring neurons in the brain. This WTEC report was intended to compile and reveal to government decision-makers and the scientific community the information on global developments and patterns in BCI research. The design of hardware, device architecture, functional electrical stimulation, non-invasive systems of communication, academic and industrial cognitive and emotional neuroprosthesis has been discussed in this chapter. The purpose of the present chapter is to review the current sensor technologies used for invasive and non-invasive BCI approaches throughout North America, Europe, and Asia. We have visited and/or interacted with key laboratories with expertise in these areas. Although not completely comprehensive, this chapter gives an overview of the major sensor technologies being developed for potential BCI applications.

Background: Brain-computer interface (BCI) is a collaboration between a brain and a device that enables signals from the brain to direct some external activity, which interface enables a direct communications pathway between the brain and the object to be controlled. By reading signals from an array of neurons and using computer chips and programs to translate the signals into action. The Magneto encephalography (MEG) is to record the firing of the neuron and absorb the brain activity as the magnetic field travels from region to region with in the brain. It has the potential to enhance the life style of the disabled person. The MEG Brain Computer Interface (BCI) impact on medicine and healthcare now may be subtle but is still revolutionary. Objectives: The purpose of this study was to develop novel methods and systems for rehabilitation and control of assistive devices using signals from the brain – BCI on motor coding and on neural plasticity. Materials and methods: The experiment was carried out in 24 albino rats with Magneto encephalography (MEG). All the rats underwent BCI procedure except the first and second group which was exposed to fake intervention. The objective of this proof of concept closed loop BCI experiment was for the subject to control the positive movement of a rat in the radial arm maze to take the food even altered position which was recorded in the system. Results: An important finding in the present study was the enhancing effect of BCI against neurodegeneration. It shows that these rats achieve to learn the task. It suggests that the BCI activated the neurons in the hippocampus and makes it sufficient for normal acquisition. Conclusion: Stress induced hippocampal degeneration leads to significant impairment of cognitive functions especially in calculation, immediate recall and attention. In this present study, stress induced loss of cognition was studied by activating the neurons of the hippocampus by grid electrodes of BCI system. Keywords: Magneto encephalography (MEG), glucocorticoid, hippocampal formation & grid electrodes.

In this paper, we propose and compare multiple criteria for selecting ElectroEncephaloGraphic (EEG) channels over the Riemannian manifold, for EEG classification in Brain- Computer Interfaces (BCI). These criteria aim to promote EEG covariance matrix classifiers to generalize well by considering EEG data non-stationarity. Our approach consists of both increasing the discriminative information between classes over the manifold and reducing the dispersion within classes. We also reduce the influence of outliers in both discriminative and dispersion measures. Using the proposed criteria, channel selection is done automatically in a backward elimination process. The criteria are evaluated on EEG signals recorded from a tetraplegic subject and dataset IVa from BCI competition III. Experimental evidences confirm that considering the dispersion within each class as a measure for quantifying the effects of non-stationarity and removing the most affected channels can improve the performance of BCI by 5% on the tetraplegic subject and by 12 % on dataset IVa.

Brain Computer Interface (BCI) is mainly divided into two phases; calibration phase for training and feedback phase. A calibration phase is usually time-consuming, thereby, being likely to raise subjects’ fatigue at the early stage. For more convenient and applicable BCI system it should be investigated to reduce such preparation (calibration) time before feedback phase. Beamformer is a source imaging technique widely used in MEG/EEG source localization problem. It passes only signals produced at the designated source point and filters out other signals such as noise. We conjecture information in source space may be consistent over well trained and good subjects. This idea facilitates to reuse existing datasets from the same or different subjects. Using IVa data in BCI competition III, we constructed a classifier from other 4 subject’s training data and performance was evaluated in source domain. In this work, we observed the proposed approach worked well, resulting in relatively good accuracies (73.21%, 74.21%) for two subjects.KeywordsBrain Computer InterfaceSource imagingTransfer learningZero training

A brain computer interface (BCI) is a novel communication system that translates brain signals into control commands. In this paper, we present a P300 BCI system based on ordinal pattern features. Compared to BCI system based on linear time domain features, we have shown that slightly better classification accuracies and bitrates can be achieved for healthy and disabled subjects.

Several studies have shown the positive clinical effect of brain computer interface (BCI) training for stroke rehabilitation. This study investigated the efficacy of the sensorimotor rhythm (SMR)-based BCI with audio-cue, motor observation and multisensory feedback for post-stroke rehabilitation. Furthermore, we discussed the interaction between training intensity and training duration in BCI training. Twenty-four stroke patients with severe upper limb (UL) motor deficits were randomly assigned to two groups: 2-week SMR-BCI training combined with conventional treatment (BCI Group, BG, n = 12) and 2-week conventional treatment without SMR-BCI intervention (Control Group, CG, n = 12). Motor function was measured using clinical measurement scales, including Fugl-Meyer Assessment-Upper Extremities (FMA-UE; primary outcome measure), Wolf Motor Functional Test (WMFT), and Modified Barthel Index (MBI), at baseline (Week 0), post-intervention (Week 2), and follow-up week (Week 4). EEG data from patients allocated to the BG was recorded at Week 0 and Week 2 and quantified by mu suppression means event-related desynchronization (ERD) in mu rhythm (8–12 Hz). All functional assessment scores (FMA-UE, WMFT, and MBI) significantly improved at Week 2 for both groups (p < 0.05). The BG had significantly higher FMA-UE and WMFT improvement at Week 4 compared to the CG. The mu suppression of bilateral hemisphere both had a positive trend with the motor function scores at Week 2. This study proposes a new effective SMR-BCI system and demonstrates that the SMR-BCI training with audio-cue, motor observation and multisensory feedback, together with conventional therapy may promote long-lasting UL motor improvement.Clinical Trial Registration: [http://www.chictr.org.cn], identifier [ChiCTR2000041119].