A Survey of Interactive Systems based on Brain-Computer Interfaces

EEG-VTTCNet: A loss joint training model based on the vision transformer and the temporal convolution network for EEG-based motor imagery classification

Brain-computer interfaces (BCIs) can provide communication and control to people who are totally paralyzed. BCIs can use noninvasive or invasive methods for recording the brain signals that convey the user's commands. Whereas noninvasive BCIs are already in use for simple applications, it has been widely assumed that only invasive BCIs, which use electrodes implanted in the brain, can provide multidimensional movement control of a robotic arm or a neuroprosthesis. We now show that a noninvasive BCI that uses scalp-recorded electroencephalographic activity and an adaptive algorithm can provide humans, including people with spinal cord injuries, with multidimensional point-to-point movement control that falls within the range of that reported with invasive methods in monkeys. In movement time, precision, and accuracy, the results are comparable to those with invasive BCIs. The adaptive algorithm used in this noninvasive BCI identifies and focuses on the electroencephalographic features that the person is best able to control and encourages further improvement in that control. The results suggest that people with severe motor disabilities could use brain signals to operate a robotic arm or a neuroprosthesis without needing to have electrodes implanted in their brains.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

在长期的载人航天任务中，复杂且极端的太空环境对宇航员的生理、心理及认知功能产生显著的影响，直接威胁任务的安全性和执行效率。脑机接口(brain-computer interface，BCI)技术作为连接大脑与外部设备的关键桥梁，通过精准采集和解析神经信号，为载人航天任务提供了创新的人机协同范式。非侵入式BCI技术在航天员选拔、训练、在轨任务执行及返回后康复等多个阶段展现出广阔的应用潜力。在任务准备阶段，BCI技术通过整合多模态信号评估与神经反馈训练，显著提升航天员的认知功能和心理韧性。在任务执行阶段，BCI技术能够实时监测航天员的生理与心理状态，并通过意念控制优化设备操作，从而提升任务效率与安全性。在任务结束返回后的康复期，BCI技术不仅能够结合非侵入式神经调控改善情绪与认知功能，还能够支持航天员的运动和认知恢复，并有助于其进行长期的健康管理。然而，BCI技术在航天应用中仍面临信号鲁棒性不足、系统自适应性有限及数据处理效率欠佳等挑战。未来，BCI技术应融合多模态生理传感器与深度学习算法，以实现精准监测和个性化干预，并结合虚拟现实及机器人技术开发智能化人机协作模式，为航天任务提供更高效的人机交互支持。

Enhancing the Social Impact of Contemporary Music with Neurotechnology

Brain Computer Interfaces (BCIs) are interfaces that put the user in communication with an electronic device through the brain activity produced by the user herself. Non-invasive BCI are mainly based on electroencephalographic (EEG) signals. While using these systems, users become able to manipulate their brain activity to produce signals that will then be used to control computers or Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. UMAP '19, June 9-12, 2019, Larnaca, Cyprus © 2019 Association for Computing Machinery. ACM ISBN 978-1-4503-6021-0/19/06. . . $15.00 https://doi.org/10.1145/3320435.3320472 other devices without the aid of motor movements. Besides active BCI systems, in which the user directly control the system by a conscious and voluntary mental activity, there are passive BCI that can be used to recognize mental states, like the user's emotional state (in particular the level of engagement) during the interaction or according to a received stimulus. In this paper we describe a passive BCI experiment and its results, wherein users are exposed to a set of emotional artworks and the engagement measured through a BCI headset is compared to their explicit engagement, in order to test the reliability of BCI-based engagement detection

Amyotrophic Lateral Sclerosis (ALS) is a neurodegenerative disease that causes progressive loss of voluntary movement, including the ability to speak. As the disease rapidly progresses, there is a need for augmented and alternative communication that removes physical barriers required for engagement. Brain-Computer Interfaces (BCI) are a technology that allow for communication using only brain signals in response to visual stimuli. This project aimed to design an accessible and practical BCI for continuous, at-home use, and to increase BCI usability by using familiar facial stimuli to elicit a stronger brain response. The overall goal for enhancement of BCI usability is to maximize ease of use for individuals with ALS and their caregivers so that BCI technologies can make a useful impact and improve the everyday lives of people with ALS. In this project, a new miniaturized and low-cost P300 BCI system was designed using wireless and battery-operated EEG neuroimaging, modular software, and the use of an individualized BCI presentation stimuli. A 3-dimensional virtual environment platform was integrated as a navigational control output mechanism that executes commands received from the BCI to move a user's avatar in a virtual maze. Together, these form the virtual environment BCI (veBCI). The BCI stimuli presentation consisted of a custom approach of integrating different human face images with the aim of increasing user engagement and elicited brain activity, and therefore improving BCI online accuracy. The impact and effectiveness of the approach was tested via a verification BCI study in which 6 different stimuli conditions were utilized: male celebrity face, female celebrity face, male stranger face, female stranger face, a family member/close friend of the participant, and blank/no face as a control. A total of 16 volunteers (average age 56.5 and eight diagnosed with ALS) used the BCI system with all 6 face conditions in ecologically valid settings. Results indicate that this new miniaturized BCI is usable for both people with ALS and healthy controls. Significant differences existed in the amplitude of the P300 response in ALS and healthy controls. Additionally, differences between male and female participants demonstrated the potential role of personalization to optimize the visually evoked potential response and further increase BCI online accuracy. In conclusion, a home-use P300 BCI system was developed with a new generation of miniaturized, low-cost, and mobile neuroimaging running on an embedded computational platform. BCI stimuli with human face inclusion could help personalization of the BCI. Furthermore, virtual environments provide unique new ways to implement BCI output beyond typical spelling tasks and communication. Future work could focus on developing dedicated online signal processing to eliminate noise and improve online accuracy during home use.

In the recent years, the field of Human-Computer Interaction (HCI) has greatly evolved to involve new and exciting interaction paradigms that allow users to interact with their environment and with technology in a more intuitive and ergonomic way. These interaction paradigms include voice, touch, virtual reality, and more recently, the brain. A brain-computer interface (BCI) is a an interface system allowing users to control devices without using the normal output pathways of peripherals, instead, by using neural activity generated in the brain. BCIs have a huge potential in a multitude of fields, all the way from providing users with severe motor disabilities with means for interaction with the external world, to entertainment, gaming, user state monitoring, and self-tracking systems. The potentials of BCI have sparked the interest of researchers, gaming markets and healthcare providers more and more in the recent years. The is due to the emergence of new commercial lightweight, low cost Electroencephalograph (EEG) equipment that made it possible to create more portable and usable BCI systems and expanded their fields of application. This Master thesis aims to explore the state of the art commercial BCI as well as the uses and challenges related to them. Commercially available EEG equipment, namely the Neurosky Brainband and Neurosky Mindset, will be investigated. User tests will be carried out to investigate whether such equipment with low accuracy and low cost can be used to recognize various mental activities. This would be performed by first collecting a dataset of brain signals during performing a set of mental tasks, which is one of the contributions of this project, followed by applying a set of signal processing algorithms, then exploring various classification techniques to classify the collected signals.

Human-machine systems designed for daily activities must balance direct user control with autonomous system behaviour to reduce cognitive effort while maintaining responsiveness. Douglas et al. have demonstrated how shared-autonomy frameworks can enable BCIs to operate across multiple levels of user involvement, from direct neural control to high-level goal specification. Rather than relying solely on low-level EEG commands, the proposed approach integrates intent inference with adaptive robotic planning, allowing multiple users to coordinate multiple robots in functional daily tasks. A key insight from this contribution is that increasing autonomy can significantly reduce user workload while preserving task accuracy and efficiency, supporting more sustainable long-term BCI use in real-world settings. This work highlights a broader shift in BCI design toward collaborative control paradigms, where machines proactively assist users instead of acting only as passive command receivers.Single-modality EEG BCIs are often limited by noise, signal ambiguity, and performance variability across users and environments. Hybrid approaches that fuse EEG with additional sensing modalities offer a promising solution, as demonstrated by Coutray and their team. Coutray et al. have proven how integrating EEG with eye-tracking improves command disambiguation, interaction speed, and overall control reliability, particularly in immersive virtual reality environments. This hybrid design reduces reliance on high-precision EEG decoding alone and enables more natural, intuitive hands-free interaction, expanding BCI applicability beyond clinical contexts into entertainment, accessibility, and smartenvironment control. This contribution reinforces the growing consensus that multimodal BCIs are more scalable, resilient, and user-friendly than EEG-only systems. Robust BCI performance depends not only on decoding algorithms but also on the user's cognitive and neural state. Mohamed et al. have provided evidence that baseline neural oscillations, particularly precue alpha activity, influence event-related desynchronisation (ERD) strength, a core signal used in motorimagery BCIs. A concrete implication of this finding is that BCIs could dynamically adjust classification thresholds, training protocols, or feedback timing based on real-time cognitive state estimates, potentially improving accuracy, consistency, and user learning rates. These insights support the development of state-aware BCIs that adapt decoding strategies in response to moment-to-moment brain dynamics, improving reliability in naturalistic, everyday environments.Beyond EEG decoding alone, combining physiological stress markers and subjective workload measures enables more comprehensive prediction of user performance in demanding tasks. Wei et al. have demonstrated that interpretable machine-learning models can forecast task outcomes while revealing which physiological factors most strongly influence performance. This is particularly relevant for realworld BCI deployment, where fatigue, stress, and cognitive overload can degrade system reliability. A key insight is that interpretable predictive models can support adaptive intervention strategies, such as adjusting task difficulty or providing rest prompts, while maintaining transparency and user trust.Across the contributions, several overarching themes emerge:1. Multimodal integration: Combining EEG with complementary sensors enhances decoding robustness and interaction flexibility. 2. Adaptive autonomy: Systems that balance user intent with automated assistance reduce workload and improve task efficiency. 3. Machine-learning innovation: Deep and interpretable models improve accuracy while supporting transparency and trust. 4. Cognitive context awareness: Accounting for neural and psychological state enables more reliable and personalised BCI performance.Together, these themes suggest a transition from static, single-user BCIs toward adaptive, context-aware, and multi-agent systems designed for long-term everyday use.

Brain Machine Interfaces (BMIs) enable direct communication between the brain or nervous system and a machine without involving the sensory-motor system. BMIs are an embryonic technology and remarkable accomplishments have recently been reported. BMIs have a high potential and possibly an enormous impact on society, and may evoke a revolution in the way we interact with computers. If we look at the Human Factors and Ergonomics community's position in the BMI field, we do not have a meaningful track record yet. However, the thesis of this paper is that we as a community are in a good position to (1) facilitate a broadening of the focus of BMIs from therapeutic applications to general use and (2) to realise a spin-in of BMI technology to domains such as Human Computer Interaction, supporting people with special needs, and training and simulation.

BackgroundFor severely paralyzed people, a brain-computer interface (BCI) provides a way of re-establishing communication. Although subjects with muscular dystrophy (MD) appear to be potential BCI users, the actual long-term effects of BCI use on brain activities in MD subjects have yet to be clarified. To investigate these effects, we followed BCI use by a chronic tetraplegic subject with MD over 5 months. The topographic changes in an electroencephalogram (EEG) after long-term use of the virtual reality (VR)-based BCI were also assessed. Our originally developed BCI system was used to classify an EEG recorded over the sensorimotor cortex in real time and estimate the user's motor intention (MI) in 3 different limb movements: feet, left hand, and right hand. An avatar in the internet-based VR was controlled in accordance with the results of the EEG classification by the BCI. The subject was trained to control his avatar via the BCI by strolling in the VR for 1 hour a day and then continued the same training twice a month at his home.ResultsAfter the training, the error rate of the EEG classification decreased from 40% to 28%. The subject successfully walked around in the VR using only his MI and chatted with other users through a voice-chat function embedded in the internet-based VR. With this improvement in BCI control, event-related desynchronization (ERD) following MI was significantly enhanced (p < 0.01) for feet MI (from -29% to -55%), left-hand MI (from -23% to -42%), and right-hand MI (from -22% to -51%).ConclusionsThese results show that our subject with severe MD was able to learn to control his EEG signal and communicate with other users through use of VR navigation and suggest that an internet-based VR has the potential to provide paralyzed people with the opportunity for easy communication.

Brain-Computer Interface

Objective. Noninvasive brain–computer interfaces (BCIs) assist paralyzed patients by providing access to the world without requiring surgical intervention. Prior work has suggested that EEG motor imagery based BCI can benefit from increased decoding accuracy through the application of deep learning methods, such as convolutional neural networks (CNNs). Approach. Here, we examine whether these improvements can generalize to practical scenarios such as continuous control tasks (as opposed to prior work reporting one classification per trial), whether valuable information remains latent outside of the motor cortex (as no prior work has compared full scalp coverage to motor only electrode montages), and the existing challenges to the practical implementation of deep-learning based continuous BCI control. Main results. We report that: (1) deep learning methods significantly increase offline performance compared to standard methods on an independent, large, and longitudinal online motor imagery BCI dataset with up to 4-classes and continuous 2D feedback; (2) our results suggest that a variety of neural biomarkers for BCI, including those outside the motor cortex, can be detected and used to improve performance through deep learning methods, and (3) tuning neural network output will be an important step in optimizing online BCI control, as we found the CNN models trained with full scalp EEG also significantly reduce the average trial length in a simulated online cursor control environment. Significance. This work demonstrates the benefits of CNNs classification during BCI control while providing evidence that electrode montage selection and the mapping of CNN output to device control will be important design choices in CNN based BCIs.

A new research methodology named Brain Computer Interface (BCI) studies novel human-computer interactions; by means of BCI electronics devices, paralyzed patients are able to interact with the environment using no muscular contractions. This technique provides an external electronics support to all persons with severe motor disabilities, by acquiring in continuous mode the electroencephalogram (EEG) signals and operating some processing to control a computer or other domotics devices. Patients are so allowed to control external devices or to communicate simple messages through the computer, just concentrating their attention either on codified movements or on a letter or icon on a digital keyboard. The use of a customized and optimized spatial filtering technique embedded in the BCI system, based on the detection of the Electroencephalographic activity, improves the accuracy of BCI system itself, thanks to the explicit separation of the signal activity of interest from artefact signals. In this chapter, after an overview of the state-of-the-art research on BCI systems, the spatial filtering problem in EEG signals acquisition will be illustrated. In particular, a spatial filtering algorithm, known as ICA (Independent Component Analysis) and its application will be discussed. Finally, the design and implementation of an embedded system for EEG signals acquisition and real-time processing for BCI applications will be presented. The system is based onto a very performing and reconfigurable hardware platform. Moreover ICA algorithm has been implemented for noise reduction and artifacts removal.

A Brain–Computer Interface (BCI) could help to restore mobility of severely paralyzed patients, for instance by prosthesis control. However, the currently achievable information transfer rate of noninvasive BCIs is insufficient to control complex prostheses continuously in many degrees of freedom. In this paper we present an autonomous system for grasping natural objects that compensates the low information flow from noninvasive BCIs. Using this system, one out of several objects can be grasped without any muscle activity. Rather, the grasp is initiated by decoded voluntary brain wave modulations. Object selection and grasping are performed in a virtual reality environment. A universal grasp planning algorithm calculates the trajectory of a gripper online. The system can be controlled after less than 10 min of training. We found that decoding accuracy increases over time and that an increased sense of agency achieved by permitting free selections renders the system to work most reliably.KeywordsBrain–computer interface (BCI)P300Oddball paradigmGrasp robotMagnetoencephalogram (MEG)Virtual reality (VR)Gaze independence