Coming Soon

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Brain-Computer Interface (BCI) enables users to interact with a computer only through their brain biological signals, without the need to use muscles. BCI is an emerging research area but it is still relatively immature. However, it is important to reflect on the different aspects of the Human-Computer Interaction (HCI) area related to BCIs, considering that BCIs will be part of interactive systems in the near future. BCIs most attend not only to handicapped users, but also healthy ones, improving interaction for end-users. Virtual Reality (VR) is also an important part of interactive systems, and combined with BCI could greatly enhance user interactions, improving the user experience by using brain signals as input with immersive environments as output. This paper addresses only noninvasive BCIs, since this kind of capture is the only one to not present risk to human health. As contributions of this work we highlight the survey of interactive systems based on BCIs focusing on HCI and VR applications, and a discussion on challenges and future of this subject matter.

BackgroundFor severely paralyzed people, a brain-computer interface (BCI) provides a way of re-establishing communication. Although subjects with muscular dystrophy (MD) appear to be potential BCI users, the actual long-term effects of BCI use on brain activities in MD subjects have yet to be clarified. To investigate these effects, we followed BCI use by a chronic tetraplegic subject with MD over 5 months. The topographic changes in an electroencephalogram (EEG) after long-term use of the virtual reality (VR)-based BCI were also assessed. Our originally developed BCI system was used to classify an EEG recorded over the sensorimotor cortex in real time and estimate the user's motor intention (MI) in 3 different limb movements: feet, left hand, and right hand. An avatar in the internet-based VR was controlled in accordance with the results of the EEG classification by the BCI. The subject was trained to control his avatar via the BCI by strolling in the VR for 1 hour a day and then continued the same training twice a month at his home.ResultsAfter the training, the error rate of the EEG classification decreased from 40% to 28%. The subject successfully walked around in the VR using only his MI and chatted with other users through a voice-chat function embedded in the internet-based VR. With this improvement in BCI control, event-related desynchronization (ERD) following MI was significantly enhanced (p < 0.01) for feet MI (from -29% to -55%), left-hand MI (from -23% to -42%), and right-hand MI (from -22% to -51%).ConclusionsThese results show that our subject with severe MD was able to learn to control his EEG signal and communicate with other users through use of VR navigation and suggest that an internet-based VR has the potential to provide paralyzed people with the opportunity for easy communication.

In the 1983 film `Brainstorm', a brain-computer interface (BCI) was developed to read and process thoughts, which are then transferred to video tape to be shared with other people. This was used to share happy memories, but also to experience a fellow scientist's heart attack, and it was the target of the military interest for torture and brainwashing. The workings of the human brain is still largely a mystery. Electrodes can offer an insight but some fear a sinister intent. The idea of being able to read a person's thoughts and exert control over their thoughts and actions has been a rich seam of material for science fiction and the medical profession alike. Today, there are some insights into the brain, but they are limited. Functional magnetic resonance imaging (fMRI) scans indicate brain activity by measuring the flow of blood to the brain, not what neurons in the brain are doing. Scientists are monitoring brain activity that controls the brain's speech regions to decode spoken words and phrases in real time. This work could eventually make it possible for people with speech loss to communicate via brain signals. Researchers at Harvard University are developing a flexible mesh of electrodes that could be implanted in the brain for targeted relief of symptoms of illness caused by neurodegeneration or ageing. Neuralink, a company founded by Elon Musk, is working on a project to implant thin, flexible threads into the brain to transfer data. To conclude, in a relatively short period of time, electrodes have moved from the surface of the head, to the surface of the brain, to being implanted mechanically into specific regions of the brain. The accumulation of this research, advances in materials and signal technology brings hope but also some concern - if knowledge is power, could the nightmare of Brainstorm await us?

Enhancing the Social Impact of Contemporary Music with Neurotechnology

Brain–computer interfaces (BCIs), also known as brain–machine interfaces (BMIs), translate brain activity into new outputs that replace, restore, enhance, supplement or improve natural brain outputs. BCI research and development has grown rapidly for the past two decades. It is beginning to provide useful communication and control capacities to people with severe neuromuscular disabilities; and it is expanding into new areas such as neurorehabilitation that may greatly increase its clinical impact. At the same time, significant challenges remain, particularly in regard to translating laboratory advances into clinical use.

Amyotrophic Lateral Sclerosis (ALS) is a neurodegenerative disease that causes progressive loss of voluntary movement, including the ability to speak. As the disease rapidly progresses, there is a need for augmented and alternative communication that removes physical barriers required for engagement. Brain-Computer Interfaces (BCI) are a technology that allow for communication using only brain signals in response to visual stimuli. This project aimed to design an accessible and practical BCI for continuous, at-home use, and to increase BCI usability by using familiar facial stimuli to elicit a stronger brain response. The overall goal for enhancement of BCI usability is to maximize ease of use for individuals with ALS and their caregivers so that BCI technologies can make a useful impact and improve the everyday lives of people with ALS. In this project, a new miniaturized and low-cost P300 BCI system was designed using wireless and battery-operated EEG neuroimaging, modular software, and the use of an individualized BCI presentation stimuli. A 3-dimensional virtual environment platform was integrated as a navigational control output mechanism that executes commands received from the BCI to move a user's avatar in a virtual maze. Together, these form the virtual environment BCI (veBCI). The BCI stimuli presentation consisted of a custom approach of integrating different human face images with the aim of increasing user engagement and elicited brain activity, and therefore improving BCI online accuracy. The impact and effectiveness of the approach was tested via a verification BCI study in which 6 different stimuli conditions were utilized: male celebrity face, female celebrity face, male stranger face, female stranger face, a family member/close friend of the participant, and blank/no face as a control. A total of 16 volunteers (average age 56.5 and eight diagnosed with ALS) used the BCI system with all 6 face conditions in ecologically valid settings. Results indicate that this new miniaturized BCI is usable for both people with ALS and healthy controls. Significant differences existed in the amplitude of the P300 response in ALS and healthy controls. Additionally, differences between male and female participants demonstrated the potential role of personalization to optimize the visually evoked potential response and further increase BCI online accuracy. In conclusion, a home-use P300 BCI system was developed with a new generation of miniaturized, low-cost, and mobile neuroimaging running on an embedded computational platform. BCI stimuli with human face inclusion could help personalization of the BCI. Furthermore, virtual environments provide unique new ways to implement BCI output beyond typical spelling tasks and communication. Future work could focus on developing dedicated online signal processing to eliminate noise and improve online accuracy during home use.

Brain computer interfaces (BCI) are the foundation of numerous therapeutic applications that use brain signals to control programs or to translate into feedback. While the technical creation of these systems may be done in the lab with limited design expertise, the translation into a therapeutic calls for the engagement of game designers. This is evermore true for BCI in virtual reality (VR). VR has the potential to elevate BCI in embodiment and immersiveness. These traits are key for neurofeedback therapies for neurobehavioral conditions like anxiety. The cooperation between game designers and scientists overcomes the hurdle in transforming an experiment into a tool. More often than not, BCI on the road to therapeutics or other practical applications are launched in original or adapted games to demonstrate the usability of the platform. In the absence of partnerships like this, slow or stalled progress ensues on the scientific translation. We demonstrate this principle in a range of examples and in-depth with Mandala Flow State—a VR neurofeedback system that first served as an interactive installation in an art museum.

A brain-computer interface (BCI) decodes the brain signals representing a desire to do something and transforms those signals into a control command. However, only a limited number of mental tasks have been previously investigated and classified. This study aimed to investigate two motor imagery (MI) commands, moving forward and moving backward, using a small number of EEG channels, to be used in a neurofeedback context. This study also aimed to simulate a BCI and investigate the offline classification between MI movements in forward and backward directions, using different features and classification methods. Ten healthy people participated in a two-session (48 min each) experiment. This experiment investigated neurofeedback of navigation in a virtual tunnel. Each session consisted of 320 trials where subjects were asked to imagine themselves moving in the tunnel in a forward or backward motion after a randomly presented (forward versus backward) command on the screen. Three electrodes were mounted bilaterally over the motor cortex. Trials were conducted with feedback. Data from session 1 were analyzed offline to train classifiers and to calculate thresholds for both tasks. These thresholds were used to form control signals that were later used online in session 2 in neurofeedback training to trigger the virtual tunnel to move in the direction requested by the user's brain signals. After 96 min of training, the online band-power neurofeedback training achieved an average classification of 76%, while the offline BCI simulation using power spectral density asymmetrical ratio and AR-modeled band power as features, and using LDA and SVM as classifiers, achieved an average classification of 80%.

ESRA19-0696 Non-pharmacological methods for comfort and pain management (virtual reality, musicotherapy, hypnosis,……)

Human emotions play a key role in numerous decision-making processes. The ability to correctly identify likes and dislikes as well as excitement and boredom would facilitate novel applications in neuromarketing, affective entertainment, virtual rehabilitation and forensic neuroscience that leverage on sub-conscious human affective states. In this neuroinformatics investigation, we seek to recognize human preferences and excitement passively through the use of electroencephalography (EEG) when a subject is presented with some 3D visual stimuli. Our approach employs the use of machine learning in the form of deep neural networks to classify brain signals acquired using a brain-computer interface (BCI). In the first part of our study, we attempt to improve upon our previous work, which has shown that EEG preference classification is possible although accuracy rates remain relatively low at 61%-67% using conventional deep learning neural architectures, where the challenge mainly lies in the accurate classification of unseen data from a cohort-wide sample that introduces inter-subject variability on top of the existing intra-subject variability. Such an approach is significantly more challenging and is known as subject-independent EEG classification as opposed to the more commonly adopted but more time-consuming and less general approach of subject-dependent EEG classification. In this new study, we employ deep networks that allow dropouts to occur in the architecture of the neural network. The results obtained through this simple feature modification achieved a classification accuracy of up to 79%. Therefore, this study has shown that the use of a deep learning classifier was able to achieve an increase in emotion classification accuracy of between 13% and 18% through the simple adoption of the use of dropouts compared to a conventional deep learner for EEG preference classification. In the second part of our study, users are exposed to a roller-coaster experience as the emotional stimuli which are expected to evoke the emotion of excitement, while simultaneously wearing virtual reality goggles, which delivers the virtual reality experience of excitement, and an EEG headset, acquires the raw brain signals detected when exposed to this excitement stimuli. Here, a deep learning approach is used to improve the excitement detection rate to well above the 90% accuracy level. In a prior similar study, the use of conventional machine learning approaches involving k-Nearest Neighbour (kNN) classifiers and Support Vector Machines (SVM) only achieved prediction accuracy rates of between 65% and 89%. Using a deep learning approach here, rates of 78%-96% were achieved. This demonstrates the superiority of adopting a deep learning approach over other machine learning approaches for detecting human excitement when immersed in an immersive virtual reality environment.

Brain-Computer Interface: Current and Emerging Rehabilitation Applications

A brain-computer interface (BCI) is a system that enables control of devices or communication with other persons, only through cerebral activity, without using muscles. The main application for BCIs is assistive technology for disabled persons. Examples for devices that can be controlled by BCIs are artificial limbs, spelling devices, or environment control systems. BCI research has seen renewed interest in recent years, and it has been convincingly shown that communication via a BCI is in principle feasible. However, present day systems still have shortcomings that prevent their widespread application. In part, these shortcomings are caused by limitations in the functionality of the pattern recognition algorithms used for discriminating brain signals in BCIs. Moreover, BCIs are often tested exclusively with able-bodied persons instead of conducting tests with the target user group, namely disabled persons. The goal of this thesis is to extend the functionality of pattern recognition algorithms for BCI systems and to move towards systems that are helpful for disabled users. We discuss extensions of linear discriminant analysis (LDA), which is a simple but efficient method for pattern recognition. In particular, a framework from Bayesian machine learning, the so-called evidence framework, is applied to LDA. An algorithm is obtained that learns classifiers quickly, robustly, and fully automatically. An extension of this algorithm allows to automatically reduce the number of sensors needed for acquisition of brain signals. More specifically, the algorithm allows to perform electrode selection. The algorithm for electrode selection is based on a concept known as automatic relevance determination (ARD) in Bayesian machine learning. The last part of the algorithmic development in this thesis concerns methods for computing accurate estimates of class probabilities in LDA-like classifiers. These probabilities are used to build a BCI that dynamically adapts the amount of acquired data, so that a preset, approximate bound on the probability of misclassifications is not exceeded. To test the algorithms described in this thesis, a BCI specifically tailored for disabled persons is introduced. The system uses electroencephalogram (EEG) signals and is based on the P300 evoked potential. Datasets recorded from five disabled and four able-bodied subjects are used to show that the Bayesian version of LDA outperforms plain LDA in terms of classification accuracy. Also, the impact of different static electrode configurations on classification accuracy is tested. In addition, experiments with the same datasets demonstrate that the algorithm for electrode selection is computationally efficient, yields physiologically plausible results, and improves classification accuracy over static electrode configurations. The classification accuracy is further improved by dynamically adapting the amount of acquired data. Besides the datasets recorded from disabled and able-bodied subjects, benchmark datasets from BCI competitions are used to show that the algorithms discussed in this thesis are competitive with state-of-the-art electroencephalogram (EEG) classification algorithms. While the experiments in this thesis are uniquely performed with P300 datasets, the presented algorithms might also be useful for other types of BCI systems based on the EEG. This is the case because functionalities such as robust and automatic computation of classifiers, electrode selection, and estimation of class probabilities are useful in many BCI systems. Seen from a more general point of view, many applications that rely on the classification of cerebral activity could possibly benefit from the methods developed in this thesis. Among the potential applications are interrogative polygraphy ("lie detection") and clinical applications, for example coma outcome prognosis and depth of anesthesia monitoring.

Real-time adjustments to task difficulty during flight training are crucial for optimizing performance and managing pilot workload. This study evaluated the functionality of a pretrained brain-computer interface (BCI) that adapts training difficulty based on real-time estimations of workload from brain signals. Specifically, an EEG-based neuro-adaptive training system was developed and tested in Virtual Reality (VR) flight simulations with military student pilots. The neuro-adaptive system was compared to a fixed sequence that progressively increased in difficulty, in terms of self-reported user engagement, workload, and simulator sickness (subjective measures), as well as flight performance (objective metric). Additionally, we explored the relationships between subjective workload and flight performance in the VR simulator for each condition. The experiments concluded with semi-structured interviews to elicit the pilots' experiences with the neuro-adaptive prototype. Results revealed no significant differences between the adaptive and fixed sequence conditions in subjective measures or flight performance. In both conditions, flight performance decreased as subjective workload increased. The semi-structured interviews indicated that, upon briefing, the pilots preferred the neuro-adaptive VR training system over the system with a fixed sequence, although individual differences were observed in the perception of difficulty and the order of changes in difficulty. Even though this study shows performance does not change, BCI-based flight training systems hold the potential to provide a more personalized and varied training experience.

Brain-Computer Interface (BCI) systems allow user to operate devices without muscular activation and experimental results indicate that they can induce activity-dependent plasticity. A BCI system has two key features: exploitation of brain signal changes induced by the cognitive task assigned to the user and mutual learning. With respect to the first feature, BCI users can learn to control and hence change their cortical activity, for example by performing a motor imagery task. Mutual learning, on the other hand, is set up between users, which have to learn how to control their brain activity, and the classifier used for learning and categorize brain signal changes. This mutual learning can be mediated by the use of feedback, which in turn can be discrete or continuous, abstract or realistic and so on. Motor imagery represents a promising approach for stroke rehabilitation, thus a BCI based on a motor imagery paradigm could be a useful tool. Nevertheless, in stroke rehabilitation settings two issues are emerging which are also crucial for motor imagery BCIs. The first one relates to the importance of recognize the imagery ability of the subject, the second one to the importance of feedback in re-learning of motor skills. The present study relates to this second issue and it is aimed at investigating attitudes of BCI users towards feedback. We considered a sample of six healthy males, using an EEG-based BCI and executing a Motor Imagery task without feedback (training phase) and with feedback (performance phase). A horizontal bar of varying length depicted on a computer screen was used as feedback medium and overall accuracy data were matched with neurophysiological data acquired from P z and O z . Results could be of importance for BCI use as neurorehabilitation tool in stroke rehabilitation protocols since they suggest that some subjects could need an increased cortical activation in order to deal with a stimulus like visual feedback. Moreover, they reveal a possible association between worsening in performance and increased cortical activation needed in order to cope with feedback.