Evaluation of Motor Training Performance in 3D Virtual Environment via Combining Brain-computer Interface and Haptic Feedback

One of the main problems in Brain Computer Interface (BCI) systems is the non-stationary behavior of the electroencephalography (EEG) signals causing problems in real time applications. Another common problem in BCI systems is the situation where the labeled data are scarce. In this study, we take a semi-supervised learning perspective and propose solving both types of problems by updating the BCI system with labels obtained from the outputs of the classifier. To test the approach, data from motor imagery BCI system are used. Attributes extracted from EEG signals are classified with Linear Discriminant Analysis (LDA) and Support Vector Machines (SVM). With respect to the static classifiers, accuracy was improved approximately 4% using the proposed adaptation approach in the case of a training dataset. Even though the difference between the performance of static and adaptive classifiers decreases as the size of training data increases, the accuracy of our proposed adaptive classifier remains higher. The proposed approach has also improved the performance of a BCI system around 4% in the case of non-stationary signals as well.

For educational purposes, virtual reality (VR) is often used to produce life-like experiences. However, the use of haptic feedback in educational practices for science and mathematics remained under-studied. Haptically-enabled science simulations (HESSs) enable students to physically experience the concepts being taught via haptic feedback. We present a study on the effect of a buoyancy HESS designed to aid in the understanding of basic physics concepts. We hypothesize that introducing both visual and haptic feedback of the underlying simulated forces will improve participants understanding. We investigate this hypothesis with a 2 (haptics: yes, no) × 2 (visuals: yes, no) between subjects design user study, where all participants were randomly assigned to one of the four conditions. Participants were given a pretest of buoyancy knowledge, then instructed to interact with the buoyancy simulation, then given a post-test of buoyancy knowledge. The present study is still in the process of data collection, with 40 out of 60 participants. Preliminary results highlight a significant improvement in performance of participants in the haptic-and-visual condition, while no significant differences were observed in other conditions.

Slip feedback is an important cue in everyday object manipulation, but it is generally missing in tele-operation systems. To test the usefulness of simple, abstract types of feedback that warn the user about slip events, we tested the effect of auditory and haptic vibration feedback in a tele-operation task. Participants were asked to hold an object in a remote robot hand, and the force profiles that they exerted in response to slip events were measured. Haptic feedback did not significantly change the response characteristics, but auditory feedback did significantly improve response latency. A small but significant difference between haptic and auditory reaction times (60 ms) found in our control experiment might explain the difference between the feedback types.

This research evaluated visual and haptic feedback for enhancing operator situational awareness during multitasking drone operations, investigating how task complexity influenced feedback effectiveness. Two virtual reality experiments were conducted: Study 1 assessed three types of visual feedback (Static, Blink, and Shake), while Study 2 compared visual, haptic, and visual-haptic multimodal feedback across varying task complexity levels determined by target density. Participants performed drone operation tasks requiring simultaneous obstacle avoidance and target counting. Results revealed that dynamic visual feedback (Shake) significantly improved primary task accuracy, reduced cognitive load, and was the preferred visual feedback type. Furthermore, haptic and visual-haptic feedback significantly shortened obstacle avoidance reaction times and decreased cognitive workload compared to visual feedback. Task complexity proved to be a critical factor: while visual feedback sufficed for low complexity scenarios, visual-haptic feedback offered distinct advantages in medium and high complexity tasks, providing insights for complexity-responsive drone interface design.

We routinely have physical interactions with others, whether it be handing someone a glass of water or jointly moving a heavy object together. These sensorimotor interactions between humans typically rely on visual feedback and haptic feedback. Recent single-participant studies have highlighted that the unique noise and time delays of each sense must be considered to estimate the state, such as the position and velocity, of one's own movement. However, we know little about how visual feedback and haptic feedback are used to estimate the state of another person. Here, we tested how humans utilize visual feedback and haptic feedback to estimate the state of their partner during a collaborative sensorimotor task. Across two experiments, we show that visual feedback dominated haptic feedback during collaboration. Specifically, we found that visual feedback led to comparatively lower task-relevant movement variability, smoother collaborative movements, and faster trial completion times. We also developed an optimal feedback controller that considered the noise and time delays of both visual feedback and haptic feedback to estimate the state of a partner. This model was able to capture both lower task-relevant movement variability and smoother collaborative movements. Taken together, our empirical and modeling results support the idea that visual accuracy is more important than haptic speed to perform state estimation of a partner during collaboration.NEW & NOTEWORTHY Physical collaboration between two or more individuals involves both visual and haptic feedback. Here, we investigated how visual and haptic feedback is used to estimate the movements of a partner during a collaboration task. Our experimental and computational modeling results parsimoniously support the notion that greater visual accuracy is more important than faster yet noisier haptic feedback when estimating the state of a partner.

This study investigates the influence of shoe type (sneakers and safety boots), age, and gender on the perception of haptic pulse feedback provided by a prototype accelerator pedal in a running stationary vehicle. Haptic feedback can be a less distracting alternative to traditionally visual and auditory in-vehicle feedback. However, to be effective, the device delivering the haptic feedback needs to be in contact with the person. Factors such as shoe type vary naturally over the season and could render feedback that is perceived well in one situation, unnoticeable in another. In this study, we evaluate factors that can influence the subjective perception of haptic feedback in a stationary but running car: shoe type, age, and gender. Thirty-six drivers within three age groups (≤39, 40-59, and ≥60) took part. For each haptic feedback, participants rated intensity, urgency, and comfort via a questionnaire. The perception of the haptic feedback is significantly influenced by the interaction between the pulse's duration and force amplitude and the participant's age and gender but not shoe type. The results indicate that it is important to consider different age groups and gender in the evaluation of haptic feedback. Future research might also look into approaches to adapt haptic feedback to the individual driver's preferences. Findings from this study can be applied to the design of an accelerator pedal in a car, for example, for a nonvisual in-vehicle warning, but also to plan user studies with a haptic pedal in general.

Brain Computer Interface (BCI) systems have gained great visibility in the last years as they represent a quite innovative way of communication and a new instrument aimed at exploring brain functions. A lot of research labs are developing their own BCI system, everyone being involved in some particular aspects of them. At the "Tor Vergata" University our purpose is to develop tools for the evaluation and the optimization of the performances of BCI systems and to delineate some criteria for the analysis and implementation of different BCI systems; also we have defined file formats for BCI data in order to allow the sharing of tools among groups and to create models for the generalization and therefore the unification of the resources. All the tools and routines mentioned are part of the Body Language Framework++ 2.0.

In this study we compared tactile and visual feedbacks for the motor imagery-based brain–computer interface (BCI) in five healthy subjects. A vertical green bar from the center of the fixing cross to the edge of the screen was used as visual feedback. Vibration motors that were placed on the forearms of the right and the left hands and on the back of the subject’s neck were used as tactile feedback. A vibration signal was used to confirm the correct classification of the EEG patterns of the motor imagery of right and left hand movements and the rest task. The accuracy of recognition in the classification of the three states (right hand movement, left hand movement, and rest) in the BCI without feedback exceeded the random level (33% for the three states) for all the subjects and was rather high (67.8% ± 13.4% (mean ± standard deviation)). Including the visual and tactile feedback in the BCI did not significantly change the mean accuracy of recognition of mental states for all the subjects (70.5% ± 14.8% for the visual feedback and 65.9% ± 12.4% for the tactile feedback). The analysis of the dynamics of the movement imagery skill in BCI users with the tactile and visual feedback showed no significant differences between these types of feedback. Thus, it has been found that the tactile feedback can be used in the motor imagery-based BCI instead of the commonly used visual feedback, which greatly expands the possibilities of the practical application of the BCI.

Brain Computer Interfacing (BCI) systems, which are a new communicating channel between humans and the computers are growing rapidly. One such a method is based on the Steady State Visual Evoked Potentials (SSVEP), which can be recorded during visual stimulating of the subject by a twinkling light source with a fixed frequency. An important parameter to be considered is the effect of the inter-sources distance on the accuracy of such BCI systems. In particular inter-sources (LEDs) distances of 4, 14, 24, 44 and 64 cm when the sources plane is located 60 cm away from the subject's eyes (producing inter-sources visual angles of 3.8°, 13.4°, 22.6°, 40.2° and 56° respectively) were examined. In addition, four different sweep lengths of 0.5, 1, 2 and 3 seconds are considered. In addition, due to the usage of the AR models for feature extraction from the SSVEP signals, selection of the best AR model together with the best classifier among the LDA, the SVM and the Naive Bayes are studied. It is showed that the BCI system with D=44 cm, AR order of 13 and either the LDA or the SVM classifiers could produce the best results compared to the other cases.

Mostly in natural disaster situations, for example an earthquake, we will face a situation where people get trapped and unable to be found as soon as possible due to the unpredictable terrain. Hence, robot is being seen as scouts that able to adapt and perceive, and usually snake type robot is able to move throughout rubble and small confines. Then, the idea of using modular self-reconfigurable (MSR) robots based on CEBOT in 1988 spark an interest to researchers to fit the search and rescue purpose. Besides that, the idea of controlling robots without manual control but by using “thinking” have fascinated researchers and lead to the development of interfaces between human and machines. In 1970, a Brain Computer Interface (BCI) system has been establish and the BCI system usually is based on the Electroencephalogram (EEG) which is the electrical potentials generated in which the information generated by user and independent of any muscular activities. The implementations of both technologies to each other will frontier for a new alternatives which improves self-reconfigurable modular robot in term of control strategy. As a preliminary framework, a simulation is needed to determine both systems capabilities, and it can be achieved by interfacing the robot simulator, OpenVIBE and Virtual Experimentation Platform (V-REP) with the Python programming language. This study presentation will focus on how to control the modular robot in V-REP environment by using Python language which can be used for interfacing with BCI system created in OpenVIBE software and the simulated robot configuration. Each joint handles of modular robot based on V-REP hierarchy being streamed by Python language. There are several methods for establishment of communication between OpenVIBE and other applications. The method purpose in this paper will utilizing Lab Streaming Layer (LSL) and Python script as mediator. The outcome that is expected to be obtained for this study will be the success of communication establish to control virtual modular robot based on data from OpenVIBE. The future work based on this study will involving the BCI system designed using OpenVIBE and implementation to real modular robot might be applied.

Spinal cord injury (SCI) impairs motor function and requires rigorous rehabilitative therapy, motivating the development of approaches that are engaging and customizable. Virtual reality (VR) motor training with augmented sensory feedback (ASF) offers a promising pathway to enhance functional outcomes, yet it remains unclear how ASF modalities affect performance and underlying psychophysiological states in persons with SCI. Five participants with chronic incomplete cervical-level SCI controlled a virtual robotic arm with semi-isometric upper-body contractions while undergoing ASF training with either visual feedback (VF) or combined visual plus haptic feedback (VHF). Motor performance (pathlength, completion time), psychophysiological measures (EEG, EMG, EDA, HR), and perceptual ratings (agency, motivation, utility) were assessed before and after ASF training. VF significantly reduced pathlength (−12.5%, p = 0.0011) and lowered EMG amplitude (−32.5%, p = 0.0063), suggesting the potential for improved motor performance and neuromuscular efficiency. VHF did not significantly improve performance, but trended toward higher cortical engagement. EEG analyses showed VF significantly decreased alpha and beta activity after training, whereas VHF trended toward mild increases. Regression revealed improved performance was significantly (p < 0.05) associated with changes in alpha power, EMG, EDA, and self-reported motivation. ASF type may differentially shape performance and psychophysiological responses in SCI participants. These preliminary findings suggest VR-based ASF as a potent multidimensional tool for personalizing rehabilitation.

Brain computer interface (BCI) system is field of neurotechnology that convey command from human brain to a computer. BCI is currently most growing field in the research area. BCI applications have different field such as medical, education, neuroergonomics, self-regulation, games and entertainment, production, security as well as marketing. Electronic devices can be controlled using brain signal called electroencephalography (EEG) to record electrical activity of the brain. The P300 wave is a positive peak of an event related potential (ERP) that occurs 300ms recorded by EEG. One major approach in the BCI research field is the P300 based oddball paradigm, subjects identify infrequent target stimuli within a series of rapid presented standard stimuli. The P300-speller is a widely used BCI system that allows users to communicate characters by focused attention. In our study, we are focusing on BCI system architecture different applications of BCI specifically P300 speller which classifies P300 waves and recognize targeted character, on the dataset II of the third BCI competition.

BackgroundFor severely paralyzed people, a brain-computer interface (BCI) provides a way of re-establishing communication. Although subjects with muscular dystrophy (MD) appear to be potential BCI users, the actual long-term effects of BCI use on brain activities in MD subjects have yet to be clarified. To investigate these effects, we followed BCI use by a chronic tetraplegic subject with MD over 5 months. The topographic changes in an electroencephalogram (EEG) after long-term use of the virtual reality (VR)-based BCI were also assessed. Our originally developed BCI system was used to classify an EEG recorded over the sensorimotor cortex in real time and estimate the user's motor intention (MI) in 3 different limb movements: feet, left hand, and right hand. An avatar in the internet-based VR was controlled in accordance with the results of the EEG classification by the BCI. The subject was trained to control his avatar via the BCI by strolling in the VR for 1 hour a day and then continued the same training twice a month at his home.ResultsAfter the training, the error rate of the EEG classification decreased from 40% to 28%. The subject successfully walked around in the VR using only his MI and chatted with other users through a voice-chat function embedded in the internet-based VR. With this improvement in BCI control, event-related desynchronization (ERD) following MI was significantly enhanced (p < 0.01) for feet MI (from -29% to -55%), left-hand MI (from -23% to -42%), and right-hand MI (from -22% to -51%).ConclusionsThese results show that our subject with severe MD was able to learn to control his EEG signal and communicate with other users through use of VR navigation and suggest that an internet-based VR has the potential to provide paralyzed people with the opportunity for easy communication.

EMG and EOG artifacts in brain computer interface systems: A survey

The modern commercial industry is increasingly dependent on the analysis of vast data records. Analysing big electroencephalogram (EEG) signal data (called brain signal data) plays an important role in a wide variety of applications such as healthcare practices, brain computer interface (BCI) systems, innovative education, privacy and security, and biometrics. The key objective of this paper is to establish a methodological framework for identifying communicative intentions of motor disabled people from EEG data for application in BCI systems. The proposed framework is designed based on common spatial pattern (CSP) data method and optimized ensemble (OE) machine learning method for the application of BCI technologies. The CSP method is used for discovering important features from EEG data and finally the extracted features are fed as an input to optimized ensemble (OE) method. The proposed method was tested on BCI Competition III dataset IVa, which contains motor imagery-based EEG signal data. The experimental results show that our proposed method can handle brain signal big data for identifying communicative intentions for an advanced BCI system. We compared the performance of our proposed method with several other existing methods. In comparison with other established methods, our method achieves higher classification accuracy performance. This research assists the experts in processing and analysing EEG signals for BCI applications. It also supports technologists to create a new EEG data analyser for BCI systems.KeywordsElectroencephalographyBrain computer interfaceCommon spatial patternMachine learning