Electrotactile Feedback in a Virtual Hand Rehabilitation Platform: Evaluation and Implementation

Humans systematically adapt their strategies for closed-loop control based on visual feedback according to the dynamics of the system. Tactile feedback is a key element in many human–machine interfaces, but it is not known if and how well human control adapts to changes in system dynamics when information about the system state is provided using this type of feedback. In this study, 11 participants tracked a pseudorandom trajectory with a virtual, position- or velocity-controlled plant using a joystick. Visual or electrotactile feedback provided the instantaneous error between the target and generated trajectory. Frequency-domain system identification indicated that human control adapted in similar ways to the different control modes (i.e., position/velocity control) for both feedback modalities. For the plant dynamics modeled as gain and integrator, the human controller behaved as a low-pass filter and gain, respectively (under the assumption of quasi-linear behavior). However, while tracking quality was largely similar for both control modes with visual feedback, velocity control enabled substantially worse control with electrotactile feedback compared to position control. Furthermore, for both control modes, the crossover frequency of open-loop transfer functions was lower for electrotactile feedback (0.9 and 1.1 rad/s) than for visual feedback (1.5 and 1.7 rad/s) indicating limited control bandwidth. To summarize, closed-loop control based on electrotactile feedback enables natural adaptations in human control strategy, which is encouraging for tactile feedback-controlled human–machine interfaces, but the lower control bandwidth and lower tracking quality with velocity control may impose functional limitations.

Tactile feedback from a remotely controlled robotic arm can facilitate certain tasks by enabling the user to experience tactile or force sensations from the robot's interaction with the environment. However, equipping both the robot and the user with tactile sensing and feedback systems can be complex, expensive, restrictive and application specific. This paper introduces a new tele-operation haptic feedback method involving electro-tactile feedback. This feedback system is inexpensive, easy to setup and versatile in that it can provide the user with a diverse range of tactile sensations and is suitable for a variety of tasks. We demonstrate the potential of our electro-tactile feedback system by providing experimental results showing how electro-tactile feedback from a tele-operated robotic arm equipped with range sensors can help with avoiding obstacles in cluttered workspace. We also show how interactive tasks, like placing a peg in a hole, can be facilitated with electro-tactile feedback from force sensors.

Virtual environments are often used in pre-wearing training and assessment of prosthetic control abilities. Here, we developed a virtual prosthetic hand training platform for evaluation of closed-loop control of grasp force. Biorealistic controllers emulated a pair of antagonistic muscles that actuated the thumb and index fingers of the hand. Surface electromyographic (sEMG) signals from a pair of antagonistic residual muscles drove the biorealistic controllers. Tactile forces from fingertip sensors were conveyed to amputees through evoked tactile sensations (ETS) elicited at the projected finger map (PFM) areas of the stump. A forearm amputee subject participated in force tracking or holding tasks using the virtual hand with residual muscle EMGs, or the contralateral intact hand. Root-mean-square error (RMSE) was used as outcome measure of motor performance. Results in this subject showed that the biorealistic controller enabled the virtual hand to track and maintain grasping forces. The best performance in both tasks was achieved by the contralateral intact hand with visual feedback. The roles of visual or tactile feedback in force tracking or maintaining were also assessed with the virtual hand. For force holding task, hybrid tactile and visual feedback with biorealistic control had a better performance than single visual or tactile feedback in terms of RMSE, success rate, and force variability. While in the force pursuing task, tactile feedback did not seem to add visual feedback in following the target force. The study suggests that training may be required for a novel virtual hand user to perceive and integrate multiple modalities of feedback information, so as to optimize the closed-loop control ability.

In this study we compared tactile and visual feedbacks for the motor imagery-based brain–computer interface (BCI) in five healthy subjects. A vertical green bar from the center of the fixing cross to the edge of the screen was used as visual feedback. Vibration motors that were placed on the forearms of the right and the left hands and on the back of the subject’s neck were used as tactile feedback. A vibration signal was used to confirm the correct classification of the EEG patterns of the motor imagery of right and left hand movements and the rest task. The accuracy of recognition in the classification of the three states (right hand movement, left hand movement, and rest) in the BCI without feedback exceeded the random level (33% for the three states) for all the subjects and was rather high (67.8% ± 13.4% (mean ± standard deviation)). Including the visual and tactile feedback in the BCI did not significantly change the mean accuracy of recognition of mental states for all the subjects (70.5% ± 14.8% for the visual feedback and 65.9% ± 12.4% for the tactile feedback). The analysis of the dynamics of the movement imagery skill in BCI users with the tactile and visual feedback showed no significant differences between these types of feedback. Thus, it has been found that the tactile feedback can be used in the motor imagery-based BCI instead of the commonly used visual feedback, which greatly expands the possibilities of the practical application of the BCI.

The rehabilitation on limb paralysis after stroke is an international scientific and technological issue. The rehabilitation in the early stage could hardly realize active participation of patients' intention, self-adaptive functional compensation and quantitative measurement on rehabilitation effect. This paper proposed a brain-computer interface (BCI) training system for rehabilitation of hand motor function after stroke. The mechanical hand mounted on patients' hand was driven by their intention through online motor imagery electroencephalogram (EEG) signal. The mechanical hand provided real-time adaptive assistance or resistance according to motor condition measured by transducers. The system displayed real-time and dynamic information on EEG, hand force and angle synchronously for rehabilitation evaluation and program optimization. The system was tested on three healthy subjects. An average accuracy and information transfer rate of 89.7% and 0.5099bit/s were obtained respectively. The system provides new idea and approach for the rehabilitation training of stroke patients. In follow-up work, we will test our system in long-term rehabilitation experiment for stroke patients to evaluate the effectiveness on improving hand movement and neural system activation.

In 2016, the number of patients with stroke is increasing in Japan. Stroke survivor's rate has doubled from 1960 to 2004 in Japan. Even though surviving from stroke, patients usually have paralyses or impairment. In recent years, new rehabilitation system with virtual reality system has been reported. It has a possibility to improve occupational therapist tasks. In the previous study, virtual reality rehabilitation system had an effect in Box and Block Test. However, a movement in virtual reality system rehabilitation was not measured quantitatively and it was not compared with real conventional rehabilitation. Rehabilitation with virtual reality system is also required to make patients same movement of conventional rehabilitation. By comparing virtual reality test motion with conventional test motion, VR device experiment has larger 30 degrees in average bending angles than imitation device performing and we examine usefulness of rehabilitation with virtual reality. This study focuses on bending motion of the middle fingers when grasping a box. To compare them, a measurement system which can measure the grasping angle was developed based on six-axis inertial sensors. Consequently, difference between two tests was observed in bending motion. We consider it is relating with physical feedback.

Acquired brain injury (ABI) is one of the main problems of disability and death in the world. Its incidence and survival rate are increasing annually. Thus, the number of chronic ABI patients is gradually growing. Traditionally, rehabilitation programs are applied to postacute and acute patients, but recent publications determine that chronic patients may benefit from rehabilitation. Also, in the last few years, the potential of virtual rehabilitation (VR) systems has been demonstrated. However, until now, no previous studies have been carried out to compare the evolution of chronic patients with acute patients in a VR program. To perform this study, we developed a VR system for ABI patients. The system, vestibular virtual rehabilitation (V2R), was designed with clinical specialists. V2R has been tested with 21 people ranging in age from 18 to 80years old that were classified in two groups: chronic patients and acute patients. The results demonstrate a similar recovery for chronic and acute patients during the intervention period. Also, the results showed that chronic patients stop their improvement when they finish their training. This conclusion encourages us to direct our developments toward VR systems that can be easily integrated at home, allowing chronic patients to have a permanent VR training program.

The virtual rehabilitation system combining virtual rehabilitation environment and upper limb rehabilitation technology is interactive and interesting, which can improve the enthusiasm and initiative of patients for rehabilitation training, improve the efficiency of rehabilitation training and improve the effect of rehabilitation treatment. This paper firstly conducts in-depth research and analysis on the research progress of the upper limb rehabilitation robot system, and deeply studies the principle and rehabilitation principle of the stroke caused by hemiplegic dyskinesia, and summarizes the goals and methods of the upper limb rehabilitation system design. Secondly, the hand motion tracking is realized by Kinect’s bone tracking, and the optimal tracking distance is determined experimentally, which verifies the stability and robustness of the tracking. Static gesture recognition adopts two gesture recognition schemes based on Kinect depth image and color space model respectively. Finally, using the rehabilitation robot and Kinect sensor as the hardware platform, the virtual rehabilitation training system experimental platform is constructed, and the horizontal rehabilitation exercise and the three-dimensional space rehabilitation exercise are respectively studied experimentally, and the exercise data obtained by using healthy subjects as the experimental object is analyzed. Based on this, the validity and feasibility of the Kinect-based upper limb exercise rehabilitation interactive system were verified.

While user's perception and performance are predominantly examined independently in virtual reality, the Action-Specific Perception (ASP) theory postulates that the performance of an individual on a task modulates this individual's spatial and time perception pertinent to the task's components and procedures. This paper examines the association between performance and perception and the potential effects that tactile feedback modalities could generate. This paper reports a user study (N=24), in which participants performed a standardized Fitts's law target acquisition task by using three feedback modalities: visual, visuo-electrotactile, and visuo-vibrotactile. The users completed 3 Target Sizes × 2 Distances × 3 feedback modalities = 18 trials. The size perception, distance perception, and (movement) time perception were assessed at the end of each trial. Performance-wise, the results showed that electrotactile feedback facilitates a significantly better accuracy compared to vibrotactile and visual feedback, while vibrotactile provided the worst accuracy. Electrotactile and visual feedback enabled a comparable reaction time, while the vibrotactile offered a substantially slower reaction time than visual feedback. Although amongst feedback types the pattern of differences in perceptual aspects were comparable to performance differences, none of them was statistically significant. However, performance indeed modulated perception. Significant action-specific effects on spatial and time perception were detected. Changes in accuracy modulate both size perception and time perception, while changes in movement speed modulate distance perception. Also, the index of difficulty was found to modulate all three perceptual aspects. However, individual differences appear to affect the magnitude of action-specific effects. These outcomes highlighted the importance of haptic feedback on performance, and importantly the significance of action-specific effects on spatial and time perception in VR, which should be considered in future VR studies.

BackgroundMyoelectric interfaces have emerged as powerful tools for human–machine interaction (HMI), enabling intuitive control of virtual and physical devices. However, most existing systems are limited by low spatial resolution and unidirectional communication. To address these limitations, we developed NeuraLoop, a wearable, high-bandwidth, bidirectional interface that integrates myoelectric (EMG) signal acquisition and electrotactile stimulation feedback within a single wearable textile-based platform.MethodsNeuraLoop comprises a flexible matrix of 32 EMG recording and 32 electrotactile stimulation pads controlled by a compact electronic unit. We evaluated the system in two experimental tasks involving ten healthy subjects to demonstrate: (1) online classification of four transient thumb micro-gestures (thumb rightwards, leftwards, upwards, and downwards swipe directions), and (2) closed-loop control of a virtual cursor using micro-gesture commands and spatially encoded tactile feedback. A time-division multiplexing (TDM) strategy was implemented to enable simultaneous stimulation and recording.ResultsThe subjects achieved a median success rate of 82% on the first attempt and over 94% within two attempts during online classification with visual feedback. All four micro-gestures were classified with similar accuracy. In the closed-loop control task with tactile feedback, participants navigated a 3 × 4 grid using only electrotactile stimulation, achieving 70% accuracy for exact target hits and 95% when including the hits in the neighboring cells (1 cell distance error).ConclusionsNeuraLoop demonstrates the feasibility of high-bandwidth, bidirectional HMI using a wearable, textile-based interface. The system enables accurate recognition of subtle micro-gestures and effective delivery of spatially encoded tactile feedback. These capabilities open new possibilities for intuitive control in applications such as prosthetics, rehabilitation, and virtual/augmented reality. Future work will explore multimodal feedback encoding and proportional gesture control.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12984-025-01815-5.

Utilizing hierarchical redundancy for accurate throwing movement.

Virtual reality can help to learn basic laparoscopic tasks. However, no haptic feedback, which alerts for tissue slippage, is provided by most simulators, although, it might be of influence for the decrease of errors. This study explored whether visual or tactile feedback can be used to alert the surgeon of tissue slippage. Twenty-four participants performed a laparoscopic grasping task and where provided with either visual or tactile feedback about tissue slippage. The reaction time with the visual feedback was compared to the reaction time with tactile feedback signal. The results showed that when tissue slippage is simulated, tactile feedback shows significant faster reaction times (269ms) than visual feedback signals (398ms).

This paper describes development of an activity based, multimodal balance rehabilitation training device. Various sensors can be used, including a force plate, inertial sensors, and depth sensing cameras, and various combinations of visual, auditory and tactile feedback can be configured depending on the rehabilitation task and activity. Tactile feedback is presented via a lightweight belt that is worn on the torso. Generally, visual feedback is only needed at the start of rehabilitation training (task orientation) while tactile feedback may be used to augment balance control. Tactile feedback can be configured as a cue that certain movement targets or limits have been reached or as an immediate indicator of the variance in postural sway. Tactile feedback allows the subject to naturally concentrate on the functional rehabilitation task, and is less reliant on visual or verbal cues.

It is evident that user training significantly affects performance of pattern-recognition-based myoelectric prosthetic device control. Despite plausible classification accuracy on offline datasets, online accuracy usually suffers from the changes in physiological conditions and electrode displacement. The user ability in generating consistent electromyographic (EMG) patterns can be enhanced via proper user training strategies in order to improve online performance. This study proposes a clustering-feedback strategy that provides real-time feedback to users by means of a visualized online EMG signal input as well as the centroids of the training samples, whose dimensionality is reduced to minimal number by dimension reduction. Clustering feedback provides a criterion that guides users to adjust motion gestures and muscle contraction forces intentionally. The experiment results have demonstrated that hand motion recognition accuracy increases steadily along the progress of the clustering-feedback-based user training, while conventional classifier-feedback methods, i.e., label feedback, hardly achieve any improvement. The result concludes that the use of proper classifier feedback can accelerate the process of user training, and implies prosperous future for the amputees with limited or no experience in pattern-recognition-based prosthetic device manipulation.It is evident that user training significantly affects performance of pattern-recognition-based myoelectric prosthetic device control. Despite plausible classification accuracy on offline datasets, online accuracy usually suffers from the changes in physiological conditions and electrode displacement. The user ability in generating consistent electromyographic (EMG) patterns can be enhanced via proper user training strategies in order to improve online performance. This study proposes a clustering-feedback strategy that provides real-time feedback to users by means of a visualized online EMG signal input as well as the centroids of the training samples, whose dimensionality is reduced to minimal number by dimension reduction. Clustering feedback provides a criterion that guides users to adjust motion gestures and muscle contraction forces intentionally. The experiment results have demonstrated that hand motion recognition accuracy increases steadily along the progress of the clustering-feedback-based user training, while conventional classifier-feedback methods, i.e., label feedback, hardly achieve any improvement. The result concludes that the use of proper classifier feedback can accelerate the process of user training, and implies prosperous future for the amputees with limited or no experience in pattern-recognition-based prosthetic device manipulation.

Virtual rehabilitation should be adaptable to the patient need and progress. To do so, patient in-game performace and ability are monitored to maintain an adequate level of challenge. A novel adaptation strategy is proposed by which patient control and speed are dynamically interrogated to adjust the game difficulty. The strategy is based on a Markov decision process seeding a therapist-guided reinforcement learning algorithm. The optimal learning scheme for the algorithm is established (α = 0.5). Convergence to an optimal therapeutic plan is demonstrated for patients with non-deterministic behaviour. The proposed adaptation algorithm can enhance existing virtual reality-based motor rehabilitation platforms by tailoring the games response to the patient changing needs.