Relative to direct haptic feedback, remote vibrotactile feedback improves but slows object manipulation

We introduce a novel posture guidance office chair and evaluate the effectiveness of vibrotactile and visual feedback methods for guiding seated postures. For visually dominant office work such as typing on the computer, it is possible that delivering posture feedback visually may overload the visual sense while haptic feedback may be a viable alternative. We performed two experiments to compare vibrotactile and visual feedback--posture compliance and dual-task cognitive workload assessment. In the first experiment, our results showed no statistically significant difference in effectiveness between using vibrotactile and visual feedback to obtain postural compliance to a reference posture. In the second experiment, participants experienced typing performance and response time degradations from both types of feedback. However the differences in performance degradation were not statistically significant between the two feedback methods. We conclude that vibrotactile and visual feedback are similarly effective for guiding quasistatic postures in routine tasks such as seated office work.

We routinely have physical interactions with others, whether it be handing someone a glass of water or jointly moving a heavy object together. These sensorimotor interactions between humans typically rely on visual feedback and haptic feedback. Recent single-participant studies have highlighted that the unique noise and time delays of each sense must be considered to estimate the state, such as the position and velocity, of one's own movement. However, we know little about how visual feedback and haptic feedback are used to estimate the state of another person. Here, we tested how humans utilize visual feedback and haptic feedback to estimate the state of their partner during a collaborative sensorimotor task. Across two experiments, we show that visual feedback dominated haptic feedback during collaboration. Specifically, we found that visual feedback led to comparatively lower task-relevant movement variability, smoother collaborative movements, and faster trial completion times. We also developed an optimal feedback controller that considered the noise and time delays of both visual feedback and haptic feedback to estimate the state of a partner. This model was able to capture both lower task-relevant movement variability and smoother collaborative movements. Taken together, our empirical and modeling results support the idea that visual accuracy is more important than haptic speed to perform state estimation of a partner during collaboration.NEW & NOTEWORTHY Physical collaboration between two or more individuals involves both visual and haptic feedback. Here, we investigated how visual and haptic feedback is used to estimate the movements of a partner during a collaboration task. Our experimental and computational modeling results parsimoniously support the notion that greater visual accuracy is more important than faster yet noisier haptic feedback when estimating the state of a partner.

This research evaluated visual and haptic feedback for enhancing operator situational awareness during multitasking drone operations, investigating how task complexity influenced feedback effectiveness. Two virtual reality experiments were conducted: Study 1 assessed three types of visual feedback (Static, Blink, and Shake), while Study 2 compared visual, haptic, and visual-haptic multimodal feedback across varying task complexity levels determined by target density. Participants performed drone operation tasks requiring simultaneous obstacle avoidance and target counting. Results revealed that dynamic visual feedback (Shake) significantly improved primary task accuracy, reduced cognitive load, and was the preferred visual feedback type. Furthermore, haptic and visual-haptic feedback significantly shortened obstacle avoidance reaction times and decreased cognitive workload compared to visual feedback. Task complexity proved to be a critical factor: while visual feedback sufficed for low complexity scenarios, visual-haptic feedback offered distinct advantages in medium and high complexity tasks, providing insights for complexity-responsive drone interface design.

Prosthetic hand users may benefit from incorporation of sensory feedback, but the optimal methods of providing such feedback are still unknown. The effect on object manipulation of adding vibrotactile and pressure feedback at remote body sites to visual feedback was assessed in 8 participants using a virtual and robotic interface. Both pressure and vibrotactile feedback improved some aspects of task performance when compared to visual feedback alone; however, no difference in object manipulation performance was seen between pressure and vibrotactile feedback.

For educational purposes, virtual reality (VR) is often used to produce life-like experiences. However, the use of haptic feedback in educational practices for science and mathematics remained under-studied. Haptically-enabled science simulations (HESSs) enable students to physically experience the concepts being taught via haptic feedback. We present a study on the effect of a buoyancy HESS designed to aid in the understanding of basic physics concepts. We hypothesize that introducing both visual and haptic feedback of the underlying simulated forces will improve participants understanding. We investigate this hypothesis with a 2 (haptics: yes, no) × 2 (visuals: yes, no) between subjects design user study, where all participants were randomly assigned to one of the four conditions. Participants were given a pretest of buoyancy knowledge, then instructed to interact with the buoyancy simulation, then given a post-test of buoyancy knowledge. The present study is still in the process of data collection, with 40 out of 60 participants. Preliminary results highlight a significant improvement in performance of participants in the haptic-and-visual condition, while no significant differences were observed in other conditions.

The focus of this research is on the perception and control of forces produced by the hand. It is known that the change in force that subjects can reliably discriminate is approximately 6%, and that subjects are able to match forces generated by two corresponding muscle groups with an accuracy of approximately 15%. The present study was concerned with measuring the ability to control finger force using only haptic feedback and both haptic and visual feedback. It was found that subjects were surprising good at controlling forces for up to 120 s using only haptic feedback with mean absolute errors of less than 0.3 N over a force range of 2–6 N. The variation in force averaged 10.5% when haptic feedback was available and decreased to 6% when visual feedback was also provided. These results provide a framework for specifying the characteristics of force-reflecting haptic interfaces that should optimize human perception.

Spinal cord injury (SCI) impairs motor function and requires rigorous rehabilitative therapy, motivating the development of approaches that are engaging and customizable. Virtual reality (VR) motor training with augmented sensory feedback (ASF) offers a promising pathway to enhance functional outcomes, yet it remains unclear how ASF modalities affect performance and underlying psychophysiological states in persons with SCI. Five participants with chronic incomplete cervical-level SCI controlled a virtual robotic arm with semi-isometric upper-body contractions while undergoing ASF training with either visual feedback (VF) or combined visual plus haptic feedback (VHF). Motor performance (pathlength, completion time), psychophysiological measures (EEG, EMG, EDA, HR), and perceptual ratings (agency, motivation, utility) were assessed before and after ASF training. VF significantly reduced pathlength (−12.5%, p = 0.0011) and lowered EMG amplitude (−32.5%, p = 0.0063), suggesting the potential for improved motor performance and neuromuscular efficiency. VHF did not significantly improve performance, but trended toward higher cortical engagement. EEG analyses showed VF significantly decreased alpha and beta activity after training, whereas VHF trended toward mild increases. Regression revealed improved performance was significantly (p < 0.05) associated with changes in alpha power, EMG, EDA, and self-reported motivation. ASF type may differentially shape performance and psychophysiological responses in SCI participants. These preliminary findings suggest VR-based ASF as a potent multidimensional tool for personalizing rehabilitation.

The ability to control index finger and elbow flexion forces was measured while subjects used either haptic feedback or both haptic and visual feedback to control the forces exerted. Over a 120-s time period subjects were able to control the finger forces ranging from 2 to 6 N to within 1 N using only haptic feedback, and elbow flexion forces to within 4.5 N over a force range of 10-30 N. At the same force amplitude there was no significant difference between the two muscle groups in the precision or accuracy with which the force could be controlled, suggesting that there is not a proximal to distal gradient in force control as has been found for the control of limb movement and position.

This study investigates the influence of shoe type (sneakers and safety boots), age, and gender on the perception of haptic pulse feedback provided by a prototype accelerator pedal in a running stationary vehicle. Haptic feedback can be a less distracting alternative to traditionally visual and auditory in-vehicle feedback. However, to be effective, the device delivering the haptic feedback needs to be in contact with the person. Factors such as shoe type vary naturally over the season and could render feedback that is perceived well in one situation, unnoticeable in another. In this study, we evaluate factors that can influence the subjective perception of haptic feedback in a stationary but running car: shoe type, age, and gender. Thirty-six drivers within three age groups (≤39, 40-59, and ≥60) took part. For each haptic feedback, participants rated intensity, urgency, and comfort via a questionnaire. The perception of the haptic feedback is significantly influenced by the interaction between the pulse's duration and force amplitude and the participant's age and gender but not shoe type. The results indicate that it is important to consider different age groups and gender in the evaluation of haptic feedback. Future research might also look into approaches to adapt haptic feedback to the individual driver's preferences. Findings from this study can be applied to the design of an accelerator pedal in a car, for example, for a nonvisual in-vehicle warning, but also to plan user studies with a haptic pedal in general.

The perceptual deadband (PD) data reduction scheme allows for a significant reduction of network load in real-time haptic communication scenarios. Aiming to ascertain how the optimum PD data reduction parameter k is best determined, an experiment was conducted in which a possible interaction of PD data reduction and visual feedback on haptic feedback quality and task performance was investigated. The results show that when haptic feedback is complemented with visual feedback, haptic data reduction affects haptic feedback quality at a lower data reduction rate than it affects task performance. When visual feedback is missing, however, this effect is reversed. These results imply that the perception of haptic feedback quality is markedly influenced by visual sensory input as well as task requirements; hence, it is recommended to consider feedback quality and task performance in the optimization of PD reduction before such schemes may be applied to industrial telepresence and teleaction systems.

Visual vs vibrotactile feedback for posture assessment during upper-limb robot-aided rehabilitation

This paper proposes a new method of object manipulation in a Virtual Environment that uses a visual and haptic feedback based manipulation aid. If the displayed position of the object is constrained on a face of another object in the virtual environment, a user can place the virtual object at a precise position as easily as in a real environment. The visual feedback based manipulation aid constrains the motion of the virtual object as if it were moved in a real environment. Furthermore, in simultaneous correspondence with the visual constraints, the haptic feedback based manipulation aid constrains the motion of the user’s hand in the real environment. The proposed method introduces a magnetic metaphor: the user has the sensation of manipulating a virtual object on a rigid surface without using a large and powerful robot arm and the time consuming simulation of detailed physical phenomena. The properties of this manipulation aid based on two kinds of feedback are discussed. Experimental results show the effectiveness of our proposed method.

In this paper, we present a study that evaluates the effect of visual and haptic feedbacks and their relevance to human performance in a needle insertion task. A virtual needle insertion simulator with a four-layer tissue model (skin, fat, muscle, and bone) and haptic feedback is used for perceptual experiments. The task was to detect the puncture of a particular layer using haptic and visual cues provided by the simulation. The results show that the addition of force feedback reduces error in detection of transitions between tissue layers by at least 55%. The addition of real-time visual feedback (image overlay) improved user performance by at least 87% in scenarios without force feedback. Presentation of both force and visual feedback improved performance by at least 43% in comparison to scenarios without feedback. These results suggest that real-time image overlay provides greater improvement in performance than force feedback.

Controlling the motion of a prosthetic upper limb without visual feedback is extremely difficult because the wearer does not know the prosthesis' configuration. This paper describes an experiment designed to determine the relative importance of visual and haptic position feedback during targeted force-based motion by non-amputee human subjects as an analogy to prosthetic use. Subjects control the angle of a virtual proxy through an admittance relationship by generating torque at the MCP joint of the right index finger. During successive repetitions of a target acquisition task, the proxy's state is selectively conveyed to the user through graphical display, finger motion, and tactile stimulation. Performance metrics for each feedback condition will provide insights on the role of haptic position feedback and may help guide the development of future upper-limb prostheses

Camera drones allow exploring remote scenes that are inaccessible or inappropriate to visit in person. However, these exploration experiences are often limited due to insufficient scene information provided by front cameras, where only 2D images or videos are supplied. Combining a camera drone vision with haptic feedback would augment users' spatial understandings of the remote environment. But such designs are usually difficult for users to learn and apply, due to the complexity of the system and unfluent UAV control. In this paper, we present a new telepresence system for remote environment exploration, with a drone agent controlled by a VR mid-air panel. The drone is capable of generating real-time location and landmark details using integrated Simultaneous Location and Mapping (SLAM). The SLAMs' point cloud generations are produced using RGB input, and the results are passed to a Generative Adversarial Network (GAN) to reconstruct 3D remote scenes in real-time. The reconstructed objects are taken advantage of by haptic devices which could improve user experience through haptic rendering. Capable of providing both visual and haptic feedback, our system allows users to examine and exploit remote areas without having to be physically present. An experiment has been conducted to verify the usability of 3D reconstruction result in haptic feedback rendering.