Reducing Error Rates with Low-Cost Haptic Feedback in Virtual Reality-Based Training Applications

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user's sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application's Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.

This paper describes an experiment that studies the effect of basic haptic feedback in creating a sense of social interaction within a shared virtual environment (SVE). Although there have been a number of studies investigating the effect of haptic feedback on collaborative task performance, they do not address the effect it has in inducing social presence. The purpose of this experiment is to show that haptic feedback enhances the sense of social presence within a mediated environment. An experiment was carried out using a shared desktop based virtual environment where 20 remotely located couples who did not know one another had to solve a puzzle together. In 10 groups they had shared haptic communication through their hands, and in another group they did not. Hence the haptic feedback was not used for completing the task itself, but rather as a means of social interacting – communicating with the other participant. The results suggest that basic haptic feedback increases the sense of social presence within the shared VE.

Stand-alone virtual environments (VEs) using haptic devices have proved useful for assembly/disassembly simulation of mechanical components. Nowadays, collaborative haptic virtual environments (CHVEs) are also emerging. A new peer-to-peer collaborative haptic assembly simulator (CHAS) has been developed whereby two users can simultaneously carry out assembly tasks using haptic devices. Two major challenges have been addressed: virtual scene synchronization (consistency) and the provision of a reliable and effective haptic feedback. A consistency-maintenance scheme has been designed to solve the challenge of achieving consistency. Results show that consistency is guaranteed. Furthermore, a force-smoothing algorithm has been developed which is shown to improve the quality of force feedback under adverse network conditions. A range of laboratory experiments and several real trials between Labein (Spain) and Queen's University Belfast (Northern Ireland) have verified that CHAS can provide an adequate haptic interaction when both users perform remote assemblies (assembly of one user's object with an object grasped by the other user). Moreover, when collisions between grasped objects occur (dependent collisions), the haptic feedback usually provides satisfactory haptic perception. Based on a qualitative study, it is shown that the haptic feedback obtained during remote assemblies with dependent collisions can continue to improve the sense of co-presence between users with regard to only visual feedback.

The haptic interface or force feedback device improves the quality of human computer interaction by accommodating the sense of touch in a computer simulation. A haptic device enables a two way communication between the user and the host computer. The applications of force feedback device cover a wide area such as in medical training, systems for the visually impaired, engineering research, and creative art-work. The role of haptic feedback varies depending on the applications in question and the devices used. This includes haptic as a teaching tool, and a communication aid. Despite the benefits of haptic feedback, its deployment to support interactions in a virtual environment is still slow. This is due to the high cost of the haptic devices in the market. We address this issue by presenting a conceptual framework in designing and implementing a cost-effective haptic device; building on similar concepts obtained from existing technologies. The device is developed using those gadgets that are mainly available in-house. It is projected that haptic feedback generated from the device could benefit teaching abstract concepts to students.

EEG correlates of haptic feedback in a visuomotor tracking task

In this paper, we present a study that evaluates the effect of visual and haptic feedbacks and their relevance to human performance in a needle insertion task. A virtual needle insertion simulator with a four-layer tissue model (skin, fat, muscle, and bone) and haptic feedback is used for perceptual experiments. The task was to detect the puncture of a particular layer using haptic and visual cues provided by the simulation. The results show that the addition of force feedback reduces error in detection of transitions between tissue layers by at least 55%. The addition of real-time visual feedback (image overlay) improved user performance by at least 87% in scenarios without force feedback. Presentation of both force and visual feedback improved performance by at least 43% in comparison to scenarios without feedback. These results suggest that real-time image overlay provides greater improvement in performance than force feedback.

BackgroundVirtual simulations (VSs) enhance clinical competencies and skills. However, a previous systematic review of 9 RCT studies highlighted a paucity of literature on the effects of haptic feedback in surgical VSs. An updated systematic and scoping review was conducted to encompass more studies and a broader range of study methodologies.MethodsA systematic literature search was conducted on July 31, 2023, in MEDLINE, Embase, and Cochrane. English language studies comparing haptic vs non-haptic conditions and using VSs were included. Studies were evaluated and reported using PRISMA-ScR guidelines.ResultsOut of 2782 initial studies, 51 were included in the review. Most studies used RCT (21) or crossover (23) methodologies with medical residents, students, and attending physicians. Most used post-intervention metrics, while some used pre- and post-intervention metrics. Overall, 34 performance results from studies favored haptics, 3 favored non-haptics, and the rest showed mixed or equal results.ConclusionThis updated review highlights the diverse application of haptic technology in surgical VSs. Haptics generally enhances performance, complements traditional teaching methods, and offers personalized learning with adequate simulator validation. However, a sparsity of orienting to the simulator, pre-/post-study designs, and small sample sizes poses concerns with the validity of the results. We underscore the urgent need for standardized protocols, large-scale studies, and nuanced understanding of haptic feedback integration. We also accentuate the significance of simulator validation, personalized learning potential, and the need for researcher, educator, and manufacturer collaboration. This review is a guidepost for navigating the complexities and advancements in haptic-enhanced surgical VSs.

This study investigated the role of haptic feedback in precision peg insertion tasks conducted via teleoperation under varying visual resolution and communication latency conditions. Experiment 1 examined the combined effects of haptic feedback and the visual resolution, revealing that haptic feedback significantly reduces the maximum normal force and mental workload, while enhancing subjective operability, particularly in low-visual-resolution conditions. Experiment 2 evaluated the impact of communication latency, showing that the maximum normal force, operability, and mental workload are affected by increased latency. Notably, the maximum normal force is sensitive even to minimal latency (100 ms), whereas the mental workload and operability remain acceptable under lower-latency conditions. These findings underscore the importance of multi-metric evaluations, as different aspects of performance respond differently to latency. Overall, the results demonstrate the critical role of haptic feedback in enhancing task performance and the user experience in teleoperated precision tasks, offering valuable insights for the design and development of more effective and user-friendly teleoperation systems.

Clinical robotic surgery systems do not currently provide haptic feedback because surgical instrument interactions are difficult to measure and display. Our laboratory recently developed a technology that allows surgeons to feel and/or hear the high-frequency vibrations of robotic instruments as they interact with patient tissue and other tools. Until now, this type of feedback had not been carefully evaluated by users. We conducted two human-subject studies to discover whether surgeons and non-surgeons value the addition of vibration feedback from surgical instruments during robotic surgery. In the first experiment, 10 surgeons and 10 non-surgeons (n = 20) used an augmented Intuitive da Vinci Standard robot to repeatedly perform up to four dry-lab tasks both with and without haptic and audio feedback. In the second experiment, 68 surgeons and 26 non-surgeons (n = 94) tested the same robot at a surgical conference: each participant spent approximately 5 min performing one or two tasks. Almost all subjects in both experiments (95 and 98 %, respectively) preferred receiving feedback of tool vibrations, and all subjects in the second experiment thought it would be useful for surgeons to have the option of such feedback. About half of the subjects (50, 60 %) preferred haptic and audio feedback together, and almost all the rest (45, 35 %) preferred haptic feedback alone. Subjects stated that the feedback made them more aware of tool contacts and did not interfere with use of the robot. There were no significant differences between the responses of different subject populations for any questions in either experiment. This study illustrates that both surgeons and non-surgeons prefer instrument vibration feedback during robotic surgery. Some participants found audio feedback useful but most preferred haptic feedback overall. This strong preference for tool vibration feedback indicates that this technology provides valuable tactile information to the surgeon.

In recent times, there have been a number of computer user interface devices that have some haptic feedback functions, such as the DUALSHOCK controller [Sony Computer Entertainment, Inc. 1997] or the Wii Remote [Nintendo Co., Ltd. 2006]. However, their haptic feedback is limited to only the vibration function since there is no other method to provide haptic feedback that can be implemented with a small and inexpensive device. On the other hand, there is an increasing demand for realistic haptic feedback; thus, a simple and inexpensive method for a highly realistic haptic display is required. To meet this requirement, we propose a wearable haptic display to present the mass of a virtual object as shown in Figure 1. We focused on the mass of a virtual object, which contributes to the weight and the inertia mass in haptic interaction. If the virtual mass is presented by a haptic device, the user can perceive a more realistic sensation of the virtual object by grasping than the vibration feedback.

As an important part of virtual reality technology, haptic devices bring much better immersion and interactive to users. Haptic devices are continuously refined so they become more and more realistic with this technology's advancement. Multiple feedbacks, including the initial vibration feedback, advanced force feedback, electrotactile, and thermo-haptic feedback devices, are valuable because they help with complex surgical operations, create immersive gaming experiences, and provide practice-oriented education and training. This paper provides an in-depth analysis of the main applications of haptic devices into virtual reality. It is shown that while haptic feedback and user comfort have been widely applied in the field of highly realistic haptic education, many issues still have not been explored in other areas. Further exploration is needed in customization for specific user needs, the integration of multiple sensory feedback systems, and the reduction of latency to enhance the overall virtual experience. Research into lightweight, wearable haptic devices could also expand accessibility and usability across diverse fields.

For most existing robot-assisted endovascular catheterization systems, the master interface takes the shape of a joystick or a haptic device, therefore potentially altering the natural behavior and motion patterns of physicians. Furthermore, existing studies on effective haptic feedback with robot-assisted endovascular catheterization system have been very limited. To address these issues, the paper proposes an MR fluids-based master haptic interface. The input catheter was utilized as the joystick, hence the physician can apply conventional motions (advance, retreat and rotate) on this interface. Meanwhile, the passive resistance force will be generated during the operating. The VeSEL (vertical-cavity surface-emitting laser) was used to measure the linear and rotational displacement of input catheter. The radial motion of the input catheter is detected by the hollow encoder. It was found that the mean accuracy of axial motion is 0.04 mm (precision, ±0.05 mm) for 100 mm length of advancing and retracting. And the considerable range of generated passive resistance force is from 28 mN to 1206 mN.

When we use a tool to explore or manipulate an object, friction between the surface of the tool and the fingerpads generates skin stretch cues that are related to the interaction forces between the tool and the object. In this study, we emulate these naturally occurring skin stretch cues in order to convey force direction and magnitude information to users during teleoperation. We hypothesize that skin stretch feedback is a useful substitute for kinesthetic force feedback in force-sensitive teleoperated tasks. In this study, ten participants performed teleoperated palpation to determine the orientation of a stiff region in a surrounding artificial tissue using five feedback conditions: skin stretch, force, reduced gain force, graphic, and vibration. When participants received skin stretch feedback, they localized the stiff region as well as with force feedback, with no increase in task completion time. Additionally, participants receiving skin-stretch feedback localized the stiff region statistically significantly more accurately than those using vibration feedback. Although participants using skin stretch exhibited higher interaction forces than when using force, vibration, and graphical feedback, skin stretch statistically significantly decreased interaction forces compared with reduced gain force feedback. Thus, skin-stretch feedback is a compelling substitute for force feedback and may be useful in scenarios where force feedback is reduced or infeasible.

A swarm of robots is a large group of individual agents that autonomously coordinate via local control laws. Their emergent behavior allows simple robots to accomplish complex tasks. Since missions may have complex objectives that change dynamically due to environmental and mission changes, human control and influence over the swarm is needed. The field of Human Swarm Interaction (HSI) is young, with few user studies, and even fewer papers focusing on giving non-visual feedback to the operator. The authors will herein present a background of haptics in robotics and swarms and two studies that explore various conditions under which haptic feedback may be useful in HSI. The overall goal of the studies is to explore the effectiveness of haptic feedback in the presence of other visual stimuli about the swarm system. The findings show that giving feedback about nearby obstacles using a haptic device can improve performance, and that a combination of feedback from obstacle forces via the visual and haptic channels provide the best performance.

This chapter examines the use of haptic feedback in a multimedia simulation as a means of conveying information about physical science concepts. The case study presented herein investigates the effects of force feedback on children’s conceptions of gravity, mass, and related concepts following experimentation with a force-feedback-enabled simulation. Two groups of 17 children conducted experiments with the simulation; the experimental group used both visual and force feedback, and the control group used visual feedback only. Evidence of positive gains by the experimental group who used the simulation with force feedback is presented. Guidelines for applying these technologies effectively for educational purposes are discussed. This chapter adds to the limited research on the application of haptic feedback for conceptual learning and provides a basis for further research into the effects of computer-based haptic feedback on children’s cognition.