The Effect of Haptic Feedback on Basic Social Interaction within Shared Virtual Environments

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user's sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application's Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.

EEG correlates of haptic feedback in a visuomotor tracking task

Stand-alone virtual environments (VEs) using haptic devices have proved useful for assembly/disassembly simulation of mechanical components. Nowadays, collaborative haptic virtual environments (CHVEs) are also emerging. A new peer-to-peer collaborative haptic assembly simulator (CHAS) has been developed whereby two users can simultaneously carry out assembly tasks using haptic devices. Two major challenges have been addressed: virtual scene synchronization (consistency) and the provision of a reliable and effective haptic feedback. A consistency-maintenance scheme has been designed to solve the challenge of achieving consistency. Results show that consistency is guaranteed. Furthermore, a force-smoothing algorithm has been developed which is shown to improve the quality of force feedback under adverse network conditions. A range of laboratory experiments and several real trials between Labein (Spain) and Queen's University Belfast (Northern Ireland) have verified that CHAS can provide an adequate haptic interaction when both users perform remote assemblies (assembly of one user's object with an object grasped by the other user). Moreover, when collisions between grasped objects occur (dependent collisions), the haptic feedback usually provides satisfactory haptic perception. Based on a qualitative study, it is shown that the haptic feedback obtained during remote assemblies with dependent collisions can continue to improve the sense of co-presence between users with regard to only visual feedback.

This study investigated the role of haptic feedback in precision peg insertion tasks conducted via teleoperation under varying visual resolution and communication latency conditions. Experiment 1 examined the combined effects of haptic feedback and the visual resolution, revealing that haptic feedback significantly reduces the maximum normal force and mental workload, while enhancing subjective operability, particularly in low-visual-resolution conditions. Experiment 2 evaluated the impact of communication latency, showing that the maximum normal force, operability, and mental workload are affected by increased latency. Notably, the maximum normal force is sensitive even to minimal latency (100 ms), whereas the mental workload and operability remain acceptable under lower-latency conditions. These findings underscore the importance of multi-metric evaluations, as different aspects of performance respond differently to latency. Overall, the results demonstrate the critical role of haptic feedback in enhancing task performance and the user experience in teleoperated precision tasks, offering valuable insights for the design and development of more effective and user-friendly teleoperation systems.

BackgroundVirtual simulations (VSs) enhance clinical competencies and skills. However, a previous systematic review of 9 RCT studies highlighted a paucity of literature on the effects of haptic feedback in surgical VSs. An updated systematic and scoping review was conducted to encompass more studies and a broader range of study methodologies.MethodsA systematic literature search was conducted on July 31, 2023, in MEDLINE, Embase, and Cochrane. English language studies comparing haptic vs non-haptic conditions and using VSs were included. Studies were evaluated and reported using PRISMA-ScR guidelines.ResultsOut of 2782 initial studies, 51 were included in the review. Most studies used RCT (21) or crossover (23) methodologies with medical residents, students, and attending physicians. Most used post-intervention metrics, while some used pre- and post-intervention metrics. Overall, 34 performance results from studies favored haptics, 3 favored non-haptics, and the rest showed mixed or equal results.ConclusionThis updated review highlights the diverse application of haptic technology in surgical VSs. Haptics generally enhances performance, complements traditional teaching methods, and offers personalized learning with adequate simulator validation. However, a sparsity of orienting to the simulator, pre-/post-study designs, and small sample sizes poses concerns with the validity of the results. We underscore the urgent need for standardized protocols, large-scale studies, and nuanced understanding of haptic feedback integration. We also accentuate the significance of simulator validation, personalized learning potential, and the need for researcher, educator, and manufacturer collaboration. This review is a guidepost for navigating the complexities and advancements in haptic-enhanced surgical VSs.

This paper reports on the effectiveness of haptic feedback in a low-cost virtual-reality training environment for military and emergency personnel. Subjects participated in simulated exercises for clearing a damaged building, implemented using a modified commercial videogame engine and USB-compatible force and vibration feedback devices. With the addition of haptic feedback, subjects made fewer procedural errors and completed some tasks more rapidly. Initially, best results were obtained with vibration feedback. After modifications to the collision detection and display algorithm, force feedback produced equivalent results and was preferred by a majority of subjects.

It is well known that the current state of mathematics education is problematic in many countries. The Interactive Learning Environments group at CID (Centre for user-oriented IT Design) at the Royal Institute of Technology (KTH) has developed an avatar-based shared virtual environment called CyberMath, aimed at improving this situation by allowing interaction with mathematical content in new and exciting ways. CyberMath is suitable for exploring and teaching mathematics in situations where both the teacher and the students are co-present and physically separated. In this virtual reality environment the participants are represented by avatars. The space concept in virtual environments is different from that of any known physical space. Yet people seem to perceive, for example, chat rooms and bulletin board systems as places. Still, avatars have limited possibilities for non-verbal expressions, such as body language, which are important in order to improve the communication quality. To investigate the importance of human-to-human expression and eye-contact between actors in the CyberMath environment, a test lecture in mathematics was carried out between students at Uppsala University and a lecturer at the Royal Institute of Technology (KTH) in Stockholm. The Media Environment group at the KTH Learning Lab was responsible for the production of a sense of presence involving the lecturer and the students, using distance technology such as networked two-way television systems and interactive storytelling. Empirical material was collected from recordings of 5 video sources and through a questionnaire given to the participants in the test. The main goal of the study was to investigate whether students at a distance could adapt to a combination of different shared virtual environments. It was found that presence production mediated as two-way television is a good way to build trust and to enhance non-verbal communication between the actors. The students treated the avatars on the computer screen and the lecturer on the display in front of them as real persons. In the same way, they treated the virtual reality space and the space distributed through two-way television as real spaces.Key wordsVirtual environmentinteractive learningvirtual realitymathematicsnonverbal expression

Inferring emotions from others’ non-verbal behavior is a pervasive and fundamental task in social interactions. Typically, real-life encounters imply the co-location of interactants, i.e., their embodiment within a shared spatial-temporal continuum in which the trajectories of the interaction partner’s Expressive Body Movement (EBM) create mutual social affordances. Shared Virtual Environments (SVEs) and Virtual Characters (VCs) are increasingly used to study social perception, allowing to reconcile experimental stimulus control with ecological validity. However, it remains unclear whether display modalities that enable co-presence have an impact on observers responses to VCs’ expressive behaviors. Drawing upon ecological approaches to social perception, we reasoned that sharing the space with a VC should amplify affordances as compared to a screen display, and consequently alter observers’ perceptions of EBM in terms of judgment certainty, hit rates, perceived expressive qualities (arousal and valence), and resulting approach and avoidance tendencies. In a between-subject design, we compared the perception of 54 10-s animations of VCs performing three daily activities (painting, mopping, sanding) in three emotional states (angry, happy, sad)—either displayed in 3D as a co-located VC moving in shared space, or as a 2D replay on a screen that was also placed in the SVEs. Results confirm the effective experimental control of the variable of interest, showing that perceived co-presence was significantly affected by the display modality, while perceived realism and immersion showed no difference. Spatial presence and social presence showed marginal effects. Results suggest that the display modality had a minimal effect on emotion perception. A weak effect was found for the expression “happy,” for which unbiased hit rates were higher in the 3D condition. Importantly, low hit rates were observed for all three emotion categories. However, observers judgments significantly correlated for category assignment and across all rating dimensions, indicating universal decoding principles. While category assignment was erroneous, though, ratings of valence and arousal were consistent with expectations derived from emotion theory. The study demonstrates the value of animated VCs in emotion perception studies and raises new questions regarding the validity of category-based emotion recognition measures.

Travel methods are the most basic and widespread interaction method with virtual environments. They are the primary and often the only way the user interactively experiences the environment. We present a study composed of three experiments that investigates how virtual collisions methods and feedback impact user perception of the realism of collisions and the virtual environment. A wand-based virtual travel method was used to navigate maze environments in an immersive projective system. The results indicated that the introduction of collision handling significantly improved the user's perception of the realism of the environment and collisions. An effect of feedback on the perceived level of realism of collisions and solidity of the environment was also found. Our results indicate that feedback should be context appropriate, e.g. fitting to a collision with the object; yet, the modality and richness of feedback were only important in that traditional color change feedback did not perform as well as audio or haptic feedback. In combination, the experiments indicated that in immersive virtual environments the stop collision handling method produced a more realistic impression than the slide method that is popular in games. In total, the study suggests that feedback fitting the collision context, coupled with the stop handling method, provides the best perceived realism of collisions and scene.

The use of Shared Virtual Environments is growing in areas such as multi-player video games, military and industrial training, and collaborative design and engineering. As a result, different mixes of computing power and graphics capabilities of the participating computers arise naturally as the variety of people/organizations sharing a virtual environment grows. This paper presents an adaptive mechanism to reduce bandwidth usage and to optimize the use of computing resources of heterogeneous computers mixes utilized in a shared virtual environment. The mechanism is based on caching of both outgoing- and incoming-messages. We also report the results of implementing the proposed scheme in a simple shared virtual environment.

Passing an object between two people is a common event that happens in various forms for example when giving someone a cup of coffee. An experimental study is presented where passing objects between two people in a virtual environment with haptic feedback was compared to passing objects in a nonhaptic virtual environment. The aim of the experiment was to investigate if and how added haptic feedback in such an environment affects perceived virtual presence, perceived social presence and perceived task performance. A within subject design was used, were nine pairs of subjects performed a hand off task with six differently sized cubes without audio communication. Results showed that haptic force feedback significantly improved perceived virtual presence, perceived social presence and perceived performance in this experiment.KeywordsHapticPresenceSocial presenceCollaborative

For most existing robot-assisted endovascular catheterization systems, the master interface takes the shape of a joystick or a haptic device, therefore potentially altering the natural behavior and motion patterns of physicians. Furthermore, existing studies on effective haptic feedback with robot-assisted endovascular catheterization system have been very limited. To address these issues, the paper proposes an MR fluids-based master haptic interface. The input catheter was utilized as the joystick, hence the physician can apply conventional motions (advance, retreat and rotate) on this interface. Meanwhile, the passive resistance force will be generated during the operating. The VeSEL (vertical-cavity surface-emitting laser) was used to measure the linear and rotational displacement of input catheter. The radial motion of the input catheter is detected by the hollow encoder. It was found that the mean accuracy of axial motion is 0.04 mm (precision, ±0.05 mm) for 100 mm length of advancing and retracting. And the considerable range of generated passive resistance force is from 28 mN to 1206 mN.

Co-presence has been considered as a very critical factor in shared virtual environments (SVEs). It is believed that by increasing the level of co-presence, the collaborative design performance in a shared virtual environment could be improved. The aim of this chapter is to reflect on the concept and characteristics of co-presence, by considering how Mixed Reality (MR)-mediated collaborative virtual environments could be specified, and therefore to provide distributed designers with a more effective design environment that improves the sense of “being there” and “being together”.

Slip feedback is an important cue in everyday object manipulation, but it is generally missing in tele-operation systems. To test the usefulness of simple, abstract types of feedback that warn the user about slip events, we tested the effect of auditory and haptic vibration feedback in a tele-operation task. Participants were asked to hold an object in a remote robot hand, and the force profiles that they exerted in response to slip events were measured. Haptic feedback did not significantly change the response characteristics, but auditory feedback did significantly improve response latency. A small but significant difference between haptic and auditory reaction times (60 ms) found in our control experiment might explain the difference between the feedback types.

Most prosthetic hand users are limited to visual feedback of movement performance. To characterize the benefit of vibrotactile feedback for a task that lacks haptic feedback, a virtual environment was used to experimentally manipulate visual, task-relevant haptic, and remote vibrotactile feedback on simple object manipulation for unimpaired subjects. The combination of visual and remote vibrotactile feedback was compared to visual feedback alone, and to simultaneous visual and direct haptic feedback to represent ideal performance. Visual and vibrotactile feedback resulted in improvement of most performance variables including difficulty ratings relative to visual feedback alone. However addition of sensory cues to visual feedback increased trial times and the increase was steeper for vibrotactile than for haptic feedback. Specifically, during vibrotactile feedback the velocity did not change, but the duration of execution increased due to improved performance, resulting in increased trial times. This result suggests future exploration of performance improvement and execution speed for augmented sensory feedback.