Methods for haptic feedback in teleoperated robot-assisted surgery.

The haptic interface or force feedback device improves the quality of human computer interaction by accommodating the sense of touch in a computer simulation. A haptic device enables a two way communication between the user and the host computer. The applications of force feedback device cover a wide area such as in medical training, systems for the visually impaired, engineering research, and creative art-work. The role of haptic feedback varies depending on the applications in question and the devices used. This includes haptic as a teaching tool, and a communication aid. Despite the benefits of haptic feedback, its deployment to support interactions in a virtual environment is still slow. This is due to the high cost of the haptic devices in the market. We address this issue by presenting a conceptual framework in designing and implementing a cost-effective haptic device; building on similar concepts obtained from existing technologies. The device is developed using those gadgets that are mainly available in-house. It is projected that haptic feedback generated from the device could benefit teaching abstract concepts to students.

Camera drones allow exploring remote scenes that are inaccessible or inappropriate to visit in person. However, these exploration experiences are often limited due to insufficient scene information provided by front cameras, where only 2D images or videos are supplied. Combining a camera drone vision with haptic feedback would augment users' spatial understandings of the remote environment. But such designs are usually difficult for users to learn and apply, due to the complexity of the system and unfluent UAV control. In this paper, we present a new telepresence system for remote environment exploration, with a drone agent controlled by a VR mid-air panel. The drone is capable of generating real-time location and landmark details using integrated Simultaneous Location and Mapping (SLAM). The SLAMs' point cloud generations are produced using RGB input, and the results are passed to a Generative Adversarial Network (GAN) to reconstruct 3D remote scenes in real-time. The reconstructed objects are taken advantage of by haptic devices which could improve user experience through haptic rendering. Capable of providing both visual and haptic feedback, our system allows users to examine and exploit remote areas without having to be physically present. An experiment has been conducted to verify the usability of 3D reconstruction result in haptic feedback rendering.

Haptic feedback provides an essential sensory stimulus crucial for interaction and analyzing three-dimensional spatio-temporal phenomena on surface visualizations. Given its ability to provide enhanced spatial perception and scene maneuverability, virtual reality (VR) catalyzes haptic interactions on surface visualizations. Various interaction modes, encompassing both mid-air and on-surface interactions-with or without the application of assisting force stimuli-have been explored using haptic force feedback devices. In this paper, we evaluate the use of on-surface and assisted on-surface haptic modes of interaction compared to a no-haptic interaction mode. A force-based haptic stylus is used for all three modalities; the on-surface mode uses collision based forces, whereas the assisted on-surface mode is accompanied by an additional snapping force. We conducted a within-subjects user study involving fundamental interaction tasks performed on surface visualizations. Keeping a consistent visual design across all three modes, our study incorporates tasks that require the localization of the highest, lowest, and random points on surfaces; and tasks that focus on brushing curves on surfaces with varying complexity and occlusion levels. Our findings show that participants took almost the same time to brush curves using all the interaction modes. They could draw smoother curves using the on-surface interaction modes compared to the no-haptic mode. However, the assisted on-surface mode provided better accuracy than the on-surface mode. The on-surface mode was slower in point localization, but the accuracy depended on the visual cues and occlusions associated with the tasks. Finally, we discuss participant feedback on using haptic force feedback as a tangible input modality and share takeaways to aid the design of haptics-based tangible interactions for surface visualizations.

Smooth haptic force feedback is an important task for multirate 6-DOF haptic rendering for both rigid and deformable objects. The update rate of the haptic force may be too low and changed during the simulation as the high computation time is required for complex physical simulation. Therefore, to implement a stable and smooth haptic rendering, we need a method to update the force in a higher regular rate during the physical simulation. We propose a prediction method using interpolation to calculate smooth haptic interaction force in a high update rate. The auto-regressive model is used to predict the force value based on the previous haptic force calculation. In addition, we introduce a spline function to interpolate force values for the haptic force output. We demonstrate that the proposed method can provide smooth and accurate haptic force feedback in a high update rate during a low frequency physical simulation of complex and deformable models. In the experiments, we show the feasibility of the proposed method and compare its accuracy and stability with the linear force prediction algorithm.

Passing an object between two people is a common event that happens in various forms for example when giving someone a cup of coffee. An experimental study is presented where passing objects between two people in a virtual environment with haptic feedback was compared to passing objects in a nonhaptic virtual environment. The aim of the experiment was to investigate if and how added haptic feedback in such an environment affects perceived virtual presence, perceived social presence and perceived task performance. A within subject design was used, were nine pairs of subjects performed a hand off task with six differently sized cubes without audio communication. Results showed that haptic force feedback significantly improved perceived virtual presence, perceived social presence and perceived performance in this experiment.KeywordsHapticPresenceSocial presenceCollaborative

Haptic feedback is of critical importance in surgical tasks. However, conventional surgical robots do not provide haptic feedback to surgeons during surgery. Thus, in this study, a combined tactile and kinesthetic feedback system was developed to provide haptic feedback to surgeons during robotic surgery. To assess haptic feasibility, the effects of two types of haptic feedback were examined empirically - kinesthetic and tactile feedback - to measure object-pulling force with a telesurgery robotics system at two desired pulling forces (1 N and 2 N). Participants answered a set of questionnaires after experiments. The experimental results reveal reductions in force error (39.1% and 40.9%) when using haptic feedback during 1 N and 2 N pulling tasks. Moreover, survey analyses show the effectiveness of the haptic feedback during teleoperation. The combined tactile and kinesthetic feedback of the master device in robotic surgery improves the surgeon's ability to control the interaction force applied to the tissue. Copyright © 2014 John Wiley & Sons, Ltd.

The use of virtual reality technology has become popular in modern theme parks across the world. Attractions using virtual reality technologies offer highly immersive experiences as if people are really staying in fictional worlds that the theme parks like to create. Haptic feedback is an important piece in virtual reality, where it can increase the immersion and enjoyment, but most virtual reality attractions in theme parks do not offer enough haptic feedback. In this paper, we present HapticDaijya, which is a waist-worn robot capable of giving various haptic and tactile feedback on the torso, neck, face, arms and hands. We present the design and implementation of HapticDaijya. Then, we show a user study and analyze the results for investigating its feasibility. Also, we present preliminary evaluation results that gauged the user’s accuracy in distinguishing the locations of taps applied on the chest, as well as general usability and user acceptance.

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user's sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application's Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.

In both teleoperation in real space and exploration in virtual space, ‘passive’ and ‘active’ haptic feedback can help to improve the performance of the task, especially in object handover and exploring. However, the current wearable haptic devices are hard to display continuous omnidirectional motion feedback simultaneously, which makes it not yet achieved. In this study, we thus propose a cutaneous haptic device, which enables continuous omnidirectional motion feedback for exhibiting ‘active’ and ‘passive’ haptic feedback. By applying small smart actuators (i.e., piezo actuators), the device can obtain contact force and be wearable. By arranging the closed loop with a plain-woven structure, our device makes continuous omnidirectional motion feedback possible. Our 35 g device can generate 0.94 N contact force and 0.5 N shear force. The passive and active haptic evaluations also proved its haptic capability. In conclusion, our proposed device with ‘active’ and ‘passive’ haptic feedback can provide continuous omnidirectional motion making it possible to be used for precise teleoperation.

In this paper, we present a study that evaluates the effect of visual and haptic feedbacks and their relevance to human performance in a needle insertion task. A virtual needle insertion simulator with a four-layer tissue model (skin, fat, muscle, and bone) and haptic feedback is used for perceptual experiments. The task was to detect the puncture of a particular layer using haptic and visual cues provided by the simulation. The results show that the addition of force feedback reduces error in detection of transitions between tissue layers by at least 55%. The addition of real-time visual feedback (image overlay) improved user performance by at least 87% in scenarios without force feedback. Presentation of both force and visual feedback improved performance by at least 43% in comparison to scenarios without feedback. These results suggest that real-time image overlay provides greater improvement in performance than force feedback.

Periodic Kinesthetic Guidance Cannot Expedite Learning Surgical Skills.

Sensory augmentation for subsea robot teleoperation

This paper outlines a study into the effects of co-location (the term `co-location' is used throughout to refer to the co-location of haptic and visual sensory modes, except where otherwise specified) of haptic and visual sensory modes in VR simulations. The study hypothesis is that co-location of these sensory modes will lead to improved task performance within a VR environment. Technical challenges and technological limitations are outlined prior to a description of the implementation adopted for this study. Experiments were conducted to evaluate the effect on user performance of co-located haptics (force feedback) in a 3D virtual environment. Results show that co-location is an important factor, and when coupled with haptic feedback the performance of the user is greatly improved.

Haptic feedback is important to improve the operability of virtual objects. However, AR systems with haptic feedback are still not many. Moreover, users cannot perform tasks that need large workspaces in those systems because of physical constraints from haptic feedback devices. In this paper, we combined a wearable haptic device and a marker-less AR technology to develop an AR system in which users can perform activities that need large workspaces. Differences of the operability of virtual objects in an assembly task of large machines are examined to evaluate effects of the haptic feedback in this system. The result indicates that haptic feedback is effective and users can perform activities that need large space in this system.

The objective of this article is to provide a thorough examination of the current state of haptic information feedback in minimally invasive surgery (MIS), including research conducted between 1985 and the present. Despite the fact that haptic information input in MIS is currently scarce, the comprehensive analysis emphasises that it represents a potential option. Surgeons might potentially get substantial benefits from receiving more input about force information, which would therefore enhance their surgical accuracy and control. The extant literature highlights a significant knowledge deficit regarding the intricate ramifications of haptic feedback, specifically with regard to the sense of slip and gripping forces. It is crucial to investigate the possible advantages of incorporating supplementary haptic information in order to protect against tissue damage when performing manipulation treatments. It is of the utmost importance to fill this void, since doing so might revolutionise the field of MIS via the mitigation of unintended tissue injury and the improvement of surgical results. By capitalising on technology progress to provide more extensive haptic feedback, surgeons have the ability to enhance their tactile perception, optimise the application of force, and reduce the likelihood of tissue injuries when performing complex surgical procedures.