A comprehensive review on haptic feedback for minimally invasive surgery

This article gives an overview of research performed in the field of haptic information feedback during minimally invasive surgery (MIS). Literature has been consulted from 1985 to present. The studies show that currently, haptic information feedback is rare, but promising, in MIS. Surgeons benefit from additional feedback about force information. When it comes to grasping forces and perceiving slip, little is known about the advantages additional haptic information can give to prevent tissue trauma during manipulation. Improvement of haptic perception through augmented haptic information feedback in MIS might be promising.

Robot-assisted minimally invasive surgery (MIS) has countless benefits over open surgery, from shorter re- covery times and lower risk procedures for the patient to higher accuracy and broader capabilities for the surgeon [1]. However, a significant detriment to these procedures is that current systems lack haptic feedback. The lack of haptic feedback in MIS forces the surgeon to depend merely on visual cues, such as the deformation of tissue under load, to estimate the forces [1]. The likely outcome of misreading these cues is torn tissue, patient discomfort or broken sutures [2]. Moreover, haptic feed- back is specifically vital for robot-assisted endoscopy procedures. A recent study evaluating an Endoscopic Operation Robot (EOR) concluded that haptic feedback is beneficial in remote manipulation of flexible endo- scopes. When haptic feedback was absent there were more incidences of overstretching of sigmoid colon in a colonoscopy training model [3]. This work presents a soft robotic glove that provides haptic feedback for endoscopic procedures (Fig. 1, A). In our previous work, we introduced a soft robotic sleeve [4] that can detect forces between a colonoscope and colon walls during navigation. The glove receives force input from the soft robotic sleeve wrapped around the colonoscope (Fig. 1, B). Any incident force on the sleeve, during endoscopic navigation, is relayed to the surgeon as haptic feedback through proportional inflation of the glove’s pneumatic actuators.

Haptic feedback is of critical importance in surgical tasks. However, conventional surgical robots do not provide haptic feedback to surgeons during surgery. Thus, in this study, a combined tactile and kinesthetic feedback system was developed to provide haptic feedback to surgeons during robotic surgery. To assess haptic feasibility, the effects of two types of haptic feedback were examined empirically - kinesthetic and tactile feedback - to measure object-pulling force with a telesurgery robotics system at two desired pulling forces (1 N and 2 N). Participants answered a set of questionnaires after experiments. The experimental results reveal reductions in force error (39.1% and 40.9%) when using haptic feedback during 1 N and 2 N pulling tasks. Moreover, survey analyses show the effectiveness of the haptic feedback during teleoperation. The combined tactile and kinesthetic feedback of the master device in robotic surgery improves the surgeon's ability to control the interaction force applied to the tissue. Copyright © 2014 John Wiley & Sons, Ltd.

Teleoperated minimally invasive surgical robots can significantly enhance a surgeon's accuracy, dexterity and visualization. However, current commercially available systems do not include significant haptic (force and tactile) feedback to the operator. This paper describes experiments to characterize this problem, as well as several methods to provide haptic feedback in order to improve surgeon's performance. There exist a variety of sensing and control methods that enable haptic feedback, although a number of practical considerations, e.g. cost, complexity and biocompatibility, present significant challenges. The ability of teleoperated robot-assisted surgical systems to measure and display haptic information leads to a number of additional exciting clinical and scientific opportunities, such as active operator assistance through "virtual fixtures" and the automatic acquisition of tissue properties.

The use of virtual reality technology has become popular in modern theme parks across the world. Attractions using virtual reality technologies offer highly immersive experiences as if people are really staying in fictional worlds that the theme parks like to create. Haptic feedback is an important piece in virtual reality, where it can increase the immersion and enjoyment, but most virtual reality attractions in theme parks do not offer enough haptic feedback. In this paper, we present HapticDaijya, which is a waist-worn robot capable of giving various haptic and tactile feedback on the torso, neck, face, arms and hands. We present the design and implementation of HapticDaijya. Then, we show a user study and analyze the results for investigating its feasibility. Also, we present preliminary evaluation results that gauged the user’s accuracy in distinguishing the locations of taps applied on the chest, as well as general usability and user acceptance.

In minimally invasive surgery (MIS), haptic or force feedback control is used when the visual feedback is insufficient or unavailable. Haptics, or haptic technology, which is essential in simulating touch, provides both the kinesthetic and the tactile feedback. The lack of haptics in robotic-assisted MIS hinders the progression of operation when visual feedback reaches its limit. This paper aims to develop a single degree-of-freedom (DOF) haptic handle controller, using a proportional-integral (PI) control, that can quantify up to 1-N of force.

An important obstacle in Minimally Invasive Surgery (MIS) is the significant degradation of haptic feedback (sensation of touch) to the surgeon about surgical instrument s interaction with tissue. This monograph is concerned with devices and methods required for incorporating haptic feedback in master-slave robotic MIS systems. In terms of devices, novel mechanisms are designed including a surgical end-effector (slave) with full force sensing capabilities and a surgeon-robot interface (master) with full force feedback capabilities. Using the master-slave system, various haptic teleoperation control schemes are compared in terms of stability and performance, and passivity-based time delay compensation for haptic teleoperation over a long distance is investigated. The monograph also compares haptic feedback with visual feedback and with substitution for haptic feedback by other sensory cues in terms of surgical task performance. Contents: Introduction; Sensorized Surgical Effector (Slave); Haptic User Interface (Master); Unilateral Teleoperation Control; Bilateral Teleoperation Control; Substitution for Haptic Feedback: Bilateral Teleoperation Control Under Time Delay.

The challenges imposed by Minimally Invasive Surgery (MIS) have been the subject of significant research in the last decade. In the case of cancer surgery, a significant limitation is the inability to effectively palpate the target tissue to localize tumor nodules for treatment or removal. Current clinical technologies are still limited and tumor localization efforts often result in the need to increase the size of the incision to allow finger access for direct palpation. New methods of MIS tumor localization under investigation involve restoring the sense of touch, or haptic feedback. The two most commonly investigated modes of haptic perception include kinesthetic and tactile sensing, each with its own advantages and disadvantages. Work in this area includes the development of customized instruments with embedded sensors that aim to solve the problem of limited haptic feedback in MIS. This chapter provides a review of the work to date in the use of kinesthetic and tactile sensing information in MIS for tissue palpation, with the goal of highlighting the benefits and limitations of each mode when used to locate hidden tumors during MIS.

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user's sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application's Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.

Mid-air haptic (MAH) feedback is an interesting means to provide augmented haptic feedback for gesture-based technology as it enables a sense of touch without physical contact with an actuator. Although quite some work already investigated the user experience (UX) of MAH feedback during initial encounter, we are not aware of studies testing the UX after repeated use, with regard to both pragmatic and hedonic UX, as well as emotional reactions. In this article, we tested how the UX of MAH feedback changed over the course of five weeks by collecting both questionnaire as well as interview data of 31 participants. Our results showed that MAH feedback significantly increased the enjoyment, engagement, valence and arousal of the emotional response. However, the added value of valence was due to a novelty effect as it was only significantly elevated during initial use, and not after repeated use. Interestingly, the added value of MAH feedback in terms of enjoyment, engagement and arousal remained elevated over the course of five weeks. Moreover, the interview data hinted at substantial individual differences underlying the global trends from the questionnaire data, showing the importance of combining quantitative and qualitative data when testing the UX of MAH feedback.

Mid-air haptic interfaces have several advantages - the haptic information is delivered directly to the user, in a manner that is unobtrusive to the immediate environment. They operate at a distance, thus easier to discover; they are more hygienic and allow interaction in 3D. We validate, for the first time, in a preliminary study with sighted and a user study with blind participants, the use of mid-air haptics for conveying Braille. We tested three haptic stimulation methods, where the haptic feedback was either: a) aligned temporally, with haptic stimulation points presented simultaneously (Constant); b) not aligned temporally, presenting each point independently (Point-By-Point); or c) a combination of the previous methodologies, where feedback was presented Row-by-Row. The results show that mid-air haptics is a viable technology for presenting Braille characters, and the highest average accuracy (94% in the preliminary and 88% in the user study) was achieved with the Point-by-Point method.

The small scale of minimally-invasive surgery (MIS) presents significant challenges to developing robust, smart, and dexterous tools for manipulating millimeter and sub-millimeter anatomical structures (vessels, nerves) and surgical equipment (sutures, staples). Robotic MIS systems offer the potential to transform this medical field by enabling precise repair of these miniature tissue structures through the use of teleoperation and haptic feedback. However, this effort is currently limited by the inability to make robust and accurate MIS end effectors with integrated force and contact sensing. In this paper, we demonstrate the use of the novel Pop-Up Book MEMS manufacturing method to fabricate the mechanical and sensing elements of an instrumented MIS grasper. A custom thin-foil strain gage was manufactured in parallel with the mechanical components of the grasper to realize a fully-integrated electromechanical system in a single manufacturing step, removing the need for manual assembly, bonding and alignment. In preliminary experiments, the integrated grasper is capable of resolving forces as low as 30 mN, with a sensitivity of approximately 408 mV/N. This level of performance will enable robotic surgical systems that can handle delicate tissue structures and perform dexterous procedures through the use of haptic feedback guidance.

During traditional surgery, the surgeons’ hands are in direct contact with organs, and surgeons rely on the sense of touch to perform surgery. In teleoperated robotic systems, all physical connections between the surgeon and both the robot and patient, are absent. The surgeon must estimate the force exerted on organs, based only on visual deformation of tissues he is pulling, pushing, gripping, or suturing. It is hard to imagine how to operate with no haptic sensations, and it is surprising that commercially available robots didn’t include until now any Haptic Feedback, despite reports about tissue injury, and inability to perform complex manipulation. The sense of touch must be created by stimuli sensed by the surgeon. Haptic sensors are required to collect and send haptic information, and display them on the operator’s side, creating telepresence, known as transparency. Multiple ways have been developed to improve transparency through force feedback and tactile feedback. However, this interferes with the stability of the closed-loop controlling interactions between master, robot and remote environment. Cutaneous feedback is more stable and less transparent; force feedback is more transparent and less stable. Thus, multimodal platforms of haptic feedback would try to find the best trade-off between both modalities.

In this article, we present an application specific integrated circuit (ASIC) for gripper finger haptic force feedback in minimally invasive surgery (MIS). It consists of a driving current source, a sensing channel, a digital to analog converter (DAC), a power management unit (PMU), a clock generator and a digital control unit (DCU). The driving current source features a 6-bit DAC to provide a temperature-insensitive current from 0.27 mA to 1.15 mA for the sensor array. The sensing channel contains a programmable instrumentation amplifier (PIA), a low-pass filter (LPF), an incremental analog-to-digital converter (ADC) with its input buffer (BUF). The gain of the sensing channel ranges from 2.76 to 140. The DAC generates a tunable reference voltage to compensate possible sensor array offset. The input referred noise of the sensing channel is around 3.6 μVrms at a sampling rate of 850S/s. A custom 2-wire communication protocol is implemented to support two chips on gripper fingers operating in parallel with low latency, ensuring real-time surgical condition estimation for surgeons. Manufactured in the TSMC 180nm CMOS technology, this chip occupies only 1.37 mm2 core area, and the entire system requires only 4 wires (including power / ground) to operate. Combined with its high accuracy, low latency, and high integration level, this work allows real-time, high-performance haptic force feedback with compact system size, particularly suitable for MIS applications.

Minimally Invasive Surgeries (MIS) such as laparoscopic procedures are increasingly preferred over conventional surgeries due to many different advantages. Laparoscopic surgical procedures are very complex compared to open surgeries and require high level of experience and expertise. Hybrid surgery simulators available for training using physical phantoms are expensive and not readily available in majority of health care facilities around the world. Therefore, computer simulation or Virtual Reality (VR) is a better way to obtain skills for MIS. A VR simulator incorporated with haptic feedback provides a comprehensive training closer to real world experience. In this paper, we present a novel approach to incorporate force feedback to VR laparoscopic surgery training. The proposed interface incorporates force feedback in all three axes to provide three levels of force feedback. Computational models of abdomen organs were generated using the cryosection data of Visible Human Project of the National Library of Medicine, USA. The organ models were developed with three basic force categories: soft, mild and hard. A hardware interface is developed to provide the force feedback for the interaction of virtual tools with the said organ models while generating the tool navigation information for the VR simulator.