A Design of Hardware Haptic Interface for Gastrointestinal Endoscopy Simulation

Vitreo-retinal surgery encompasses the surgical procedures performed on the vitreous humor and the retina. A procedure typically consists of the removal of the vitreous humor, the peeling of a membrane and/or the repair of a retinal detachment. Operations are performed with needle shaped instruments which enter the eye through surgeon made scleral openings. An instrument is moved by hand in four degrees of freedom (three rotations and one translation) through this opening. Two rotations (? and ? ) are for a lateral instrument tip movement. The other two DoFs (z and ?) are the translation and rotation along the instrument axis. Actuation of for example a forceps can be considered as a fifth DoF. Characteristically, the manipulation of delicate, micrometer range thick intraocular tissue is required. Today, eye surgery is performed with a maximum of two instruments simultaneously. The surgeon relies on visual feedback only, since instrument forces are below the human detection limit. A microscope provides the visual feedback. It forces the surgeon to work in a static and non ergonomic body posture. Although the surgeon’s proficiency improves throughout his career, hand tremor may become a problem around his mid-fifties. Robotically assisted surgery with a master-slave system enhances dexterity. The slave with instrument manipulators is placed over the eye. The surgeon controls the instrument manipulators via haptic interfaces at the master. The master and slave are connected by electronic hardware and control software. Implementation of tremor filtering in the control software and downscaling of the hand motion allow prolongation of the surgeon’s career. Furthermore, it becomes possible to do tasks like intraocular cannulation which can not be done by manually performed surgery. This thesis focusses on the master console. Eye surgery procedures are observed in the operating room of different hospitals to gain insight in the requirements for the master. The master console as designed has an adjustable frame, a 3D display and two haptic interfaces with a coarse adjustment arm each. The console is mounted at the head of the operating table and is combined with the slave. It is compact, easy to place and allows the surgeon to have a direct view on and a physical contact with the patient. Furthermore, it fits in today’s manual surgery arrangement. Each haptic interface has the same five degrees of freedom as the instrument inside the eye. Through these interfaces, the surgeon can feel the augmented instrument forces. Downscaling of the hand motion results in a more accurate instrument movement compared to manually performed surgery. Together with the visual feedback, it is like the surgeon grasps the instrument near the tip inside the eye. The similarity between hand motion and motion of the instrument tip as seen on the display results in an intuitive manipulation. Pre-adjustment of the interface is done via the coarse adjustment arm. Mode switching enables to control three or more instruments manipulators with only two interfaces. Two one degree of freedom master-slave systems with force feedback are built to derive the requirements for the haptic interface. Hardware in the loop testing provides valuable insights and shows the possibility of force feedback without the use of force sensors. Two five DoF haptic interfaces are realized for bimanual operation. Each DoF has a position encoder and a force feedback motor. A correct representation of the upscaled instrument forces is only possible if the disturbance forces are low. Actuators are therefore mounted to the fixed world or in the neighborhood of the pivoting point for a low contribution to the inertia. The use of direct drive for ' and and low geared, backdriveable transmissions for the other three DoFs gives a minimum of friction. Disturbance forces are further minimized by a proper cable layout and actuator-amplifier combinations without torque ripple. The similarity in DoFs between vitreo-retinal eye surgery and minimally invasive surgery (MIS) enables the system to be used for MIS as well. Experiments in combination with a slave robot for laparoscopic and thoracoscopic surgery show that an instrument can be manipulated in a comfortable and intuitive way. User experience of surgeons and others is utilized to improve the haptic interface further. A parallel instead of a serial actuation concept for the ' and DoFs reduces the inertia, eliminates the flexible cable connection between frame and motor and allows that the heat of the motor is transferred directly to the frame. A newly designed z-?? module combines the actuation and suspension of the hand held part of the interface and has a three times larger z range than in the first design of the haptic interface.

Many high-degree-of-freedom haptic devices and teleoperator systems either do not have grippers or do not provide force feedback in the gripper degree of freedom (DOF). The purpose of this work is to determine the effect of gripper force feedback in relation to Cartesian (translational) force feedback on the execution of telemanipulation tasks. We developed a system for adding an additional DOF of grip force feedback on a 3-DOF Phantom haptic device master, as well as 6-DOF force/torque sensing on each finger of a coupled gripper on a Phantom haptic device slave. The internal (grip) and external (translational) forces were measured as users performed a soft peg-in-hole task with various DOFs of force feedback: (1) full force feedback, (2) translational force feedback only, (3) grip force feedback only, and (4) no force feedback. Results show that the level of force applied in the translational and gripping DOFs are decoupled for a 3-DOF telemanipulator with added grip force feedback. This is likely due to the decoupled dynamics of internal and external hand forces

Haptic User Interface for a Telerobotic System – Design based on a Network Model

Application research of master-slave cranio-maxillofacial surgical robot based on force feedback

Multifinger haptic interface is the ultimate human haptic interface that would provide natural manipulation and feeling to the tip of each finger, same as the real hand interaction with the real world in daily-life tasks. This chapter introduced the second version of HIRO, including the hardware design, the control algorithm, and the software design including the system architecture, haptic rendering, and practical VR application. The hand with the arm provides a large workspace and reflects exactly the human hand movement as you are looking to your hand and arm in the mirror, with some differences related to hardware limitations and control methods that yet need to be improved. A comprehensive haptic system through demonstration of the future haptic science encyclopedia is presented. The architecture used was a multi-layer network framework that decoupled the haptic from graphic on to separate machines. The architecture maintained the high update rate needed for the haptic servoloop. Also, an accurate collision detection algorithm that considers the shape of human fingertips is developed. A physically based modeling using the finite element method (FEM) combined with an elementary displacement approach proved to ensure stable interaction and realistic forces calculation for a relatively complex deformable model in real time. The approach takes into considerations interaction with multipoint contacts where other approaches failed to achieve such interaction due difficulty in calculating forces at 1 KHz. The FHSE application was demonstration at World Expo 2005. People enjoyed the demonstration, where the majority were satisfied with the system as it provided a unique experience. Finally, combining technologies such as virtual reality and haptic robotics enables us to extend the capabilities and effectiveness of simulators. We anticipate that the proposed system will bring the notice of many researchers to the feasibility of implementing multifinger haptic interfaces. Although, every one agree on the importance of such interface, but many obstacles are still preventing the advancement of this research field. This chapter provided a complete picture on how this could be achieved. The greatest lesson of such project was the collaboration between robotics community and virtual reality community is necessary to achieve a better and comprehensive sophisticated haptic robot. Eventhough , I was the leader of the software development side including the haptic rendering and VR application, I was involved from the beginning in the process of designing the hardware and the control algorithms as well, useful feedback was the result of such collaboration. There are a number of future directions that can apply to this research. First, it is desirable to test the presented architecture in the presence of large delay and apply some techniques to compensate it. Second, develop more robust control algorithm for the arm movements to reflect the exact motion of the human hand, this will make the interface manipulation very natural. Also, make the hardware itself smarter and safer.

In addition to the visual and auditory feedbacks, the force and torque feedbacks will definitely make the virtual environment more immersive. This idea has encouraged the researchers to go through the non-ground-based haptic interfaces containing these capabilities. In this paper, a new ungrounded haptic interface will be developed which can provide real sense of linear and angular momentum in three-dimensional space. Incorporation of a pair of propellers into the haptic interface will make it to continuously produce the desired forces and torques in less than 350 ms up to 15 N and 1 N.m, respectively. To orient the produced forces and torques in the desired and arbitrary directions in the three-dimensional space, these propellers are mounted on a set of sliders and gimbal rings. Due to the simple structure and effortless control of the proposed mechanism, rooted in the developed novelty in the mechanism design, the haptic interface can be more practical for the illusion of complex tasks in the daily uses. In this paper, the detailed design and force analysis of the haptic interface are followed by analytically derived kinematic and dynamic model, verified in MATLAB/Simscape Multibody. Then, the mathematical framework for the proposed controller is addressed, and afterward the robust stability analysis in the presence of structured and unstructured uncertainties is investigated. Eventually, a measuring prototype has been designed and manufactured to experimentally evaluate the performance of the haptic device and proposed control algorithm. The experiments show good performance of the proposed system and controller in the tracking of the various forces and torques on different paths.

A haptic interface is a device for operators to touch, feel and manipulate virtual environments. It senses the contact forces and gives operators a sensation of interacting with a real physical environment. Force feedback can improve the quality of the haptic feedback control in virtual reality applications. In this paper, a real time haptic interface is developed via control models in the Labview virtual environment, which is a combination of a touch force measurement haptic interface and a vibro-tactile CyberGlove serial haptic interface. The haptic interface is a prototype for real time feedback control on robotic integration of smart censors. This approach is easy to apply on various types of sensors, such as acoustic wave sensors, ultrasound array, etc., which will enhance the ability to communicate between the operators and the virtual environment. The flexiforce sensor and tactile sensor based haptic interface in force feedback systems will improve traditional robot visual sensing systems by a virtual feel.

In this paper the design of a multi-finger force-reflecting haptic interface device for teleoperational grasping is introduced. The haptic interface or “master” controller device is worn on the human operator’s hand and measured human finger positions are used to control the finger positions of a remote grasping manipulator or “slave” device. The slave may be a physical robotic grasping manipulator, or a computer generated representation of a human hand such as used in virtual reality applications. The forces measured by the robotic slave, or calculated for the virtual slave, are presented to the operator’s fingertips through the master providing a means for deeper human sensation of presence and better control of grasping tasks in the slave environments. Design parameters and performance measures for haptic interfaces for teleoperation are discussed. One key performance issue involving the high-speed display of forces during initial contact, especially when interacting with rigid surfaces, is addressed by the present design, reducing slave controller computation requirements and overcoming actuator response time constraints. The design presented utilizes a planar four-bar linkage for each finger, to represent each finger bend motion as a single degree of freedom, and to provide a finger bend resistance force that is substantially perpendicular to the distal finger pad throughout the full 180 degrees of finger bend motion represented. The finger linkage design, in combination with a remote position measurement and force display assembly, provides a very lightweight and low inertia system with a large workspace. The concept of a replicated finger is introduced which, in combination with a decoupled actuator and feed forward control, provides improved performance in transparent free motion, and rapid, stable touch sensation of initial contact with rigid surfaces. A distributed computation architecture with a PC based haptic interface controller and associated control algorithms are also discussed.

This paper proposes a nonlinear virtual coupling fur haptic interface, which offers better performance while maintaining stability of the system. The nonlinear virtual coupling is designed based on a human response model. This human response model exploits delay between the human Intention and the actual change of arm impedance. The proposed approach provides with less conservative constraints for the design of stable haptic interface, compared with the traditional passivity condition. This allows increased performance that is verified through experiments.

A new portable device for haptic interaction with virtual environments is presented. It is a lightweight interface for the fingertips, designed for providing cutaneous feedback and displaying the contact - non contact transition in highly immersive virtual environments. In this paper the kinematics, the mechanical design and the control system are described. The device has been mounted on a kinesthetic haptic interface which tracks its position: the overall system can provide both cutaneous and kinesthetic feedback. Finally the performance of the portable device in a simple task of shape exploration has been evaluated and compared to a kinesthetic haptic interface.

In this study, the developed 7-DOF haptic device (HaP-7) and its virtual reality interface are presented. The design issues, design constraints, and alternative design configurations are discussed and the potential advantages of the HaP-7 are put forward. The kinematic model of the proposed device looks like a simplified human arm kinematic model. The redundant characteristic of the device provides larger workspace and allows for appropriate posture selection for the purposes of maximization of the rigidity, transparency and stability, while minimizing the inertia and power consumption in addition to the singularity and obstacle avoidance optimization.

We discuss the possibility of improving eyehand coordination in children diagnosed with this problem, using a robotic mapping from a haptic user interface to a virtual environment. Our goal is to develop an assessment and training procedure that will result in improving handwriting taking the advantage of the force feedback provided by the haptic device. Force feedback can be used to guide the subject's hand in a predetermined trajectory when he/she is unable to move in response to visual feedback. We also incorporate the inertia and viscosity effects to decrease the tremor in the hand as well as to stimulate the muscles involved in the task of holding a pencil (known as facilitation technique in the occupational therapy field). The robotic system is operated within ranges established by baseline data for accuracy of eye-hand coordination and grip strength. A set of assessment tests, commonly used by occupational therapists, were chosen to implement the various functions using force, inertia and viscosity effects. The test bed used for these tasks consisted of a six-degree-of-freedom force reflecting haptic interface device, PHANToM with the GHOST SDK Software. Preliminary data obtained from two children diagnosed with eye-hand coordination problems show that the assistance provided, improved their ability to execute and complete the tasks, suggesting that the system could be used for assessment and training purposes. In addition, these results suggest that the introduction of the haptic rendering capabilities, including the force feedback, offers special benefit and measures the impact on the occupation of handwriting. Furthermore, a real application of this system could be implemented in the future, by identifying specific job tasks that may be made accessible through a desktop in classrooms by the aforementioned haptic system.

Interactions with virtual worlds using the sense of touch, called haptic rendering, have natural applications in many domains such as health or industry. For an accurate and realistic haptic feedback, the haptic device must receive orders at high frequencies, especially to render stiff contacts between rigid bodies. Therefore, it is today still challenging to provide consistent haptic feedback in complex virtual worlds. In this paper, we present a new coupling scheme for haptic display of contacts between rigid bodies, based on the generation of a sub-world around the haptic interaction. This sub-world allows the simulation of physical models at higher frequencies using a reduced quantity of data. We introduce the use of a graph to manage the contacts between the bodies. Our results show that our coupling scheme enables to increase the complexity of the virtual world without having perceptible loss in the haptic display quality.

Virtual reality training simulations to acquire surgical skills are important for increasing patient safety and save valuable resources, e.g., cadavers, supervision and operating room time. However, as surgery is a craft, simulators must not only provide a high degree of visual realism, but especially a realistic haptic behavior. While such simulators exist for surgeries like laparoscopy or arthroscopy, other surgical fields, especially where large forces need to be exerted, like total hip arthroplasty (THA; implantation of a hip joint protheses), lack realistic VR training simulations. In this paper we present for the first time a novel VR training simulation for the five steps of THA (from femur head resection to stem implantation) with realis-tic haptic feedback. To achieve this, a novel haptic hammering device, an upgraded version of the Virtuose 6D haptic device from Haption, novel algorithms for collision detection, haptic rendering, and material removal are introduced. In a study with 17 surgeons of diverse experience levels, we confirmed the realism, usefulness and usability of our novel methods.

In this paper, we propose the use of a nonlinear disturbance-observer for estimation of contact forces during haptic interactions. Most commonly used impedance-type haptic interfaces employ open-loop force control under the assumption of pseudostatic interactions. Advanced force control in such interfaces can increase simulation fidelity through improvement of the transparency of the device. However, closed-loop force feedback is limited both due to the bandwidth limitations of force sensing and the associated cost of force sensors required for its implementation. Using a disturbance-observer, we estimate contact forces at the tool tip, then use these estimates for closed-loop control of the haptic interface. Simulation and experimental results, utilizing a custom single degree-of-freedom haptic interface, are presented to demonstrate the efficacy of the proposed disturbance-observer (DO)-based control approach. This approach circumvents the traditional drawbacks of force sensing while exhibiting the advantages of closed-loop force control in haptic devices. Results show that the proposed disturbance-observer can reliably estimate contact forces at the human-robot interface. The DO-based control approach is experimentally shown to improve haptic interface fidelity over a purely open-loop display while maintaining stable and vibration-free interactions between the human user and virtual environment.