Design, Fabrication and Control of a Handheld Force-Feedback Device

Vitreo-retinal surgery encompasses the surgical procedures performed on the vitreous humor and the retina. A procedure typically consists of the removal of the vitreous humor, the peeling of a membrane and/or the repair of a retinal detachment. Operations are performed with needle shaped instruments which enter the eye through surgeon made scleral openings. An instrument is moved by hand in four degrees of freedom (three rotations and one translation) through this opening. Two rotations (? and ? ) are for a lateral instrument tip movement. The other two DoFs (z and ?) are the translation and rotation along the instrument axis. Actuation of for example a forceps can be considered as a fifth DoF. Characteristically, the manipulation of delicate, micrometer range thick intraocular tissue is required. Today, eye surgery is performed with a maximum of two instruments simultaneously. The surgeon relies on visual feedback only, since instrument forces are below the human detection limit. A microscope provides the visual feedback. It forces the surgeon to work in a static and non ergonomic body posture. Although the surgeon’s proficiency improves throughout his career, hand tremor may become a problem around his mid-fifties. Robotically assisted surgery with a master-slave system enhances dexterity. The slave with instrument manipulators is placed over the eye. The surgeon controls the instrument manipulators via haptic interfaces at the master. The master and slave are connected by electronic hardware and control software. Implementation of tremor filtering in the control software and downscaling of the hand motion allow prolongation of the surgeon’s career. Furthermore, it becomes possible to do tasks like intraocular cannulation which can not be done by manually performed surgery. This thesis focusses on the master console. Eye surgery procedures are observed in the operating room of different hospitals to gain insight in the requirements for the master. The master console as designed has an adjustable frame, a 3D display and two haptic interfaces with a coarse adjustment arm each. The console is mounted at the head of the operating table and is combined with the slave. It is compact, easy to place and allows the surgeon to have a direct view on and a physical contact with the patient. Furthermore, it fits in today’s manual surgery arrangement. Each haptic interface has the same five degrees of freedom as the instrument inside the eye. Through these interfaces, the surgeon can feel the augmented instrument forces. Downscaling of the hand motion results in a more accurate instrument movement compared to manually performed surgery. Together with the visual feedback, it is like the surgeon grasps the instrument near the tip inside the eye. The similarity between hand motion and motion of the instrument tip as seen on the display results in an intuitive manipulation. Pre-adjustment of the interface is done via the coarse adjustment arm. Mode switching enables to control three or more instruments manipulators with only two interfaces. Two one degree of freedom master-slave systems with force feedback are built to derive the requirements for the haptic interface. Hardware in the loop testing provides valuable insights and shows the possibility of force feedback without the use of force sensors. Two five DoF haptic interfaces are realized for bimanual operation. Each DoF has a position encoder and a force feedback motor. A correct representation of the upscaled instrument forces is only possible if the disturbance forces are low. Actuators are therefore mounted to the fixed world or in the neighborhood of the pivoting point for a low contribution to the inertia. The use of direct drive for ' and and low geared, backdriveable transmissions for the other three DoFs gives a minimum of friction. Disturbance forces are further minimized by a proper cable layout and actuator-amplifier combinations without torque ripple. The similarity in DoFs between vitreo-retinal eye surgery and minimally invasive surgery (MIS) enables the system to be used for MIS as well. Experiments in combination with a slave robot for laparoscopic and thoracoscopic surgery show that an instrument can be manipulated in a comfortable and intuitive way. User experience of surgeons and others is utilized to improve the haptic interface further. A parallel instead of a serial actuation concept for the ' and DoFs reduces the inertia, eliminates the flexible cable connection between frame and motor and allows that the heat of the motor is transferred directly to the frame. A newly designed z-?? module combines the actuation and suspension of the hand held part of the interface and has a three times larger z range than in the first design of the haptic interface.

Haptic User Interface for a Telerobotic System – Design based on a Network Model

Multifinger haptic interface is the ultimate human haptic interface that would provide natural manipulation and feeling to the tip of each finger, same as the real hand interaction with the real world in daily-life tasks. This chapter introduced the second version of HIRO, including the hardware design, the control algorithm, and the software design including the system architecture, haptic rendering, and practical VR application. The hand with the arm provides a large workspace and reflects exactly the human hand movement as you are looking to your hand and arm in the mirror, with some differences related to hardware limitations and control methods that yet need to be improved. A comprehensive haptic system through demonstration of the future haptic science encyclopedia is presented. The architecture used was a multi-layer network framework that decoupled the haptic from graphic on to separate machines. The architecture maintained the high update rate needed for the haptic servoloop. Also, an accurate collision detection algorithm that considers the shape of human fingertips is developed. A physically based modeling using the finite element method (FEM) combined with an elementary displacement approach proved to ensure stable interaction and realistic forces calculation for a relatively complex deformable model in real time. The approach takes into considerations interaction with multipoint contacts where other approaches failed to achieve such interaction due difficulty in calculating forces at 1 KHz. The FHSE application was demonstration at World Expo 2005. People enjoyed the demonstration, where the majority were satisfied with the system as it provided a unique experience. Finally, combining technologies such as virtual reality and haptic robotics enables us to extend the capabilities and effectiveness of simulators. We anticipate that the proposed system will bring the notice of many researchers to the feasibility of implementing multifinger haptic interfaces. Although, every one agree on the importance of such interface, but many obstacles are still preventing the advancement of this research field. This chapter provided a complete picture on how this could be achieved. The greatest lesson of such project was the collaboration between robotics community and virtual reality community is necessary to achieve a better and comprehensive sophisticated haptic robot. Eventhough , I was the leader of the software development side including the haptic rendering and VR application, I was involved from the beginning in the process of designing the hardware and the control algorithms as well, useful feedback was the result of such collaboration. There are a number of future directions that can apply to this research. First, it is desirable to test the presented architecture in the presence of large delay and apply some techniques to compensate it. Second, develop more robust control algorithm for the arm movements to reflect the exact motion of the human hand, this will make the interface manipulation very natural. Also, make the hardware itself smarter and safer.

Autonomous vehicles will significantly change the existing driver-vehicle relationship, since only a destination input from the human driver will suffice. However, reduced degree of human-control could result in lack of driving pleasure and excitement. Thus, we proposed a method to increase the flexibility in controlling of an autonomous vehicle by allowing the driver to control its lateral and longitudinal motions with a time lag. We first derived a set of vehicle movements to improve driver experience and related them to a set of control functions that the driver can input. We then created two types of driver-vehicle interfaces (DVIs) for vehicle control; a haptic interface with kinesthetic and tactile feedback, and a hand-gesture interface with augmented reality feedback. The joystick-type haptic interface provides feedback on driver input by dynamically varying its degrees of freedom through controlling the current supplied to axis motors, and through vibration motors. The gesture interface is based on Leap Motion controller and provides visual feedback to driver. We conducted driving experiments in a VR simulator using twenty drivers to evaluate the effectiveness of these DVIs. The results showed that haptic interface significantly reduced the average input time and input error, and drivers preferred the haptic interface due to its ability to provide immediate, active, and passive feedback.

A haptic interface is a device for operators to touch, feel and manipulate virtual environments. It senses the contact forces and gives operators a sensation of interacting with a real physical environment. Force feedback can improve the quality of the haptic feedback control in virtual reality applications. In this paper, a real time haptic interface is developed via control models in the Labview virtual environment, which is a combination of a touch force measurement haptic interface and a vibro-tactile CyberGlove serial haptic interface. The haptic interface is a prototype for real time feedback control on robotic integration of smart censors. This approach is easy to apply on various types of sensors, such as acoustic wave sensors, ultrasound array, etc., which will enhance the ability to communicate between the operators and the virtual environment. The flexiforce sensor and tactile sensor based haptic interface in force feedback systems will improve traditional robot visual sensing systems by a virtual feel.

In this paper the design of a multi-finger force-reflecting haptic interface device for teleoperational grasping is introduced. The haptic interface or “master” controller device is worn on the human operator’s hand and measured human finger positions are used to control the finger positions of a remote grasping manipulator or “slave” device. The slave may be a physical robotic grasping manipulator, or a computer generated representation of a human hand such as used in virtual reality applications. The forces measured by the robotic slave, or calculated for the virtual slave, are presented to the operator’s fingertips through the master providing a means for deeper human sensation of presence and better control of grasping tasks in the slave environments. Design parameters and performance measures for haptic interfaces for teleoperation are discussed. One key performance issue involving the high-speed display of forces during initial contact, especially when interacting with rigid surfaces, is addressed by the present design, reducing slave controller computation requirements and overcoming actuator response time constraints. The design presented utilizes a planar four-bar linkage for each finger, to represent each finger bend motion as a single degree of freedom, and to provide a finger bend resistance force that is substantially perpendicular to the distal finger pad throughout the full 180 degrees of finger bend motion represented. The finger linkage design, in combination with a remote position measurement and force display assembly, provides a very lightweight and low inertia system with a large workspace. The concept of a replicated finger is introduced which, in combination with a decoupled actuator and feed forward control, provides improved performance in transparent free motion, and rapid, stable touch sensation of initial contact with rigid surfaces. A distributed computation architecture with a PC based haptic interface controller and associated control algorithms are also discussed.

A new portable device for haptic interaction with virtual environments is presented. It is a lightweight interface for the fingertips, designed for providing cutaneous feedback and displaying the contact - non contact transition in highly immersive virtual environments. In this paper the kinematics, the mechanical design and the control system are described. The device has been mounted on a kinesthetic haptic interface which tracks its position: the overall system can provide both cutaneous and kinesthetic feedback. Finally the performance of the portable device in a simple task of shape exploration has been evaluated and compared to a kinesthetic haptic interface.

In this paper, we propose the use of a nonlinear disturbance-observer for estimation of contact forces during haptic interactions. Most commonly used impedance-type haptic interfaces employ open-loop force control under the assumption of pseudostatic interactions. Advanced force control in such interfaces can increase simulation fidelity through improvement of the transparency of the device. However, closed-loop force feedback is limited both due to the bandwidth limitations of force sensing and the associated cost of force sensors required for its implementation. Using a disturbance-observer, we estimate contact forces at the tool tip, then use these estimates for closed-loop control of the haptic interface. Simulation and experimental results, utilizing a custom single degree-of-freedom haptic interface, are presented to demonstrate the efficacy of the proposed disturbance-observer (DO)-based control approach. This approach circumvents the traditional drawbacks of force sensing while exhibiting the advantages of closed-loop force control in haptic devices. Results show that the proposed disturbance-observer can reliably estimate contact forces at the human-robot interface. The DO-based control approach is experimentally shown to improve haptic interface fidelity over a purely open-loop display while maintaining stable and vibration-free interactions between the human user and virtual environment.

This work presents a study of performance improvement of unskilled people to drawn simple sketches. We have assisted the unskilled people when drawn using a haptic interface which acts as a virtual guide taking advantage of its force feedback capabilities. In the first part of the study, an application has been developed to extract fixed templates from image files; the application extract the principal edges in the images to build output trajectories (templates) that are used by haptic interface controller, after that, the user can the virtual templates with the assistance of the force feedback capabilities of the interface. Based on the obtained results for fixed templates, a second application was developed; the user can generate interactive templates indicating where he/she desires to put a geometrical template (circle, line or arc) inside the haptic interface's workplace; once that the position of the template is defined, the interface shows its position graphically and then user can fill it assisted by the haptic interface force feedback.

Inspection of the colon with an endoscope for early signs of cancer (colonoscopy) has become an extremely widespread procedure, since early treatment radically improves the outlook of patients. The procedure requires a close coordination between the sense of touch and vision to navigate the endoscope along the colon. This raises the need to develop efficient training methods for physicians. Training simulators based on virtual reality, where realistic graphics are combined with a mechatronic system providing haptic feedback, are alternative to traditional training methods. To provide physicians with realistic haptic sensations of an endoscopic procedure, we have designed a haptic interface, instrumented a clinical endoscope and combined them with a simulation software for colonoscopy. In this contribution, we present the mechatronic components of the simulator. The haptic interface is able to generate high forces using the combination of electrical motors and brakes in a compact design. Experiments were performed to determine the characteristics of the device. A model-based control has been implemented and the results show that the control successfully compensates for the device nonlinearities, such as friction. The proposed haptic interface, together with the virtual reality, form a highly realistic training simulator for endoscopic surgeons, applicable not only to colonoscopy, but also to similar interventions.

Endoscopic endonasal approach surgery is now the preferred treatment for most pituitary and related skull base tumors. However, this procedure requires a high level of hands-on skills and rich clinical experience. During the operation, haptic feedback, as the only one sense of bidirectional information interaction, plays an important role in surgical decision-making especially for bone-drilling. Existing surgical simulators provide either no haptic device or multipurpose haptic devices, which is difficult to reproduce the characteristics of surgical tool handling. In this paper, a custom-designed 3-DOF (pitch, yaw, radial) compact haptic interface for this surgery simulation is presented. It is dedicated to mimicking the touch sense of the surgical tools inserted through the nostril. Its main innovation is the mechanism design to maintain as much fidelity of the tool handling in the surgical training as in a real operation. The mechanism design is presented in detail as well as the kinematics and the force transmission. The mechanical characteristics of this haptic interface are also analyzed and presented.

Gastrointestinal endoscopy simulations have been developed to train endoscopic procedures which require hundreds of practices to be competent in the skills. Even though realistic haptic feedback is important to provide realistic sensation to the user, most of previous simulations including commercialized simulation have mainly focused on providing realistic visual feedback. In this paper, we propose a novel design of portable haptic interface, which provides 2DOF force feedback, for the gastrointestinal endoscopy simulation. The haptic interface consists of translational and rotational force feedback mechanism which are completely decoupled, and gripping mechanism for controlling connection between the endoscope and the force feedback mechanism.

We discuss the possibility of improving eyehand coordination in children diagnosed with this problem, using a robotic mapping from a haptic user interface to a virtual environment. Our goal is to develop an assessment and training procedure that will result in improving handwriting taking the advantage of the force feedback provided by the haptic device. Force feedback can be used to guide the subject's hand in a predetermined trajectory when he/she is unable to move in response to visual feedback. We also incorporate the inertia and viscosity effects to decrease the tremor in the hand as well as to stimulate the muscles involved in the task of holding a pencil (known as facilitation technique in the occupational therapy field). The robotic system is operated within ranges established by baseline data for accuracy of eye-hand coordination and grip strength. A set of assessment tests, commonly used by occupational therapists, were chosen to implement the various functions using force, inertia and viscosity effects. The test bed used for these tasks consisted of a six-degree-of-freedom force reflecting haptic interface device, PHANToM with the GHOST SDK Software. Preliminary data obtained from two children diagnosed with eye-hand coordination problems show that the assistance provided, improved their ability to execute and complete the tasks, suggesting that the system could be used for assessment and training purposes. In addition, these results suggest that the introduction of the haptic rendering capabilities, including the force feedback, offers special benefit and measures the impact on the occupation of handwriting. Furthermore, a real application of this system could be implemented in the future, by identifying specific job tasks that may be made accessible through a desktop in classrooms by the aforementioned haptic system.

The last decade has seen significant advances in research on haptic interfaces. High-performance haptic devices have been developed, some of which are commercially available. What problems will we meet in the next decade?This panel will explore cutting-edge issues and the scope of new frontiers in haptic interfaces. Topics to be discussed are: New mechanisms of haptic interfaces. Various exoskeletons and manipulators have been developed. How will they be extended? What other configurations of haptic interfaces are possible?New application areas. Medical applications of haptic interface are very active now. What will be the next successful application? Home-use haptic interfaces may be one of the ultimate goals. What barriers are there for home use? Full-body haptics and foot haptics. One of the new frontiers of haptic interfaces is full-body haptics, including foot haptics. Force applied to a whole body plays a very important role in locomotion. What are the issues for full-body haptics?

A newly designed haptic interface enables an operator to control a remote robot precisely. It transmits position information to the remote robot and feeds back the interaction force from it. A control algorithm of haptic interface has been studied to improve the robustness and stability to uncertain dynamic environments with a proposed contact dynamic model that incorporates human hand dynamics. A simplified hybrid parallel robot dynamic model fur a 6 DOF haptic device was proposed to from a real time control system, which does not include nonlinear components. LQC/LTR scheme was adopted in this paper for the compensation of un-modeled dynamics. The recovery of the farce from the remote robot at the haptic interface was demonstrated through the experiments.