Five-fingered haptic interface robot: HIRO III

Haptic interfaces are becoming more commonplace in virtual environment and teleoperation systems. There is a growing need to not only continue to improve hardware platforms and rendering algorithms, but evaluate human performance with haptic interfaces. This review summarizes two recent studies inspired by perception problems in using haptic interfaces to interact with virtual environments. The first study evaluated perceived quality of virtual haptic textures and discovered several types of perceived instability and their sources. We found that the buzzing type of perceived instability was most likely due to the mechanical resonance of the haptic interface hardware, and the aliveness type of perceived instability due to our inability to sense the slight movements of our hands in free space. The second study focused on the motor strategy employed during interaction with a virtual surface via a force-feedback haptic interface. We found that users tended to maintain a constant penetration force into a virtual surface when interacting with the surface. This can result in a reversal in perceived relative surface heights if the taller surface is rendered with a lower stiffness, thereby resulting in an erroneous perception of the virtual environment being rendered. For both studies, possible solutions to improving human perception of virtual and remote objects via hardware and/or software are discussed.

A haptic interface is a device for operators to touch, feel and manipulate virtual environments. It senses the contact forces and gives operators a sensation of interacting with a real physical environment. Force feedback can improve the quality of the haptic feedback control in virtual reality applications. In this paper, a real time haptic interface is developed via control models in the Labview virtual environment, which is a combination of a touch force measurement haptic interface and a vibro-tactile CyberGlove serial haptic interface. The haptic interface is a prototype for real time feedback control on robotic integration of smart censors. This approach is easy to apply on various types of sensors, such as acoustic wave sensors, ultrasound array, etc., which will enhance the ability to communicate between the operators and the virtual environment. The flexiforce sensor and tactile sensor based haptic interface in force feedback systems will improve traditional robot visual sensing systems by a virtual feel.

Haptic interaction in slow virtual environments (VEs) can become unstable due to the phase lag introduced in the control loop by the slow update rate of the VE. Increasing the physical damping and/or limiting the contact stiffness rendered to users can mitigate the destabilizing effect of the low VE update rate. However, large physical damping and compliant virtual contacts decrease the sense of presence in VEs, especially during interaction with rigid virtual objects. To increase the maximum virtual contact stiffness that can be rendered to users without increasing the interface damping, this paper proposes a control strategy based on multirate wave communications between a haptic interface and a VE updated at a slow and fixed rate. The multirate wave communications are shown to be guaranteed passive only if the decrease of the wave sampling rate at the connection between the haptic interface and the VE does not cause aliasing. Therefore, an antialiasing low-pass filter is placed before the wave rate drop in the communications. The passivity condition is verified analytically and numerically for multirate haptic interaction in VEs with various contact stiffnesses and update rates. The transparency of haptic interaction in slow VEs to which users connect via passive multirate wave communications is investigated analytically in the frequency domain. Experiments validate that passive multirate wave communications can render stiffer contact in slow VEs than conventional direct coupling, and illustrate the destabilizing effect of the aliasing caused by the sampling rate drop in the communications.

In this paper the design of a multi-finger force-reflecting haptic interface device for teleoperational grasping is introduced. The haptic interface or “master” controller device is worn on the human operator’s hand and measured human finger positions are used to control the finger positions of a remote grasping manipulator or “slave” device. The slave may be a physical robotic grasping manipulator, or a computer generated representation of a human hand such as used in virtual reality applications. The forces measured by the robotic slave, or calculated for the virtual slave, are presented to the operator’s fingertips through the master providing a means for deeper human sensation of presence and better control of grasping tasks in the slave environments. Design parameters and performance measures for haptic interfaces for teleoperation are discussed. One key performance issue involving the high-speed display of forces during initial contact, especially when interacting with rigid surfaces, is addressed by the present design, reducing slave controller computation requirements and overcoming actuator response time constraints. The design presented utilizes a planar four-bar linkage for each finger, to represent each finger bend motion as a single degree of freedom, and to provide a finger bend resistance force that is substantially perpendicular to the distal finger pad throughout the full 180 degrees of finger bend motion represented. The finger linkage design, in combination with a remote position measurement and force display assembly, provides a very lightweight and low inertia system with a large workspace. The concept of a replicated finger is introduced which, in combination with a decoupled actuator and feed forward control, provides improved performance in transparent free motion, and rapid, stable touch sensation of initial contact with rigid surfaces. A distributed computation architecture with a PC based haptic interface controller and associated control algorithms are also discussed.

Multifinger haptic interface is the ultimate human haptic interface that would provide natural manipulation and feeling to the tip of each finger, same as the real hand interaction with the real world in daily-life tasks. This chapter introduced the second version of HIRO, including the hardware design, the control algorithm, and the software design including the system architecture, haptic rendering, and practical VR application. The hand with the arm provides a large workspace and reflects exactly the human hand movement as you are looking to your hand and arm in the mirror, with some differences related to hardware limitations and control methods that yet need to be improved. A comprehensive haptic system through demonstration of the future haptic science encyclopedia is presented. The architecture used was a multi-layer network framework that decoupled the haptic from graphic on to separate machines. The architecture maintained the high update rate needed for the haptic servoloop. Also, an accurate collision detection algorithm that considers the shape of human fingertips is developed. A physically based modeling using the finite element method (FEM) combined with an elementary displacement approach proved to ensure stable interaction and realistic forces calculation for a relatively complex deformable model in real time. The approach takes into considerations interaction with multipoint contacts where other approaches failed to achieve such interaction due difficulty in calculating forces at 1 KHz. The FHSE application was demonstration at World Expo 2005. People enjoyed the demonstration, where the majority were satisfied with the system as it provided a unique experience. Finally, combining technologies such as virtual reality and haptic robotics enables us to extend the capabilities and effectiveness of simulators. We anticipate that the proposed system will bring the notice of many researchers to the feasibility of implementing multifinger haptic interfaces. Although, every one agree on the importance of such interface, but many obstacles are still preventing the advancement of this research field. This chapter provided a complete picture on how this could be achieved. The greatest lesson of such project was the collaboration between robotics community and virtual reality community is necessary to achieve a better and comprehensive sophisticated haptic robot. Eventhough , I was the leader of the software development side including the haptic rendering and VR application, I was involved from the beginning in the process of designing the hardware and the control algorithms as well, useful feedback was the result of such collaboration. There are a number of future directions that can apply to this research. First, it is desirable to test the presented architecture in the presence of large delay and apply some techniques to compensate it. Second, develop more robust control algorithm for the arm movements to reflect the exact motion of the human hand, this will make the interface manipulation very natural. Also, make the hardware itself smarter and safer.

This book covers all topics relevant for the design of haptic interfaces and teleoperation systems. The book provides the basic knowledge required for understanding more complex approaches and more importantly it introduces all issues that must be considered for designing efficient and safe haptic interfaces. Topics covered in this book provide insight into all relevant components of a haptic system. The reader is guided from understanding the virtual reality concept to the final goal of being able to design haptic interfaces for specific tasks such as nanomanipulation. The introduction chapter positions the haptic interfaces within the virtual reality context. In order to design haptic interfaces that will comply with human capabilities at least basic understanding of human sensors-motor system is required. An overview of this topic is provided in the chapter related to human haptics. The book does not try to introduce the state-of-the-art haptic interface solutions because these tend to change quickly. Only a careful selection of different kinematic configurations is shown to introduce the reader into this field. Mathematical models of virtual environment, collision detection and force rendering topics are strongly interrelated and are described in the next two chapters. The interaction with the virtual environment is simulated with a haptic interface. Impedance and admittance based approaches to haptic robot control are presented. Stability issues of haptic interaction are analyzed in details and solutions are proposed for guaranteeing stable and safe operation. Finally, haptic interaction is extended to teleoperation systems. Virtual fixtures which improve the teleoperation and human-robot cooperation in complex environments are covered next and the last chapter presents nanomanipulation as one specific example of teleoperation.

The haptic interface provides the virtual environment for the remote servo manipulator, and it can make the operator feel and manipulate the remote or virtual environment. For arbitrary passive operator (human), the stability not only relates to the parameters of the haptic device and the virtual environment, but also relates to the type of the sampling signal of the system. The passivity conditions for haptic interface systems with position sampling are given in existing works, but the parameters scope for passivity design is very small. In view of this, velocity of the haptic device is used for the sampling signal in this paper, and a wider passive region is derived based on the frequency response of a sampled-data control system. At last, through overall consideration of the bandwidth, damping ratio and passivity requirements, a reasonable design result is given in the paper.

The purpose of evaluation procedures is to achieve both qualitative and quantitative statements on haptic rendering realism and performance. Since a haptic interface provides an interaction between a user and a virtual environment, fidelity of a haptic interface directly affects the performance. To our knowledge, a standard, generic and reusable validation method which comprehensively addresses all the attributes of haptic feedback has not been realized yet. Despite the large number of human factor studies, only few of them have been proposed as well for haptic interface performance measurements. For this reason, we review validation procedures for haptic rendering and propose an evaluation method based on testbeds to obtain a systematic haptic interface assessment. We integrated the approaches of human factor studies into the testbeds to obtain a simple and yet complete measure of human-machine interaction performance. The testbeds were tested on a haptic interface, the IHP of Xitact SA, and performance results are presented. In the testbeds, performance metrics for generic haptic interaction tasks are expressed in terms of information transfer (bits) and sensory thresholds which are indeed device specific benchmark metrics. Thus, the suitability of a haptic interface for a defined task can be verified, device comparisons become possible and the obtained information can be used to identify possible improvements.

It is useful to compute sound field of a source in a virtual environment which is different from the measurement environment. For example, some properties of sound source, such as directivity index and frequency response curve, are required to be measured in an anechoic room or free space, but both of them cannot be always accessible. Consequently, it will be useful to compute sound field of a source in free space when sound field of the source is not measured in the free space. In the aforementioned example, the free space is a virtual environment. Based on reciprocity theorem and modal expansion, a method to predict sound field of a source in a virtual environment is given in this paper when the scattering effect of the source can be neglected. Reciprocity theorem builds the relationship between measured sound field and predicted sound field, which plays an important role in the method. Green’s function in the virtual environment is needed in the method. To restrict measurement points on an enclosed surface, the Green’s function is expanded by a set of modes. A simulation is given to examine the validity of the method.

A new portable device for haptic interaction with virtual environments is presented. It is a lightweight interface for the fingertips, designed for providing cutaneous feedback and displaying the contact - non contact transition in highly immersive virtual environments. In this paper the kinematics, the mechanical design and the control system are described. The device has been mounted on a kinesthetic haptic interface which tracks its position: the overall system can provide both cutaneous and kinesthetic feedback. Finally the performance of the portable device in a simple task of shape exploration has been evaluated and compared to a kinesthetic haptic interface.

The performance levels of human subjects in size identification and size discrimination experiments in both real and virtual environments are presented. The virtual environments are displayed with a PHANToM desktop three degree-of-freedom haptic interface. Results indicate that performance of the size identification and size discrimination tasks in the virtual environment is comparable to that in the real environment, implying that the haptic device does a good job of simulating reality for these tasks. Additionally, performance in the virtual environment was measured at below maximum machine performance levels for two machine parameters. The tabulated scores for the perception tasks in a sub-optimal virtual environment were found to be comparable to that in the real environment, supporting previous claims that haptic interface hardware may be able to convey, for these perceptual tasks, sufficient perceptual information to the user with relatively low levels of machine quality in terms of the following parameters: maximum endpoint force and maximum virtual surface stiffness. Results are comparable to those found for similar experiments conducted with other haptic interface hardware, further supporting this claim. Finally, it was found that varying maximum output force and virtual surface stiffness simultaneously does not have a compounding effect that significantly affects performance for size discrimination tasks.

Design of Haptic Interface Controller under Noise Uncertainty and Delay Condition

In this paper, we propose the use of a nonlinear disturbance-observer for estimation of contact forces during haptic interactions. Most commonly used impedance-type haptic interfaces employ open-loop force control under the assumption of pseudostatic interactions. Advanced force control in such interfaces can increase simulation fidelity through improvement of the transparency of the device. However, closed-loop force feedback is limited both due to the bandwidth limitations of force sensing and the associated cost of force sensors required for its implementation. Using a disturbance-observer, we estimate contact forces at the tool tip, then use these estimates for closed-loop control of the haptic interface. Simulation and experimental results, utilizing a custom single degree-of-freedom haptic interface, are presented to demonstrate the efficacy of the proposed disturbance-observer (DO)-based control approach. This approach circumvents the traditional drawbacks of force sensing while exhibiting the advantages of closed-loop force control in haptic devices. Results show that the proposed disturbance-observer can reliably estimate contact forces at the human-robot interface. The DO-based control approach is experimentally shown to improve haptic interface fidelity over a purely open-loop display while maintaining stable and vibration-free interactions between the human user and virtual environment.

A high-quality haptic interface is typically characterized by low apparent inertia and damping, high structural stiffness, minimal backlash and absence of mechanical singularities in the workspace. In addition to these specifications, exoskeleton haptic interface design involves consideration of additional parameters and constraints including space and weight limitations, workspace requirements and the kinematic constraints placed on the device by the human arm. In this context, we present the design of a five degree-of-freedom haptic arm exoskeleton for training and rehabilitation in virtual environments. The design of the device, including actuator and sensor selection, is discussed. Limitations of the device that result from the above selections are also presented. The device is capable of providing kinesthetic feedback to the joints of the lower arm and wrist of the operator, and will be used in future work for robot-assisted rehabilitation and training.

This paper presents a high-intensity focus ultrasound (HIFU) robotic system for treating breast fibroadenoma under the guidance of magnetic resonance imaging (MRI). Based on the thermal and mechanical effects of ultrasound, the system aims to deliver ultrasound energy to a target precisely without damaging the normal tissue. The temperature elevation can be monitored in real time by MRI, and the treatment plan can be adjusted during the procedure. The requirements, design specifications, control system and registration of the robotic system are specified. The robotic system was designed with a 3 degrees of freedom manipulator with limit switches and encoders, a customised MRI-compatible breast coil, a water bladder with sets of breast-conforming brackets, and a probe capable of generating ultrasound. Twenty volunteers were recruited for this study, and their data were analysed to provide more precise data for the design. The accuracy of the robot was evaluated in free space using a coordinate measuring machine, phantom and ex vivo porcine tissue in MRI room. The study also verified the signal-to-noise ratio (SNR) of the MRI with the effect of the robotic system. The research findings revealed that the manipulator exhibited a translational precision of 0.10±0.14mm, a rotational fidelity around the X direction of 0.11±0.09°, and an oscillatory exactness around the Y direction of 0.10±0.08°. The investigation of positioning accuracy demonstrated that the robot's error in free space was 0.26±0.07mm. When subjected to MRI room with agar-silica phantom and ex vivo porcine tissue, the positioning accuracy amounted to 1.11±0.47mm and 1.57±0.52mm. In the presence of the robotic system, the SNR of the MRI experienced a 4.2% reduction, which had a negligible impact on image quality. The conducted experiments validate the efficacy of the proposed MRI-guided HIFU robotic system in executing agar-silica phantom and ex vivo porcine tissue investigations with adequate positioning accuracy. Consequently, this system exhibits certain feasibility for the treatment of breast fibroadenomas.