Simulation of ICSI Procedure Using Virtual Haptic Feedback Model

Accompanying the progress of medical technology, all kinds of treatment and therapy are being practiced in healthcare industry. It is expected that the regional difference in terms of the quality of healthcare in the area of surgery could be reduced through the remote operations with the help of a robot-based surgery system. Such a remote surgery approach, however, is not yet practical, because it solely relies on the visual perception with no input for the haptic perception. Since it is a lot faster for a human being to sense through touching than by watching especially when he is manipulating an object, sensing through touching is considered essential to the success of remote surgery. In this research, a system that is able to sense through a force sensor is developed. The system simulates the process of intracytoplasmic sperm injection (ICSI). PHANToM, a force sensing system, is employed for collecting the information for the haptic perception. With this force sensing system, it is possible to have force input in 6 degrees of freedom and force output in 3 degrees of freedom. OpenGL is used for the 3D graphics rendering in the visual information processing system. By synchronizing the information for both the visual and haptic perception, a virtual reality system that reacts to both the visual and force interactions has been developed. The simulation results reported in this paper proved that using the information collected through the force sensing system makes a significant difference in the medical treatment.

"Laparoscopist's thumb," or thenar paresthesia, can result from prolonged or excessive grip force during laparoscopy, as can more general syndromes, such as carpal tunnel syndrome. This is particularly relevant in gynecology, where laparoscopic procedures are standard. Although this method of injury is well known, there is a paucity of data to guide surgeons in selecting more efficient, ergonomic instruments. This study compared the ratio of applied tissue force and required surgeon input in a sample of common ratcheting laparoscopic graspers in a small-handed surgeon, to provide potential metrics applicable to surgical ergonomics and surgeon instrument choice. Laparoscopic graspers with varied ratcheting mechanisms and tip shapes were evaluated. Brands included Snowden-Pencer, Covidien, Aesculap, and Ethicon. A Kocher was used as an open instrument comparison. Flexiforce A401 thin-film force sensors were used to measure applied forces. Data were collected and calibrated using an Arduino Uno microcontroller board with Arduino and MATLAB software. Single-handed, complete closure of each device's ratcheting mechanism was performed 3 times. The maximum required input force in Newtons was recorded and averaged. The average output force was measured with a bare sensor and the same sensor between 2 different thicknesses of LifeLike BioTissue. The most ergonomic ratcheting grasper for a small-handed surgeon was identified by the output ratio: the highest output force relative to the required surgeon input (the most force for the least amount of effort). The Kocher required an average input force of 33.66 N, with its highest output ratio of 3.46 (112 N output). The Covidien Endo Grasp was the most ergonomic, with an output ratio of 0.96 on the bare force sensor (31.4 N output). The Snowden-Pencer Wavy grasper was the least ergonomic, with an output ratio of 0.06 when applied to the bare force sensor (5.9 N output). All graspers except for the Endo Grasp had improving output ratios as tissue thickness and subsequent grasper contact area increased. Input force above that provided by the ratcheting mechanisms did not increase output force in a clinically relevant amount for any of the instruments evaluated. Laparoscopic graspers vary widely in their ability to provide reliable tissue force without requiring excessive input by the surgeon, and a point of diminishing returns often exists with increased surgeon input over designed ratcheting mechanisms. Output force and output ratio are potential quantitative measures of the efficiency of laparoscopic instruments. Providing users with this type of data could assist in optimizing instrument ergonomics.

This paper describes a task-oriented approach to cope with the difference between the three dimensional (3D) space in a virtual reality (VR) system and that perceived by the user from its stereoscopic images with binocular disparity. This difference occurs for the reason that human visual perception is affected not only by binocular disparity but also by various kinds of visual cues. Although it is possible to set some of those cues consistent with each other when they are presented by a stereoscopic display, it is difficult to do the same for other cues especially vergence and focus. As the result, the users of VR systems often fail to manipulate virtual objects due to this difference. In this article, we propose to adjust the virtual space in a VR system through object manipulation by the user so that the user does not fail in each manipulation. We verified by experiments that this adjustment is effective to improve the rate of successful manipulation.

In this paper, a precise small 5-DOF (degree of freedom) force sensor is proposed for use in a strong EMF (electromagnetic field) environment. Detecting modules using CFPs (carbon fiber plates) and transducing modules using the optical modulation principle are adopted in order not to be affected by the EMF. For miniaturization of the multi-DOF force sensor, a 2-DOF transducing module using a spherical mirror and a 3-DOF transducing module using a plane mirror were designed and integrated. The design parameters of highly sensitive transducing modules were investigated and determined experimentally. To combine with these transducing modules, a 2-DOF detecting module using CFP single leaf springs and a 3-DOF detecting module using a CFP tripod spring were also designed. Considering the easy calibration process and convenient design change, the elastic detecting modules were designed so that they deform independently according to each input force component. A calibration test confirmed that the detecting modules deform linearly and independently of the input force. The results of the evaluation tests showed that the range and resolution of forces were ±4 N and 0.94–7.1 mN and the range and resolution of moments were ±120 N mm and 0.023–0.034 N mm, respectively. The high sensitivity and the linearity of the measuring results were also verified.

Here, we used functional magnetic resonance imaging to investigate the multisensory processing of object shape in the human cerebral cortex and explored the role of mental imagery in such processing. Regions active bilaterally during both visual and haptic shape perception, relative to texture perception in the respective modality, included parts of the superior parietal gyrus, the anterior intraparietal sulcus, and the lateral occipital complex. Of these bimodal regions, the lateral occipital complexes preferred visual over haptic stimuli, whereas the parietal areas preferred haptic over visual stimuli. Whereas most subjects reported little haptic imagery during visual shape perception, experiences of visual imagery during haptic shape perception were common. Across subjects, ratings of the vividness of visual imagery strongly predicted the amount of haptic shape-selective activity in the right, but not in the left, lateral occipital complex. Thus, visual imagery appears to contribute to activation of some, but not all, visual cortical areas during haptic perception.

In this paper, Haptic End-effector for Medicine and Manufacturing (HEM2) with five degree of freedom (DoF) motion is developed for safer and more intuitive robotic surgery. In resent years, multi DoF forceps robots have been studied and developed to support surgeons. However, conventional multi DoF forceps robots cannot transmit force sensation to surgeons adequately. Force sensation is important for safe and accurate teleoperation. In this paper, a master-slave five DoF HEM2 is developed as a haptic forceps robot. Five DoF motion corresponds with the motion of human hand which can realize complex tasks. By implementing acceleration based bilateral control, force sensation can be transmitted. Spherical joint driven by parallel link mechanism realizes the bending motion of the end-effector. The coaxial driving power transmission mechanism is used to decrease the weight of movable part and improve backdrivability. Besides, the mechanism is composed of rigid parts. The performance of the five DoF HEM2 is validated by experiments.

This paper presents a force-sensorless collision detection approach based on virtual contacts. A series of indices based on virtual instantaneous powers are introduced associating to several virtual contacts defined on corresponding links of a manipulator. A possible contact link can be determined by comparing the introduced indices without any force or tactile sensors. Simulation results of 2D collision detection tasks by using a three DOF (Degree of Freedom) planar manipulator and 3D collision detection tasks by using a six DOF spatial manipulator validate the effectiveness of the proposed collision detection approach.

Haptic devices need to render accurate force to a human operator in order to create a virtual environment from a remote real environment. Existing end-effector type haptic devices employ parallel mechanisms to achieve force interaction in multiple degrees-of-freedom (DoF). Those parallel mechanisms have non-constant Jacobian matrices that create difficulties in the actuator and controller design for accurate haptic force interaction. This paper presents a 3-DoF translational parallel manipulator (TPM) with a constant Jacobian matrix to facilitate the force and impedance controller design. Instead of using external force sensors, series elastic actuators (SEAs) are used for the proposed device to realize the force rendering. SEAs can render more accurate output force and stiffness while the inertia and friction problems of using a gearhead can be ignored. In this paper, the modeling, design, and control of the haptic device are established to show the merits of the proposed TPM. Initial force control experiments demonstrate the accuracy and speed of SEA-based force control. We expect that this novel haptic device can serve as an alternative for robotic teleoperations requiring accurate force sensing and control.

Although virtual reality (VR) programs to provide firefighting training continue to be developed and adopted, our investigation with 15 firefighters indicates that a current VR training system tends to convey behavioral tips and does not sufficiently reflect actual firefighters’ needs and realities in the field. It often provides somewhat simplified fire simulations and actually lowers the effectiveness of the training. In this paper, we employ Human-Computer Interaction (HCI) methods to examine and identify core elements in firefighting scenarios and develop a VR system that incorporates such elements. We evaluate our system with respect to presence and three design components of the VR simulation (i.e., reality, meaning, play) through a user study with 22 participants. Our study results demonstrate greater user experience and perception toward the four elements in firefighting training with our VR system compared to the existing one. We discuss design implications (e.g., move control, degree of freedom, sight hindrance by smoke, unexpected events) of our study that are expected to help implement and provide an effective VR training system for firefighters.

Vascular applications of telepresence surgery: Initial feasibility studies in swine

This paper presents the fingertip which can detect the characteristic of object. In previous work, our research group found the sign for fracture appears before total fracture of the object when compressing ductility objects (such as Tofu) with fingertip filled with viscoelastic fluid. The sign is small change rate of pressure inside the fingertip. We call it "Initial break". However, the detail of the phenomenon was unclear. In this paper, we experimentally examine which conditions of fingertips affect expression of initial break.

This study analyzes the recognition process of examining the texture of fine particles through tactile sensing using human fingers and aims at establishing the structure of texture recognition. Characteristic values of the tactile sensing are studied in order to investigate a relationship between tactile sensing and its verbal response. In this study, the SD method (Semantic Differential method) is used to execute the language estimation (the evaluation value) of fine particles. In this paper, the recognition of tactile sensing in mid teens is shown by the relationship between particle size of fine particles and the evaluation value.

With the growing interest of integrating robotics into everyday life and industry, the requirements towards the quality and quantity of applications grows equally hard. This trend is profoundly recognized in applications involving visual perception. Whereas visual sensing in home environments tend to be mainly used for recognition and localization, safety becomes the driving factor for developing intelligent visual control algorithms. More specifically, a robot operating in a human environment should not collide with obstacles and executed motion should be as smooth as possible. Furthermore, as the environment is not known on beforehand, a high demand on the robustness of visual processing is a necessity. On the other hand, in an industrial setting, the environment is known on beforehand and safety is mainly guaranteed by excluding a human operator. Despite these reasons, and the fact that visual servoing has gained much attention from industry to become a standard solution for robotic automation tasks, applications are highly simplified. For example, methods such as visual fault detection are already a mature technique in industrial manufacturing, where a fixed camera observes a product (e.g., on a conveyor belt) and checks whether it meets certain requirements. These operations can be executed at a fairly high rate due to the simplicity of the system (e.g., static camera) and the simplification of the processing task (e.g., binary images). For both areas the identified difficulties are similar. Foremost, this is the slow nature of (robust) visual processing, in respect to the ever growing demand of increasing speed and reducing delay. These two application areas with analogous limitations motivate the design of more direct approaches of vision in visual control systems. Therefore, in order to meet the requirements for next generation visual control systems, this thesis presents approaches which employ visual measurements as a direct feedback to design constrained motion. First, for industrial robotics, in order to obtain the required positioning accuracy, the measurement and fixation system have to be highly rigid and welldesigned, implying high cost and long design time. By measuring the position of objects directly with a camera, instead of indirectly by motor encoders, the requirements of the measurement and fixation system are less demanding. Moreover, this motivates the miniaturization of the complete control system. The approach is validated in experiments on a simplified 2D planar stage (i.e., considerable friction, poor fixation), which attains similar performance compared to encoder-based positioning systems. Secondly, in a human-centred environment, this direct sensing can improve traditional visual control systems, when subject to certain disturbances. More specifically, a method is proposed that uses an image-based feedforward controller on top of traditional position-based visual servo control to overcome disturbances such as friction or poorly designed local motor controllers. This visual feedforward control action is only active when an image-based error is present and vanishes when that error goes to zero. The method is validated on an anthropomorphic robotic manipulator with 7 degrees of freedom, intended for operation in the human care environment. Third, sensing the product directly gives rise to designing motion directly. Whereas in traditional approaches the motion trajectory is designed offline and can not be changed at runtime, direct trajectory generation computes the motion of the next step based on current state and events. This means that at any instance in time, the trajectory of a motion system can be altered with respect to certain desired kinematic or dynamic constraints. For industrial applications this makes manufacturing on near-repetitive or non-rigid structures (e.g. flexible displays) possible. When applied to a robotic manipulator, this enables obstacle avoidance to no longer be on path planning level, but on trajectory planning level, where kinematic or dynamic constraints can be taken into account. This results in a motion that is smoother than when obstacle avoidance with path planning is employed. For both application areas this direct trajectory generation method is implemented and shows high flexibility in constrained motion trajectory design.

Tactile sensors can be used to build human-machine interfaces, for instance in isometric joysticks or handlebars. When used as input sensor device for control, questions arise related to the contact with the human, which involve ergonomic aspects. This paper focuses on the example application of driving a powered wheelchair as attendant. Since other proposals use force and torque sensors as control input variables, this paper explores the relationship between these variables and others obtained from the tactile sensor. For this purpose, a handlebar is instrumented with tactile sensors and a 6-axis force torque sensor. Several experiments are carried out with this handlebar mounted on a wheelchair and also fixed to a table. It is seen that it is possible to obtain variables well correlated with those provided by force and torque sensors. However, it is necessary to contemplate the influence of issues such as the gripping force of the human hand on the sensor or the different kinds of grasps due to different physical constitutions of humans and to the inherent random nature of the grasp. Moreover, it is seen that a first step is necessary where the contact with the hands has to stabilize, and its characteristics and settle time are obtained.

This paper describes design and simulation of five degrees of freedom (5-DOF) Micro-Electro-Mechanical systems (MEMS) based tactile force sensor. Tactile sensing involves with measuring physical parameters such as force, temperature, etc. with the aid of physical touch. Over the past decades tactile sensors are gaining popularity over non-contact sensors in biomedical and robotic applications. Proposed sensor design with 3mm × 3mm × 300µm dimensions, has the capability to measure not only the magnitude but also the direction of the force applied. A wagon wheel spring structure was proposed, where 8 beams work as springs to relief the force applied. Behavior of these 8 beams are monitored under each loading conditions using defused piezoresistive sensing elements. A finite element analysis of structure was performed to optimize and validate the structure and Multiphysics analysis was performed to validate the working principal of the proposed sensor.