Real-Time Sclera Force Feedback for Enabling Safe Robot-Assisted Vitreoretinal Surgery.

Control elements with active haptic feedback have become established in modern automotive user interfaces, but numerous media reports and studies indicate customer dissatisfaction with the current design. To investigate the customer preference that has not been well-characterized so far and to deduce whether differences in preference can be linked to a driving task or customer attributes, a Preliminary-Study with adjective rating, an Expert-Study with pairwise comparison, and a Customer-Study with an additional real driving task were conducted. The results show different customer preference groups, with the vast majority favoring short haptic feedback within 13.6 ms and 20.6 ms length. A driving task does not influence preference, nor is the preference affected by attributes such as gender, age, or thumb size. These findings can be used to optimize active haptic feedback according to customer preferences. As a result, well-designed haptics can increase customer satisfaction and the perceived value of automotive controls.

Hysteroscopy is a type of gynecological procedure that enables diagnosis and treatment of intrauterine pathologies using a minimally-invasive approach. Recent advancements, especially in optics, made it possible to perform this procedure in an ambulatory, outpatient setting without anaesthesia. Yet, such approach introduces additional challenges to gynecologists, as the surgical instruments have to pass through the cervix, a narrow curved canal. This process can be traumatic and even cause tissue damage or rupture. Current training curricula mainly focus on general minimally-invasive surgical skills or on inpatient hysteroscopy. This paper describes the first virtual reality training system designed purposely for in-office hysteroscopy training featuring both passive and active haptic feedback. Design requirements were defined through a literature review and a survey of clinical experts. A dedicated platform was developed and tested with clinicians and medical students. The results of the conducted experiments verified the skill acquisition capabilities of the platform showing a.o. shorter execution time and path length, lower integral forces in most of non-expert participants. The results also demonstrated the capability of the system to differentiate between experts and non-expert groups. Reported findings suggest that the platform has the potential to evolve into an effective solution for in-office hysteroscopy training.

Retinal microsurgery is technically demanding and requires high surgical skill with very little room for manipulation error. During surgery the tool needs to be inserted into the eyeball while maintaining constant contact with the sclera. Any unexpected manipulation could cause extreme tool-sclera contact force (scleral force) thus damage the sclera. The introduction of robotic assistance could enhance and expand the surgeon's manipulation capabilities during surgery. However, the potential intra-operative danger from surgeon's misoperations remains difficult to detect and prevent by existing robotic systems. Therefore, we propose a method to predict imminent unsafe manipulation in robot-assisted retinal surgery and generate feedback to the surgeon via auditory substitution. The surgeon could then react to the possible unsafe events in advance. This work specifically focuses on minimizing sclera damage using a force-sensing tool calibrated to measure small scleral forces. A recurrent neural network is designed and trained to predict the force safety status up to 500 milliseconds in the future. The system is implemented using an existing "steady hand" eye robot. A vessel following manipulation task is designed and performed on a dry eye phantom to emulate the retinal surgery and to analyze the proposed method. Finally, preliminary validation experiments are performed by five users, the results of which indicate that the proposed early warning system could help to reduce the number of unsafe manipulation events.

Haptic technology allows producing computer-generated signals that a perceiver experiences through touch. In general, haptic devices can be classified as active and passive. In this paper, we present a performance evaluation of a group of 10 voluntary subjects in a task of shape perception when both active and passive kinesthetic haptic feedback are provided. The results of a series of psychophysical experiments show that active haptics is superior for shape recognition: active haptics enhances the perception of surfaces while passive haptics conveys the sensation of being guided along pathways. In addition, the guidance nature of passive haptics makes the pathway direction easy to recognize. This last observation can be effectively exploited in more challenging tasks such as navigation/mobility assistance.

We present a technique that enables both grasping and injecting a biological cell with real-time haptic force feedback by using miniature compliant mechanisms. The compliant mechanisms serve the dual purpose of grasping and force-sensing. The setup developed comprises an inverted microscope, two XYZ stages for micro-positioning, two usage-specific polydimethylsiloxane (PDMS) tools, a haptic robot, and a camera for image capture. Soft PDMS grippers help in holding the cells gently without excessive force because their stiffness can be matched with that of the cells. The setup allows the transfer of user-induced motion from the stylus of the haptic device (master) to the miniature grasper/injector (slave) and, in reverse, the force senses at the grasper is returned to the user with suitable de-amplification and amplification, respectively. It is demonstrated that a single haptic device can be used to grasp and inject by switching between two modes by toggling a button on the stylus of the device. The haptic cell-manipulation system was tested by grasping and injecting a zebrafish egg cell.KeywordsCompliant mechanismsCell-manipulationRemote-hapticsMiniature gripperIntracytoplasmic injection

New technological progress in robotics has brought many beneficial clinical applications. Currently, computer integrated robotic surgery has gained clinical acceptance for several surgical procedures. Robotically assisted eye surgery is envisaged as a promising solution to overcome the shortcomings inherent to conventional surgical procedures as in vitreoretinal surgeries. Robotics by its high precision and fine mechanical control can improve dexterity, cancel tremor, and allow highly precise remote surgical capability, delicate vitreoretinal manipulation capabilities. Combined with magnified three-dimensional imaging of the surgical site, it can enhance surgical precision. Tele-manipulation can provide the ability for tele-surgery or haptic feedback of forces generated by the manipulation of intraocular tissues. It presents new solutions for some sight-threatening conditions such as retinal vein cannulation where, due to physiological limitations of the surgeon's hand, the procedure cannot be adequately performed. In this paper, we provide an overview of the research and advances in robotically assisted vitreoretinal eye surgery. Additionally the barriers to the integration of this method in the field of ocular surgery are summarized. Finally, we discuss the possible applications of the method in the area of vitreoretinal surgery.

Pseudo haptic sensation is an illusion based on visual stimuli. In virtual environments the principle of pseudo haptic feedback is used to simulate material properties such as stiffness, mass and friction. Transfering the principle of pseudo haptic feedback to real haptic teleoperation systems can provide a haptic sensation for properties of material, manipulated without active haptic feedback. Hence, the implementation of pseudo haptic feedback to teleoperation systems reduces the requirements for kinematic structures and actuators of the used haptic user interface.

Acute and chronic pain after median sternotomy is common and often underestimated. The mechanical retractors used for median sternotomy exert significant forces on the skeletal cage. We hypothesized that instrumented retractors could be developed to enable real-time monitoring and control of retraction forces, functions that may provide equivalent exposure with significantly reduced forces and tissue damage, and thus, less postoperative pain. We developed a novel instrumented retractor designed to enable real-time force monitoring during surgical retraction and then tested it by performing median sternotomies on 16 mature sheep. For 8 of these median sternotomies, retraction was performed to 7.5 cm at a standard "clinical pace" of 7.25 +/- 0.97 minutes without real-time monitoring of retraction forces. For the other 8 median sternotomies, we performed retraction to the same exposure using real-time visual force feedback and, consequently, a more deliberate pace of 12.05 +/- 1.73 minutes (P <.001). Retraction forces, blood pressure, and heart rate were monitored throughout the procedure. Full retraction resulted in an average force of 102.99 +/- 40.68 N at the standard clinical pace, compared to 64.68 +/- 17.60 N with force feedback (a 37.2% reduction, P = .021). Standard retraction produced peak forces of 368.79 +/- 133.61 N, whereas force feedback yielded peak forces of 254.84 +/- 75.77 N (a 30.9% reduction, P = .1152). Heart rate was significantly higher during standard clinical retraction (P = .025). Use of the novel instrumented retractor resulted in lower average and peak retraction forces during median sternotomy. Moreover, these reduced retraction forces correlated to a reduction in animal stress, as documented by heart rate.

Ion homeostasis and tension-sensitive surface area regulation: modeling autonomous cellular processes for trans-membrane and within-membrane homeostasis

BackgroundA lack of force feedback in laparoscopic surgery often leads to a steep learning curve to the novices and traditional training system equipped with force feedback need a high educational cost. This study aimed to use a laparoscopic grasper providing force feedback in laparoscopic training which can assist in controlling of gripping forces and improve the learning processing of the novices.MethodsFirstly, we conducted a pre-experiment to verify the role of force feedback in gripping operations and establish the safe gripping force threshold for the tasks. Following this, we proceeded with a four-week training program. Unlike the novices without feedback (Group A2), the novices receiving feedback (Group B2) underwent training that included force feedback. Finally, we completed a follow-up period without providing force feedback to assess the training effect under different conditions. Real-time force parameters were recorded and compared.ResultsIn the pre-experiment, we set the gripping force threshold for the tasks based on the experienced surgeons’ performance. This is reasonable as the experienced surgeons have obtained adequate skill of handling grasper. The thresholds for task 1, 2, and 3 were set as 0.731 N, 1.203 N and 0.938 N, respectively. With force feedback, the gripping force applied by the novices with feedback (Group B1) was lower than that of the novices without feedback (Group A1) (p < 0.005). During the training period, the Group B2 takes 6 trails to achieve gripping force of 0.635 N, which is lower than the threshold line, whereas the Group A2 needs 11 trails, meaning that the learning curve of Group B2 was significantly shorter than that of Group A2. Additionally, during the follow-up period, there was no significant decline in force learning, and Group B2 demonstrated better control of gripping operations. The training with force feedback received positive evaluations.ConclusionOur study shows that using a grasper providing force feedback in laparoscopic training can help to control the gripping force and shorten the learning curve. It is anticipated that the laparoscopic grasper equipped with FBG sensor is promising to provide force feedback during laparoscopic training, which ultimately shows great potential in laparoscopic surgery.

This paper describes the design and implementation of a control system that integrates visual and haptic information to give assistive force feedback through a haptic controller (Omni Phantom) to the user. A sensor-based assistive function and velocity scaling program provides force feedback that helps the user complete trajectory following exercises for rehabilitation purposes. This system also incorporates a PUMA robot for teleoperation, which implements a camera and a laser range finder, controlled in real time by a PC, were implemented into the system to help the user to define the intended path to the selected target. The real-time force feedback from the remote robot to the haptic controller is made possible by using effective multithreading programming strategies in the control system design and by novel sensor integration. The sensor-based assistant function concept applied to teleoperation as well as shared control enhances the motion range and manipulation capabilities of the users executing rehabilitation exercises such as trajectory following along a sensor-based defined path. The system is modularly designed to allow for integration of different master devices and sensors. Furthermore, because this real-time system is versatile the haptic component can be used separately from the telerobotic component; in other words, one can use the haptic device for rehabilitation purposes for cases in which assistance is needed to perform tasks (e.g., stroke rehab) and also for teleoperation with force feedback and sensor assistance in either supervisory or automatic modes.

6-DOF force feedback control of robot-assisted bone fracture reduction system using double F/T sensors and adjustable admittances to protect bones against damage

Force feedback gloves allow users to touch and manipulate virtual objects intuitively. Compared with gloves providing active feedback force, gloves with passive feedback force are promising in terms of safety and low weight, but simulating the variable stiffness of virtual objects is more challenging. Addressing this difficulty, we propose a five-fingered glove with passive force feedback employing a variable ratio lever mechanism. The stiffness of the proposed glove is tuned by changing the structural stiffness of this mechanism rather than by applying torque control at each joint of the finger. The switch between free and constrained space is realized in real time by locking/unlocking the revolute joints of the glove using a servo motor. Furthermore, a predictive control mode is proposed to reduce the response time of the control system, and the actual response time is less than the limit of the delay (45 ms) that humans can perceive between visual and haptic stimuli. Experimental results show that the linear stiffness at the fingertip ranges from 0.89 to 619.89 N/m, and the maximum backdrive force of the proposed glove is less than 0.147 N.

Micro Air Vehicles (MAVs) have become more attractive for various missions including surveillance or reconnaissance in recent years. MAVs should be capable of maintaining their attitudes through either inherent passive stability or active feedback in order to successfully perform their directives. Stability and Controllability Augmentation Systems (SCASs) are usually employed to enhance the flight performance of conventional aircrafts and Unmanned Aerial Vehicles (UAVs). However, it is no simple task to obtain an accurate numerical model for the flight dynamics of a MAV. An alternative approach for SCASs would be to incorporate reinforcement learning in order to address this numerical complexity. Such implementation has already been successful in other vehicles, such as unmanned ground vehicles (UGVs), because of their bettered stability compared to aerial vehicles. However, in order to train MAVs to learn how to fly, they must first be airborne. Similar to teaching infants how to walk, this paper presents a new method to provide an effective environment where a MAV can learn how to fly. A test setup was constructed to enable magnetic levitation of a MAV embedded with a permanent magnet. This apparatus allows for flexible experimentation: the position and the altitude of the MAV, the constraint forces, and the resulting moments are all adjustable and fixable. This ‘Pseudo Flight Environment’ was demonstrated with a fixed wing MAV model. In order for the model to maintain a constant altitude, a height hold control system was devised and implemented.

In this paper, we designed a 8 degrees of freedom (DOFs) haptic device for applications in minimally invasive surgical robot. The device can provide three translational, three rotational and a grasping motion and force feedback capability. It is composed of three parts, including an arm mechanism, a redundant wrist mechanism and a grasper mechanism. The kinematics and gravity compensation algorithms are also detailed in the paper. In addition, the haptic device and a slave surgical robot for minimally invasive surgery (MIS) developed by our lab are integrated as a master-slave surgical robotic system in this paper. In the master-slave robotic system, a new control system is designed to realize real-time mater-slave control based on EtherCAT bus technology. Experiments show that the haptic device can effectively compensate gravity at any position in its workspace and successfully realize master-slave operation by the control method, which prove the haptic device designed in this paper can be used as a master manipulator to control the surgical robot.