Robot Force Estimation with Learned Intraoperative Correction

Abstract. Force sensing plays an important role in minimally invasive surgery (MIS). Force sensing makes it possible for the surgeon to feel the tissue properties and apply an appropriate level force and avoid tissue damage. The micromanipulators are compact and to allow appropriate disinfection, it is inappropriate to integrate sensors at the end of the micromanipulator. In this study, a new asymmetric cable-driven type of micromanipulator for a surgical robot was designed, and a joint angle estimator (JAE) was designed based on the dynamical model of the single cable-driven joint. Closed-loop control of the joint angle was carried out by regarding the JAE output as the feedback signal. On this basis, an external force estimator was designed using a disturbance observer (DOB). The experimental results show an average accuracy of the joint angle estimator of about −0.150°, with excellent control precision, the largest absolute error of about 0.95°, and an average error of 0.175°. The accuracy of the force estimator was at a high level during static loading. The estimated accuracy was 94 % at external force is greater than 1 N, and the estimated accuracy was 82 % for an external force of 0.3 N. These results predict that force sensing of a cable-driven micromanipulator in this paper can used to realize the micromanipulator's force feedback of a minimally invasive surgical robot.

Although technological advancements in robotic minimally invasive surgery (RMIS) have refined maneuverability and the risk of conducting surgical procedures, RMIS nonetheless necessitates that the surgeon is disassociated with the laparoscopic tool. By this very nature, RMIS elicits the approximation of pressure applied to the patient. Current robotic systems have a unilateral control structure, which lacks force feedback. The implementation of a force feedback system to monitor physical interactions between the robot and the patient provides the surgeon with an accurate force sensation. For this paper, the research aims to develop a Direct Force Reflection bilateral teleoperation system for a laparoscopic grasper. A force sensor was placed on the grasper to measure the amount of force needed for the controller. Likewise, an encoder was used to determine the angle of the grasper. Factors such as temperature and system response time were observed in the study to verify the performance of the bilateral teleoperation system. Results show that

A parallel spherical manipulator can provide three degrees of freedom of pure rotation around the virtual center of its platform. Parallel manipulators, where all actuators are fixed to the base of the manipulator, provide better load-carrying capacity and better dynamic properties. This advantage makes it possible to use a parallel spherical manipulator as an additional tool for rotating the end-effector in robots with an incomplete set of degrees of freedom or compensate rotation/position error of the master robot. This solution can be used in precise machining operations and other problems where external forces have great influence or other sources of position errors are significant.Under the influence of external forces robot's end-effector could have an offset in the positioning. In this work, we propose to use a specially designed parallel robot that mounted on the master robot flange between the master robot and its tool. In order to compensate position errors following steps is performed: 1) Estimation of master robot stiffness model; 2) Estimation of spherical parallel robot stiffness model; 3) Estimation of external forces; 4) Estimation of compliance errors for both robots; 5) Trajectory modification for spherical parallel robot; 6) Trajectory execution.In this way, the overall accuracy of the system could be greatly improved, without any modification to the master robot program.

Multi-axial force and torque sensing is of importance for robot control and many force-feedback applications. Minimal invasive robotic surgery (MIRS) is a possible field of application of force and torque sensors with up to six degrees of freedom. Although these sensors are not yet employed in current commercial MIRS systems, extensive work has been carried out on the development of these sensors. Some of their issues are related to their electric working principle: they are limited in performance by thermal noise, need electric power inside the patient and are not usable under influence of strong magnet fields (e. g. in MRI machines). One possible alternative is seen in fiber optic force torque sensors, since the signal demodulation may be located in some distance to the actual sensor and they also do not have to include any magnetic material. This article presents a fiber optic force and torque sensor with six degrees of freedom. The general setup resembles a Stewart Platform, whereas its connecting beams are formed by the fiber itself, and the element creating stiffness may be of arbitrary form. Only a single fiber is needed to extract all six parameters since they are measured on six multiplexed fiber Bragg grating sensors. We demonstrate how the sensor is realized and show results of torque measurements with variable load.

Although substantial progresses have been made in robot-assisted laparoscopic surgery, the graspers for existing surgical systems generally remain nonsensorized forceps design with limited functions. This article presents the design, development, and preliminary evaluation of the MUSHA Hand II, a multifunctional hand with force sensors for robot-assisted laparoscopic surgery. The proposed hand has three snake-like underactuated fingers that can be folded into a $\phi$ 12 mm cylindrical form. Each finger has a three-axis force sensor, to provide force information. After been deployed into an abdominal cavity, the hand can be configured to either grasper mode, retractor mode, or palpation mode for different tasks. Underactuated finger design enhances the adaptivity in grasping and the compliance in interaction with the environment. In addition, fingertip force sensors can be utilized for palpation to obtain a real-time stiffness map of organs. Using the da Vinci Research Kit as a robotic testbed, the functionality of the hand has been demonstrated and experiments have been conducted, including robotic palpation and organ manipulation. The results suggest that the hand can effectively enhance the functionality of a robotic surgical system and overcome the limits on force sensing introduced by the use of robots in laparoscopic surgery.

Surgery can become safer and more reliable if the conditions of the organs can be physically and quantitatively assessed through simultaneous intraoperative measurements and physical estimations of deformed organs. This study proposes a new method for estimating external forces based on local displacement of elastic bodies. Using this method, one can estimate the magnitude and position of forces not only on the forceps but also in the invisible field. Simulation studies confirmed the effectiveness of this method. The experimental results showed that partial observation of a deformed shape achieved successful estimation of the external forces.

Precise force control and position control is essential for many robotic applications. During unilateral or bilateral object handling remote applications, human operators apply excess forces on the actuators with the intention of better grip and it is possible to damage the handled objects. This paper proposes position control and force control techniques when human applies excess force. The controllers consist of predefined force and equilibrium position which are correspond to remote environmental object safety. The operator applied excess force is modeled as a spring force through a virtual stiffness model. The paper introduces proportional derivative position control loop to conventional virtual stiffness controller. The force control loop and the position control loop references are combined in the common dimension of acceleration to produce current reference to the actuator. The sensorless sensing techniques of disturbance observer and reaction force observer are utilized to disturbance suppression and external force estimation. In the force controller, estimated reaction force is reduced to desired value by introducing virtual force loop between reaction force observer output and reaction force feedback input. The position control is achieved by introducing a scaling factor to reaction force feedback path. The switching between force control and position control is possible with scaling factor value. The performances of proposed methods are compared with conventional method and validity of the proposed methods is verified by simulations and experiments.

This paper presents a method to estimate external force which is applied at the tip of a robotic arm. Piezoelectric actuator has been used as linear motor into this system and provide force estimation. The disturbance system has been divided into two parts: external force and design trajectory tracking. An external force is applied to the piezoelectric linear model to estimate current external force, without force sensor which is normally used in telesurgery or micro robotics. The displacement can be controlled by feedback signal from adaptive control to track the design trajectory tracking displacement. If the unknown parameters in this system, this adaptive controller can also approximate unknown parameters and make system stable. Sensorless force estimation is applied with four controllers to compare the results.

To address inaccurate external force estimation caused by nonlinear friction in robotic systems, this paper proposes a friction compensation and external force estimation method based on an adaptive neuro-fuzzy inference system (ANFIS). The approach integrates Stribeck friction modeling with a Takagi–Sugeno fuzzy inference structure to identify joint friction parameters from measured data. Experimental results show that ANFIS yields lower identification errors and better generalization performance than baseline methods including fuzzy neural networks, particle swarm optimization, and least squares. The implemented feedforward compensation strategy achieves maximum torque errors of 0.263 Nm and 0.184 Nm for the two joints, lower than those obtained by the compared approaches. By incorporating the identified friction model into a generalized momentum observer with median and Butterworth filtering, the proposed method reduces the root mean square error and maximum absolute error by 18.3 % and 27.9 %, respectively, and achieves a coefficient of determination (R²) of 0.994. In collision detection tests, the method identifies impact events with reduced false alarm rates under the same experimental settings, supporting its applicability to high-precision force control in robotic applications.

In human-robot collaboration, physical hand-over-hand communication of the human operator’s intention is essential for realizing direct and intuitive cooperation. When an external force acts on a robot hand holding an object, the torque exerted on the finger motor becomes a combination of the grasping torque required to maintain the grasping pose and reaction torque to the external force. To analyze intended interaction, we need to recognize multiple contacts inside and outside the fingertips. In this study, an external force estimation method was developed that uses tactile sensors to measure both grasping and external forces acting on the fingertips of a robot hand. Because a tactile sensor cannot measure all the external force components, it is impossible to directly compute the magnitude and contact angle of an external force. The shear and frictional forces lost during tactile sensing are backwardly estimated according to the difference between the estimated and measured torques. The magnitude and contact angle are computed to estimate external force, which can be used to determine the intention of the user. Simulations based on real sensor responses and experiments were performed to validate the proposed estimation method.

External disturbance forces caused by nonlinear springy electrical cables in the Master Tool Manipulator (MTM) of the da Vinci Research Kit (dVRK) limits the usage of the existing gravity compensation methods. Significant motion drifts at the MTM tip are often observed when the MTM is located far from its identification trajectory, preventing the usage of these methods for the entire workspace reliably. In this paper, we propose a general and systematic framework to address the problems of the gravity compensation for the MTM of the dVRK. Particularly, high order polynomial models were used to capture the highly nonlinear disturbance forces and integrated with the Multi-step Least Square Estimation (MLSE) framework. This method allows us to identify the parameters of both the gravitational and disturbance forces for each link sequentially, preventing residual error passing among the links of the MTM with uneven mass distribution. A corresponding gravity compensation controller was developed to compensate the gravitational and disturbance forces. The method was validated with extensive experiments in the majority of the manipulator's workspace, showing significant performance enhancements over existing methods. Finally, a deliverable software package in MATLAB and C++ was integrated with dVRK and published in the dVRK community for open-source research and development.

This paper establishes an approach to external force estimation through the use of a mathematical model and current sensing, without employing a force/torque sensor. The advantages and need for force feedback have been well established in the field of telerobotics. This paper presents the requirement for sensorless force estimation and comparative results between a force sensor and the presented approach using an industrial robot. The approach presents not only a cost effective solution but also a solution for force sensing in hazardous environments, especially ionizing radiation prone environments where the dose rates limit the use of sensing equipment. The paper also discusses the applications and advantages presented by this work in various fields.

An application of psychophysical ratings to external force estimation

In this paper, a new robust adaptive nonlinear teleoperation system using an improved extended active observer (IEAOB), adaptive Smith predictor (ASP) and sliding mode control is developed to address the time delay in the communication channels and the nonlinear robot model uncertainties. Firstly, an ASP based on Padé approximation and active observer is designed to compensate for the time delay effect. Specifically, the total network time delay is modelled by Padé approximation, and then an active observer is deployed to estimate the time delay. To ensure the time-varying delay effect is completely suppressed, a sliding mode control algorithm is further developed. The main added value of this teleoperation approach is that it requires neither specific mathematic delay-time model, nor strict assumptions on time delay from a practical point of view. Finally, the stability of the designed teleoperation system is theoretically studied and the system effectiveness is demonstrated by applying it to a pair of Phantom Omni haptic devices connected via a communication channel with time-varying delays.

Human-robot interaction (HRI) is a rapidly growing research area and it occurs in many applications including human-robot collaboration, human power augmentation, and rehabilitation robotics. As it is hard to exactly calculate intended motion trajectory, generally, interaction control is applied in HRI instead of pure motion control. To implement the interaction control, force information is necessary and force sensor is widely used in force feedback. However, force sensor has some limitations as 1) it is subject to breakdown, 2) it imposes additional volume and weight to the system, and 3) its applicable places are constrained. In this situation, force estimation can be a good solution. However, if force in static situation should be measured, using position and velocity is not sufficient because they are not influenced by the exerted force anymore. Therefore, we proposed sEMG-based static force estimation using deep learning. sEMG provides a useful information about human-exerting force because it reflects the human intention. Also, to extract the complex relationship between sEMG and force, deep learning approach is used. Experimental results show that when force with maximal value of 63.2 N is exerted, average force estimation error was 3.67 N. Also, the proposed method shows that force onset timing of estimated force is faster than force sensor signal. This result would be advantageous for faster human intention recognition.