SPRK: A Low-Cost Stewart Platform For Motion Study In Surgical Robotics

Automation of surgical processes (SPs) is an utterly complex, yet highly demanded feature by medical experts. Currently, surgical tools with advanced sensory and diagnostic capabilities are only available. A major criticism towards the newly developed instruments that they are not fitting into the existing medical workflow often creating more annoyance than benefit for the surgeon. The first step in achieving streamlined integration of computer technologies is gaining a better understanding of the SP. Surgical ontologies provide a generic platform for describing elements of the surgical procedures. Surgical Process Models (SPMs) built on top of these ontologies have the potential to accurately represent the surgical workflow. SPMs provide the opportunity to use ontological terms as the basis of automation, allowing the developed algorithm to easily integrate into the surgical workflow, and to apply the automated SPMs wherever the linked ontological term appears in the workflow. In this work, as an example to this concept, the subtask level ontological term “blunt dissection” was targeted for automation. We implemented a computer vision-driven approach to demonstrate that automation on this task level is feasible. The algorithm was tested on an experimental silicone phantom as well as in several ex vivo environments. The implementation used the da Vinci surgical robot, controlled via the Da Vinci Research Kit (DVRK), relying on a shared code-base among the DVRK institutions. It is believed that developing and linking further building blocks of lower level surgical subtasks could lead to the introduction of automated soft tissue surgery. In the future, the building blocks could be individually unit tested, leading to incremental automation of the domain. This framework could potentially standardize surgical performance, eventually improving patient outcomes.

Automating repetitive surgical subtasks such as suturing, cutting and debridement can reduce surgeon fatigue and procedure times and facilitate supervised tele-surgery. Programming is difficult because human tissue is deformable and highly specular. Using the da Vinci Research Kit (DVRK) robotic surgical assistant, we explore a “Learning By Observation” (LBO) approach where we identify, segment, and parameterize motion sequences and sensor conditions to build a finite state machine (FSM) for each subtask. The robot then executes the FSM repeatedly to tune parameters and if necessary update the FSM structure. We evaluate the approach on two surgical subtasks: debridement of 3D Viscoelastic Tissue Phantoms (3d-DVTP), in which small target fragments are removed from a 3D viscoelastic tissue phantom; and Pattern Cutting of 2D Orthotropic Tissue Phantoms (2d-PCOTP), a step in the standard Fundamentals of Laparoscopic Surgery training suite, in which a specified circular area must be cut from a sheet of orthotropic tissue phantom. We describe the approach and physical experiments with repeatability of 96% for 50 trials of the 3d-DVTP subtask and 70% for 20 trials of the 2d-PCOTP subtask. A video is available at: http://j.mp/Robot-Surgery-Video-Oct-2014.

Surgical movements have an important stylistic quality that individuals without formal surgical training can use to identify expertise. In our prior work, we sought to characterize quantitative metrics associated with surgical style and developed a near-real-time detection framework for stylistic deficiencies using a commercial haptic device. In this paper, we implement bimanual stylistic detection on the da Vinci Research Kit (dVRK) and focus on one stylistic deficiency, "Anxious", which may describe movements under stressful conditions. Our goal is to potentially correct these "Anxious" movements by exploring the effects of three different types of haptic cues (time-variant spring, damper, and spring-damper feedback) on performance during a basic surgical training task using the da Vinci Research Kit (dVRK). Eight subjects were recruited to complete peg transfer tasks using a randomized order of haptic cues and with baseline trials between each task. Overall, all cues lead to a significant improvement over baseline economy of volume and time-variant spring haptic cues lead to significant improvements in reducing the classified "Anxious" movements and also corresponded with significantly lower path length and economy of volume for the non-dominant hand. This work is the first step in evaluating our stylistic detection model on a surgical robot and could lay the groundwork for future methods to actively and adaptively reduce the negative effect of stress in the operating room.

Research in surgical robotics and automation has made remarkable advancements in recent years thanks to new methods in computer vision, control, and deep learning. Autonomous end-effector manipulation is a challenging task in surgical robotics, and cutting with scissor tools is largely unexplored. A concurrent work explored path and trajectory generation for cutting deformable materials using the da Vinci Research Kit (dVRK) [1]. However, an efficient and realistic simulation is necessary for methods such as reinforcement learning (RL) or learned trajectory planning. Our previous work built a simulation for the dVRK in Unity for training RL algorithms on rigid body tasks [2]. To our knowledge, there is no dVRK simulation available that includes the cutting of deformable materials. This paper introduces a cutting simulation of a deformable mesh, which can represent a tissue layer, built onto our Unity dVRK simulation.

Although substantial progresses have been made in robot-assisted laparoscopic surgery, the graspers for existing surgical systems generally remain nonsensorized forceps design with limited functions. This article presents the design, development, and preliminary evaluation of the MUSHA Hand II, a multifunctional hand with force sensors for robot-assisted laparoscopic surgery. The proposed hand has three snake-like underactuated fingers that can be folded into a $\phi$ 12 mm cylindrical form. Each finger has a three-axis force sensor, to provide force information. After been deployed into an abdominal cavity, the hand can be configured to either grasper mode, retractor mode, or palpation mode for different tasks. Underactuated finger design enhances the adaptivity in grasping and the compliance in interaction with the environment. In addition, fingertip force sensors can be utilized for palpation to obtain a real-time stiffness map of organs. Using the da Vinci Research Kit as a robotic testbed, the functionality of the hand has been demonstrated and experiments have been conducted, including robotic palpation and organ manipulation. The results suggest that the hand can effectively enhance the functionality of a robotic surgical system and overcome the limits on force sensing introduced by the use of robots in laparoscopic surgery.

In a teleoperated system, misalignment between the master and slave manipulators can result from clutching, errors in the kinematic model, and/or sensor errors. This study examines the effects of type and magnitude of misalignment on the performance of the teleoperator. We first characterized the magnitude and direction of orientation misalignment created when clutching and unclutching during use of two surgical robots: the Raven II and the da Vinci Research Kit. We then purposely generated typical misalignments in order to measure the impact of such misalignment on user performance of a peg transfer task with the Raven II. Users were able to compensate for misalignment angles up to approximately 20 degrees in both tool orientation and camera viewpoint misalignment. These results can be used to guide the design and control of teleoperated systems for a variety of applications.

Robotic fabric manipulation has applications in home robotics, textiles, senior care and surgery. Existing fabric manipulation techniques, however, are designed for specific tasks, making it difficult to generalize across different but related tasks. We extend the Visual Foresight framework to learn fabric dynamics that can be efficiently reused to accomplish different fabric manipulation tasks with a single goal-conditioned policy. We introduce VisuoSpatial Foresight (VSF), which builds on prior work by learning visual dynamics on domain randomized RGB images and depth maps simultaneously and completely in simulation. We experimentally evaluate VSF on multi-step fabric smoothing and folding tasks against 5 baseline methods in simulation and on the da Vinci Research Kit (dVRK) surgical robot without any demonstrations at train or test time. Furthermore, we find that leveraging depth significantly improves performance. RGBD data yields an 80% improvement in fabric folding success rate over pure RGB data. Code, data, videos, and supplementary material are available at https://sites.google.com/view/fabric-vsf/.

With progressive advancements in modern technologies, image-guided automation, which plans and manipulates particular tasks by utilizing visual cues, is becoming an emerging sector in medical robotics. Its prevalence nowadays requires a reliable hand-to-eye calibration owing to various configurations of surgical robots, especially for those with remote center of motion (RCM) constraints, e.g., da Vinci Research Kit (dVRK). In this article, we proposed a novel unified calibration method, which leverages the structure characteristics and establishes an explicit geometric model to solve the hand-to-eye relationship only with a monocular camera's feedback. By freely moving the patient side manipulator in dVRK, our method can first recover the 3-D poses of its shaft at various configurations using Plücker coordinates. To obtain the hand-to-eye translation relationship, these recovered poses are further converged by softly intersecting them within a minimal spatial sphere while preserving their coordinates using an aggregating sphere loss. Meanwhile, the orientation mapping can be recovered by leveraging the corresponding longitudinal-axis information. To improve the calibration accuracy, a local optimization strategy that utilizes the bundle adjustment to refine the transformation using the previous outcomes is also introduced. With such a hierarchical calibration scheme, the hand-to-eye relationship of any RCM constrained robot can be successfully achieved. Extensive simulations and experiments are conducted based on dVRK platform. By comparing with the traditional method, the validation results prove the feasibility and superiority of our unified algorithm in calibration accuracy and robustness.

This paper describes the implementation of a heart motion compensation system on the da Vinci surgical system (Intuitive Surgical Inc.) with the da Vinci Research Kit, for the purpose of simulating minimally invasive coronary artery bypass surgery on the beating heart. A Novint Falcon device is used to simulate the motion of the coronary bypass site. The 3D position of the heart simulator is measured optically in real time using an NDI Optotrak Certus tracker. The NDI measurements are used to command the da Vinci patient side manipulators to track, from a fixed distance, the simulated heart surface. Visual stabilization is achieved by having a stereo endoscopic camera carried by another patient-side manipulator that is also tracking the heart surface. The system performance is evaluated and discussed. In a preliminary evaluation study, surgeons were asked to perform a bimanual teleoperation suturing task on the simulated heart, with the gold standard being a suturing task on a stationary surface. When motion compensation was enabled, the median completion time dropped from 1.24 to 1.07 of the gold standard, the number of errors was reduced, and subjective measures show higher preference for the use of motion compensation in the da Vinci controllers.

In this paper, a real-time control network based on Ethernet and FireWire is presented, where Ethernet provides a convenient, cross-platform interface between a central control PC and a FireWire subnetwork that contains multiple distributed nodes (I/O boards). Real-time performance is achieved because this architecture limits the number of Ethernet transactions on the host PC, benefits from the availability of real-time Ethernet drivers, and uses the broadcast and peer-to-peer capabilities of FireWire to efficiently transfer data among the distributed nodes. This approach and resulting benefits are comparable to EtherCAT, but preserves existing investments in FireWire-based controllers and relies only on conventional, vendor-neutral network hardware and protocols. The system performance is demonstrated on the da Vinci Research Kit (dVRK), which consists of 8 FireWire nodes that control 2 Master Tool Manipulators (MTMs) and 2 Patient Side Manipulators (PSMs), for a total of 28 axes. This approach is generally applicable to interface existing FireWire-based systems to new control PCs via Ethernet or to serve as an open-source alternative to EtherCAT for new designs.

Master manipulators represent a key component of Robot-Assisted Minimally Invasive Surgery (RAMIS). In this paper, an Analytic Hierarchy Process (AHP) method is used to construct an effective workspace analysis framework, which can assist the configuration selection and design evaluation of a portable master manipulator for surgical robot control and training. The proposed framework is designed based on three criteria: 1) compactness, 2) workspace quality, and 3) mapping efficiency. A hardware prototype, called the Hamlyn Compact Robotic Master (Hamlyn CRM), is constructed following the proposed framework. Experimental verification of the platform is conducted on the da Vinci Research Kit (dVRK) with which a da Vinci robot is controlled as a slave. The proposed Hamlyn CRM is compared with Phantom Omni, a commercial portable master device, with results demonstrating the relative merits of the new platform in terms of task completion time, average control speed and number of clutching.

Conformity of tumour volumes and dose plans in prostate brachytherapy (BT) can be constrained by unwanted needle deflections, needle access restrictions and visualisation limitations. This work validates the feasibility of teleoperated robotic control of an active steerable needle using magnetic resonance (MR) for guidance. With this system, perturbations can be counteracted and critical structures can be circumvented to access currently inaccessible areas. The system comprises of (1) a novel steerable needle, (2) the minimally invasive robotics in an MR environment (MIRIAM) system, and (3) the daVinci Research Kit (dVRK). MR scans provide visual feedback to the operator controlling the dVRK. Needle steering is performed along curved trajectories to avoid the urethra towards targets (representing tumour tissue) in a prostate phantom with a targeting error of 1.2 ± 1.0 mm. This work shows the potential clinical applicability of active needle steering for prostate BT with a teleoperated robotic system in an MR environment.

The use of flight simulators for pilot training and evaluations is common in the modern aeronautical sector. Whether for airplanes or helicopters, the military and airline companies use this feature to keep their crews operational, safety-oriented and resource-saving. This work presents a proposal for adjusting the washout filter parameters that allows the use of Stewart's platform in helicopter simulators. The identification of the filter parameters was carried out based on tests on an available commercial platform, certified by the company Moog, model MB-E-6DOF/24/1800KG, currently used in the SHEFE helicopter simulator of the Brazilian Army. The work dealt with the physiological aspects of the human vestibular system and its dynamics. Subsequently, the methods for choosing the filter model, and configuration parameters are presented. Three series of tests were carried on the commercial platform and in a real helicopter prepared with special flight test instrumentation. The tests were completed, and the motion platform filter adjustments were made to minimize errors between the movement cues perceived by the pilot in the aircraft and the flight simulator. The results of this work will be used as a basis for the development of a national movement platform for another flight simulator in the development phase.

Most of the calibration methods proposed for the Stewart platform require complex computation or low noise data for the platform's accuracy to be determined. They are not suitable for practical use in a production environment, where the measurement and calibration method should be simple and robust. Using an external laser measuring device to determine the actual accuracy of a Stewart platform, a practical and simple leg length compensating calibration method, that improves the accuracy of the Stewart platform by a magnitude of around 7, is proposed. The procedures and computation algorithms of the calibration method are shown.

Measurement of environment interaction forces during robotic minimally-invasive surgery would enable haptic feedback to the surgeon, thereby solving one long-standing limitation. Estimating this force from existing sensor data avoids the challenge of retrofitting systems with force sensors, but is difficult due to mechanical effects such as friction and compliance in the robot mechanism. We have previously shown that neural networks can be trained to estimate the internal robot joint torques, thereby enabling estimation of external forces on the da Vinci Research Kit (dVRK). In this work, we extend the method to estimate external Cartesian forces and torques, and also present a two-step approach to adapt to the specific surgical setup by compensating for forces due to the interactions between the instrument shaft and cannula seal and between the trocar and patient body. Experiments show that this approach provides estimates of external forces and torques within a mean root-mean-square error (RMSE) of 1.8N and 0.1Nm, respectively. Furthermore, the two-step approach can add as little as 5 minutes to the surgery setup time, with about 4 minutes to collect intraoperative training data and 1 minute to train the second-step network.