Image-based Visual Servoing of a Gough—Stewart Parallel Manipulator using Leg Observations

IMAGE-BASED VISUAL SERVOING OF GOUGH-STEWART PARALLEL MANIPULATORS USING LEGS OBSERVATION

This letter deals with the development of a vision-based controller for a continuum robot architecture. More precisely, the controlled robotic structure is based on three-tube concentric tube robot (CTR), an emerging paradigm to design accurate, miniaturized, and flexible endoscopic robots. This approach has grown considerably in the recent years finding applications in numerous surgical disciplines. In contrast to conventional robotic structures, CTR kinematics arise many challenges for an optimal control, such as friction, torsion, shear, and nonlinear constitutive behavior. In fact, in order to ensure efficient and reliable control, in addition to computing an analytical and complete kinematic model, it is also important to close the control loop. To do this, we developed an eye-in-hand visual servoing scheme using a millimeter-sized camera embedded at the robot's tip. Both the kinematic model and the visual servoing controller were successfully validated in simulation with visual servoing platform and using an experimental setup. The obtained results showed satisfactory performances for three-degrees of freedom positioning and path following tasks with adaptive gain control.

In this paper, the robustness of a new visual servoing scheme with respect to camera calibration errors is analyzed. This scheme, called 2D 1/2 visual servoing, is based on the estimation of the partial camera displacement from the current to the desired camera poses at each iteration of the control law. Visual features and data extracted from the partial displacement allow as to design a decoupled control law controlling the six camera DOF. The necessary and sufficient conditions for local asymptotic stability in presence of camera calibration errors are easily obtained. Then, thanks to the simple structure of the system, sufficient conditions for global asymptotic stability are proposed. Finally, experimental results show the validity of our approach and its robustness not only with respect camera calibration errors but also to robot calibration errors.

Since quadrotors are becoming more popular, several techniques appeared to control robustly the Unmanned Aerial Vehicle. This article proposes an intelligent image-based visual servoing (IBVS) control method for an unmanned aerial vehicle (UAV). This work consists of computing the proportional controller gain by smoothing the error between the desired and actual feature vectors values, as opposed to the classical visual servoing (VS) scheme. The weak handling of this gain, affect the maintaining of tracked features in the area of view. Unlike the classical VS scheme that requires a known value of gain, selected in a heuristic way, which lead the controller to fall in the velocity convergence problem. We adjust an adaptive gain value by the use of a fuzzy logic controller. The proposed controller exploits the error between feature vectors and its derivative as inputs. Furthermore, we design an intelligent based-fuzzy logic method for gain tuning of PD controllers applied for handling the heading and position of the quadrotor. The proposed controllers are applied to the inner and outer loop, offer several advantages in dealing the uncertainty occurred within trajectory tracking control. Various simulations are developed in MATLAB, shows the outperformance of the proposed method against the classical one.

In minimally invasive surgery, endoscopes provide visual guidance to surgical operations. To tackle the problems caused by traditional hand-held endoscope, robots are introduced to steer the endoscope.Image based instrument tracking is an effective way to ease the human-robot-interaction. In this article, we aim to design a 3D visual servoing scheme for a robotic flexible surgical endoscope with algorithm-based RCM constraint. The endoscope consists of two identical cameras and is based on continuum mechanism, mounted on the tip of a UR5 robot arm.This article conducts kinematic modeling of the flexible robotic endoscope to get the mapping from joint space to task space. Then the relationship between depth and joint variables is derived. Finally, to validate the constrained visual servoing scheme, simulation is conducted in V-REP. The results show that it is feasible and effective to achieve 3D visual servoing of the flexible robotic endoscope with RCM constraints.KeywordsRobotic flexible endoscope3D visual servo controlRemote center of motion

This paper considers image-guided assembly for large composite panels. By using fiducial markers on the panels and robot gripper mounted cameras, we are able to use an industrial robot to align the panels to sub-millimeter accuracy. We considered two commonly used visual servoing schemes: position-based visual servoing (PBVS) and image-based visual servoing (IBVS). It has been noted that IBVS possesses superior robustness with respect to the camera calibration accuracy. However, we have found that in our case, PBVS is both faster and slightly more accurate than IBVS. This result is due to the fact that the visual servoing target in the image plane is derived from a reference target, which depends on the accuracy of the camera model. This additional dependency essentially nullifies the robustness advantage of IBVS. We also implemented a simple scheme to combine inputs from multiple cameras to improve the visual servoing accuracy. Both simulation and experimental results are included to show the effectiveness of visual servoing in an industrial setting.

A pose-estimation-based approach to perform visual control for differential-drive robots is presented in this paper. Our scheme recovers the camera location (position and orientation) using an extended kalman filter (EKF) algorithm with the 1D trifocal tensor (TT) as measurement model. This new visual servoing scheme allows knowing the real world path performed by the robot without the computational load introduced by position-based approaches. A state-estimated feedback control law is designed to solve a tracking problem for the lateral and longitudinal robot coordinates. The desired trajectories to be tracked ensure total correction of both position and orientation using a single control law, even though the orientation is a DOF in the control system. The effectiveness of our approach is tested via simulations.

Vision allows a robotic system to obtain a lot of information on the surrounding environment to be used for motion planning and control. When the control is based on feedback of visual information is called Visual Servoing. Visual Servoing is a powerful tool which allows a robot to increase its interaction capabilities and tasks complexity. In this chapter we describe the architecture of the Robotenis system in order to design two different control strategies to carry out tracking tasks. Robotenis is an experimental stage that is formed of a parallel robot and vision equipment. The system was designed to test joint control and Visual Servoing algorithms and the main objective is to carry out tasks in three dimensions and dynamical environments. As a result the mechanical system is able to interact with objects which move close to 2m=s. The general architecture of control strategies is composed by two intertwined control loops: The internal loop is faster and considers the information from the joins, its sample time is 0:5ms. Second loop represents the visual Servoing system and it is an external loop to the first mentioned. The second loop represents the main study purpose, it is based in the prediction of the object velocity that is obtained from visual information and its sample time is 8:3ms. The robot workspace analysis plays an important role in Visual Servoing tasks, by this analysis is possible to bound the movements that the robot is able to reach. In this article the robot jacobian is obtained by two methods. First method uses velocity vector-loop equations and the second is calculated from the time derivate of the kinematical model of the robot. First jacobian requires calculating angles from the kinematic model. Second jacobian instead, depends on physical parameters of the robot and can be calculated directly. Jacobians are calculated from two different kinematic models, the first one determines the angles each element of the robot. Fist jacobian is used in the graphic simulator of the system due to the information that can be obtained from it. Second jacobian is used to determine off-line the work space of the robot and it is used in the joint and visual controller of the robot (in real time). The work space of the robot is calculated from the condition number of the jacobian (this is a topic that is not studied in article). The dynamic model of the mechanical system is based on Lagrange multipliers, and it uses forearms and end effector platform of non-negligible inertias for the

The visual servoing strategy of a continuum robot has wide-ranging applications and significant engineering value. However, limited by the accuracy of its motion control, research on visual servoing of continuum robots is currently limited to close-range conditions (distance < 1m). Therefore, a novel visual servoing strategy for continuum robots is proposed in this paper that can track external targets in real-time and across distance from centimeters to tens of meters. Firstly, a continuous mapping relationship was proposed by integrating the visual coordinate transformation process with the mechanical model of the continuum robot, establishing a mapping relationship between pixel error and drive values. Secondly, a closed-loop control system for trajectory tracking of moving targets is designed by combining the continuous mapping relationship. Finally, trajectory tracking experiments are conducted on a moving target (i.e., human-shaped target) at different distance scales. Compared with the eye-in-hand strategy at different distances (0.2m, 2m and 3m), the experimental results amply demonstrate the tracking accuracy and stability of the proposed visual servoing strategy. Furthermore, under the experimental condition of long distance (20.5m), the proposed visual servoing strategy can still achieve accurate tracking of the moving target, with an average tracking error of 27.97 pixels (about 2.31cm).

The research on acceleration-level visual servoing of manipulators is crucial yet insufficient, which restricts the potential application range of visual servoing. To address this issue, this paper proposes a quadratic programming-based acceleration-level image-based visual servoing (AIVS) scheme, which considers joint constraints. Besides, aiming to address the unknown problems in visual servoing systems, a data-driven learning algorithm is proposed to facilitate estimating structural information. Building upon this foundation, a data-driven acceleration-level image-based visual servoing (DAIVS) scheme is proposed, integrating learning and control capabilities. Subsequently, a recurrent neural network (RNN) is developed to tackle the DAIVS scheme, followed by theoretical analyses substantiating its stability. Afterwards, simulations and experiments on a Franka Emika Panda manipulator with eye-in-hand structure and comparisons among the existing methods are provided. The obtained results demonstrate the feasibility and practicality of the proposed schemes and highlight the superior learning and control ability of the proposed RNN. This method is particularly well-suited for visual servoing applications of manipulators with unknown structure.

The design and implementation of a real-time object motion sensing and trajectory replicating system, dependent on the image-based visual servoing capabilities of an electrohydraulic eye-in-hand parallel manipulator has been described in this paper. The tracking system consists of a monocular vision apparatus affixed to the top platform of a six-cylinder parallel manipulator constructed with industry grade double-acting single rod hydraulic actuators and budget proportional valves. The control algorithm is an amalgamation of three principles: machine vision based object tracking, image based visual servoing and actuator-level feedback control utilizing the inverse kinematics of the parallel manipulator. The real-time experiments are performed on a laboratory scale parallel manipulator setup with the aid of a virtual tracking object rendered on a computer monitor. The results for a few reference trajectories reveal that accurate tracking of target motions can be achieved with visual servoing algorithms designed for parallel manipulators in real-time.

This paper introduces a visual servoing scheme for humanoid walking. Though most of the existing approaches follow a perception-decision-action scheme, we close the loop so that the control is robust to model error. Our approach is based on a new reactive pattern generator which modifies, at the control level, the footsteps, the center of mass and the center of pressure trajectories for the center of mass to track a reference velocity. And, in this paper, the reference velocity is directly given by a visual servoing control law. Since, the HRP-2 walk induces a sway motion that disturbs the regulation of the visual control law, we introduce a control law allowing convergence in the image space and taking into account this sway motion.

&lt;p&gt;Visual servoing (VS) is a technique which utilizes information acquired from a visual sensor to control the motion of a robot. Despite the advantages of the classic VS techniques in controlling of the robots- specifically in unstructured environments- they suffer from some system’s shortcomings such as the lack of an effective mechanism for handling the existing constraints within the control procedure. To alleviate these issues, among the several approaches proposed in the literature, the receding horizon or model predictive control (MPC) technique is an appropriate candidate to be employed by the system. Any combination of the model predictive control and visual servoing is called visual predictive control (VPC) which is categorized as the classic and hybrid methods. The classic VPC schemes are developed based on the classic VS approaches and the hybrid ones employ the hybrid VS schemes to be integrated by MPC. On the other hand, to guarantee the convergence of the system in presence of the noises and uncertainties, the proposed VPC schemes are embedded in internal model control (IMC) framework.&lt;/p&gt; &lt;p&gt;This work contributes by utilizing the available VPC scheme in control of different robots, such as 3 DOF catheter as a continuum robot and a 6 DOF manipulator from Denso robotics as a robot arm, and by developing two hybrid predictive controllers, conjugated VPC and depth-based VPC, with higher constraint handling capabilities. By utilizing the proposed VPC methods, some of the available limitations could be resolved in the single-robot control level. In addition, to enable the system to perform more sophisticated tasks under difficult conditions such as unexpected occlusions, a multi-agent system controlled by the mentioned VPC methods is employed. Under these conditions, by utilizing the data sharing ability among neighboring agents inside the network, the obstructed one could be provided with enough information to complete the task. At each stage, the performance of the proposed algorithm is verified through numerous simulations and experiments to show their capability in handling existing constraints while accounting for system uncertainties.&lt;/p&gt;

A monocular visual servoing scheme for constrained robots is considered in this chapter. Inspired by the Orthogonalization Principle (OP) introduced by Suguru Arimoto in the context of robot force control, a Visual Orthogonalization Principle (VOP) is proposed and a novel control scheme for adaptive image-based visual servoing is presented. The scheme guarantees a global exponential convergence for the image-based position-velocity and contact forces even when the robot parameters are considered unknown. The stability of the new control scheme is tested under experiments. The experimental results comply to the theoretical considerations.

This paper proposes a Kullback-Leibler (K-L) divergence-based visual servoing scheme. K-L divergence, also known as relative entropy, is a measure of the difference between two probability distributions. By employing the K-L divergence as a new error metric to evaluate the similarity between the actual and desired images, and then formulating the visual servoing process as an optimization problem with the K-L divergence as the optimization objective, the servoing control law can be designed to achieve the positioning task of the robot. The proposed method does not require any feature detection, tracking and matching. In addition, compared with the histogram-based visual servoing method, the proposed method has a faster convergence rate. Numerical simulations carried out on the ViSP platform validate the effectiveness of proposed method.