A data-driven acceleration-level scheme for image-based visual servoing of manipulators with unknown structure.

This paper considers image-guided assembly for large composite panels. By using fiducial markers on the panels and robot gripper mounted cameras, we are able to use an industrial robot to align the panels to sub-millimeter accuracy. We considered two commonly used visual servoing schemes: position-based visual servoing (PBVS) and image-based visual servoing (IBVS). It has been noted that IBVS possesses superior robustness with respect to the camera calibration accuracy. However, we have found that in our case, PBVS is both faster and slightly more accurate than IBVS. This result is due to the fact that the visual servoing target in the image plane is derived from a reference target, which depends on the accuracy of the camera model. This additional dependency essentially nullifies the robustness advantage of IBVS. We also implemented a simple scheme to combine inputs from multiple cameras to improve the visual servoing accuracy. Both simulation and experimental results are included to show the effectiveness of visual servoing in an industrial setting.

In this paper, a practical discrete-time control method with adaptive image feature prediction for the image-based visual servoing (IBVS) scheme is presented. In the discrete-time IBVS inner-loop/outer-loop control architecture, the time delay caused by image capture and computation is noticed. Considering the dynamic characteristics of a 6-DOF manipulator velocity input system, we propose a linear dynamic model to describe the motion of a robot end effector. Furthermore, for better estimation of image features and smoothing of the robot's velocity input, we propose an adaptive image feature prediction method that employs past image feature data and real robot velocity data to adopt the prediction parameters. The experimental results on a 6-DOF robotic arm demonstrate that the proposed method can ensure system stability and accelerate system convergence.

Main issues in visual servoing of manipulators mainly include rapid convergence of feature errors to zero and the safety of joints regarding joint physical limits. To address the two issues, in this paper, an image-based visual servoing scheme is proposed for manipulators with an eye-in-hand configuration. Compared with existing schemes, the proposed one does not require performing pseudoinversion for the image Jacobian matrix or inversion for the Jacobian matrix associated with the forward kinematics of the manipulators. Theoretical analysis shows that the proposed scheme not only guarantees the asymptotic convergence of feature errors to zero but also the compliance with joint angle and velocity limits of the manipulators. Besides, simulation results based on a PUMA560 manipulator with a camera mounted on the end effector verify the theoretical conclusions and the efficacy of the proposed scheme.

Since quadrotors are becoming more popular, several techniques appeared to control robustly the Unmanned Aerial Vehicle. This article proposes an intelligent image-based visual servoing (IBVS) control method for an unmanned aerial vehicle (UAV). This work consists of computing the proportional controller gain by smoothing the error between the desired and actual feature vectors values, as opposed to the classical visual servoing (VS) scheme. The weak handling of this gain, affect the maintaining of tracked features in the area of view. Unlike the classical VS scheme that requires a known value of gain, selected in a heuristic way, which lead the controller to fall in the velocity convergence problem. We adjust an adaptive gain value by the use of a fuzzy logic controller. The proposed controller exploits the error between feature vectors and its derivative as inputs. Furthermore, we design an intelligent based-fuzzy logic method for gain tuning of PD controllers applied for handling the heading and position of the quadrotor. The proposed controllers are applied to the inner and outer loop, offer several advantages in dealing the uncertainty occurred within trajectory tracking control. Various simulations are developed in MATLAB, shows the outperformance of the proposed method against the classical one.

This letter for the first time extends the virtual camera image-based visual servoing (IBVS) scheme to enable an underactuated rotorcraft UAV to regulate its translational motion and heading relative to a planar visual target of arbitrary orientation. The conversion from real camera images to virtual camera images of visual targets are proposed based on a set of rotation matrices. Hence, image moment features can be reused due to the simplicity and decoupled structure of the interaction matrix, and satisfactory 3D Cartesian trajectory of UAVs. In the design of the IBVS control law, the external disturbance and model uncertainties are estimated by an integral-based filter. In addition, to enable tracking of a moving visual target, a velocity estimator is developed. The global asymptotic stability of the error dynamics is proven. Both of the simulation and experimental results of tracking of a tilted moving planar target are provided to show the efficacy of the proposed IBVS scheme.

A 2-1/2-D visual servoing strategy, which is based on a novel motion-estimation technique, is presented for the stabilization of a nonholonomic mobile robot (which is also called the “parking problem”). By taking into account the planar motion constraint of mobile robots, the proposed motion-estimation technique can be applied in both planar and nonplanar scenes. In addition, this approach requires no matrix estimation or decomposition, and it avoids ambiguity and degeneracy problems for the homography or fundamental matrix-based algorithms. Moreover, the field-of-view (FOV) constraint of the onboard camera is largely alleviated because the presented algorithm works well with few feature points. In order to incorporate the advantages of position-based visual servoing and image-based visual servoing, a composite error vector is defined that includes both image signals and the estimated rotational angle. Subsequently, a smooth time-varying feedback controller is adopted to cope with the nonholonomic constraints, which yields global exponential convergent rate for the closed-loop system. On the basis of the perturbed linear system theory, we show that practical exponential stability can be achieved, despite the lack of depth information, which is inherent for monocular camera systems. Both simulation and experiment results are collected to investigate the feasibility of the proposed approach.

U ovom preglednom radu su prikazane metode vizuelnog upravljanja robotskih sistema, sa primarnim fokusom na mobilne robote sa diferencijalnim pogonom. Analizirane su standardne metode vizuelnog upravljanja bazirane na (i) greškama u parametrima slike (engl. Image-Based Visual Servoing - IBVS) i (ii) izdvojenim karakteristikama sa slike neophodnim za estimaciju položaja izabranog objekta (engl. Position-Based Visual Servoing - PBVS) i poređene sa novom metodom direktnog vizuelnog upravljanja (engl. Direct Visual Servoing - DVS). U poređenju sa IBVS i PBVS metodama, DVS metod se odlikuje višom tačnošću, ali i manjim domenom konvergencije. Zbog ovog razloga je DVS metod upravljanja pogodan za integraciju u hibridne sisteme vizuelnog upravljanja. Takođe, predstavljeni su radovi koji unapređuju sistem vizuelnog upravljanja korišćenjem stereo sistema (sistem sa dve kamere). Stereo sistem, u poređenju sa alternativnim metodama, omogućava tačniju ocenu dubine karakterističnih objekata sa slike, koja je neophodna za zadatke vizuelnog upravljanja. Predmet analize su i radovi koji integrišu tehnike veštačke inteligencije u sistem vizuelnog upravljanja. Ovim tehnikama sistemi vizuelnog upravljanja dobijaju mogućnost da uče, čime se njihov domen primene znatno proširuje. Na kraju, napominje se i mogućnost integracije vizuelne odometrije u sisteme vizuelnog upravljanja, što prouzrokuje povećanje robusnosti čitavog robotskog sistema.

A review on visual servoing for underwater vehicle manipulation systems automatic control and case study

This thesis deals with Visual Servoing and its strictly connected disciplines like projective geometry, image processing, robotics and non-linear control. More specifically the work addresses the problem to control a robotic manipulator through one of the largely used Visual Servoing techniques: the Image Based Visual Servoing (IBVS). In Image Based Visual Servoing the robot is driven by on-line performing a feedback control loop that is closed directly in the 2D space of the camera sensor. The work considers the case of a monocular system with the only camera mounted on the robot end effector (eye in hand configuration). Through IBVS the system can be positioned with respect to a 3D fixed target by minimizing the differences between its initial view and its goal view, corresponding respectively to the initial and the goal system configurations: the robot Cartesian Motion is thus generated only by means of visual informations. However, the execution of a positioning control task by IBVS is not straightforward because singularity problems may occur and local minima may be reached where the reached image is very close to the target one but the 3D positioning task is far from being fulfilled: this happens in particular for large camera displacements, when the the initial and the goal target views are noticeably different. To overcame singularity and local minima drawbacks, maintaining the good properties of IBVS robustness with respect to modeling and camera calibration errors, an opportune image path planning can be exploited. This work deals with the problem of generating opportune image plane trajectories for tracked points of the servoing control scheme (a trajectory is made of a path plus a time law). The generated image plane paths must be feasible i.e. they must be compliant with rigid body motion of the camera with respect to the object so as to avoid image jacobian singularities and local minima problems. In addition, the image planned trajectories must generate camera velocity screws which are smooth and within the allowed bounds of the robot. We will show that a scaled 3D motion planning algorithm can be devised in order to generate feasible image plane trajectories. Since the paths in the image are off-line generated it is also possible to tune the planning parameters so as to maintain the target inside the camera field of view even if, in some unfortunate cases, the feature target points would leave the camera images due to 3D robot motions. To test the validity of the proposed approach some both experiments and simulations results have been reported taking also into account the influence of noise in the path planning strategy. The experiments have been realized with a 6DOF anthropomorphic manipulator with a fire-wire camera installed on its end effector: the results demonstrate the good performances and the feasibility of the proposed approach.

This paper presents a visual servoing system for the task of tracking and grasping a moving target in 3D space. A multiple-cue feature visual tracking module combining known 3D CAD models of the object is used to extract the target pose in the RGB-D images. An Extended Kalman Filter (EKF) is developed to further compensate for the delay of the system and estimate the object motion. A two-stage controller for the visual servo is proposed with the first part being able to control the gripper to track the target, while the second part is a pose based visual servo (PBVS) scheme for the grasping motion. Experiment results show that the presented system is able to perform real time tracking- grasping tasks to a moving target.

This paper addresses the challenges of choosing proper image features for planar symmetric shape objects and designing visual servoing controller to enhance the tracking performance in image-based visual servoing (IBVS). Six image moments are chosen as the image features and the analytical image interaction matrix related to the image features are derived. A controller is designed to efficiently increase the robustness of the visual servoing system. Experimental results on a 6-DOF robot visual servoing system are provided to illustrate the effectiveness of the proposed method.

This letter deals with the development of a vision-based controller for a continuum robot architecture. More precisely, the controlled robotic structure is based on three-tube concentric tube robot (CTR), an emerging paradigm to design accurate, miniaturized, and flexible endoscopic robots. This approach has grown considerably in the recent years finding applications in numerous surgical disciplines. In contrast to conventional robotic structures, CTR kinematics arise many challenges for an optimal control, such as friction, torsion, shear, and nonlinear constitutive behavior. In fact, in order to ensure efficient and reliable control, in addition to computing an analytical and complete kinematic model, it is also important to close the control loop. To do this, we developed an eye-in-hand visual servoing scheme using a millimeter-sized camera embedded at the robot's tip. Both the kinematic model and the visual servoing controller were successfully validated in simulation with visual servoing platform and using an experimental setup. The obtained results showed satisfactory performances for three-degrees of freedom positioning and path following tasks with adaptive gain control.

Prosthesis is a device that replaces a missing body part. These devices are used to reinstate the lost functionality due to an amputation. Above elbow amputees are equipped with trans-humeral prostheses. The existing prostheses use biological signals such as electromyography (EMG) in combine with external sensors for the controlling purposes. Vision is the latest addition to external sensor domain to eliminate the drawbacks of existing hybrid prostheses. This paper proposes a vision based task planning method in combination with available EMG signals to plan reach-to-grasp tasks of a trans-humeral prosthesis. The visual servoing system consists of an Image Based Visual Servoing (IBVS) system in conjunction with an Artificial Neural Network (ANN) to center the object in the image frame. Moreover, a gradient descent algorithm is proposed for the elbow motion to reach the palm towards the object of interest. EMG based system is capable of controlling the prosthetic elbow motion with available muscles' signals. EMG based system is fused with the IBVS system to perform elbow motions when an object is visible to the camera. Experiments are carried out using a virtual prosthesis to validate the proposed task planning method.

The visual servoing strategy of a continuum robot has wide-ranging applications and significant engineering value. However, limited by the accuracy of its motion control, research on visual servoing of continuum robots is currently limited to close-range conditions (distance < 1m). Therefore, a novel visual servoing strategy for continuum robots is proposed in this paper that can track external targets in real-time and across distance from centimeters to tens of meters. Firstly, a continuous mapping relationship was proposed by integrating the visual coordinate transformation process with the mechanical model of the continuum robot, establishing a mapping relationship between pixel error and drive values. Secondly, a closed-loop control system for trajectory tracking of moving targets is designed by combining the continuous mapping relationship. Finally, trajectory tracking experiments are conducted on a moving target (i.e., human-shaped target) at different distance scales. Compared with the eye-in-hand strategy at different distances (0.2m, 2m and 3m), the experimental results amply demonstrate the tracking accuracy and stability of the proposed visual servoing strategy. Furthermore, under the experimental condition of long distance (20.5m), the proposed visual servoing strategy can still achieve accurate tracking of the moving target, with an average tracking error of 27.97 pixels (about 2.31cm).

This paper presents a self-memory prediction model to mitigate the effects of image based visual servoing (IBVS) system under uncertainty. The performance of IBVS system is easily influenced by different tasks, diverse environments and uncertain disturbances. Through building a self-memory prediction model to keep previous movement tendency in the every current movement, the framework of a self-memory is proposed for incremental self-learning capability. The main challenge in IBVS system is avoid abrupt change in the operation of the system. The proposed approach is processed from state pattern, and introduces the memory function which contain multiple historical observed values. According to past events, predict the current state and avoid abrupt changes in the every current movement. Simulations on IBVS system illustrate the effectiveness of the proposed method.