Robust visual servoing with spheres projected in cameras obeying the unified model

This work presents different proposals to deal with common problems in robot visual servoing based on the application of discontinuous control methods. The feasibility and effectiveness of the proposed approaches are substantiated by simulation results and real experiments using a 6R industrial manipulator. The main contributions are:\n\n- Geometric invariance using sliding mode control (Chapter 3): the defined higher-order invariance is used by the proposed approaches to tackle problems in visual servoing. Proofs of invariance condition are presented.\n\n- Fulfillment of constraints in visual servoing (Chapter 4): the proposal uses sliding mode methods to satisfy mechanical and visual constraints in visual servoing, while a secondary task is considered to properly track the target object. The main advantages of the proposed approach are: low computational cost, robustness and fully utilization of the allowed space for the constraints.\n\n- Robust auto tool change for industrial robots using visual servoing (Chapter 4): visual servoing and the proposed method for constraints fulfillment are applied to an automated solution for tool changing in industrial robots. The robustness of the proposed method is due to the control law of the visual servoing, which uses the information acquired by the vision system to close a feedback control loop. Furthermore, sliding mode control is simultaneously used in a prioritized level to satisfy the aforementioned constraints. Thus, the global control accurately places the tool in the warehouse, but satisfying the robot constraints.\n\n- Sliding mode controller for reference tracking (Chapter 5): an approach based on sliding mode control is proposed for reference tracking in robot visual servoing using industrial robot manipulators. The novelty of the proposal is the introduction of a sliding mode controller that uses a high-order discontinuous control signal, i.e., joint accelerations or joint jerks, in order to obtain a smoother behavior and ensure the robot system stability, which is demonstrated with a theoretical proof.\n\n- PWM and PFM for visual servoing in fully decoupled approaches (Chapter 6): discontinuous control based on pulse width and pulse frequency modulation is proposed for fully decoupled position based visual servoing approaches, in order to get the same convergence time for camera translation and rotation.\n\nMoreover, other results obtained in visual servoing applications are also described.

For soft continuum arms, visual servoing is a popular control strategy that relies on visual feedback to close the control loop. However, robust visual servoing is challenging as it requires reliable feature extraction from the image, accurate control models and sensors to perceive the shape of the arm, both of which can be hard to implement in a soft robot. This letter circumvents these challenges by presenting a deep neural network-based method to perform smooth and robust 3D positioning tasks on a soft arm by visual servoing using a camera mounted at the distal end of the arm. A convolutional neural network is trained to predict the actuations required to achieve the desired pose in a structured environment. Integrated and modular approaches for estimating the actuations from the image are proposed and are experimentally compared. A proportional control law is implemented to reduce the error between the desired and current image as seen by the camera. The model together with the proportional feedback control makes the described approach robust to several variations such as new targets, lighting, loads, and diminution of the soft arm. Furthermore, the model lends itself to be transferred to a new environment with minimal effort.

In this paper, a tuning methodology based on predictive model approach for visual servoing system is investigated. The proposed approach uses features prediction based method to estimate the camera velocity and thus to calculate the optimal control tuning parameter. In order to have a faster convergence and in the same time a desired behaviour of the servoing system, a new control parameter is computed every sampling time. To evaluate the designed tuning strategy, a visual servoing architecture with an eye-in-hand configuration has been considered. The experimental results showed that the proposed tuning algorithm based on prediction model has a stable and convergent behavior when dealing with visual servoing applications. To the knowledge of the authors, the proposed methodology is the first approach which enables an automatic selection of control parameter for the proportional visual control law.

Vision allows a robotic system to obtain a lot of information on the surrounding environment to be used for motion planning and control. When the control is based on feedback of visual information is called Visual Servoing. Visual Servoing is a powerful tool which allows a robot to increase its interaction capabilities and tasks complexity. In this chapter we describe the architecture of the Robotenis system in order to design two different control strategies to carry out tracking tasks. Robotenis is an experimental stage that is formed of a parallel robot and vision equipment. The system was designed to test joint control and Visual Servoing algorithms and the main objective is to carry out tasks in three dimensions and dynamical environments. As a result the mechanical system is able to interact with objects which move close to 2m=s. The general architecture of control strategies is composed by two intertwined control loops: The internal loop is faster and considers the information from the joins, its sample time is 0:5ms. Second loop represents the visual Servoing system and it is an external loop to the first mentioned. The second loop represents the main study purpose, it is based in the prediction of the object velocity that is obtained from visual information and its sample time is 8:3ms. The robot workspace analysis plays an important role in Visual Servoing tasks, by this analysis is possible to bound the movements that the robot is able to reach. In this article the robot jacobian is obtained by two methods. First method uses velocity vector-loop equations and the second is calculated from the time derivate of the kinematical model of the robot. First jacobian requires calculating angles from the kinematic model. Second jacobian instead, depends on physical parameters of the robot and can be calculated directly. Jacobians are calculated from two different kinematic models, the first one determines the angles each element of the robot. Fist jacobian is used in the graphic simulator of the system due to the information that can be obtained from it. Second jacobian is used to determine off-line the work space of the robot and it is used in the joint and visual controller of the robot (in real time). The work space of the robot is calculated from the condition number of the jacobian (this is a topic that is not studied in article). The dynamic model of the mechanical system is based on Lagrange multipliers, and it uses forearms and end effector platform of non-negligible inertias for the

In this paper, we present a robot positioning task with respect to a static target using visual servoing and optimization techniques. The vision system is uncalibrated and the kinematic model of the robot may be totally unknown. The displacements of the robot are generated in real time in order to minimize an objective function. The objective function includes the quadratic error between the current target image and the desired target image. A simplex method is used to minimize the objective function. Our method allows the system to include constraints in the image as well as in joint-space. We successfully validate this method, with simulations under the graphic library QpenGL and with experiments on a 6-DOF industrial manipulator.

Hand gestures are a natural way of communication in our everyday life. Gestures provide a user friendly alternative of various types of devices available in our surroundings such as keyboard, mouse, and joysticks. This paper presents a hand gesture recognition system based on new set of features along with the classifier combining techniques to enhance the performance of the existing system. The study reveals that the unwanted movement within a gesture (self co-articulation) may be used as one of the complementary feature to improve the performance of the system. The self co-articulation was detected from the gesture trajectory by exploiting velocity information along with the pause information. The proposed new set of features (such as position of the hand, ratio feature and first half trajectory features) along with the self co-articulated features were used to define the feature space. Individual models as well as hybrid models were developed using the classifiers such as artificial neural network (ANN), multiclass support vector machine (SVM), k-nearest neighbours (k-NN) and Naive Bayes model. The models based on conditional random field (CRF) and hidden conditional random field (HCRF) were used to develop the baseline system for the present study. The experimental results suggest that the proposed new set of features combined with the hybrid classifier provides an improvement in the accuracy of the system by 14.58 and 8.41 % as compared to CRF and HCRF based models respectively. An analysis of variance test has also been conducted to validate the statistical significance of results.

In this paper, we develop a robust visual servo system for object tracking applications of a nonholonomic mobile robot. The system mainly consists of an adaptive shape tracking algorithm and a robust visual servo controller. The adaptive shape tracking algorithm is designed to automatically detect the shape contours of moving objects, extract the shape parameters, and continuously track the object in shape parameter space. Based on direct measurements of the shape parameters, the visual servo controller is designed by using the sliding mode control technique and is robust to uncertainties of the object's motion. Through a Lyapunov-based stability analysis, a sufficient condition on selection of control gains to achieve the tracking goal in finite time is provided.

Recently, a unified camera model has been introduced in visual control systems in order to describe through a unique mathematical model conventional perspective cameras, fisheye cameras, and catadioptric systems. In this paper, a path-planning strategy for visual servoing is proposed for any camera obeying this unified model. The proposed strategy is based on the projection onto a virtual plane of the available image projections. This has two benefits. First, it allows one to perform camera pose estimation and 3D object reconstruction by using methods for conventional camera that are not valid for other cameras. Second, it allows one to perform image path-planning for multi-constraint satisfaction by using a simplified but equivalent projection model, that in this paper is addressed by introducing polynomial parametrizations of the rotation and translation. The planned image trajectory is hence tracked by using an IBVS controller. The proposed strategy is validated through simulations with image noise and calibration errors typical of real experiments. It is worth remarking that visual servoing path-planning for non conventional perspective cameras has not been proposed yet in the literature.

Intra-operative imaging is widely used to provide visual feedback to a clinician when he/she performs a procedure. In visual servoing, surgical instruments and parts of tissue/body are tracked by processing the acquired images. This information is then used within a control loop to manoeuvre a robotic manipulator during a procedure. A comprehensive search of electronic databases was completed for the period 2000-2013 to provide a survey of the visual servoing applications in medical robotics. The focus is on medical applications where image-based tracking is used for closed-loop control of a robotic system. Detailed classification and comparative study of various contributions in visual servoing using endoscopic or direct visual images are presented and summarized in tables and diagrams. The main challenges in using visual servoing for medical robotic applications are identified and potential future directions are suggested. 'Supervised automation of medical robotics' is found to be a major trend in this field.

The research on acceleration-level visual servoing of manipulators is crucial yet insufficient, which restricts the potential application range of visual servoing. To address this issue, this paper proposes a quadratic programming-based acceleration-level image-based visual servoing (AIVS) scheme, which considers joint constraints. Besides, aiming to address the unknown problems in visual servoing systems, a data-driven learning algorithm is proposed to facilitate estimating structural information. Building upon this foundation, a data-driven acceleration-level image-based visual servoing (DAIVS) scheme is proposed, integrating learning and control capabilities. Subsequently, a recurrent neural network (RNN) is developed to tackle the DAIVS scheme, followed by theoretical analyses substantiating its stability. Afterwards, simulations and experiments on a Franka Emika Panda manipulator with eye-in-hand structure and comparisons among the existing methods are provided. The obtained results demonstrate the feasibility and practicality of the proposed schemes and highlight the superior learning and control ability of the proposed RNN. This method is particularly well-suited for visual servoing applications of manipulators with unknown structure.

This paper presents an application of feature-based visual servoing to achieve accurate and robust 3D filament alignment. Two orthogonal cameras are used to localize the five degrees of freedom of an axi-symmetric filament. An algorithm for precisely estimating the center and orientation features of a filament in a camera view is described. The use of feature-based servoing overcomes the difficulty of 3D camera calibration in a factory environment. These features drive a PID control loop for each view. The two views are coupled in one degree of freedom, and the multiple-view controller switches between two single-view controllers to achieve 3D servoing. The overall accuracy achieved is in thousandths of an inch. Experimental results on the performance of the control algorithm are discussed. Models explaining the system behavior are presented. The use of visual servoing in this application far exceeds the positioning accuracy and the repeatability of human operators.

IBVS based on adaptive sliding mode control for a quadrotor target tracking under perturbations

Visual servoing allows to control the motion of a robot using information from its visual sensors to achieve manipulation tasks. In this work we design and implement a robust visual servoing framework for reaching and grasping behaviours for a humanoid service robot with limited control capabilities. Our approach successfully exploits a 5-degrees of freedom manipulator, overcoming the control limitations of the robot while avoiding singularities and stereo vision techniques. Using a single camera, we combine a marker-less model based tracker for the target object, a pattern tracking for the end-effector to deal with the robot’s inaccurate kinematics, and alternate pose based visual servo technique with eye-in-hand and eye-to-hand configurations to achieve a fully functional grasping system. The overall method shows better results for grasping than conventional motion planing and simple inverse kinematics techniques for this robotic morphology, demonstrating a 48.8% of increment in the grasping success rate.

Phoneme recognition using wavelet based features

A new set of features for a bimodal system based on on-line signature and speech