New Visual Servoing Control Strategies in Tracking Tasks Using a PKM

U ovom preglednom radu su prikazane metode vizuelnog upravljanja robotskih sistema, sa primarnim fokusom na mobilne robote sa diferencijalnim pogonom. Analizirane su standardne metode vizuelnog upravljanja bazirane na (i) greškama u parametrima slike (engl. Image-Based Visual Servoing - IBVS) i (ii) izdvojenim karakteristikama sa slike neophodnim za estimaciju položaja izabranog objekta (engl. Position-Based Visual Servoing - PBVS) i poređene sa novom metodom direktnog vizuelnog upravljanja (engl. Direct Visual Servoing - DVS). U poređenju sa IBVS i PBVS metodama, DVS metod se odlikuje višom tačnošću, ali i manjim domenom konvergencije. Zbog ovog razloga je DVS metod upravljanja pogodan za integraciju u hibridne sisteme vizuelnog upravljanja. Takođe, predstavljeni su radovi koji unapređuju sistem vizuelnog upravljanja korišćenjem stereo sistema (sistem sa dve kamere). Stereo sistem, u poređenju sa alternativnim metodama, omogućava tačniju ocenu dubine karakterističnih objekata sa slike, koja je neophodna za zadatke vizuelnog upravljanja. Predmet analize su i radovi koji integrišu tehnike veštačke inteligencije u sistem vizuelnog upravljanja. Ovim tehnikama sistemi vizuelnog upravljanja dobijaju mogućnost da uče, čime se njihov domen primene znatno proširuje. Na kraju, napominje se i mogućnost integracije vizuelne odometrije u sisteme vizuelnog upravljanja, što prouzrokuje povećanje robusnosti čitavog robotskog sistema.

Asymptotic Stability of Uncalibrated Eye-in-Hand Visual Servoing: An Affine Invariance Perspective

Robotics has been a paradigm for science in the last few decades. At first, scientists’ efforts were devoted to the solution of the problem of planning and control of themotion of robot manipulators. However, the motion control of robot manipulators in unstructured environments is today an attractive scientific problem. An interesting solution for motion control is the use of sensor information, such as computer vision, in the system’s feedback. Several works and tools have been developed in recent years in this field (Corke; 2005), (Chaumette and Hutchinson; 2006). The more typical approaches consider visual perception for servoing and for the so called look and move (Hutchinson et al.; 1996). Visual servoing (Kelly et al.; 2000) can be classified into two approaches: camera-in-hand or camera-to-hand (Flandin et al.; 2000). In camera-to-hand robotic systems, multiple cameras or a single camera fixed in the worldcoordinate frame capture images of both, the robot and its environment. The tracking of the object with visual feedback can be made in 2D or 3D. An interesting solution for the visual servoing of camera-in hand robot manipulators in 2D can be found in e. g. (Bonfe et al.; 2002) and (Hernandez et al.; 2008) where stability demonstration of a decoupled controller have been presented. For 3D tracking some solutions reported are look and move controller, with one camera (Sim et al.; 2002) or more than one camera (Xie et al.; 2005). A 3D visual servoing with stability analysis in continuous time is presented by (Hernandez et al.; 2008a) and (Kelly et al.; 2006) present a direct visual servoing with transpose Jacobian control technique for regulation of robot manipulators in the 3D Cartesian space. In a similar vein (Enescu et al.; 2006) present mobile robot navigation for person tracking using a stereo head-camera. In this chapter, we consider the control problem of camera-in-hand robot manipulators in 2D and 3D. In both cases only one camera is mounted on the robot’s arm, which supplies visual information of the environment, with the aim of moving the manipulator by maintaining the image of the tracked object (a sphere) in the centre of the image plane, despite the possible movements of the object. In the 3D control the constant radius is used as a feature too. In this work, the proposed control system considers two loops in cascade, an internal loop solving the robots’ joint control, and an external loop implementing a dynamic look and move visual controller. A stability analysis in discrete time is developed under the conditions that it is possible to approximate the dynamic effect of the internal loop as an external loop time delay (Corke; 1996) and (Bonfe et al.; 2002). The more classic presentations of servovisual control are velocity controllers, based in the term feature Jacobian,(Chaumette and Hutchinson; 2006) 2

For a visual servo system, there usually exists the problem of time-delay likely caused by long image processing and data transmission. The visual servo system of our robot is subject to two main limitations stemming from the specific commercial mobile manipulator, of which one is the large time-delay due to image transmission, whereas the other is failure to directly command each joint velocity as the visual servo's control input. In this paper, we propose a novel visual servo system to compensate the large time-delay, and we use integral of velocity which incorporating the image estimation and prediction with the kinematic model so as to achieve indirect velocity control with position based control mode joint. Our proposed visual servo system controls the neck joints of robot and keeps the object within the field of view of the camera during the approaching phase. We evaluate the proposed approach by experiment on a real wheeled humanoid robot and find the resulting performance quite promising, showing that our method is feasible and practical.

Vision is in fact the richest source of information for ourself and also for outdoors Robotics, and can be considered the most complex and challenging problem in signal processing for pattern recognition. The first results using Vision in the control loop have been obtained in indoors and structured environments, in which a line or known patterns are detected and followed by a robot (Feddema & Mitchell (1989), Masutani et al. (1994)). Successful works have demonstrated that visual information can be used in tasks such as servoing and guiding, in robot manipulators and mobile robots (Conticelli et al. (1999), Mariottini et al. (2007), Kragic & Christensen (2002).) Visual Servoing is an open issue with a long way for researching and for obtaining increasingly better and more relevant results in Robotics. It combines image processing and control techniques, in such a way that the visual information is used within the control loop. The bottleneck of Visual Servoing can be considered the fact of obtaining robust and on-line visual interpretation of the environment, which can be usefully treated by control structures and algorithms. The solutions provided in Visual Servoing are typically divided into Image Based Control Techniques and Pose Based Control Techniques, depending on the kind of information provided by the vision system that determine the kind of references that have to be sent to the control structure (Hutchinson et al. (1996), Chaumette & Hutchinson (2006) and Siciliano & Khatib (2008)). Another classical division of the Visual Servoing algorithms considers the physical disposition of the visual system, yielding to eye-in-hand systems and eye-to-hand systems, that in the case of Unmanned Aerial Vehicles (UAV) can be translated as on-board visual systems (Mejias (2006)) and ground visual systems (Martinez et al. (2009)). The challenge of Visual Servoing is to be useful in outdoors and non-structured environments. For this purpose the image processing algorithms have to provide visual information that has to be robust and works in real time. UAV can therefore be considered as a challenging testbed for visual servoing, that combines the difficulties of abrupt changes in the image sequence (i.e. vibrations), outdoors operation (non-structured environments) and 3D information changes (Mejias et al. (2006)). In this chapter we give special relevance to the fact of obtaining robust visual information for the visual servoing task. In section (2).we overview the main algorithms used for visual tracking and we discuss their robustness when they are applied to image sequences taken from the UAV. In sections (3). and (4). we analyze how vision systems can perform 3D pose estimation that can be used for

This letter deals with the development of a vision-based controller for a continuum robot architecture. More precisely, the controlled robotic structure is based on three-tube concentric tube robot (CTR), an emerging paradigm to design accurate, miniaturized, and flexible endoscopic robots. This approach has grown considerably in the recent years finding applications in numerous surgical disciplines. In contrast to conventional robotic structures, CTR kinematics arise many challenges for an optimal control, such as friction, torsion, shear, and nonlinear constitutive behavior. In fact, in order to ensure efficient and reliable control, in addition to computing an analytical and complete kinematic model, it is also important to close the control loop. To do this, we developed an eye-in-hand visual servoing scheme using a millimeter-sized camera embedded at the robot's tip. Both the kinematic model and the visual servoing controller were successfully validated in simulation with visual servoing platform and using an experimental setup. The obtained results showed satisfactory performances for three-degrees of freedom positioning and path following tasks with adaptive gain control.

This paper proposes a new controller for dynamic visual servoing of robotic manipulators. In this work the image is acquired from a CCD camera mounted on planar robotic manipulator end-effector, and processed in a standard PC based vision system in order to close the visual servoing loop with PD controllers. First, we propose an extension to dynamic visual servoing based on a pose estimation algorithm and then compare it with another Lyapunov stable controller. The proposed dynamic controller uses only information from the image and robot encoders. This approach is inspired on classical 3D visual servoing and the Jacobian transpose controller approach. An experimental setup developed for visual servoing experiments, including software and hardware is presented. Experimental results for a planar robotic manipulator are presented.

Based on the real-time image capturing and recognition, the robot is responsible to inspect the working status of various equipments in the smart substation. In this paper, an adapted efficient robotics visual servo algorithm is proposed in order to improve the accuracy of capturing the target images. When the robot captures image of the equipment, SIFT method is utilized to match with the template image to verify the captured image containing the region of interesting (ROI) or not. Once the ROI is part or entirety out of the real-time image, the PTZ is controlled to make up for the deficiency by calculating the offset pixels. Then the robot will capture more accurate target image. Farther more, a “high zoom ratio” visual servo solution is also figured out by this proposed algorithm. The experiment results proved that this algorithm improved the efficiency of image capture to the robot. And it is effective to save the computer resource to achieve long battery life when the robot is working the inspection missions. The application of this proposed algorithm will provide facilities for robot to realize 24 hour all-weather inspection in the substation.

We propose a novel geometry-constrained learning-based method for camera-in-hand visual servoing systems that eliminates the need for camera intrinsic parameters, depth information, and the robot's kinematic model. Our method uses a cerebellar model articulation controller (CMAC) to execute online Jacobian estimation within the control framework. Specifically, we introduce a fixed-dimension, uniform-magnitude error function based on the projective homography matrix. The fixed-dimension error function ensures a constant Jacobian size regardless of the number of feature points, thereby reducing computational complexity. By not relying on individual feature points, the approach maintains robustness even when some features are occluded. The uniform magnitude of the error vector elements simplifies neural network input normalization, thereby enhancing online training efficiency. Furthermore, we incorporate geometric constraints between feature points (such as collinearity preservation) into the network update process, ensuring that model predictions conform to the fundamental principles of projective geometry and eliminating physically impossible control outputs. Experimental and simulation results demonstrate that our approach achieves superior robustness and faster learning rates compared to other model-free image-based visual servoing methods.

Visual servoing (VS) is an automatic control technique which uses the feedback of visual information to guide the robot motion to converge to the desired location. This paper proposes new moment-based features for robust VS with spheres, especially in cameras obeying the unified model. Specifically, we introduce a virtual plane to reload contours of image projections of spheres, in the process of which image noises will however be greatly magnified and induce large converge error. To solve this, new moment-based features are proposed and extracted on this virtual plane to enhance the robustness of VS to calibration errors and image noises. Simulation results verify the effectiveness of the proposed scheme and demonstrate the high robustness of these set of new features to image noises. The altering trends of converge errors versus increasing noise levels are randomly tested to present the obvious advantage of new moment-based features over the usual ones. At last, a path-planning based VS application is performed with fisheye views of some spheres, benefiting from the robustness and an interpolation-based tracking scheme.

One key technology for a humanoid robot is to sense the environment accurately and control the movement of the robot in realtime. This paper addresses this problem and focuses on the visual servoing control of an anthropomorphic robotic arm. A high precision, image-based visual servo algorithm for an eye-in-hand visual system is proposed to control the movement of the anthropomorphic arm. The control objective of the controller is to lock all the feature points at fixed points on the image plane. Then, impedance control law is implemented to keep the flexible of the arm. Finally, the effectiveness of the proposed methods are verified by experiments on our humanoid robotic arm platform.

In this paper, a tight coupling between computer vision and parallel robotics is exhibited through the projective line geometry. Indeed, contrary to the usual methodology where the robot is modeled independently from the control law that will be implemented, the proposed method takes into account, from the early modeling stage, the fact that vision will be used for control. Hence, kinematic modeling and projective geometry are fused into a control-devoted projective kinematic model. Thus, starting from a vision-based kinematic modeling of a Gough—Stewart manipulator, a visual servoing scheme is presented, where the image projection (edges) of the non-rigidly linked legs are servoed, rather than the end-effector pose or the leg directions.

Robot path planning algorithms and vision based solutions have gained importance among the algorithms developed to realize path planning in real time. In this paper, a new vision control model was developed for differential drive robot, using Fuzzy Logic decision sets that are independent of wheel encoder sensors. Real-time robot tracking and navigation was done using colored labels. A kinematic model with a graph based virtual sensor is utilized to develop a different solution to robot control problem.

The design and implementation of a real-time object motion sensing and trajectory replicating system, dependent on the image-based visual servoing capabilities of an electrohydraulic eye-in-hand parallel manipulator has been described in this paper. The tracking system consists of a monocular vision apparatus affixed to the top platform of a six-cylinder parallel manipulator constructed with industry grade double-acting single rod hydraulic actuators and budget proportional valves. The control algorithm is an amalgamation of three principles: machine vision based object tracking, image based visual servoing and actuator-level feedback control utilizing the inverse kinematics of the parallel manipulator. The real-time experiments are performed on a laboratory scale parallel manipulator setup with the aid of a virtual tracking object rendered on a computer monitor. The results for a few reference trajectories reveal that accurate tracking of target motions can be achieved with visual servoing algorithms designed for parallel manipulators in real-time.

This paper presents a visual servoing system for the task of tracking and grasping a moving target in 3D space. A multiple-cue feature visual tracking module combining known 3D CAD models of the object is used to extract the target pose in the RGB-D images. An Extended Kalman Filter (EKF) is developed to further compensate for the delay of the system and estimate the object motion. A two-stage controller for the visual servo is proposed with the first part being able to control the gripper to track the target, while the second part is a pose based visual servo (PBVS) scheme for the grasping motion. Experiment results show that the presented system is able to perform real time tracking- grasping tasks to a moving target.