Visual servoing with redundant features

This paper proposes a control theoretic formulation and a controller design method for the feature-based visual servoing with redundant features. The linear time-invariant (LTI) formulation copes with the redundant features and provides a simple framework for controller design. The proposed linear quadratic (LQ) method can deal with the redundant features, which is important because the previous LQ methods are not applicable to redundant systems. Moreover, this LQ method gives flexibility for performance improvement instead of the very limited design parameters provided by the generalized inverse and task function controllers. Validity of the LTI model and effectiveness and flexibility of the LQ optimal controller are evaluated by real-time experiments on a PUMA 560 manipulator.

This work presents different proposals to deal with common problems in robot visual servoing based on the application of discontinuous control methods. The feasibility and effectiveness of the proposed approaches are substantiated by simulation results and real experiments using a 6R industrial manipulator. The main contributions are:\n\n- Geometric invariance using sliding mode control (Chapter 3): the defined higher-order invariance is used by the proposed approaches to tackle problems in visual servoing. Proofs of invariance condition are presented.\n\n- Fulfillment of constraints in visual servoing (Chapter 4): the proposal uses sliding mode methods to satisfy mechanical and visual constraints in visual servoing, while a secondary task is considered to properly track the target object. The main advantages of the proposed approach are: low computational cost, robustness and fully utilization of the allowed space for the constraints.\n\n- Robust auto tool change for industrial robots using visual servoing (Chapter 4): visual servoing and the proposed method for constraints fulfillment are applied to an automated solution for tool changing in industrial robots. The robustness of the proposed method is due to the control law of the visual servoing, which uses the information acquired by the vision system to close a feedback control loop. Furthermore, sliding mode control is simultaneously used in a prioritized level to satisfy the aforementioned constraints. Thus, the global control accurately places the tool in the warehouse, but satisfying the robot constraints.\n\n- Sliding mode controller for reference tracking (Chapter 5): an approach based on sliding mode control is proposed for reference tracking in robot visual servoing using industrial robot manipulators. The novelty of the proposal is the introduction of a sliding mode controller that uses a high-order discontinuous control signal, i.e., joint accelerations or joint jerks, in order to obtain a smoother behavior and ensure the robot system stability, which is demonstrated with a theoretical proof.\n\n- PWM and PFM for visual servoing in fully decoupled approaches (Chapter 6): discontinuous control based on pulse width and pulse frequency modulation is proposed for fully decoupled position based visual servoing approaches, in order to get the same convergence time for camera translation and rotation.\n\nMoreover, other results obtained in visual servoing applications are also described.

In this paper, a visual servoing robotic control scheme of fast convergence and calibration-free is proposed. The objective of this paper is to achieve a better convergence around the equilibrium for uncalibrated visual servoing systems. Moreover, there are threefold unknown parameters, the visual kinematic parameters, the dynamic parameters and the parameters of the feature points, considered in the design of the proposed control scheme. In order to achieve the above control objective, the finite-time tracking controller and three adaptive laws are proposed. The adaptability to the unknown parameters is guaranteed by the online adaptive laws. The finite-time convergence is achieved by the continuously non-smooth fractional order function in the controller. The rigorously mathematic proof of stability is given by homogeneous theory and the Lyapunov function formalism. Three real-time experiments are conducted to demonstrate the practical performance of the proposed scheme.

The design and implementation of a real-time object motion sensing and trajectory replicating system, dependent on the image-based visual servoing capabilities of an electrohydraulic eye-in-hand parallel manipulator has been described in this paper. The tracking system consists of a monocular vision apparatus affixed to the top platform of a six-cylinder parallel manipulator constructed with industry grade double-acting single rod hydraulic actuators and budget proportional valves. The control algorithm is an amalgamation of three principles: machine vision based object tracking, image based visual servoing and actuator-level feedback control utilizing the inverse kinematics of the parallel manipulator. The real-time experiments are performed on a laboratory scale parallel manipulator setup with the aid of a virtual tracking object rendered on a computer monitor. The results for a few reference trajectories reveal that accurate tracking of target motions can be achieved with visual servoing algorithms designed for parallel manipulators in real-time.

This paper presents a new approach to evaluate the performances of point feature detectors for eye-in-hand visual servoing systems. The performances are analyzed in terms of stability and robustness criteria defined for a sequence of images. The first image represents the start position and the last one contains the reaching of the desired position for grasping the target object. The performances are back-analyzed, every image being compared with the last one which contains the desired features. Real-time experiments with a set-up consisting of a six d.o.f ABB robot with an eye-in-hand configuration were used to evaluate the performance. Point features are extracted with Harris and SIFT detectors and experiments show better results for the last descriptor.

A learning control technique which uses an extension of the CMAC (cerebellar model articulation controller) network developed by J.S. Albus is discussed, and the results of real-time control experiments which involved learning the dynamics of a five-axis industrial robot during high-speed, nonrepetitive movements are presented. During each control cycle, a training scheme was used to adjust the weights in the network in order to form an approximate dynamic model of the robot in appropriate regions of the control space. Simultaneously, the network was used during each control cycle to predict the actuator drives required to follow a desired trajectory, and these drives were used as feedforward terms in parallel to a fixed gain linear feedback controller. Trajectory tracking errors were found to converge to low values within a few training trails for both repetitive and nonrepetitive operations.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The conventional image-based visual servoing leads to image singularities that might cause control instabilities. To avoid this problem, in this paper, the image moments are used as features for visual servoing, where the Jacobian matrix is full rank and upper triangular. Thus, it has the maximal decoupled structure and simplifies the controller. The general analytical form of the interaction matrix or the Jacobian matrix considering the camera parameters related to any image moments is derived in this paper. As a servoing controller, an optimal visual PD controller is presented to improve the performance of the visual servoing system instead of the P controller, which is the method extensively used in visual servoing. A genetic algorithm-based PD parameters tuning method is applied to obtain the optimal parameters. The method proposed is used to align the micropeg and hole, and the simulation results show that the object can reach its desired position faster and more smoothly.

This thesis deals with Visual Servoing and its strictly connected disciplines like projective geometry, image processing, robotics and non-linear control. More specifically the work addresses the problem to control a robotic manipulator through one of the largely used Visual Servoing techniques: the Image Based Visual Servoing (IBVS). In Image Based Visual Servoing the robot is driven by on-line performing a feedback control loop that is closed directly in the 2D space of the camera sensor. The work considers the case of a monocular system with the only camera mounted on the robot end effector (eye in hand configuration). Through IBVS the system can be positioned with respect to a 3D fixed target by minimizing the differences between its initial view and its goal view, corresponding respectively to the initial and the goal system configurations: the robot Cartesian Motion is thus generated only by means of visual informations. However, the execution of a positioning control task by IBVS is not straightforward because singularity problems may occur and local minima may be reached where the reached image is very close to the target one but the 3D positioning task is far from being fulfilled: this happens in particular for large camera displacements, when the the initial and the goal target views are noticeably different. To overcame singularity and local minima drawbacks, maintaining the good properties of IBVS robustness with respect to modeling and camera calibration errors, an opportune image path planning can be exploited. This work deals with the problem of generating opportune image plane trajectories for tracked points of the servoing control scheme (a trajectory is made of a path plus a time law). The generated image plane paths must be feasible i.e. they must be compliant with rigid body motion of the camera with respect to the object so as to avoid image jacobian singularities and local minima problems. In addition, the image planned trajectories must generate camera velocity screws which are smooth and within the allowed bounds of the robot. We will show that a scaled 3D motion planning algorithm can be devised in order to generate feasible image plane trajectories. Since the paths in the image are off-line generated it is also possible to tune the planning parameters so as to maintain the target inside the camera field of view even if, in some unfortunate cases, the feature target points would leave the camera images due to 3D robot motions. To test the validity of the proposed approach some both experiments and simulations results have been reported taking also into account the influence of noise in the path planning strategy. The experiments have been realized with a 6DOF anthropomorphic manipulator with a fire-wire camera installed on its end effector: the results demonstrate the good performances and the feasibility of the proposed approach.

Vision is in fact the richest source of information for ourself and also for outdoors Robotics, and can be considered the most complex and challenging problem in signal processing for pattern recognition. The first results using Vision in the control loop have been obtained in indoors and structured environments, in which a line or known patterns are detected and followed by a robot (Feddema & Mitchell (1989), Masutani et al. (1994)). Successful works have demonstrated that visual information can be used in tasks such as servoing and guiding, in robot manipulators and mobile robots (Conticelli et al. (1999), Mariottini et al. (2007), Kragic & Christensen (2002).) Visual Servoing is an open issue with a long way for researching and for obtaining increasingly better and more relevant results in Robotics. It combines image processing and control techniques, in such a way that the visual information is used within the control loop. The bottleneck of Visual Servoing can be considered the fact of obtaining robust and on-line visual interpretation of the environment, which can be usefully treated by control structures and algorithms. The solutions provided in Visual Servoing are typically divided into Image Based Control Techniques and Pose Based Control Techniques, depending on the kind of information provided by the vision system that determine the kind of references that have to be sent to the control structure (Hutchinson et al. (1996), Chaumette & Hutchinson (2006) and Siciliano & Khatib (2008)). Another classical division of the Visual Servoing algorithms considers the physical disposition of the visual system, yielding to eye-in-hand systems and eye-to-hand systems, that in the case of Unmanned Aerial Vehicles (UAV) can be translated as on-board visual systems (Mejias (2006)) and ground visual systems (Martinez et al. (2009)). The challenge of Visual Servoing is to be useful in outdoors and non-structured environments. For this purpose the image processing algorithms have to provide visual information that has to be robust and works in real time. UAV can therefore be considered as a challenging testbed for visual servoing, that combines the difficulties of abrupt changes in the image sequence (i.e. vibrations), outdoors operation (non-structured environments) and 3D information changes (Mejias et al. (2006)). In this chapter we give special relevance to the fact of obtaining robust visual information for the visual servoing task. In section (2).we overview the main algorithms used for visual tracking and we discuss their robustness when they are applied to image sequences taken from the UAV. In sections (3). and (4). we analyze how vision systems can perform 3D pose estimation that can be used for

A PC-based visual-servoing automatic microassembly system was developed and tested. With compliant PU microgripper, regional-edge-statistics algorithm, and PD controller, a visual servoing system was implemented for gripping micro object, gluing adhesive, and operating adhesive bonding. The main specifications of the system are: gripping range of 60~80 mum , working space of 7 mmtimes5.74 mmtimes15 mm, system bandwidth of 15 Hz. In the performance test, a copper wire with diameter 60 mum was automatically gripped and transported for adhesive bonding with a copper wire of 380 mum.

For a visual servo system, there usually exists the problem of time-delay likely caused by long image processing and data transmission. The visual servo system of our robot is subject to two main limitations stemming from the specific commercial mobile manipulator, of which one is the large time-delay due to image transmission, whereas the other is failure to directly command each joint velocity as the visual servo's control input. In this paper, we propose a novel visual servo system to compensate the large time-delay, and we use integral of velocity which incorporating the image estimation and prediction with the kinematic model so as to achieve indirect velocity control with position based control mode joint. Our proposed visual servo system controls the neck joints of robot and keeps the object within the field of view of the camera during the approaching phase. We evaluate the proposed approach by experiment on a real wheeled humanoid robot and find the resulting performance quite promising, showing that our method is feasible and practical.

U ovom preglednom radu su prikazane metode vizuelnog upravljanja robotskih sistema, sa primarnim fokusom na mobilne robote sa diferencijalnim pogonom. Analizirane su standardne metode vizuelnog upravljanja bazirane na (i) greškama u parametrima slike (engl. Image-Based Visual Servoing - IBVS) i (ii) izdvojenim karakteristikama sa slike neophodnim za estimaciju položaja izabranog objekta (engl. Position-Based Visual Servoing - PBVS) i poređene sa novom metodom direktnog vizuelnog upravljanja (engl. Direct Visual Servoing - DVS). U poređenju sa IBVS i PBVS metodama, DVS metod se odlikuje višom tačnošću, ali i manjim domenom konvergencije. Zbog ovog razloga je DVS metod upravljanja pogodan za integraciju u hibridne sisteme vizuelnog upravljanja. Takođe, predstavljeni su radovi koji unapređuju sistem vizuelnog upravljanja korišćenjem stereo sistema (sistem sa dve kamere). Stereo sistem, u poređenju sa alternativnim metodama, omogućava tačniju ocenu dubine karakterističnih objekata sa slike, koja je neophodna za zadatke vizuelnog upravljanja. Predmet analize su i radovi koji integrišu tehnike veštačke inteligencije u sistem vizuelnog upravljanja. Ovim tehnikama sistemi vizuelnog upravljanja dobijaju mogućnost da uče, čime se njihov domen primene znatno proširuje. Na kraju, napominje se i mogućnost integracije vizuelne odometrije u sisteme vizuelnog upravljanja, što prouzrokuje povećanje robusnosti čitavog robotskog sistema.

Hand-eye relation in visual servoing involves eye-in-hand and eye-to-hand configuration. Both have its own merit and drawback regarding to its precision and sight range. This paper addresses this problem and introduces the camera utilization as eye-to-hand configuration for the second robot, while it retains as eye-in-hand for the first robot. Hence, the camera becomes mobile and provides more precision and more global sight of scene. Moreover, this paper also addresses the real-time constraints within real-time vision system and real-time data exchange due to different processing units for robot and vision system. Here, a pattern design and simplified image processing are considered. This paper considers a 3D visual servoing within dynamic look and move scheme based on object pose. The system performance is validated by the experimental result.

In the field of civil engineering, magnetorheological fluid (MRF) damper-based semi-active control systems have received considerable attention for use in protecting structures from natural hazards such as strong earthquakes and high winds. In this paper, the MRF damper-based semi-active control system is applied to a long-span spatially extended structure and its feasibility is discussed. Meanwhile, a trust-region method based instantaneous optimal semi-active control algorithm (TIOC) is proposed to improve the performance of the semi-active control system in a multiple damper situation. The proposed TIOC describes the control process as a bounded constraint optimization problem, in which an optimal semiactive control force vector is solved by the trust-region method in every control step to minimize the structural responses. A numerical example of a railway station roof structure installed with MRF-04K dampers is presented. First, a modified Bouc-Wen model is utilized to describe the behavior of the selected MRF-04K damper. Then, two semi-active control systems, including the well-known clipped-optimal controller and the proposed TIOC controller, are considered. Based on the characteristics of the long-span spatially extended structure, the performance of the control system is evaluated under uniform earthquake excitation and travelling-wave excitation with different apparent velocities. The simulation results indicate that the MR fluid damper-based semi-active control systems have the potential to mitigate the responses of full-scale long-span spatially extended structures under earthquake hazards. The superiority of the proposed TIOC controller is demonstrated by comparing its control effectiveness with the clipped-optimal controller for several different cases.

In particular, it is useful to pay attention to control strategies to reduce energy consumption. In this article, the purpose is to regulate and control the temperature and humidity of the air in a room with specific dimensions. The dynamical equations of the HVAC system are presented as equations of temperature and humidity variation. A control system based on LQR optimal control method is designed for this air conditioning system including two heating and cooling operations. Then, after examining the stability of the system based on the Lyapunov stability criterion, and also a temperature control performance index is defined to express and compare the performance of control system with different gains. Finally, by performing the simulation, the control and HVAC system performance is evaluated with the help of defined performance index in the presence of designed algorithm which results show that LQR gains have direct effect on energy consumption and control system performance.