Implementation of an improved moment-based visual servoing controller on an industrial robot

In this paper, a proportional-integral sliding mode control (PISMC) method is presented to achieve the attitude and position control of an unmanned helicopter system for autonomous hovering. The PISMC includes the advantages of both conventional sliding mode control (SMC) and integral sliding mode control (ISMC). The proposed PISMC algorithm is capable of stabilized attitude and position control for unmanned helicopter autonomous hovering, and it offers a performance improvement over the conventional SMC and ISMC algorithms as well. The system performance is quantified and an experimental unmanned helicopter system is developed to access the performance of the PISMC. Both simulations and actual flight tests are conducted to verify the PISMC attitude and position control. The simulation results show that the PISMC tracking error is smaller and that PISMC converges faster than SMC does. The flight test results prove that the proposed PISMC is effective in the attitude and position control of unmanned helicopter autonomous hovering.

This work presents different proposals to deal with common problems in robot visual servoing based on the application of discontinuous control methods. The feasibility and effectiveness of the proposed approaches are substantiated by simulation results and real experiments using a 6R industrial manipulator. The main contributions are:\n\n- Geometric invariance using sliding mode control (Chapter 3): the defined higher-order invariance is used by the proposed approaches to tackle problems in visual servoing. Proofs of invariance condition are presented.\n\n- Fulfillment of constraints in visual servoing (Chapter 4): the proposal uses sliding mode methods to satisfy mechanical and visual constraints in visual servoing, while a secondary task is considered to properly track the target object. The main advantages of the proposed approach are: low computational cost, robustness and fully utilization of the allowed space for the constraints.\n\n- Robust auto tool change for industrial robots using visual servoing (Chapter 4): visual servoing and the proposed method for constraints fulfillment are applied to an automated solution for tool changing in industrial robots. The robustness of the proposed method is due to the control law of the visual servoing, which uses the information acquired by the vision system to close a feedback control loop. Furthermore, sliding mode control is simultaneously used in a prioritized level to satisfy the aforementioned constraints. Thus, the global control accurately places the tool in the warehouse, but satisfying the robot constraints.\n\n- Sliding mode controller for reference tracking (Chapter 5): an approach based on sliding mode control is proposed for reference tracking in robot visual servoing using industrial robot manipulators. The novelty of the proposal is the introduction of a sliding mode controller that uses a high-order discontinuous control signal, i.e., joint accelerations or joint jerks, in order to obtain a smoother behavior and ensure the robot system stability, which is demonstrated with a theoretical proof.\n\n- PWM and PFM for visual servoing in fully decoupled approaches (Chapter 6): discontinuous control based on pulse width and pulse frequency modulation is proposed for fully decoupled position based visual servoing approaches, in order to get the same convergence time for camera translation and rotation.\n\nMoreover, other results obtained in visual servoing applications are also described.

With the growing interest of integrating robotics into everyday life and industry, the requirements towards the quality and quantity of applications grows equally hard. This trend is profoundly recognized in applications involving visual perception. Whereas visual sensing in home environments tend to be mainly used for recognition and localization, safety becomes the driving factor for developing intelligent visual control algorithms. More specifically, a robot operating in a human environment should not collide with obstacles and executed motion should be as smooth as possible. Furthermore, as the environment is not known on beforehand, a high demand on the robustness of visual processing is a necessity. On the other hand, in an industrial setting, the environment is known on beforehand and safety is mainly guaranteed by excluding a human operator. Despite these reasons, and the fact that visual servoing has gained much attention from industry to become a standard solution for robotic automation tasks, applications are highly simplified. For example, methods such as visual fault detection are already a mature technique in industrial manufacturing, where a fixed camera observes a product (e.g., on a conveyor belt) and checks whether it meets certain requirements. These operations can be executed at a fairly high rate due to the simplicity of the system (e.g., static camera) and the simplification of the processing task (e.g., binary images). For both areas the identified difficulties are similar. Foremost, this is the slow nature of (robust) visual processing, in respect to the ever growing demand of increasing speed and reducing delay. These two application areas with analogous limitations motivate the design of more direct approaches of vision in visual control systems. Therefore, in order to meet the requirements for next generation visual control systems, this thesis presents approaches which employ visual measurements as a direct feedback to design constrained motion. First, for industrial robotics, in order to obtain the required positioning accuracy, the measurement and fixation system have to be highly rigid and welldesigned, implying high cost and long design time. By measuring the position of objects directly with a camera, instead of indirectly by motor encoders, the requirements of the measurement and fixation system are less demanding. Moreover, this motivates the miniaturization of the complete control system. The approach is validated in experiments on a simplified 2D planar stage (i.e., considerable friction, poor fixation), which attains similar performance compared to encoder-based positioning systems. Secondly, in a human-centred environment, this direct sensing can improve traditional visual control systems, when subject to certain disturbances. More specifically, a method is proposed that uses an image-based feedforward controller on top of traditional position-based visual servo control to overcome disturbances such as friction or poorly designed local motor controllers. This visual feedforward control action is only active when an image-based error is present and vanishes when that error goes to zero. The method is validated on an anthropomorphic robotic manipulator with 7 degrees of freedom, intended for operation in the human care environment. Third, sensing the product directly gives rise to designing motion directly. Whereas in traditional approaches the motion trajectory is designed offline and can not be changed at runtime, direct trajectory generation computes the motion of the next step based on current state and events. This means that at any instance in time, the trajectory of a motion system can be altered with respect to certain desired kinematic or dynamic constraints. For industrial applications this makes manufacturing on near-repetitive or non-rigid structures (e.g. flexible displays) possible. When applied to a robotic manipulator, this enables obstacle avoidance to no longer be on path planning level, but on trajectory planning level, where kinematic or dynamic constraints can be taken into account. This results in a motion that is smoother than when obstacle avoidance with path planning is employed. For both application areas this direct trajectory generation method is implemented and shows high flexibility in constrained motion trajectory design.

U ovom preglednom radu su prikazane metode vizuelnog upravljanja robotskih sistema, sa primarnim fokusom na mobilne robote sa diferencijalnim pogonom. Analizirane su standardne metode vizuelnog upravljanja bazirane na (i) greškama u parametrima slike (engl. Image-Based Visual Servoing - IBVS) i (ii) izdvojenim karakteristikama sa slike neophodnim za estimaciju položaja izabranog objekta (engl. Position-Based Visual Servoing - PBVS) i poređene sa novom metodom direktnog vizuelnog upravljanja (engl. Direct Visual Servoing - DVS). U poređenju sa IBVS i PBVS metodama, DVS metod se odlikuje višom tačnošću, ali i manjim domenom konvergencije. Zbog ovog razloga je DVS metod upravljanja pogodan za integraciju u hibridne sisteme vizuelnog upravljanja. Takođe, predstavljeni su radovi koji unapređuju sistem vizuelnog upravljanja korišćenjem stereo sistema (sistem sa dve kamere). Stereo sistem, u poređenju sa alternativnim metodama, omogućava tačniju ocenu dubine karakterističnih objekata sa slike, koja je neophodna za zadatke vizuelnog upravljanja. Predmet analize su i radovi koji integrišu tehnike veštačke inteligencije u sistem vizuelnog upravljanja. Ovim tehnikama sistemi vizuelnog upravljanja dobijaju mogućnost da uče, čime se njihov domen primene znatno proširuje. Na kraju, napominje se i mogućnost integracije vizuelne odometrije u sisteme vizuelnog upravljanja, što prouzrokuje povećanje robusnosti čitavog robotskog sistema.

With the rapid progress of science and technology, industrial manufacturing and processing technology have also developed rapidly. Industrial robots are widely used in many industrial fields. In this paper, the design and implementation of trajectory planning based on 4 DOF industrial robot is accomplished. Firstly, trajectory planning of the 4 DOF industrial robot is analyzed. Secondly, the operation path is planned, and detailed operation steps are designed. Finally, the simulation verification process of trajectory planning for the 4 DOF industrial robot is completed, and some research results are obtained.

Purpose Real-time implementation of sophisticated algorithms on robotic systems demands a rewarding interface between hardware and software components. Individual robot manufacturers have dedicated controllers and languages. However, robot operation would require either the knowledge of additional software or expensive add-on installations for effective communication between the robot controller and the computation software. This paper aims to present a novel method of interfacing the commercial robot controllers with most widely used simulation platform, e.g. MATLAB in real-time with a demonstration of visual predictive controller. Design/methodology/approach A remote personal computer (PC), running MATLAB, is connected with the IRC5 controller of an ABB robotic arm through the File Transfer Protocol (FTP). FTP server on the IRC5 responds to a request from an FTP client (MATLAB) on a remote computer. MATLAB provides the basic platform for programming and control algorithm development. The controlled output is transferred to the robot controller through Ethernet port as files and, thereby, the proposed scheme ensures connection and control of the robot using the control algorithms developed by the researchers without the additional cost of buying add-on packages or mastering vendor-specific programming languages. Findings New control strategies and contrivances can be developed with numerous conditions and constraints in simulation platforms. When the results are to be implemented in real-time systems, the proposed method helps to establish a simple, fast and cost-effective communication with commercial robot controllers for validating the real-time performance of the developed control algorithm. Practical implications The proposed method is used for real-time implementation of visual servo control with predictive controller, for accurate pick-and-place application with different initial conditions. The same strategy has been proven effective in supervisory control using two cameras and artificial neural network-based visual control of robotic manipulators. Originality/value This paper elaborates a real-time example using visual servoing for researchers working with industrial robots, enabling them to understand and explore the possibilities of robot communication.

ROS-Industrial based robotic cell for Industry 4.0: Eye-in-hand stereo camera and visual servoing for flexible, fast, and accurate picking and hooking in the production line

In image-based visual servoing (IBVS), parametric uncertainties tend to cause the model inaccuracy and limit the control performance. Considering these uncertainties can be embodied by the output–input data from the visual servoing system, this brief proposes an eye-in-hand visual servoing control (VSC) scheme based on the input mapping method, which directly utilizes the past output–input data to enhance the original feedback control law rather than identifying the model. The system with the input mapping method is proven to not only maintain the stability of the original VSC but also accelerate the convergent rate. The results of the experiments on a manipulator with an eye-in-hand camera demonstrate the superiority of our proposed method.

This paper aims to develop a visual servo control of a robotic manipulator for cherry tomato harvesting. In the robotic manipulator, an RGB-depth camera was mounted to the end effector to acquire the poses of the target cherry tomatoes in space. The eye-in-hand-based visual servo controller guides the end effector to implement eye–hand coordination to harvest the target cherry tomatoes, in which a hybrid visual servo control method (HVSC) with the fuzzy dynamic control parameters was proposed by combining position-based visual servo (PBVS) control and image-based visual servo (IBVS) control for the tradeoff of both performances. In addition, a novel cutting and clipping integrated mechanism was designed to pick the target cherry tomatoes. The proposed tomato-harvesting robotic manipulator with HVSC was validated and evaluated in a laboratory testbed based on harvesting implementation. The results show that the developed robotic manipulator using HVSC has an average harvesting time of 9.40 s/per and an average harvesting success rate of 96.25% in picking cherry tomatoes.

This paper has two purposes: investigating a featureless visual servoing approach based on mutation analysis and proposing a visual servo control method for nanomanipulations. For the first purpose, the featureless visual servoing method is needed because traditional visual servoing relies heavily on robust feature extraction and tracking, which are very difficult in natural environment. The mutation analysis based approach in this paper considers the image as a set, and designs a controller to make the distance between the initial and goal image sets converge to zero, thereby steering the initial image to the goal image. For the second purpose, atomic force microscopic (AFM) based nanomanipulations with subnanometer accuracy are very difficult because the position sensor cannot provide valuable feedback due to large noises at this precision level. We propose to use the images obtained by AFM and perform a visual servo control. This method, independent of external sensors, can directly perform control on the AFM end tip's position. The featureless controller is successfully validated on AFM images and the results suggest a potential precision enhancement for nanomanipulations.

A new stable Adaptive Interval Type-2 Fuzzy Proportional Integral Sliding Mode Controller (AI2FPISMC) is introduced here to control a class of nonlinear systems. The proposed method is based on interval type-2 fuzzy logic system (IT2FLS) whose antecedent and consequent membership functions are interval type-2 fuzzy sets. IT2FLS is utilized to approximate unknown nonlinear functions. To achieve high performance, optimizing membership functions (MFs) of interval type-2 fuzzy sets (IT2FS) is required. Genetic algorithm (GA) is a parallel search optimization method; that here contributes to optimize the MFs. In order to cope with the chattering of sliding mode controller, PI control law is proposed and Lyapunov analysis is utilized to prove asymptotic stability of the proposed approach. The adaptation laws are derived using Lyapunov approach. Two nonlinear system simulation examples are presented to verify the effectiveness of the proposed method, and their results confirm the optimization merits.

Accurate and robust pollinations for watermelons using intelligence guided visual servoing

For a visual servo system, there usually exists the problem of time-delay likely caused by long image processing and data transmission. The visual servo system of our robot is subject to two main limitations stemming from the specific commercial mobile manipulator, of which one is the large time-delay due to image transmission, whereas the other is failure to directly command each joint velocity as the visual servo's control input. In this paper, we propose a novel visual servo system to compensate the large time-delay, and we use integral of velocity which incorporating the image estimation and prediction with the kinematic model so as to achieve indirect velocity control with position based control mode joint. Our proposed visual servo system controls the neck joints of robot and keeps the object within the field of view of the camera during the approaching phase. We evaluate the proposed approach by experiment on a real wheeled humanoid robot and find the resulting performance quite promising, showing that our method is feasible and practical.

A review on visual servoing for underwater vehicle manipulation systems automatic control and case study

This paper addresses the adaptive visual servoing control for robotic manipulator under uncalibrated eye-to-hand configuration with unknown actuator backlash. Most existing control approaches to visual servoing manipulator system ignore actuator backlash constraint, which is inevitable in mechanical robot and will significantly affect the dynamic performance. To alleviate the deleterious and undesirable effect caused by the unknown actuator backlash, a smooth adaptive inverse is proposed to compensate the backlash effect Based on the visual servoing system, a novel updating law is presented to estimate the backlash model parameters. Moreover, an adaptive image-based visual servoing(IBVS) controller is constructed to guarantee the global stability of closed-loop system and the boundary of parameters. The stability of dynamic system and convergence of the image error are proved by Lyapunov method. Lastly, simulations are carried out to validate the availability of the proposed controller.