Vision-based 2D and 3D Control of Robot Manipulators

Real-time motion planning and control of multi-joint robot manipulators are studied using neural networks. The proposed neural network approach consists of two modules: one for real-time collision-free motion planning, the other for real-time fine control of the robot manipulators. The motion planning module is a biologically inspired, parallel connected, topologically organised neural network, where each neuron is characterised by a shunting equation, whose state space is the robot configuration joint space. This module is capable of planning a real-time optimal path for robot manipulators through the dynamic activity landscape of the neural network. The motion control module involves of a feedforward neural network together with a traditional PD feedback loop. It is capable of achieving real-time fine motion control of robot manipulators under significant uncertainties and without any prior knowledge of the robot dynamics. This module can quickly compensate sudden changes in the robot dynamics. The real-time fine control of robot manipulators is achieved through the online learning of the neural network. Both the motion planning and motion control modules are computationally efficient, and their global stabilities are proved using Lyapunov stability theory.

Inspite of so much research, key technical problem, naming chattering of conventional, simple and robust SMC is still a challenge to the researchers and hence limits its practical application. However, newly developed soft computing based techniques can provide solution. In order to have advantages of conventional and heuristic soft computing based control techniques, in this paper various commonly used intelligent techniques, neural network, fuzzy logic and adaptive neuro fuzzy inference system (ANFIS) have been combined with sliding mode controller (SMC). For validation, proposed hybrid control schemes have been implemented for tracking a predefined trajectory by robotic manipulator, incorporating structured and unstructured uncertainties in the system. After reviewing numerous papers, all the commonly occurring uncertainties like continuous disturbance, uniform random white noise, static friction like coulomb friction and viscous friction, dynamic friction like Dhal friction and LuGre friction have been inserted in the system. Various performance indices like norm of tracking error, chattering in control input, norm of input torque, disturbance rejection, chattering rejection have been used. Comparative results show that with almost eliminated chattering the intelligent SMC controllers are found to be more efficient over simple SMC. It has also been observed from results that ANFIS based controller has the best tracking performance with the reduced burden on the system. No paper in the literature has found to have all these structured and unstructured uncertainties together for motion control of robotic manipulator.

We present a new technique to the design and real-time implementation of fuzzy control system basedon digital signal processors in order to improve the precision and robustness for system of industrial robot in this paper. The need to meet demanding control requirement in increasingly complex dynamical control systems under significant uncertainties, leads toward design of intelligent manipulation robots. The TMS320C80 is used in implementing real time fuzzy control to provide an enhanced motion control for robot manipulators. In this paper, a Self-Organizing Fuzzy Controller for the industrial robot manipulator with a actuator located at the base is studied. A fuzzy logic composed of linguistic conditional statements is employed by defining the relations of input-output variables of the controller. In the synthesis of a Fuzzy Logic Controller, one of the most difficult problems is the determination of linguistic control rules from the human operators. To overcome this difficult Self-Organizing Fuzzy Controller is proposed for a hierarchical control structure consisting of basic and high levels that modify control rules. The proposed Self-Organizing Fuzzy Controller scheme is simple in structure, fast in computation, and suitable for implementation of real-time control. Performance of the SOFC is illustrated by simulation and experimental results for a Dual-Arm robot with eight joints.

This book is the second edition of a textbook published in 1996 by McGraw Hill and originates from a graduate level course given by the authors at the University of Naples. The topics include kinematics, statics and dynamics of robot manipulators together with trajectory planning and active control. There are only minor additions the first edition, which are mainly the use of quaternion to describe the orientation of the end effector and a short description of a closed chain architecture for a manipulator (parallelogram arm). The book is largely devoted to serial manipulators, with special developments about active control including adaptative control, robust controls and stability analysis. Another strength of this book is the great number of problems proposed at the end of each chapter, together with a list of references related to it. The fundamental features covered by the text are illustrated on simple examples of serial manipulators (two-link planar arm, parallelogram arm) including analytical results and numerical tests.The book has nine chapters followed by three appendices. The first appendix is devoted to linear algebra, the second recalls some fundamental aspects of rigid body mechanics and the third gives some basic principles of feedback control of linear systems.Chapter one is an introduction to the study of robot manipulators, giving an interesting classification of their architectures, the corresponding workspace and describes the tasks for which theyare used. After some standard examples of industrial manipulators, bibliographical reference texts are proposed, including textbooks on modelling and control of robots, general books on robotics, specialized texts, scientific robotic reviews and some international conferences on robotics.Chapters two, three and four are devoted to mechanical modelling of robot manipulators. The fundamental basics of kinematics are given in chapter two, including the representation of finite rotations by Euler angles or unit quaternions, homogeneous transformations, Denavit-Hartenberg parameters and workspace. The direct and inverse kinematical problems are solved in analytical form for some typical manipulator structures. The differential kinematics of robots are presented in chapter three, with an introduction to the geometric and analytic Jacobian matrices, kinematic singularities and redundancy. The inverse kinematic problem is presented, with special attention to the case of redundant robots where the solution is obtained by a linear optimization problem leading to the introduction of the pseudo-inverse Jacobian matrix and to the solution of several objectives such as avoidance of collision with an obstacle or moving away from singularities. Several inverse kinematics algorithms are given wih an interesting application to a three-link planar arm. Finally, a property of kineto-statics duality is deduced from the principle of virtual work applied to an equilibrium configuration of the robot. Chapter four is a standard presentation of the derivation of the dynamical model by Lagrange formulation and then by the Newton-Euler method. In the Lagrange formulation method, the linearity with respect to inertial parameters is shown and a detailed formulation of the dynamical model is obtained for a two-link Cartesian arm, a two-link planar arm and a parallelogram arm. The problem of dynamic parameter identification is also briefly presented from a numerical point of view. The recursive algorithm constructed from the Newton-Euler formulation is presented and illustrated by considering a two-link planar arm. Finally, the operational space dynamic model is introduced.In chapter five, paths and trajectory planning in joints and in operational spaces are presented; several classical methods of interpolation are described.Chapter six is an extensive study of active control of manipulators. Several methods are presented, involving classical independent joint control, non-linear centralized control, robust control and adaptative control. Both joint-space control and operational-space control are studied together with stability analysis by using Liapounoff functions. An interesting application to the two-link planar arm already used shows the comparison between various control schemes.Chapter seven deals with interaction control of serial manipulators with the working environment. Several strategies involving compliance control, impedance control, force control and hybrid control are presented.Chapter eight describes the actuators and the sensors used in robotics. Several types of servomotors (electric and hydraulic) are presented, together with the model giving their input/output relationship. Several kinds of sensors are also described including encoders, tachometers, force and vision sensors.The last chapter gives a short presentation of the functional architecture of a robot's control system, including characteristics of the programing environment and the hardware architecture.In conclusion, the book provides a good insight about simulation and control of robot manipulators, with a detailed study of the various control strategies and several interesting and pedagogical applications. This book is an excellent review of the standard knowledge needed not only for graduate students but also for researchers interested in robot manipulators.M Pascal

Fine motion control of robotic manipulators has become a desired goal in the last few years as a result of new robot morphology and the definition of new tasks involving high velocity motions and end effector tracking precision. In order to achieve better performance of robotic manipulators the artificial intelligence can be introduced into the control system. One way for accomplishing this is application of neural nets. Main advantage claimed for neural based controllers is their ability to learn and generalize from partial data and ability to perform parall processing. Many papers present possibilities of neural nets application for control [Hunt,92],[Sontag 93],[Anstaklis,92]. They have discussed the use in adaptive control, predictive control, optimal control, gain sheduling in PID with variable gain. A large area of neural nets application for control is control of nonlinear systems. Neural nets can be applied as a forward or inverse model of manipulator. Such neural net can learn a behaviour of the real system, without knowledge about robot’s model structure.

This paper presents the current state of the art in the adaptive control of single rigid robotic manipulators in the constrained motion tasks. A complete mathematical model of a single rigid robotic manipulator in contact with dynamic environment is presented. The basic approaches in deriving the environment model are given. The significance of the dynamic environment in the scope of the stability problem of the whole system robot-dynamic environment is emphasized. A classification of the adaptive contact control concepts in manipulation robotics is presented. The main characteristics of the most important adaptive strategies in constrained manipulation are given. The advantages and the drawbacks of the presented methods are emphasized. The paper covers results published a few years ago, as well as some recent trends in this field. One important result in the stability analysis of robotic manipulators in the constrained motion tasks is reported. Finally, some concluding remarks are given and possible future investigation trends in adaptive control of robotic manipulators are indicated.

This study presents the design and validation of a novel sampled-time state feedback controller for a multi-robotic system composed of a Cartesian robot and a Delta robot, providing five degrees of freedom – two from the Cartesian and three from the Delta robot. The novelty of this work lies in the integration of a machine learning-based visual servoing strategy with the discrete-time control design, where the controller's sampling period is carefully selected based on the computational demands of the vision-based feedback. The end-effector's pose is estimated in real time using a modified eye-to-hand YOLOv8 algorithm embedded within a neural network visual servoing system. This approach enables the robotic system to continuously update its control actions based on visual feedback, enhancing trajectory tracking accuracy during task execution. A key contribution of this work is the investigation of the impact of sampling time on the convergence quality of the end-effector's tracking error. The proposed control architecture was rigorously evaluated in a virtual environment, where the end-effector pose estimated by the visual system was compared against ground truth data. The experimental results demonstrate that adjusting the sampling time significantly improves control accuracy, thereby justifying the proposed discrete-time controller design. This integrated machine learning and sampled-time control approach shows promising potential for real-time, vision-guided robotic manipulation. Highlights A synthetic database for the proposed controller using the Segment Anything Model (SAM) for image processing. A CNN algorithm to detect the position and orientation of the end-effector of a composite robotic manipulator based on a sequential configuration of an open-closed kinematic chain. A sampled controller that drives the position/orientation of the composite robotic manipulator to time-varying specific target configurations, including its convergence analysis.

This paper presents an uncalibrated visual servoing control system based on the human–robot–robot cooperation (HRRC). In case of malfunctions of the joint sensors of a robotic manipulator, the proposed system enables the mobile robot to continue operating the manipulator to complete the task that requires careful handling. With the aid of a virtual exoskeleton, an operator may use a human–computer interaction (HCI) device to guide the malfunctioning manipulator. During the guiding process, the virtual exoskeleton serves as a connector between the HCI device and the manipulator. However, when using the HCI device to guide the virtual exoskeleton, there could be a risk of a large-residual problem at any time caused by non-uniform guiding. To solve this problem, a residual switching algorithm (RSA) has been proposed that can identify whether the residual should be calculated based on the motion characteristics of the artificial guiding, reducing the computational cost and ensuring the tracking stability. To enhance the virtual exoskeleton’s ability to drive the manipulator, a multi-joint fuzzy driving controller has been proposed, which can drive the corresponding joint of the manipulator in accordance with an offset vector between the virtual exoskeleton and the manipulator. Lastly, the guiding experiments have verified that, compared with the contrast algorithm, the proposed RSA has a better tracking performance. A peg-in-hole assembly experiment has shown that the proposed control system can assist the operator to control efficiently the robotic manipulator with malfunctioning joint sensors.

Model-based control such as computed torque control is frequently employed to ensure the accurate control of a robot manipulator. However, in some cases control performance is not satisfactory due to unmodeled nonlinearities or dynamics. To overcome this issue, this study investigates how using a quaternion neural network can adaptively compensate for the computed torque control. The control system consists of the quaternion neural network, feedforward model and feedback controller, resulting in a feedback error learning scheme utilised for the training of the quaternion neural network with a backpropagation algorithm extended to quaternion numbers. In computational experiments, the trajectory control of a three-link robot manipulator is performed using the proposed control system. Simulation results confirm the effectiveness of the quaternion neural network in practical control applications.

Position-based visual servo control of autonomous robotic manipulators

In this article we present the stability analysis of a class of PD-type controllers for position and motion control of robot manipulators. The main feature of this class of controllers is that the proportional and derivative gains can be nonlinear functions of the robot states. These controllers can be obtained from control strategies that adjust the controller gains depending on the robot states. It is shown that global asymptotic stability of the control system is achieved provided that the P and D gains have suitable structure. As an outcome, we propose a global regulator constrained to deliver torques within prescribed limits of the actuator's capabilities. Experimental results on a two degrees of freedom direct drive arm show the usefulness of the proposed scheme. © 1996 John Wiley & Sons, Inc.

Presents an approach to real time implementation of visual feedback control for a robotic manipulator. There are mainly two ways to put the visual feedback into practice. One is called look-and-move and the other is visual servoing. The former is the method which transforms the position and orientation of an object obtained by a visual sensor into those in the world frame fixed to an environment and guides the arm of the manipulator to a desired location in the world frame. Visual servoing uses the Jacobian matrix which relates the displacement of an image feature to the displacement of a camera motion and performs a closed-loop control regarding the feature as a scale of the state. Therefore, we can construct a servo system based only on the image and can have a robust control against the calibration error because there is no need to calculate the corresponding location in the world frame. In order to control the position and orientation of a robot with respect to an object, a new technique is proposed using binocular stereo vision. The stereo vision enables us to calculate an exact image Jacobian not only around a desired location but also at other locations. The suggested technique can guide a robot manipulator to the desired location without giving such priori knowledge as the relative distance to the desired location or the model of an object even if the initial positioning error is large. The performance of the proposed visual servoing system is illustrated by the experimental results and compared with the case of conventional method for an assembling robot with four joints.

This paper describes the implementation of hierarchical control on a robotic manipulator using fuzzy logic. A decentralized control approach is implemented, i.e., individual controllers control the two links of the robot. The kinematic aspect of the control is treated as the supervisory mode at a higher level and the joint control is treated as the lower level. Fuzzy logic based rules determine the inverse kinematic mapping which maps the Cartesian coordinates to the individual joint angles. This scheme is implemented using Togai Infra Logic software and the entire simulation software is implemented using C language. The results of the simulation are discussed. This experiment is a proof of principle to show that the fuzzy controller can be used to map the nonlinear mapping which can be implemented to the more complex problem of inverse kinematics of higher degree of freedom robots. A fuzzy PD controller is implemented on a Rhino robot and the performance is compared with a traditional PD controller.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

In a robot control system there are dynamics and sensor faults, modeling errors, and system and measurement noises. This paper addresses several techniques to control the robot manipulator under the uncertainty model and sensor failure. First, a neural network is proposed to control the robot to follow the generated path. This network model is able to compensate the structured and unstructured dynamic uncertainties of the robot by using both online and off-line training. The PD controller, computed torque controller and adaptive controller with and without exact model are used to control the robot and compared with the proposed method. Next, a residual generator for detection and isolation of the sensor failure is designed. Assuming one fault at a time, the proposed method can detect and isolate the fault sensor. The simulation study based on a 2-DOF planar robot is included.

Observer-Based Adaptive Sliding Control of Robotic Manipulators