Robot visual servo control based on fuzzy adaptive PID

This paper aims to develop a visual servo control of a robotic manipulator for cherry tomato harvesting. In the robotic manipulator, an RGB-depth camera was mounted to the end effector to acquire the poses of the target cherry tomatoes in space. The eye-in-hand-based visual servo controller guides the end effector to implement eye–hand coordination to harvest the target cherry tomatoes, in which a hybrid visual servo control method (HVSC) with the fuzzy dynamic control parameters was proposed by combining position-based visual servo (PBVS) control and image-based visual servo (IBVS) control for the tradeoff of both performances. In addition, a novel cutting and clipping integrated mechanism was designed to pick the target cherry tomatoes. The proposed tomato-harvesting robotic manipulator with HVSC was validated and evaluated in a laboratory testbed based on harvesting implementation. The results show that the developed robotic manipulator using HVSC has an average harvesting time of 9.40 s/per and an average harvesting success rate of 96.25% in picking cherry tomatoes.

With the growing interest of integrating robotics into everyday life and industry, the requirements towards the quality and quantity of applications grows equally hard. This trend is profoundly recognized in applications involving visual perception. Whereas visual sensing in home environments tend to be mainly used for recognition and localization, safety becomes the driving factor for developing intelligent visual control algorithms. More specifically, a robot operating in a human environment should not collide with obstacles and executed motion should be as smooth as possible. Furthermore, as the environment is not known on beforehand, a high demand on the robustness of visual processing is a necessity. On the other hand, in an industrial setting, the environment is known on beforehand and safety is mainly guaranteed by excluding a human operator. Despite these reasons, and the fact that visual servoing has gained much attention from industry to become a standard solution for robotic automation tasks, applications are highly simplified. For example, methods such as visual fault detection are already a mature technique in industrial manufacturing, where a fixed camera observes a product (e.g., on a conveyor belt) and checks whether it meets certain requirements. These operations can be executed at a fairly high rate due to the simplicity of the system (e.g., static camera) and the simplification of the processing task (e.g., binary images). For both areas the identified difficulties are similar. Foremost, this is the slow nature of (robust) visual processing, in respect to the ever growing demand of increasing speed and reducing delay. These two application areas with analogous limitations motivate the design of more direct approaches of vision in visual control systems. Therefore, in order to meet the requirements for next generation visual control systems, this thesis presents approaches which employ visual measurements as a direct feedback to design constrained motion. First, for industrial robotics, in order to obtain the required positioning accuracy, the measurement and fixation system have to be highly rigid and welldesigned, implying high cost and long design time. By measuring the position of objects directly with a camera, instead of indirectly by motor encoders, the requirements of the measurement and fixation system are less demanding. Moreover, this motivates the miniaturization of the complete control system. The approach is validated in experiments on a simplified 2D planar stage (i.e., considerable friction, poor fixation), which attains similar performance compared to encoder-based positioning systems. Secondly, in a human-centred environment, this direct sensing can improve traditional visual control systems, when subject to certain disturbances. More specifically, a method is proposed that uses an image-based feedforward controller on top of traditional position-based visual servo control to overcome disturbances such as friction or poorly designed local motor controllers. This visual feedforward control action is only active when an image-based error is present and vanishes when that error goes to zero. The method is validated on an anthropomorphic robotic manipulator with 7 degrees of freedom, intended for operation in the human care environment. Third, sensing the product directly gives rise to designing motion directly. Whereas in traditional approaches the motion trajectory is designed offline and can not be changed at runtime, direct trajectory generation computes the motion of the next step based on current state and events. This means that at any instance in time, the trajectory of a motion system can be altered with respect to certain desired kinematic or dynamic constraints. For industrial applications this makes manufacturing on near-repetitive or non-rigid structures (e.g. flexible displays) possible. When applied to a robotic manipulator, this enables obstacle avoidance to no longer be on path planning level, but on trajectory planning level, where kinematic or dynamic constraints can be taken into account. This results in a motion that is smoother than when obstacle avoidance with path planning is employed. For both application areas this direct trajectory generation method is implemented and shows high flexibility in constrained motion trajectory design.

At the beginning of its application, arm-robot manipulator worked blindly in completing the assigned task, and along with the improvement of camera technology as it is getting cheaper and smaller, the application of visual sensor lead to visual servoing control which utilized target detection as the input to control/move the end-effector of an arm-robot. The application of camera in arm-robot manipulator is divided into eye-in-hand where a camera is mounted to the robot, and eye-to-hand where the camera is fixed elsewhere. Visual control involves kinematics modeling and target position in image plane provided by image processing to ensure the motion accuracy in grasping and harvesting the fruit. The end-effector movement for harvesting relies heavily on the accuracy of target position detection that can be obtained by image processing method. This paper is a review study that discusses the design of visual servoing control of an arm-robot manipulator applied in agriculture starting from kinematics and inverse kinematics for controlling the motion and trajectory of the robot, and image processing for target detecting. An example of an image processing method is given by using Edge Detection method for detecting a dragon fruit simulated in SCILAB.

Multi-axis motion system is widely applied in commercial industrial machines such as precision CNC machine tools, Robot manipulator and laser cutting machines, etc. Contour accuracy plays a major role for the multi-axis servo motion system. The contour machining accuracy is related to the synthesis of single-axis position accuracy and multi-axis linkage accuracy. Only improving the single-axis tracking performance cannot effectively guarantee the machining accuracy of multi-axis system. The primary objective of this study was to design a contour control method to improve single-axis tracking accuracy and multi-axis contour accuracy. A control strategy that combines a sliding mode tracking controller, a disturbance observer and an adaptive fuzzy PID cross coupled controller is proposed. Sliding mode control is simple and has strong robustness to parameter changes and disturbance, which is especially suitable for control of such as non-linear multi-axis motion system. Besides, disturbance is inevitable in practical application, which degrades the motion accuracy. In order to eliminate the influence of external disturbance and uncertainty, disturbance observer is adopted to accurately estimate external disturbance and reduce the chattering phenomenon of sliding mode control, then improve the single-axis tracking accuracy. In order to further consider the coordination between different motion axes and improve the contour accuracy, the PID cross coupled control is used. Owing to conventional PID control cannot satisfy the multi-axis servo motion system with nonlinearity and uncertainty, an adaptive fuzzy method with on-line real-time PID parameters adjustment is proposed. The three-axis motion platform driven by PMLSM is used as the control object, to analysis the influence of disturbance observer on sliding mode control signal and analysis adaptive fuzzy PID cross coupled control performance respectively. The disturbance observer is used to observe the disturbance signal and estimate the disturbance well. The chattering of the sliding mode control signal is obviously improved. Next, compared with the conventional PID-CCC control, adaptive fuzzy PID- CCC control can significantly reduce the tracking error, the contour accuracy is also obviously improved. The disturbance observer can effectively eliminate the influence of external disturbance, reduce the chattering of sliding mode control, and ensure the single-axis accurate tracking. The self-adaptive fuzzy PID cross coupled controller can eliminate the influence of the dynamic characteristics mismatching and parameter difference of each axis, and improve contour accuracy. The simulation results clearly demonstrate the effectiveness of the proposed control method.

This paper focuses on the real-time target tracking for a Cartesian robot in 2 dimensions by using visual servo control. It is the process of controlling a system in servoloop by using visual image data as a referenced command to control the motion of a robot. The visual servo control can be divided into 2 loops. The first loop is image processing loop for tracking a target in video frame and the second loop is servo loop for feedback control of a Cartesian robot. A target tracking is based on color probability image using a Continuously Adaptive Mean Shift (CAMShift) algorithm that finds the most probable target position from color histogram in the current frame with a single stationary camera. The experimental results illustrate that a Cartesian robot is automatically follow a moving target and record it. The visual servo control can be adapted to more complex control applications.

Aiming at the network control system with time delay and packet loss, this paper designs a fuzzy adaptive PID controller to realize the adjustment of the network control system. Through the design of wavelet neural algorithm to predict the time delay, the fuzzy adaptive PID can realize the online adjustment of Kp, Ki and Kd, which overcomes the problem of constant input parameter adjustment of the traditional PID controller. Fuzzy control is realized by designing fuzzy rules, and finally through comparison The Matlab simulation renderings of traditional PID controller and fuzzy adaptive PID controller show that the robustness and accuracy of fuzzy adaptive PID controller are better.

SummaryThe goal of target tracking is to compute motion strategies for a robot equipped with visual sensors, so that it can effectively track a moving target despite obstruction by obstacles. It is an important problem with many applications in robotics. Existing work focuses mostly on the 2-D version of the problem, partly due to the complexity of dealing with 3-D visibility relationships. This paper proposes an online algorithm for 3-D target tracking among obstacles, using only local geometric information available to a robot’s visual sensors. Key to this new algorithm is the definition and efficient computation of a risk function, which tries to capture a target’s ability in escaping from the robot sensors’ visibility region in both short and long terms. The robot then moves to minimize this risk function locally in a greedy fashion. In the absence of occlusion by obstacles, the standard tracking algorithm based on visual servo control can be considered a special case of our algorithm. Experiments show that the new algorithm generated interesting tracking behaviors in three dimensions and performed substantially better than visual servo control in simulation.

The paper realizes a design of digital fuzzy adaptive tuning PID electronic governor. It is based on VHDL description and implemented on FPGA. The paper proposes to detect the speed with duty cycle calculation and use fuzzy-control rules to amend the PID parameters on line to achieve the fuzzy adaptive tuning PID control. On this basis, the paper introduces the implementation of the system in detail, including speed detection, fuzzy logic inference, and fuzzy adaptive tuning PID algorithm. The experiment proves that digital wide-range electronic speed governor implemented on control chip FPGA can be used widely and fuzzy adaptive tuning PID control algorithm can improve the dynamic response of the system, which makes the electronic speed governor under good control in a wider work scope.

In this brief, the robust position-based visual servoing control problem is addressed for a quadrotor with a down-facing monocular camera to perform ground target tracking tasks in outdoor environments. The object is to regulate the position of the quadrotor relative to the ground target by using a minimum sensor set, including an inertial measurement unit, an ultrasonic sensor, and a vision sensor. The robust compensators are designed in the visual servoing control system to enhance the robustness against nonlinearities, couplings, and uncertainties, including parameter uncertainties and external disturbances in the translational and rotational quadrotor dynamics. Theoretical analysis and experimental results demonstrate the effectiveness of the proposed robust visual servoing control method.

Wheeled Mobile robots (WMRs) are very popular because they are appropriate for applications with relatively low mechanical complexity and energy consumption. WMRs are used in many military and civilian applications. These applications include surveillance, transportation, inspection, exploration, and many other applications either in structured or unstructured environments. This paper provides a review on the developments of the wheeled mobile robots (WMRs) control. It reviews the kinematic and dynamic modeling of the robots, the path following and tracking, the avoiding obstacles control algorithms, and finally a review on robot vision and visual servoing control system is presented including the main challenges in each field of research.

The use of unmanned aerial vehicles (UAVs) for military, scientific, and civilian sectors are increasing drastically in recent years. This study presents algorithms for the visual-servo control of an UAV, in which a quadrotor helicopter has been stabilized with visual information through the control loop. Unlike previous study that use pose estimation approach which is time consuming and subject to various errors, the visual-servo control is more reliable and fast. The method requires a camera on-board the vehicle, which is already available on various UAV systems. The UAV with a camera behaves like an eye-in-hand visual servoing system. In this study the controller was designed by using two different approaches; image based visual servo control method and hybrid visual servo control method. Various simulations are developed on Matlab, in which the quadrotor aerial vehicle has been visual-servo controlled. In order to show the effectiveness of the algorithms, experiments were performed on a model quadrotor UAV, which suggest successful performance.

In this paper we study image based visual servo (IBVS) control of a four rotor vertical take-off and landing (VTOL) craft known as an X4-flyer. The approach taken is tailored to planar visual targets and is based on a reformulation and extension of 2 1/2 D image-based visual servo design to the control of dynamic under-actuated systems. Exponential stability of the closed loop system is proved and simulation results are presented.

Purpose Commercial bioprinting systems are typically characterized by high costs and limited adaptability, presenting a significant barrier for academic research. This study aims to introduce and empirically validate an integral prototyping methodology for developing a low-cost, accessible robotic bioprinters. The primary objective is to provide a holistic framework that integrates mechanical design, electronics and control systems, aimed at addressing key engineering challenges. This work also addresses the critical issue of maintaining high positional accuracy in a system with a variable payload mass. Design/methodology/approach Following the Integral Prototyping framework, a multi-link robotic manipulator bioprinter was developed through an iterative process that integrated mechanical, electronic and control subsystems. A direct-drive extruder was implemented to minimize material waste by embedding cartridges into the end-effector, a design choice that created the central challenge of a variable payload. This study develops a payload-agnostic visual servoing controller. The system’s adaptability and the controller’s performance were validated by testing with five biomaterials and two 50:50 composites with diverse viscosities (gelatin-based hydrogel, lipid-rich emulsion, ground pork, starch-based hydrogel and composites), confirming the framework’s control strategy and the overall adaptability to heterogeneous biomaterials. Findings Application of the Integral Prototyping methodology resulted in a robotic bioprinter capable of sustaining high positional accuracy despite variable payloads. The visual servoing controller improved the system’s precision, reducing average positional deviations for key joint angles (e.g. from 13.18% to 6.62% for q1 and from 6.74% to 4.08% for q2), demonstrating effective compensation for mass variability introduced by the direct-drive extruder. The control system interfaces with standardized G-code to ensure both accuracy and accessibility. This control robustness translated into accurate fabrication outcomes, with dimensional deviations below 2.0% and scaffold fidelity deviations between 2.0% and 3.3%. Scaffold integrity was partially constrained given the configuration of the extrusion module. Results confirm that the control system successfully decouples positional precision from payload variability. These outcomes validate Integral Prototyping as a framework that not only integrates design, electronics and software but also resolves emergent engineering challenges central to accessible bioprinting. Originality/value The primary contribution is a reproducible Integral Prototyping methodology for developing low-cost, adaptable robotic manufacturing bioprinting platform that remain robust under variable operating condition. This work contributes in uniting design, electronics and control into a single framework while providing a validated visual servoing solution to the long-standing robotics challenge of variable end-effector mass. Beyond the specific controller, this study establishes a generalizable blueprint for rapid prototyping and tissue engineering, demonstrating that cost and flexibility barriers in bioprinting can be addressed through integrative, methodology-driven development.

A balancing control algorithm for a wheel-legged robot is designed for current logistics and distribution mobile robots. Since the wheel-legged robot is a nonlinear underactuated system, it is crucial to realize the balanced control and robustness of the wheel-legged robot in the absence of an accurate dynamics model. In this paper, an optimal control scheme for the wheel-legged robot based on the fusion control of variable-height adaptive fuzzy PID and conventional PID is designed. The balance of the wheel-legged robot is controlled by adaptive fuzzy PID control, the lifting and attitude changes of the two sides of the wheel-legs are controlled with high precision by traditional PID control, and the speed and steering control of the wheel-legged robot is not affected by the structure of the model, and is controlled by traditional PID control with linearization. The attitude of the robot is detected in real time using the BMI088 attitude sensor to realize the positional control. Experimental and simulation results show that the control algorithm of the wheel-legged robot designed in this paper is reliable, and the robot runs smoothly and robustly.

Focusing on strong nonlinearity and time variation of switching-inverter power, this paper presents a fuzzy adaptive PID control algorithm. This algorithm not only avoids complex modeling of the controlled object, but also combines advantages of fuzzy control and PID control. In addition, this algorithm realizes real-time and effective online control for switching-inverter power by setting parameters of PID based on fuzzy reasoning. Furthermore, the control algorithm improves steady-state precision and dynamic performance of system, either of which the PID algorithm can be realized simultaneously. A simulation model of system was made aiming at the response of unit step, constant load and cyclical load simulation. The simulation results show that the switching-inverter power based on fuzzy adaptive PID has excellent control performance and self-adjustable ability, and this controller improves the output waveform.