Intelligent optimization of robotic impedance control parameters based on reinforcement learning

This paper proposes an intelligent massage control system by using multi-fingered robot hand with hybrid impedance and force control, which is able to create the movement and force of robot to behave as similar as the human’s massage. Therefore, the various massage points, such as the changing of hardness and softness of human skin muscle condition, can be appropriately controlled by using hybrid impedance and force control method. The impedance control is used to control the lateral position of massage on the human skin muscle. The perpendicular direction one, while force control is carried out for the vertical direction on human skin muscle. This paper also gives a modeling of human skin muscle through robot perception of impedance to decide the parameter of impedance controller. The control strategy using a hybrid impedance and force control to implement an adaptive control system is presented, when human condition is changed with soft and hard skin muscle. For the robot fingertip force, PI servo compensation is used together with impedance control. Based on using both impedance and PI servo compensation, the effectiveness of the massage position and force control is shown through actual massage motion experiments.

Robotic polishing is crucial for achieving superior surface finishes in manufacturing. However, precise force control presents significant challenges, particularly for curved workpieces exhibiting time-varying stiffness. Traditional methods typically struggle to adapt to these dynamic conditions, often leading to inconsistent results and suboptimal surface quality. This study proposes an Adaptive Impedance Control based on Visual Guidance (AICVG) strategy for robotic polishing. This approach integrates real-time visual feedback for geometric perception and adaptive tool path generation with a fuzzy logic system that dynamically adjusts impedance parameters to account for unforeseen surface stiffness variations. Simulations and experimental validations conducted on a robotic platform demonstrate that the AICVG strategy significantly outperforms both traditional impedance control and conventional fuzzy logic-based adaptive impedance control. Specifically, it maintains force control errors within ±1.5 N under dynamic stiffness conditions and achieves a 60% reduction in workpiece surface roughness compared to the aforementioned alternative methods. This study presents a robust and precise control framework that significantly enhances the adaptability and efficacy of robotic polishing for complex geometries, thereby advancing automated solutions in high-precision manufacturing.

Focusing on the mechanical characteristics of robotic dexterous hand tea-picking, this paper takes the harvesting of the premium tea Huangshan Maofeng as an example and proposes an adaptive impedance control method for tea-picking dexterous hands based on the Whale Optimization Algorithm (WOA) and Kolmogorov-Arnold Network (KAN). Firstly, within the impedance control framework, a KAN neural network with cubic B-spline functions as activation functions is introduced. Subsequently, the WOA is applied to optimize the B-splines, enhancing the network´s nonlinear fitting and global optimization capabilities, thereby achieving dynamic mapping and real-time adjustment of impedance parameters to improve the accuracy of tea bud contact force-tracking. Finally, simulation results show that under working conditions such as stiffness mutation and dynamic changes in desired force, the proposed method reduces the overshoot by 14.2% compared to traditional fixed-parameter impedance control, while the steady-state error is reduced by 99.89%. Experiments on tea-picking using a dexterous hand equipped with tactile sensors show that at a 50Hz control frequency, the maximum overshoot is about 6%, further verifying the effectiveness of the proposed control algorithm.

With the rapid development of powered transtibial prostheses, the control of motorized actuators is a research hotspot. This study designs a double-loop impedance controller to improve the performance of the traditional controller. Including time-varying parameters, the designed controller is implemented on a novel elastic actuator. Under different loads of users’ weights, simulation tests are conducted in a working condition of walking, and the results show that the designed control system has outstanding performances in the control of an entire gait cycle. The error between output and reference of ankle joint position during walking is less than 0.013rad in each sample time. A position disturbance test demonstrates that the control also has a high tolerance of any position disturbance at the ankle joint. The double-loop impedance controller can replace traditional finite-state impedance control in the field of the powered transtibial prosthesis and has the potential to be applied in other relative fields.

Piezoelectric inkjet printing has important applications in high-precision manufacturing, but precise control of droplet quality faces challenges such as the nonlinear coupling between drive waveform parameters and droplet characteristics, as well as environmental disturbances. To address these issues, this paper proposes a closed-loop droplet control method based on a curriculum deep deterministic policy gradient (DDPG) algorithm, with the standard deviation of droplet volume and velocity as the main optimization objectives. To tackle the complex mapping relationships and boost training efficiency, we first built a high-fidelity simulation environment using a four-layer deep residual network trained on real-world industrial ink droplet data. Following that, we designed a dynamic weight-adaptive composite reward mechanism and integrated a curriculum learning strategy with the Deep Deterministic Policy Gradient (DDPG) algorithm. Relying on our self-developed droplet observation platform, we achieve closed-loop control of the standard deviation of droplet volume and velocity. Experimental results show that this method can precisely control the droplet volume within ±0.1 pl of the target value, while keeping the velocity standard deviation below 40 mm/s. The adjustment process is stable without severe oscillations, and the method demonstrates strong robustness against environmental disturbances such as ink supply pressure. This provides an efficient and practical solution for the automated control of droplet quality in piezoelectric inkjet printing.

Mitigating collision is a fundamental issue in contact problems, and is required to ensure the safety of a robotic cell. Research into the contact problem between robots and their environment is divided into two parts: one uses the environmental contact model and parameter estimation, the other uses the robot force control method. There are two main problems with this research method. One is that the two research levels are not effectively combined to form a complete solution for force control in practice. The other problem is that research on excessive contact force in the collision phase has not been studied in depth for force control. In this paper, a sensing-executing bionic system is proposed that combines environmental detection and robotic force control based on the way an ant functions. The bionic system clearly explains the process from environment detection to robot control, which can provide guidance when designing a new robot control system. An adaptive switching control algorithm is proposed to mitigate the collision force in the collision phase. From the simulation results, the collision force is significantly reduced due to the implementation of adaptive switching control. Finally, a new self-sensing device is designed which can be integrated into the robot control device. However, as there are no additional sensors or computational complexity in the system, the effectiveness of the circuit and superiority of the adaptive parameter update must be verified by experimentation.

To solve the problem of insufficient adjustment ability of traditional impedance control due to uncertainty of environmental parameters when the robot grinds castings, an adaptive variable impedance constant force control method for robot grinding with unknown environmental parameters is proposed. In the proposed method, first, self-impedance impedance control is used as inner loop control to realize fast tracking response of actual trajectory to reference trajectory and reduce overshoot of grinding force; secondly, fuzzy adaptive variable impedance control is used as outer loop control to realize real-time prediction of unknown environmental parameters and real-time adjustment of impedance parameters; finally, the constant force control of robot grinding in unknown environment is realized by combining inner and outer loop control. In order to verify the effectiveness of the proposed method, the simulation model of the algorithm is constructed based on the MATLAB-SIMULINK simulation platform for simulation analysis, and experimental verification is carried out on the casting surface defects robot grinding platform. The simulation and experimental results show that the algorithm can control the grinding force well when grinding unknown environmental parameters, the force control accuracy is within ±1N, the average grinding force error of casting fin grinding is about 7.59%, the average grinding force error of casting riser grinding is about 5.31%, and the contour accuracy of casting surface defects is good after grinding by force control, and the surface quality after grinding is improved.

PurposeThis paper aims to present a pose correction method based on integrated virtual impedance control for avoiding collision and reducing impact.Design/methodology/approachThe authors first constructed the artificial potential field (APF) considering the geometric characteristics of the end-effector. The characteristics of the proposed field were analyzed considering the position and orientation misalignment. Then, an integrated virtual impedance control was proposed by adding resultant virtual repulsive force into traditional impedance control. Finally, the authors modified a correction trajectory for avoiding collision and reducing impact with virtual force and contact force.FindingsThe APF the authors constructed can get rid of a local minimum. Comparing with linear correction, this method is able to avoid collision effectively. When the capturing target has intrinsic estimation error, the pose correction can ensure smooth transitions among different stages.Practical implicationsThis method can be implemented on a manipulator with inner position control. It can be applied to an industrial robot with applications on robotic assembly for achieving a softer and smoother process. The method can also be expanded to the kind of claw-shaped end-effectors for capturing target.Originality valueAs the authors know, it is the first time that the characteristics of the end-effector are considered for avoiding collision in capturing application. The proposed integrated virtual impedance control can provide smooth transitions among different stages without switching different force/position controllers.

PurposeThis study aims to propose a force control algorithm based on neural networks, which enables a robot to follow a changing reference force trajectory when in contact with human skin while maintaining a stable tracking force.Design/methodology/approachAiming at the challenge of robots having difficulty tracking changing force trajectories in skin contact scenarios, a single neuron algorithm adaptive proportional – integral – derivative online compensation is used based on traditional impedance control. At the same time, to better adapt to changes in the skin contact environment, a gated recurrent unit (GRU) network is used to model and predict skin elasticity coefficients, thus adjusting to the uncertainty of skin environments.FindingsIn two robot–skin interaction experiments, compared with the traditional impedance control and robot force control algorithm based on the radial basis function model and iterative algorithm, the maximum absolute force error, the average absolute force error and the standard deviation of the force error are all decreased.Research limitations/implicationsAs the training process of the GRU network is currently conducted offline, the focus in the subsequent phase is to refine the network to facilitate real-time computation of the algorithm.Practical implicationsThis algorithm can be applied to robot massage, robot B-ultrasound and other robot-assisted treatment scenarios.Originality/valueAs the proposed approach obtains effective force tracking during robot–skin contact and is verified by the experiment, this approach can be used in robot–skin contact scenarios to enhance the accuracy of force application by a robot.

Application of μ theory in compliant force control system is studied. Compliant force control strategy is developed based on the inner loop position control of 6-DOF parallel robot in order to simulate pushing and pulling procedure of forcible alignment of space docking. The cascade structure of "out-force-loop containing inner-position-loop" is depicted, and the method is set forth using classical control strategy to realize compliant force control. Considering uncertainties such as parameter perturbations, model perturbations and external disturbances, etc, robust force controller is designed using μ synthesis theory. It is listed in detail that the selection method of weighting functions in robust compliant force control loop and the designing procedure of robust force controller. Robust stability and robust performance are compared between robust force controller and classical force controllers by μ analysis. Experiment of robust force controller and classical force controllers is presented. The results indicate that the designed robust force controller is of efficiency and superiority.

The research on autonomous driving based on deep reinforcement learning algorithms is a research hotspot. Traditional autonomous driving requires human involvement, and the autonomous driving algorithms based on supervised learning must be trained in advance using human experience. To deal with autonomous driving problems, this paper proposes an improved end-to-end deep deterministic policy gradient (DDPG) algorithm based on the convolutional block attention mechanism, and it is called multi-input attention prioritized deep deterministic policy gradient algorithm (MAPDDPG). Both the actor network and the critic network of the model have the same structure with symmetry. Meanwhile, the attention mechanism is introduced to help the vehicles focus on useful environmental information. The experiments are conducted in the open racing car simulator (TORCS)and the results of five experiment runs on the test tracks are averaged to obtain the final result. Compared with the state-of-the-art algorithm, the maximum reward increases from 62,207 to 116,347, and the average speed increases from 135 km/h to 193 km/h, while the number of success episodes to complete a circle increases from 96 to 147. Also, the variance of the distance from the vehicle to the center of the road is compared, and the result indicates that the variance of the DDPG is 0.6 m while that of the MAPDDPG is only 0.2 m. The above results indicate that the proposed MAPDDPG achieves excellent performance.

Second order matrix sensitivity analysis of force-based impedance control for leg hydraulic drive system

The electro-hydraulic hybrid drive, which synergistically combines the high power density of hydraulic systems with the precise controllability of electric actuators, presents an ideal solution for heavy-duty legged robots in unstructured, off-road environments. Addressing the challenge of significant impact forces induced by the substantial mass of heavy-duty humanoid robots during dynamic interactions with uncertain environments, this study investigates compliance control to enhance impact resistance. A kinematic model of the legged system, conceptualized as an equivalent serial mechanism, is established. This model delineates the forward/inverse kinematic relationships and the corresponding Jacobian matrices, mapping joint angles to foot-end poses and drive module outputs to joint angles. Building upon this foundation, an impedance control algorithm incorporating dynamic position error compensation is proposed. The efficacy of the proposed method is validated through both simulation and experimental studies. In terms of computational efficiency, the maximum computation times for forward and inverse kinematics are 0.2665 ms and 0.0377 ms, respectively. As the control frequency is set to 1 kHz, the computational efficiency can fully meet the real-time control requirements of the system. Results demonstrate that the controller achieves a foot-end stiffness control accuracy of over $\mathbf{9 0 \%}$. Furthermore, comparative experiments reveal that the proposed method reduces the peak landing impact force by 16.1% compared to traditional joint-based impedance control, thereby substantially improving the dynamic impact resilience of the legged system. This work provides a key contribution to the high-dynamic motion control of heavy-duty legged robots operating in complex terrains.

Objective As the core issue of ship navigation safety, ship collision avoidance always depends on the captain's personal status and judgment, which has certain potential safety hazards. In order to coordinate all ships (cruise ships, cargo ships, etc.) in key waters and predict their paths, it is necessary to establish an anti-collision early warning mechanism. Methods Using the deep deterministic policy gradient (DDPG) algorithm and the ship domain model, the electronic chart is used to simulate the ship's navigation path, and the improved strategy of DDPG algorithm based on the key learning of failure region and the improved parameters of ship domain model according to the characteristics of cruise ships are proposed, Improve the accuracy of route prediction and anti-collision. Results Using the improved DDPG algorithm and the improved ship domain model, compared with the previous algorithm, the accuracy of ship collision avoidance is improved from 84.9% to 89.7%, and the average error between the simulated route and the real route is reduced from 25.2 m to 21.4 m. Conclusion Through the ship collision avoidance path planning based on the improved DDPG algorithm and the improved ship domain model, the ship route supervision function in water area can be realized, and when the predicted route intersects with other ships, the dispatcher will be alerted to realize the anti-collision early warning mechanism.

This paper presents a parameter selection method of notch filters for suppressing mechanical resonances in industrial servo systems using the deep deterministic policy gradient (DDPG) algorithm. Several methods for tuning the notch filter parameters were studied such as the fast-Fourier-transform-based methods, extended-Kalman-filter-based methods and adaptive notch filter methods. However, these methods do not find the Q parameters of notch filters which play an important role in determining the system stability, and do not consider the cases in which multiple notch filters are required. Deep-Q-network-based method was developed to solve these problems, but the notch filter parameter tuning is limited to discrete action spaces. This paper develops a new parameter selection method of notch filters, using the DDPG algorithm. DDPG algorithm, which is a model-free and actor-critic algorithm using deep neural networks, is utilized for its capability to operate over continuous action spaces. Experiments are performed using an actual industrial servo system to demonstrate that the developed parameter selection method successfully finds the notch filter parameters to suppress the resonances of the system.