Optimal Bipartite Tracking Control for Heterogeneous Systems Under DoS Attacks

Learning-based optimal cooperative control using edge-based triggering strategy

This paper investigates the optimal couple-group consensus control (OCGCC) for heterogeneous multi-agent systems (HeMASs) with completely unknown dynamics. The agents in HeMASs are divided into two groups according to order differences. Meanwhile, heterogeneous systems are transformed into homogeneous ones by adding virtual velocities. Then, a novel data-driven distributed control protocol for HeMASs is proposed based on policy gradient reinforcement learning (RL). The proposed algorithm is implemented asynchronously, and is specifically designed to address the issue of computational imbalance caused by individual differences among participants. It achieves this by constructing an actor-critic (AC) framework. The system’s learning efficacy is optimized using offline data sets. The convergence and stability are ensured by applying functional analysis and the Lyapunov stability theory. Finally, the effectiveness of the proposed algorithm is confirmed by various simulation examples.

This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems. Unlike existing optimal state feedback control, the control input of the optimal parallel control is introduced into the feedback system. However, due to the introduction of control input into the feedback system, the optimal state feedback control methods can not be applied directly. To address this problem, an augmented system and an augmented performance index function are proposed firstly. Thus, the general nonlinear system is transformed into an affine nonlinear system. The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically. It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function. Moreover, an adaptive dynamic programming ( ADP ) technique is utilized to implement the optimal parallel tracking control using a critic neural network ( NN ) to approximate the value function online. The stability analysis of the closed-loop system is performed using the Lyapunov theory, and the tracking error and NN weights errors are uniformly ultimately bounded ( UUB ) . Also, the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals. Finally, the effectiveness of the developed optimal parallel control method is verified in two cases.

The optimal output tracking control problem for nonlinear large-scale dynamic systems is considered. A series-based approximate approach for designing the optimal controller is proposed with respect to quadratic performance indexes. By using the approach, the high order, coupling, nonlinear two-point boundary value problem is transformed into a sequence of linear decoupling two-point boundary value problems. By intercepting the frontal finite terms of the solution series, we obtained a suboptimal tracking control law. A simulation example illustrates the efficiency of the optimal output tracking control law designed.

The optimal output tracking control(OOTC) problem is considered for continuous linear time-invariable systems, which include control delays and external sinusoidal disturbances. We first transform the original system into a new system without time-delay, then construct a new state vector, which containing a external system’s vector. The external system described the aim which we want to track. Following the law of internal model principle, we transform the original and external systems into a new augmented system. Finally we get an optimal control law according to the optimal performance index, and prove the method is effective by emulation.

In this paper, we study optimal LQG tracking control problems for linear continuous-time hybrid systems. We adopt an approach to prepare some candidates of mode distributions and minimize value functions at each time for the systems and performance indices. By the approach, system states and mode distributions are simultaneously optimized so that the performance indices are minimized for available information of reference signals. We consider optimization problems for averaged systems and averaged performance indices throughout mode distributions. Finally we give numerical examples and verify that we can obtain both optimal tracking performance and optimal candidate of mode distributions by the algorithm presented in this paper.

Optimal tracking control with zero steady-state error for time-delay systems with sinusoidal disturbances

With the increasing deployment of Unmanned Aerial Vehicle (UAV) swarms in uncertain and dynamically changing environments, optimal cooperative control has become essential for ensuring robust and efficient system coordination. To this end, this paper designs a data-driven optimal bipartite containment tracking control scheme for multi-UAV systems under compound uncertainties. A novel Dynamic Iteration Regulation Strategy (DIRS) is proposed, which enables real-time adjustment of the learning iteration step according to the task-specific demands. Compared with conventional fixed-step data-driven algorithms, the DIRS provides greater flexibility and computational efficiency, allowing for better trade-offs between the performance and cost. First, the optimal bipartite containment tracking control problem is formulated, and the associated coupled Hamilton–Jacobi–Bellman (HJB) equations are established. Then, a data-driven iterative policy learning algorithm equipped with the DIRS is developed to solve the optimal control law online. The stability and convergence of the proposed control scheme are rigorously analyzed. Furthermore, the control law is approximated via the neural network framework without requiring full knowledge of the model. Finally, numerical simulations are provided to demonstrate the effectiveness and robustness of the proposed DIRS-based optimal containment tracking scheme for multi-UAV systems, which can reduce the number of iterations by 88.27% compared to that for the conventional algorithm.

This article addresses the optimal tracking control problem with prescribed performance for uncertain nonlinear systems subject to input constraint and unknown disturbances. First, a fixed‐time monotonic convergence function is introduced to restrain tracking error, and a nonlinear mapping technique is employed to transform the constrained error into an unconstrained variable, then the fixed‐time output tracking issue is boiled down to the boundedness problem of the transformed variable. With the aid of a nonquadratic cost function, the input constraint is encoded into the optimization problem. To solve the unknown disturbances, an auxiliary system and an auxiliary disturbance policy are constructed, and the optimal control problem is formulated as a two‐player zero‐sum game. Moreover, a Hamilton–Jacobi–Isaacs (HJI) equation associated with this nonquadratic zero‐sum game is established to give the optimal control and the worst‐case disturbance policy solution. Subsequently, to avoid using knowledge of the system dynamics, three neural network approximators, namely, actor, critic, and disturbance, which are tuned online and simultaneously for approximating the solution of HJI, are constructed based on the integral reinforcement learning algorithm. Theoretical analysis shows that the reconstructed error system states and the weight estimation errors are semi‐globally uniformly ultimately bounded. Finally, the simulation study further tests the availability of the proposed control strategy.

Deep reinforcement learning based finite-horizon optimal tracking control for nonlinear system

For the optimal tracking control problem of affine nonlinear systems, a general value iteration algorithm based on adaptive dynamic programming is proposed in this paper. By system transformation, the optimal tracking problem is converted into the optimal regulating problem for the tracking error dynamics. Then, general value iteration algorithm is developed to obtain the optimal control with convergence analysis. Considering the advantages of echo state network, we use three echo state networks with levenberg-Marquardt (LM) adjusting algorithm to approximate the system, the cost function and the control law. A simulation example is given to demonstrate the effectiveness of the presented scheme.

In this paper, a novel distributed H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> optimal tracking control scheme is designed for a class of physically interconnected large-scale nonlinear systems in the presence of strict-feedback form, external disturbance and saturating actuators. First, by designing feedforward control, the distributed H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> optimal tracking control problem of a physically interconnected large-scale system is transformed into equivalent control of a decoupled multiagent system. Subsequently, a feedback control algorithm is designed to learn the optimal control input and the worst-case disturbance policy. The algorithm guarantees that the function approximation error and the distributed tracking error are uniformly ultimately bounded while the cost function converges to the bounded L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -gain optimal value. Finally, the effectiveness of the proposed scheme is demonstrated by simulation results of distributed control for the mobile multirobot system.

This paper introduces an optimal tracking controller for robot manipulators with asymmetrically saturated torques and partially - unknown dynamics based on a reinforcement learning method using a neural network. Firstly, the feedforward control inputs are designed based on the backstepping technique to convert the tracking control problem into the optimal tracking control problem. Secondly, a cost function of the system with asymmetrically saturated input is defined, and the constrained Hamilton-Jacobi-Bellman equation is built, which is solved by the online reinforcement learning algorithm using only a single neural network. Then, the asymmetric saturation optimal control rule is determined. Additionally, the concurrent learning technique is used to relax the demand for the persistence of excitation conditions. The built algorithm ensures that the closed-loop system is asymptotically stable, the approximation error is uniformly ultimately bounded (UUB), and the cost function converges to the near-optimal value. Finally, the effectiveness of the proposed algorithm is shown through comparative simulations.

Distributed Optimal Output Feedback Control of Heterogeneous Multi-agent Systems under a Directed Graph

This paper develops an approximate design procedure of optimal tracking controllers for a class of nonlinear systems with persistent disturbances in the infinite time domain. By using the successive approximation approach (SAA), the two-point boundary value (TPBV) problem, which is derived from the optimal control theory, is transformed into solving a sequence of linear TPBV problems. The solution sequence of the linear TPBV problems uniformly converges to the solution of the original optimal tracking control (OTC) problem. A feedforward and feedback optimal tracking control (FFOTC) law is derived from a Riccati equation, matrix equations and a sequence of adjoint vectors. The existence and uniqueness conditions of the FFOTC are given. At last, a design algorithm of solving the approximate FFOTC law is proposed, and simulation results demonstrate the validity of the algorithm.