Optimal bipartite consensus for discrete-time multi-agent systems with event-triggered mechanism based on adaptive dynamic programming

This paper studies the problem of event-triggered optimal control (ETOC) for continuous-time nonlinear systems and proposes a novel event-triggering condition that enables designing ETOC methods directly based on the solution of the Hamilton-Jacobi-Bellman (HJB) equation. We provide formal performance guarantees by proving a predetermined upper bound. Moreover, we also prove the existence of a lower bound for interexecution time. For implementation purposes, an adaptive dynamic programming (ADP) method is developed to realize the ETOC using a critic neural network (NN) to approximate the value function of the HJB equation. Subsequently, we prove that semiglobal uniform ultimate boundedness can be guaranteed for states and NN weight errors with the ADP-based ETOC. Simulation results demonstrate the effectiveness of the developed ADP-based ETOC method.

In this article, the optimal control scheme for ice-storage air conditioning (IAC) system is solved via a data-based adaptive dynamic programming (ADP) method. It is the first time that ADP is employed to design a self-learning scheme, which obtains the optimal control policy of IAC system. First, based on the data of the temperature, irradiance, and cooling load in an actual project, a prediction model of cooling load is built by a three-layer neural network with the performance verification. Second, the operation of the IAC system is analyzed. Third, a data-based ADP method is designed to realize a self-learning optimal control for the IAC system. Then, numerical results show that using the data-based optimal control method can reduce the operation costs. Finally, the comparison results show that the developed ADP method improves the system efficiency, minimizing the overall cost. Thus, the superiority of the developed algorithm is verified.

In this article, a novel data-based adaptive dynamic programming (ADP) method is presented to solve the optimal consensus tracking control problem for discrete-time (DT) multiagent systems (MASs) with multiple time delays. Necessary and sufficient conditions of the corresponding equivalent time-delay system are provided on the basis of the causal transformations. Benefitting from the construction of tracking error dynamics, the optimal tracking problem can be transformed into settling the Nash-equilibrium in the graphical game, which can be completed by solving the coupled Hamilton-Jacobi (HJ) equations. An error estimator is introduced to construct the tracking error of the MASs only using the input and output (I/O) data. Therefore, the designed data-based ADP algorithm can minimize the cost functions and ensure the consensus of MASs without the knowledge of system dynamics. Finally, a numerical example is given to demonstrate the effectiveness of the proposed method.

Event-triggered adaptive optimal tracking control for nonlinear stochastic systems with dynamic state constraints

This paper presents an optimal trajectory tracking control approach for wheeled mobile robots (WMRs) using the adaptive dynamic programming (ADP) method integrated with radial basis function neural network (RBFNN). Firstly, the kine-matic model and error dynamics of the wheeled mobile robots are derived, with the tracking error defined in a transformed coordinate system. Then, the control input is decomposed into feedforward and feedback components to track the reference trajectory. To circumvent the issue of solving the Hamilton-Jacobi-Bellman (HJB) equation, the critic learning approach based on RBFNN is employed to approximate the optimal value function. The modified critic network weight update rule is introduced, which combines stability indicators and concurrent learning terms. The feedback control strategy is formulated by combining the output of critic network with the error dynamics, ensuring convergence of tracking errors to the neighborhood of zero. Finally, the simulation experiments validate the effectiveness of the method.

This paper investigates the problem of event-based consensus of discretetime multi-agent systems with general linear dynamics subject to actuator saturation. Leader-following and leaderless semi-global consensus are achieved by a distributed event-trigger strategy for the multi-agent systems. In spite of the input saturation constraint, it is shown that feedback control updating is unnecessary until an event is triggered. The combinational-state variables and the Riccati equation are used to construct a Lyapunov function and to design the event-triggering conditions and the low-gain control laws. Numerical examples are provided to illustrate the theoretical results. Keywords: Event-triggered control, Input saturation, Riccati equation, Semi-global consensus.

This paper is concerned with the problem of active complementary control design for affine nonlinear control systems with actuator faults. The outage and loss of effectiveness fault cases are considered. In order to achieve the performance enhancement of the faulty control system, the complementary control scheme is designed in two steps. Firstly, a novel fault estimation scheme is developed. Then, by using the fault estimations to reconstruct the faulty system dynamics and introducing a cost function as the optimization objective, a nearly optimal complementary control is obtained online based on the adaptive dynamic programming (ADP) method. Unlike most of the previous ADP methods with the addition of a probing signal, new adaptive weight update laws are derived to guarantee the convergence of neural network weights and the stability of the closed-loop system, which strongly supports the online implementation of the ADP method. Finally, two simulation examples are given to illustrate the performance and effectiveness of the proposed method.

Optimal control of Markovian jump systems via a neural network-based ADP iterative algorithm

Benefitting from the technology of integral reinforcement learning, the nonzero sum (NZS) game for distributed parameter systems is effectively solved in this paper when the information of system dynamics are unavailable. The Karhunen-Loève decomposition (KLD) is employed to convert the partial differential equation (PDE) systems into high-order ordinary differential equation (ODE) systems. Moreover, the off-policy IRL technology is introduced to design the optimal strategies for the NZS game. To confirm that the presented algorithm will converge to the optimal value functions, the traditional adaptive dynamic programming (ADP) method is first discussed. Then, the equivalence between the traditional ADP method and the presented off-policy method is proved. For implementing the presented off-policy IRL method, actor and critic neural networks are utilized to approach the value functions and control strategies in the iteration process, individually. Finally, a numerical simulation is shown to illustrate the effectiveness of the proposal off-policy algorithm.

The proposed event-triggered control design uses the adaptive dynamic programming (ADP) technique to solve the infinite-horizon optimal control of nonlinear continuous time system in affine form with complete unknown system dynamics in a forward time and online manner. The approximation property of the neural network (NN) is used to estimate the system dynamics and the value function with event-based sampling of state vector. Subsequently the estimated values are used to design the near optimal control policy. In addition, the NN weights are updated as a jump at every trigger instant, hence aperiodic in nature, to save computation when compared to the traditional NN-based approaches. Further, the closed-loop dynamics are formulated as a nonlinear impulsive dynamical system and the extension of the Lyapunov technique is utilized to prove the locally ultimate boundedness of all the closed-loop signals by deriving an adaptive event-trigger condition. Nonetheless, a positive lower bound on the inter-event time is guaranteed to avoid accumulation point. Finally, the analytical design is evaluated by using an example.

This paper is concerned with a new data-driven zero-sum neuro-optimal control problem for continuous-time unknown nonlinear systems with disturbance. According to the input-output data of the nonlinear system, an effective recurrent neural network is introduced to reconstruct the dynamics of the nonlinear system. Considering the system disturbance as a control input, a two-player zero-sum optimal control problem is established. Adaptive dynamic programming (ADP) is developed to obtain the optimal control under the worst case of the disturbance. Three single-layer neural networks, including one critic and two action networks, are employed to approximate the performance index function, the optimal control law, and the disturbance, respectively, for facilitating the implementation of the ADP method. Convergence properties of the ADP method are developed to show that the system state will converge to a finite neighborhood of the equilibrium. The weight matrices of the critic and the two action networks are also convergent to finite neighborhoods of their optimal ones. Finally, the simulation results will show the effectiveness of the developed data-driven ADP methods.

ABSTRACTThis article has investigated the problem of data‐driven distributed optimal consensus control for discrete‐time multiagent systems (MASs) with unknown dynamics. A novel event‐triggered goal representation heuristic dynamic programming (GrHDP)‐based control approach has been proposed. Note that traditional adaptive dynamic programming (ADP) methods usually establish two neural networks (NNs), the critic‐actor NNs, to approximate the performance index function and control inputs, respectively. In comparison, in this work, a goal NN has been used to generate a new local internal reinforcement signal (IRS) that contains more information from the environment, and then, a three neural‐networks structure, namely the goal‐critic‐actor NNs, has been established for each agent. Furthermore, to alleviate the communication load and computation cost, the event‐triggered mechanism has been constructed to update the control input aperiodically. Using the Lyapunov stability theory, detailed stability analysis results for the closed‐loop MASs with event‐triggered controller have been shown for both cases of the triggering and non‐triggering instants. Meanwhile, the weight estimation errors of the goal‐critic‐actor NNs and the local neighborhood consensus tracking errors have been guaranteed to be all uniformly ultimately bounded (UUB). Lastly, simulation studies have shown the effectiveness and superiority of our proposed method over existing works.

In this paper, a learning-based predictive control (LPC) scheme is proposed for adaptive optimal control of discrete-time nonlinear systems under stochastic disturbances. The proposed LPC scheme is different from conventional model predictive control (MPC), which uses open-loop optimization or simplified closed-loop optimal control techniques in each horizon. In LPC, the control task in each horizon is formulated as a closed-loop nonlinear optimal control problem and a finite-horizon iterative reinforcement learning (RL) algorithm is developed to obtain the closed-loop optimal/suboptimal solutions. Therefore, in LPC, RL and adaptive dynamic programming (ADP) are used as a new class of closed-loop learning-based optimization techniques for nonlinear predictive control with stochastic disturbances. Moreover, LPC also decomposes the infinite-horizon optimal control problem in previous RL and ADP methods into a series of finite horizon problems, so that the computational costs are reduced and the learning efficiency can be improved. Convergence of the finite-horizon iterative RL algorithm in each prediction horizon and the Lyapunov stability of the closed-loop control system are proved. Moreover, by using successive policy updates between adjoint time horizons, LPC also has lower computational costs than conventional MPC which has independent optimization procedures between two different prediction horizons. Simulation results illustrate that compared with conventional nonlinear MPC as well as ADP, the proposed LPC scheme can obtain a better performance both in terms of policy optimality and computational efficiency.

This article investigates the distributed approximate optimal control problem for pursuit-evasion differential games (PEDGs) of multiagent systems (MASs). Initially, interactions between pursuer agents and the evader agents are formulated using a divide-and-conquer algebraic graph approach, where all agents desire to maintain cohesion with their teammates. Subsequently, a state event-triggered mechanism (ETM) is introduced to conserve communication resources. Meanwhile, a polymeric hierarchical sliding mode surface (HSMS) incorporating local neighbor errors is constructed such that the system response rate is improved. To enhance team coordination, a novel dynamic target allocation algorithm is designed to execute the rational allocation among pursuers. Furthermore, based on the adaptive dynamic programming (ADP) with a single-critic neural network (NN) architecture, the HSMS-based event-triggered optimal control policies are further designed via solving the coupling Hamilton-Jacobi-Bellman (HJB) equations. Finally, a simulation conducted in the representative two-pursuer-two-evader scenario is presented to validate the effectiveness of the proposed control scheme.

In this paper, we propose a new approach to address robust control problem of nonlinear systems with uncertainties based on an adaptive dynamic programming (ADP) algorithm. After reformulating the robust control problem as an optimal control problem, we propose a modified ADP method to solve the derived Hamilton-Jacobi-Bellman (HJB) equation, where the optimal cost function is approximated by online training a critic neural network (NN). Then the approximated optimal control action can be derived to guarantee the stability of the controlled system with uncertainties. The closed-loop system stability and convergence have been proved. A simulation example is provided to illustrate the effectiveness of the method.