Event-Triggered Optimal Control With Performance Guarantees Using Adaptive Dynamic Programming.

This article investigates the adaptive fuzzy distributed optimal event-triggered (ET) control problem for high-order nonlinear multiagent (NMA) system under weight-unbalanced directed graph. An optimal signal generator with ET mechanism is first established to estimate optimization point and optimal network channel utilization between agents and neighbors. Second, since the ET mechanism is introduced in optimal signal generator, the first-order derivative of optimal virtual signals is not continuous at the trigger instant, a high-order filter is established to estimate high-order derivatives of optimal virtual signals. Based on the optimal signal generator with ET mechanism and high-order filter, an adaptive fuzzy distributed optimal ET control scheme is proposed by backstepping control design technology. It is shown that controlled NMA system is asymptotically stable and the global cost function is minimized. Finally, we apply the proposed distributed optimal ET control approach to marine surface vehicle systems, the simulation and comparison results verify its effectiveness.

The precise control of the spatiotemporal process in a roller kiln is crucial in the production of Ni-Co-Mn layered cathode material of lithium-ion batteries. Since the product is extremely sensitive to temperature distribution, temperature field control is of great significance. In this article, an event-triggered optimal control (ETOC) method with input constraints for the temperature field is proposed, which takes up an important position in reducing the communication and computation costs. A nonquadratic cost function is adopted to describe the system performance with input constraints. First, we present the problem description of the temperature field event-triggered control, where this field is described by a partial differential equation (PDE). Then, the event-triggered condition is designed according to the information of system states and control inputs. On this basis, a framework of the event-triggered adaptive dynamic programming (ETADP) method that is based on the model reduction technology is proposed for the PDE system. A critic network is used to approach the optimal performance index by a neural network (NN) together with that an actor network is used to optimize the control strategy. Furthermore, an upper bound of the performance index and a lower bound of interexecution times, as well as the stabilities of the impulsive dynamic system and the closed-loop PDE system, are also proved. Simulation verification demonstrates the effectiveness of the proposed method.

Optimal bipartite consensus for discrete-time multi-agent systems with event-triggered mechanism based on adaptive dynamic programming

Event-triggered adaptive optimal tracking control for nonlinear stochastic systems with dynamic state constraints

Event-triggered explorized IRL-based decentralized fault-tolerant guaranteed cost control for interconnected systems against actuator failures

In this paper, a learning-based predictive control (LPC) scheme is proposed for adaptive optimal control of discrete-time nonlinear systems under stochastic disturbances. The proposed LPC scheme is different from conventional model predictive control (MPC), which uses open-loop optimization or simplified closed-loop optimal control techniques in each horizon. In LPC, the control task in each horizon is formulated as a closed-loop nonlinear optimal control problem and a finite-horizon iterative reinforcement learning (RL) algorithm is developed to obtain the closed-loop optimal/suboptimal solutions. Therefore, in LPC, RL and adaptive dynamic programming (ADP) are used as a new class of closed-loop learning-based optimization techniques for nonlinear predictive control with stochastic disturbances. Moreover, LPC also decomposes the infinite-horizon optimal control problem in previous RL and ADP methods into a series of finite horizon problems, so that the computational costs are reduced and the learning efficiency can be improved. Convergence of the finite-horizon iterative RL algorithm in each prediction horizon and the Lyapunov stability of the closed-loop control system are proved. Moreover, by using successive policy updates between adjoint time horizons, LPC also has lower computational costs than conventional MPC which has independent optimization procedures between two different prediction horizons. Simulation results illustrate that compared with conventional nonlinear MPC as well as ADP, the proposed LPC scheme can obtain a better performance both in terms of policy optimality and computational efficiency.

This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems. Unlike existing optimal state feedback control, the control input of the optimal parallel control is introduced into the feedback system. However, due to the introduction of control input into the feedback system, the optimal state feedback control methods can not be applied directly. To address this problem, an augmented system and an augmented performance index function are proposed firstly. Thus, the general nonlinear system is transformed into an affine nonlinear system. The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically. It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function. Moreover, an adaptive dynamic programming ( ADP ) technique is utilized to implement the optimal parallel tracking control using a critic neural network ( NN ) to approximate the value function online. The stability analysis of the closed-loop system is performed using the Lyapunov theory, and the tracking error and NN weights errors are uniformly ultimately bounded ( UUB ) . Also, the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals. Finally, the effectiveness of the developed optimal parallel control method is verified in two cases.

This paper presents a novel method of global adaptive dynamic programming (ADP) for the adaptive optimal control of nonlinear polynomial systems. The strategy consists of relaxing the problem of solving the Hamilton-Jacobi-Bellman (HJB) equation to an optimization problem, which is solved via a new policy iteration method. The proposed method distinguishes from previously known nonlinear ADP methods in that the neural network approximation is avoided, giving rise to significant computational improvement. Instead of semiglobally or locally stabilizing, the resultant control policy is globally stabilizing for a general class of nonlinear polynomial systems. Furthermore, in the absence of the a priori knowledge of the system dynamics, an online learning method is devised to implement the proposed policy iteration technique by generalizing the current ADP theory. Finally, three numerical examples are provided to validate the effectiveness of the proposed method.

Benefitting from the technology of integral reinforcement learning, the nonzero sum (NZS) game for distributed parameter systems is effectively solved in this paper when the information of system dynamics are unavailable. The Karhunen-Loève decomposition (KLD) is employed to convert the partial differential equation (PDE) systems into high-order ordinary differential equation (ODE) systems. Moreover, the off-policy IRL technology is introduced to design the optimal strategies for the NZS game. To confirm that the presented algorithm will converge to the optimal value functions, the traditional adaptive dynamic programming (ADP) method is first discussed. Then, the equivalence between the traditional ADP method and the presented off-policy method is proved. For implementing the presented off-policy IRL method, actor and critic neural networks are utilized to approach the value functions and control strategies in the iteration process, individually. Finally, a numerical simulation is shown to illustrate the effectiveness of the proposal off-policy algorithm.

In this paper, the infinite horizon optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems is investigated using neural-network-based online solution of Hamilton-Jacobi-Bellman (HJB) equation. By establishing an appropriate bounded function and defining a modified cost function, the optimal robust guaranteed cost control problem is transformed into an optimal control problem. It can be observed that the optimal cost function of the nominal system is nothing but the optimal guaranteed cost of the original uncertain system. A critic neural network is constructed to facilitate the solution of the modified HJB equation corresponding to the nominal system. More importantly, an additional stabilizing term is introduced for helping to verify the stability, which reinforces the updating process of the weight vector and reduces the requirement of an initial stabilizing control. The uniform ultimate boundedness of the closed-loop system is analyzed by using the Lyapunov approach as well. Two simulation examples are provided to verify the effectiveness of the present control approach.

This article studies the multi-H∞ controls for the input-interference nonlinear systems via adaptive dynamic programming (ADP) method, which allows for multiple inputs to have the individual selfish component of the strategy to resist weighted interference. In this line, the ADP scheme is used to learn the Nash-optimization solutions of the input-interference nonlinear system such that multiple H∞ performance indices can reach the defined Nash equilibrium. First, the input-interference nonlinear system is given and the Nash equilibrium is defined. An adaptive neural network (NN) observer is introduced to identify the input-interference nonlinear dynamics. Then, the critic NNs are used to learn the multiple H∞ performance indices. A novel adaptive law is designed to update the critic NN weights by minimizing the Hamiltonian-Jacobi-Isaacs (HJI) equation, which can be used to directly calculate the multi-H∞ controls effectively by using input-output data such that the actor structure is avoided. Moreover, the control system stability and updated parameter convergence are proved. Finally, two numerical examples are simulated to verify the proposed ADP scheme for the input-interference nonlinear system.

This paper proposes an adaptive critic design-based event-triggered optimal control method for input-constrained continuous-time nonlinear systems. Adaptive critic design is a special framework of adaptive dynamic programming that approximates the value function by a critic neural network and derives the approximate optimal control policy through analytical methods. The proposed adaptive critic design considers the control input constraints by introducing a non-quadratic cost function and employs an event-triggered mechanism to reduce the number of controller executions. Unlike the existing event-triggered adaptive critic design, this paper proposes a novel finite-time adaptive law based on regression filtering scheme. The adaptive law utilizes the error information of the network weights to ensure fast convergence to the optimal control law under the event-triggered mechanism, which improves the real-time performance of the system. Additionally, explicit bounds for each parameter in the compact set and the specific convergence time estimates are provided in the convergence analysis. Finally, the effectiveness and practicality of the proposed method for real-time online applications are validated through two simulation examples.

The proposed event-triggered control design uses the adaptive dynamic programming (ADP) technique to solve the infinite-horizon optimal control of nonlinear continuous time system in affine form with complete unknown system dynamics in a forward time and online manner. The approximation property of the neural network (NN) is used to estimate the system dynamics and the value function with event-based sampling of state vector. Subsequently the estimated values are used to design the near optimal control policy. In addition, the NN weights are updated as a jump at every trigger instant, hence aperiodic in nature, to save computation when compared to the traditional NN-based approaches. Further, the closed-loop dynamics are formulated as a nonlinear impulsive dynamical system and the extension of the Lyapunov technique is utilized to prove the locally ultimate boundedness of all the closed-loop signals by deriving an adaptive event-trigger condition. Nonetheless, a positive lower bound on the inter-event time is guaranteed to avoid accumulation point. Finally, the analytical design is evaluated by using an example.

In this article, the optimal control scheme for ice-storage air conditioning (IAC) system is solved via a data-based adaptive dynamic programming (ADP) method. It is the first time that ADP is employed to design a self-learning scheme, which obtains the optimal control policy of IAC system. First, based on the data of the temperature, irradiance, and cooling load in an actual project, a prediction model of cooling load is built by a three-layer neural network with the performance verification. Second, the operation of the IAC system is analyzed. Third, a data-based ADP method is designed to realize a self-learning optimal control for the IAC system. Then, numerical results show that using the data-based optimal control method can reduce the operation costs. Finally, the comparison results show that the developed ADP method improves the system efficiency, minimizing the overall cost. Thus, the superiority of the developed algorithm is verified.

This paper presents a functional model predictive control (MPC) approach based on an adaptive dynamic programming (ADP) algorithm with the abilities of handling control constraints and disturbances for the optimal control of nonlinear discrete-time systems. In the proposed ADP-based nonlinear MPC (NMPC) structure, a neural-network-based identification is established first to reconstruct the unknown system dynamics. Then, the actor-critic scheme is adopted with a critic network to estimate the index performance function and an action network to approximate the optimal control input. Meanwhile, as the MPC strategy can effectively determine the current control by solving a finite horizon open-loop optimal control problem, in the proposed algorithm, the infinite horizon is decomposed into a series of finite horizons to obtain the optimal control. In each finite horizon, the finite ADP algorithm solves the optimal control problem subject to the terminal constraint, the control constraint, and the disturbance. The uniform ultimate boundedness of the closed-loop system is verified by the Lyapunov approach. Finally, the ADP-based NMPC is conducted on two different cases and the simulation results demonstrate the quick response and strong robustness of the proposed method.