Online adaptive dynamic programming algorithm for linear fixed-time impulse hybrid systems

In this paper, an on-line adaptive dynamic programming (ADP) algorithm is presented to solve the optimal control problem for continuous-time nonlinear systems with control constraints. First, a suitable non-quadratic performance function is introduced to confront control constraints, and then we present an on-line ADP algorithm in which dynamic programming techniques and neural networks are used to solve the optimal control problem for nonlinear systems with control constraints, which is completely different with the traditional ADP algorithm using off-line training, and the corresponding convergence proof is given. Finally, a simulation examples is given to demonstrate the convergence and feasibility of the proposed algorithm.

In this paper, an optimal tracking control approach based on adaptive dynamic programming (ADP) algorithm is proposed to solve the linear quadratic regulation (LQR) problems for unknown discrete-time systems in an online fashion. First, we convert the optimal tracking problem into designing infinite-horizon optimal regulator for the tracking error dynamics based on the system transformation. Then we expand the error state equation by the history data of control and state. The iterative ADP algorithm of policy iteration (PI) and value iteration (VI) are introduced to solve the value function of the controlled system. It is shown that the proposed ADP algorithm solves the LQR without requiring any knowledge of the system dynamics. The simulation results show the convergence and effectiveness of the proposed control scheme.

In this paper, a finite-horizon ε-optimal control for a class of nonlinear systems with state delay is proposed by Adaptive Dynamic Programming (ADP) algorithm. First of all, the performance index function is defined and the Hamilton-Jacobi-Bellman (HJB) equation is obtained for the problem, the convergence of the iterative algorithm is also presented. Then, ADP algorithm for finite-horizon optimal control is introduced with an ε-error bound so as to get the ε-optimal control, and BP neural network is used to implement ADP algorithm. At last, an example is given to demonstrate the effectiveness of the proposed algorithm.

This paper deals with optimal tracking control problems for a class of discrete-time nonlinear systems using a generalized policy iteration adaptive dynamic programming (ADP) algorithm. First, by system transformation, the optimal tracking control problem is transformed into an optimal regulation problem. Then the generalized policy iteration ADP algorithm is employed to obtain the optimal tracking controller with convergence and optimality analysis. The developed algorithm uses the idea of two iteration procedures to obtain the iterative tracking control laws and the iterative value functions. Three neural networks, including model network, critic network and action network, are used to implement the developed algorithm. At last, an simulation example is given to demonstrate the effectiveness of the developed method.

The convergence and stability of a value-iteration-based adaptive dynamic programming (ADP) algorithm are considered for discrete-time nonlinear systems accompanied by a discounted quadric performance index. More importantly than sufficing to achieve a good approximate structure, the iterative feedback control law must guarantee the closed-loop stability. Specifically, it is firstly proved that the iterative value function sequence will precisely converge to the optimum. Secondly, the necessary and sufficient condition of the optimal value function serving as a Lyapunov function is investigated. We prove that for the case of infinite horizon, there exists a finite horizon length of which the iterative feedback control law will provide stability, and this increases the practicability of the proposed value iteration algorithm. Neural networks (NNs) are employed to approximate the value functions and the optimal feedback control laws, and the approach allows the implementation of the algorithm without knowing the internal dynamics of the system. Finally, a simulation example is employed to demonstrate the effectiveness of the developed optimal control method.

This paper is concerned with a new iterative adaptive dynamic programming (ADP) algorithm to solve optimal control problems for infinite horizon discrete-time nonlinear systems using a numerical controller. The convergence conditions of the iterative ADP are developed considering the errors by the numerical controller which show that the iterative performance index functions can converge to the greatest lower bound of all performance indices within a finite error bound. Neural networks and digital computer are used to approximate the iterative performance index function and compute the numerically iterative control policy, respectively, for facilitating the implementation of the iterative ADP algorithm. Finally, a simulation example is given to illustrate the performance of the present method.

In this paper, a novel iterative adaptive dynamic programming (ADP) algorithm is developed to solve infinite horizon optimal control problems for discrete-time nonlinear systems. When the iterative control law and iterative performance index function in each iteration cannot be accurately obtained, it is shown that the iterative controls can make the performance index function converge to within a finite error bound of the optimal performance index function. Stability properties are presented to show that the system can be stabilized under the iterative control law which makes the present iterative ADP algorithm feasible for implementation both on-line and off-line. Neural networks are used to approximate the iterative performance index function and compute the iterative control policy, respectively, to implement the iterative ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the present method.

In this paper, an event-triggered adaptive dynamic programming (ADP) algorithm is developed to solve the two-player zero-sum game problem of continuous-time nonlinear systems. First, a critic neural network is employed to approximate the optimal value function. Then, an event-triggered ADP method is proposed, which guarantees the stability of the closed-loop system. The developed method can save the amount of computation as the control law and disturbance law that update only when the pre-designed triggering condition is violated. Finally, its effectiveness is verified through simulation results.

An optimal control strategy based on adaptive dynamic programming (ADP) algorithm is proposed for the typical optimal control problem of missile interceptor with control constraints. This method is based on ADP algorithm control architecture and using BP neural network has great advantages in approximating nonlinear systems to approach the cost function and optimal control strategy by on-line iteration. Experimental results show that: Compared with the traditional control policy, this controller improves the control accuracy of missile interception process, and the system robustness is also significantly enhanced, and the control performance is better.

This paper presents a functional model predictive control (MPC) approach based on an adaptive dynamic programming (ADP) algorithm with the abilities of handling control constraints and disturbances for the optimal control of nonlinear discrete-time systems. In the proposed ADP-based nonlinear MPC (NMPC) structure, a neural-network-based identification is established first to reconstruct the unknown system dynamics. Then, the actor-critic scheme is adopted with a critic network to estimate the index performance function and an action network to approximate the optimal control input. Meanwhile, as the MPC strategy can effectively determine the current control by solving a finite horizon open-loop optimal control problem, in the proposed algorithm, the infinite horizon is decomposed into a series of finite horizons to obtain the optimal control. In each finite horizon, the finite ADP algorithm solves the optimal control problem subject to the terminal constraint, the control constraint, and the disturbance. The uniform ultimate boundedness of the closed-loop system is verified by the Lyapunov approach. Finally, the ADP-based NMPC is conducted on two different cases and the simulation results demonstrate the quick response and strong robustness of the proposed method.

In this paper, an innovative adaptive dynamic programming (ADP) algorithm with fast convergence speed is designed for the optimal containment control problem of discrete‐time linear multi‐agent systems. Precisely, a quadratic input energy cost function, including local containment error information and actuator information in the neighborhood, is designed for each follower. Solving the stationary condition of the cost function, the optimal containment controllers are obtained. Traditional ADP methods use actor–critic neural networks to approximate optimal costs and control strategies, it is time‐consuming to solve large‐scale multi‐agent problems due to the computational complexity of neural networks. In order to seek faster convergence speed of optimal containment control without knowing the model information, the fast ADP algorithm framework is designed, it is proved theoretically that the convergence speed is determined by some configurable parameters, and the whale optimization algorithm is employed to globally optimize the parameters of given spaces to derive the optimal configuration. Finally, numerical simulation results are given to verify the effectiveness of the designed algorithm.

Online learning is an important property of adaptive dynamic programming (ADP). Online observations contain plentiful dynamics information, and ADP algorithms can utilize them to learn the optimal control policy. This paper reviews the research of online ADP algorithms for the optimal control of continuous-time systems. With the intensive study, ADP has been developed towards model free and data efficient. After separately introducing the algorithms, we compare their performance on the same problem. This paper is desired to provide a comprehensive understanding of continuous-time online ADP algorithms.

In this paper, a control method based on periodic sliding mode technology and adaptive dynamic programming algorithm is proposed for the attitude dynamics of hypersonic flight vehicle. The method takes into account a case that there is unknown control direction in a single channel. First, under the premise that the disturbance observers estimate the coupling and other disturbances, the attitude dynamics are decomposed into mathematical models of pitch channel and yaw-roll channel, respectively. Then, a periodic sliding mode control law with both stability and rapidity is designed for the unknown control direction in pitch channel. In the yaw-roll channel, an optimal attitude control law satisfying cost function is designed based on adaptive dynamic programming algorithm. What’s more, the closed-loop system is proved to be stable by Lyapunov stability theorem. Finally, the simulation results show that the attitude angle can track the command stably, which proves the effectiveness of the control scheme. Compared with the backstepping control scheme, the simulation results show that the proposed scheme can effectively avoid the divergence of the system because of the control direction switch, and the system have better dynamic performance and robustness.

In this article, a new data‐based adaptive dynamic programming algorithm is proposed to solve the optimal control policy for discrete‐time systems with uncertainties. Firstly, for uncertain systems, the corresponding Hamiltonian function is designed, and then the robust adaptive dynamic programming algorithm is obtained. Next, by using the input and output data of the system, the data‐based Bellman equation is constructed, and the data‐based robust adaptive dynamic programming algorithm is derived, which does not require the accurate model of the system. Finally, a simulation example shows the effectiveness of the proposed algorithm.

Optimal control of Markovian jump systems via a neural network-based ADP iterative algorithm