The comparative dynamics of closed-loop controls for discounted infinite horizon optimal control problems

On chaos control of nonlinear fractional chaotic systems via a neural collocation optimization scheme and some applications

This article introduces a droop-free, approximate optimal feedback control strategy to optimally control distributed generators (DGs) in islanded dc microgrids (DCMGs). Each DG is modeled as a control affine dynamical system, and constrained input of each DG is designed to minimize the infinite horizon quadratic state cost and nonquadratic control effort. The approximate dynamic programming (ADP) method is employed to solve the infinite horizon optimal control problem by successive approximation of the value function via a linear in the parameter (LIP) neural network (NN). The NN weights are updated online by a reinforcement learning (RL)-based tuning algorithm, and the convergence of the unknown weights to a neighborhood of the optimal weights is guaranteed without the persistence of excitation (PE). Simulation and experimental results are presented to demonstrate the effectiveness and applicability of the proposed concept.

In this paper, we study an infinite horizon optimal control problem arising in economic dynamics. We establish convergence of approximate solutions to the golden-rule stock (turnpike) and show the existence of overtaking optimal solutions.

The Viability Kernel Algorithm for Computing Value Functions of Infinite Horizon Optimal Control Problems

Infinite horizon optimal control problem of mean-field backward stochastic delay differential equation under partial information

Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem

This studies the infinite horizon optimal control problem for a class of continuous‐time systems subjected to multiplicative noises and Markovian jumps by using a data‐driven policy iteration algorithm. The optimal control problem is equivalent to solve a stochastic coupled algebraic Riccatic equation (CARE). An off‐line iteration algorithm is first established to converge the solutions of the stochastic CARE, which is generalised from an implicit iterative algorithm. By applying subsystems transformation (ST) technique, the off‐line iterative algorithm is decoupled into N parallel Kleinman's iterative equations. To learn the solution of the stochastic CARE from N decomposed linear subsystems data, an ST‐based data‐driven policy iteration algorithm is proposed and the convergence is proved. Finally, a numerical example is given to illustrate the effectiveness and applicability of the proposed two iterative algorithms.

This paper proposes an optimal control algorithm for a polynomial system with a quadratic criterion over infinite horizon. The designed regulator gives a closed form solution to the infinite horizon optimal control problem for a polynomial system with a quadratic criterion. The obtained solution consists in a feedback control law obtained by solving a Riccati algebraic equation dependent on the state. Numerical simulations in the example show advantages of the developed algorithm.

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

In this paper, a learning-based predictive control (LPC) scheme is proposed for adaptive optimal control of discrete-time nonlinear systems under stochastic disturbances. The proposed LPC scheme is different from conventional model predictive control (MPC), which uses open-loop optimization or simplified closed-loop optimal control techniques in each horizon. In LPC, the control task in each horizon is formulated as a closed-loop nonlinear optimal control problem and a finite-horizon iterative reinforcement learning (RL) algorithm is developed to obtain the closed-loop optimal/suboptimal solutions. Therefore, in LPC, RL and adaptive dynamic programming (ADP) are used as a new class of closed-loop learning-based optimization techniques for nonlinear predictive control with stochastic disturbances. Moreover, LPC also decomposes the infinite-horizon optimal control problem in previous RL and ADP methods into a series of finite horizon problems, so that the computational costs are reduced and the learning efficiency can be improved. Convergence of the finite-horizon iterative RL algorithm in each prediction horizon and the Lyapunov stability of the closed-loop control system are proved. Moreover, by using successive policy updates between adjoint time horizons, LPC also has lower computational costs than conventional MPC which has independent optimization procedures between two different prediction horizons. Simulation results illustrate that compared with conventional nonlinear MPC as well as ADP, the proposed LPC scheme can obtain a better performance both in terms of policy optimality and computational efficiency.

The optimal control of a system whose states are governed by a nonlinear autonomous Volterra integrodifferential equation with unbounded time interval is considered. Specifically, it is assumed that the delay occurs only in the state variable. The results obtained extend those of Brock and Haurie [Math. Oper. Res., 1 (1976), pp. 337–346] and Leizarowitz [Math. Oper. Res., 10 (1985), pp. 450–461]. In particular, it is shown that (under appropriate hypotheses) catching-up optimal solutions asymptotically approach a unique optimal steady state, and thus enjoy the so-called “turnpike” property found in the economics literature. By combining this result with an associated optimal control problem, the desired existence result is obtained. Furthermore, it is remarked that, in addition to extending these earlier works to the time-delay case, the results presented below utilize convexity, seminormality, and growth hypotheses that in some cases are weaker than those encountered in the above-mentioned papers.

This paper introduces a framework for solving time-autonomous nonlinear infinite horizon optimal control problems, under the assumption that all minimizers satisfy Pontryagin’s necessary optimality conditions. In detail, we use methods from the field of symplectic geometry to analyze the eigenvalues of a Koopman operator that lifts Pontryagin’s differential equation into a suitably defined infinite dimensional symplectic space. This has the advantage that methods from the field of spectral analysis can be used to characterize globally optimal control laws. A numerical method for constructing optimal feedback laws for nonlinear systems proceeds by computing the eigenvalues and eigenvectors of a matrix that is obtained by projecting the Pontryagin–Koopman operator onto a finite dimensional space. We illustrate the effectiveness of this approach by computing accurate approximations of the optimal nonlinear feedback law for a Van der Pol control system, which cannot be stabilized by a linear control law.

We consider a family of star-shaped planar domains Ωϵ, made of N non intersecting semi-infinite strips of thickness ϵ and of a central region whose diameter is proportional to ϵ. As ϵ → 0, Ωϵ tends to a network 𝒢 made of half-lines sharing an endpoint O. We study infinite horizon optimal control problems in which the state is constrained to remain in . We prove that the value function tends to the solution of a Hamilton-Jacobi equation on 𝒢, with an effective transmission condition at O. The effective equation is linked to an optimal control problem.

In this article, a novel multistep look-ahead policy iteration with isoperimetric constraints (MLPIIC) method is developed to solve infinite horizon optimal control problems (OCPs) with isoperimetric constraints for discrete-time nonlinear systems. In order to overcome the difficulty that Bellman’s principle of optimality does not hold directly in OCPs with isoperimetric constraints, a method to approximate OCPs with isoperimetric constraints by OCPs with new constraints is developed. For the MLPIIC method initialized with an admissible control law, the convergence and optimality of the iterative value function and the feasibility of the iterative control law are proven. Utilizing the function approximator, the implementation of the MLPIIC method is described. Finally, simulation results are provided.

We consider the undiscounted infinite horizon optimal control problem under state constraints in the absence of convexity/concavity assumptions. Then the value function is, in general, nonsmooth. Using the tools of set-valued and nonsmooth analysis, the necessary optimality conditions and sensitivity relations are derived in such a framework. We also investigate relaxation theorems and uniqueness of solutions of the Hamilton–Jacobi–Bellman equation arising in this setting.