Necessary conditions for infinite horizon optimal control problems with state constraints

On chaos control of nonlinear fractional chaotic systems via a neural collocation optimization scheme and some applications

The Viability Kernel Algorithm for Computing Value Functions of Infinite Horizon Optimal Control Problems

We consider the undiscounted infinite horizon optimal control problem under state constraints in the absence of convexity/concavity assumptions. Then the value function is, in general, nonsmooth. Using the tools of set-valued and nonsmooth analysis, the necessary optimality conditions and sensitivity relations are derived in such a framework. We also investigate relaxation theorems and uniqueness of solutions of the Hamilton–Jacobi–Bellman equation arising in this setting.

Properties of the value function are examined in an infinite horizon optimal control problem with an unlimited integrand index appearing in the quality functional with a discount factor. Optimal control problems of such type describe solutions in models of economic growth. Necessary and sufficient conditions are derived to ensure that the value function satisfies the infinitesimal stability properties. It is proved that value function coincides with the minimax solution of the Hamilton–Jacobi equation. Description of the growth asymptotic behavior for the value function is provided for the logarithmic, power and exponential quality functionals and an example is given to illustrate construction of the value function in economic growth models.

Receding horizon control (RHC) or model predictive control (MPC) solves online a finite horizon open-loop optimal control problem repeatedly in an infinite horizon context and provides a suboptimal control solution. It has been widely used in industry. For continuous-time (CT) systems, two categories of RHC have been investigated in literature, namely instantaneous RHC and sampled-data RHC. This paper focuses on the sampled-data RHC for continuous linear time-invariant (LTI) systems. The usual way to ensure stability of RHC scheme is to impose some constraints on the terminal cost and/or the terminal state and/or the horizon length. A new RHC algorithm is proposed, which we call updated terminal cost receding horizon control (UTC-RHC). Compared with the standard RHC which uses the same terminal cost all the time, our algorithm updates the terminal cost at each sampling instant. The advantage of UTC-RHC is that the uniform exponential stability of the closed-loop system can be guaranteed without imposing the usual constraints on the terminal state or the terminal cost or the horizon size. Moreover, the UTC-RHC control gain approaches the optimal value associated with the infinite horizon optimal control problems.

Turnpike theorem for terminal functionals in infinite horizon optimal control problems

In this article, a novel multistep look-ahead policy iteration with isoperimetric constraints (MLPIIC) method is developed to solve infinite horizon optimal control problems (OCPs) with isoperimetric constraints for discrete-time nonlinear systems. In order to overcome the difficulty that Bellman’s principle of optimality does not hold directly in OCPs with isoperimetric constraints, a method to approximate OCPs with isoperimetric constraints by OCPs with new constraints is developed. For the MLPIIC method initialized with an admissible control law, the convergence and optimality of the iterative value function and the feasibility of the iterative control law are proven. Utilizing the function approximator, the implementation of the MLPIIC method is described. Finally, simulation results are provided.

Infinite horizon optimal control problem of mean-field backward stochastic delay differential equation under partial information

In this paper, technological interventions in a simple airshed model of emission propagation coupled with a social model of technology adoption are studied. An optimal control problem is introduced where a central planner aims to balance pollution concentration against intervention costs over an infinite time horizon. Interventions are classified into two types. Preventive interventions promote the adoption of low emission (green) technologies in the underlying population whereas dissipative interventions reduce the concentration of pre-existing pollutants in the air. Next, an infinite-horizon optimal control problem is set up and the notion of optimality is established. Applying Pontryagin's Minimum Principle, a sustainable equilibrium is derived and the existence of its stable manifold is explored. Imposition of green technology reduces both the steady-state pollutant concentration and dissipative Effort simultaneously but the planning variables induce a tradeoff between concentration and dissipative Effort. Lastly, the stable manifolds are simulated concluding with a discussion on policy implications for the system under consideration.

Direct numerical approximation of a continuous‐time infinite horizon control problem, requires to recast the model as a discrete‐time, finite‐horizon control model. The quality of the optimization results can be heavily degraded if the discretization process does not take into account features of the original model to be preserved. Restricting their attention to optimal growh problems with a steady state, Mercenier and Michel in [1] and [2], studied the conditions to be imposed for ensuring that discrete first‐order approximation models have the same steady states as the infinite‐horizon continuous‐times counterpart. Here we show that Mercenier and Michel scheme is a first order partitioned Runge‐Kutta method applied to the state‐costate differential system which arises from the Pontryagin maximum principle. The main consequence is that it is possible to consider high order schemes which generalize that algorithm by preserving the steady‐growth invariance of the solutions with respect to the discretization process. Numerical examples show the efficiency and accuracy of the proposed methods when applied to the classical Ramsey growth model.

We consider a family of star-shaped planar domains Ωϵ, made of N non intersecting semi-infinite strips of thickness ϵ and of a central region whose diameter is proportional to ϵ. As ϵ → 0, Ωϵ tends to a network 𝒢 made of half-lines sharing an endpoint O. We study infinite horizon optimal control problems in which the state is constrained to remain in . We prove that the value function tends to the solution of a Hamilton-Jacobi equation on 𝒢, with an effective transmission condition at O. The effective equation is linked to an optimal control problem.

This paper proposes an optimal control algorithm for a polynomial system with a quadratic criterion over infinite horizon. The designed regulator gives a closed form solution to the infinite horizon optimal control problem for a polynomial system with a quadratic criterion. The obtained solution consists in a feedback control law obtained by solving a Riccati algebraic equation dependent on the state. Numerical simulations in the example show advantages of the developed algorithm.

This studies the infinite horizon optimal control problem for a class of continuous‐time systems subjected to multiplicative noises and Markovian jumps by using a data‐driven policy iteration algorithm. The optimal control problem is equivalent to solve a stochastic coupled algebraic Riccatic equation (CARE). An off‐line iteration algorithm is first established to converge the solutions of the stochastic CARE, which is generalised from an implicit iterative algorithm. By applying subsystems transformation (ST) technique, the off‐line iterative algorithm is decoupled into N parallel Kleinman's iterative equations. To learn the solution of the stochastic CARE from N decomposed linear subsystems data, an ST‐based data‐driven policy iteration algorithm is proposed and the convergence is proved. Finally, a numerical example is given to illustrate the effectiveness and applicability of the proposed two iterative algorithms.

We consider a controlled state equation of parabolic type on the halfline (0,+∞) with boundary conditions of Dirichlet type in which the unknown is equal to the sum of the control and of a white noise in time. We study finite horizon and infinite horizon optimal control problem related by means of backward stochastic differential equations.

The comparative dynamics of closed-loop controls for discounted infinite horizon optimal control problems