The Viability Kernel Algorithm for Computing Value Functions of Infinite Horizon Optimal Control Problems

On chaos control of nonlinear fractional chaotic systems via a neural collocation optimization scheme and some applications

Properties of the value function are examined in an infinite horizon optimal control problem with an unlimited integrand index appearing in the quality functional with a discount factor. Optimal control problems of such type describe solutions in models of economic growth. Necessary and sufficient conditions are derived to ensure that the value function satisfies the infinitesimal stability properties. It is proved that value function coincides with the minimax solution of the Hamilton–Jacobi equation. Description of the growth asymptotic behavior for the value function is provided for the logarithmic, power and exponential quality functionals and an example is given to illustrate construction of the value function in economic growth models.

In this article, a novel multistep look-ahead policy iteration with isoperimetric constraints (MLPIIC) method is developed to solve infinite horizon optimal control problems (OCPs) with isoperimetric constraints for discrete-time nonlinear systems. In order to overcome the difficulty that Bellman’s principle of optimality does not hold directly in OCPs with isoperimetric constraints, a method to approximate OCPs with isoperimetric constraints by OCPs with new constraints is developed. For the MLPIIC method initialized with an admissible control law, the convergence and optimality of the iterative value function and the feasibility of the iterative control law are proven. Utilizing the function approximator, the implementation of the MLPIIC method is described. Finally, simulation results are provided.

We consider a family of star-shaped planar domains Ωϵ, made of N non intersecting semi-infinite strips of thickness ϵ and of a central region whose diameter is proportional to ϵ. As ϵ → 0, Ωϵ tends to a network 𝒢 made of half-lines sharing an endpoint O. We study infinite horizon optimal control problems in which the state is constrained to remain in . We prove that the value function tends to the solution of a Hamilton-Jacobi equation on 𝒢, with an effective transmission condition at O. The effective equation is linked to an optimal control problem.

Partial and full sensitivity relations are obtained for nonauto-nomous optimal control problems with infinite horizon subject to state constraints, assuming the associated value function to be locally Lipschitz in the state. Sufficient structural conditions are given to ensure such a Lipschitz regularity in presence of a positive discount factor, as it is typical of macroeconomics models.

We consider the undiscounted infinite horizon optimal control problem under state constraints in the absence of convexity/concavity assumptions. Then the value function is, in general, nonsmooth. Using the tools of set-valued and nonsmooth analysis, the necessary optimality conditions and sensitivity relations are derived in such a framework. We also investigate relaxation theorems and uniqueness of solutions of the Hamilton–Jacobi–Bellman equation arising in this setting.

The comparative dynamics of closed-loop controls for discounted infinite horizon optimal control problems

In this paper, we study an infinite horizon optimal control problem arising in economic dynamics. We establish convergence of approximate solutions to the golden-rule stock (turnpike) and show the existence of overtaking optimal solutions.

Receding horizon control (RHC) or model predictive control (MPC) solves online a finite horizon open-loop optimal control problem repeatedly in an infinite horizon context and provides a suboptimal control solution. It has been widely used in industry. For continuous-time (CT) systems, two categories of RHC have been investigated in literature, namely instantaneous RHC and sampled-data RHC. This paper focuses on the sampled-data RHC for continuous linear time-invariant (LTI) systems. The usual way to ensure stability of RHC scheme is to impose some constraints on the terminal cost and/or the terminal state and/or the horizon length. A new RHC algorithm is proposed, which we call updated terminal cost receding horizon control (UTC-RHC). Compared with the standard RHC which uses the same terminal cost all the time, our algorithm updates the terminal cost at each sampling instant. The advantage of UTC-RHC is that the uniform exponential stability of the closed-loop system can be guaranteed without imposing the usual constraints on the terminal state or the terminal cost or the horizon size. Moreover, the UTC-RHC control gain approaches the optimal value associated with the infinite horizon optimal control problems.

Turnpike theorem for terminal functionals in infinite horizon optimal control problems

This studies the infinite horizon optimal control problem for a class of continuous‐time systems subjected to multiplicative noises and Markovian jumps by using a data‐driven policy iteration algorithm. The optimal control problem is equivalent to solve a stochastic coupled algebraic Riccatic equation (CARE). An off‐line iteration algorithm is first established to converge the solutions of the stochastic CARE, which is generalised from an implicit iterative algorithm. By applying subsystems transformation (ST) technique, the off‐line iterative algorithm is decoupled into N parallel Kleinman's iterative equations. To learn the solution of the stochastic CARE from N decomposed linear subsystems data, an ST‐based data‐driven policy iteration algorithm is proposed and the convergence is proved. Finally, a numerical example is given to illustrate the effectiveness and applicability of the proposed two iterative algorithms.

We consider a controlled state equation of parabolic type on the halfline (0,+∞) with boundary conditions of Dirichlet type in which the unknown is equal to the sum of the control and of a white noise in time. We study finite horizon and infinite horizon optimal control problem related by means of backward stochastic differential equations.

In this paper, technological interventions in a simple airshed model of emission propagation coupled with a social model of technology adoption are studied. An optimal control problem is introduced where a central planner aims to balance pollution concentration against intervention costs over an infinite time horizon. Interventions are classified into two types. Preventive interventions promote the adoption of low emission (green) technologies in the underlying population whereas dissipative interventions reduce the concentration of pre-existing pollutants in the air. Next, an infinite-horizon optimal control problem is set up and the notion of optimality is established. Applying Pontryagin's Minimum Principle, a sustainable equilibrium is derived and the existence of its stable manifold is explored. Imposition of green technology reduces both the steady-state pollutant concentration and dissipative Effort simultaneously but the planning variables induce a tradeoff between concentration and dissipative Effort. Lastly, the stable manifolds are simulated concluding with a discussion on policy implications for the system under consideration.

The optimal control of a system whose states are governed by a nonlinear autonomous Volterra integrodifferential equation with unbounded time interval is considered. Specifically, it is assumed that the delay occurs only in the state variable. The results obtained extend those of Brock and Haurie [Math. Oper. Res., 1 (1976), pp. 337–346] and Leizarowitz [Math. Oper. Res., 10 (1985), pp. 450–461]. In particular, it is shown that (under appropriate hypotheses) catching-up optimal solutions asymptotically approach a unique optimal steady state, and thus enjoy the so-called “turnpike” property found in the economics literature. By combining this result with an associated optimal control problem, the desired existence result is obtained. Furthermore, it is remarked that, in addition to extending these earlier works to the time-delay case, the results presented below utilize convexity, seminormality, and growth hypotheses that in some cases are weaker than those encountered in the above-mentioned papers.

Direct numerical approximation of a continuous‐time infinite horizon control problem, requires to recast the model as a discrete‐time, finite‐horizon control model. The quality of the optimization results can be heavily degraded if the discretization process does not take into account features of the original model to be preserved. Restricting their attention to optimal growh problems with a steady state, Mercenier and Michel in [1] and [2], studied the conditions to be imposed for ensuring that discrete first‐order approximation models have the same steady states as the infinite‐horizon continuous‐times counterpart. Here we show that Mercenier and Michel scheme is a first order partitioned Runge‐Kutta method applied to the state‐costate differential system which arises from the Pontryagin maximum principle. The main consequence is that it is possible to consider high order schemes which generalize that algorithm by preserving the steady‐growth invariance of the solutions with respect to the discretization process. Numerical examples show the efficiency and accuracy of the proposed methods when applied to the classical Ramsey growth model.