Turnpike theorem for terminal functionals in infinite horizon optimal control problems

In this paper, technological interventions in a simple airshed model of emission propagation coupled with a social model of technology adoption are studied. An optimal control problem is introduced where a central planner aims to balance pollution concentration against intervention costs over an infinite time horizon. Interventions are classified into two types. Preventive interventions promote the adoption of low emission (green) technologies in the underlying population whereas dissipative interventions reduce the concentration of pre-existing pollutants in the air. Next, an infinite-horizon optimal control problem is set up and the notion of optimality is established. Applying Pontryagin's Minimum Principle, a sustainable equilibrium is derived and the existence of its stable manifold is explored. Imposition of green technology reduces both the steady-state pollutant concentration and dissipative Effort simultaneously but the planning variables induce a tradeoff between concentration and dissipative Effort. Lastly, the stable manifolds are simulated concluding with a discussion on policy implications for the system under consideration.

On chaos control of nonlinear fractional chaotic systems via a neural collocation optimization scheme and some applications

This paper proposes an optimal control algorithm for a polynomial system with a quadratic criterion over infinite horizon. The designed regulator gives a closed form solution to the infinite horizon optimal control problem for a polynomial system with a quadratic criterion. The obtained solution consists in a feedback control law obtained by solving a Riccati algebraic equation dependent on the state. Numerical simulations in the example show advantages of the developed algorithm.

Partial and full sensitivity relations are obtained for nonauto-nomous optimal control problems with infinite horizon subject to state constraints, assuming the associated value function to be locally Lipschitz in the state. Sufficient structural conditions are given to ensure such a Lipschitz regularity in presence of a positive discount factor, as it is typical of macroeconomics models.

The Viability Kernel Algorithm for Computing Value Functions of Infinite Horizon Optimal Control Problems

Theturnpike propertyin contemporary macroeconomics asserts that if an economic planner seeks to move an economy from one level of capital to another, then the most efficient path, as long as the planner has enough time, is to rapidly move stock to a level close to the optimal stationary or constant path, then allow for capital to develop along that path until the desired term is nearly reached, at which point the stock ought to be moved to the final target. Motivated in part by its nature as a resource allocation strategy, over the past decade, the turnpike property has also been shown to hold for several classes of partial differential equations arising in mechanics. When formalized mathematically, the turnpike theory corroborates insights from economics: for an optimal control problem set in a finite-time horizon, optimal controls and corresponding states are close (often exponentially) most of the time, except near the initial and final times, to the optimal control and the corresponding state for the associated stationary optimal control problem. In particular, the former are mostly constant over time. This fact provides a rigorous meaning to the asymptotic simplification that some optimal control problems appear to enjoy over long time intervals, allowing the consideration of the corresponding stationary problem for computing and applications. We review a slice of the theory developed over the past decade – the controllability of the underlying system is an important ingredient, and can even be used to devise simple turnpike-like strategies which are nearly optimal – and present several novel applications, including, among many others, the characterization of Hamilton–Jacobi–Bellman asymptotics, and stability estimates in deep learning via residual neural networks.

Receding horizon control (RHC) or model predictive control (MPC) solves online a finite horizon open-loop optimal control problem repeatedly in an infinite horizon context and provides a suboptimal control solution. It has been widely used in industry. For continuous-time (CT) systems, two categories of RHC have been investigated in literature, namely instantaneous RHC and sampled-data RHC. This paper focuses on the sampled-data RHC for continuous linear time-invariant (LTI) systems. The usual way to ensure stability of RHC scheme is to impose some constraints on the terminal cost and/or the terminal state and/or the horizon length. A new RHC algorithm is proposed, which we call updated terminal cost receding horizon control (UTC-RHC). Compared with the standard RHC which uses the same terminal cost all the time, our algorithm updates the terminal cost at each sampling instant. The advantage of UTC-RHC is that the uniform exponential stability of the closed-loop system can be guaranteed without imposing the usual constraints on the terminal state or the terminal cost or the horizon size. Moreover, the UTC-RHC control gain approaches the optimal value associated with the infinite horizon optimal control problems.

In the previous chapters it has been seen that the turnpike properties of extremal trajectories play an important role in developing existence results for autonomous and discounted infinite horizon optimal control problems. In particular we remind the reader of the sufficient conditions for overtaking optimality of Theorems 2.4, 2.5, and 4.4 as well as the results of Chapter 5 regarding the reduction to finite rewards. We notice that the results of Chapter 2 are given for the general nonautonomous control system where the results given in Chapters 4 and 5 apply only to autonomous problems.

In this work we revisit the linear-quadratic optimal control problem for differential-algebraic systems on the infinite time horizon with zero terminal state. Based on the recently developed Lur'e equation for differential-algebraic equations we obtain new equivalent conditions for feasibility. These are related to the existence of a stabilizing solution of the Lur'e equation. This approach also allows us to determine optimal controls if they exist. In particular, we can characterize regularity of the optimal control problem. The latter refers to existence and uniqueness of optimal controls for any consistent initial condition.

In optimal control problems with infinite time horizon, arising in models of economic growth, there are essential difficulties in analytical and even in numerical construction of solutions of Hamiltonian systems. The problem is in stiff properties of differential equations of the maximum principle and in non-stable character of equilibrium points connected with corresponding transversality conditions. However, if a steady state exists and meets several conditions of regularity then it is possible to construct a nonlinear stabilizer for the Hamiltonian system. This stabilizer inherits properties of the maximum principle, generates a nonlinear system with excluded adjoint variables and leads its trajectories to the steady state. Basing on the qualitative theory of differential equations, it is possible to prove that trajectories generated by the nonlinear stabilizer are close to solutions of the original Hamiltonian system, at least locally, in a neighborhood of the steady state. This analysis allows to create stable algorithms for construction of optimal solutions.Keywordsoptimal controlnonlinear control systemnonlinear stabilizereconomic systems

The paper is devoted to analysis of the structure of approximation schemes for value functions in optimal control problems with the infinite time horizon. The problem peculiarity is a quality functional with an unbounded integrand, which is discounted by an exponential index. The main result is reduction of the optimal control problem to an equivalent problem with the value function that has the compact definition domain and the compact value range. This goal is reached by a series of nonlinear changes of variables and the transformations of the system dynamics and the Hamilton-Jacobi equations. The proposed method is resulted, first, in obtaining the strongly invariant calculation domains of approximation schemes and, second, in compactification of ranges for the value functions. The first property provides the opportunity to implement approximation schemes in the form of numerical finite difference schemes for the value functions in compact definition domains. The second property guarantees convergence of approximation schemes to the value functions.

In this article, a novel multistep look-ahead policy iteration with isoperimetric constraints (MLPIIC) method is developed to solve infinite horizon optimal control problems (OCPs) with isoperimetric constraints for discrete-time nonlinear systems. In order to overcome the difficulty that Bellman’s principle of optimality does not hold directly in OCPs with isoperimetric constraints, a method to approximate OCPs with isoperimetric constraints by OCPs with new constraints is developed. For the MLPIIC method initialized with an admissible control law, the convergence and optimality of the iterative value function and the feasibility of the iterative control law are proven. Utilizing the function approximator, the implementation of the MLPIIC method is described. Finally, simulation results are provided.

Infinite horizon optimal control problem of mean-field backward stochastic delay differential equation under partial information

This paper is concerned with a stochastic linear-quadratic (LQ) optimal control problem on infinite time horizon, with regime switching, random coefficients, and cone control constraint. To tackle the problem, two new extended stochastic Riccati equations (ESREs) on infinite time horizon are introduced. The existence of the nonnegative solutions, in both standard and singular cases, is proved through a sequence of ESREs on finite time horizon. Based on this result and some approximation techniques, we obtain the optimal state feedback control and optimal value for the stochastic LQ problem explicitly. Finally, we apply these results to solve a lifetime portfolio selection problem of tracking a given wealth level with regime switching and portfolio constraint.

This paper focuses on addressing the linear quadratic (LQ) optimal control problem on an infinite time horizon for stochastic systems controlled by impulses. No constraint regarding the sign of the quadratic functional is applied. That is why our first concern is to find conditions which guarantee that the considered optimal control problem is well posed. Then, when the optimal control problem is well posed, it is natural to look for conditions which guarantee the attainability of the optimal control problem that is being evaluated. The main tool involved in the solution of the problems stated before is a backward jump matrix linear differential equation (BJMLDE) with a Riccati-type jumping operator. This is formulated using the matrix coefficients of the controlled system and the weight matrices of the performance criterion. We show that the well posedness of the optimal control problem under investigation is guaranteed by the existence of the maximal and bounded solution of the associated BJMLDE with a Riccati-type jumping operator. Further, we show that when the associated BJMLDE with a Riccati-type jumping operator has a maximal solution which satisfies a suitable sign condition, then the optimal control problem is attainable if and only if it has an optimal control in a state feedback form, or if and only if the maximal solution of the BJMLDE with a Riccati-type jumping operator is a stabilizing solution. In order to make the paper more self-contained, we present a set of conditions that correspond to the existence of the maximal solution of the BJMLDE satisfying the desired sign condition.