Discrete-time indefinite stochastic linear quadratic optimal control with equality constraints

In this paper, we use the solutions of forward-backward stochastic differential equations with random jumps to get the optimal controls for forward-backward stochastic linear quadratic optimal control problem and the Nash equilibrium point for nonzero sum differential games problem.

SummaryIn this article, we study the open‐loop and closed‐loop solvability for indefinite mean‐field stochastic linear quadratic (MF‐SLQ) optimal control problem and its application in finance, where the controlled stochastic system is driven by a Brownian motion and a Poisson random martingale measure and also disturbed by some stochastic processes. The intrinsic property of stochastic systems results in the inequivalence of those two solvabilities, which is different from deterministic case. Based on a well‐posedness result of the problem, it is shown that the uniform convexity of cost functional is sufficient for the open‐loop solvability of the problem. By a matrix minimum principle, the necessity condition of regular solvability for a decoupled Riccati equation is established, meanwhile, the closed‐loop solvability is turned out to be equivalent to the regular solvability of Riccati equations with some constraints on the solutions of disturbances equations, moreover, the optimal closed‐loop strategy is characterized by regular solutions of Riccati equations and adapted solutions of disturbances equations. And then a mean‐variance model is considered to solve optimal portfolio selection strategy problem of the insurance company with liability. Our study fills a gap in the field of solvability for mean‐field stochastic optimal control with random jumps and provides a different way for solving mean‐variance portfolio selection model.

Well-posedness of stochastic Riccati equations and closed-loop solvability for stochastic linear quadratic optimal control problems

A stochastic control problem is formulated and we get the explicit form of the optimal control for initial-data-parameterized linear quadratic stochastic optimal control problems with random jumps. The optimal control can be proved to be unique. A stochastic Riccati equation is rigorous derived from the stochastic Hamilton system, which provides an optimal feedback control. This completes the the interrelationship between the stochastic Riccati equation and stochastic Hamilton system as two different but equivalent tools for the stochastic linear quadratic problem.

We consider a sufficient maximum principle of optimal control for a stochastic control problem. This problem is governed by a system of fully coupled multi-dimensional forward-backward doubly stochastic differential equation with Poisson jumps. Moreover, all the coefficients appearing in this system are allowed to be random and depend on the control variable. We derive, in particular, sufficient conditions for optimality for this stochastic optimal control problem. We apply our result to treat a kind of forward-backward doubly stochastic linear quadratic optimal control problems with jumps.

A Class of Linear Quadratic Gaussian Hybrid Optimal Control Problems with Realization–Independent Riccati Equations

We consider the stochastic linear quadratic optimal control problem where the state equation is given by a stochastic differential equation of the Ito–Skorokhod type with respect to fractional Brownian motion. The dynamics are driven by strongly continuous semigroups and the cost functional is quadratic. We use the fractional isometry mapping defined between the space of square integrable stochastic processes with respect to fractional Gaussian white noise measure and the space of integrable stochastic processes with respect to the classical Gaussian white noise measure. By this mapping we transform the fractional state equation to a state equation with Brownian motion. Applying the chaos expansion approach, we can solve the optimal control problem with respect to a state equation with the standard Brownian motion. We recover the solution of the original problem by the inverse of the fractional isometry mapping. Finally, we consider a general form of the state equation related to the Gaussian colored noise, we study the control problem, a system with an algebraic constraint and a particular example involving generalized operators from the Malliavin calculus.

Stochastic Linear Quadratic Optimal Control Problems with Regime-Switching Jumps in Infinite Horizon

An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closed-loop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In this paper, the existence of an optimal closed-loop strategy for the system (also called the closed-loop solvability of the problem) is characterized by the existence of a regular solution to the coupled two (generalized) Riccati equations, together with some constraints on the adapted solution to a linear backward stochastic differential equation and a linear terminal value problem of an ordinary differential equation.

This paper is concerned with backward stochastic differential equations with Markov switching driven by Brownian motion and Poisson random measure. The motivation is a constrained stochastic Riccati equation derived from a stochastic linear quadratic optimal control problem with both Poisson and Markovian jumps. The existence and uniqueness of an adapted solution under global Lipschitz condition on the generator is obtained. The continuous dependence of the solution on parameters is proved. Two comparison theorems are also derived by a generalized Girsanov transformation theorem.

In this chapter we present applications of the chaos expansion method in optimal control and stochastic partial differential equations. In particular, we consider the stochastic linear quadratic optimal control problem where the state equation is given by a stochastic differential equation of the Ito-Skorokhod type with different forms of noise disturbances, operator differential algebraic equations arising in fluid dynamics, stationary equations and fractional versions of the studied equations. Moreover, we provide a numerical framework based on chaos expansions and perform numerical simulations.

Convergence of policy gradient for stochastic linear quadratic optimal control problems in infinite horizon

In this paper, the existence of a mean-square stabilizing solution to a discrete-time modified algebraic Riccati equation (MARE), which arises in the study of some stochastic linear quadratic optimal control problems, is investigated. The theory of cone-invariant operators is employed as the mathematical tool to tackle this problem. We provide some criteria for the cone stability, cone observability, and cone detectability of a class of cone-invariant systems in terms of the associated distinguished eigenvalues. Then two scenarios concerning the problem of existence in terms of the positive definiteness of the input weighting matrix are considered. We first study the MARE under the assumption that the input weighting matrix in the cost function is positive definite. Then an explicit necessary and sufficient condition is obtained. Such a condition is derived for the very first time and it indicates that the common condition of observability or detectability of certain stochastic systems is unnecessary. Onl...

Solvability of a stochastic linear quadratic optimal control problem

This article adopts a reinforcement learning (RL) method to solve infinite horizon continuous-time stochastic linear quadratic problems, where the drift and diffusion terms in the dynamics may depend on both the state and control. Based on the Bellman’s dynamic programming principle, we presented an online RL algorithm to attain optimal control with partial system information. This algorithm computes the optimal control, rather than estimates the system coefficients, and solves the related Riccati equation. It only requires local trajectory information, which significantly simplifies the calculation process. We shed light on our theoretical findings using two numerical examples.