The Stochastic LQ Framework with Markov Modulation Parameters

This paper is concerned with a stochastic linear-quadratic (LQ) optimal control problem on infinite time horizon, with regime switching, random coefficients, and cone control constraint. To tackle the problem, two new extended stochastic Riccati equations (ESREs) on infinite time horizon are introduced. The existence of the nonnegative solutions, in both standard and singular cases, is proved through a sequence of ESREs on finite time horizon. Based on this result and some approximation techniques, we obtain the optimal state feedback control and optimal value for the stochastic LQ problem explicitly. Finally, we apply these results to solve a lifetime portfolio selection problem of tracking a given wealth level with regime switching and portfolio constraint.

A stochastic linear quadratic (LQ) control problem is indefinite when the cost weighting matrices for the state and the control are allowed to be indefinite. Indefinite stochastic LQ theory has been extensively developed and has found interesting applications in finance. However, there remains an outstanding open problem, which is to identify an appropriate Riccati-type equation whose solvability is {\it equivalent} to the solvability of the indefinite stochastic LQ problem. This paper solves this open problem for LQ control in a finite time horizon. A new type of differential Riccati equation, called the generalized (differential) Riccati equation, is introduced, which involves algebraic equality/inequality constraints and a matrix pseudoinverse. It is then shown that the solvability of the generalized Riccati equation is not only sufficient, but also necessary, for the well-posedness of the indefinite LQ problem and the existence of optimal feedback/open-loop controls. Moreover, all of the optimal controls can be identified via the solution to the Riccati equation. An example is presented to illustrate the theory developed.

In this paper we study the optimization of the discrete-time stochastic linear-quadratic (LQ) control problem with conic control constraints on an infinite horizon, considering multiplicative noises. Stochastic control systems can be formulated as Markov Decision Problems (MDPs) with continuous state spaces and therefore we can apply the direct-comparison based optimization approach to solve the problem. We first derive the performance difference formula for the LQ problem by utilizing the state separation property of the system structure. Based on this, we successfully derive the optimality conditions and the stationary optimal feedback control. By introducing the optimization, we establish a general framework for infinite horizon stochastic control problems. The direct-comparison based approach is applicable to both linear and nonlinear systems. Our work provides a new perspective in LQ control problems; based on this approach, learning based algorithms can be developed without identifying all of the system parameters.

This paper analyzes one kind of linear quadratic (LQ) stochastic control problem of forward backward stochastic control system associated with Lévy process. We obtain the explicit form of the optimal control, then prove it to be unique, and get the linear feedback regulator by introducing one kind of generalized Riccati equation. Finally, we discuss the solvability of the generalized Riccati equation, and its existence and uniqueness of the solutions are proved in a special case.

A behavioral mean-risk portfolio selection problem in continuous time is formulated and studied in this article. Based on the standard mean-variance portfolio selection problem, the cumulative distribution function of the cash flow is distorted by a probability distortion function. This probability distortion function represents the risk preference in a different way. Then the problem is no longer a convex optimization problem. This feature distinguishes it from the conventional linear-quadratic (LQ) problems. The stochastic optimal LQ control theory no longer applies. We take the quantile function of the terminal cash flow as the decision variable. The corresponding optimal terminal cash flow can be recovered by the optimal quantile function. Then the efficient strategy is the hedging strategy of the optimal terminal cash flow.

In this article, we investigate the multistage stochastic constrained control problem with linear dynamics and quadratic costs when only partial information of the disturbance distribution (i.e., the first two moments) is known. We adopt a distributionally robust chance constraint (DRCC)-based approach, where the DRCC holds (with high probability) as long as the true distribution of uncertainty belongs to a distribution family or the ambiguity set (constructed based on the known information). We approximate the DRCC with the worst-case conditional value-at-risk (CVaR) constraint, which bounds the expected constraint violation with respect to all distributions in the ambiguity set. The worst-case CVaR problem is known to be computationally tractable under linear decision rules (LDRs). To improve the performance of LDRs, we propose to apply the segregated linear decision rules (SLDRs) on dynamical control systems with the worst-case CVaR approximation. To deal with the tractability issue of the worst-case CVaR constraint, we construct a special group of segregation (of the random disturbance) that is shown to be without loss of optimality. The proposed segregation method enables us to establish the equivalence between the stochastic control problem (with worst-case CVaR constraints and under SLDRs) and a tractable semidefinite program.

This paper investigates a backward stochastic linear quadratic control problem with an expected-type equality constraint on the initial state. By using the Lagrange multiplier method, the problem with a uniformly convex cost functional is first transformed into an equivalent unconstrained parameterized backward stochastic linear quadratic control problem. Then, under the surjectivity of the linear constraint, the equivalence between the original problem and the dual problem is proven by Lagrange duality theory. Subsequently, with the help of the maximum principle, an explicit solution of the optimal control for the unconstrained problem is obtained. This solution is feedback-based and determined by an adjoint stochastic differential equation, a Riccati-type ordinary differential equation, a backward stochastic differential equation, and an equality, thereby yielding the optimal control for the original problem. Finally, an optimal control for an investment portfolio problem with an expected-type equality constraint on the initial state is explicitly provided.

Infinite horizon linear quadratic Pareto game of the stochastic singular systems

This paper is concerned with the sampled-data based adaptive linear quadratic (LQ) control of hybrid systems with both unmeasurable Markov jump processes and stochastic noises. By the least matching error estimation algorithm, parameter estimates are presented. By a double-step (DS) sampling approach and the certainty equivalence principle, a sampled-data based adaptive LQ control is designed. The DS-approach is characterized by a comparatively large estimation step for parameter estimation and a sufficient small control step for control updating. Under mild conditions, the closed-loop system is shown to be stable. It is found that the key factor determining the perfor- mance index is the estimation step rather than the control step. When the estimation step becomes too small, the system performance will become worse. When the estimation step is fixed, the system performance can indeed be improved by reducing the control step, but cannot reach the optimal value. The index difference between the sampled-data based adaptive LQ control and the conventional LQ optimal control is asymptotically bounded by a constant depending on the estimation step and the priori information of the parameter set.

In this paper, we examine a stochastic linear-quadratic control problem characterized by regime switching and Poisson jumps. All the coefficients in the problem are random processes adapted to the filtration generated by Brownian motion and Poisson random measure for each given regime. The model incorporates two distinct types of controls: the first is a conventional control that appears in the continuous diffusion component, while the second is an unconventional control, dependent on the variable z , which influences the jump size in the jump diffusion component. Both controls are constrained within general closed cones. By employing the Meyer-Itô formula in conjunction with a generalized squares completion technique, we rigorously and explicitly derive the optimal value and optimal feedback control. These depend on solutions to certain multi-dimensional fully coupled stochastic Riccati equations, which are essentially backward stochastic differential equations with jumps (BSDEJs). We establish the existence of a unique nonnegative solution to the BSDEJs. One of the major tools used in the proof is the newly established comparison theorems for multi-dimensional BSDEJs.

The paper gives a fairly general approach to the stochastic optimal control problem, using some recent results on semimartingales. After a short, review on these results, an abstract model of stochastic optimal control is described, where the influence of the control is modeled by exponentials of martingales. This model gives a synthesis of the martingale approaches of Davis and Varaiya on the one hand, and Striebel on the other hand. Necessary and sufficient conditions for optimality from these papers are adapted to the abstract model. Topological existence results, which apply to the complete observation case as well as to the partial observation case, are described when the likelihood ratios describing the dynamics of the system are subsets respectively of an $L_2 $- and an $L_1 $-space.These abstract results are specialized to the problem where the martingales are represented as stochastic integrals. The results can then be written in a more explicit form; in particular it is possible to formulate the...

A Stochastic Control Approach to Defined Contribution Plan Decumulation: 'The Nastiest, Hardest Problem in Finance'

Some recent concepts and results on asymptotically optimal control rules in several classical stochastic adaptive control problems are reviewed and discussed. In particular, ways to resolve the apparent dilemma between the need for information and the objective of efficient control in these problems are presented. Asymptotic lower bounds on the minimal amount of information needed and a general heuristic principle are also given.

This article examines an economic growth model that expresses the interaction between production, technology stock, and research and development (R&D) investments. The goal of this study is to maximize production. Considering the presence of Gaussian white noises, this model is reformulated as a stochastic optimal control problem, where the R&D investment rate is defined as the control input. We aim to explore the efficiency of Kalman filtering approaches for solving this problem. Here, the extended Kalman filter (EKF) and unscented Kalman filter (UKF) are applied for state estimation. The state equation linearization is made in the EKF, while the unscented transform is taken in the UKF for generating a set of sigma points. These approaches aim to estimate the state dynamics from different perspectives. With these state estimates, two different computational algorithms are proposed, the EKF for state-control (EKF4SC) and UKF for state-control (UKF4SC) algorithms. The optimal control policy is designed to minimize the cost function. For illustration, the model's parameters are considered in the simulation experiment. The simulation results showed that the UKF has higher accuracy in the state estimation with the smallest mean squares of error compared with the EKF. Moreover, the optimal control policy based on the state estimate generated from the UKF could optimize the cost function of the problem. Hence, the results show the efficiency of the algorithms proposed. In conclusion, the application of the Kalman filtering algorithms to the economic growth model is presented. The significance of this study is to provide a stochastic optimal control model for the economic growth problem and to suggest an efficient computational approach for solving this stochastic optimal control problem.

Reliable approximations of probability-constrained stochastic linear-quadratic control