Convex duality for finite-fuel problems in singular stochastic control

Infinite horizon singular optimal control problems with control taking values in a closed cone U in Rn lead to a dynamic programming equation of the form: maxF2x,v, v',v'' ,F1x,v,v' =0, forall x∈Q, where Q , the state space of the control problem, is some nonempty connected open subset of Rn , and F1, F2 are continuous real-valued functions on QxR2 and QxR3 respectively, with the coercivity assumption for F 2, that is, the function r&rarrr;F2x,u,p,r is nonincreasing on R . A major concern is to determine how smooth the value function v is across the free boundary of the problem. Linear-convex deterministic and stochastic singular control problems in dimension one are considered. We present the analysis of the above equation together with smoothness of the value function across the free boundary. The interest here is the explicitness of the results and the fact that the smooth fit principle depends on the parameters of the control problem. In particular, the value function for the stochastic control problem is founded explicitly, and at the same time, optimal controls are identified using a verification theorem.

In this research article, we focus on a stochastic optimal control problem with two types of terminal constraints. These specific conditions provide real-valued and stochastic Lagrange multipliers. Our model evolves according to a Markov regime-switching jump diffusion model with memory. In this context, the memory is represented by a Stochastic Differential Delay Equation. We present two theorems for each constraint within the general formulation of stochastic optimal control theory in a Lagrangian environment. We approach to this task from a theoretical perspective and provide mild technical assumptions, which make our theorems applicable for a broad class of stochastic control problems as well as for a wide range of disciplines such as engineering, biology, operations research, medicine, computer science and economics. In this work, we apply Stochastic Maximum Principle to demonstrate an optimal dividend policy corresponding to a time-delayed wealth process of a company. Moreover, we determine the real-valued Lagrange multiplier of this control problem explicitly.

A stochastic control problem is formulated and we get the explicit form of the optimal control for initial-data-parameterized linear quadratic stochastic optimal control problems with random jumps. The optimal control can be proved to be unique. A stochastic Riccati equation is rigorous derived from the stochastic Hamilton system, which provides an optimal feedback control. This completes the the interrelationship between the stochastic Riccati equation and stochastic Hamilton system as two different but equivalent tools for the stochastic linear quadratic problem.

In this paper we study the optimization of the discrete-time stochastic linear-quadratic (LQ) control problem with conic control constraints on an infinite horizon, considering multiplicative noises. Stochastic control systems can be formulated as Markov Decision Problems (MDPs) with continuous state spaces and therefore we can apply the direct-comparison based optimization approach to solve the problem. We first derive the performance difference formula for the LQ problem by utilizing the state separation property of the system structure. Based on this, we successfully derive the optimality conditions and the stationary optimal feedback control. By introducing the optimization, we establish a general framework for infinite horizon stochastic control problems. The direct-comparison based approach is applicable to both linear and nonlinear systems. Our work provides a new perspective in LQ control problems; based on this approach, learning based algorithms can be developed without identifying all of the system parameters.

A Stochastic Adaptive Control Problem and Related Non-adaptive Problems

A nonlinear stochastic control problem related to ow control is considered. It is assumed that the state of a link is described by a controlled hidden Markov process with a finite state set, while the loss ow is described by a counting process with intensity depending on a current transmission rate and an unobserved link state. The control is the transmission rate, and it has to be chosen as a nonanticipating process depending on the observation of the loss process. The aim of the control is to achieve the maximum of some utility function that takes into account losses of the transmitted information. Originally, the problem belongs to the class of stochastic control problems with incomplete information; however, optimal filtering equations that provide estimation of the current link state based on observations of the loss process allow one to reduce the problem to a standard stochastic control problem with full observations. Then a necessary optimality condition is derived in the form of a stochastic maximum principle, which allows us to obtain explicit analytic expressions for the optimal control in some particular cases. Optimal and suboptimal controls are investigated and compared with the ow control schemes used in TCP/IP (Transmission Control Protocols/Internet Protocols) networks. In particular, the optimal control demonstrates a much smoother behavior than the TCP/IP congestion control currently used.

The stochastic optimal control solution to the Linear-Quadratic-Gaussian problem with partial information is considered where the system parameters operating on the control are assumed to be unknown Gaussian random processes. This class of problems has special properties that are exploited. First, the state and the random parameters are conditionally Gaussian with respect to the measurement history. This allows the optimal stochastic control problem to be reformulated as a stochastic optimal control problem of full information where the new state space is the state and parameter estimates and the error variance of the state and parameters. Even in this somewhat simple form, there is no closed form solution. Therefore, it is assumed that there is only a first order dependence on the random fluctuation of the parameter about its mean. In this way an asymptotic series solution is obtained in terms of the new state space. For this problem a closed form solution for the controller is obtained up to second order. The controller is derived for the scalar case and then extended to the case of vector state with single input and multiple outputs. The characteristics of the new solution is investigated and a scalar numerial example is given to assess the performance of this new controller.

We consider a singular stochastic control problem with state constraints that arises in problems of optimal consumption and investment under transaction costs. Numerical approximations for the value function using the Markov chain approximation method of Kushner and Dupuis are studied. The main result of the paper shows that the value function of the Markov decision problem (MDP) corresponding to the approximating controlled Markov chain converges to that of the original stochastic control problem as various parameters in the approximation approach suitable limits. All our convergence arguments are probabilistic; the main assumption that we make is that the value function be finite and continuous. In particular, uniqueness of the solutions of the associated HJB equations is neither needed nor available (in the generality under which the problem is considered). Specific features of the problem that make the convergence analysis nontrivial include unboundedness of the state and control space and the cost function; degeneracies in the dynamics; mixed boundary (Dirichlet-Neumann) conditions; and presence of both singular and absolutely continuous controls in the dynamics. Finally, schemes for computing the value function and optimal control policies for the MDP are presented and illustrated with a numerical study.

We consider a stochastic control problem which is composed of a controlled stochastic differential equation and whose associated cost functional is defined through a controlled backward stochastic differential equation. Under appropriate convexity assumptions on the coefficients of the forward and the backward equations, we prove the existence of an optimal control on a suitable reference stochastic system. The proof is based on an approximation of the stochastic control problem by a sequence of control problems with smooth coefficients, admitting an optimal feedback control. The quadruplet formed by this optimal feedback control and the associated solution of the forward and the backward equations are shown to converge in law, at least along a subsequence. The convexity assumptions on the coefficients then allow to construct from this limit an admissible control process which, on an appropriate reference stochastic system, is optimal for our stochastic control problem.

In this paper, necessary conditions of optimality, in the form of a maximum principle, are obtained for singular stochastic control problems. This maximum principle is derived for a state process satisfying a general stochastic differential equation where the coefficient associated to the control process can be dependent on the state. We consider the class of so called robust nonlinear impulsive systems, those discontinuous solutions can be considered also as point-wise limits of ordinary solutions. The special conditions of robustness permit to derive the backward equations for adjoint variables in concise form of differential equation with measure and thereby to derive the optimality condition in the form of strong (point-wise) maximum principle.

We consider a sufficient maximum principle of optimal control for a stochastic control problem. This problem is governed by a system of fully coupled multi-dimensional forward-backward doubly stochastic differential equation with Poisson jumps. Moreover, all the coefficients appearing in this system are allowed to be random and depend on the control variable. We derive, in particular, sufficient conditions for optimality for this stochastic optimal control problem. We apply our result to treat a kind of forward-backward doubly stochastic linear quadratic optimal control problems with jumps.

On Controllability of Infinite Dimensional Linear Stochastic Systems

European Option Pricing under Jump Diffusion with Proportional Transaction Costs

We consider a class of stochastic impulse control problems of linear diffusions arising in studies considering the determination of optimal dividend policies. This class of problems appears also in studies analyzing the optimal management of renewable resources. We state a set of weak conditions guaranteeing both existence and uniqueness of the boundary characterizing the optimal policy and its value. We also analyze two associated stochastic control problems and establish a general ordering for both the values and the marginal values of the considered stochastic control problems. In this way we extend previous findings obtained by relying on linear payoff characterizations.

In this article, we investigate the multistage stochastic constrained control problem with linear dynamics and quadratic costs when only partial information of the disturbance distribution (i.e., the first two moments) is known. We adopt a distributionally robust chance constraint (DRCC)-based approach, where the DRCC holds (with high probability) as long as the true distribution of uncertainty belongs to a distribution family or the ambiguity set (constructed based on the known information). We approximate the DRCC with the worst-case conditional value-at-risk (CVaR) constraint, which bounds the expected constraint violation with respect to all distributions in the ambiguity set. The worst-case CVaR problem is known to be computationally tractable under linear decision rules (LDRs). To improve the performance of LDRs, we propose to apply the segregated linear decision rules (SLDRs) on dynamical control systems with the worst-case CVaR approximation. To deal with the tractability issue of the worst-case CVaR constraint, we construct a special group of segregation (of the random disturbance) that is shown to be without loss of optimality. The proposed segregation method enables us to establish the equivalence between the stochastic control problem (with worst-case CVaR constraints and under SLDRs) and a tractable semidefinite program.