A class of solvable singular stochastic control problems

In this paper, necessary conditions of optimality, in the form of a maximum principle, are obtained for singular stochastic control problems. This maximum principle is derived for a state process satisfying a general stochastic differential equation where the coefficient associated to the control process can be dependent on the state. We consider the class of so called robust nonlinear impulsive systems, those discontinuous solutions can be considered also as point-wise limits of ordinary solutions. The special conditions of robustness permit to derive the backward equations for adjoint variables in concise form of differential equation with measure and thereby to derive the optimality condition in the form of strong (point-wise) maximum principle.

In this article, the singular extremal control problem with time delay is studied from the viewpoint of the calculus of variation and matrix theory. Some sufficient conditions for the non-negativity of the second variation are obtained for an optimal control problem with a delayed state vector. The hypothesis of Hass is first assumed, and the total singular situation of Jacobson and Bell is also considered. A second-order necessary condition for optimality is also presented.

This report summarizes some of our recent work (Guo and Tomecek, SIAM J Control Optim 47(1):421–443, 2008; A class of singular control problems and smooth fit principle, 2008) on a new theoretical connection between singular control of finite variation and optimal switching problems. This correspondence not only provides a novel method for analyzing multi-dimensional singular control problems, but also builds links among singular controls, Dynkin games, and sequential optimal stopping problems.

In this paper we provide a complete theoretical analysis of a two-dimensional degenerate nonconvex singular stochastic control problem. The optimisation is motivated by a storage-consumption model in an electricity market, and features a stochastic real-valued spot price modelled by Brownian motion. We find analytical expressions for the value function, the optimal control, and the boundaries of the action and inaction regions. The optimal policy is characterised in terms of two monotone and discontinuous repelling free boundaries, although part of one boundary is constant and the smooth fit condition holds there.

A singular stochastic control problem with state constraints in two-dimensions is studied. We show that the value function is $C^1$ and its directional derivatives are the value functions of certain optimal stopping problems. Guided by the optimal stopping problem, we then introduce the associated no-action region and the free boundary and show that, under appropriate conditions, an optimally controlled process is a Brownian motion in the no-action region with reflection at the free boundary. This proves a conjecture of Martins, Shreve and Soner [SIAM J. Control Optim. 34 (1996) 2133--2171] on the form of an optimal control for this class of singular control problems. An important issue in our analysis is that the running cost is Lipschitz but not $C^1$. This lack of smoothness is one of the key obstacles in establishing regularity of the free boundary and of the value function. We show that the free boundary is Lipschitz and that the value function is $C^2$ in the interior of the no-action region. We then use a verification argument applied to a suitable $C^2$ approximation of the value function to establish optimality of the conjectured control.

Some second-order necessary conditions in optimal control

We consider a singular stochastic control problem with state constraints that arises in problems of optimal consumption and investment under transaction costs. Numerical approximations for the value function using the Markov chain approximation method of Kushner and Dupuis are studied. The main result of the paper shows that the value function of the Markov decision problem (MDP) corresponding to the approximating controlled Markov chain converges to that of the original stochastic control problem as various parameters in the approximation approach suitable limits. All our convergence arguments are probabilistic; the main assumption that we make is that the value function be finite and continuous. In particular, uniqueness of the solutions of the associated HJB equations is neither needed nor available (in the generality under which the problem is considered). Specific features of the problem that make the convergence analysis nontrivial include unboundedness of the state and control space and the cost function; degeneracies in the dynamics; mixed boundary (Dirichlet-Neumann) conditions; and presence of both singular and absolutely continuous controls in the dynamics. Finally, schemes for computing the value function and optimal control policies for the MDP are presented and illustrated with a numerical study.

This paper analyzes a mathematical model for the growth of bone marrow cells under cell-cycle-specific cancer chemotherapy originally proposed by Fister and Panetta [8]. The model is formulated as an optimal control problem with control representing the drug dosage ( respectively its effect ) and objective of Bolza type depending on the control linearly, a so-called L(1)- objective. We apply the Maximum Principle, followed by high-order necessary conditions for optimality of singular arcs and give sufficient conditions for optimality based on the method of characteristics. Singular controls are eliminated as candidates for optimality, and easily verifiable conditions for strong local optimality of bang-bang controls are formulated in the form of transversality conditions at switching surfaces. Numerical simulations are given.

This work generalizes the connection between the optimal singular stochastic control problem and the optimal stopping problem for regime-switching processes. Via the optimal singular stochastic control, the optimal stopping time and the continuation region are characterized. Moreover, we prove the existence of optimal singular stochastic control for a finite horizon singular control problem with the cost function containing the terminal cost. We prove it directly by the compactification method, which is based on an elaborate application of the properties of probability measures over the càdlàg space. Such a problem was left open in Haussmann and Suo [SIAM J. Control Optim., 33 (1995), pp. 916–936] and [SIAM J. Control Optim., 33 (1995), pp. 937–959]). In addition, our compactification method can remove the convexity condition on the coefficients used in Dufour and Miller [SIAM J. Control Optim., 43 (2004), pp. 708–730].

We consider a singular control problem that aims to maximize the expected cumulative rewards, where the instantaneous returns depend on the state of a controlled process. The contributions of this paper are twofold. Firstly, to establish sufficient conditions for determining the optimality of the one-barrier strategy when the uncontrolled process X follows a spectrally negative Lévy process with a Lévy measure defined by a completely monotone density. Secondly, to verify the optimality of the (2n+1)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(2n+1)$$\\end{document}-barrier strategy when X is a Brownian motion with a drift. Additionally, we provide an algorithm to compute the barrier values in the latter case.

Infinite horizon singular optimal control problems with control taking values in a closed cone U in Rn lead to a dynamic programming equation of the form: maxF2x,v, v',v'' ,F1x,v,v' =0, forall x∈Q, where Q , the state space of the control problem, is some nonempty connected open subset of Rn , and F1, F2 are continuous real-valued functions on QxR2 and QxR3 respectively, with the coercivity assumption for F 2, that is, the function r&rarrr;F2x,u,p,r is nonincreasing on R . A major concern is to determine how smooth the value function v is across the free boundary of the problem. Linear-convex deterministic and stochastic singular control problems in dimension one are considered. We present the analysis of the above equation together with smoothness of the value function across the free boundary. The interest here is the explicitness of the results and the fact that the smooth fit principle depends on the parameters of the control problem. In particular, the value function for the stochastic control problem is founded explicitly, and at the same time, optimal controls are identified using a verification theorem.

Systematic application of Pontryagin's maximum principle to a general class of nonlinear mixed boundary value control problems yields necessary conditions for optimal control. Also, the features of certain pathological situations including the abnormal case and singular control are clearly revealed in this treatment.

This paper studies the problem of a company which expands its stochastic production capacity in irreversible investments by purchasing capital at a given price. The profit production function is of a very general form satisfying minimal standard assumptions. The objective of the company is to find optimal production decisions to maximize its expected total net profit in an infinite horizon. The resulting dynamic programming principle is a singular stochastic control problem. The value function is analyzed in great detail relying on viscosity solutions of the associated Bellman variational inequation: we state several general properties and in particular regularity results on the value function. We provide a complete solution with explicit expressions of the value function and the optimal control: the firm invests in capital so as to maintain its capacity above a certain threshold. This boundary may be computed quite explicitly.

We consider a problem of controlling the production rate of a single machine, single product, stochastic manufacturing system in order to minimize the total discounted inventory /backlog costs. The demand has two components: one is deterministic with constant rate d and the other is stochastic with random demand batches. Under heavy loading (or heavy traffic) conditions, that is, when the average production capacity is close to the average demand, the control problem is approximated by a singular stochastic control problem. The approximate problem can be solved expli­citly. The solution is then interpreted in terms of the actual manufactur­ing system and a control policy for this system is derived. We prove that the resulting policy is nearly optimal under the heavy traffic condition. This policy is characterized by a single critical level z0• The commodity should be produced only when inventory is less than or equal to z0 : The production rate is maximal if the inventory is less than z0 and equal to the deterministic component d of the demand rate if the inventory is equal to z0•.

Linear formulations for the occupation measures of a diffusion process with singular boundary behavior and an arbitrary but fixed control process are motivated and derived. Under specific assumptio...