An Optimal Multibarrier Strategy for a Singular Stochastic Control Problem with a State-Dependent Reward

Infinite horizon singular optimal control problems with control taking values in a closed cone U in Rn lead to a dynamic programming equation of the form: maxF2x,v, v',v'' ,F1x,v,v' =0, forall x∈Q, where Q , the state space of the control problem, is some nonempty connected open subset of Rn , and F1, F2 are continuous real-valued functions on QxR2 and QxR3 respectively, with the coercivity assumption for F 2, that is, the function r&rarrr;F2x,u,p,r is nonincreasing on R . A major concern is to determine how smooth the value function v is across the free boundary of the problem. Linear-convex deterministic and stochastic singular control problems in dimension one are considered. We present the analysis of the above equation together with smoothness of the value function across the free boundary. The interest here is the explicitness of the results and the fact that the smooth fit principle depends on the parameters of the control problem. In particular, the value function for the stochastic control problem is founded explicitly, and at the same time, optimal controls are identified using a verification theorem.

A singular stochastic control problem with state constraints in two-dimensions is studied. We show that the value function is $C^1$ and its directional derivatives are the value functions of certain optimal stopping problems. Guided by the optimal stopping problem, we then introduce the associated no-action region and the free boundary and show that, under appropriate conditions, an optimally controlled process is a Brownian motion in the no-action region with reflection at the free boundary. This proves a conjecture of Martins, Shreve and Soner [SIAM J. Control Optim. 34 (1996) 2133--2171] on the form of an optimal control for this class of singular control problems. An important issue in our analysis is that the running cost is Lipschitz but not $C^1$. This lack of smoothness is one of the key obstacles in establishing regularity of the free boundary and of the value function. We show that the free boundary is Lipschitz and that the value function is $C^2$ in the interior of the no-action region. We then use a verification argument applied to a suitable $C^2$ approximation of the value function to establish optimality of the conjectured control.

In this paper we provide a complete theoretical analysis of a two-dimensional degenerate nonconvex singular stochastic control problem. The optimisation is motivated by a storage-consumption model in an electricity market, and features a stochastic real-valued spot price modelled by Brownian motion. We find analytical expressions for the value function, the optimal control, and the boundaries of the action and inaction regions. The optimal policy is characterised in terms of two monotone and discontinuous repelling free boundaries, although part of one boundary is constant and the smooth fit condition holds there.

This work generalizes the connection between the optimal singular stochastic control problem and the optimal stopping problem for regime-switching processes. Via the optimal singular stochastic control, the optimal stopping time and the continuation region are characterized. Moreover, we prove the existence of optimal singular stochastic control for a finite horizon singular control problem with the cost function containing the terminal cost. We prove it directly by the compactification method, which is based on an elaborate application of the properties of probability measures over the càdlàg space. Such a problem was left open in Haussmann and Suo [SIAM J. Control Optim., 33 (1995), pp. 916–936] and [SIAM J. Control Optim., 33 (1995), pp. 937–959]). In addition, our compactification method can remove the convexity condition on the coefficients used in Dufour and Miller [SIAM J. Control Optim., 43 (2004), pp. 708–730].

ABSTRACTWe consider a class of singular control problems driven by a double exponential jump diffusion process, which come from the reversible investment problem. In some interesting cases (e.g., the running cost function is given by the so-called Cobb-Douglas production function), we give the explicit solutions to the singular control problem by using the connection between singular control and optimal switching. We solve a collection of consistent optimal switching problems and yield the explicit solution for the singular control problem. We then give an application to a particular inventory control problem in a single random period.

A linear quadratic regulator problem where a small parameter μ2multiplies the control cost is considered. As \mu \rightarrow 0 , it exhibits a dynamic behavior with multiple time-scales, whereas for \mu = 0 , it is a singular control problem. At first a multiparameter singularly perturbed model of the given problem is constructed by transferring the singularity that exists in the performance index to the dynamic equations. Then the problem is decomposed into several subproblems of minimal order, each pertaining to only one time-scale. The method developed permits under a single framework the characterization of asymptotic behavior of optimal closed-loop poles, state and control trajectories, performance index, and optimal transfer function as \mu \rightarrow 0 . Moreover, the asymptotic solution brings into focus 1) the nature of singular control and its connection to transmission zeros and 2) the presence of various types of impulses and other higher order distributions in both state and control. The approach unifies and extends significantly several ideas previously somewhat disjoint.

In this article, the singular extremal control problem with time delay is studied from the viewpoint of the calculus of variation and matrix theory. Some sufficient conditions for the non-negativity of the second variation are obtained for an optimal control problem with a delayed state vector. The hypothesis of Hass is first assumed, and the total singular situation of Jacobson and Bell is also considered. A second-order necessary condition for optimality is also presented.

A Variational Approach to Determine the Optimal Power Distribution for Cycling in a Time Trial

In this paper, necessary conditions of optimality, in the form of a maximum principle, are obtained for singular stochastic control problems. This maximum principle is derived for a state process satisfying a general stochastic differential equation where the coefficient associated to the control process can be dependent on the state. We consider the class of so called robust nonlinear impulsive systems, those discontinuous solutions can be considered also as point-wise limits of ordinary solutions. The special conditions of robustness permit to derive the backward equations for adjoint variables in concise form of differential equation with measure and thereby to derive the optimality condition in the form of strong (point-wise) maximum principle.

This report summarizes some of our recent work (Guo and Tomecek, SIAM J Control Optim 47(1):421–443, 2008; A class of singular control problems and smooth fit principle, 2008) on a new theoretical connection between singular control of finite variation and optimal switching problems. This correspondence not only provides a novel method for analyzing multi-dimensional singular control problems, but also builds links among singular controls, Dynkin games, and sequential optimal stopping problems.

We consider a singular stochastic control problem with state constraints that arises in problems of optimal consumption and investment under transaction costs. Numerical approximations for the value function using the Markov chain approximation method of Kushner and Dupuis are studied. The main result of the paper shows that the value function of the Markov decision problem (MDP) corresponding to the approximating controlled Markov chain converges to that of the original stochastic control problem as various parameters in the approximation approach suitable limits. All our convergence arguments are probabilistic; the main assumption that we make is that the value function be finite and continuous. In particular, uniqueness of the solutions of the associated HJB equations is neither needed nor available (in the generality under which the problem is considered). Specific features of the problem that make the convergence analysis nontrivial include unboundedness of the state and control space and the cost function; degeneracies in the dynamics; mixed boundary (Dirichlet-Neumann) conditions; and presence of both singular and absolutely continuous controls in the dynamics. Finally, schemes for computing the value function and optimal control policies for the MDP are presented and illustrated with a numerical study.

A N IMPORTANT problem in air traffic management (ATM) is the design of aircraft trajectories that meet certain arrival time constraints at given waypoints, for instance, at the top of descent (TOD), at the initial approach fix (IAF), or at the runway threshold. These are called 4-D trajectories, which are a key element in the trajectory-based-operations (TBO) concept proposed by NextGen and SESAR for the future ATM system. The time constraints must also be met in certain cases in which the nominal trajectories have to be modified to resolve detected conflicts (e.g., lost of separation minima) between aircraft; for example, Bilimoria and Lee [1] analyze aircraft conflict resolution with an arrival time constraint at a downstream waypoint. On the other hand, the design of fuel-optimal trajectories that lead to energy-efficient flights is another important problem which has been treated extensively in the literature, see, for instance, Burrows [2], Neuman andKreindler [3],Menon [4] and the references therein. Fuel-optimal trajectories with fixed arrival times are studied by Sorensen and Waters [5], Burrows [6] and Chakravarty [7], who analyze the 4-D fuel-optimization problem as a minimum directoperating-cost (DOC) problem with free final time, that is, the problem is to find the time cost for which the corresponding free final-time DOC-optimal trajectory arrives at the assigned time. In this work we analyze the problem of minimum-fuel cruise at constant altitudewith afixed arrival time as a singular optimal control problem, building upon the works of Pargett and Ardema [8] and Rivas and Valenzuela [9], who analyze the problem of maximumrange cruise at constant altitude also as a singular optimal control problem; the case of unsteady cruise is considered. The singular arcs and the corresponding optimal control are obtained as a function of the final time. The optimal paths are obtained as well, which define a variable-Mach cruise at constant altitude. The influence of cruise altitude on the optimal paths is analyzed, and the minimum fuel is calculated. The final-time constraint may be defined, for example, by a flight delay imposed on the nominal (preferred) cruise trajectory (which in our case is the minimum-fuel cruise trajectory with free final time); comparison with a standard constant-Mach procedure to absorb delays is made. Results are presented for a model of a Boeing 767-300ER. Problem Formulation

We consider the singular stochastic control problem of a linear, time-homogeneous and regular diffusion process. By relying on a combination of stochastic calculus the classical theory of diffusions, and ordinary nonlinear programming techniques, we find the optimal policies and their value functions in the three most common cases appearing in the applications of singular controls. Especially we demonstrate that the smooth fit principle can be interpreted as an ordinary first order necessary condition for optimality

We consider a problem of controlling the production rate of a single machine, single product, stochastic manufacturing system in order to minimize the total discounted inventory /backlog costs. The demand has two components: one is deterministic with constant rate d and the other is stochastic with random demand batches. Under heavy loading (or heavy traffic) conditions, that is, when the average production capacity is close to the average demand, the control problem is approximated by a singular stochastic control problem. The approximate problem can be solved expli­citly. The solution is then interpreted in terms of the actual manufactur­ing system and a control policy for this system is derived. We prove that the resulting policy is nearly optimal under the heavy traffic condition. This policy is characterized by a single critical level z0• The commodity should be produced only when inventory is less than or equal to z0 : The production rate is maximal if the inventory is less than z0 and equal to the deterministic component d of the demand rate if the inventory is equal to z0•.

This paper presents an algorithm for the computation of totally singular optimal control problems. The algorithm is an extension of a modified quasilinearization technique which was originally developed for the solution of non-linear two-point boundary-value problems. A numerical example is included for illustration.