Maximum principle in nonlinear optimal stochastic singular control problems

Infinite horizon singular optimal control problems with control taking values in a closed cone U in Rn lead to a dynamic programming equation of the form: maxF2x,v, v',v'' ,F1x,v,v' =0, forall x∈Q, where Q , the state space of the control problem, is some nonempty connected open subset of Rn , and F1, F2 are continuous real-valued functions on QxR2 and QxR3 respectively, with the coercivity assumption for F 2, that is, the function r&rarrr;F2x,u,p,r is nonincreasing on R . A major concern is to determine how smooth the value function v is across the free boundary of the problem. Linear-convex deterministic and stochastic singular control problems in dimension one are considered. We present the analysis of the above equation together with smoothness of the value function across the free boundary. The interest here is the explicitness of the results and the fact that the smooth fit principle depends on the parameters of the control problem. In particular, the value function for the stochastic control problem is founded explicitly, and at the same time, optimal controls are identified using a verification theorem.

A nonlinear stochastic control problem related to ow control is considered. It is assumed that the state of a link is described by a controlled hidden Markov process with a finite state set, while the loss ow is described by a counting process with intensity depending on a current transmission rate and an unobserved link state. The control is the transmission rate, and it has to be chosen as a nonanticipating process depending on the observation of the loss process. The aim of the control is to achieve the maximum of some utility function that takes into account losses of the transmitted information. Originally, the problem belongs to the class of stochastic control problems with incomplete information; however, optimal filtering equations that provide estimation of the current link state based on observations of the loss process allow one to reduce the problem to a standard stochastic control problem with full observations. Then a necessary optimality condition is derived in the form of a stochastic maximum principle, which allows us to obtain explicit analytic expressions for the optimal control in some particular cases. Optimal and suboptimal controls are investigated and compared with the ow control schemes used in TCP/IP (Transmission Control Protocols/Internet Protocols) networks. In particular, the optimal control demonstrates a much smoother behavior than the TCP/IP congestion control currently used.

This work generalizes the connection between the optimal singular stochastic control problem and the optimal stopping problem for regime-switching processes. Via the optimal singular stochastic control, the optimal stopping time and the continuation region are characterized. Moreover, we prove the existence of optimal singular stochastic control for a finite horizon singular control problem with the cost function containing the terminal cost. We prove it directly by the compactification method, which is based on an elaborate application of the properties of probability measures over the càdlàg space. Such a problem was left open in Haussmann and Suo [SIAM J. Control Optim., 33 (1995), pp. 916–936] and [SIAM J. Control Optim., 33 (1995), pp. 937–959]). In addition, our compactification method can remove the convexity condition on the coefficients used in Dufour and Miller [SIAM J. Control Optim., 43 (2004), pp. 708–730].

We consider the singular stochastic control problem of a linear, time-homogeneous and regular diffusion process. By relying on a combination of stochastic calculus the classical theory of diffusions, and ordinary nonlinear programming techniques, we find the optimal policies and their value functions in the three most common cases appearing in the applications of singular controls. Especially we demonstrate that the smooth fit principle can be interpreted as an ordinary first order necessary condition for optimality

In this article, the singular extremal control problem with time delay is studied from the viewpoint of the calculus of variation and matrix theory. Some sufficient conditions for the non-negativity of the second variation are obtained for an optimal control problem with a delayed state vector. The hypothesis of Hass is first assumed, and the total singular situation of Jacobson and Bell is also considered. A second-order necessary condition for optimality is also presented.

European Option Pricing under Jump Diffusion with Proportional Transaction Costs

Some second-order necessary conditions in optimal control

The paper is concerned with a stochastic optimal control problem in which the controlled system is described by a fully coupled nonlinear forward-backward stochastic differential equation driven by a Brownian motion. It is required that all admissible control processes are adapted to a given subfiltration of the filtration generated by the underlying Brownian motion. For this type of partial information control, one sufficient (a verification theorem) and one necessary conditions of optimality are proved. The control domain need to be convex and the forward diffusion coefficient of the system can contain the control variable.

We consider a singular control problem that aims to maximize the expected cumulative rewards, where the instantaneous returns depend on the state of a controlled process. The contributions of this paper are twofold. Firstly, to establish sufficient conditions for determining the optimality of the one-barrier strategy when the uncontrolled process X follows a spectrally negative Lévy process with a Lévy measure defined by a completely monotone density. Secondly, to verify the optimality of the (2n+1)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(2n+1)$$\\end{document}-barrier strategy when X is a Brownian motion with a drift. Additionally, we provide an algorithm to compute the barrier values in the latter case.

Upon introducing a finite-fuel constraint in a stochastic control system, the convex duality formulation can be set up to represent the original singular control problem as a minimization problem over the space of vector measures at each level of available fuel. This minimization problem is imbedded tightly into a related weak problem, which is actually a mathematical programming problem over a convex,w*-compact space of vector-valued Radon measures. Then, through the Fenchel duality principle, the dual for the finite-fuel control problems is to seek the maximum of smooth subsolutions to a dynamic programming variational inequality. The approach is basically in the spirit of Fleming and Vermes, and the results of this paper extend those of Vinter and Lewis in deterministic control problems to the finite-fuel problems in singular stochastic control. Meanwhile, we also obtain the characterization of the value function as a solution to the dynamic programming variational inequality in the sense of the Schwartz distribution.

We consider a singular stochastic control problem with state constraints that arises in problems of optimal consumption and investment under transaction costs. Numerical approximations for the value function using the Markov chain approximation method of Kushner and Dupuis are studied. The main result of the paper shows that the value function of the Markov decision problem (MDP) corresponding to the approximating controlled Markov chain converges to that of the original stochastic control problem as various parameters in the approximation approach suitable limits. All our convergence arguments are probabilistic; the main assumption that we make is that the value function be finite and continuous. In particular, uniqueness of the solutions of the associated HJB equations is neither needed nor available (in the generality under which the problem is considered). Specific features of the problem that make the convergence analysis nontrivial include unboundedness of the state and control space and the cost function; degeneracies in the dynamics; mixed boundary (Dirichlet-Neumann) conditions; and presence of both singular and absolutely continuous controls in the dynamics. Finally, schemes for computing the value function and optimal control policies for the MDP are presented and illustrated with a numerical study.

A conventional approach to solving stochastic optimal control problems with time-dependent uncertainties involves the use of the stochastic maximum principle (SMP) technique. For large-scale problems, however, such an algorithm frequently leads to convergence complexities when solving the two-point boundary value problem resulting from the optimality conditions. An alternative approach consists of using continuous random variables to capture uncertainty through sampling-based methods embedded within an optimization strategy for the decision variables; such a technique may also fail due to the computational intensity involved in excessive model calculations for evaluating the objective function and its derivatives for each sample. This paper presents a new approach to solving stochastic optimal control problems with time-dependent uncertainties based on BONUS (Better Optimization algorithm for Nonlinear Uncertain Systems). The BONUS has been used successfully for non-linear programming problems with static uncertainties, but we show here that its scope can be extended to the case of optimal control problems with time-dependent uncertainties. A batch reactor for biodiesel production was used as a case study to illustrate the proposed approach. Results for a maximum profit problem indicate that the optimal objective function and the optimal profiles were better than those obtained by the maximum principle.

A global maximum principle for stochastic optimal control problems with delay and applications

This paper studies the necessary and sufficient conditions for near-optimal singular stochastic controls for the systems driven by non-linear stochastic differential equations with jump processes. The proof of our result is based on Ekeland's variational principle and some delicate estimates of the state and adjoint processes. We apply convex perturbation for continuous and singular components of the control. It is shown that optimal singular controls may fail to exist even in simple cases. This justifies the use of near-optimal stochastic singular controls, which exist under minimal hypothesis and are sufficient in most practical cases. Moreover, since there are many near-optimal singular controls, it is possible to choose suitable ones that are convenient for implementation. The set of controls under consideration is necessarily convex. We prove that under an additional hypothesis, the near-maximum condition on the Hamiltonian function is a sufficient condition for near optimality.

Optimal control of systems with noisy memory and BSDEs with Malliavin derivatives