Sequential optimal contracting in continuous time

In this article, we study the Schrödinger bridge problem (SBP) with nonlinear prior dynamics. In control-theoretic language, this is a problem of minimum effort steering of a given joint state probability density function (PDF) to another over a finite-time horizon, subject to a controlled stochastic differential evolution of the state vector. As such, it can be seen as a stochastic optimal control problem in continuous time with endpoint density constraints—A topic that originated in the physics literature in 1930s, and in the recent years, has garnered burgeoning interest in the systems-control community. For generic nonlinear drift, we reduce the SBP to solving a system of forward and backward Kolmogorov partial differential equations (PDEs) that are coupled through the boundary conditions, with unknowns being the “Schrödinger factors”—so named since their product at any time yields the optimal controlled joint state PDF at that time. We show that if the drift is a gradient vector field, or is of mixed conservative–dissipative nature, then it is possible to transform these PDEs into a pair of initial value problems (IVPs) involving the same forward Kolmogorov operator. Combined with a recently proposed fixed point recursion that is contractive in the Hilbert metric, this opens up the possibility to numerically solve the SBPs in these cases by computing the Schrödinger factors via a single IVP solver for the corresponding (uncontrolled) forward Kolmogorov PDE. The flows generated by such forward Kolmogorov PDEs, for the two aforementioned types of drift, in turn, enjoy gradient descent structures on the manifold of joint PDFs with respect to suitable distance functionals. We employ a proximal algorithm developed in our prior work that exploits this geometric viewpoint, to solve these IVPs and compute the Schrödinger factors via weighted scattered point cloud evolution in the state space. We provide the algorithmic details and illustrate the proposed framework of solving the SBPs with nonlinear prior dynamics by numerical examples.

This paper develops a preconditioning theory for the standard optimal control problem. Preconditioning is the execution of a coordinate transformation in order to increase the convergence rate of gradient based numerical algorithms. The transforming operator employed for this purpose is the square root of a symmetric positive definite matrix which is called the preconditioner. It was shown in [l] that the optimal preconditioner for nonlinear constrained mathematical programming problems is a matrix, denoted as 'H+, which is the positive definite reflection of the Hessian matrix 'H. By positive definite reflection, it is meant that matrix whose eigenvectors are identical to those of the Hessian and whose eigenvalues are the moduli of the latter's eigenvalues. It was demonstrated that the preconditioner 'H+ produces a transformed Hessian with a condition number of unity; it was further shown how H+ can be efficiently constructed from a singular value decomposition of 7-t. Since the Hessian is a function of the optimizing variables in a nonlinear problem, periodic application of the preconditioner is required for best results. Here, this preconditioning theory is extended to the standard optimal control problem using the direct approach of transcription. The optimal preconditioner, as well as suboptimal alternatives, are presented based on the Hessian of the resulting finite dimensional problem. Additionally, by allowing the discretization level to approach zero, the infinite-dimensional Hessian of the optimal control problem is obtained, which turns out to be a sparse matrix-valued hybrid surface consisting of ridges and discrete points. This is a nice geometrical result which provides great insight into the optimal control problem. It is shown for completeness, that as At -+ 0, the necessary conditions and the second differential of the mathematical programming problem converge to the necessary conditions and the second variation of the control problem. It is further shown that the second variation of the control problem has the form ;AZ~ 'HAZ, where the Hessian is a hybrid surface and Ar is an infinite-dimensional vector. For the second variation to be positive, the Hessian sur'Associate Fellow, AIAA 'Member, AIAA This paper is declared a work of the U.S. Government and is not subject to copyright protection in the United States. face must be positive definite with respect to surface multiplication, surface multiplication being the infinite-dimensional extension of matrix multiplication. Finally, it is noted that the extended theory is also valid in the indirect approach to optimal control provided a new integration implementation technique is employed, as shown in the separate work [2].

Optimal Hedging in Discrete and Continuous Time

We revisit the linear distributed consensus problem in continuous time, to provide a simple and elegant proof under very mild assumptions. Our approach is based on a novel extension of the contraction coefficient that can be adapted to the continuous time version of the model. We apply our results in non-linear second order consensus networks of Cucker- Smale type and we obtain new initial conditions for asymptotic flocking.

A behavioral mean-risk portfolio selection problem in continuous time is formulated and studied in this article. Based on the standard mean-variance portfolio selection problem, the cumulative distribution function of the cash flow is distorted by a probability distortion function. This probability distortion function represents the risk preference in a different way. Then the problem is no longer a convex optimization problem. This feature distinguishes it from the conventional linear-quadratic (LQ) problems. The stochastic optimal LQ control theory no longer applies. We take the quantile function of the terminal cash flow as the decision variable. The corresponding optimal terminal cash flow can be recovered by the optimal quantile function. Then the efficient strategy is the hedging strategy of the optimal terminal cash flow.

Jump-Diffusions with Controlled Jumps: Existence and Numerical Methods

The paper develops a method to solve higher-dimensional stochastic\ncontrol problems in continuous time. A finite difference type\napproximation scheme is used on a coarse grid of low discrepancy\npoints, while the value function at intermediate points is obtained\nby regression. The stability properties of the method are discussed,\nand applications are given to test problems of up to 10 dimensions.\nAccurate solutions to these problems can be obtained on a personal\ncomputer.

We extend the numerical methods of [Kushner, H.J. and Dupuis, P., 1992, Numerical Methods for Stochastic Control Problems in Continuous Time, 2nd ed., 2001 (Berlin and New York: Springer Verlag], known as the Markov chain approximation methods, to controlled general nonlinear delayed reflected diffusion models. Both the path and the control can be delayed. For the no-delay case, the method covers virtually all models of current interest. The method is robust, the approximations have physical interpretations as control problems closely related to the original one, and there are many effective methods for getting the approximations, and for solving the Bellman equation for low-dimensional problems. These advantages carry over to the delay problem. It is shown how to adapt the methods for getting the approximations, and the convergence proofs are outlined for the discounted cost function. Extensions to all of the cost functions of current interest as well as to models with Poisson jump terms are possible. The paper is particularly concerned with representations of the state and algorithms that minimize the memory requirements.

A nonlinear stochastic control problem related to ow control is considered. It is assumed that the state of a link is described by a controlled hidden Markov process with a finite state set, while the loss ow is described by a counting process with intensity depending on a current transmission rate and an unobserved link state. The control is the transmission rate, and it has to be chosen as a nonanticipating process depending on the observation of the loss process. The aim of the control is to achieve the maximum of some utility function that takes into account losses of the transmitted information. Originally, the problem belongs to the class of stochastic control problems with incomplete information; however, optimal filtering equations that provide estimation of the current link state based on observations of the loss process allow one to reduce the problem to a standard stochastic control problem with full observations. Then a necessary optimality condition is derived in the form of a stochastic maximum principle, which allows us to obtain explicit analytic expressions for the optimal control in some particular cases. Optimal and suboptimal controls are investigated and compared with the ow control schemes used in TCP/IP (Transmission Control Protocols/Internet Protocols) networks. In particular, the optimal control demonstrates a much smoother behavior than the TCP/IP congestion control currently used.

A stochastic control problem is formulated and we get the explicit form of the optimal control for initial-data-parameterized linear quadratic stochastic optimal control problems with random jumps. The optimal control can be proved to be unique. A stochastic Riccati equation is rigorous derived from the stochastic Hamilton system, which provides an optimal feedback control. This completes the the interrelationship between the stochastic Riccati equation and stochastic Hamilton system as two different but equivalent tools for the stochastic linear quadratic problem.

The moral hazard problem with high stakes

Discrete-time hybrid control in Borel spaces: Average cost optimality criterion

Regulatory risk constraints as in the European Solvency II standard formula for insurance companies may lead to wealth-dependent constraints on the investment strategy. We develop two solution approaches for portfolio optimization problems in continuous time with wealth-dependent constraint sets. In the first approach, we reduce the optimization problem to an associate problem with constraints independent of wealth and a different utility function. The associate problem is then solved using known convex duality results. In the second approach, we use a change of control. We apply these results to Solvency II constraint sets and find that even for an investor with HARA utility who inherently reduces risk in times of distress, the constraints help to prevent the investor from taking too much risk in an optimistic market. Furthermore, we measure significant loss in utility and reduction in risk caused by the constraints, and we also evaluate the trade-off between these two effects.

Novikov and Shiryaev (2004) give explicit solutions to a class of optimal stopping problems for random walks based on other similar examples given in Darling et al. (1972). We give the analogue of their results when the random walk is replaced by a Lévy process. Further we show that the solutions show no contradiction with the conjecture given in Alili and Kyprianou (2004) that there is smooth pasting at the optimal boundary if and only if the boundary of the stopping reigion is irregular for the interior of the stopping region.

This paper presents an optimal trajectory planning solution to a special class of the pursuit-evasion game; the herding problem in continuous time. In this model, the pursuer’s objective is to herd the evader to a certain location in the Euclidian co-ordinate system over the shortest path. First, the problem is modeled as a constrained optimization problem. Then the optimal trajectory that the pursuer should follow is obtained as the solution for that problem. After that, the derived solution is applied to a non-holonomic realization of the problem.