Backward SDEs for optimal control of partially observed path-dependent stochastic systems: A control randomization approach

Infinite horizon optimal control problem of mean-field backward stochastic delay differential equation under partial information

Considerably much work has been done on Backward Stochastic Differential Equations (BSDEs) in continuous-time with deterministic terminal horizon or stopping times. Various new models have been introduced in these years in order to generalize BSDEs to solve new practical financial problems.
\nOne strand is focused on discrete-time models. Backward Stochastic Difference Equations (also called BSDEs if no ambiguity) on discrete-time finite-state space were introduced by Cohen and Elliott (2010a). The associated theory required only weak assumptions. In the first topic of this thesis, properties of non-linear expectations defined using the discrete-time finite-state BSDEs were studied. A converse comparison theorem was established. Properties of risk measures defined by non-linear expectations, especially the representation theorems, were given. Then the theory of BSDEs was applied to optimal design of dynamic risk measures. Another strand is about a general random terminal time, which is not necessarily a stopping time. The motivation of this model is a financial problem of hedging of defaultable contingent claims, where BSDEs with stopping times are not applicable. In the second topic of this thesis, discrete-time finite-state BSDEs under progressively enlarged filtration were considered. Martingale representation
\ntheorem, existence and uniqueness theorem and comparison theorem were established. Application to nonlinear expectations was also explored. Using the theory of BSDEs, the explicit solution for optimal design of dynamic default risk measures was obtained.
\nIn recent work on continuous-time BSDEs under progressively enlarged filtration, the reference filtration is generated by Brownian motions. In order to deal with cases with jumps, in the third topic of this thesis, a general reference filtration with predictable representation property and an initial time with immersion property were considered. The martingale representation theorem for square-integrable martingales under progressively enlarged filtration was established. Then the existence and uniqueness theorem of BSDEs under enlarged filtration using Lipschitz continuity of the driver was proved. Conditions for a comparison theorem were also presented. Finally applications to nonlinear expectations and hedging of defaultable contingent claims on Brownian-Poisson setting were explored.

We provide existence results and comparison principles for solutions of backward stochastic difference equations (BS$\Delta$Es) and then prove convergence of these to solutions of backward stochastic differential equations (BSDEs) when the mesh size of the time-discretizaton goes to zero. The BS$\Delta$Es and BSDEs are governed by drivers $f^N(t,\omega,y,z)$ and $f(t,\omega,y,z),$ respectively. The new feature of this paper is that they may be non-Lipschitz in z. For the convergence results it is assumed that the BS$\Delta$Es are based on d-dimensional random walks $W^N$ approximating the d-dimensional Brownian motion W underlying the BSDE and that $f^N$ converges to f. Conditions are given under which for any bounded terminal condition $\xi$ for the BSDE, there exist bounded terminal conditions $\xi^N$ for the sequence of BS$\Delta$Es converging to $\xi$, such that the corresponding solutions converge to the solution of the limiting BSDE. An important special case is when $f^N$ and f are convex in z. We show that in this situation, the solutions of the BS$\Delta$Es converge to the solution of the BSDE for every uniformly bounded sequence $\xi^N$ converging to $\xi$. As a consequence, one obtains that the BSDE is robust in the sense that if $(W^N,\xi^N)$ is close to $(W,\xi)$ in distribution, then the solution of the Nth BS$\Delta$E is close to the solution of the BSDE in distribution too.

In this article, we consider a linear-quadratic optimal control problem (LQ problem) for a controlled linear stochastic differential equation driven by a multidimensional Browinan motion and a Poisson random martingale measure in the general case, where the coefficients are allowed to be predictable processes or random matrices. By the duality technique, the dual characterization of the optimal control is derived by the optimality system (so-called stochastic Hamilton system), which turns out to be a linear fully coupled forward-backward stochastic differential equation with jumps. Using a decoupling technique, the connection between the stochastic Hamilton system and the associated Riccati equation is established. As a result, the state feedback representation is obtained for the optimal control. As the coefficients for the LQ problem are random, here, the associated Riccati equation is a highly nonlinear backward stochastic differential equation (BSDE) with jumps, where the generator depends on the unknown variables K, L, and H in a quadratic way (see (5.9) herein). For the case where the generator is bounded and is linearly dependent on the unknown martingale terms L and H, the existence and uniqueness of the solution for the associated Riccati equation are established by Bellman's principle of quasi-linearization.

This paper is concerned with backward stochastic differential equations with Markov switching driven by Brownian motion and Poisson random measure. The motivation is a constrained stochastic Riccati equation derived from a stochastic linear quadratic optimal control problem with both Poisson and Markovian jumps. The existence and uniqueness of an adapted solution under global Lipschitz condition on the generator is obtained. The continuous dependence of the solution on parameters is proved. Two comparison theorems are also derived by a generalized Girsanov transformation theorem.

In this paper we study by probabilistic techniques the convergence of the value function for a two-scale, infinite-dimensional, stochastic controlled system as the ratio between the two evolution speeds diverges. The value function is represented as the solution of a backward stochastic differential equation (BSDE) that it is shown to converge towards a reduced BSDE. The noise is assumed to be additive both in the slow and the fast equations for the state. Some non degeneracy condition on the slow equation is required. The limit BSDE involves the solution of an ergodic BSDE and is itself interpreted as the value function of an auxiliary stochastic control problem on a reduced state space.

The theory of backward stochastic differential equations (BSDEs) was pioneered by Pardoux and Peng [PaPe90]. It became now very popular, and is an important field of research due to its connections with stochastic control, mathematical finance, and partial differential equations. BSDEs provide a probabilistic representation of nonlinear PDEs, which extends the famous Feynman-Kac formula for linear PDEs. As a consequence, BSDEs can be used for designing numerical algorithms to nonlinear PDEs.

Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes

This article considers the decentralized linear–quadratic (LQ) optimal control problem for the multiplicative-noise stochastic systems in discrete time with asymmetric information. There exist two controllers in the system, and each controller has its own information in general structure. The adopted information structure is adapted open loop for both controllers. The main contribution is to derive the necessary and sufficient condition for the unique solvability of the LQ control problem with a general asymmetric information structure and present the explicitly optimal controllers in terms of the proposed Riccati equations. The key is to explicitly solve the forward and backward stochastic difference equations, which are derived from the maximum principle.

We consider a class of backward stochastic differential equations (BSDEs) driven by Brownian motion and Poisson random measure, and subject to constraints on the jump component. We prove the existence and uniqueness of the minimal solution for the BSDEs by using a penalization approach. Moreover, we show that under mild conditions the minimal solutions to these constrained BSDEs can be characterized as the unique viscosity solution of quasi-variational inequalities (QVIs), which leads to a probabilistic representation for solutions to QVIs. Such a representation in particular gives a new stochastic formula for value functions of a class of impulse control problems. As a direct consequence, this suggests a numerical scheme for the solution of such QVIs via the simulation of the penalized BSDEs.

We construct a stochastic maximum principle (SMP) which provides necessary conditions for the existence of Nash equilibria in a certain form of N-agent stochastic differential game (SDG) of a mean-field type. The information structure considered for the SDG is of a possible asymmetric and partial type. To prove our SMP we take an approach based on spike-variations and adjoint representation techniques, analogous to that of S. Peng (SIAM J. Control Optim. 28(4):966–979, 1990) in the optimal stochastic control context. In our proof we apply adjoint representation procedures at three points. The first-order adjoint processes are defined as solutions to certain mean-field backward stochastic differential equations, and second-order adjoint processes of a first type are defined as solutions to certain backward stochastic differential equations. Second-order adjoint processes of a second type are defined as solutions of certain backward stochastic equations of a type that we introduce in this paper, and which we term conditional mean-field backward stochastic differential equations. From the resulting representations, we show that the terms relating to these second-order adjoint processes of the second type are of an order such that they do not appear in our final SMP equations. A comparable situation exists in an article by R. Buckdahn, B. Djehiche, and J. Li (Appl. Math. Optim. 64(2):197–216, 2011) that constructs a SMP for a mean-field type optimal stochastic control problem; however, the approach we take of using these second-order adjoint processes of a second type to deal with the type of terms that we refer to as the second form of quadratic-type terms represents an alternative to a development, to our setting, of the approach used in their article for their analogous type of term.

A noncooperative, two person, zero sum, stochastic differential game is formulated and solved that is described by a linear stochastic equation in a Hilbert space with a fractional Brownian motion and a quadratic payoff functional for the two players. The stochastic equation can model stochastic partial differential equations not only with distributed strategies and noise but also with control strategies and noise restricted to the boundary of the domain. The optimal strategies for the two players are given explicitly. The verification method is a generalization of completion of squares and provides the optimal strategies directly without solving partial differential equations or backward stochastic differential equations. Some examples of games described by stochastic partial differential equations are given.

The decentralized linear–quadratic–Gaussian (LQG) control problem for networked control systems (NCSs) with asymmetric information is investigated, where controller 1 shares its historical information with controller 2, and not vice versa. The asymmetry of the information structure leads to the coupling between controller 2 and estimator 1, and hence the classical separation principle fails. Through the assumption of linear control strategy, the coupling between controller 2 and estimator 1 (CCE) is decoupled, but the estimation gain is still coupled with the control gain. It is noted that the control gain conforms to the backward Riccati equation while estimation gain abides by the forward equation, which is computationally challenging. Applying the stochastic maximum principle, the solvability of the decentralized LQG control problem is reduced to that of corresponding forward and backward stochastic difference equations (FBSDEs). Further, necessary and sufficient conditions for the solvability of optimal control problem are presented by two Riccati equations, one of which is nonsymmetric. Moreover, a novel iterative forward method is proposed to calculate the coupled backward control gain and forward estimation gain.

We consider the $L_2$-regularity of solutions to backward stochastic differential equations (BSDEs) with Lipschitz generators driven by a Brownian motion and a Poisson random measure associated with a L\'{e}vy process $(X_t)_{t\in[0,T]}$. The terminal condition may be a Borel function of finitely many increments of the L\'{e}vy process which is not necessarily Lipschitz but only satisfies a fractional smoothness condition. The results are obtained by investigating how the special structure appearing in the chaos expansion of the terminal condition is inherited by the solution to the BSDE.

In the present paper, the notion of a weak solution of a general backward stochastic differential equation (BSDE), which was introduced by the authors and A. Răşcanu in [Theory Probab. Appl., 49 (2005), pp. 16–50], will be discussed. The relationship between continuity of solutions, pathwise uniqueness, uniqueness in law, and existence of a pathwise unique strong solution is investigated. The main result asserts that if all weak solutions of a BSDE are continuous, then the solution is pathwise unique. One should notice that this is a specific result for BSDEs and there is of course no counterpart for (forward) stochastic differential equations (SDEs). As a consequence, if a weak solution exists and all solutions are continuous, then there exists a pathwise unique solution and this solution is strong. Moreover, if the driving process is a continuous local martingale satisfying the previsible representation property, then the converse is also true. In other words, the existence of discontinuous solutions to a BSDE is a natural phenomenon, whenever pathwise uniqueness or, in particular, uniqueness in law is not satisfied. Examples of discontinuous solutions of a certain BSDE were already given in [R. Buckdahn and H.-J. Engelbert, Proceedings of the Fourth Colloquium on Backward Stochastic Differential Equations and Their Applications, to appear]. This was the motivation for the present paper which is aimed at exploring the general situation.