Forward–backward stochastic differential equations and controlled McKean–Vlasov dynamics

The purpose of this chapter is to provide a detailed probabilistic analysis of the optimal control of nonlinear stochastic dynamical systems of McKean-Vlasov type. We tackle the characterization and construction of solutions of this special type of optimal control problem by means of forward-backward stochastic differential equations. Because of the presence of the distribution of the controlled state in the coefficients, the approach based on the Pontryagin stochastic maximum principle requires special attention. We provide a version of this maximum principle based on the differential calculus for functions of probability measures introduced and developed in Chapter 5 We test the results of the analysis on linear quadratic models and a few other models already considered in the framework of mean field games. Finally, we highlight the similarities and the differences between this problem and MFG problems with which it is often confused.

An approximation theory of optimal control for nonlinear stochastic dynamic systems has been established. Based on the generalized Hamilton-Jacobi-Bellman equation of the cost function of nonlinear stochastic systems, general iterative procedures for approximating the optimal control are developed by successively improving the performance of a feedback control law until a satisfactory suboptimal solution is achieved. A successive design scheme using upper and lower bounds of the exact cost function has been developed for the infinite-time stochastic regulator problem. The determination of the upper and lower bounds requires the solution of a partial differential inequality instead of equality. Therefore it provides a degree of flexibility in the design method over the exact design method. Stability of the infinite-time sub-optimal control problem was established under not very restrictive conditions, and stable sequences of controllers can be generated. Several examples are used to illustrate the applicati on of the proposed approximation theory to stochastic control. It has been shown that in the case of linear quadratic Gaussian problems, the approximation theory leads to the exact solution of optimal control.

We consider a mean field game (MFG) of optimal portfolio liquidation under asymmetric information. We prove that the solution to the MFG can be characterized in terms of a forward-backward stochastic differential equation (FBSDE) with a possibly singular terminal condition on the backward component or, equivalently, in terms of an FBSDE with a finite terminal value yet a singular driver. Extending the method of continuation to linear-quadratic FBSDEs with a singular driver, we prove that the MFG has a unique solution. Our existence and uniqueness result allows proving that the MFG with a possibly singular terminal condition can be approximated by a sequence of MFGs with finite terminal values.

The paper introduces a new method for finding optimal control of algebraic dynamic systems. The structure of algebraic dynamical systems is nonlinear with quadratic and bilinear terms. A new hybrid extended Fourier series is introduced, and state and control variables of the system are expanded by this series. Moreover, properties of new series are presented, and integration and product operational matrices are obtained. Using operational matrices, optimal control of the systems is converted to a set of simultaneous nonlinear algebraic relations. An illustrative example is included to compare our results with those in the literature.

This paper presents a method for obtaining optimal stable control for general nonlinear nonautonomous dynamical systems. The approach is inspired by recent developments in analytical dynamics and the observation that the Lyapunov criterion for stability of dynamical systems can be recast as a constraint to be imposed on the system. A closed-form expression for control is obtained that minimizes a user-defined control cost at each instant of time and enforces the Lyapunov constraint simultaneously. The derivation of this expression closely mirrors the development of the fundamental equation of motion used in the study of constrained motion. For this control method to work, the positive definite functions used in the Lyapunov constraint should satisfy a consistency condition. A class of positive definite functions has been provided for mechanical systems that meet this criterion. To illustrate the broad scope of the method, for linear systems it is shown that a proper choice of these positive definite functions results in conventional LQR control. Control of the Lorenz system and a multi-degree of freedom nonlinear mechanical system are considered. Numerical examples demonstrating the efficacy and simplicity of the method are provided.

In this study, we investigate multi-player noncooperative minmax H∞ target tracking game strategy with conflicting target strategies and cooperative H∞ target tracking game strategy with common target strategy of nonlinear mean-field stochastic jump diffusion (MFSJD) system with external disturbance. Due to the nonlinear terms and mean-field (average) behaviors in the stochastic nonlinear MFSJD system and minmax H∞ payoff function, the multi-player noncooperative and cooperative minmax H∞ mean-field game strategy of nonlinear MFSJD system are more difficult than the linear MFSJD system and conventional nonlinear stochastic system. To avoid solving complex Hamilton Jacobi Isaacs inequalities (HJIIs) of multi-player noncooperative minmax H∞ mean-field target tracking game strategy of nonlinear MFSJD system, the nonlinear MFSJD system is interpolated by a set of local linearized MFSJD system through smoothing functions by the global linearization method. Then the multi-player noncooperative minmax H∞ nonlinear mean-field target tracking game strategy design can be transformed to a linear matrix inequalities (LMIs)-constrained multi-objective optimization problem (MOP). The LMIs-constrained MOP could be efficiently solved by the help of the proposed LMIs-constrained multi-objective evolution algorithm (MOEA). We can prove that the Pareto optimal solution of LMIs-constrained MOP is the Nash equilibrium solution of noncooperative minmax H∞ mean-field target tracking strategy of nonlinear mean-field MFSJD system. Further, the cooperative minmax H∞ mean-field common target tracking strategy of nonlinear mean-field stochastic system is reduced to an LMIs-constrained single-objective optimization problem (SOP). Finally, two simulation examples of cyber-financial mean-field systems are given to illustrate the design procedure and compare the efficacies of the proposed noncooperative and cooperative minmax H∞ mean-field target tracking strategy of nonlinear MFSJD systems.

Stochastic optimal control based on reliability plays a crucial role in mitigating structural failure and ensuring safe operation. For general stochastic vibration systems, a data-driven method for reliability-based optimal control is proposed based on random state data. Firstly, the feedback control is split into conservative and dissipative components in coherence with physical intuition, and each component is expanded upon using pre-selected basic functions, resulting in a modified system with undetermined coefficients. Then, from discrete random samples, the expressions for the probability densities of the first-passage time and the stationary response can be identified, explicitly including system and excitation parameters, as well as initial and safety boundary conditions. Finally, a performance index, combining system reliability and control cost, is constructed and can be explicitly expressed in terms of the undetermined coefficients of the control forces. The optimal control problem for determining the optimal control forces is reformulated as an optimization problem for determining the coefficients that minimize the performance index. This unconstrained optimization problem can be easily solved. Three typical nonlinear stochastic systems, a controlled Duffing oscillator, a controlled nonlinear hysteretic system, and a controlled 2-DOF nonlinear dynamical system, are given to illustrate the validity and accuracy of the proposed data-driven method. Additionally, the control performance of the optimal control strategy in improving the system reliability is also discussed.

formula omitted]-Nash equilibrium in stochastic differential games with mean-field interaction and controlled jumps

This paper presents a simple methodology for obtaining the entire set of continuous controllers that cause a nonlinear dynamical system to exactly track a given trajectory. The trajectory is provided as a set of algebraic and/or differential equations that may or may not be explicitly dependent on time. Closed-form results are also provided for the real-time optimal control of such systems when the control cost to be minimized is any given weighted norm of the control, and the minimization is done not just of the integral of this norm over a span of time but also at each instant of time. The method provided is inspired by results from analytical dynamics and the close connection between nonlinear control and analytical dynamics is explored. The paper progressively moves from mechanical systems that are described by the second-order differential equations of Newton and/or Lagrange to the first-order equations of Poincaré, and then on to general first-order nonlinear dynamical systems. A numerical example illustrates the methodology.

The stability of stochastic dynamic systems have widespread application prospects and greater theoretical significance. Its theory are mainly applied to engineering control, communication equipment, military technology, biology, finance, etc. In this article, we consider the stability of a Class of nonlinear impulsive neutral stochastic differential dynamic systems. A new set of conditions proving the mean square stability of this nonlinear impulsive neutral stochastic dynamic systems are derived by means of the Banach fixed point theorem. Some well-known results are improved and generalized.

Existence of optimal controls for systems driven by FBSDEs

Nonlinear [formula omitted]-gain verification for nonlinear systems

A linear quadratic (LQ) stochastic optimization system involving large population, which is driven by forward-backward stochastic differential equation (FBSDE), is investigated in this paper. Agents cooperate with each other to minimize the so-called social objective, which is rather different from mean field (MF) game. Employing forward-backward person-by-person optimality principle, we derive an auxiliary LQ control problem by decentralized information. A decentralized strategy is obtained by virtue of an MF-type forward-backward stochastic differential equation consistency condition. Applying Riccati equation decoupling method, we solve the consistency condition system. We also verify the asymptotic social optimality in this framework.

Sufficient conditions for the controllability of nonlinear stochastic fractional boundary control systems are established. The equivalent integral equations are derived for both linear and nonlinear systems, and the control function is given in terms of the pseudoinverse operator. The Banach contraction mapping theorem is used to obtain the result. A controllability result for nonlinear stochastic fractional integrodifferential systems is also attained. Examples are included to illustrate the theory.

Optimal control and properties of nonlinear multistage dynamical system for planning horizontal well paths