Stochastic Processes, Estimation, and Control

SummaryIn this article, we study the open‐loop and closed‐loop solvability for indefinite mean‐field stochastic linear quadratic (MF‐SLQ) optimal control problem and its application in finance, where the controlled stochastic system is driven by a Brownian motion and a Poisson random martingale measure and also disturbed by some stochastic processes. The intrinsic property of stochastic systems results in the inequivalence of those two solvabilities, which is different from deterministic case. Based on a well‐posedness result of the problem, it is shown that the uniform convexity of cost functional is sufficient for the open‐loop solvability of the problem. By a matrix minimum principle, the necessity condition of regular solvability for a decoupled Riccati equation is established, meanwhile, the closed‐loop solvability is turned out to be equivalent to the regular solvability of Riccati equations with some constraints on the solutions of disturbances equations, moreover, the optimal closed‐loop strategy is characterized by regular solutions of Riccati equations and adapted solutions of disturbances equations. And then a mean‐variance model is considered to solve optimal portfolio selection strategy problem of the insurance company with liability. Our study fills a gap in the field of solvability for mean‐field stochastic optimal control with random jumps and provides a different way for solving mean‐variance portfolio selection model.

Stochastic bifurcation characteristics and optimal control of giant magnetostrictive film (GMF)-shape memory alloy (SMA) composite plate subjected to in-plane stochastic excitation were studied in this paper. Nonlinear difference item was introduced to interpret the hysteretic phenomena of both GMF and SMA, and then the nonlinear dynamic model of GMF-SMA composite plate subjected to in-plane stochastic excitation was developed. The stochastic stability of the system was analyzed, and the condition of stochastic Hopf bifurcation was discussed. The reliability function of the system was solved from backward Kolmogorov equation, and then the probability density of the first-passage time was obtained. Finally, the stochastic optimal control strategy was obtained in stochastic dynamic programming method. Numerical simulation shows that the stability of the trivial solution varies with bifurcation parameters, and stochastic Hopf bifurcation appears in the process; the reliability of the system is improved by stochastic optimal control, and the first-passage time is delayed. GMF-SMA composite plate combines the advantages of both GMF and SMA, and can reduce vibration through passive control and active control effectively. The results of this paper are helpful to application of GMF-SMA composite plate in engineering fields.

The stochastic optimal nonlinear control of coupled adjacent building structures is studied based on the stochastic dynamical programming principle and the stochastic averaging method. The coupled structures with control devices under random seismic excitation are first condensed to form a reduced-order structural model for the control analysis. The stochastic averaging method is applied to the reduced model to yield stochastic differential equations for structural modal energies as controlled diffusion processes. Then a dynamical programming equation for the energy processes is established based on the stochastic dynamical programming principle, and solved to determine the optimal nonlinear control law. The seismic response mitigation of the coupled structures is achieved through the structural energy control and the dimension of the optimal control problem is reduced. The seismic excitation spectrum is taken into account according to the stochastic dynamical programming principle. Finally, the nonlinear controlled structural response is predicted by using the stochastic averaging method and compared with the uncontrolled structural response to evaluate the control efficacy. Numerical results are given to demonstrate the response mitigation capabilities of the proposed stochastic optimal control method for coupled adjacent building structures.

A stochastic minimax optimal time-delay state feedback control strategy for uncertain quasi-integrable Hamiltonian systems is proposed. First, a stochastic optimal state feedback control problem of uncertain quasi-integrable Hamiltonian system with time delay in feedback control subjected to Gaussian white noise is formulated. Then, the time-delayed state feedback control forces are approximated by the control forces without time delay and the original problem is converted into a stochastic optimal state feedback control problem of an uncertain system without time delay. After that, by following a procedure based on the stochastic averaging method and stochastic differential game, the worst-case disturbances and the optimal controls are obtained from solving a Hamiltonian–Jacobi–Isaacs (HJI) equation. As an example, the stochastic minimax optimal state feedback control of a Duffing oscillator with parametric disturbances and time-delayed feedback control is worked out in detail to illustrate the procedure and effectiveness of the proposed control strategy.

This article examines an economic growth model that expresses the interaction between production, technology stock, and research and development (R&D) investments. The goal of this study is to maximize production. Considering the presence of Gaussian white noises, this model is reformulated as a stochastic optimal control problem, where the R&D investment rate is defined as the control input. We aim to explore the efficiency of Kalman filtering approaches for solving this problem. Here, the extended Kalman filter (EKF) and unscented Kalman filter (UKF) are applied for state estimation. The state equation linearization is made in the EKF, while the unscented transform is taken in the UKF for generating a set of sigma points. These approaches aim to estimate the state dynamics from different perspectives. With these state estimates, two different computational algorithms are proposed, the EKF for state-control (EKF4SC) and UKF for state-control (UKF4SC) algorithms. The optimal control policy is designed to minimize the cost function. For illustration, the model's parameters are considered in the simulation experiment. The simulation results showed that the UKF has higher accuracy in the state estimation with the smallest mean squares of error compared with the EKF. Moreover, the optimal control policy based on the state estimate generated from the UKF could optimize the cost function of the problem. Hence, the results show the efficiency of the algorithms proposed. In conclusion, the application of the Kalman filtering algorithms to the economic growth model is presented. The significance of this study is to provide a stochastic optimal control model for the economic growth problem and to suggest an efficient computational approach for solving this stochastic optimal control problem.

Reliability-based stochastic optimal control of frame building under near-fault ground motions

Optimal control simulations have shown that both musculoskeletal dynamics and physiological noise are important determinants of movement. However, due to the limited efficiency of available computational tools, deterministic simulations of movement focus on accurately modelling the musculoskeletal system while neglecting physiological noise, and stochastic simulations account for noise while simplifying the dynamics. We took advantage of recent approaches where stochastic optimal control problems are approximated using deterministic optimal control problems, which can be solved efficiently using direct collocation. We were thus able to extend predictions of stochastic optimal control as a theory of motor coordination to include muscle coordination and movement patterns emerging from non-linear musculoskeletal dynamics. In stochastic optimal control simulations of human standing balance, we demonstrated that the inclusion of muscle dynamics can predict muscle co-contraction as minimal effort strategy that complements sensorimotor feedback control in the presence of sensory noise. In simulations of reaching, we demonstrated that nonlinear multi-segment musculoskeletal dynamics enables complex perturbed and unperturbed reach trajectories under a variety of task conditions to be predicted. In both behaviors, we demonstrated how interactions between task constraint, sensory noise, and the intrinsic properties of muscle influence optimal muscle coordination patterns, including muscle co-contraction, and the resulting movement trajectories. Our approach enables a true minimum effort solution to be identified as task constraints, such as movement accuracy, can be explicitly imposed, rather than being approximated using penalty terms in the cost function. Our approximate stochastic optimal control framework predicts complex features, not captured by previous simulation approaches, providing a generalizable and valuable tool to study how musculoskeletal dynamics and physiological noise may alter neural control of movement in both healthy and pathological movements.

A minimax stochastic optimal semi-active control strategy for stochastically excited quasi-integrable Hamiltonian systems with parametric uncertainty by using magneto-rheological (MR) dampers is proposed. Firstly, the control problem is formulated as an n-degree-of-freedom (DOF) controlled, uncertain quasi-integrable Hamiltonian system and the control forces produced by MR dampers are split into the passive part and the semi-active part. Then the passive part is incorporated into the uncontrolled system. After that, the stochastic optimal semi-active control problem is solved by applying the minimax stochastic optimal control strategy based on the stochastic averaging method and stochastic differential game. The worst-case disturbances and the optimal controls are obtained by the minimax dynamical programming equation with the constraints of disturbance bounds and MR damper dynamics. Finally, the system response and the control performance are evaluated by using Monte Carlo simulation. An example of a two-DOF system with coupling damping and parametric uncertainty under Gaussian white noise excitations is worked out in detail to illustrate the procedure and effectiveness of the proposed control strategy, which is also compared with the clipped linear-quadratic-Gaussian control strategy to show the advantages.

In this paper, we use the solutions of forward-backward stochastic differential equations with random jumps to get the optimal controls for forward-backward stochastic linear quadratic optimal control problem and the Nash equilibrium point for nonzero sum differential games problem.

In this paper, the stochastic optimal control problems, which frequently occur in economic and finance are investigated. First, using Bellman’s dynamic programming method the stochastic optimal control problems are converted to Hamilton-Jacobi-Bellman (HJB) equation. Then, obtained HJB equation is solved through the method of separation of variables by guessing a solution via its terminal condition. Also, the non-linear optimal feedback control law is constructed. Finally, the solution procedure is illustrated for solving some examples that two of them are financial models. In fact, to highlight the applications of stochastic optimal control problems in financial mathematics, some financial models are presented.

Innovative procedures for the stochastic optimal time-delay control and stabilization are proposed for a quasi-integrable Hamiltonian system subject to Gaussian white noises. First, the problem of stochastic optimal control with time delay is formulated. Then, the problem is converted into a stochastic optimal control without time delay, and the converted control problems are then solved by applying the stochastic averaging method and the stochastic dynamical programming principle. The optimal time-delay stabilization of quasi-integrable Hamiltonian systems is formulated as an ergodic control with a cost function determined by minimizing the largest Lyapunov exponent of the controlled system. As an example, a two-degree-of-freedom quasi-integrable Hamiltonian system with time delay in feedback control forces is investigated in detail to illustrate the procedures and their effectiveness.KeywordsTime-delay feedback controlStochastic averaging methodStochastic optimal controlFeedback stabilization

The principal techniques used up to now for the analysis of stochastic adaptive control systems have been (i) super-martingale (often called stochastic Lyapunov) methods and (ii) methods relying upon the strong consistency of some parameter estimation scheme. Optimal stochastic control and filtering methods have also been employed. Although there have been some successes, the extension of these techniques to a broad class of adptive control problems, including the case of time varying parameters, has been difficult. In this paper a new approach is adopted: If an underlying Markovian state space system for the controlled process is available, and if this process possesses stationary transition probabilities, then the powerful ergodic theory of Markov processes may be applied. Subject to technical conditions one may deduce (amongst other facts) (i) the existence of an invariant measure µ∞ for the process and (ii) the convergence almost surely of the sample averages of a function of the state process (and of its expectation) to its conditional expectation [µ∞] with respect to a sub-σ-field of invariant sets ΣI. The technique is illustrated by an application to a previously unsolved problem involving a linear system with unbounded random time varying parameters. Work suppoted by Canada NSERC Grant No.: 1329 and a UK SERC Visiting Research Fellowship.

We provide a data-driven framework for optimal control of a continuous-time control-affine stochastic dynamical system. The proposed framework relies on the linear operator theory involving Perron-Frobenius (P-F) and Koopman operators. Our first result involving the P-F operator provides a convex formulation to the optimal control problem in the dual space of densities. This convex formulation of the stochastic optimal control problem leads to an infinite-dimensional convex program. The finite-dimensional approximation of the convex program is obtained using a data-driven approximation of the linear operators. We provide rigorous error bounds and convergence rate for the data-driven approximation of the linear operators and comment on the convergence rate of the optimal control w.r.t. data size. Our second results demonstrate using the Koopman operator, dual to the P-F operator, for the stochastic optimal control design. We show that the Hamilton Jacobi Bellman (HJB) equation can be expressed using the Koopman operator. We provide an iterative procedure along the lines of a popular policy iteration algorithm based on the data-driven approximation of the Koopman operator for solving the HJB equation. The two formulations, namely the convex formulation involving the P-F operator and the Koopman-based formulation using the HJB equation, can be viewed as dual to each other where the duality follows due to the dual nature of the P-F and Koopman operators. Finally, we present examples to demonstrate the efficacy of the developed framework and verify the convergence rates for the operator and optimal control numerically.

The aim of this chapter is to develop methods and algorithms for determining the optimal solutions of stochastic discrete control problems and Markov decision problems with an infinite time horizon.

Mathematical formulation and numerical solutions of an optimal Dirichlet boundary control problem for the Boussinesq equations are considered. The solution of the optimal control problem is obtained by adjusting of the temperature on the boundary. We analyze finite element approximations. A gradient method for the solution of the discrete optimal control problem is presented and analyzed. Finally, the results of some computational experiments are presented.