On Successive Approximation of Optimal Control of Stochastic Dynamic Systems

The stability of stochastic dynamic systems have widespread application prospects and greater theoretical significance. Its theory are mainly applied to engineering control, communication equipment, military technology, biology, finance, etc. In this article, we consider the stability of a Class of nonlinear impulsive neutral stochastic differential dynamic systems. A new set of conditions proving the mean square stability of this nonlinear impulsive neutral stochastic dynamic systems are derived by means of the Banach fixed point theorem. Some well-known results are improved and generalized.

Mean-square practical stability for stochastic large-scale dynamical systems is studied. The vector Lyapunov functions and the basic comparison principle of stochastic systems as well as the decomposition-aggregation method for large-scale systems were used to obtain results for various types of mean-square practical stability of nonlinear stochastic large-scale dynamical systems. The objective is to analyze stochastic large-scale systems in terms of their lower-order subsystems and in terms of their interconnecting structure. These results make it possible to determine the mean-square practical stability of stochastic large-scale dynamical systems by testing the corresponding practical stability of the auxiliary deterministic systems. >

Adaptive fault tolerant tracking control for a class of stochastic nonlinear systems with output constraint and actuator faults

The Markov chain approximation method is an effective and widely used approach for computing optimal values and controls for stochastic systems. It was extended to nonlinear (and possibly reflected) diffusions with delays in a recent book. The path, control, and reflection terms can all be delayed. In models of communications systems, reflection terms correspond to buffer overflows, and to the adjustments needed to keep the buffers and rates nonnegative and bounded. The book concentrated on convergence proofs for general algorithms of promising forms. If the control and/or reflection terms are delayed, the memory requirements can make the problem intractable, since these have little or no regularity. For such models, the problem was recast in terms of a "wave" or "transportation" equation, to get practical algorithms with very much reduced computational needs. The overall approach to getting usable algorithms is outlined. The methods are applied to an ideal model of a communications system with transportation delay. and it is seen that nearly optimal controls for appropriately chosen criteria can improve the performance considerably and be robust in the face of changing operating conditions. The data illustrate the potential usefulness of numerical methods for nonlinear stochastic systems with delays. It opens up new avenues of control for consideration.

Adaptive neural optimal tracking control of stochastic nonstrict-feedback nonlinear systems with output constraints

SummaryIn this paper, we derive stability margins for optimal and inverse optimal stochastic feedback regulators. Specifically, gain, sector, and disk margin guarantees are obtained for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton‐Jacobi‐Bellman controllers that minimize a nonlinear‐nonquadratic performance criterion with cross‐weighting terms. Furthermore, using the newly developed notion of stochastic dissipativity, we derive a return difference inequality to provide connections between stochastic dissipativity and optimality of nonlinear controllers for stochastic dynamical systems. In particular, using extended Kalman‐Yakubovich‐Popov conditions characterizing stochastic dissipativity, we show that our optimal feedback control law satisfies a return difference inequality predicated on the infinitesimal generator of a controlled Markov diffusion process if and only if the controller is stochastically dissipative with respect to a specific quadratic supply rate.

In this study, we investigate multi-player noncooperative minmax H∞ target tracking game strategy with conflicting target strategies and cooperative H∞ target tracking game strategy with common target strategy of nonlinear mean-field stochastic jump diffusion (MFSJD) system with external disturbance. Due to the nonlinear terms and mean-field (average) behaviors in the stochastic nonlinear MFSJD system and minmax H∞ payoff function, the multi-player noncooperative and cooperative minmax H∞ mean-field game strategy of nonlinear MFSJD system are more difficult than the linear MFSJD system and conventional nonlinear stochastic system. To avoid solving complex Hamilton Jacobi Isaacs inequalities (HJIIs) of multi-player noncooperative minmax H∞ mean-field target tracking game strategy of nonlinear MFSJD system, the nonlinear MFSJD system is interpolated by a set of local linearized MFSJD system through smoothing functions by the global linearization method. Then the multi-player noncooperative minmax H∞ nonlinear mean-field target tracking game strategy design can be transformed to a linear matrix inequalities (LMIs)-constrained multi-objective optimization problem (MOP). The LMIs-constrained MOP could be efficiently solved by the help of the proposed LMIs-constrained multi-objective evolution algorithm (MOEA). We can prove that the Pareto optimal solution of LMIs-constrained MOP is the Nash equilibrium solution of noncooperative minmax H∞ mean-field target tracking strategy of nonlinear mean-field MFSJD system. Further, the cooperative minmax H∞ mean-field common target tracking strategy of nonlinear mean-field stochastic system is reduced to an LMIs-constrained single-objective optimization problem (SOP). Finally, two simulation examples of cyber-financial mean-field systems are given to illustrate the design procedure and compare the efficacies of the proposed noncooperative and cooperative minmax H∞ mean-field target tracking strategy of nonlinear MFSJD systems.

A mathematical framework for new fault detection schemes in nonlinear stochastic continuous-time dynamical systems

Stability analysis for a class of stochastic delay nonlinear systems driven by G-Brownian motion

This note is devoted to study the stabilization problem of stochastic nonlinear delay systems with exogenous disturbances and the event-triggered feedback control. By introducing the notation of input-to-state practical stability and an event-triggered strategy, we establish the input-to-state practically exponential mean-square stability of the suggested system. Moreover, we investigate the stabilization result by designing the feedback gain matrix and the event-triggered feedback controller, which is expressed in terms of linear matrix inequalities. Also, the lower bounds of interexecution times by the proposed event-triggered control method are obtained. Finally, an example is given to show the effectiveness of the proposed method. Compared with a large number of results for discrete-time stochastic systems, only a few results have appeared on the event-triggered control for continuous-time stochastic systems. In particular, there have been no published papers on the event-triggered control for continuous-time stochastic delay systems. This note is a first try to fill the gap on the topic.

Sufficient conditions for the controllability of nonlinear stochastic fractional boundary control systems are established. The equivalent integral equations are derived for both linear and nonlinear systems, and the control function is given in terms of the pseudoinverse operator. The Banach contraction mapping theorem is used to obtain the result. A controllability result for nonlinear stochastic fractional integrodifferential systems is also attained. Examples are included to illustrate the theory.

Stochastic optimal control based on reliability plays a crucial role in mitigating structural failure and ensuring safe operation. For general stochastic vibration systems, a data-driven method for reliability-based optimal control is proposed based on random state data. Firstly, the feedback control is split into conservative and dissipative components in coherence with physical intuition, and each component is expanded upon using pre-selected basic functions, resulting in a modified system with undetermined coefficients. Then, from discrete random samples, the expressions for the probability densities of the first-passage time and the stationary response can be identified, explicitly including system and excitation parameters, as well as initial and safety boundary conditions. Finally, a performance index, combining system reliability and control cost, is constructed and can be explicitly expressed in terms of the undetermined coefficients of the control forces. The optimal control problem for determining the optimal control forces is reformulated as an optimization problem for determining the coefficients that minimize the performance index. This unconstrained optimization problem can be easily solved. Three typical nonlinear stochastic systems, a controlled Duffing oscillator, a controlled nonlinear hysteretic system, and a controlled 2-DOF nonlinear dynamical system, are given to illustrate the validity and accuracy of the proposed data-driven method. Additionally, the control performance of the optimal control strategy in improving the system reliability is also discussed.

The purpose of this chapter is to provide a detailed probabilistic analysis of the optimal control of nonlinear stochastic dynamical systems of McKean-Vlasov type. We tackle the characterization and construction of solutions of this special type of optimal control problem by means of forward-backward stochastic differential equations. Because of the presence of the distribution of the controlled state in the coefficients, the approach based on the Pontryagin stochastic maximum principle requires special attention. We provide a version of this maximum principle based on the differential calculus for functions of probability measures introduced and developed in Chapter 5 We test the results of the analysis on linear quadratic models and a few other models already considered in the framework of mean field games. Finally, we highlight the similarities and the differences between this problem and MFG problems with which it is often confused.

The purpose of this paper is to provide a detailed probabilistic analysis of the optimal control of nonlinear stochastic dynamical systems of McKean–Vlasov type. Motivated by the recent interest in mean-field games, we highlight the connection and the differences between the two sets of problems. We prove a new version of the stochastic maximum principle and give sufficient conditions for existence of an optimal control. We also provide examples for which our sufficient conditions for existence of an optimal solution are satisfied. Finally we show that our solution to the control problem provides approximate equilibria for large stochastic controlled systems with mean-field interactions when subject to a common policy.

We propose a framework for safe reinforcement learning that can handle stochastic nonlinear dynamical systems. We focus on the setting where the nominal dynamics are known, and are subject to additive stochastic disturbances with known distribution. Our goal is to ensure the safety of a control policy trained using reinforcement learning, e.g., in a simulated environment. We build on the idea of model predictive shielding (MPS), where a backup controller is used to override the learned policy as needed to ensure safety. The key challenge is how to compute a backup policy in the context of stochastic dynamics. We propose to use a tube-based robust nonlinear model predictive controller (NMPC) as the backup controller. We estimate the tubes using sampled trajectories, leveraging ideas from statistical learning theory to obtain high-probability guarantees. We empirically demonstrate that our approach can ensure safety in stochastic systems, including cart-pole and a non-holonomic particle with random obstacles.