A new approach to stochastic adaptive control

A computational approach is taken to solve the optimal nonlinear stochastic control problem. The approach is to systematically solve the stochastic dynamic programming equations forward in time, using a nested stochastic approximation technique. Although computationally intensive, this provides a straightforward numerical solution for this class of problems and provides an alternative to the usual dimensionality problem associated with solving the dynamic programming equations backward in time. It is shown that the cost degrades monotonically as the complexity of the algorithm is reduced. This provides a strategy for suboptimal control with clear performance/computation tradeoffs. A numerical study focusing on a generic optimal stochastic adaptive control example is included to demonstrate the feasibility of the method.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The stochastic optimal nonlinear control of coupled adjacent building structures is studied based on the stochastic dynamical programming principle and the stochastic averaging method. The coupled structures with control devices under random seismic excitation are first condensed to form a reduced-order structural model for the control analysis. The stochastic averaging method is applied to the reduced model to yield stochastic differential equations for structural modal energies as controlled diffusion processes. Then a dynamical programming equation for the energy processes is established based on the stochastic dynamical programming principle, and solved to determine the optimal nonlinear control law. The seismic response mitigation of the coupled structures is achieved through the structural energy control and the dimension of the optimal control problem is reduced. The seismic excitation spectrum is taken into account according to the stochastic dynamical programming principle. Finally, the nonlinear controlled structural response is predicted by using the stochastic averaging method and compared with the uncontrolled structural response to evaluate the control efficacy. Numerical results are given to demonstrate the response mitigation capabilities of the proposed stochastic optimal control method for coupled adjacent building structures.

Reliability-based stochastic optimal control of frame building under near-fault ground motions

Stochastic optimal control for aircraft conflict resolution under wind uncertainty

This paper is devoted to exposition of some results that are related to numerical synthesis of stochastic optimal control systems and also to numerical analysis of different approximate analytical synthesis methods. This trend of investigations was desoribed in detail in the paper

Stochastic bifurcation characteristics and optimal control of giant magnetostrictive film (GMF)-shape memory alloy (SMA) composite plate subjected to in-plane stochastic excitation were studied in this paper. Nonlinear difference item was introduced to interpret the hysteretic phenomena of both GMF and SMA, and then the nonlinear dynamic model of GMF-SMA composite plate subjected to in-plane stochastic excitation was developed. The stochastic stability of the system was analyzed, and the condition of stochastic Hopf bifurcation was discussed. The reliability function of the system was solved from backward Kolmogorov equation, and then the probability density of the first-passage time was obtained. Finally, the stochastic optimal control strategy was obtained in stochastic dynamic programming method. Numerical simulation shows that the stability of the trivial solution varies with bifurcation parameters, and stochastic Hopf bifurcation appears in the process; the reliability of the system is improved by stochastic optimal control, and the first-passage time is delayed. GMF-SMA composite plate combines the advantages of both GMF and SMA, and can reduce vibration through passive control and active control effectively. The results of this paper are helpful to application of GMF-SMA composite plate in engineering fields.

The topic is a model of stochastic optimal control with Markovian base, which is determined by a set of objects. If certain states are accessible for direct observation, the problem of dynamic decision making in conditions of the type of risk outlet created by accidenting of states takes place. Under given conditions, game ideology and constructive methods of successive identification and adaptive decision-making are developing. The best decisions are understood in the sense of dynamic balances of Shtakelberg. Together the described results realize system methodology of decision-making. It permits to investigate problem of decision-making not in particular suppositions of concrete tasks but as a common problem for sufficiently broad class of operated systems marked by appropriate postulates and bases. That opens possibility to describe and solve problems which could not be solved before.

A comprehensive treatment of stochastic systems beginning with the foundations of probability and ending with stochastic optimal control. The book divides into three interrelated topics. First, the concepts of probability theory, random variables and stochastic processes are presented, which leads easily to expectation, conditional expectation, and discrete time estimation and the Kalman filter. With this background, stochastic calculus and continuous-time estimation are introduced. Finally, dynamic programming for both discrete-time and continuous-time systems leads to the solution of optimal stochastic control problems resulting in controllers with significant practical application. This book will be valuable to first year graduate students studying systems and control, as well as professionals in this field.

Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses

Innovative procedures for the stochastic optimal time-delay control and stabilization are proposed for a quasi-integrable Hamiltonian system subject to Gaussian white noises. First, the problem of stochastic optimal control with time delay is formulated. Then, the problem is converted into a stochastic optimal control without time delay, and the converted control problems are then solved by applying the stochastic averaging method and the stochastic dynamical programming principle. The optimal time-delay stabilization of quasi-integrable Hamiltonian systems is formulated as an ergodic control with a cost function determined by minimizing the largest Lyapunov exponent of the controlled system. As an example, a two-degree-of-freedom quasi-integrable Hamiltonian system with time delay in feedback control forces is investigated in detail to illustrate the procedures and their effectiveness.KeywordsTime-delay feedback controlStochastic averaging methodStochastic optimal controlFeedback stabilization

Optimal control simulations have shown that both musculoskeletal dynamics and physiological noise are important determinants of movement. However, due to the limited efficiency of available computational tools, deterministic simulations of movement focus on accurately modelling the musculoskeletal system while neglecting physiological noise, and stochastic simulations account for noise while simplifying the dynamics. We took advantage of recent approaches where stochastic optimal control problems are approximated using deterministic optimal control problems, which can be solved efficiently using direct collocation. We were thus able to extend predictions of stochastic optimal control as a theory of motor coordination to include muscle coordination and movement patterns emerging from non-linear musculoskeletal dynamics. In stochastic optimal control simulations of human standing balance, we demonstrated that the inclusion of muscle dynamics can predict muscle co-contraction as minimal effort strategy that complements sensorimotor feedback control in the presence of sensory noise. In simulations of reaching, we demonstrated that nonlinear multi-segment musculoskeletal dynamics enables complex perturbed and unperturbed reach trajectories under a variety of task conditions to be predicted. In both behaviors, we demonstrated how interactions between task constraint, sensory noise, and the intrinsic properties of muscle influence optimal muscle coordination patterns, including muscle co-contraction, and the resulting movement trajectories. Our approach enables a true minimum effort solution to be identified as task constraints, such as movement accuracy, can be explicitly imposed, rather than being approximated using penalty terms in the cost function. Our approximate stochastic optimal control framework predicts complex features, not captured by previous simulation approaches, providing a generalizable and valuable tool to study how musculoskeletal dynamics and physiological noise may alter neural control of movement in both healthy and pathological movements.

The aim of this paper is to address optimality of control strategies for stochastic discrete time control systems subject to conditional distribution uncertainty. This type of uncertainty is motivated from the fact that the value function involves expectation with respect to the conditional distribution. The issues which will be discussed are the following. 1) Optimal stochastic control systems subject to conditional distribution uncertainty, 2) optimality criteria for stochastic control systems with conditional distribution uncertainty, including principle of optimality and dynamic programming.

In this paper, considering the simultaneous influence of multi-source disturbances, system modeling uncertainties and input–output constraints, an adaptive robust attitude tracking control scheme is proposed for near space vehicles (NSVs) which is expressed as a stochastic nonlinear system. A multi-dimensional Taylor polynomial network (MTPN) is utilized to handle the system uncertainties, and the nonlinear disturbance observer (NDO) based on MTPN is designed to estimate the external disturbances. Furthermore, by constructing the auxiliary system to tackle the input saturation and introducing the Tan-type barrier Lyapunov function (TBLF) to solve the output constraint, the constrained control strategy can be obtained. Combining with backstepping control (BC) technique and stochastic control method, an adaptive robust stochastic control scheme is developed based on NDO, MTPN, and auxiliary system, and the closed-loop system stability in the sense of probability is analyzed based on stochastic Lyapunov stability theory. Finally, numerical simulations further demonstrate the feasibility of the proposed tracking control scheme.

The control of probabilistic Boolean networks as a model of genetic regulatory networks is formulated as an optimal stochastic control problem and has been solved using dynamic programming; however, the proposed methods fail when the number of genes in the network goes beyond a small number. There are two dimensionality problems. First, the complexity of optimal stochastic control exponentially increases with the number of genes. Second, the complexity of estimating the probability distributions specifying the model increases exponentially with the number of genes. We propose an approximate stochastic control method based on reinforcement learning that mitigates the curses of dimensionality and provides polynomial time complexity. Using a simulator, the proposed method eliminates the complexity of estimating the probability distributions and, because the method is a model-free method, it eliminates the impediment of model estimation. The method can be applied on networks for which dynamic programming cannot be used owing to computational limitations. Experimental results demonstrate that the performance of the method is close to optimal stochastic control.

In this paper, we use the solutions of forward-backward stochastic differential equations with random jumps to get the optimal controls for forward-backward stochastic linear quadratic optimal control problem and the Nash equilibrium point for nonzero sum differential games problem.