Stochastic Systems with Averaging in the Scheme of Diffusion Approximation

Markovian jumping systems have arisen naturally in the mathematical modeling of phenomena spanning disciplines in the social sciences, natural sciences, and engineering. This kind of stochastic dynamical systems can be employed to model the dynamics when parameters are subject to random abrupt changes due to sudden environment changes, subsystem switching, system noises, executor faults, and so forth. Much attention has been given to modeling, optimization, and real applications of such stochastic dynamical systems in the literature in recent years. As the advanced control and optimization will provide a basis for the design and application of such stochastic systems, these advanced techniques would result in substantial and sustainable benefits. The accepted papers in this special issue include stochastic stability, stabilization, stochastic control optimization, system modeling and identification methods, predictive control, signal processing, robust filtering, multiagent systems, networked control systems, time-delayed systems, neural networks, the Takagi-Sugeno fuzzy systems, simulated annealing, and fault detection methods.

In this chapter we consider stochastic stability of Markov and semi-Markov processes, random evolutions and stochastic evolutionary systems, using analogue of Dynkin’s formula for them and martingale theory.

The authors study the asymptotic behavior of stochastic evolutionary systems with Markov-modulated Poisson perturbation in the diffusion approximation scheme. They consider combining the Poisson process with the Markov process, which allows for describing random transitions between different modes of evolution. The ergodic properties of the Markov-modulated Poisson process that ensure the stable behavior of the system on average are presented. Boundary generators for the original system of stochastic differential equations are constructed. The results allow for studying stochastic optimization and optimal control problems. Keywords: stochastic evolution, Markov-modulated Poisson process, diffusion approximation schema.

Conditions are given under which a semi-Markov process $Z(t)$ can be obtained from a Markov process $Y(t)$ by a time change (i.e. $Z(t) = Y(\gamma(t))$). Estimates are given for $P\{\sup_{s\leqq t} |s - \gamma(s)| > \varepsilon\}$ and the construction is used to give conditions under which a sequence of semi-Markov processes will have the same convergence properties as the corresponding sequence of Markov processes.

Chapter 10 - Diffusion approximation

6 Dependence notions in reliability theory

Automated verification and synthesis of stochastic hybrid systems: A survey

In this chapter, we construct stochastic integral operator equations for the limiting, in the sense of weak convergence, semi-Markov random evolutions in the diffusion approximation.

Markov and Semi-Markov Processes with Fuzzy States. Part 1. Markov Processes

We show that diffusion approximations, including modified diffusion approximations, can be problematic since the proper choice of local boundary conditions (if any exist) is not obvious. For a class of Markov processes in one dimension, we show that to leading order it is proper to use a diffusion (Fokker-Planck) approximation to compute mean exit times with a simple absorbing boundary condition. However, this is only true for the leading term in the asymptotic expansion of the mean exit time. Higher order correction terms do not, in general, satisfy simple absorbing boundary conditions. In addition, the diffusion approximation for the calculation of mean exit times is shown to break down as the initial point approaches the boundary, and leads to an increasing relative error. By introducing a boundary layer, we show how to correct the diffusion approximation to obtain a uniform approximation of the mean exit time. We illustrate these considerations with a number of examples, including a jump process which leads to Kramers' diffusion model. This example represents an extension to a multivariate process.

The effect of averaging for semi-Markov random evolutions appears in the case where semi-Markov processes have many (infinitely many, in the limit) changes of states on a finite time interval. The processes determining the states of semi-Markov random evolutions (the switched processes) and the processes describing the changes of states of the random medium (the switching processes) should be considered in different time scales. A small parameter e > 0 is used as a scaling factor.

A robust and efficient triangulation-based optimization algorithm for stochastic black-box systems

This paper discusses the asymptotic behavior of the stochastic evolutionary system underthe Markov-modulated Poisson perturbations in an averaging schema. Such a perturbationprocess combines the Poisson process with the Markov process that modulates the intensityof jumps. This allows us to model systems with transitions between different modes orrare but significant jumps. Initially, the asymptotic properties of the Markov-modulated Poissonperturbation are investigated. For this purpose, we build the generator for the limitprocess solving the singular perturbation problem for the original process. Then we introducea compensated Poisson process with a zero mean value, and it is used to center the jumps.The stochastic evolutionary system perturbed by the compensated Poisson process with anadditional jump size function is described. We build the generator for an evolution process andinvestigate its asymptotic properties. Solving the singular perturbation problem we obtain theform of the limit process and its generator. This allows us to formulate and prove the theoremabout weak convergence of the evolution process to the averaged one. The limit process forthe stochastic evolutionary system at increasing time intervals is determined by the solutionof a deterministic differential equation. The obtained result makes it possible to study therate of convergence of the perturbed process to the limit one, as well as to consider stochasticapproximation and optimization procedures for problems in which the system is described byan evolutionary equation with the Markov-modulated Poisson perturbation.

Stability analysis of semi-Markov switching stochastic mode-dependent delay systems with unstable subsystems

A Markov Renewal Process (M.R.P.) is a process similar to a Markov chain, except that the time required to move from one state to another is not fixed, but is a random variable whose distribution may depend on the two states between which the transition is made. For an M.R.P. ofm (<∞) states we derive a goodness-of-fit test for a hypothetical matrix of transition probabilities. This test is similar to the test Bartlett has derived for Markov chains. We calculate the first two moments of the test statistic and modify it to fit the moments of a standard χ2. Finally, we illustrate the above procedure numeerically for a particular case of a two-state M.R.P.