Nonlinearly Perturbed Markov Chains and Large Deviations for Lifetime Functionals

We present several numerical approaches to investigate rare events in stochastic systems, with a specific focus on application to biological models. We first present several aspects concerning variance reduction of Monte-Carlo methods, with a focus on importance sampling. We show how these methods can be applied to basic biological models. We show that these techniques can be useful in dealing with multiscale continuous-time Markov chains that are important in the context of biochemical reaction networks. We treat with more detail the problem of first passage time for a linear diffusion process, arising from the integrate-and-fire neuron model, to show the kind of mathematical problems that may arise when trying to design an optimal importance sampling scheme. This leads to the observation that large deviation theory can be helpful to solve these questions. We also review a numerical method designed to estimate large deviations quantities such as the quasipotential and the optimal path and apply this method to estimate numerically the quasipotential of the Morris–Lecar neuron model perturbed by small noise.

Importance sampling (IS) is a change-of-measure technique for speeding up the simulation of rare events in stochastic systems. In this paper, we presented an approach uses Importance Sampling technique for efficient estimation of software reliability via Markov software usage models in statistical testing. By suitable changes of the probabilities of state transitions during test, an iterative method based on the Ali-Silvey distance is proposed for this choice, and an unbiased reliability estimator with zero variance is obtained. A learning algorithm for the computation of optimal transition probabilities of the Markov chain usage model is also presented and experimental results of this algorithm are reported.

Importance sampling is a change-of-measure technique for speeding up the simulation of rare events in stochastic systems. In this paper the authors establish a number of properties characterizing optimal importance sampling measures for Markovian systems. The authors show how these properties may be used to compute the optimal measure and give specific results for a tandem queueing system. The authors' approach has no apparent computational advantage over other direct methods, but it suggests a number of heuristic approximations which may lead to computationally attractive methods.

1. The main notions.- 2. The main lemmas.- 2.1. General lemmas on the approximation of distribution of an arbitrary random variable by the normal distribution.- 2.2. Proof of lemmas 2.1-2.4.- 3. Theorems on large deviations for the distributions of sums of independent random variables.- 3.1. Theorems on large deviations under Bernstein's condition.- a) Sums of non-identically distributed random variables.- b) Sums of weighted random variables.- 3.2. A theorem of large deviations in terms of Lyapunov's fractions.- 4. Theorems of large deviations for sums of dependent random variables.- 4.1. Estimates of the kth order centered moments of random processes with mixing.- 4.2. Estimates of mixed cumulants of random processes with mixing.- 4.3. Estimates of cumulants of sums of dependent random variables.- 4.4. Theorems and inequalities of large deviations for sums of dependent random variables.- 5. Theorems of large deviations for polynomial forms, multiple stochastic integrals and statistical estimates.- 5.1. Estimates of cumulants and theorems of large deviations for polynomial forms, polynomial Pitman estimates and U-statistics.- 5.2. Cumulants of multiple stochastic integrals and theorems of large deviations.- 5.3. Large deviations for estimates of the spectrum of a stationary sequence.- 6. Asymptotic expansions in the zones of large deviations.- 6.1. Asymptotic expansion for distribution density of an arbitrary random variable.- 6.2. Estimates for characteristic functions.- 6.3. Asymptotic expansion in the Cramer zone for distribution density of sums of independent random variables.- 6.4. Asymptotic expansions in integral theorems with large deviations.- 7. Probabilities of large deviations for random vectors.- 7.1. General lemmas on large deviations for a random vector with regular behaviour of cumulants.- 7.2. Theorems on large deviations for sums of random vectors and quadratic forms.- a) Sums of non-identically distributed random vectors.- b) Sums of weighted random vectors.- c) Sums of random number of random vectors.- d) Quadratic forms.- Appendices.- Appendix 1. Proof of inequalities for moments and Lyapunov's fractions.- Appendix 2. Proof of the lemma on the representation of cumulants.- Appendix 3. Leonov - Shiryaev's formula.- References.

On asymptotic convergence and boundedness of stochastic systems with time-delay

Markovian jumping systems have arisen naturally in the mathematical modeling of phenomena spanning disciplines in the social sciences, natural sciences, and engineering. This kind of stochastic dynamical systems can be employed to model the dynamics when parameters are subject to random abrupt changes due to sudden environment changes, subsystem switching, system noises, executor faults, and so forth. Much attention has been given to modeling, optimization, and real applications of such stochastic dynamical systems in the literature in recent years. As the advanced control and optimization will provide a basis for the design and application of such stochastic systems, these advanced techniques would result in substantial and sustainable benefits. The accepted papers in this special issue include stochastic stability, stabilization, stochastic control optimization, system modeling and identification methods, predictive control, signal processing, robust filtering, multiagent systems, networked control systems, time-delayed systems, neural networks, the Takagi-Sugeno fuzzy systems, simulated annealing, and fault detection methods.

As a primary branch of Operations Research, Queueing Theory models and analyzes engineering systems with random fluctuations. With the development of internet and computation techniques, the engineering systems today are much bigger in scale and more complicated in structure than 20 years ago, which raises numerous new problems to researchers in the field of queueing theory. The aim of this thesis is to explore new methods and tools, from both algorithmic and analytical perspectives, that are useful to solve such problems. In Chapter 1 and 2, we introduce some techniques of asymptotic analysis that are relatively new to queueing applications in order to give more accurate probabilistic characterization of queueing models with large scale and complicated structure. In particular, Chapter 1 gives the first functional large deviation result for infinite-server system with general inter-arrival and service times. The functional approach we use enables a nice description of the whole system over the entire time horizon of interest, which is important in real problems. In Chapter 2, we construct a queueing model for the so-called limit order book that is used in main financial markets worldwide. We use an asymptotic approach called multi-scale modeling to disentangle the complicated dependence among the elements in the trading system and to reduce the model dimensionality. The asymptotic regime we use is inspired by empirical observations and the resulting limit process explains and reproduces stylized features of real market data. Chapter 2 also provides a nice example of novel applications of queueing models in systems, such as the electronic trading system, that are traditionally outside the scope of queueing theory. Chapter 3 and 4 focus on stochastic simulation methods for performance evaluation of queueing models where analytic approaches fail. In Chapter 3, we develop a perfect sampling algorithm to generate exact samples from the stationary distribution of stochastic fluid networks in polynomial time. Our approach can be used for time-varying networks with general inter-arrival and service times, whose stationary distributions have no analytic expression. In Chapter 4, we focus on the stochastic systems with continuous random fluctuations, for instance, the workload arrives to the system in continuous flow like a Levy process. We develop a general framework of simulation algorithms featuring a deterministic error bound and an almost square root convergence rate. As an application, we apply this framework to estimate the stationary distributions of reflected Brownian motions and the performance of our algorithm is better than existing prevalent numeric methods.

We present a large deviation theory that characterizes the exponential estimate for rare events in stochastic dynamical systems in the limit of weak noise. We aim to consider a next-to-leading-order approximation for more accurate calculation of the mean exit time by computing large deviation prefactors with the aid of machine learning. More specifically, we design a neural network framework to compute quasipotential, most probable paths and prefactors based on the orthogonal decomposition of a vector field. We corroborate the higher effectiveness and accuracy of our algorithm with two toy models. Numerical experiments demonstrate its powerful functionality in exploring the internal mechanism of rare events triggered by weak random fluctuations.

Precise asymptotics of certain Wiener functionals

In this work we develop stochastic control methods for the study of large deviation principles (LDP) for certain interacting particle systems. Although such methods have been well studied for analyzing large deviation properties of small noise stochastic dynamical systems [7] and of weakly interacting particle systems [6], this is the first work to implement the approach for Brownian particle systems with a local interaction. As an application of these methods we give a new proof of the large deviation principle from the hydrodynamic limit for the Ginzburg-Landau model studied in [10]. Along the way, we establish regularity properties of the densities of certain controlled Markov processes and certain results relating large deviation principles and Laplace principles in non-Polish topological spaces that are of independent interest. The proof of the LDP is based on characterizing subsequential hydrodynamic limits of controlled diffusions with nearest neighbor interaction that arise from a variational representation of certain Laplace functionals. This proof also yields a new representation for the rate function which is very natural from a control theoretic point of view. Proof techniques are very similar to those used for the law of large number analysis, namely in the proof of convergence to the hydrodynamic limit (cf. [15]). Specifically, the key step in the proof is establishing suitable bounds on relative entropies and Dirichlet forms associated with certain controlled laws. This general approach has the promise to be applicable to other interacting Brownian systems as well.

Fifty years ago, when Claude Shannon was developing the Mathematical Theory of Communications, for reliable data transmission, which evolved into the subject of information theory, another discipline was developing dealing with Feedback Control of Dynamical System, which evolved into a scientific subject dealing with decision, stability, and optimization. More recently, a separate discipline dealing with robustness of uncertain systems was born in response to the codification of high performance and reliability in the presence of modeling uncertainties. In principle, robustness in dynamical systems is captured through power dissipation via induced norms and dynamic games, while reliable data transmission is captured through measures of information via entropy, relative entropy, and certain laws of Large Deviations theory. The main ingredient in Large Deviations is the rate functional (or action functional in the classical mechanics terminology), often identified through the Cramer or Legendre-Fenchel Transform. On the other hand, robustness of stochastic uncertain systems is currently under development, using information theoretic as well as statistical mechanics concepts, such as, partition functions, free energy, relative entropy, and entropy rate functional. This lecture will summarize certain connections between fundamental concepts of robustness, information theory, and statistical mechanics, and possibly make future projections into the convergence of these disciplines.

We consider ALOHA and slotted ALOHA protocols as medium access rules for a multi-channel message delivery system. Users decide randomly and independently with a minimal amount of knowledge about the system at random times to make a message emission attempt. We consider the two cases that the system has a fixed number of independent available channels, and that interference constraints make the delivery of too many messages at a time impossible. We derive probabilistic formulas for the most important quantities like the number of successfully delivered messages and the number of emission attempts, and we derive large-deviation principles for these quantities in the limit of many participants and many emission attempts. We analyse the rate functions and their minimizers and derive laws of large numbers for the throughput. We optimize it over the probability parameter. Furthermore, we are interested in questions like ``if the number of successfully delivered messages is significantly lower than the expectation, was the reason that too many or too few sending attempts were made?''. Our main tools are basic tools from probability and the theory of (the probabilities of) large deviations.

We consider an infinite-horizon minimax optimal control problem for nonlinear stochastic uncertain systems governed by a discrete-state continuous-time chain. The chain and system dynamics are subject to uncertain disturbances. Using the large deviations theory we construct a robust stabilizing suboptimal guaranteed-performance controller. Conditions are presented under which this controller is optimal. We then present a numeric algorithm for calculating a robust (sub)optimal controller using a Markov chain approximation technique.

A general theory which provides asymptotic tail expansions for density, survival, and hazard rate functions is developed for both absolutely continuous and integer-valued distributions. The expansions make use of Tauberian theorems which apply to moment generating functions (MGFs) with boundary singularities that are of gamma-type or log-type. Standard Tauberian theorems from Feller [An Introduction to Probability Theory and Its Applications II (1971) Wiley] can provide a limited theory but these theorems do not suffice in providing a complete theory as they are not capable of explaining tail behaviour for compound distributions and other complicated distributions which arise in stochastic modelling settings. Obtaining such a complete theory for absolutely continuous distributions requires introducing new “Ikehara” conditions based upon Tauberian theorems whose development and application have been largely confined to analytic number theory. For integer-valued distributions, a complete theory is developed by applying Darboux’s theorem used in analytic combinatorics. Characterizations of asymptotic hazard rates for both absolutely continuous and integer-valued distributions are developed in conjunction with these expansions. The main applications include the ruin distribution in the Cramer–Lundberg and Sparre Andersen models, more general classes of compound distributions, and first-passage distributions in finite-state semi-Markov processes. Such first-passage distributions are shown to have exponential-like/geometric-like tails which mimic the behaviour of first-passage distributions in Markov processes even though the holding-time MGFs involved with such semi-Markov processes are typically not rational.

The asymptotic exponential convergence rate of ordinal comparisons follows from well-known results in large deviations theory, where the critical condition is the existence of a finite moment generating function. In this note, we show that this is both a necessary and sufficient condition, and also show how one can recover the exponential convergence rate in cases where the moment generating function is not finite. In particular, by working with appropriately truncated versions of the original random variables, the exponential convergence rate can be recovered.