Monte Carlo simulation and large deviations theory for uniformly recurrent Markov chains

Importance sampling is a Monte Carlo simulation technique in which the simulation distribution is different from the true underlying distribution. In order to obtain an unbiased Monte Carlo estimate of the desired parameter, simulated events are weighted to reflect their true relative frequency. In this paper, we consider the estimation via simulation of certain large deviations probabilities for time-homogeneous Markov chains. We first demonstrate that when the simulation distribution is also a homogeneous Markov chain, the estimator variance will vanish exponentially as the sample size n tends to∞. We then prove that the estimator variance is asymptotically minimized by the same exponentially twisted Markov chain which arises in large deviation theory, and furthermore, this optimization is unique among uniformly recurrent homogeneous Markov chain simulation distributions.

This article is a review article on the optimal Markov chain Monte Carlo (MCMC) sampling. The focus is on homogeneous Markov chains. This article first reviews the problem of finding the optimal transition matrix, which is defined to minimize the asymptotic variance of MCMC estimators. The article later reviews the locally optimal sampler (LOS), an MCMC sampling that performs local updates based on the optimal transition matrix. We conducted a simulation study to compare the LOS with the Metropolis–Hastings and the Gibbs Sampler. The LOS was shown to provide an improved rate of convergence over these two most popular sampling schemes. The implementation of the LOS requires only minor modifications in existing Gibbs sampling code. WIREs Comput Stat 2013. doi: 10.1002/wics.1265This article is categorized under: Statistical and Graphical Methods of Data Analysis > Markov Chain Monte Carlo (MCMC)

Importance sampling (IS) is a powerful method for reducing simulation run times when estimating the probabilities of rare events in communication systems using Monte Carlo simulation and is made feasible and effective for the simulation of networks of queues by regenerative techniques. However, using the most favorable IS settings very often makes the length of regeneration cycles infinite or impractically long. To address this problem, a methodology that uses IS dynamically within each regeneration cycle to drive the system back to the regeneration state after an accurate estimate has been obtained is discussed. A statistically based technique for optimizing IS parameter values for simulations of queueing systems, including complex systems with bursty arrival processes, is formulated. A deterministic variant of stochastic simulated annealing (SA), called mean field annealing (MFA), is used to minimize statistical estimates of the IS estimator variance. The technique is demonstrated by evaluating blocking probabilities.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

This dissertation focuses on the importance sampling (IS) variance reduction technique in the Monte Carlo simulation of the performance of digital communication systems over a memoryless, additive Gaussian noise channel. The result is a unbiased estimate of the decision error with significantly smaller estimation variance than conventional Monte Carlo simulation provides. Several common digital (n, k) block codes are used to illustrate the importance sampling approach. Bounds on the variance of the error estimate are developed where the channel description is simple, but where the multiple dimension decision regions are potentially complex. Coherent detection is used as a venue for the comparison of these IS techniques with respect to increasing dimension. Bounds based on the union bound are presented for several IS techniques. An overlap stratification variance reduction technique is introduced and extended with the application of IS. A mixed IS biasing function is employed for concentrating more samples in a particular error region and is employed for several of the common lower rate ([special characters omitted] &lt; 1) block codes. For the random phase channel the IS biasing function is developed as a 2n dimensional pdf consisting of n Rayleigh-Rician pdf pairs. The error performance of several common lower rate block codes is presented. An alternate formulation, having the potential advantage of being able to use both mean translation and variance scaling, is presented. This approach is based on a commonly used technique to numerically generate a single Rayleigh-Rician pdf pair.

In the practical engineering, to solve the small failure probabilities with correlated high-dimensional variables, the subset simulation (SS) is combined together with the Monte Carlo and importance sampling (IS) method. The samples from the probability density functions (PDF) of the importance sampling are used to construct the intermediate failure events, by which the small failure probabilities are turned into a Markov chain (MC), which is a continuous product made of a series large failure probability or conditional failure probability (CFP) which is easily answered, on which the structural reliability can be efficiently simulated by directly obtaining the samples with correlated ones. Finally, the 3 planet carriers of 3 grade planetary reducers in shield tunneling machine(STM) are as examples to check the algorithm above, the results show that the SS of the IS with correlated variables can highly simulate failure probability.

1 - Discrete state space Markov processes

At the multiplexer of an ATM network, the admissible cell loss probability is less than 10−12 for some service categories. Ordinary Monte Carlo (MC) simulation requires considerable computation time to obtain such a small probability. The importance sampling (IS) technique is effective in accclerating the MC simulation. In an IS simulation, the underlying distribution is biased so as to generate more samples in the target event. In order to achieve a fast simulation, it is important to have a simulation distribution that yields an estimate with minimum variance. This is called the optimal simulation distribution. In this paper, we investigate the survivor function P(Q > q) of a stationary queue length Q in ATM queuing and obtain an IS estimate of P(Q > q) at about 10−2 by means of the optimal simulation distribution. Conventional methods [1, 2] for determining the optimal distribution require a lot of computation time. We apply the large deviation theory to this problem to obtain the optimal simulation distribution analytically and carry out an IS simulation. We present some numerical results for M/D/1 and 2-state MMPP/D/1 which demonstrate that our estimates are obtained in a very small computation time and are accurate when compared to MC simulation. © Scripta Technica, Inc. Electron Comm Jpn Pt 1, 80(12): 28–37, 1997

Importance sampling is recognized as a potentially powerful method for reducing simulation runtimes when estimating the bit error rate (BER) of communications systems using Monte Carlo simulation. Analytically, minimizing the variance of the importance sampling (IS) estimator with respect to the biasing parameters has typically yielded solutions for systems for which the BER could be found analytically. A technique for finding an asymptotically optimal set of biasing parameter values, in the sense that as the resolution of the search and the number of runs used both approach infinity, the algorithm converges to the true optimum, is proposed. The algorithm determines the amount of biasing that minimizes a statistical measure of the variance of the BER estimate and exploits a theoretically justifiable relationship, for small sample sizes, between the BER estimate and the amount of biasing. The translation biasing scheme is considered, although the algorithm is applicable to other parametric IS techniques. Only mild assumptions are required of the noise distribution and system. Experimentally, improvement factors ranging from two to eight orders of magnitude are obtained for a number of distributions for both linear and nonlinear systems with memory.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Let X 1, X 2, · ·· be independent and identically distributed random variables such that ΕΧ 1 &amp;lt; 0 and P (X 1 ≥ 0) ≥ 0. Fix M ≥ 0 and let T = inf {n: X 1 + X 2 + · ·· + Xn ≥ M} (T = +∞, if for every n = 1,2, ···). In this paper we consider the estimation of the level-crossing probabilities P (T &amp;lt;∞) and , by using Monte Carlo simulation and especially importance sampling techniques. When using importance sampling, precision and efficiency of the estimation depend crucially on the choice of the simulation distribution. For this choice we introduce a new criterion which is of the type of large deviations theory; consequently, the basic large deviations theory is the main mathematical tool of this paper. We allow a wide class of possible simulation distributions and, considering the case that M →∞, we prove asymptotic optimality results for the simulation of the probabilities P (T &amp;lt;∞) and . The paper ends with an example.

This paper deals with methods for constructing and solving large Markov chain models of computer system availability and reliability. A set of powerful high level modeling constructs is discussed that can be used to represent the failure and repair behavior of the components that comprise a system, including important component interactions. If time independent failure and repair rates are assumed then a time homogeneous continuous time Markov chain can be constructed automatically from the modeling constructs used to describe the system. Since, the size of Markov chains grows exponentially with the number of components modeled, simulation appears to be a practical way for solving models of large systems. However, the standard simulation takes very long simulation runs to estimate availability and reliability measures because the system failure event is a rare event. Therefore, variance reduction techniques which can aid in computing rare-event probabilities quickly are of interest. Specifically, the Importance Sampling technique has been found to be most useful. The modeling language and the simulation methods discussed in this paper have been implemented in a program package called the System Availability Estimator (SAVE).

We consider weak lumpability of finite homogeneous Markov chains, which is when a lumped Markov chain with respect to a partition of the initial state space is also a homogeneous Markov chain. We show that weak lumpability is equivalent to the existence of a direct sum of polyhedral cones that is positively invariant by the transition probability matrix of the original chain. It allows us, in a unified way, to derive new results on lumpability of reducible Markov chains and to obtain spectral properties associated with lumpability.

The known approaches to project management are typically based on the intuitive decisions by project managers, not on theoretical foundations, patterns and laws of project management. In order to solve this contradiction, it is necessary to conduct the generalization and develop applied aspects for using the Markov chains to represent and model the weakly structured project management systems. To construct the Markov chain, we used a modified system of project management, which is represented in the standard of project management and defines the scheme of interaction between project participants. A method for the transformation of this scheme into a homogeneous Markov chain with discrete states and time is developed. It is demonstrated that iteration solution of the system of equations of the Markov model makes it possible to construct a trajectory of development of virtual or actual project systems. We examined practical aspects in measuring the effectiveness of trajectory of project development by using the devised Markov model. An evaluation of results of changing the probability distribution of project states in the coordinates of probabilities of the system states by steps revealed a significant effect on the progress and efficiency of a project only in the case of variation of terms of cooperation within a project team.

Using long-term direct observations in a Polytrichum-Myrtillus pine forest, we have constructed and verified a homogeneous Markov chain model for two dominant species (Vaccinium myrtillus and V. vitis-idaed) at the late stages of succession. The sampling design features a large sample size (2000 quadrats) on permanent transects, several re-examinations with the interval of 5 years, and the use of species rooted frequency. As a model of the process under concern, the discrete Markov chain accounts for the following four states: both species being absent on the quadrat, one of them being present alone, and the joint presence of the both; the model time step coincides with the time interval between observations. The model is calibrated on the data of two successive examinations and verified on that of one more examination. All possible transitions between the states are revealed to realize in quadrats for one time interval, as well as the absence of transitions at each state, which results in the complete digraph (directed graph) of transitions. Major model results are obtained by the formulae of finite Markov chain theory: the steady-state square distribution, cyclicity characteristics, and the mean durations of stages in the fine-scale dynamics. As a steady-state (stable) outcome of succession, the distribution among quadrats is expected where 30% of quadrats are occupied by V. myrtillus alone, 11% by V. vitis-idaea alone, both species are present on 18% of quadrats, and 41% of quadrats are 'empty'. This demonstrates a possibility for V. myrtillus and V. vitis-idaea to coexist stably at the latest stages of succession, with the clear predominance of V. myrtillus, yet without competitive exclusion. The quantitative characteristics of cyclicity and the durations of stages in the fine-scale dynamics enable us to estimate the total duration of secondary post-fire succession as about 45 years (to reach a distribution of states that differs less than 5% from the steady-state one). Out of the four states specified, the quadrats with V. vitis-idaea alone persist for the least time (8 years) on the average, while 'empty' ones persist for the greatest time (18 years). Forecasting the dynamics for one model time step forward and comparing the forecast with the real square distribution have revealed the measure of difference to be 5.4%. This illustrates the efficiency of the (time-)homogeneous Markov chain as a short-term forecast tool, yet leaves open the question whether the homogeneity hypothesis be true in the longer term.

Summary We develop new statistical theory for probabilistic principal component analysis models in high dimensions. The focus is the estimation of the noise variance, which is an important and unresolved issue when the number of variables is large in comparison with the sample size. We first unveil the reasons for an observed downward bias of the maximum likelihood estimator of the noise variance when the data dimension is high. We then propose a bias-corrected estimator by using random-matrix theory and establish its asymptotic normality. The superiority of the new and bias-corrected estimator over existing alternatives is checked by Monte Carlo experiments with various combinations of (p, n) (the dimension and sample size). Next, we construct a new criterion based on the bias-corrected estimator to determine the number of the principal components, and a consistent estimator is obtained. Its good performance is confirmed by a simulation study and real data analysis. The bias-corrected estimator is also used to derive new asymptotics for the related goodness-of-fit statistic under the high dimensional scheme.

The problem of discriminating between two Markov chains is considered. It is assumed that the common state space of the chains is finite and all the finite dimensional distributions are mutually absolutely continuous. The Bayes risk is expressed through large deviation probabilities for sums of random variables defined on an auxiliary Markov chain. The proofs are based on a large deviation theorem recently established by Z. Szewczak.