Optimal importance sampling for Markovian systems

Importance sampling (IS) is a change-of-measure technique for speeding up the simulation of rare events in stochastic systems. In this paper, we presented an approach uses Importance Sampling technique for efficient estimation of software reliability via Markov software usage models in statistical testing. By suitable changes of the probabilities of state transitions during test, an iterative method based on the Ali-Silvey distance is proposed for this choice, and an unbiased reliability estimator with zero variance is obtained. A learning algorithm for the computation of optimal transition probabilities of the Markov chain usage model is also presented and experimental results of this algorithm are reported.

We present several numerical approaches to investigate rare events in stochastic systems, with a specific focus on application to biological models. We first present several aspects concerning variance reduction of Monte-Carlo methods, with a focus on importance sampling. We show how these methods can be applied to basic biological models. We show that these techniques can be useful in dealing with multiscale continuous-time Markov chains that are important in the context of biochemical reaction networks. We treat with more detail the problem of first passage time for a linear diffusion process, arising from the integrate-and-fire neuron model, to show the kind of mathematical problems that may arise when trying to design an optimal importance sampling scheme. This leads to the observation that large deviation theory can be helpful to solve these questions. We also review a numerical method designed to estimate large deviations quantities such as the quasipotential and the optimal path and apply this method to estimate numerically the quasipotential of the Morris–Lecar neuron model perturbed by small noise.

We review two types of adaptive Monte Carlo methods for rare event simulations. These methods are based on importance sampling. The first approach selects importance sampling distributions by minimizing the variance of importance sampling estimator. The second approach selects importance sampling distributions by minimizing the cross entropy to the optimal importance sampling distribution. We also review the basic concepts of importance sampling in the rare event simulation context. To make the basic concepts concrete, we introduce these ideas via the study of rare events of M/M/1 queues.

Command-based importance sampling for statistical model checking

Rare event simulation in the context of queueing networks has been an active area of research for more than two decades. A commonly used technique to increase the efficiency of Monte Carlo simulation is importance sampling. However, there are few rigorous results on the design of efficient or asymptotically optimal importance sampling schemes for queueing networks. Using a recently developed game/subsolution approach, we construct simple and efficient state-dependent importance sampling schemes for simulating buffer overflows in stable open Jackson networks. The sampling distributions do not depend on the particular event of interest, and hence overflow probabilities for different events can be estimated simultaneously. A by-product of the analysis is the identification of the minimizing trajectory for the calculus of variation problem that is associated with the sample-path large deviation rate function.

This paper surveys efficient techniques for estimating, via simulation, the probabilities of certain rare events in queueing and reliability models. The rare events of interest are long waiting times or buffer overflows in queueing systems, and system failure events in reliability models of highly dependable computing systems. The general approach to speeding up such simulations is to accelerate the occurrence of the rare events by using importance sampling. In importance sampling, the system is simulated using a new set of input probability distributions, and unbiased estimates are recovered by multiplying the simulation output by a likelihood ratio. Our focus is on describing asymptotically optimal importance sampling techniques. Using asymptotically optimal importance sampling, the number of samples required to get accurate estimates grows slowly compared to the rate at which the probability of the rare event approaches zero. In practice, this means that run lengths can be reduced by many orders of magnitude, compared to standard simulation. In certain cases, asymptotically optimal importance sampling results in estimates having bounded relative error. With bounded relative error, only a fixed number of samples are required to get accurate estimates, no matter how rare the event of interest is. The queueing systems studied include simple queues (e.g., GI/GI/1), Jackson networks, discrete time queues with multiple autocorrelated arrival processes that arise in the analysis of Asynchronous Transfer Mode communications switches, and tree structured networks of such switches. Both Markovian and non-Markovian reliability models are treated.

In this paper, the optimal importance sampling, which is a variance reduction technique in Monte Carlo estimation, is theoretically studied in the case of electronic circuits. To the best of our knowledge, this problem has not previously been studied in electronic circuits. In this study, the theoretical basis of the optimal importance sampling is developed in the case where no approximate model of the circuit responses is given. Simulation results show good agreement with the theoretical basis.

Rare events are an important limitation of discrete-event simulation in some areas of applications. The existing rare-event simulation (RES) techniques, such as importance sampling (IS), splitting, and RESTART, have been used for RES so far. However, these techniques not only have some limitations, but also are not general or easy to use. In this paper, the partition of the region (PoR) method is used as a variance reduction technique in a new RES technique for usage with a wide range of modeling and simulation languages. Three variants of the proposed technique will be introduced: simple PoR, recursive PoR, and advanced PoR. Simple and recursive PoR methods are powerful techniques that have some important features such as bounded relative error; however, they are restricted to some specific models. Advanced PoR is a more general technique that integrates the PoR method and the IS technique. To evaluate the efficiency of the proposed technique and its variants, the comparative simulation results of some illustrative examples using stochastic activity networks will be presented.

In this paper, we present a variance minimization (VM) procedure for rare event simulation in tandem queueing networks. We prove that the VM method can produce a zero variance. The VM method is suitable to compute optimal importance sampling (IS) parameters for small scale tandem networks. For large scale tandem networks we propose a sub-optimal IS (SOIS) method, which projects the optimal biased transition probabilities of the corresponding small scale system into those of a large scale system. In other words, we establish an efficient IS method for a large scale system by zooming into a small scale system and then projecting our findings into the large scale system. The numerical results show that our SOIS method can produce accurate results with very short CPU time, while many other methods often require much longer.

A new type of exponential asymptotical expansions in mixed large deviation and quasi-ergodic theorems and asymptotical expansions for quasistationary distributions are presented for nonlinearly perturbed Markov chains. Applications to analysis of rare events in stochastic systems are discussed.

Optimal conditional importance sampling (OCIS) provides the unique optimal conditional estimator for error probabilities. The OCIS method essentially obtains a Monte Carlo average over conditional error probabilities. The efficiency of OCIS is studied for a binary symmetric ISI channel with AWGN. The results show that the technique is at least twice as efficient as that of the linearly shifted method.

In future cellular networks, it will be possible to estimate the channel parameters of non-line-of-sight propagation paths providing unique opportunities for simultaneous localization and mapping (SLAM) with commodity user equipments (UEs). Radio-SLAM generally relies on generating samples of the UE trajectory and constructing a trajectory-conditioned map. To reduce the number of samples and complexity, we propose an iterative method to approximate the optimal sampling density. The numerical results demonstrate that the added computational complexity of the proposed method can be easily justified by the more efficient use of particles. As an outcome, the presented filter nearly achieves the lower bound and still runs in real-time.

This article describes an efficient technique for estimating, via simulation, the probability of buffer overflows in a queueing model that arises in the analysis of ATM (Asynchronous Transfer Mode) communication switches. There are multiple streams of (autocorrelated) traffic feeding the switch that has a buffer of finite capacity. Each stream is designated as being of either high or low priority. When the queue length reaches a certain threshold, only high priority packets are admitted to the switch's buffer. The problem is to estimate the loss rate of high priority packets. An asymptotically optimal importance sampling approach is developed for this rare event simulation problem. In this approach, the importance sampling is done in two distinct phases. In the first phase, an importance sampling change of measure is used to bring the queue length up to the threshold at which low priority packets get rejected. In the second phase a different importance sampling change of measure is used to move the queue length from the threshold to the buffer capacity.

Simulation is widely used to estimate losses due to default and other credit events in financial portfolios. The accurate measurement of credit risk can be modeled as a rare event simulation problem. While Monte Carlo simulation is time-consuming for rare events, importance sampling techniques can effectively reduce the simulation time, thus improving simulation efficiency. This chapter proposes a new importance sampling method to estimate rare event probability in simulation models. The optimal importance sampling distributions are derived in terms of expectation in the normal copula model developed in finance. In the normal copula model, dependency is introduced through a set of common factors of multiple obligors. The intriguing dependence between defaults of multiple obligors imposes hurdles in simulation. The simulated results demonstrate the effectiveness of the proposed approach to solving the portfolio credit risk problem.

Importance sampling is a variance reduction technique for efficient simulation via a change of measure. In particular, it can be applied to rare event simulation of Markov chains. An optimal change of measure yielding a zero variance estimator always exists, but it explicitly depends on the unknown quantity of interest. Thus, it is typically neither known in advance nor available in simulations. In this paper, we investigate the form of optimal importance sampling for estimating state probabilities in discrete-time Markov chains, both over finite horizon and in steady state. We derive the optimal change of measure by utilizing only a general property, without using a priori knowledge of the quantity of interest. Our results show that optima! importance sampling for Markov chains cannot be performed by simulating an alternative Markov chain, since transition probabilities in each step must depend on the history of the already simulated process. This particularly indicates that combined dynamic/adaptive techniques should be applied, and that some traditional approaches are not really promising, which holds not only for Markov chains but also for higher level models such as Petri nets and Markovian queueing models.