Dynamic importance sampling for uniformly recurrent Markov chains

The paper is dealing with estimation of rare event probabilities in stochastic networks. The well known variance reduction technique, called Importance Sampling (IS) is an effective tool for doing this. The main idea of IS is to simulate the random system under a modified set of parameters, so as to make the occurrence of the rare event more likely. The major problem of the IS technique is that the optimal modified parameters, called reference parameters to be used in IS are usually very difficult to obtain. Rubinstein (Eur J Oper Res 99:89–112, 1997) developed the Cross Entropy (CE) method for the solution of this problem of IS technique and then he and his collaborators applied this for estimation of rare event probabilities in stochastic networks with exponential distribution [see De Boer et al. (Ann Oper Res 134:19–67, 2005)]. In this paper, we test this simulation technique also for medium sized stochastic networks and compare its effectiveness to the simple crude Monte Carlo (CMC) simulation. The effectiveness of a variance reduction simulation algorithm is measured in the following way. We calculate the product of the necessary CPU time and the estimated variance of the estimation. This product is compared to the same for the simple Crude Monte Carlo simulation. This was originally used for comparison of different variance reduction techniques by Hammersley and Handscomb (Monte Carlo Methods. Methuen & Co Ltd, London, 1967). The main result of the paper is the extension of CE method for estimation of rare event probabilities in stochastic networks with beta distributions. In this case the calculation of reference parameters of the importance sampling distribution requires numerical solution of a nonlinear equation system. This is done by applying a Newton–Raphson iteration scheme. In this case the CPU time spent for calculation of the reference parameter values cannot be neglected. Numerical results will also be presented.

This paper studies the applications of state-dependent importance sampling in pricing exotic rainbow options. We demonstrate that for many rainbow options, efficient dynamic importance sampling schemes can be constructed from subsolutions to appropriate partial differential equations. We also introduce an alternative large deviations scaling, which leads to universal importance sampling for rainbow options.

In this paper we propose a new importance sampling (IS) scheme to estimate the low error-rate performance of LDPC codes over a Rayleigh fading channel. It biases the Rayleigh fading distribution as well as the noise distribution in order to improve the quality of estimation. The proposed IS scheme is compared with the Monte-Carlo (MC) simulator and other IS schemes in terms of its efficiency and accuracy. Numerical results show that the proposed IS estimator is much more efficient than the MC estimator and other IS estimators from the viewpoint of the required simulation time. Furthermore, the proposed IS scheme provides a more accurate performance estimation of LDPC codes over a Rayleigh fading channel with only a limited simulation time than other IS schemes.

This paper deals with the estimation of rare event probabilities using importance sampling (IS), where an optimal proposal distribution is computed with the cross-entropy (CE) method. Although, IS optimized with the CE method leads to an efficient reduction of the estimator variance, this approach remains unaffordable for problems where the repeated evaluation of the score function represents a too intensive computational effort. This is often the case for score functions related to the solution of a partial differential equation (PDE) with random inputs. This work proposes to alleviate computation by the parsimonious use of a hierarchy of score function approximations in the CE optimization process. The score function approximation is obtained by selecting the surrogate of lowest dimensionality, whose accuracy guarantees to pass the current CE optimization stage. The selection relies on certified upper bounds on the error norm. An asymptotic analysis provides some theoretical guarantees on the efficiency and convergence of the proposed algorithm. Numerical results demonstrate the gain brought by the method in the context of pollution alerts and a system modeled by a PDE.

We present several numerical approaches to investigate rare events in stochastic systems, with a specific focus on application to biological models. We first present several aspects concerning variance reduction of Monte-Carlo methods, with a focus on importance sampling. We show how these methods can be applied to basic biological models. We show that these techniques can be useful in dealing with multiscale continuous-time Markov chains that are important in the context of biochemical reaction networks. We treat with more detail the problem of first passage time for a linear diffusion process, arising from the integrate-and-fire neuron model, to show the kind of mathematical problems that may arise when trying to design an optimal importance sampling scheme. This leads to the observation that large deviation theory can be helpful to solve these questions. We also review a numerical method designed to estimate large deviations quantities such as the quasipotential and the optimal path and apply this method to estimate numerically the quasipotential of the Morris–Lecar neuron model perturbed by small noise.

Mixed-level orthogonal arrays are basic structures in experimental design. We develop three algorithms that compute Rao- and Gilbert-Varshamov-type bounds for mixed-level orthogonal arrays. The computational complexity of the terms involved in the original combinatorial representations of these bounds can grow fast as the parameters of the arrays increase and this justifies the construction of these algorithms. The first is a recursive algorithm that computes the bounds exactly, the second is based on an asymptotic analysis, and the third is a simulation algorithm. They are all based on the representation of the combinatorial expressions that appear in the bounds as expectations involving a symmetric random walk. The Markov property of the underlying random walk gives the recursive formula to compute the expectations. A large deviation (LD) analysis of the expectations provides the asymptotic algorithm. The asymptotically optimal importance sampling (IS) of the same expectation provides the simulation algorithm. Both the LD analysis and the construction of the IS algorithm use a representation of these problems as a sequence of stochastic optimal control problems converging to a limit calculus of a variations problem. The construction of the IS algorithm uses a recently discovered method of using subsolutions to the Hamilton-Jacobi-Bellman equations associated with the limit problem.

Almost all previous acquisition schemes for direct-sequence spread-spectrum systems assume that there is no data modulation. However, in some applications, there is need for acquisition systems with the presence of data modulation, which is what we consider in this paper. First, we derive an optimal scheme using maximum likelihood estimate, by treating the phase of the incoming sequence as a signal parameter to be estimated. Two practical parallel search schemes as approximations to the optimal scheme are presented. We also consider serial and hybrid search schemes. Numerical results show that significant degradation of the standard coherent acquisition schemes occurs when there is data modulation. Using the proposed search schemes, performance is restored to the level which is almost equal to that of the standard schemes without data modulation. >

Although importance sampling is an established and effective sampling and estimation technique, it becomes unstable and unreliable for high- dimensional problems. The main reason is that the likelihood ratio in the importance sampling estimator degenerates when the dimension of the problem becomes large. Various remedies to this problem have been suggested, including heuristics such as resampling. Even so, the consen- sus is that for large-dimensional problems, likelihood ratios (and hence importance sampling) should be avoided. In this paper we introduce a new adaptive simulation approach that does away with likelihood ratios, while retaining the multi-level approach of the cross-entropy method. Like the latter, the method can be used for rare-event probability estimation, optimization, and counting. Moreover, the method allows one to sam- ple from the target distribution rather than asymptotically as in Markov chain Monte Carlo. Numerical examples demonstrate the effectiveness of the method.

Let X 1, X 2, · ·· be independent and identically distributed random variables such that ΕΧ 1 < 0 and P (X 1 ≥ 0) ≥ 0. Fix M ≥ 0 and let T = inf {n: X 1 + X 2 + · ·· + Xn ≥ M} (T = +∞, if for every n = 1,2, ···). In this paper we consider the estimation of the level-crossing probabilities P (T <∞) and , by using Monte Carlo simulation and especially importance sampling techniques. When using importance sampling, precision and efficiency of the estimation depend crucially on the choice of the simulation distribution. For this choice we introduce a new criterion which is of the type of large deviations theory; consequently, the basic large deviations theory is the main mathematical tool of this paper. We allow a wide class of possible simulation distributions and, considering the case that M →∞, we prove asymptotic optimality results for the simulation of the probabilities P (T <∞) and . The paper ends with an example.

We propose a class of strongly efficient rare-event simulation estimators for random walks and compound Poisson processes with a regularly varying increment/jump-size distribution in a general large deviations regime. Our estimator is based on an importance sampling strategy that hinges on a recently established heavy-tailed sample-path large deviations result. The new estimators are straightforward to implement and can be used to systematically evaluate the probability of a wide range of rare events with bounded relative error. They are “universal” in the sense that a single importance sampling scheme applies to a very general class of rare events that arise in heavy-tailed systems. In particular, our estimators can deal with rare events that are caused by multiple big jumps (therefore, beyond the usual principle of a single big jump) as well as multidimensional processes such as the buffer content process of a queueing network. We illustrate the versatility of our approach with several applications that arise in the context of mathematical finance, actuarial science, and queueing theory.

We present a novel method, called the transform likelihood ratio (TLR) method, for estimation of rare event probabilities with heavy-tailed distributions. Via a simple transformation (change of variables) technique the TLR method reduces the original rare event probability estimation with heavy tail distributions to an equivalent one with light tail distributions. Once this transformation has been established we estimate the rare event probability via importance sampling, using the classical exponential change of measure or the standard likelihood ratio change of measure. In the latter case the importance sampling distribution is chosen from the same parametric family as the transformed distribution. We estimate the optimal parameter vector of the importance sampling distribution using the cross-entropy method. We prove the polynomial complexity of the TLR method for certain heavy-tailed models and demonstrate numerically its high efficiency for various heavy-tailed models previously thought to be intractable. We also show that the TLR method can be viewed as a universal tool in the sense that not only it provides a unified view for heavy-tailed simulation but also can be efficiently used in simulation with light-tailed distributions. We present extensive simulation results which support the efficiency of the TLR method.

Mobile data demand is increasing tremendously in wireless social networks, and thus an efficient pricing scheme for social-enabled services is urgently needed. Though static pricing is dominant in the actual data market, price intuitively ought to be dynamically changed to yield greater revenue. The critical question is how to design the optimal dynamic pricing scheme, with prospects for maximizing the expected long-term revenue. In this paper, we study the sequential dynamic pricing scheme of a monopoly mobile network operator in the social data market. In the market, the operator, i.e., the seller, individually offers each mobile user, i.e., the buyer, a certain price in multiple time periods sequentially and repeatedly. The proposed scheme exploits the network effects in the mobile users' behaviors that boost the social data demand. Furthermore, due to limited radio resource, the impact of wireless network congestion is taken into account in the pricing scheme. Thereafter, we propose a modified sequential pricing policy in order to ensure social fairness among mobile users in terms of their individual utilities. To gain more insights, we further study a simultaneous dynamic pricing scheme in which the operator offers the data price simultaneously. We analytically demonstrate that the proposed dynamic pricing scheme can help the operator gain greater revenue and users achieve higher total utilities than those of the baseline static pricing scheme. We construct the social graph using Erdös-Rényi (ER) model and the real dataset based social network for performance evaluation. The numerical results corroborate that the dynamics of pricing schemes over static ones can significantly improve the revenue of the operator.

High-order finite difference scheme for compressible multi-component flow computations

Rare event simulation in the context of queueing networks has been an active area of research for more than two decades. A commonly used technique to increase the efficiency of Monte Carlo simulation is importance sampling. However, there are few rigorous results on the design of efficient or asymptotically optimal importance sampling schemes for queueing networks. Using a recently developed game/subsolution approach, we construct simple and efficient state-dependent importance sampling schemes for simulating buffer overflows in stable open Jackson networks. The sampling distributions do not depend on the particular event of interest, and hence overflow probabilities for different events can be estimated simultaneously. A by-product of the analysis is the identification of the minimizing trajectory for the calculus of variation problem that is associated with the sample-path large deviation rate function.

This paper develops importance resampling into a variance reduction technique for Monte Carlo integration. Importance resampling is a sample generation technique that can be used to generate more equally weighted samples for importance sampling. This can lead to significant variance reduction over standard importance sampling for common rendering problems. We show how to select the importance resampling parameters for near optimal variance reduction. We demonstrate the robustness of this technique on common global illumination problems and achieve a 10%-70% variance reduction over standard importance sampling for direct lighting. We conclude that further variance reduction could be achieved with cheaper sampling methods.