A Study of Variance Reduction Techniques for Estimating Circuit Yields

The paper is dealing with estimation of rare event probabilities in stochastic networks. The well known variance reduction technique, called Importance Sampling (IS) is an effective tool for doing this. The main idea of IS is to simulate the random system under a modified set of parameters, so as to make the occurrence of the rare event more likely. The major problem of the IS technique is that the optimal modified parameters, called reference parameters to be used in IS are usually very difficult to obtain. Rubinstein (Eur J Oper Res 99:89–112, 1997) developed the Cross Entropy (CE) method for the solution of this problem of IS technique and then he and his collaborators applied this for estimation of rare event probabilities in stochastic networks with exponential distribution [see De Boer et al. (Ann Oper Res 134:19–67, 2005)]. In this paper, we test this simulation technique also for medium sized stochastic networks and compare its effectiveness to the simple crude Monte Carlo (CMC) simulation. The effectiveness of a variance reduction simulation algorithm is measured in the following way. We calculate the product of the necessary CPU time and the estimated variance of the estimation. This product is compared to the same for the simple Crude Monte Carlo simulation. This was originally used for comparison of different variance reduction techniques by Hammersley and Handscomb (Monte Carlo Methods. Methuen & Co Ltd, London, 1967). The main result of the paper is the extension of CE method for estimation of rare event probabilities in stochastic networks with beta distributions. In this case the calculation of reference parameters of the importance sampling distribution requires numerical solution of a nonlinear equation system. This is done by applying a Newton–Raphson iteration scheme. In this case the CPU time spent for calculation of the reference parameter values cannot be neglected. Numerical results will also be presented.

A new global variance reduction technique based on pseudo flux method

This dissertation focuses on the importance sampling (IS) variance reduction technique in the Monte Carlo simulation of the performance of digital communication systems over a memoryless, additive Gaussian noise channel. The result is a unbiased estimate of the decision error with significantly smaller estimation variance than conventional Monte Carlo simulation provides. Several common digital (n, k) block codes are used to illustrate the importance sampling approach. Bounds on the variance of the error estimate are developed where the channel description is simple, but where the multiple dimension decision regions are potentially complex. Coherent detection is used as a venue for the comparison of these IS techniques with respect to increasing dimension. Bounds based on the union bound are presented for several IS techniques. An overlap stratification variance reduction technique is introduced and extended with the application of IS. A mixed IS biasing function is employed for concentrating more samples in a particular error region and is employed for several of the common lower rate ([special characters omitted] < 1) block codes. For the random phase channel the IS biasing function is developed as a 2n dimensional pdf consisting of n Rayleigh-Rician pdf pairs. The error performance of several common lower rate block codes is presented. An alternate formulation, having the potential advantage of being able to use both mean translation and variance scaling, is presented. This approach is based on a commonly used technique to numerically generate a single Rayleigh-Rician pdf pair.

This chapter develops methods for increasing the efficiency of Monte Carlo simulation by reducing the variance of simulation estimates. These methods draw on two broad strategies for reducing variance: taking advantage of tractable features of a model to adjust or correct simulation outputs, and reducing the variability in simulation inputs. We discuss control variates, antithetic variates, stratified sampling, Latin hypercube sampling, moment matching methods, and importance sampling, and we illustrate these methods through examples. Two themes run through this chapter: The greatest gains in efficiency from variance reduction techniques result from exploiting specific features of a problem, rather than from generic applications of generic methods. Reducing simulation error is often at odds with convenient estimation of the simulation error itself; in order to supplement a reduced-variance estimator with a valid confidence interval, we sometimes need to sacrifice some of the potential variance reduction. KeywordsImportance SamplingVariance ReductionLatin Hypercube SamplingGeometric Brownian MotionUnderlying AssetThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Chapter 5 - Variance Reduction Techniques

This paper develops importance resampling into a variance reduction technique for Monte Carlo integration. Importance resampling is a sample generation technique that can be used to generate more equally weighted samples for importance sampling. This can lead to significant variance reduction over standard importance sampling for common rendering problems. We show how to select the importance resampling parameters for near optimal variance reduction. We demonstrate the robustness of this technique on common global illumination problems and achieve a 10%-70% variance reduction over standard importance sampling for direct lighting. We conclude that further variance reduction could be achieved with cheaper sampling methods.

We provide a thorough analysis of the effectiveness of different Variance Reduction Techniques (VRTs). We consider both stand-alone and combined applications of two input techniques, Antithetic Variates (AV) and Latin Hypercube Sampling (LHS), and two output techniques, Control Variates (CV) and Poststratified Sampling (PS). Previous research in the area mainly focuses on asymptotic variance reduction. In this experimental study, we measure the performance of VRTs under finite simulation run lengths and analyze their effects. Our findings show that the asymptotic variance reduction results do not readily apply to finite-length simulations. We consider three different types of systems (M/M/1, serial production line and (s, S) inventory control systems) and compare the VRTs under various experimental conditions. We observe that a variance reduction cannot be guaranteed for every instance a VRT is applied. Our results also indicate that the output VRTs (CV, PS) are better than input VRTs (AV, LHS) on the average for the single systems considered in this study. More interestingly, the less-sophisticated techniques (AV, CV) often perform better than the relatively more-complex techniques (LHS, PS). A comprehensive bibliography is also provided.

In this paper, we propose a calibration estimator of population mean in stratified sampling using the known mean and variance information from multi-auxiliary variables. The problem of determining the optimum calibrated weights is formulated as a Nonlinear Programming Problem (NLPP) that is solved using the Lagrange multiplier technique. Numerical example with real data is presented to illustrate the computational details of the proposed estimator. A comparison study is also carried out using real and simulated data to evaluate the performance and the usefulness of the proposed estimator. The study reveals that the proposed estimator with multi-auxiliary information is more efficient estimator of the population mean as it provides least estimated variance and highest gain in relative efficiency (RE). References Jean Claude Deville and Carl Erik Sarndal. Calibration estimators in survey sampling. Journal of the American statistical Association, 87(418):376–382, 1992. doi:10.1080/01621459.1992.10475217 . Victor M Estevao and Carl Erik Sarndal. Survey estimates by calibration on complex auxiliary information. International Statistical Review, 74(2):127–147, 2006. doi:110.1111/j.1751-5823.2006.tb00165.x Patrick J Farrell and Sarjinder Singh. Model-assisted higher-order calibration of estimators of variance. Australian and New Zealand Journal of Statistics, 47(3):375–383, 2005. doi:10.1111/j.1467-842X.2005.00402.x Wolfram Research, Inc. Mathematica, Version 11.3. Champaign, IL, 2018. Jong Min Kim, Engin A Sungur, and Tae Young Heo. Calibration approach estimators in stratified sampling. Statistics and probability letters, 77(1):99–103, 2007. doi:10.1016/j.spl.2006.05.015 Phillip S Kott. Using calibration weighting to adjust for nonresponse and coverage errors. Survey Methodology, 32(2):133, 2006. Dinesh K Rao. Mathematical programing in stratified random sampling. PhD thesis, School of Computing, Information and Mathematical Sciences, The University of the South Pacific, Fiji, February 2017. Dinesh K. Rao, Tokaua. Tekabu, and Mohammad G M Khan. New calibration estimators in stratified sampling. In Proceedings of Asia-Pacific World Congress on Computer Science and Engineering, pages 66–70. IEEE, 2016. Gurmindar K Singh, Dinesh K Rao, and Mohammed GM Khan. Calibration estimator of population mean in stratified random sampling. In Proceedings of Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE), pages 1–5. IEEE, 2014. Sarjindar Singh, Stephen Horn, and Frank Yu. Estimation of variance of general regression estimator: Higher level calibration approach. Survey Methodology, 24:41–50, 1998. Sarjinder Singh. Advanced Sampling Theory With Applications: How Michael Selected Amy, volume I and II. Kluwer Academic Publishers, Netherlands, 2003. Sarjinder Singh. On the calibration of design weights using a displacement function. Metrika, 75(1):85–107, 2012. doi:10.1007/s00184-010-0316-6 Sarjinder Singh, Stephen Horn, Sadeq Chowdhury, and Frank Yu. Theory and methods: Calibration of the estimators of variance. Australian and New Zealand Journal of Statistics, 41(2):199–212, 1999. doi:10.1111/1467-842X.00074 D S Tracy, S Singh, and R Arnab. Note on calibration in stratified and double sampling. Survey Methodology, 29(1):99–104, 2003. Changbao Wu and Randy R Sitter. A model-calibration approach to using complete auxiliary information from survey data. Journal of the American Statistical Association, 96(453):185–193, 2001. doi:10.1198/016214501750333054

A coprime sensor array (CSA) is a sparse array geometry that interleaves two spatially undersampled uniform linear arrays (ULAs) with coprime undersampling factors. The CSA spans the same aperture and has equivalent resolution as a fully populated ULA, but uses fewer sensors. Previous works have introduced three CSA processors as spatial power spectral density (PSD) estimators: the Product, Min, and Blend processors. However, two of these processors display increased PSD esti-mate variance over the ULA PSD estimate. This dissertation focuses on reducing the variance of the CSA PSD estimators. PSD estimation traditionally relies on averag-ing uncorrelated coherent measurements (snapshots) to reduce variance. However,non-stationary underwater sonar environments often preclude increasing the num-ber of snapshots required to achieve a desirable PSD variance. In the traditional signal processing literature, there are two alternative methods to reduce variance without additional snapshot cost. The multitaper (MT) method and Welch's over-lapping segment averaging (WOSA) method improve PSD variance by O(K) at the expense of an O(K) resolution decrease by averaging K uncorrelated PSD estimates. Multitaper obtains these uncorrelated estimates by windowing the array with K or-thogonal tapers that span the entire array aperture. In contrast, WOSA obtains the uncorrelated estimates by subdividing the array into K possibly overlapping segments.This dissertation extends these two existing ULA variance reduction techniques to the CSA processors, making four main contributions. The first proposes the multitapered Product processor (MT-Product) that estimates the spatial PSD with reduced variance with respect to the traditional CSA Product processor but still uses fewer sensors than a fully populated ULA. The second proposes the multita-pered Min processor (MT-Min) that reduces the PSD estimate variance further than either MT-Product or Product. The MT-Min estimator has variance comparable to a multitapered ULA. The third proposes the multitapered Blend processor (MT-Blend) that blends MT-Product's attenuation of multiple source cross-terms with MT-Min's low variance while still achieving an O(K) variance reduction over the traditional Blend processor. The final contribution proposes the Welch overlapping segment averaging Product processor (WOSA-Product) that estimates the spatial PSD with reduced variance with respect to the traditional CSA Product processor but still uses fewer sensors than a fully populated ULA. WOSA-Product is gener-ally able to form a larger number of uncorrelated PSD estimates than MT-Product. Closed-form statistics for spatially white Gaussian processes are derived for MT-Product, MT-Min, and WOSA-Product. Simulations verify the variance reduction predicted by the analytical derivations each processor, and the effects of WOSA-Product segment length on resolution, variance reduction, and peak sidelobe levels are discussed.

Gives an overview of the main techniques for improving the statistical efficiency of simulation estimators. Efficiency improvement is typically (but not always) achieved through variance reduction. We discuss methods such as common random numbers, antithetic variates, control variates, importance sampling, conditional Monte Carlo, stratified sampling, and some others, as well as the combination of certain of those methods. We also survey the recent literature on this topic.

In this paper, we consider a simulation technique for stochastic trees. One of the most important areas in computational genetics is the calculation and subsequent maximization of the likelihood function associated with such models. This typically consists of using importance sampling and sequential Monte Carlo techniques. The approach proceeds by simulating the tree, backward in time from observed data, to a most recent common ancestor. However, in many cases, the computational time and variance of estimators are often too high to make standard approaches useful. In this paper, we propose to stop the simulation, subsequently yielding biased estimates of the likelihood surface. The bias is investigated from a theoretical point of view. Results from simulation studies are also given to investigate the balance between loss of accuracy, saving in computing time and variance reduction.

The Internet’s complexity restricts analysis or simulation to assess its parameters. Instead, actual measurements provide a reality check. Many statistical measurements of the Internet estimate rare event probabilities. Collection of such statistics renders sampling methods as a primary substitute. Within the context of this inquiry, we have presented the conventional Monte Carlo approach to estimate the Internet event probability. As a variance reduction technique, Importance Sampling is introduced which is a modified Monte Carlo approach resulting in a significant reduction of effort to obtain an accurate estimate. This method works particularly well when estimating the probability of rare events. It has great appeal to use as an efficient sampling scheme for estimating the information server density on the Internet. In this chapter, we have proposed the Importance Sampling approaches to track the prevalence and growth of Web service, where an improved Importance Sampling scheme is introduced. We present a thorough analysis of the sampling approaches. Based on the periodic measurement of the number of active Web servers conducted over the past five years, an exponential growth of the Web is observed and modeled. Also discussed in this chapter is the increasing security concerns on Web servers.

We provide the first theoretical analysis on the convergence rate of asynchronous mini-batch gradient descent with variance reduction (AsySVRG) for non-convex optimization. Asynchronous stochastic gradient descent (AsySGD) has been broadly used for deep learning optimization, and it is proved to converge with rate of O(1/\sqrt{T}) for non-convex optimization. Recently, variance reduction technique is proposed and it is proved to be able to accelerate the convergence of SGD greatly. It is shown that asynchronous SGD method with variance reduction technique has linear convergence rate when problem is strongly convex. However, there is still no work to analyze the convergence rate of this method for non-convex problem. In this paper, we consider two asynchronous parallel implementations of mini-batch gradient descent method with variance reduction: one is on distributed-memory architecture and the other is on shared-memory architecture. We prove that both methods can converge with a rate of O(1/T) for non-convex optimization, and linear speedup is accessible when we increase the number of workers. We evaluate our methods by optimizing multi-layer neural networks on two real datasets (MNIST and CIFAR-10), and experimental results demonstrate our theoretical analysis.

Rare events are an important limitation of discrete-event simulation in some areas of applications. The existing rare-event simulation (RES) techniques, such as importance sampling (IS), splitting, and RESTART, have been used for RES so far. However, these techniques not only have some limitations, but also are not general or easy to use. In this paper, the partition of the region (PoR) method is used as a variance reduction technique in a new RES technique for usage with a wide range of modeling and simulation languages. Three variants of the proposed technique will be introduced: simple PoR, recursive PoR, and advanced PoR. Simple and recursive PoR methods are powerful techniques that have some important features such as bounded relative error; however, they are restricted to some specific models. Advanced PoR is a more general technique that integrates the PoR method and the IS technique. To evaluate the efficiency of the proposed technique and its variants, the comparative simulation results of some illustrative examples using stochastic activity networks will be presented.

Quantiles, which are also known as values-at-risk in finance, frequently arise in practice as measures of risk. This article develops asymptotically valid confidence intervals for quantiles estimated via simulation using variance-reduction techniques (VRTs). We establish our results within a general framework for VRTs, which we show includes importance sampling, stratified sampling, antithetic variates, and control variates. Our method for verifying asymptotic validity is to first demonstrate that a quantile estimator obtained via a VRT within our framework satisfies a Bahadur-Ghosh representation. We then exploit this to show that the quantile estimator obeys a central limit theorem (CLT) and to develop a consistent estimator for the variance constant appearing in the CLT, which enables us to construct a confidence interval. We provide explicit formulae for the estimators for each of the VRTs considered.