Variance Reduction in Population-Based Optimization

Gives an overview of the main techniques for improving the statistical efficiency of simulation estimators. Efficiency improvement is typically (but not always) achieved through variance reduction. We discuss methods such as common random numbers, antithetic variates, control variates, importance sampling, conditional Monte Carlo, stratified sampling, and some others, as well as the combination of certain of those methods. We also survey the recent literature on this topic.

Parametric sensitivity of biochemical networks is an indispensable tool for studying system robustness properties, estimating network parameters, and identifying targets for drug therapy. For discrete stochastic representations of biochemical networks where Monte Carlo methods are commonly used, sensitivity analysis can be particularly challenging, as accurate finite difference computations of sensitivity require a large number of simulations for both nominal and perturbed values of the parameters. In this paper we introduce the common random number (CRN) method in conjunction with Gillespie's stochastic simulation algorithm, which exploits positive correlations obtained by using CRNs for nominal and perturbed parameters. We also propose a new method called the common reaction path (CRP) method, which uses CRNs together with the random time change representation of discrete state Markov processes due to Kurtz to estimate the sensitivity via a finite difference approximation applied to coupled reaction paths that emerge naturally in this representation. While both methods reduce the variance of the estimator significantly compared to independent random number finite difference implementations, numerical evidence suggests that the CRP method achieves a greater variance reduction. We also provide some theoretical basis for the superior performance of CRP. The improved accuracy of these methods allows for much more efficient sensitivity estimation. In two example systems reported in this work, speedup factors greater than 300 and 10,000 are demonstrated.

Comparison with a standard is a general multiple comparison problem, where each system is required to be compared with a single system, referred to as a ‘standard’, as well as with other alternative systems. Screening procedures specially designed to be used for comparison with a standard have been proposed to find a subset that includes all the systems better than the standard in terms of the expected performance. Selection procedures are derived to determine the best system among a number of systems that are better than the standard, or to select the standard when it is equal to or better than the other alternatives. We develop new procedures for screening and selection through the use of two variance reduction techniques, common random numbers and control variates, which are particularly useful in the context of simulation experiments. Empirical results and a realistic example are also provided to compare our procedures with the existing ones.

Other than common random numbers, control variates is the most promising variance reduction technique in terms of its potential for widespread use: Control variates is applicable in single or multiple response simulation, it does not require altering the simulation run in any way, and any stochastic simulation contains potential control variates. A rich theory of control variates has been developed in recent years. Most of this theory assumes a specific probabilistic structure for the simulation output process, usually joint normality of the response and the control variates. When these assumptions are not satisfied, desirable properties of the estimator, such as unbiasedness, may be lost. A number of remedies for violations of the assumptions have been proposed, including jackknifing and splitting. However, there has been no systematic analytical and empirical evaluation of these remedies. This paper presents such an evaluation, including evaluation of the small-sample statistical properties of the proposed remedies.

Agent-based simulation provides a methodology to investigate complex systems behavior, such as supply chains, while incorporating many empirical elements relative to both systems structure and agent behavior. While there is a significant amount of simulation and analytical research investigating the impact of information sharing in supply chains, few studies have used empirical demand for the model. This research utilizes empirical distributions in order to determine the demand process faced by distribution centers in a distribution network. Therefore, the distribution centers face independent and heterogeneous demand that is not normal, and exhibits a much larger coefficient of variation than is generally utilized in similar research. With so much complexity and variability, contrasting different inter-organizational information sharing configurations provides an ideal setting for utilizing common random numbers for variance reduction. Comparisons made using this methodology show clear differences between the different information sharing schemes.

The popularity of search heuristics has lead to numerous new approaches in the last two decades. Since algorithm performance is problem dependent and parameter sensitive, it is difficult to consider any single approach as of greatest utility overall problems. In contrast, differential evolution (DE) is a numerical optimization approach that requires hardly any parameter tuning and is very efficient and reliable on both benchmark and real-world problems. However, the results presented in this paper demonstrate that standard methods of evolutionary optimization are able to outperform DE on noisy problems when the fitness of candidate solutions approaches the fitness variance caused by the noise.

The efficiency of several variance reduction techniques (in particular, importance sampling, stratified sampling, and control variates) are studied with respect to their application in estimating circuit yields. This study suggests that one essentially has to have a good approximation of the region of acceptability in order to achieve significant variance reduction. Further, all the methods considered are based, either explicitly or implicity, on the use of a model. The control variate method appears to be more practical for implementation in a general purpose statistical circuit analysis program. Stratified sampling is the most simple to implement, but yields only very modest reductions in the variance of the yield estimator. Lastly, importance sampling is very useful when there are few parameters and the yield is very high or very low; however, a good practical technique for its implementation, in general, has not been found.

Chapter 5 - Variance Reduction Techniques

Link prediction based on temporal similarity metrics using continuous action set learning automata

We propose a new particle swarm optimization algorithm for problems where objective functions are subject to zero-mean, independent, and identically distributed stochastic noise. While particle swarm optimization has been successfully applied to solve many complex deterministic nonlinear optimization problems, straightforward applications of particle swarm optimization to noisy optimization problems are subject to failure because the noise in objective function values can lead the algorithm to incorrectly identify positions as the global/personal best positions. Instead of having the entire swarm follow a global best position based on the sample average of objective function values, the proposed new algorithm works with a set of statistically global best positions that include one or more positions with objective function values that are statistically equivalent, which is achieved using a combination of statistical subset selection and clustering analysis. The new PSO algorithm can be seamlessly integrated with adaptive resampling procedures to enhance the capability of PSO to cope with noisy objective functions. Numerical experiments demonstrate that the new algorithm is able to consistently find better solutions than the canonical particle swarm optimization algorithm in the presence of stochastic noise in objective function values with different resampling procedures.

Population statistics for particle swarm optimization: Resampling methods in noisy optimization problems

Refined Stratified Sampling for efficient Monte Carlo based uncertainty quantification

In this article, we propose several quantization-based stratified sampling methods to reduce the variance of a Monte Carlo simulation. Theoretical aspects of stratification lead to a strong link between optimal quadratic quantization and the variance reduction that can be achieved with stratified sampling. We first put the emphasis on the consistency of quantization for partitioning the state space in stratified sampling methods in both finite and infinite-dimensional cases. We show that the proposed quantization-based strata design has uniform efficiency among the class of Lipschitz continuous functionals. Then a stratified sampling algorithm based on product functional quantization is proposed for path-dependent functionals of multi-factor diffusions. The method is also available for other Gaussian processes such as Brownian bridge or Ornstein–Uhlenbeck processes. We derive in detail the case of Ornstein–Uhlenbeck processes. We also study the balance between the algorithmic complexity of the simulation and the variance reduction factor.

We provide a thorough analysis of the effectiveness of different Variance Reduction Techniques (VRTs). We consider both stand-alone and combined applications of two input techniques, Antithetic Variates (AV) and Latin Hypercube Sampling (LHS), and two output techniques, Control Variates (CV) and Poststratified Sampling (PS). Previous research in the area mainly focuses on asymptotic variance reduction. In this experimental study, we measure the performance of VRTs under finite simulation run lengths and analyze their effects. Our findings show that the asymptotic variance reduction results do not readily apply to finite-length simulations. We consider three different types of systems (M/M/1, serial production line and (s, S) inventory control systems) and compare the VRTs under various experimental conditions. We observe that a variance reduction cannot be guaranteed for every instance a VRT is applied. Our results also indicate that the output VRTs (CV, PS) are better than input VRTs (AV, LHS) on the average for the single systems considered in this study. More interestingly, the less-sophisticated techniques (AV, CV) often perform better than the relatively more-complex techniques (LHS, PS). A comprehensive bibliography is also provided.

This chapter develops methods for increasing the efficiency of Monte Carlo simulation by reducing the variance of simulation estimates. These methods draw on two broad strategies for reducing variance: taking advantage of tractable features of a model to adjust or correct simulation outputs, and reducing the variability in simulation inputs. We discuss control variates, antithetic variates, stratified sampling, Latin hypercube sampling, moment matching methods, and importance sampling, and we illustrate these methods through examples. Two themes run through this chapter: The greatest gains in efficiency from variance reduction techniques result from exploiting specific features of a problem, rather than from generic applications of generic methods. Reducing simulation error is often at odds with convenient estimation of the simulation error itself; in order to supplement a reduced-variance estimator with a valid confidence interval, we sometimes need to sacrifice some of the potential variance reduction. KeywordsImportance SamplingVariance ReductionLatin Hypercube SamplingGeometric Brownian MotionUnderlying AssetThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.