Efficient Importance Sampling for High-sigma Yield Analysis with Adaptive Online Surrogate Modeling

An efficient time-dependent (TD) Monte Carlo (MC) importance sampling method has recently been developed [G. Tao and W. H. Miller, J. Chem. Phys. 135, 024104 (2011)] for the evaluation of time correlation functions using the semiclassical (SC) initial value representation (IVR) methodology. In this TD-SC-IVR method, the MC sampling uses information from both time-evolved phase points as well as their initial values, and only the "important" trajectories are sampled frequently. Even though the TD-SC-IVR was shown in some benchmark examples to be much more efficient than the traditional time-independent sampling method (which uses only initial conditions), the calculation of the SC prefactor-which is computationally expensive, especially for large systems-is still required for accepted trajectories. In the present work, we present an approximate implementation of the TD-SC-IVR method that is completely prefactor-free; it gives the time correlation function as a classical-like magnitude function multiplied by a phase function. Application of this approach to flux-flux correlation functions (which yield reaction rate constants) for the benchmark H + H(2) system shows very good agreement with exact quantum results. Limitations of the approximate approach are also discussed.

In this paper, we present a variance minimization (VM) procedure for rare event simulation in tandem queueing networks. We prove that the VM method can produce a zero variance. The VM method is suitable to compute optimal importance sampling (IS) parameters for small scale tandem networks. For large scale tandem networks we propose a sub-optimal IS (SOIS) method, which projects the optimal biased transition probabilities of the corresponding small scale system into those of a large scale system. In other words, we establish an efficient IS method for a large scale system by zooming into a small scale system and then projecting our findings into the large scale system. The numerical results show that our SOIS method can produce accurate results with very short CPU time, while many other methods often require much longer.

The failure rate of static RAM (SRAM) cells is restricted to be extremely low to ensure sufficient high yield for the entire chip. In addition, multiple performances of interest and influences from peripherals make SRAM failure rate estimation a high-dimensional multiple-failure-region problem. This paper proposes a new method featuring a multistart-point sequential quadratic programming (SQP) framework to extend minimized norm importance sampling (IS) to address this problem. Failure regions in the variation space are first found by the low-discrepancy sampling sequence. Afterward, start points are generated in all identified failure regions and local optimizations based on SQP are invoked from these start points searching for the optimal shift vectors (OSVs). Based on the OSVs, a Gaussian mixture distorted distribution is constructed for IS. To further reduce the computational cost of IS while fully considering the influence of increasing dimensionality, an adaptive model training framework is proposed to keep high efficiency for both low- and high-dimensional problems. The experimental results show that the proposed method can not only approximate failure rate with high accuracy and efficiency in low-dimensional cases but also keep these features in high-dimensional ones.

An importance sampling (IS) simulation method is developed for evaluating the bit-error rate of one-shot and recursive Bayesian equalizers (symbol-by-symbol detectors.) It is shown that the one-shot Bayesian equalizer structure closely resembles that of a multi-user detector for CDMA. Hence, noise biasing IS methods developed previously for CDMA detectors should also be appropriate for Bayesian equalizers. At low SNRs, the decision boundary far the one-shot Bayesian equalizer approaches a hyperplane. This hyperplane is viewed as an approximation to the true decision boundary at high SNRs, and can thus be used to construct an appropriate noise bias vector. With a slight modification, it is shown that the IS technique for the one-shot Bayesian equalizer can also be applied to the Abend and Fritchman (1970) recursive algorithm, and finally, to an adaptive Bayesian blind equalizer. IS simulations are carried out for both the nonadaptive Bayesian equalizers and for the blind recursive equalizer. >

Vehicle routing problem belongs to classical Combination Optimization hard problem and has been extensively studied by many researchers. Vehicle Routing Problem with Weight Coefficients and Stochastic Demands (WVRPSD) is established and a new implementation for importance sampling method to solve this model is proposed in this paper, in which the classical exponential change of measure is adopted to construct the family of importance sampling distributions, and the optimal importance sampling distribution is obtained by minimizing the variance of importance sampling estimator. Numerical experiments have been conducted and the results indicate that the method can effectively solve this problem.

Estimating the distribution of total default losses on the Spanish financial system

The importance sampling method is an extensively used numerical simulation method in reliability analysis. In this paper, a modification to the importance sampling method (ISM) is proposed, and the modified ISM divides the sample set of input variables into different subsets based on the contributive weight of the importance sample defined in this paper and the maximum super-sphere denoted by β-sphere in the safe domain defined by the truncated ISM. By this proposed modification, only samples with large contributive weight and locating outside of the β-sphere need to call the limit state function. This amelioration remarkably reduces the number of limit state function evaluations required in the simulation procedure, and it doesn’t sacrifice the precision of the results by controlling the level of relative error. Based on this modified ISM and the space-partition idea in variance-based sensitivity analysis, the global reliability sensitivity indices can be estimated as byproducts, which is especially useful for reliability-based design optimization. This process of estimating the global reliability sensitivity indices only needs the sample points used in reliability analysis and is independent of the dimensionality of input variables. A roof truss structure and a composite cantilever beam structure are analyzed by the modified ISM. The results demonstrate the efficiency, accuracy, and robustness of the proposed method.

Statistical analysis of SRAM has emerged as a challenging issue because the failure rate of SRAM cells is extremely small. In this paper, we develop an efficient importance sampling algorithm to capture the rare failure event of SRAM cells. In particular, we adapt the Gibbs sampling technique from the statistics community to find the optimal probability distribution for importance sampling with minimum computational cost (i.e., a small number of transistor-level simulations). The proposed Gibbs sampling method applies an integrated optimization engine to adaptively explore the failure region by sampling a sequence of one-dimensional probability distributions. Several implementation issues such as one-dimensional random sampling and starting point selection are carefully studied to make the Gibbs sampling method efficient and accurate for SRAM failure rate prediction. Our experimental results of a commercial 65nm SRAM cell demonstrate that the proposed Gibbs sampling method achieves 3~10x runtime speed-up over other state-of-the-art techniques without surrendering any accuracy.

An importance sampling method is presented for computing the first passage probability of elasto-plastic structures under stochastic excitations. The importance sampling distribution corresponds to shifting the mean of the excitation to an \'adapted\' stochastic process whose future is determined based on information only up to the present. A stochastic control approach is adopted for designing the adapted process. The optimal control law is determined by a control potential, which satisfies the Bellman\'s equation, a nonlinear partial differential equation on the response state-space. Numerical results for a single-degree-of freedom elasto-plastic structure shows that the proposed method leads to significant improvement in variance reduction over importance sampling using design points reported recently.

To reduce the substantial leakage current, the supply voltage of SRAM cells has being scaled down towards its lower limit, which is called the data Retention Voltage (DRV). Although the power consumption is largely reduced, this down-scaling trend, however, impacts the stability of the SRAM cell due to the unpredictable process or device parameter variations. In this work, we propose a novel method to evaluate the DRV of SRAM cells at the presence of variations. The DRV issue is first formulated as a time domain worst performance bound problem. To accurately and efficiently evaluate the DRV, a multi-start point (MSP) optimization strategy is then studied and developed with the use of practical circuit simulator. One feature of the proposed method is that it can efficiently evaluate the DRV without suffering from any process/model accuracy. Experiment results show that it achieves a speedup of 3 and 5-7 order over the Importance Sampling (IS) and Monte Carlo (MC) method respectively under the context of the DRV evaluation in this paper. The proposed method can serve as an efficient DRV evaluation tool on any specific technology process or in-house circuit simulator. In this work, the DRVs at the technology node from 130 nm to 45 nm under the influence of different variation sources are also presented and analyzed.

Value at Risk (VaR) is an important tool for estimating the risk of a financial portfolio under significant loss. Although Monte Carlo simulation is a powerful tool for estimating VaR, it is quite inefficient since the event of significant loss is usually rare. Previous studies suggest that the performance of the Monte Carlo simulation can be improved by impor-tance sampling if the market returns follow the normality or the distributions. The first contribution of our paper is to extend the importance sampling method for dealing with jump-diffusion market returns, which can more precisely model the phenomenon of high peaks, heavy tails, and jumps of market returns mentioned in numerous empirical study papers. This paper also points out that for portfolios of which the huge loss is triggered by significantly distinct events, naively applying importance sampling method can result in poor performance. The second contribution of our paper is to develop the hybrid importance sampling method for the aforementioned problem. Our method decomposes a Monte Carlo simulation into sub simulations, and each sub simulation focuses only on one huge loss event. Thus the perform-ance for each sub simulation is improved by importance sampling method, and overall performance is optimized by determining the allotment of samples to each sub simulation by Lagrange’s multiplier. Numerical experiments are given to verify the superiority of our method.

A Markov chain simulation was performed to extract points in a failure region. A Kriging metamodel was constructed to approximate a limit state based on the points extracted by the Markov chain simulation. A kernel sampling density was constructed to approximate the optimal importance sampling density. The points extracted in the failure region by the Markov chain simulation were assumed as a mean of each kernel. An importance sampling method was applied to calculate the failure probability. In the importance sampling method, points are extracted from the kernel in the vicinity of a limit state. Considering the statistical distance as well as the learning function, additional experimental points were selected for the kriging metamodel. A stable numerical calculation method was applied to find the parameters of the kernel sampling density. The completeness of the Kriging metamodel was evaluated on the basis of possible changes in failure probability.

Stochastic gradient descent (SGD) usually samples training data based on the uniform distribution, which may not be a good choice because of the high variance of its stochastic gradient. Thus, importance sampling methods are considered in the literature to improve the performance. Most previous work on SGD-based methods with importance sampling requires the knowledge of Lipschitz constants of all component gradients, which are in general difficult to estimate. In this paper, we study an adaptive importance sampling method for common SGD-based methods by exploiting the local first-order information without knowing any Lipschitz constants. In particular, we periodically changes the sampling distribution by only utilizing the gradient norms in the past few iterations. We prove that our adaptive importance sampling non-asymptotically reduces the variance of the stochastic gradients in SGD, and thus better convergence bounds than that for vanilla SGD can be obtained. We extend this sampling method to several other widely used stochastic gradient algorithms including SGD with momentum and ADAM. Experiments on common convex learning problems and deep neural networks illustrate notably enhanced performance using the adaptive sampling strategy.

Numerical aspects of application of FORM in node snapping truss structures

Importance sampling (IS) is a powerful Monte Carlo (MC) methodology for approximating integrals, for instance in the context of Bayesian inference. In IS, the samples are simulated from the so-called proposal distribution, and the choice of this proposal is key for achieving a high performance. In adaptive IS (AIS) methods, a set of proposals is iteratively improved. AIS is a relevant and timely methodology although many limitations remain yet to be overcome, e.g., the curse of dimensionality in high-dimensional and multi-modal problems. Moreover, the Hamiltonian Monte Carlo (HMC) algorithm has become increasingly popular in machine learning and statistics. HMC has several appealing features such as its exploratory behavior, especially in high-dimensional targets, when other methods suffer. In this letter, we introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. The proposed HAIS efficiently adapts a population of proposals, extracting the advantages of HMC. HAIS can be understood as a particular instance of the generic layered AIS family with an additional resampling step. HAIS achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms. We discuss the statistical properties of HAIS and show its high performance in two challenging examples.