An adaptive importance sampling method with a Kriging metamodel to calculate failure probability

The surrogate model-based sampling method has been an active area of research due to its evident efficiency over direct sampling method. The fusion of the adaptive kriging (AK) model with the adaptive radial-based importance sampling (ARBIS) method is developed as an improvement of the AK model combined with Monte Carlo simulation (MCS) method especially for estimating the small failure probabilities. In the AK–ARBIS method, the number of candidate samples in each updating process of the kriging model is reduced through partitioning the whole MCS sample domain into different subdomains and removing the samples inside the last subdomain from the participating candidate sampling pool, which can avoid the time-consuming matrix operation involved in kriging predictions in order to find the best training sample in each updating step. In the existing AK–ARBIS method, kriging model stops updating according to the learning function-based stopping criterion, which cannot directly reflect the error of the estimated failure probability. In this regard, this paper proposes the error-based stopping criterion (ESC) enhanced AK–ARBIS method, where the processes of finding the optimal hypersphere and accurately identifying the sign of candidate samples are decoupled in order to successfully embed the ESC. In addition to embedding the ESC into the AK–ARBIS method, two target-oriented candidate sampling reduction strategies are investigated, respectively, to improve the efficiency of finding the optimal hypersphere and accurately identifying the sign of the participating candidate samples. Based on the candidate sampling pool reduction strategies and ESC, both the computational time and the required number of calls to the real limit state function are reduced over the existing AK–ARBIS method. Three case studies are analyzed to demonstrate the accuracy and efficiency of the proposed enhanced AK–ARBIS method, especially for estimating the small failure probabilities.

This paper presents a new methodology for reliability-based inspection planning focusing on robust and accurate computational strategies for fatigue-reliability updating using inspection results. The core of the proposed strategy is a conditioned sampling-based method, implemented by a Fast Probability Analyzer (FPA) software where efficiency is achieved by using the importance sampling principal. For a single component or limit state, FPA first generates Markov-Chain Monte Carlo (MCMC) samples in the failure domain, then applies an adaptive stratified importance sampling (ASIS) method to compute probability of failure (PoF) with error control. Once the MCMC samples have been created, solving a reliability updating problem is fairly straightforward and computationally robust relative to the conventional system reliability methods that rely on linearization of the limit states. The new approach is demonstrated using examples including stiffened panels of a ship-shaped vessel where reliability is updated using inspection results from 100 panel connections.

Rare-event yield analysis is challenging for high-dimensional circuit cases. In this paper, we propose a non-Gaussian adaptive importance sampling (NGAIS) method. In order to approximate the failure region in high-dimensional space, we model it as a mixture of von Mises-Fisher distributions. We formulate the parameter estimation problem as a maximum likelihood estimation problem, and then solve with expectation-maximization algorithm. Experiments on bit cell, amplifier and SRAM column circuit validate that the proposed NGAIS method outperforms other state-of-the-art approaches in terms of accuracy and efficiency.

Reliability-based evaluation of automotive wind noise quality

This paper is a survey of the major techniques and approaches available for the numerical approximation of integrals in statistics. We classify these into five broad categories; namely, asymptotic methods, importance sampling, adaptive importance sampling, multiple quadrature and Markov chain methods. Each method is discussed, giving an outline of the basic supporting theory and particular features of the technique. Conclusions are drawn concerning the relative merits of the methods based on the discussion and their application to three examples. The following broad recommendations are made. Asymptotic methods should only be considered in contexts where the integrand has a dominant peak with approximate ellipsoidal symmetry. Importance sampling (and preferably adaptive importance sampling) based on a multivariate Student should be used instead of asymptotics methods in such a context. Multiple quadrature and, in particular, subregion adaptive integration are the algorithms of choice for lower-dimensional integrals. Due to the difficulties in assessing convergence to stationarity and the error in estimates, Markov chain methods are recommended only when there is no adequate alternative. In certain very high dimensional problems, however, Markov chain methods are the only hope. The importance of the parameterization of the integral is noted for the success of all the methods, and several useful reparameterizations are presented.

Importance sampling (IS) is a powerful Monte Carlo (MC) methodology for approximating integrals, for instance in the context of Bayesian inference. In IS, the samples are simulated from the so-called proposal distribution, and the choice of this proposal is key for achieving a high performance. In adaptive IS (AIS) methods, a set of proposals is iteratively improved. AIS is a relevant and timely methodology although many limitations remain yet to be overcome, e.g., the curse of dimensionality in high-dimensional and multi-modal problems. Moreover, the Hamiltonian Monte Carlo (HMC) algorithm has become increasingly popular in machine learning and statistics. HMC has several appealing features such as its exploratory behavior, especially in high-dimensional targets, when other methods suffer. In this letter, we introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. The proposed HAIS efficiently adapts a population of proposals, extracting the advantages of HMC. HAIS can be understood as a particular instance of the generic layered AIS family with an additional resampling step. HAIS achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms. We discuss the statistical properties of HAIS and show its high performance in two challenging examples.

Regional importance effect analysis of the input variables on failure probability

Stochastic gradient descent (SGD) usually samples training data based on the uniform distribution, which may not be a good choice because of the high variance of its stochastic gradient. Thus, importance sampling methods are considered in the literature to improve the performance. Most previous work on SGD-based methods with importance sampling requires the knowledge of Lipschitz constants of all component gradients, which are in general difficult to estimate. In this paper, we study an adaptive importance sampling method for common SGD-based methods by exploiting the local first-order information without knowing any Lipschitz constants. In particular, we periodically changes the sampling distribution by only utilizing the gradient norms in the past few iterations. We prove that our adaptive importance sampling non-asymptotically reduces the variance of the stochastic gradients in SGD, and thus better convergence bounds than that for vanilla SGD can be obtained. We extend this sampling method to several other widely used stochastic gradient algorithms including SGD with momentum and ADAM. Experiments on common convex learning problems and deep neural networks illustrate notably enhanced performance using the adaptive sampling strategy.

Adaptive importance sampling (AIS) methods are increasingly used for the approximation of distributions and related intractable integrals in the context of Bayesian inference. Population Monte Carlo (PMC) algorithms are a subclass of AIS methods, widely used due to their ease in the adaptation. In this paper, we propose a novel algorithm that exploits the benefits of the PMC framework and includes more efficient adaptive mechanisms, exploiting geometric information of the target distribution. In particular, the novel algorithm adapts the location and scale parameters of a set of importance densities (proposals). At each iteration, the location parameters are adapted by combining a versatile resampling strategy (i.e., using the information of previous weighted samples) with an advanced optimization-based scheme. Local second-order information of the target distribution is incorporated through a preconditioning matrix acting as a scaling metric onto a gradient direction. A damped Newton approach is adopted to ensure robustness of the scheme. The resulting metric is also used to update the scale parameters of the proposals. We discuss several key theoretical foundations for the proposed approach. Finally, we show the successful performance of the proposed method in three numerical examples, involving challenging distributions.

In this paper, we present a variance minimization (VM) procedure for rare event simulation in tandem queueing networks. We prove that the VM method can produce a zero variance. The VM method is suitable to compute optimal importance sampling (IS) parameters for small scale tandem networks. For large scale tandem networks we propose a sub-optimal IS (SOIS) method, which projects the optimal biased transition probabilities of the corresponding small scale system into those of a large scale system. In other words, we establish an efficient IS method for a large scale system by zooming into a small scale system and then projecting our findings into the large scale system. The numerical results show that our SOIS method can produce accurate results with very short CPU time, while many other methods often require much longer.

The performance of low-density parity-check (LDPC) codes can be evaluated by using the dual adaptive importance sampling (DAIS) method. To further accelerate the convergence speed of simulation, an improved method is proposed in which the biased distribution is multiplied with an auxiliary biased function. In this proposed method, samples are limited in the restricted region, where most of the events result in decoding failure, by multiplying the original biased distribution of DAIS method by a second bias. This scheme speeds up the convergence of simulation in the restricted region. Compared with the traditional DAIS method, simulation results for two specific LDPC codes verify that this proposed method achieves the significant simulation gain without sacrificing the accuracy.

Structural systems often utilize scheduled maintenance to sustain structural reliability throughout the service life or even extend the original design life. However, inspection schedules are often arbitrary, e.g., use equal intervals, which tend to be on the conservative side for new structures and non-conservatively for aging structures. Although many reliability methods are useful for maintenance planning, there is a no generic methodology to conduct risk or reliability-based maintenance optimization (RBMO). The progress in RBMO technology development has been slow partly because there is no integrated methodology and because computationally efficient tools to study and resolve RBMO issues are lacking. This paper presents an integrated, efficient, and versatile RBMO methodology built on a two-stage random simulation framework and features three tightly-integrated efficient methods: (1) a new adaptive reliability-based adaptive meta-modeling (RAM) approach to establish an accurate and fast-running approximate damage tolerance model, (2) an adaptive stratified importance sampling (ASIS) method for computing probability-offailure without inspection and to generate random failure samples, and (3) the recursive probability integration (RPI) method for computing probability-offailure with inspections and repairs. The RBMO methodology has been implemented in a Fast Probability Analyzer software and is demonstrated using a fracture mechanics code with several random variables, a random POD, several repair qualities, and multiple inspections.

The population Monte Carlo (PMC) algorithm is a popular adaptive importance sampling (AIS) method used for approximate computation of intractable integrals. Over the years, many advances have been made in the theory and implementation of PMC schemes. The mixture PMC (M-PMC) algorithm, for instance, optimizes the parameters of a mixture proposal distribution in a way that minimizes that Kullback-Leibler divergence to the target distribution. The parameters in M-PMC are updated using a single step of expectation maximization (EM), which limits its accuracy. In this work, we introduce a novel M-PMC algorithm that optimizes the parameters of a mixture proposal distribution, where parameter updates are resolved via stochastic optimization instead of EM. The stochastic gradients w.r.t. each of the mixture parameters are approximated using a population of Markov chain Monte Carlo samplers. We validate the proposed scheme via numerical simulations on an example where the considered target distribution is multimodal.

Adaptive Monte Carlo methods for solving hyperbolic telegraph equation

The concept of virtual power plant (VPP) has been proposed to manage distributed renewable energy sources as packaging to engage in the energy and reserve planning on contemporary generating portfolios. In this context, an efficient tool is needed to support analysis of generating flexibility of conventional units in combination with VPPs to cooperatively counterbalance the fluctuation of net demand. In this paper, an adaptive importance sampling method is proposed, intentionally for efficiently evaluating specific indices capturing the possibility and severity of rare inadequate spinning reserve events of a deployed unit comment schedule for a generating system incorporating VPPs, in terms of short-term stochastic unit failures and power fluctuation of VPPs. The proposed method is based on the standard cross-entropy (CE) method, which newly introduces a mathematical transformation aiming at diverting evaluations of customized risk indices to a generic rare-event probability estimation problem. A Markov chain Monte Carlo method is employed to train the proposal density, to efficiently gain on the best, owing to avoiding the iterative parameter-updating mechanism of the standard CE method. The efficacy of the proposed method is tested on a modified RTS-96 generating system emulating a portfolio of conventional and renewable generating sources modeled as VPPs.