Optimized Population Monte Carlo

The population Monte Carlo (PMC) algorithm is a popular adaptive importance sampling (AIS) method used for approximate computation of intractable integrals. Over the years, many advances have been made in the theory and implementation of PMC schemes. The mixture PMC (M-PMC) algorithm, for instance, optimizes the parameters of a mixture proposal distribution in a way that minimizes that Kullback-Leibler divergence to the target distribution. The parameters in M-PMC are updated using a single step of expectation maximization (EM), which limits its accuracy. In this work, we introduce a novel M-PMC algorithm that optimizes the parameters of a mixture proposal distribution, where parameter updates are resolved via stochastic optimization instead of EM. The stochastic gradients w.r.t. each of the mixture parameters are approximated using a population of Markov chain Monte Carlo samplers. We validate the proposed scheme via numerical simulations on an example where the considered target distribution is multimodal.

The surrogate model-based sampling method has been an active area of research due to its evident efficiency over direct sampling method. The fusion of the adaptive kriging (AK) model with the adaptive radial-based importance sampling (ARBIS) method is developed as an improvement of the AK model combined with Monte Carlo simulation (MCS) method especially for estimating the small failure probabilities. In the AK–ARBIS method, the number of candidate samples in each updating process of the kriging model is reduced through partitioning the whole MCS sample domain into different subdomains and removing the samples inside the last subdomain from the participating candidate sampling pool, which can avoid the time-consuming matrix operation involved in kriging predictions in order to find the best training sample in each updating step. In the existing AK–ARBIS method, kriging model stops updating according to the learning function-based stopping criterion, which cannot directly reflect the error of the estimated failure probability. In this regard, this paper proposes the error-based stopping criterion (ESC) enhanced AK–ARBIS method, where the processes of finding the optimal hypersphere and accurately identifying the sign of candidate samples are decoupled in order to successfully embed the ESC. In addition to embedding the ESC into the AK–ARBIS method, two target-oriented candidate sampling reduction strategies are investigated, respectively, to improve the efficiency of finding the optimal hypersphere and accurately identifying the sign of the participating candidate samples. Based on the candidate sampling pool reduction strategies and ESC, both the computational time and the required number of calls to the real limit state function are reduced over the existing AK–ARBIS method. Three case studies are analyzed to demonstrate the accuracy and efficiency of the proposed enhanced AK–ARBIS method, especially for estimating the small failure probabilities.

Importance sampling (IS) is a powerful Monte Carlo (MC) methodology for approximating integrals, for instance in the context of Bayesian inference. In IS, the samples are simulated from the so-called proposal distribution, and the choice of this proposal is key for achieving a high performance. In adaptive IS (AIS) methods, a set of proposals is iteratively improved. AIS is a relevant and timely methodology although many limitations remain yet to be overcome, e.g., the curse of dimensionality in high-dimensional and multi-modal problems. Moreover, the Hamiltonian Monte Carlo (HMC) algorithm has become increasingly popular in machine learning and statistics. HMC has several appealing features such as its exploratory behavior, especially in high-dimensional targets, when other methods suffer. In this letter, we introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. The proposed HAIS efficiently adapts a population of proposals, extracting the advantages of HMC. HAIS can be understood as a particular instance of the generic layered AIS family with an additional resampling step. HAIS achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms. We discuss the statistical properties of HAIS and show its high performance in two challenging examples.

Reliability-based evaluation of automotive wind noise quality

This paper is a survey of the major techniques and approaches available for the numerical approximation of integrals in statistics. We classify these into five broad categories; namely, asymptotic methods, importance sampling, adaptive importance sampling, multiple quadrature and Markov chain methods. Each method is discussed, giving an outline of the basic supporting theory and particular features of the technique. Conclusions are drawn concerning the relative merits of the methods based on the discussion and their application to three examples. The following broad recommendations are made. Asymptotic methods should only be considered in contexts where the integrand has a dominant peak with approximate ellipsoidal symmetry. Importance sampling (and preferably adaptive importance sampling) based on a multivariate Student should be used instead of asymptotics methods in such a context. Multiple quadrature and, in particular, subregion adaptive integration are the algorithms of choice for lower-dimensional integrals. Due to the difficulties in assessing convergence to stationarity and the error in estimates, Markov chain methods are recommended only when there is no adequate alternative. In certain very high dimensional problems, however, Markov chain methods are the only hope. The importance of the parameterization of the integral is noted for the success of all the methods, and several useful reparameterizations are presented.

Adaptive Monte Carlo methods for solving hyperbolic telegraph equation

Rare-event yield analysis is challenging for high-dimensional circuit cases. In this paper, we propose a non-Gaussian adaptive importance sampling (NGAIS) method. In order to approximate the failure region in high-dimensional space, we model it as a mixture of von Mises-Fisher distributions. We formulate the parameter estimation problem as a maximum likelihood estimation problem, and then solve with expectation-maximization algorithm. Experiments on bit cell, amplifier and SRAM column circuit validate that the proposed NGAIS method outperforms other state-of-the-art approaches in terms of accuracy and efficiency.

A Markov chain simulation was performed to extract points in a failure region. A Kriging metamodel was constructed to approximate a limit state based on the points extracted by the Markov chain simulation. A kernel sampling density was constructed to approximate the optimal importance sampling density. The points extracted in the failure region by the Markov chain simulation were assumed as a mean of each kernel. An importance sampling method was applied to calculate the failure probability. In the importance sampling method, points are extracted from the kernel in the vicinity of a limit state. Considering the statistical distance as well as the learning function, additional experimental points were selected for the kriging metamodel. A stable numerical calculation method was applied to find the parameters of the kernel sampling density. The completeness of the Kriging metamodel was evaluated on the basis of possible changes in failure probability.

Stochastic gradient descent (SGD) usually samples training data based on the uniform distribution, which may not be a good choice because of the high variance of its stochastic gradient. Thus, importance sampling methods are considered in the literature to improve the performance. Most previous work on SGD-based methods with importance sampling requires the knowledge of Lipschitz constants of all component gradients, which are in general difficult to estimate. In this paper, we study an adaptive importance sampling method for common SGD-based methods by exploiting the local first-order information without knowing any Lipschitz constants. In particular, we periodically changes the sampling distribution by only utilizing the gradient norms in the past few iterations. We prove that our adaptive importance sampling non-asymptotically reduces the variance of the stochastic gradients in SGD, and thus better convergence bounds than that for vanilla SGD can be obtained. We extend this sampling method to several other widely used stochastic gradient algorithms including SGD with momentum and ADAM. Experiments on common convex learning problems and deep neural networks illustrate notably enhanced performance using the adaptive sampling strategy.

This paper presents a new methodology for reliability-based inspection planning focusing on robust and accurate computational strategies for fatigue-reliability updating using inspection results. The core of the proposed strategy is a conditioned sampling-based method, implemented by a Fast Probability Analyzer (FPA) software where efficiency is achieved by using the importance sampling principal. For a single component or limit state, FPA first generates Markov-Chain Monte Carlo (MCMC) samples in the failure domain, then applies an adaptive stratified importance sampling (ASIS) method to compute probability of failure (PoF) with error control. Once the MCMC samples have been created, solving a reliability updating problem is fairly straightforward and computationally robust relative to the conventional system reliability methods that rely on linearization of the limit states. The new approach is demonstrated using examples including stiffened panels of a ship-shaped vessel where reliability is updated using inspection results from 100 panel connections.

Regional importance effect analysis of the input variables on failure probability

Structural systems often utilize scheduled maintenance to sustain structural reliability throughout the service life or even extend the original design life. However, inspection schedules are often arbitrary, e.g., use equal intervals, which tend to be on the conservative side for new structures and non-conservatively for aging structures. Although many reliability methods are useful for maintenance planning, there is a no generic methodology to conduct risk or reliability-based maintenance optimization (RBMO). The progress in RBMO technology development has been slow partly because there is no integrated methodology and because computationally efficient tools to study and resolve RBMO issues are lacking. This paper presents an integrated, efficient, and versatile RBMO methodology built on a two-stage random simulation framework and features three tightly-integrated efficient methods: (1) a new adaptive reliability-based adaptive meta-modeling (RAM) approach to establish an accurate and fast-running approximate damage tolerance model, (2) an adaptive stratified importance sampling (ASIS) method for computing probability-offailure without inspection and to generate random failure samples, and (3) the recursive probability integration (RPI) method for computing probability-offailure with inspections and repairs. The RBMO methodology has been implemented in a Fast Probability Analyzer software and is demonstrated using a fracture mechanics code with several random variables, a random POD, several repair qualities, and multiple inspections.

Optimization methods have been widely applied in sheet metal forming area. However, the existence of variations during manufacturing processes significantly may influence final product quality, rendering non-robust optimal solutions. In this paper, experiments were conducted to investigate how a stochastic behavior of noise factors affected stamping quality. Robust design models for sheet metal forming process integrated adaptive importance sampling with response surface method, in order to minimize impact of the variations and achieve reliable process parameters. Support Vector Machine with nonlinear capability in both pattern recognition and regression was adopted to map the relation between input process parameters and part quality. A cup drawing example was employed to verify the feasibility of the proposed method. Comparisons were conducted between different optimization models to demonstrate robustness of the adaptive importance sampling method. Final results showed that the stamping part quality was optimized under the specified constraint requirements.

The concept of virtual power plant (VPP) has been proposed to manage distributed renewable energy sources as packaging to engage in the energy and reserve planning on contemporary generating portfolios. In this context, an efficient tool is needed to support analysis of generating flexibility of conventional units in combination with VPPs to cooperatively counterbalance the fluctuation of net demand. In this paper, an adaptive importance sampling method is proposed, intentionally for efficiently evaluating specific indices capturing the possibility and severity of rare inadequate spinning reserve events of a deployed unit comment schedule for a generating system incorporating VPPs, in terms of short-term stochastic unit failures and power fluctuation of VPPs. The proposed method is based on the standard cross-entropy (CE) method, which newly introduces a mathematical transformation aiming at diverting evaluations of customized risk indices to a generic rare-event probability estimation problem. A Markov chain Monte Carlo method is employed to train the proposal density, to efficiently gain on the best, owing to avoiding the iterative parameter-updating mechanism of the standard CE method. The efficacy of the proposed method is tested on a modified RTS-96 generating system emulating a portfolio of conventional and renewable generating sources modeled as VPPs.

A probabilistic procedure for the assessment of the time-dependent reliability of existing prestressed concrete (PSC) box-girder bridges is presented. These bridges are subject to increased traffic loads and an aggressive environment, which result in structural deterioration such as cracking and corrosion. In this paper, a multiple-peak vehicle load model is proposed based on acquired traffic load data, which cannot be properly described by previous single-peak probability distributions. With the proposed vehicle load model, the maximal vehicle loads during the remaining life of bridges are obtained for evaluating the influence of the increase in traffic loads on bridge reliability. Time-dependent corrosion models are adopted to account for pitting corrosion because of chloride attack as well as uniform corrosion because of concrete carbonation. A degenerated shell element is used for accurate and efficient modeling of the PSC box-girder. Postcracking behaviors of reinforced concrete box-girders, simulated by using different crack models and elements, are compared with laboratory test results. To increase the efficiency of probabilistic analyses, an adaptive importance sampling method in conjunction with the truncated Latin hypercube sampling is used. Application of the proposed methodologies is demonstrated by a case study, in which the time-dependent reliabilities of a deteriorating PSC box-girder bridge are obtained.