Adaptive Monte Carlo methods for solving hyperbolic telegraph equation

The surrogate model-based sampling method has been an active area of research due to its evident efficiency over direct sampling method. The fusion of the adaptive kriging (AK) model with the adaptive radial-based importance sampling (ARBIS) method is developed as an improvement of the AK model combined with Monte Carlo simulation (MCS) method especially for estimating the small failure probabilities. In the AK–ARBIS method, the number of candidate samples in each updating process of the kriging model is reduced through partitioning the whole MCS sample domain into different subdomains and removing the samples inside the last subdomain from the participating candidate sampling pool, which can avoid the time-consuming matrix operation involved in kriging predictions in order to find the best training sample in each updating step. In the existing AK–ARBIS method, kriging model stops updating according to the learning function-based stopping criterion, which cannot directly reflect the error of the estimated failure probability. In this regard, this paper proposes the error-based stopping criterion (ESC) enhanced AK–ARBIS method, where the processes of finding the optimal hypersphere and accurately identifying the sign of candidate samples are decoupled in order to successfully embed the ESC. In addition to embedding the ESC into the AK–ARBIS method, two target-oriented candidate sampling reduction strategies are investigated, respectively, to improve the efficiency of finding the optimal hypersphere and accurately identifying the sign of the participating candidate samples. Based on the candidate sampling pool reduction strategies and ESC, both the computational time and the required number of calls to the real limit state function are reduced over the existing AK–ARBIS method. Three case studies are analyzed to demonstrate the accuracy and efficiency of the proposed enhanced AK–ARBIS method, especially for estimating the small failure probabilities.

Reliability-based evaluation of automotive wind noise quality

Adaptive importance sampling (AIS) methods are increasingly used for the approximation of distributions and related intractable integrals in the context of Bayesian inference. Population Monte Carlo (PMC) algorithms are a subclass of AIS methods, widely used due to their ease in the adaptation. In this paper, we propose a novel algorithm that exploits the benefits of the PMC framework and includes more efficient adaptive mechanisms, exploiting geometric information of the target distribution. In particular, the novel algorithm adapts the location and scale parameters of a set of importance densities (proposals). At each iteration, the location parameters are adapted by combining a versatile resampling strategy (i.e., using the information of previous weighted samples) with an advanced optimization-based scheme. Local second-order information of the target distribution is incorporated through a preconditioning matrix acting as a scaling metric onto a gradient direction. A damped Newton approach is adopted to ensure robustness of the scheme. The resulting metric is also used to update the scale parameters of the proposals. We discuss several key theoretical foundations for the proposed approach. Finally, we show the successful performance of the proposed method in three numerical examples, involving challenging distributions.

Importance sampling (IS) is a powerful Monte Carlo (MC) methodology for approximating integrals, for instance in the context of Bayesian inference. In IS, the samples are simulated from the so-called proposal distribution, and the choice of this proposal is key for achieving a high performance. In adaptive IS (AIS) methods, a set of proposals is iteratively improved. AIS is a relevant and timely methodology although many limitations remain yet to be overcome, e.g., the curse of dimensionality in high-dimensional and multi-modal problems. Moreover, the Hamiltonian Monte Carlo (HMC) algorithm has become increasingly popular in machine learning and statistics. HMC has several appealing features such as its exploratory behavior, especially in high-dimensional targets, when other methods suffer. In this letter, we introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. The proposed HAIS efficiently adapts a population of proposals, extracting the advantages of HMC. HAIS can be understood as a particular instance of the generic layered AIS family with an additional resampling step. HAIS achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms. We discuss the statistical properties of HAIS and show its high performance in two challenging examples.

The combination of the three-dimensional (3D) target mirror and laser interferometers can achieve 3D precision measurement, and the orthogonality between the target mirror has a significant impact on the accuracy of the measurement system. In order to ensure the accuracy of the measurement system, it is necessary to measure the orthogonal deviation angle of the 3D target mirror and evaluate the uncertainty to verify whether the machining accuracy of the target mirror meets the requirements. In this paper, to test the orthogonality of the X-Y reflection planes of the 3D target mirror, which is glued by three plane mirrors, the evaluation methods of measurement uncertainty of the orthogonal deviation angle θxy are studied. The measurement model of uncertainty evaluation is built based on the quantitative characteristics analysis method, and the evaluation results of measurement uncertainty are obtained based on the adaptive Monte Carlo method (AMCM). Moreover, the evaluation results are compared with those evaluation results obtained respectively based on the traditional assessment method of measurement uncertainty (called as GUM method) and Monte Carlo method (MCM). Through the comparison of the evaluation results, it can be concluded that the best estimate of the orthogonal deviation angle θxy is about -3.09”, the corresponding standard uncertainty is about 0.69”, and the machining accuracy of the X-Y reflection planes of the target mirror meets the requirements. Moreover, the AMCM evaluation results is reliable, and AMCM is more convenient and efficient than GUM and MCM methods. The research work provides valuable reference for solving practical measurement evaluation problems.

This paper presents a new methodology for reliability-based inspection planning focusing on robust and accurate computational strategies for fatigue-reliability updating using inspection results. The core of the proposed strategy is a conditioned sampling-based method, implemented by a Fast Probability Analyzer (FPA) software where efficiency is achieved by using the importance sampling principal. For a single component or limit state, FPA first generates Markov-Chain Monte Carlo (MCMC) samples in the failure domain, then applies an adaptive stratified importance sampling (ASIS) method to compute probability of failure (PoF) with error control. Once the MCMC samples have been created, solving a reliability updating problem is fairly straightforward and computationally robust relative to the conventional system reliability methods that rely on linearization of the limit states. The new approach is demonstrated using examples including stiffened panels of a ship-shaped vessel where reliability is updated using inspection results from 100 panel connections.

This paper is a survey of the major techniques and approaches available for the numerical approximation of integrals in statistics. We classify these into five broad categories; namely, asymptotic methods, importance sampling, adaptive importance sampling, multiple quadrature and Markov chain methods. Each method is discussed, giving an outline of the basic supporting theory and particular features of the technique. Conclusions are drawn concerning the relative merits of the methods based on the discussion and their application to three examples. The following broad recommendations are made. Asymptotic methods should only be considered in contexts where the integrand has a dominant peak with approximate ellipsoidal symmetry. Importance sampling (and preferably adaptive importance sampling) based on a multivariate Student should be used instead of asymptotics methods in such a context. Multiple quadrature and, in particular, subregion adaptive integration are the algorithms of choice for lower-dimensional integrals. Due to the difficulties in assessing convergence to stationarity and the error in estimates, Markov chain methods are recommended only when there is no adequate alternative. In certain very high dimensional problems, however, Markov chain methods are the only hope. The importance of the parameterization of the integral is noted for the success of all the methods, and several useful reparameterizations are presented.

The population Monte Carlo (PMC) algorithm is a popular adaptive importance sampling (AIS) method used for approximate computation of intractable integrals. Over the years, many advances have been made in the theory and implementation of PMC schemes. The mixture PMC (M-PMC) algorithm, for instance, optimizes the parameters of a mixture proposal distribution in a way that minimizes that Kullback-Leibler divergence to the target distribution. The parameters in M-PMC are updated using a single step of expectation maximization (EM), which limits its accuracy. In this work, we introduce a novel M-PMC algorithm that optimizes the parameters of a mixture proposal distribution, where parameter updates are resolved via stochastic optimization instead of EM. The stochastic gradients w.r.t. each of the mixture parameters are approximated using a population of Markov chain Monte Carlo samplers. We validate the proposed scheme via numerical simulations on an example where the considered target distribution is multimodal.

Rare-event yield analysis is challenging for high-dimensional circuit cases. In this paper, we propose a non-Gaussian adaptive importance sampling (NGAIS) method. In order to approximate the failure region in high-dimensional space, we model it as a mixture of von Mises-Fisher distributions. We formulate the parameter estimation problem as a maximum likelihood estimation problem, and then solve with expectation-maximization algorithm. Experiments on bit cell, amplifier and SRAM column circuit validate that the proposed NGAIS method outperforms other state-of-the-art approaches in terms of accuracy and efficiency.

Stochastic gradient descent (SGD) usually samples training data based on the uniform distribution, which may not be a good choice because of the high variance of its stochastic gradient. Thus, importance sampling methods are considered in the literature to improve the performance. Most previous work on SGD-based methods with importance sampling requires the knowledge of Lipschitz constants of all component gradients, which are in general difficult to estimate. In this paper, we study an adaptive importance sampling method for common SGD-based methods by exploiting the local first-order information without knowing any Lipschitz constants. In particular, we periodically changes the sampling distribution by only utilizing the gradient norms in the past few iterations. We prove that our adaptive importance sampling non-asymptotically reduces the variance of the stochastic gradients in SGD, and thus better convergence bounds than that for vanilla SGD can be obtained. We extend this sampling method to several other widely used stochastic gradient algorithms including SGD with momentum and ADAM. Experiments on common convex learning problems and deep neural networks illustrate notably enhanced performance using the adaptive sampling strategy.

Optimization methods have been widely applied in sheet metal forming area. However, the existence of variations during manufacturing processes significantly may influence final product quality, rendering non-robust optimal solutions. In this paper, experiments were conducted to investigate how a stochastic behavior of noise factors affected stamping quality. Robust design models for sheet metal forming process integrated adaptive importance sampling with response surface method, in order to minimize impact of the variations and achieve reliable process parameters. Support Vector Machine with nonlinear capability in both pattern recognition and regression was adopted to map the relation between input process parameters and part quality. A cup drawing example was employed to verify the feasibility of the proposed method. Comparisons were conducted between different optimization models to demonstrate robustness of the adaptive importance sampling method. Final results showed that the stamping part quality was optimized under the specified constraint requirements.

In order to assess the precision of the parameter estimation of the nonlinear mixed additive and multiplicative errors model, there is a complicated derivative problem. This paper firstly derives the precision assessment formula of the existing solution method of the model based on the approximate function expression method, and then introduces the Monte Carlo method in the probability distribution method of the approximate function to assess the precision of the parameter estimation results of the model, which can avoid the complicated derivative operation. Meanwhile, in response to the low computational efficiency of the Monte Carlo method and the blindness in the selection of the number of simulations, an adaptive Monte Carlo method with relatively stable results is derived by using a given numerical tolerance to obtain the required number of simulations.Finally, through the analysis of the results of the example, it can be concluded that the adaptive Monte Carlo method has wider applicability and potential application prospects.

Regional importance effect analysis of the input variables on failure probability

Structural systems often utilize scheduled maintenance to sustain structural reliability throughout the service life or even extend the original design life. However, inspection schedules are often arbitrary, e.g., use equal intervals, which tend to be on the conservative side for new structures and non-conservatively for aging structures. Although many reliability methods are useful for maintenance planning, there is a no generic methodology to conduct risk or reliability-based maintenance optimization (RBMO). The progress in RBMO technology development has been slow partly because there is no integrated methodology and because computationally efficient tools to study and resolve RBMO issues are lacking. This paper presents an integrated, efficient, and versatile RBMO methodology built on a two-stage random simulation framework and features three tightly-integrated efficient methods: (1) a new adaptive reliability-based adaptive meta-modeling (RAM) approach to establish an accurate and fast-running approximate damage tolerance model, (2) an adaptive stratified importance sampling (ASIS) method for computing probability-offailure without inspection and to generate random failure samples, and (3) the recursive probability integration (RPI) method for computing probability-offailure with inspections and repairs. The RBMO methodology has been implemented in a Fast Probability Analyzer software and is demonstrated using a fracture mechanics code with several random variables, a random POD, several repair qualities, and multiple inspections.

Coverage interval estimation of the measurement of Gross Heat of Combustion of fuel by bomb calorimetry: Comparison of ISO GUM and adaptive Monte Carlo method