Support Vector Machine-Assisted Importance Sampling for Optimal Reliability Design

Command-based importance sampling for statistical model checking

It is well known that in estimating performance measures associated with a stochastic system a good importance sampling distribution (IS) can give orders of magnitude of variance reduction while a bad one may lead to large, even infinite, variance. In this paper we study how this sensitivity of the estimator variance to the importance sampling change of measure may be "dampened" by combining importance sampling with stochastic approximation based temporal difference (TD) method. We consider a finite state space discrete time Markov chain (DTMC) with one-step transition rewards and an absorbing set of states and focus on estimating the cumulative expected reward to absorption starting from any state. In this setting we develop sufficient conditions under which the estimate resulting from the combined approach has a mean square error that asymptotically equals zero even when the estimate formed by using only importance sampling change of measure has infinite variance. In particular, we consider the problem of estimating the small buffer overflow probability in a queuing network, where the change of measure suggested in literature is shown to have infinite variance under certain parameters and where the appropriate combination of IS and TD method can be empirically seen to have a much faster convergence rate compared to naive simulation.

Simulation-based validation methods exhibit properties which should allow them to contribute to the release of highly automated driving functions (HADF). Monte-Carlo (MC) simulations make it possible to generate statistical statements about a HADF. However, the fact that accidents happen very rarely in the real world can strongly enlarge the number of MC simulation runs which are needed to obtain statistically stable results. In the past, importance sampling (IS) was used to reduce these simulation costs. This paper considers some of the still existent limitations of IS: The definition of proper safety metrics needed to find good IS distributions and the application of IS to high dimensional parameter spaces. The approach in this paper was able to reduce an IS optimization of 2700 dimensions to 17 optimizations of 9 dimensions. The number of MC runs could be reduced by a factor of 350.

Vehicle routing problem belongs to classical Combination Optimization hard problem and has been extensively studied by many researchers. Vehicle Routing Problem with Weight Coefficients and Stochastic Demands (WVRPSD) is established and a new implementation for importance sampling method to solve this model is proposed in this paper, in which the classical exponential change of measure is adopted to construct the family of importance sampling distributions, and the optimal importance sampling distribution is obtained by minimizing the variance of importance sampling estimator. Numerical experiments have been conducted and the results indicate that the method can effectively solve this problem.

We review two types of adaptive Monte Carlo methods for rare event simulations. These methods are based on importance sampling. The first approach selects importance sampling distributions by minimizing the variance of importance sampling estimator. The second approach selects importance sampling distributions by minimizing the cross entropy to the optimal importance sampling distribution. We also review the basic concepts of importance sampling in the rare event simulation context. To make the basic concepts concrete, we introduce these ideas via the study of rare events of M/M/1 queues.

Traditional reliability-based design optimization(RBDO)is either computational intensive or not accurate enough.In this work,a new RBDO method based on Support Vector Machines(SVM)is proposed.For reliability analysis,SVM is used to create a surrogate model of the limit-state function at the Most Probable Point(MPP).The uniqueness of the new method is the use of the gradient of the limit-state function at the MPP.This guarantees that the surrogate model not only passes through the MPP but also is tangent to the limit-state function at the MPP.Then Importance Sampling(IS)is used to calculate the probability of failure based on the surrogate model.This treatment significantly improves the accuracy of reliability analysis.For optimization,the Sequential Optimization and Reliability Assessment(SORA)is employed,which decouples deterministic optimization from the SVM reliability analysis.The decoupling makes RBDO more efficient.The two examples show that the new method is more accurate with a moderately increased computational cost.

This paper introduces a new surrogate-based optimization algorithm to optimize a deterministic objective function with non-computable constraint functions (a.k.a. hidden constraints). Both the objective function and the feasible domain are defined within a known rectangular domain. The objective function might be nonconvex, computationally expensive, and without analytic expression. Moreover, the feasible domain boundaries are not explicitly defined, but can be determined via oracle calls (feasible or not) and learned as the algorithm proceeds. To solve this class of optimization problems, the proposed algorithm, in each iteration, approximates the feasible domain boundary by incorporating a Support Vector Machine (SVM) classifier model as an approximation for the non-computable constraint function, which characterizes the feasible domain. The uncertainty associated with this surrogate is modeled using an artificially-generated uncertainty function built on the framework of Delaunay triangulation. This work extends the Delaunay-based optimization algorithm with nonconvex constraints, dubbed \(\Delta \)-DOGS(\(\Omega \)), and extends this approach to estimate the feasible domain with binary oracle calls. Similarly, this algorithm at each iteration determines a minimizer of the objective function surrogate model with the highest probability of being feasible. We evaluate the performance of the algorithm through the numerical experiments on a representative test problem.

Bayesian support vector machine for optimal reliability design of modular systems

We present a novel method, called the transform likelihood ratio (TLR) method, for estimation of rare event probabilities with heavy-tailed distributions. Via a simple transformation (change of variables) technique the TLR method reduces the original rare event probability estimation with heavy tail distributions to an equivalent one with light tail distributions. Once this transformation has been established we estimate the rare event probability via importance sampling, using the classical exponential change of measure or the standard likelihood ratio change of measure. In the latter case the importance sampling distribution is chosen from the same parametric family as the transformed distribution. We estimate the optimal parameter vector of the importance sampling distribution using the cross-entropy method. We prove the polynomial complexity of the TLR method for certain heavy-tailed models and demonstrate numerically its high efficiency for various heavy-tailed models previously thought to be intractable. We also show that the TLR method can be viewed as a universal tool in the sense that not only it provides a unified view for heavy-tailed simulation but also can be efficiently used in simulation with light-tailed distributions. We present extensive simulation results which support the efficiency of the TLR method.

Computation of failure probability via hierarchical clustering

The paper is dealing with estimation of rare event probabilities in stochastic networks. The well known variance reduction technique, called Importance Sampling (IS) is an effective tool for doing this. The main idea of IS is to simulate the random system under a modified set of parameters, so as to make the occurrence of the rare event more likely. The major problem of the IS technique is that the optimal modified parameters, called reference parameters to be used in IS are usually very difficult to obtain. Rubinstein (Eur J Oper Res 99:89–112, 1997) developed the Cross Entropy (CE) method for the solution of this problem of IS technique and then he and his collaborators applied this for estimation of rare event probabilities in stochastic networks with exponential distribution [see De Boer et al. (Ann Oper Res 134:19–67, 2005)]. In this paper, we test this simulation technique also for medium sized stochastic networks and compare its effectiveness to the simple crude Monte Carlo (CMC) simulation. The effectiveness of a variance reduction simulation algorithm is measured in the following way. We calculate the product of the necessary CPU time and the estimated variance of the estimation. This product is compared to the same for the simple Crude Monte Carlo simulation. This was originally used for comparison of different variance reduction techniques by Hammersley and Handscomb (Monte Carlo Methods. Methuen & Co Ltd, London, 1967). The main result of the paper is the extension of CE method for estimation of rare event probabilities in stochastic networks with beta distributions. In this case the calculation of reference parameters of the importance sampling distribution requires numerical solution of a nonlinear equation system. This is done by applying a Newton–Raphson iteration scheme. In this case the CPU time spent for calculation of the reference parameter values cannot be neglected. Numerical results will also be presented.

Battery thermal management has significant effect on the performance of electric vehicles (EVs) under high current rates. In this research, a comprehensive thermal analysis and multi-objective optimization design framework is proposed to enhance the thermal performance of a novel air–liquid cooling coupled battery pack under higher discharging rate (3C). Computational fluid dynamics (CFD) numerical calculation is utilized to compare the cooling efficiency of the battery pack designs. Furthermore, a surrogate model is generated by using Latin hypercube sampling (LHS) and support vector machine. The design parameters include different mini-channels’ mass flowrates and the air flow inlet velocity, the objectives are the temperature rise, temperature distribution, and the energy consumption. Sensitivity analysis results indicate that the air flow inlet velocity is the main factor affecting the temperature rise and temperature distribution, while the mass flowrates of mini-channels have important influence on the pressure drop. Finally, the nondominated sorting genetic algorithm-II (NSGA-II) is used to select the optimal battery pack design, the maximum temperature, and temperature standard deviation (TSD) get improved by 1.8 K and 0.06 K, respectively. And the energy consumption of the cooling system can be controlled within the appropriate range after optimization design.

Support vector machine (SVM) active learning is one popular and successful technique for relevance feedback in content-based image retrieval (CBIR). Despite the success, conventional SVM active learning has two main drawbacks. First, the performance of SVM is usually limited by the number of labeled examples. It often suffers a poor performance for the small-sized labeled examples, which is the case in relevance feedback. Second, conventional approaches do not take into account the redundancy among examples, and could select multiple examples that are similar (or even identical). In this work, we propose a novel scheme for explicitly addressing the drawbacks. It first learns a kernel function from a mixture of labeled and unlabeled data, and therefore alleviates the problem of small-sized training data. The kernel will then be used for a batch mode active learning method to identify the most informative and diverse examples via a min-max framework. Two novel algorithms are proposed to solve the related combinatorial optimization: the first approach approximates the problem into a quadratic program, and the second solves the combinatorial optimization approximately by a greedy algorithm that exploits the merits of submodular functions. Extensive experiments with image retrieval using both natural photo images and medical images show that the proposed algorithms are significantly more effective than the state-of-the-art approaches. A demo is available at http://msm.cais.ntu.edu.sg/LSCBIR/.

The objective of this paper is two fold: The first is to give a rigorous relaxation for the problem of minimum compliance design for reinforced Kirchoff plates subject to random loads. This relaxation is new and is the extension to the two dimensional setting of the relaxation given by Cheng and Olhoff (see [1,2]) for one dimensional plate problems; when the plate thickness is allowed to take two values. The second is to illustrate reliability based optimal design for stiffener reinforced Kirchoff plates as well as other plate models. An illustrative numerical example is carried out for optimal reliability design for the laminated plate model of Suzuki and Kikuchi [3]. The numerical method used was developed and implemented by A. Diaz and C. Soto [14] and is an adaptation of the multi—load numerical method given in [13].

According to the spring’s failures in the tools storage and mechnical tools locking, the thesis will discuss the reliability optimization design for spring. With the known probability characteristics of basic random parameters, the random perturbation method, the reliability theory and optimal design technology are used to give the spring reliability design problems in locking knife spring in the tools storage and mechanical tools of machining center, to put forward the practical and effective reliability-based optimization design method, which can shorten the design cycle, save the testing cost, improve the design level and the reliability of parts. DOI: http://dx.doi.org/10.11591/telkomnika.v10i5.1276