Numerical aspects of application of FORM in node snapping truss structures

In this paper, we present a variance minimization (VM) procedure for rare event simulation in tandem queueing networks. We prove that the VM method can produce a zero variance. The VM method is suitable to compute optimal importance sampling (IS) parameters for small scale tandem networks. For large scale tandem networks we propose a sub-optimal IS (SOIS) method, which projects the optimal biased transition probabilities of the corresponding small scale system into those of a large scale system. In other words, we establish an efficient IS method for a large scale system by zooming into a small scale system and then projecting our findings into the large scale system. The numerical results show that our SOIS method can produce accurate results with very short CPU time, while many other methods often require much longer.

Summary Two new implementations of the EM algorithm are proposed for maximum likelihood fitting of generalized linear mixed models. Both methods use random (independent and identically distributed) sampling to construct Monte Carlo approximations at the E-step. One approach involves generating random samples from the exact conditional distribution of the random effects (given the data) by rejection sampling, using the marginal distribution as a candidate. The second method uses a multivariate t importance sampling approximation. In many applications the two methods are complementary. Rejection sampling is more efficient when sample sizes are small, whereas importance sampling is better with larger sample sizes. Monte Carlo approximation using random samples allows the Monte Carlo error at each iteration to be assessed by using standard central limit theory combined with Taylor series methods. Specifically, we construct a sandwich variance estimate for the maximizer at each approximate E-step. This suggests a rule for automatically increasing the Monte Carlo sample size after iterations in which the true EM step is swamped by Monte Carlo error. In contrast, techniques for assessing Monte Carlo error have not been developed for use with alternative implementations of Monte Carlo EM algorithms utilizing Markov chain Monte Carlo E-step approximations. Three different data sets, including the infamous salamander data of McCullagh and Nelder, are used to illustrate the techniques and to compare them with the alternatives. The results show that the methods proposed can be considerably more efficient than those based on Markov chain Monte Carlo algorithms. However, the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension.

espanolExact tests, fuzzy p-values, Gibbs sampling, iterval censoring, linear extensions, linear rank statistics, perfect MCMC, proportional hazard model, topological sorting EnglishWe discuss distribution-free exact rank tests from partially ordered data that arise in various biological and other applications where the primary objective is to conduct testing of significance to assess the linear dependence or to compare different groups. The tests here are obtained by treating the usual rank statistics, based on the completely ordered data as “latent” or missing, and conceptualizing the “latent” p-value as the random probability under the null hypothesis of a test statistic that is as extreme, or more extreme, than the latent test statistics based on the completely ordered data. The latent p-value is then predicted by sampling linear extensions or the complete orderings that are consistent with the observed partially ordered data. The sampling methods explored here include importance sampling methods based on randomized topological sorting algorithms, Gibbs sampling methods, random-walk based Metropolis–Hasting sampling methods and random-walk based modern perfect Markov chain Monte Carlo sampling methods. We discuss running times of these sampling methods and their strength and weaknesses. A simulation experiment and three data examples are given. The simulation experiment illustrates how the exact rank tests from partially ordered data work when the desired result is known. The first data example concerns the light preference behavior of fruit flies and tests whether heterogeneity observed in average light-preference behavior can be explained by manipulations in serotonin signaling. The second one is a reanalysis of the lead absorption data in children of employees who worked in a lead battery factory and consolidates the results reported in Rosenbaum [Ann. Statist. 19 (1991) 1091–1097]. The third one reexamines the breast cosmesis data from Finkelstein [Biometrics 42 (1986) 845–854].

Massively repeated structures such as SRAM cells usually require extremely low failure rate. This brings on a challenging issue for Monte Carlo based statistical yield analysis, as huge amount of samples have to be drawn in order to observe one single failure. Fast Monte Carlo methods, e.g. importance sampling methods, are still quite expensive as the anticipated failure rate is very low. In this paper, a new method is proposed to tackle this issue. The key idea is to improve traditional importance sampling method with an efficient online surrogate model. The proposed method improves the performance for both stages in importance sampling, i.e. finding the distorted probability density function, and the distorted sampling. Experimental results show that the proposed method is 1e2X∼1e5X faster than the standard Monte Carlo approach and achieves 5X∼22X speedup over existing state-of-the-art techniques without sacrificing estimation accuracy.

Stochastic gradient descent (SGD) usually samples training data based on the uniform distribution, which may not be a good choice because of the high variance of its stochastic gradient. Thus, importance sampling methods are considered in the literature to improve the performance. Most previous work on SGD-based methods with importance sampling requires the knowledge of Lipschitz constants of all component gradients, which are in general difficult to estimate. In this paper, we study an adaptive importance sampling method for common SGD-based methods by exploiting the local first-order information without knowing any Lipschitz constants. In particular, we periodically changes the sampling distribution by only utilizing the gradient norms in the past few iterations. We prove that our adaptive importance sampling non-asymptotically reduces the variance of the stochastic gradients in SGD, and thus better convergence bounds than that for vanilla SGD can be obtained. We extend this sampling method to several other widely used stochastic gradient algorithms including SGD with momentum and ADAM. Experiments on common convex learning problems and deep neural networks illustrate notably enhanced performance using the adaptive sampling strategy.

Automated importance sampling via optimal control for stochastic reaction networks: A Markovian projection–based approach

The importance sampling method is an extensively used numerical simulation method in reliability analysis. In this paper, a modification to the importance sampling method (ISM) is proposed, and the modified ISM divides the sample set of input variables into different subsets based on the contributive weight of the importance sample defined in this paper and the maximum super-sphere denoted by β-sphere in the safe domain defined by the truncated ISM. By this proposed modification, only samples with large contributive weight and locating outside of the β-sphere need to call the limit state function. This amelioration remarkably reduces the number of limit state function evaluations required in the simulation procedure, and it doesn’t sacrifice the precision of the results by controlling the level of relative error. Based on this modified ISM and the space-partition idea in variance-based sensitivity analysis, the global reliability sensitivity indices can be estimated as byproducts, which is especially useful for reliability-based design optimization. This process of estimating the global reliability sensitivity indices only needs the sample points used in reliability analysis and is independent of the dimensionality of input variables. A roof truss structure and a composite cantilever beam structure are analyzed by the modified ISM. The results demonstrate the efficiency, accuracy, and robustness of the proposed method.

To reduce the substantial leakage current, the supply voltage of SRAM cells has being scaled down towards its lower limit, which is called the data Retention Voltage (DRV). Although the power consumption is largely reduced, this down-scaling trend, however, impacts the stability of the SRAM cell due to the unpredictable process or device parameter variations. In this work, we propose a novel method to evaluate the DRV of SRAM cells at the presence of variations. The DRV issue is first formulated as a time domain worst performance bound problem. To accurately and efficiently evaluate the DRV, a multi-start point (MSP) optimization strategy is then studied and developed with the use of practical circuit simulator. One feature of the proposed method is that it can efficiently evaluate the DRV without suffering from any process/model accuracy. Experiment results show that it achieves a speedup of 3 and 5-7 order over the Importance Sampling (IS) and Monte Carlo (MC) method respectively under the context of the DRV evaluation in this paper. The proposed method can serve as an efficient DRV evaluation tool on any specific technology process or in-house circuit simulator. In this work, the DRVs at the technology node from 130 nm to 45 nm under the influence of different variation sources are also presented and analyzed.

Vehicle routing problem belongs to classical Combination Optimization hard problem and has been extensively studied by many researchers. Vehicle Routing Problem with Weight Coefficients and Stochastic Demands (WVRPSD) is established and a new implementation for importance sampling method to solve this model is proposed in this paper, in which the classical exponential change of measure is adopted to construct the family of importance sampling distributions, and the optimal importance sampling distribution is obtained by minimizing the variance of importance sampling estimator. Numerical experiments have been conducted and the results indicate that the method can effectively solve this problem.

An importance sampling (IS) simulation method is developed for evaluating the bit-error rate of one-shot and recursive Bayesian equalizers (symbol-by-symbol detectors.) It is shown that the one-shot Bayesian equalizer structure closely resembles that of a multi-user detector for CDMA. Hence, noise biasing IS methods developed previously for CDMA detectors should also be appropriate for Bayesian equalizers. At low SNRs, the decision boundary far the one-shot Bayesian equalizer approaches a hyperplane. This hyperplane is viewed as an approximation to the true decision boundary at high SNRs, and can thus be used to construct an appropriate noise bias vector. With a slight modification, it is shown that the IS technique for the one-shot Bayesian equalizer can also be applied to the Abend and Fritchman (1970) recursive algorithm, and finally, to an adaptive Bayesian blind equalizer. IS simulations are carried out for both the nonadaptive Bayesian equalizers and for the blind recursive equalizer. >

An importance sampling method is presented for computing the first passage probability of elasto-plastic structures under stochastic excitations. The importance sampling distribution corresponds to shifting the mean of the excitation to an \'adapted\' stochastic process whose future is determined based on information only up to the present. A stochastic control approach is adopted for designing the adapted process. The optimal control law is determined by a control potential, which satisfies the Bellman\'s equation, a nonlinear partial differential equation on the response state-space. Numerical results for a single-degree-of freedom elasto-plastic structure shows that the proposed method leads to significant improvement in variance reduction over importance sampling using design points reported recently.

An efficient time-dependent (TD) Monte Carlo (MC) importance sampling method has recently been developed [G. Tao and W. H. Miller, J. Chem. Phys. 135, 024104 (2011)] for the evaluation of time correlation functions using the semiclassical (SC) initial value representation (IVR) methodology. In this TD-SC-IVR method, the MC sampling uses information from both time-evolved phase points as well as their initial values, and only the "important" trajectories are sampled frequently. Even though the TD-SC-IVR was shown in some benchmark examples to be much more efficient than the traditional time-independent sampling method (which uses only initial conditions), the calculation of the SC prefactor-which is computationally expensive, especially for large systems-is still required for accepted trajectories. In the present work, we present an approximate implementation of the TD-SC-IVR method that is completely prefactor-free; it gives the time correlation function as a classical-like magnitude function multiplied by a phase function. Application of this approach to flux-flux correlation functions (which yield reaction rate constants) for the benchmark H + H(2) system shows very good agreement with exact quantum results. Limitations of the approximate approach are also discussed.

Importance sampling (IS) is a powerful Monte Carlo (MC) methodology for approximating integrals, for instance in the context of Bayesian inference. In IS, the samples are simulated from the so-called proposal distribution, and the choice of this proposal is key for achieving a high performance. In adaptive IS (AIS) methods, a set of proposals is iteratively improved. AIS is a relevant and timely methodology although many limitations remain yet to be overcome, e.g., the curse of dimensionality in high-dimensional and multi-modal problems. Moreover, the Hamiltonian Monte Carlo (HMC) algorithm has become increasingly popular in machine learning and statistics. HMC has several appealing features such as its exploratory behavior, especially in high-dimensional targets, when other methods suffer. In this letter, we introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. The proposed HAIS efficiently adapts a population of proposals, extracting the advantages of HMC. HAIS can be understood as a particular instance of the generic layered AIS family with an additional resampling step. HAIS achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms. We discuss the statistical properties of HAIS and show its high performance in two challenging examples.

In this paper we propose a new importance sampling (IS) scheme to estimate the low error-rate performance of LDPC codes over a Rayleigh fading channel. It biases the Rayleigh fading distribution as well as the noise distribution in order to improve the quality of estimation. The proposed IS scheme is compared with the Monte-Carlo (MC) simulator and other IS schemes in terms of its efficiency and accuracy. Numerical results show that the proposed IS estimator is much more efficient than the MC estimator and other IS estimators from the viewpoint of the required simulation time. Furthermore, the proposed IS scheme provides a more accurate performance estimation of LDPC codes over a Rayleigh fading channel with only a limited simulation time than other IS schemes.

Value at Risk (VaR) is an important tool for estimating the risk of a financial portfolio under significant loss. Although Monte Carlo simulation is a powerful tool for estimating VaR, it is quite inefficient since the event of significant loss is usually rare. Previous studies suggest that the performance of the Monte Carlo simulation can be improved by impor-tance sampling if the market returns follow the normality or the distributions. The first contribution of our paper is to extend the importance sampling method for dealing with jump-diffusion market returns, which can more precisely model the phenomenon of high peaks, heavy tails, and jumps of market returns mentioned in numerous empirical study papers. This paper also points out that for portfolios of which the huge loss is triggered by significantly distinct events, naively applying importance sampling method can result in poor performance. The second contribution of our paper is to develop the hybrid importance sampling method for the aforementioned problem. Our method decomposes a Monte Carlo simulation into sub simulations, and each sub simulation focuses only on one huge loss event. Thus the perform-ance for each sub simulation is improved by importance sampling method, and overall performance is optimized by determining the allotment of samples to each sub simulation by Lagrange’s multiplier. Numerical experiments are given to verify the superiority of our method.