Multifidelity Monte Carlo estimation for efficient uncertainty quantification in climate-related modeling

Uncertainties in an output of interest that depends on the solution of a complex system (e.g., of partial differential equations with random inputs) are often, if not nearly ubiquitously, determined in practice using Monte Carlo (MC) estimation. While simple to implement, MC estimation fails to provide reliable information about statistical quantities (such as the expected value of the output of interest) in application settings such as climate modeling for which obtaining a single realization of the output of interest is a costly endeavor. Specifically, the dilemma encountered is that many samples of the output of interest have to be collected in order to obtain an MC estimator having sufficient accuracy; so many, in fact, that the available computational budget is not large enough to effect the number of samples needed. To circumvent this dilemma, we consider using multifidelity Monte Carlo (MFMC) estimation which leverages the use of less costly and less accurate surrogate models (such as coarser grids, reduced-order models, simplified physics, interpolants, etc.) to achieve, for the same computational budget, higher accuracy compared to that obtained by an MC estimator or, looking at it another way, an MFMC estimator obtains the same accuracy as the MC estimator at lower computational cost. The key to the efficacy of MFMC estimation is the fact that most of the required computational budget is loaded onto the less costly surrogate models, so that very few samples are taken of the more expensive model of interest. We first provide a more detailed discussion about the need to consider an alternate to MC estimation for uncertainty quantification. Subsequently, we present a review, in an abstract setting, of the MFMC approach along with its application to three climate-related benchmark problems as a proof-of-concept exercise.

Automated importance sampling via optimal control for stochastic reaction networks: A Markovian projection–based approach

A novel unbiased estimator for estimating the probability mass of a multivariate exponential distribution over a measurable set is introduced and is called the exponential simplex (ES) estimator. For any measurable set and given sample size, the statistical efficiency of the ES estimator is higher than or equal to the statistical efficiency of the well-known Monte Carlo (MC) estimator. For non-radially shaped measurable sets, the ES estimator has a strictly higher statistical efficiency than the MC estimator. For ray-convex sets, such as convex sets, the ES estimator can be expressed in a simple analytical form.

Multilevel and multifidelity uncertainty quantification for cardiovascular hemodynamics

It is very time-consuming to obtain a high-precision Monte Carlo (MC) estimate of the fuel temperature reactivity coefficient (FTC) through direct subtraction of two reactivity values from MC calculations at two different fuel temperatures. As an alternative to the direct subtraction MC estimate of the FTC, this paper presents a new method based on the adjoint-weighted correlated sampling technique. The new method translates the change in fuel temperature as the corresponding changes in both the microscopic cross sections and the transfer probabilities in scattering kernels described by the free gas model. The effectiveness of the new method is examined through continuous-energy MC neutronics calculations for pressurized water reactor pin cell and CANDU pressurized heavy water reactor lattice problems. The isotope-wise and reaction-type–wise contributions to the FTCs in the two problems are examined for two free gas models: the constant-cross-section and the resonance-cross-section models. It is demonstrated that the new MC method can predict the reactivity change due to fuel temperature variation as accurately as the conventional, more time-consuming direct subtraction MC method.

The Monte Carlo (MC) method is most exact but computationally expensive for uncertainty quantification (UQ). Instead, surrogate models can approximate UQ at a lower computational cost. However, as the input uncertainty dimensionality increases, the number of samples required to construct the surrogate models also increases rapidly; thus, the advantage of using surrogate models disappears. Therefore, it is necessary to combine the surrogate models with dimensionality reduction for effective UQ. This study compares the performance of UQ for the direct MC method and four different combinations of surrogate models and dimensionality reduction methods in benchmark test functions. The results clarify the effectiveness of training a surrogate model and a dimensionality reduction technique simultaneously rather than separately.

In epidemiological and occupational health studies it is not uncommon to find that the distribution of exposure to a quantitative risk factor is highly skewed, as, for example, when most of a population suffers little or no exposure to a hazardous agent while a small subset receives very high exposures. As a result the asymptotic significance levels of conditional tests for monotone trends in rates or proportions can be profoundly anticonservative when applied to small or even moderate numbers of events. Monte Carlo (MC) estimation of observed levels of significance (”p-values”) provides a useful method for accurately assessing statistical significance in such situations. We describe a simple technique of importance sampling (IS) which can greatly improve the efficiency of MC estimation in this setting. Use of the IS technique is illustrated with data regarding cancer mortality among atomic bomb survivors.

Uncertainty quantification has proven to be an indispensable study for enhancing reliability and robustness of engineering systems in the early design phase. Single and multi-fidelity surrogate modelling methods have been used to replace the expensive high fidelity analyses which must be repeated many times for uncertainty quantification. However, since the number of analyses required to build an accurate surrogate model increases exponentially with the number of random input variables, most surrogate modelling methods suffer from the curse of dimensionality. As an alternative approach, the Low-Rank Approximation method can be applied to high-dimensional uncertainty quantification studies with a low computational cost, where the number of coefficients for building the surrogate model increases only linearly with the number of random input variables. In this study, the Low-Rank Approximation method is implemented for multi-fidelity applications with additive and multiplicative correction approaches to make the high-dimensional uncertainty quantification analysis more efficient and accurate. The developed uncertainty quantification methodology is tested on supersonic aircraft design problems and its predictions are compared with the results of single- and multi-fidelity Polynomial Chaos Expansion and Monte Carlo methods. For the same computational cost, the Low-Rank Approximation method outperformed both in surrogate modeling and uncertainty quantification cases for all the benchmarks and real-world engineering problems addressed in the present study.

Variability in multiple independent input parameters makes it difficult to estimate the resultant variability in a system's overall response. The propagation of errors (PE) and Monte Carlo (MC) techniques are two major methods to predict the variability of a system. However, the formalism of PE can lead to an inaccurate estimate for systems that have parameters varying over a wide range. For the latter, the results give a direct estimate of the variance of the response, but for complex systems with many parameters, the number of trials necessary to yield an accurate estimate can be sizeable to the point the technique becomes impractical. The effectiveness of a designed experiment (orthogonal array) methodology, as employed in Taguchi tolerance design (TD) method to estimate variability in complex systems is studied. We use a linear elastic three-point bending beam model and a nonlinear extended finite elements crack growth model to test and compare the PE and MC methods with the TD method. Our focus is on assessing their effectiveness under those specific circumstances. Results from an MC estimate, using 10,000 trials, serve as a referent to evaluate the result in both cases. We find that the PE method works suboptimal for a coefficient of variation above 5% in the input variables. In addition, we find that the TD method works very well with moderately sized trials of designed experiments for both models. Our results demonstrate how the variability estimation methods perform in the deterministic domain of numerical simulations and can assist in designing physical tests by providing a guideline performance measure.

Exceptional points can be found for specific sets of parameters in thermoacoustic systems. At an exceptional point, two eigenvalues and their corresponding eigenfunctions coalesce. Given that the sensitivity of these eigenvalues to parameter changes becomes infinite at the exceptional point, their occurrence may greatly affect the outcome and reliability of numerical stability analysis. We propose a new method to identify exceptional points in thermoacoustic systems. By iteratively updating the system parameters, two initially selected eigenvalues are shifted toward each other, ultimately colliding and generating the exceptional point. Using this algorithm, we were able to identify for the first time a physically meaningful exceptional point with positive growth rate in a thermoacoustic model. Furthermore, our analysis goes beyond previous studies inasmuch as we employ a more realistic flame transfer function to model flame dynamics. Building on these results, we analyze the effect of exceptional points on the reliability of thermoacoustic stability analysis. In the context of uncertainty quantification, we show that surrogate modeling is not reliable in the vicinity of an exceptional point, even when large sets of training samples are provided. The impact of exceptional points on the propagation of input uncertainties is demonstrated via Monte Carlo computations. The increased sensitivity associated with the exceptional point results in large variances for eigenvalue predictions, which needs to be taken into account for reliable stability analysis.

Many Monte Carlo computations involve computing quantities that can be expressed as g(EX), where g is nonlinear and smooth, and X is an easily simulatable random variable. The nonlinearity of g makes the conventional Monte Carlo estimator for such quantities biased. In this paper, we show how such quantities can be estimated without bias. However, our approach typically increases the variance. Thus, our approach is primarily of theoretical interest in the above setting. However, our method can also be applied to the computation of the inner expectation associated with Eg(EX|Z)), and in this setting, the application of this method can have a significant positive effect on improving the rate of convergence relative to conventional nested schemes for carrying out such calculations.

In this analytical study we compare the risk of the Monte Carlo (MC) and the least-squares TD (LSTD) estimator. We prove that for the case of acyclic Markov Reward Processes (MRPs) LSTD has minimal risk for any convex loss function in the class of unbiased estimators. When comparing the Monte Carlo estimator, which does not assume a Markov structure, and LSTD, we find that the Monte Carlo estimator is equivalent to LSTD if both estimators have the same amount of information. Theoretical results are supported by an empirical evaluation of the estimators.

Intelligent sensitivity analysis framework based on machine learning for spacecraft thermal design

In inverse problems like tomography reconstruction we need to solve an over-determined linear system corrupted with noise. The ML-EM algorithm finds the solution for Poisson noise as the fixed point of iterating a forward projection and a non-linear back projection. In tomography we have several hundred million equations and unknowns. The elements of the huge matrix are high-dimensional integrals, which cannot be stored, but must be re-computed with Monte Carlo (MC) quadrature when needed. In this paper we address the problems of how the quadrature error affects the accuracy of the reconstruction, whether it is possible to modify the back projection to speed up convergence without compromising the accuracy, and whether we should always take the same MC estimate or modify it in every projection.

Monte Carlo techniques are used to estimate the integrals of a function using randomly generated samples. The interest in uncertainty quantification and robust design makes calculating the expected values of such functions (e.g., performance measures) important. Recent developments in scramjets, aircraft technology forecasting, structural reliability, and robust low-boom aircraft designs use Monte Carlo techniques to ensure the appropriate quantification of uncertainties. Because of high variance and slow convergence, Monte Carlo techniques require a large number of function evaluations, limiting the fidelity of the tools that can be used to predict performance. Stacked Monte Carlo is presented, which is a new method for postprocessing an existing set of Monte Carlo samples to improve integral estimation. Stacked Monte Carlo is based on combining fitting functions with cross-validation and should reduce the variance of any type of Monte Carlo integral estimate (importance sampling, quasi-Monte Carlo, etc.) without adding bias. An extensive set of experiments is reported, confirming that the stacked Monte Carlo estimate is more accurate than both the unprocessed Monte Carlo estimate and the estimate from a functional fit. Stacked Monte Carlo is applied to estimate the fuel-burn metrics of future commercial aircraft and sonic boom loudness measures, and the efficiency of Monte Carlo is compared with that of more standard methods. It is shown that for negligible, additional, computational cost, significant increases in accuracy are gained.