Refined Stratified Sampling for efficient Monte Carlo based uncertainty quantification

In earth and environmental sciences applications, uncertainty analysis regarding the outputs of models whose parameters are spatially varying (or spatially distributed) is often performed in a Monte Carlo framework. In this context, alternative realizations of the spatial distribution of model inputs, typically conditioned to reproduce attribute values at locations where measurements are obtained, are generated via geostatistical simulation using simple random (SR) sampling. The environmental model under consideration is then evaluated using each of these realizations as a plausible input, in order to construct a distribution of plausible model outputs for uncertainty analysis purposes. In hydrogeological investigations, for example, conditional simulations of saturated hydraulic conductivity are used as input to physically-based simulators of flow and transport to evaluate the associated uncertainty in the spatial distribution of solute concentration. Realistic uncertainty analysis via SR sampling, however, requires a large number of simulated attribute realizations for the model inputs in order to yield a representative distribution of model outputs; this often hinders the application of uncertainty analysis due to the computational expense of evaluating complex environmental models. Stratified sampling methods, including variants of Latin hypercube sampling, constitute more efficient sampling aternatives, often resulting in a more representative distribution of model outputs (e.g., solute concentration) with fewer model input realizations (e.g., hydraulic conductivity), thus reducing the computational cost of uncertainty analysis. The application of stratified and Latin hypercube sampling in a geostatistical simulation context, however, is not widespread, and, apart from a few exceptions, has been limited to the unconditional simulation case. This paper proposes methodological modifications for adopting existing methods for stratified sampling (including Latin hypercube sampling), employed to date in an unconditional geostatistical simulation context, for the purpose of efficient conditional simulation of Gaussian random fields. The proposed conditional simulation methods are compared to traditional geostatistical simulation, based on SR sampling, in the context of a hydrogeological flow and transport model via a synthetic case study. The results indicate that stratified sampling methods (including Latin hypercube sampling) are more efficient than SR, overall reproducing to a similar extent statistics of the conductivity (and subsequently concentration) fields, yet with smaller sampling variability. These findings suggest that the proposed efficient conditional sampling methods could contribute to the wider application of uncertainty analysis in spatially distributed environmental models using geostatistical simulation.

The RAVEN code has been under development at the Idaho National Laboratory since 2012. Its main goal is to create a multi-purpose platform for the deploying of all the capabilities needed for Probabilistic Risk Assessment, uncertainty quantification, data mining analysis and optimization studies. RAVEN is currently equipped with three different sampling categories: Forward samplers (Monte Carlo, Latin Hyper Cube, Stratified, Grid Sampler, Factorials, etc.), Adaptive Samplers (Limit Surface search, Adaptive Polynomial Chaos, etc.) and Dynamic Event Tree (DET) samplers (Deterministic and Adaptive Dynamic Event Trees). The main subject of this document is to report the activities that have been done in order to: start the migration of the RAVEN/RELAP-7 control logic system into MOOSE, and develop advanced dynamic sampling capabilities based on the Dynamic Event Tree approach. In order to provide to all MOOSE-based applications a control logic capability, in this Fiscal Year an initial migration activity has been initiated, moving the control logic system, designed for RELAP-7 by the RAVEN team, into the MOOSE framework. In this document, a brief explanation of what has been done is going to be reported. The second and most important subject of this report is about the development of a Dynamic Event Tree (DET) sampler named “Hybrid Dynamic Event Tree” (HDET) and its Adaptive variant “Adaptive Hybrid Dynamic Event Tree” (AHDET). As other authors have already reported, among the different types of uncertainties, it is possible to discern two principle types: aleatory and epistemic uncertainties. The classical Dynamic Event Tree is in charge of treating the first class (aleatory) uncertainties; the dependence of the probabilistic risk assessment and analysis on the epistemic uncertainties are treated by an initial Monte Carlo sampling (MCDET). From each Monte Carlo sample, a DET analysis is run (in total, N trees). The Monte Carlo employs a pre-sampling of the input space characterized by epistemic uncertainties. The consequent Dynamic Event Tree performs the exploration of the aleatory space. In the RAVEN code, a more general approach has been developed, not limiting the exploration of the epistemic space through a Monte Carlo method but using all the forward sampling strategies RAVEN currently employs. The user can combine a Latin Hyper Cube, Grid, Stratified and Monte Carlo sampling in order to explore the epistemic space, without any limitation. From this pre-sampling, the Dynamic Event Tree sampler starts its aleatory space exploration. As reported by the authors, the Dynamic Event Tree is a good fit to develop a goal-oriented sampling strategy. The DET is used to drive a Limit Surface search. The methodology that has been developed by the authors last year, performs a Limit Surface search in the aleatory space only. This report documents how this approach has been extended in order to consider the epistemic space interacting with the Hybrid Dynamic Event Tree methodology.

The generalization of Latin hypercube sampling

<p>Reliability of pipe structure is one aspect to be considered in reactor safety analysis. MSC NASTRAN is a computer code that can be used to calculate pipe deflection for reliability evaluation. MSC PATRAN can be used to generate input for this code. Uncertainty evaluation needs to be done in the input variable to understand uncertainty range in the analysis results. A computer code for evaluating structure reliability has been developed in our previous study. The code has implemented latin hypercube sampling (LHS) to assess uncertainty in the input variable, such as load and modulus of elasticity. In this study, comparison of two uncertainty methods, i.e. simple random sampling (SRS) and LHS, was carried out for the developed software. The comparison was subjected to pipe deflection calculation using 100 samples. Comparison analysis shows that LHS method produces a robust mean of variance for all sample size. The results also confirm that variance of pipe deflection using LHS is smaller by 3% than SRS one. It can be concluded that LHS is appropriate to be implemented for uncertainty analysis in the developed code.</p><p><strong> </strong></p>

We provide a thorough analysis of the effectiveness of different Variance Reduction Techniques (VRTs). We consider both stand-alone and combined applications of two input techniques, Antithetic Variates (AV) and Latin Hypercube Sampling (LHS), and two output techniques, Control Variates (CV) and Poststratified Sampling (PS). Previous research in the area mainly focuses on asymptotic variance reduction. In this experimental study, we measure the performance of VRTs under finite simulation run lengths and analyze their effects. Our findings show that the asymptotic variance reduction results do not readily apply to finite-length simulations. We consider three different types of systems (M/M/1, serial production line and (s, S) inventory control systems) and compare the VRTs under various experimental conditions. We observe that a variance reduction cannot be guaranteed for every instance a VRT is applied. Our results also indicate that the output VRTs (CV, PS) are better than input VRTs (AV, LHS) on the average for the single systems considered in this study. More interestingly, the less-sophisticated techniques (AV, CV) often perform better than the relatively more-complex techniques (LHS, PS). A comprehensive bibliography is also provided.

This chapter develops methods for increasing the efficiency of Monte Carlo simulation by reducing the variance of simulation estimates. These methods draw on two broad strategies for reducing variance: taking advantage of tractable features of a model to adjust or correct simulation outputs, and reducing the variability in simulation inputs. We discuss control variates, antithetic variates, stratified sampling, Latin hypercube sampling, moment matching methods, and importance sampling, and we illustrate these methods through examples. Two themes run through this chapter: The greatest gains in efficiency from variance reduction techniques result from exploiting specific features of a problem, rather than from generic applications of generic methods. Reducing simulation error is often at odds with convenient estimation of the simulation error itself; in order to supplement a reduced-variance estimator with a valid confidence interval, we sometimes need to sacrifice some of the potential variance reduction. KeywordsImportance SamplingVariance ReductionLatin Hypercube SamplingGeometric Brownian MotionUnderlying AssetThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Adapting a global stratified random sample for regional estimation of forest cover change derived from satellite imagery

This paper examines the application of Latin Hypercube Sampling (LHS) and Antithetic Variables (AVs) to reduce the variance of estimated performance measures from microscopic traffic simulators. LHS and AV allow for a more representative coverage of input probability distributions through stratification, reducing the standard error of simulation outputs. Two methods of implementation are examined, one where stratification is applied to headways and routing decisions of individual vehicles and another where vehicle counts and entry times are more evenly sampled. The proposed methods have wider applicability in general queuing systems. LHS is found to outperform AV, and reductions of up to 71% in the standard error of estimates of traffic network performance relative to independent sampling are obtained. LHS allows for a reduction in the execution time of computationally expensive microscopic traffic simulators as fewer simulations are required to achieve a fixed level of precision with reductions of up to 84% in computing time noted on the test cases considered. The benefits of LHS are amplified for more congested networks and as the required level of precision increases.

This report was centred on quantifying uncertainties associated with edge detection in image processing. The Canny edge-detection operator was used in this work due to its commonality within the image processing field. The proposed method employs a Monte Carlo approach to propagate uncertainties through the measurement model and two sampling methods were examined: simple random sampling and Latin hypercube sampling. The uncertainty-aware Canny operator method was successfully applied to several case-study images in which an uncertainty was quantified for the measurand – in this case the measurand was the area of and extracted region within the input image.

Fire safety design based on calculations: Uncertainty analysis and safety verification

This report presents guidelines for evaluating uncertainty in mathematical equations and computer models applied to assess human health and environmental risk. Uncertainty analyses involve the propagation of uncertainty in model parameters and model structure to obtain confidence statements for the estimate of risk and identify the model components of dominant importance. Uncertainty analyses are required when there is no a priori knowledge about uncertainty in the risk estimate and when there is a chance that the failure to assess uncertainty may affect the selection of wrong options for risk reduction. Uncertainty analyses are effective when they are conducted in an iterative mode. When the uncertainty in the risk estimate is intolerable for decision-making, additional data are acquired for the dominant model components that contribute most to uncertainty. This process is repeated until the level of residual uncertainty can be tolerated. A analytical and numerical methods for error propagation are presented along with methods for identifying the most important contributors to uncertainty. Monte Carlo simulation with either Simple Random Sampling (SRS) or Latin Hypercube Sampling (LHS) is proposed as the most robust method for propagating uncertainty through either simple or complex models. A distinction is made between simulating a stochastically varying assessment endpoint (i.e., the distribution of individual risks in an exposed population) and quantifying uncertainty due to lack of knowledge about a fixed but unknown quantity (e.g., a specific individual, the maximally exposed individual, or the mean, median, or 95%-tile of the distribution of exposed individuals). Emphasis is placed on the need for subjective judgement to quantify uncertainty when relevant data are absent or incomplete.

A framework for assessing convergence and validation of non-intrusive uncertainty quantification (UQ) methods is studied and applied to a complex industrial problem in ship design, namely the high-speed Delft Catamaran advancing in calm water, with variable Froude number and geometry. Relationship between UQ studies and deterministic verification and validation is discussed. Computations are performed using high- (URANS) and low- (potential flow) fidelity simulations. Froude number has expected value and standard deviation equal to 0.5 and 0.05, respectively, on a truncated normal distribution. Geometric uncertainty is related to the research space of a simulation-based design optimization, and assessed through the Karhunen–Loeve expansion (KLE). Monte Carlo method with Latin hypercube sampling (MC-LHS) is used to compute expected value, standard deviation, distribution and uncertainty intervals for resistance, sinkage and trim. MC-LHS with CFD is used as a benchmark for validating less costly UQ methods, including MC-LHS with metamodels and standard quadrature formulas. Gaussian quadrature is found the most efficient method; however, MC-LHS with metamodels is preferred since provides with confidence intervals and distributions in a straightforward way and at reasonably small computational cost. UQ results are compared to earlier deterministic single- and multi-objective optimization; reduced-dimensional KLE studies for geometric variability indicate that stochastic optimization would not be of great benefit for the present problem.

The accuracy of outdoor sound propagation predictions is often limited by imperfect knowledge of the atmospheric and ground properties, and random environmental variations such as turbulence. This article describes the impact of such uncertainties, and how they can be efficiently addressed and quantified with stochastic sampling techniques such as Monte Carlo and Latin hypercube sampling (LHS). Extensions to these techniques, such as importance sampling based on simpler, more efficient propagation models, and adaptive importance sampling, are described. A relatively simple example problem involving the Lloyd's mirror effect for an elevated sound source in a homogeneous atmosphere is considered first, followed by a more complicated example involving near-ground sound propagation with refraction and scattering by turbulence. When uncertainties in the atmospheric and ground properties dominate, LHS with importance sampling is found to converge to an accurate estimate with the fewest samples. When random turbulent scattering dominates, the sampling method has little impact. A comprehensive computational approach is demonstrated that is both efficient and accurate, while simultaneously incorporating broadband sources, turbulent scattering, and uncertainty in the environmental properties.

Much uncertainty is derived from the application of conceptual rainfall runoff models. In this paper, HYSIM, an ‘off‐the‐shelf’ conceptual rainfall runoff model, is applied to a suite of catchments throughout Ireland in preparation for use in climate impact assessment. Parameter uncertainty is assessed using the GLUE methodology. Given the lack of source code available for the model, parameter sampling is carried out using Latin hypercube sampling. Uncertainty bounds are constructed for model output. These bounds will be used to quantify uncertainty in future simulations as they include error derived from data measurement, model structure and parameterization.

History matching is a critical step within the reservoir management process to synchronize the simulation model with the production data. The history-matched model can be used for planning optimum field development and performing optimization and uncertainty quantifications. We present a novel history matching workflow based on a Bayesian framework that accommodates subsurface uncertainties. Our workflow involves three different model resolutions within the Bayesian framework: 1) a coarse low-fidelity model to update the prior range, 2) a fine low-fidelity model to represent the high-fidelity model, and 3) a high-fidelity model to re-construct the real response. The low-fidelity model is constructed by a multivariate polynomial function, while the high-fidelity model is based on the reservoir simulation model. We firstly develop a coarse low-fidelity model using a two-level Design of Experiment (DoE), which aims to provide a better prior. We secondly use Latin Hypercube Sampling (LHS) to construct the fine low-fidelity model to be deployed in the Bayesian runs, where we use the Metropolis-Hastings algorithm. Finally, the posterior is fed into the high-fidelity model to evaluate the matching quality. This work demonstrates the importance of including uncertainties in history matching. Bayesian provides a robust framework to allow uncertainty quantification within the reservoir history matching. Under uniform prior, the convergence of the Bayesian is very sensitive to the parameter ranges. When the solution is far from the mean of the parameter ranges, the Bayesian introduces bios and deviates from the observed data. Our results show that updating the prior from the coarse low-fidelity model accelerates the Bayesian convergence and improves the matching convergence. Bayesian requires a huge number of runs to produce an accurate posterior. Running the high-fidelity model multiple times is expensive. Our workflow tackles this problem by deploying a fine low-fidelity model to represent the high-fidelity model in the main runs. This fine low-fidelity model is fast to run, while it honors the physics and accuracy of the high-fidelity model. We also use ANOVA sensitivity analysis to measure the importance of each parameter. The ranking gives awareness to the significant ones that may contribute to the matching accuracy. We demonstrate our workflow for a geothermal reservoir with static and operational uncertainties. Our workflow produces accurate matching of thermal recovery factor and produced-enthalpy rate with physically-consistent posteriors. We present a novel workflow to account for uncertainty in reservoir history matching involving multi-resolution interaction. The proposed method is generic and can be readily applied within existing history-matching workflows in reservoir simulation.