Extended Monte Carlo Simulation for Parametric Global Sensitivity Analysis and Optimization

Computer models are widely used to help make decisions about real-world systems. As computer models of large and complex systems can have long run-times and high-dimensional input spaces, it is often necessary to use emulation to assess uncertainties in computer model output. This paper presents methodology for emulation of complex computer models motivated by a real-world example in energy policy. The computer model studied is an economic model of investment in electricity generation in Great Britain. The computer model was used to select parameters in a government policy designed to incentivize investment in renewable technologies to meet government targets. Limited computing time meant that few runs of the computer model were available to fit an emulator. The statistical methodology developed was therefore focused on accurately capturing the uncertainty in computer model output arising from the small number of available model runs. A varying coefficient emulator is proposed to model uncertainty in model output when extrapolating away from model runs. To maximize use of the small number of runs available, this varying coefficient emulator is paired with a criterion-based procedure for design selection.

A sensitivity analysis of a personalized pulse wave propagation model for arteriovenous fistula surgery. Part A: Identification of most influential model parameters

A novel procedure to analyse the uncertainty associated to the output of GIS-based models is presented. The procedure can handle models of any degree of complexity that accept any kind of input data. Two important aspects of spatial modelling are addressed: the propagation of uncertainty from model inputs and model parameters up to the model output (uncertainty analysis); and the assessment of the relative importance of the sources of uncertainty in the output uncertainty (sensitivity analysis). Two main applications are proposed. The procedure allows implementation of a GIS-based model whose output can reliably support the decision process with an optimized allocation of resources for spatial data acquisition. This is possible in low cost strategy, based on numerical simulations on a small prototype of the GIS-based model. Furthermore, the procedure provides an effective model building tool to choose, from a group of alternative models, the best one in terms of cost-benefit analysis. A comprehensive case study is described. It concerns the implementation of a new GIS-based hydrologic model, whose goal is providing near real-time flood forecasting.

Global sensitivity analysis has been widely used to detect the relative contributions of input variables to the uncertainty of model output, and then more resources can be assigned to the important input variables to reduce the uncertainty of model output more efficiently. In this paper, a new kind of global sensitivity index based on Gini’s mean difference is proposed. The proposed sensitivity index is more robust than the variance-based first order sensitivity index for the cases with non-normal distributions. Through the decomposition of Gini’s mean difference, it shows that the proposed sensitivity index can be represented by the energy distance, which measures the difference between probability distributions. Therefore, the proposed sensitivity index also takes the probability distribution of model output into consideration. In order to estimate the proposed sensitivity index efficiently, an efficient Monte Carlo simulation method is also proposed, which avoids the nested sampling procedure. The test examples show that the proposed sensitivity index is more robust than the variance-based first order sensitivity index for the cases with non-normal distributions.

Recent efforts to ensure the reliability of computational model-based predictions in healthcare, such as the ASME V&V40 Standard, emphasize the importance of uncertainty quantification (UQ) and sensitivity analysis (SA) when evaluating computational models. UQ involves empirically determining the uncertainty in model inputs—typically resulting from natural variability or measurement error—and then calculating the resultant uncertainty in model outputs. SA involves calculating how uncertainty in model outputs can be apportioned to input uncertainty. Rigorous comprehensive UQ/SA provides confidence that model-based decisions are robust to underlying uncertainties. However, comprehensive UQ/SA is not currently feasible for whole heart models, due to numerous factors including model complexity and difficulty in measuring variability in the many parameters. Here, we present a significant step to developing a framework to overcome these limitations. We: (i) developed a novel action potential (AP) model of moderate complexity (six currents, seven variables, 36 parameters); (ii) prescribed input variability for all parameters (not empirically derived); (iii) used a single “hyper-parameter” to study increasing levels of parameter uncertainty; (iv) performed UQ and SA for a range of model-derived quantities with physiological relevance; and (v) present quantitative and qualitative ways to analyze different behaviors that occur under parameter uncertainty, including “model failure”. This is the first time uncertainty in every parameter (including conductances, steady-state parameters, and time constant parameters) of every ionic current in a cardiac model has been studied. This approach allowed us to demonstrate that, for this model, the simulated AP is fully robust to low levels of parameter uncertainty — to our knowledge the first time this has been shown of any cardiac model. A range of dynamics was observed at larger parameter uncertainty (e.g., oscillatory dynamics); analysis revealed that five parameters were highly influential in these dynamics. Overall, we demonstrate feasibility of performing comprehensive UQ/SA for cardiac cell models and demonstrate how to assess robustness and overcome model failure when performing cardiac UQ analyses. The approach presented here represents an important and significant step toward the development of model-based clinical tools which are demonstrably robust to all underlying uncertainties and therefore more reliable in safety-critical decision-making.

Parameter sensitivity analysis can determine the influence of the input parameters on the model output. Identification and calibration of critical parameters are the crucial points of the process model optimization. Based on the Extended Fourier Amplitude Sensitivity Test (EFAST) and the Morris method, this paper analyzes and compares the parameter sensitivity of the annual mean net primary productivity (NPP) of Larix olgensis Henry forests in Jilin Province simulated by the Lund–Potsdam–Jena dynamic global vegetation model (LPJ model) in 2009–2014 and 2000–2019, and deeply examines the sensitivity and influence of the two methods to each parameter and their respective influence on the model’s output. Moreover, it optimizes some selected parameters and re-simulates the NPP of Larix olgensis forests in Jilin Province from 2010 to 2019. The conclusions are the following: (1) For the LPJ model, the sensitive and non-influential parameters could be identified, which could guide the optimization order of the model and was valuable for model area applications. (2) The results of the two methods were similar but not identical. The sensitivity parameters were significantly correlated (p < 0.05); parameter krp was the most sensitive parameter, followed by parameters αm, αa and gm. These sensitive parameters were mainly found in the photosynthesis, water balance, and allometric growth modules. (3) The EFAST method had a higher precision than the Morris method, which could calculate quantitatively the contribution rate of each parameter to the variances of the model results; however, the Morris method involved fewer model running times and higher efficiency. (4) The mean relative error (MRE) and mean absolute error (MAE) of the simulated value of LPJ model after parameter optimization decreases. The optimized annual mean value of NPP from 2010 to 2019 was 580 g C m−2 a−1, with a mean annual growth rate of 2.13%, exhibiting a fluctuating growth trend. The MAE of the simulated value of LPJ model after parameter optimization decreases.

SummaryMoment‐independent regional sensitivity analysis (RSA) is a very useful guide tool for assessing the effect of a specific range of an individual input on the uncertainty of model output, while large computational burden is involved to perform RSA, which would certainty lead to the limitation of engineering application. Main tasks for performing RSA are to estimate the probability density function (PDF) of model output and the joint PDF of model output and the input variable by some certain smart techniques. Firstly, a method based on the concepts of maximum entropy, fractional moment and sparse grid integration is utilized to estimate the PDF of the model output. Secondly, Nataf transformation is applied to obtain the joint PDF of model output and the input variable. Finally, according to an integral transformation, those regional sensitivity indices can be easily computed by a Monte Carlo procedure without extra function evaluations. Because all the PDFs can be estimated with great efficiency, and only a small amount of function evaluations are involved in the whole process, the proposed method can greatly decrease the computational burden. Several examples with explicit or implicit input–output relations are introduced to demonstrate the accuracy and efficiency of the proposed method. Copyright © 2015 John Wiley & Sons, Ltd.

We dealt with a rather frequent and difficult situation while modelling extreme floods, namely, model output uncertainty in data sparse regions. A historical extreme flood event was chosen to illustrate the challenges involved. Our aim was to understand what the causes might have been and specifically to show how input and model parameter uncertainties affect the output. For this purpose, a conceptual model was calibrated and validated with recent data rich time period. Resulting model parameters were used to model the historical event which subsequently resulted in a rather poor hydrograph. Due to the bad model performance, a spatial simulation technique was used to invert the model for precipitation. Constraints, such as taking the precipitation values at historical observation locations in to account, with correct spatial structures and following the observed regional distributions were used to generate realistic precipitation fields. Results showed that the inverted precipitation improved the performance significantly even when using many different model parameters. We conclude that while modelling in data sparse conditions both model input and parameter uncertainties have to be dealt with simultaneously to obtain meaningful results.

Modelling uncertainty of vehicular emissions inventory: A case study of Ireland

An alternative model to explain the vectorial capacity using as example Aedes aegypti case in dengue transmission

Sensitivity analysis is a vital tool in numerical modeling to identify important parameters and processes that contribute to the overall uncertainty in model outputs. We developed a new sensitivity analysis method to quantify the relative importance of uncertain model processes that contain multiple uncertain parameters. The method is based on the concepts of Bayesian networks (BNs) to account for complex hierarchical uncertainty structure of a model system. We derived a new set of sensitivity indices using the methodology of variance‐based global sensitivity analysis with the Bayesian inference. The framework is capable of representing the detailed uncertainty information of a complex model system using BNs and affords flexible grouping of different uncertain inputs given their characteristics and dependency structures. We have implemented the method on a real‐world biogeochemical model at the groundwater‐surface water interface within the Hanford Site's 300 Area. The uncertainty sources of the model were first grouped into forcing scenario and three different processes based on our understanding of the complex system. The sensitivity analysis results indicate that both the reactive transport and groundwater flow processes are important sources of uncertainty for carbon‐consumption predictions. Within the groundwater flow process, the structure of geological formations is more important than the permeability heterogeneity within a given geological formation. Our new sensitivity analysis framework based on BNs offers substantial flexibility for investigating the importance of combinations of interacting uncertainty sources in a hierarchical order, and it is expected to be applicable to a wide range of multiphysics models for complex systems.

The global animal food chain has a large contribution to the global anthropogenic greenhouse gas (GHG) emissions, but its share and sources vary highly across the world. However, the assessment of GHG emissions from livestock production is subject to various uncertainties, which have not yet been well quantified at large spatial scale. We assessed the uncertainties in the relations between animal production (milk, meat, egg) and the CO2, CH4, and N2O emissions in Africa, Latin America and the European Union, using the MITERRA-Global model. The uncertainties in model inputs were derived from time series of statistical data, literature review or expert knowledge. These model inputs and parameters were further divided into nine groups based on type of data and affected greenhouse gas. The final model output uncertainty and the uncertainty contribution of each group of model inputs to the uncertainty were quantified using a Monte Carlo approach, taking into account their spatial and cross-correlation. GHG emissions and their uncertainties were determined per livestock sector, per product and per emission source category. Results show large variation in the GHG emissions and their uncertainties for different continents, livestock sectors products or source categories. The uncertainty of total GHG emissions from livestock sectors is higher in Africa and Latin America than in the European Union. The uncertainty of CH4 emission is lower than that for N2O and CO2. Livestock parameters, CH4 emission factors and N emission factors contribute most to the uncertainty in the total model output. The reliability of GHG emissions from livestock sectors is relatively high (low uncertainty) at continental level, but could be lower at country level.

Parametric sensitivity analysis (SA) is an essential tool in optical data analysis aiming to quantify the relative importance of optical model parameters and identify those with a low influence which should remain fixed during analysis in order to simplify a model. Typically, such sensitivity studies are performed using a local sensitivity analysis, also referred to as “one-at-a-time” analysis, which estimates parameter sensitivity by evaluating the changes in model output while perturbing one parameter at a time and holding all other parameters fixed. All the pitfalls of the local sensitivity approach are quite obvious and well known; however, local SA is still prevailing in the area of data analysis for spectroscopic ellipsometry. In this study, we discuss applications of a global qualitative analysis introduced by Morris in 1991 which determines an overall significance for each model parameter and describes its nonlinear effect on model’s output and/or interactions with other parameters. The Morris method allows effective separation of the model parameters into different groups such as parameters with negligible effects, parameters with large linear effects without interactions, and parameters with large nonlinear and/or interaction effects. Here, we carried out the Morris sensitivity analysis to reveal the model parameter significance ranking in a test case of the B-spline model for titanium nitride (TiN). Such a screening approach can be considered as a practical tool for the initial global SA strategy prior to true “all-at-a-time” global sensitivity analysis.

As part of a research program to understand and model eye damage produced by exposure to cw and pulsed lasers, the U.S. Air Force has created a granular model of laser retinal damage. The Thompson granular model simulates absorption of light by melanosomes distributed in the retinal pigmented epithelium, melanosome heating, and subsequent photothermal damage from bulk tissue heating. Various biological input parameters required for the model, such as the density, size, spatial distribution, and absorption coefficient of melanosomes, are not well known, creating uncertainty in the results. This problem is being addressed both experimentally, through measurements of biological parameters for various species, and theoretically, through analysis of parameter sensitivity in the model. In the current study, the parameter sensitivity was analyzed using a technique known as 'design of experiments,' which allows statistical estimation of the relative importance of independent experimental variables. A matrix of 20 cases has been analyzed, using 7 input parameters as independent variables. Cases have been confined to the long pulse regime (greater than or equal to 10 microseconds), where photothermal damage is dominant. Results were assessed using both temperature rise and Arrhenius damage integral values. Corneal fluence was found to be the most important physical parameter and melanosome absorption the most important biological parameter.

Uncertainty in hydrological modelling is often associated with model structure or calibration coefficients, and little consideration is given to the visualisation of uncertainty in either input or output data. As hydrological modelling is often a component of landscape design, there is continuing interest in the visualisation of uncertainty in model inputs and outputs, which could include catchment size or landcover (spatial inputs), and the subsequent impact on uncertainty in model predictions (such as streamflow). In this paper, inputs and outputs associated with hydrological models are discussed in the context of spatial uncertainty and how this information may be presented to users by means of statistical and spatial visualisation techniques. A regional study in hydrological modelling is used to demonstrate the sensitivity to spatial uncertainty and its consequences. Tools and techniques are discussed for the visual presentation of uncertainty, with implications for land management and policy decisions. A description is given of an experimental software tool developed for visualisation of spatial uncertainty in hydrological landscape cross-sections, where depth-to-water-table is used as a predictor for dryland salinity. Uncertainty in the estimate of this depth is important to analysts and policy makers.