Tree-based ensemble methods for sensitivity analysis of environmental models: A performance comparison with Sobol and Morris techniques

The main focus of this thesis is the uncertainty propagation (UP) and global sensitivity analysis (GSA) in complex hydrogeological numerical models. Various methods are presented with applications on numerical models for the groundwater flow and mean lifetime expectancy (MLE) in the scope of Andra's (French National Radioactive Waste Management Agency) project for the geological disposal of high-level and intermediate-level long-lived radioactive wastes in a highly impermeable layer from Callovo-Oxfordian age (COX) in France. <br> A state of the art is provided for the theory of uncertainty propagation and for the methodologies of sensitivity analyses in a broad sense. Methods for UP are provided from 2-levels Factorial Designs to quasi-random samplings. GSA techniques encompass screening methods such as the Morris Measures and the Derivative-based Global Sensitivity Measures (DGSM), and also the so-called Sobol' indices based upon the variance decomposition of the response of interest. Meta-modelling techniques such as polynomial regression and Polynomial Chaos Expansions (PCE) meta-models are employed as surrogate models for UP and GSA purpose at negligible computational-costs. A comparison of GSA techniques upon various complex analytical test-functions was undertaken with the purpose of determining a relevant method to be employed in the context of “screening” out unimportant variables in computer-intensive, high-dimensional models. <br> A numerical model of groundwater flow and lifetime expectancy is employed for assessing the effect of uncertain advection-dispersion parameters and their spatial distributions upon the MLE from a target zone inside the domain. The model is a 2-dimensions synthetic cross-section of the eastern region of the Paris Basin (Meuse/Haute-Marne sector). This model was used as an exploratory tool for sensitivity analysis methods applied upon numerous sets of uncertain hydrodynamic and dispersion parameters in 15 layers. The uncertainty characterizing the permeability-porosity values in aquifer formations encompassing the COX have proved to add much of variability to the MLE calculated from the target zone. The model also served at exploring the effect of the spatial variability of permeability-porosity parameters in two main aquifer sequences on the groundwater flow rates and MLE in the model. The variabilities of the output responses are mainly due to the uncertainty upon the means and variances of the permeability-porosity distributions, as well as the longitudinal correlation lengths, in each sequence. <br> Then, a 3-dimensions high-definition hydrogeological model representing the Meuse/Haute-Marne sector in the eastern region of the Paris Basin is a more comprehensive numerical model incorporating realistic geometries, fractures, heterogeneities and discontinuities encountered on field. A sensitivity analysis of the MLE from a given zone located in the middle of the COX layer was performed by perturbing the hydraulic conductivities and porosities values in fourteen hydrogeological formations. The uncertain permeability-porosity parameters in the aquifer formations from Bathonian in the Dogger sequence, and Rauracian-Sequanian in the Oxfordian sequence, have strong and rather non-linear effects on the variability of the output response of interest. <br> The methodologies for UP and GSA employed in the present thesis have proved to be very efficient when applied to large hydrogeological models of groundwater flow and MLE. In particular, quasi-random sampling methods offer a flexible frame for providing the uncertainty distribution of the output response of interest at low computational costs. Screening techniques provide a fast estimation for the overall contribution of each input variable to the variability of the output. Meta-modelling techniques such as PCE proved highly accurate in individualising the low- and high-order effects of each input variable upon the output response of interest.

In pharmacometrics, understanding a covariate effect on an interested outcome is essential for assessing the importance of the covariate. Variance-based global sensitivity analysis (GSA) can simultaneously quantify contribution of each covariate effect to the variability for the interested outcome considering with random effects. The aim of this study was to apply GSA to pharmacometric models to assess covariate effects. Simulations were conducted with pharmacokinetic models to characterize the GSA for assessment of covariate effects and with an example of quantitative systems pharmacology (QSP) models to apply the GSA to a complex model. In the simulations, covariate and random variables were generated to simulate the outcomes using the models. Ratios of variance explained by each factor (each covariate and random effect) over the overall variance of the outcome were used as sensitivity indices. The sensitivity indices were consistent with the effect size of covariate. The sensitivity indices identified the importance of creatinine clearance on the pharmacokinetic exposure for a renally-excreted drug. These sensitivity indices could be applied to plasma concentrations over time (repeated measurable outcomes over time) as interested outcomes. Using the GSA, each contribution of all of the covariate effects could be efficiently identified even in the complex QSP model. Variance-based GSA can provide insight when considering the importance of covariate effects by simultaneously and quantitatively assessing all covariate and random effects on interested outcomes in pharmacometrics.

A variance-based global sensitivity analysis (GSA) was applied to the dose assessment model used in the risk-based corrective action methodology of environmental risk analysis to identify key sources of variability and uncertainty and quantify the relative contribution of these sources to the variance of estimated dose. GSA was performed applying extended Fourier amplitude sensitivity test technique. The soil-to-air contaminant transport pathway within an inhalation exposure scenario was addressed. Three persistent semi-volatile carcinogenic chemicals, including polychlorinated biphenyls, benzo(a)pyrene, and 2,3,7,8-tetrachlorodibenzo-p-dioxin, were chosen as contaminants of concern.

Non-Linear Partial-Least-Squares-based Polynomial Chaos Expansion (NLPLS-based PCE) approach for global sensitivity analysis of a High-Q Partially Air-Filled Pedestal Resonator Integrated in a Printed Circuit Board (PCB)

Quantification of the uncertainty in constitutive model predictions describing arterial wall mechanics is vital towards non-invasive assessment of vascular drug therapies. Therefore, we perform uncertainty quantification to determine uncertainty in mechanical characteristics describing the vessel wall response upon loading. Furthermore, a global variance-based sensitivity analysis is performed to pinpoint measurements that are most rewarding to be measured more precisely. We used previously published carotid diameter–pressure and intima–media thickness (IMT) data (measured in triplicate), and Holzapfel–Gasser–Ogden models. A virtual data set containing 5000 diastolic and systolic diameter–pressure points, and IMT values was generated by adding measurement error to the average of the measured data. The model was fitted to single-exponential curves calculated from the data, obtaining distributions of constitutive parameters and constituent load bearing parameters. Additionally, we (1) simulated vascular drug treatment to assess the relevance of model uncertainty and (2) evaluated how increasing the number of measurement repetitions influences model uncertainty. We found substantial uncertainty in constitutive parameters. Simulating vascular drug treatment predicted a 6% point reduction in collagen load bearing (L_mathrm {coll}), approximately 50% of its uncertainty. Sensitivity analysis indicated that the uncertainty in L_{mathrm {coll}} was primarily caused by noise in distension and IMT measurements. Spread in L_{mathrm {coll}} could be decreased by 50% when increasing the number of measurement repetitions from 3 to 10. Model uncertainty, notably that in L_{mathrm {coll}}, could conceal effects of vascular drug therapy. However, this uncertainty could be reduced by increasing the number of measurement repetitions of distension and wall thickness measurements used for model parameterisation.

Event Abstract Back to Event Statistical uncertainty and sensitivity analysis of intracellular signaling models - through approximate Bayesian computation and variance based global sensitivity analysis Olivia Eriksson1, 2*, Alexandra Jauhiainen1, 3, Sara Maad Sasane4, Anu G Nair5, Carolina Sartorius4 and Jeanette Hellgren Kotaleski2, 5, 6 1 Contributed equally to this work, Sweden 2 Stockholm University, Department of Numerical Analysis and Computer Science, Sweden 3 AstraZeneca AB R&D, Early Clinical Biometrics, Sweden 4 Lund University, Centre for Mathematical Sciences, Sweden 5 KTH Royal Institute of Technology, School of Computer Science and Communication, Sweden 6 Karolinska Institute, Department of Neuroscience, Sweden Computational dynamical models describing intracellular signaling pathways of nerve cells are increasing in size and complexity as more information is obtained from experiments. Models describing protein interactions within neurons can contain over hundred protein reactions [1, 2]. These models are built from knowledge about the interaction topology, inferred from e.g. gene knock-out experiments, as well as from experimental quantitative data describing the input-output relationship of the observed system. The quantitative data are often sparse as compared to the size of the system, and translating the experimental information into dynamical models often results in large uncertainties in the parameter (e.g reaction rates) values. This uncertainty is partly due to the sparseness in experimental time series data (practical unidentifiability), but it is also an intrinsic feature of the modeled system (structural unidentifiability) and could reflect the possibility for biological variation [3]; in many biological systems the same function can be achieved in partly different ways due to degeneracy and redundancy. Here we investigate the extent of this parameter uncertainty, in a model describing calcium (Ca)-dependent activation of Calmodulin (CaM), Protein phosphatase 2B (PP2B) and Ca/CaM-dependent protein kinase II (CamKII) [4], which is an intracellular pathway of importance to synaptic plasticity. We presume that the structure of the model is correct and then characterize the uncertainty in the parameters based on data. This is done by sampling the subspace of parameter values which has an equally good fit to data (corresponding to the “viable space” in [5]), through a statistical approach known as Approximate Bayesian Computation (ABC) with Markov Chain Monte Carlo (MCMC) [6]. ABC has been proposed for parameter estimation in system biology models previously, albeit in slightly different settings, e.g. [7, 8], and can be applied to a wide range of problems. In order to characterize the viable space efficiently with several experimental datasets, we employ multivariate probability distributions called copulas as a part of the ABC sampling. The result is a multivariate posterior distribution for the parameters, which represents how much they are constrained by the data (i.e. the uncertainty) as illustrated in Figure 1 (pairwise plots of viable space for four parameters indicating constraints and dependencies before and after calibration to data). The next step is to look into how the parameter uncertainty is translated to uncertainty of the predictions made from the model. We also investigate the role and importance of different parameters, with respect to different model outputs including the predictions, by means of global sensitivity analysis [9]. Figure 2 illustrates the sensitivity of one of the outputs calculated by a method based on the decomposition of the variance [10, 11, 12], showing different types of sensitivity measures. Finally, we explore how the sensitivity of different subsets of parameters change as different parts of the viable space are investigated, which is shown in Figure 3, where sensitivity profiles are clustered into groups with different sensitivity characteristics. Figure 1 Figure 2 Figure 3 Acknowledgements This research was supported by funding from the Swedish e-Science Research Centre and the Human Brain Project References [1] Gutierrez-Arenas O, Eriksson O, Hellgren Kotaleski J (2014) Segregation and Crosstalk of D1 Receptor-Mediated Activation of ERK in Striatal Medium Spiny Neurons upon Acute Administration of Psychostimulants. PLoS Comput Biol 10, e1003445. [2] Nair AG, Gutierrez-Arenas O, Eriksson O, Vincent P, and Hellgren Kotaleski J (2015) Sensing Positive versus Negative Reward Signals through Adenylyl Cyclase-Coupled GPCRs in Direct and Indirect Pathway Striatal Medium Spiny Neurons. J. Neuroscience 35, 14017-14030. [3] Marder E, Goeritz ML, Otopalik AG (2015) Robust circuit rhythms in small circuits arise from variable circuit components and mechanisms. Curr Opin Neurobiol 31, 156-163. [4] Nair AG, Gutierrez-Arenas O, Eriksson O, Jauhiainen A, Blackwell KT and Kotaleski, JH (2014). Modeling intracellular signaling underlying striatal function in health and disease. Progress in molecular biology and translational science, 123, 277 [5] Zamora-Sillero E, Hafner M, Ibig A, Stelling J, and Wagner, A. (2011). Efficient characterization of high-dimensional parameter spaces for systems biology. BMC systems biology 5, 142. [6] Marjoram P, Molitor J, Plagnol V, and Tavare S (2003) Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. U.S.A. 100, 15324–15328. [7] Toni T, Welch D, Strelkowa N, Ipsen A, and Stumpf MP. Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J R Soc Interface, 6, 187–202. [8] Liepe J, Kirk P, Filippi S, Toni T, Barnes CP, and Stumpf MP. A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation. Nat Protoc, 9, 439–456. [9] Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D, Saisana M, Tarantola S Global sensitivity analysis: the primer. In Wiley 2008 New York, NY:Wiley. [10] Sobol IM (2001) Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates, Math. Comput. Simul. 55, 271–280. [11] Saltelli A (2002) Making best use of model evaluations to compute sensitivity indices. Comp Phys Comm 145, 280 – 297. [12] Halnes G, Ulfhielm E Ljunggren EE, Kotaleski JH and Rospars JP (2009) Modelling and sensitivity analysis of the reactions involving receptor, G-protein and effector in vertebrate olfactory receptor neurons. Journal of computational neuroscience 27, 471-491. Keywords: sensitivity analysis, Approximate Bayesian Computation, copulas, intracellular signalling, Dynamical Modeling, global sensitivity analysis, Markov chain Monte Carlo, variance decomposition Conference: Neuroinformatics 2016, Reading, United Kingdom, 3 Sep - 4 Sep, 2016. Presentation Type: Investigator presentations Topic: Computational neuroscience Citation: Eriksson O, Jauhiainen A, Maad Sasane S, Nair A, Sartorius C and Hellgren Kotaleski J (2016). Statistical uncertainty and sensitivity analysis of intracellular signaling models - through approximate Bayesian computation and variance based global sensitivity analysis. Front. Neuroinform. Conference Abstract: Neuroinformatics 2016. doi: 10.3389/conf.fninf.2016.20.00014 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 31 May 2016; Published Online: 18 Jul 2016. * Correspondence: PhD. Olivia Eriksson, Contributed equally to this work, Stockholm, Sweden, olivia@kth.se Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Olivia Eriksson Alexandra Jauhiainen Sara Maad Sasane Anu G Nair Carolina Sartorius Jeanette Hellgren Kotaleski Google Olivia Eriksson Alexandra Jauhiainen Sara Maad Sasane Anu G Nair Carolina Sartorius Jeanette Hellgren Kotaleski Google Scholar Olivia Eriksson Alexandra Jauhiainen Sara Maad Sasane Anu G Nair Carolina Sartorius Jeanette Hellgren Kotaleski PubMed Olivia Eriksson Alexandra Jauhiainen Sara Maad Sasane Anu G Nair Carolina Sartorius Jeanette Hellgren Kotaleski Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Identifying key sources of uncertainty in the modelling of greenhouse gas emissions from wastewater treatment

This paper presents the use of Global Sensitivity Analysis (GSA) methods to analyze the sensitivity level of the performance of the impedance measurement algorithm of SEL-421 distance relay protection when used to perform transmission line protection with compensation circuit. This GSA technique was developed to see the dominance effect of system uncertainty (or factors). Two techniques that might be used for sensitivity analysis using GSA are Quasi-Monte Carlo (QMC) and data sampling with Sobol technique. Experimental is done with DIgSILENT software to do modeling of the power system, simulation, and calculation of impedance and algorithm error. The testing is carried out automatically with the help of the developed algorithm using DPL (DIgSILENT Programming Language). SIMLAB software, in this case, (with Sobol technique) is applied to generate some non-linear factor data and analyze the output of variations in impedance measurement errors.

Global Sensitivity Analysis of Arrhythmic Risk Biomarkers in Cardiac Single Cell Models

Stroke Treatment Prediction Using Features Selection Methods and Machine Learning Classifiers

Optimal measurement design for parameter identification in mechanized tunneling

Influence of selecting secondary settling tank sub-models on the calibration of WWTP models – A global sensitivity analysis using BSM2

The advent of next-generation sequencing has greatly accelerated the field of human microbiome studies. Currently, investigators are seeking, struggling and competing to find new ways to diagnose, treat and prevent human diseases through the human microbiome. Machine learning is a promising approach to help such an effort, especially due to the high complexity of microbiome data. However, many of the current machine learning algorithms are in a "black box", i.e., they are difficult to understand and interpret. In addition, clinicians, public health practitioners and biologists are not usually skilled at computer programming, and they do not always have high-end computing devices. Thus, in this study, we introduce a unified web cloud analytic platform, named MiTree, for user-friendly and interpretable microbiome data mining. MiTree employs tree-based learning methods, including decision tree, random forest and gradient boosting, that are well understood and suited to human microbiome studies. We also stress that MiTree can address both classification and regression problems through covariate-adjusted or unadjusted analysis. MiTree should serve as an easy-to-use and interpretable data mining tool for microbiome-based disease prediction modeling, and should provide new insights into microbiome-based diagnostics, treatment and prevention. MiTree is an open-source software that is available on our web server.

In recent years manyLife Cycle Assessment (LCA) studies have beenconducted to quantify the environmental performance of products andservices. Some of these studies propagated numerical uncertaintiesin underlying data to LCA results, and several applied Global SensitivityAnalysis (GSA) to some parts of the LCA model to determine its mainuncertainty drivers. However, only a few studies have tackled theGSA of complete LCA models due to the high computational cost of suchanalysis and the lack of appropriate methods for very high-dimensionalmodels. This study proposes a new GSA protocol suitable for largeLCA problems that, unlike existing approaches, does not make assumptionson model linearity and complexity and includes extensive validationof GSA results. We illustrate the benefits of our protocol by comparingit with an existing method in terms of filtering of noninfluentialand ranking of influential uncertainty drivers and include an applicationexample of Swiss household food consumption. We note that our protocolobtains more accurate GSA results, which leads to better understandingof LCA models, and less data collection efforts to achieve more robustestimation of environmental impacts. Implementations supporting thiswork are available as free and open source Python packages.

BackgroundCoronavirus disease (COVID-19) presents an unprecedented threat to global health worldwide. Accurately predicting the mortality risk among the infected individuals is crucial for prioritizing medical care and mitigating the healthcare system’s burden. The present study aimed to assess the predictive accuracy of machine learning methods to predict the COVID-19 mortality risk.MethodsWe compared the performance of classification tree, random forest (RF), extreme gradient boosting (XGBoost), logistic regression, generalized additive model (GAM) and linear discriminant analysis (LDA) to predict the mortality risk among 49,216 COVID-19 positive cases in Toronto, Canada, reported from March 1 to December 10, 2020. We used repeated split-sample validation and k-steps-ahead forecasting validation. Predictive models were estimated using training samples, and predictive accuracy of the methods for the testing samples was assessed using the area under the receiver operating characteristic curve, Brier’s score, calibration intercept and calibration slope.ResultsWe found XGBoost is highly discriminative, with an AUC of 0.9669 and has superior performance over conventional tree-based methods, i.e., classification tree or RF methods for predicting COVID-19 mortality risk. Regression-based methods (logistic, GAM and LASSO) had comparable performance to the XGBoost with slightly lower AUCs and higher Brier’s scores.ConclusionsXGBoost offers superior performance over conventional tree-based methods and minor improvement over regression-based methods for predicting COVID-19 mortality risk in the study population.