Parameter Estimation for Reynolds-Averaged Navier–Stokes Models Using Approximate Bayesian Computation

Event Abstract Back to Event Statistical uncertainty and sensitivity analysis of intracellular signaling models - through approximate Bayesian computation and variance based global sensitivity analysis Olivia Eriksson1, 2*, Alexandra Jauhiainen1, 3, Sara Maad Sasane4, Anu G Nair5, Carolina Sartorius4 and Jeanette Hellgren Kotaleski2, 5, 6 1 Contributed equally to this work, Sweden 2 Stockholm University, Department of Numerical Analysis and Computer Science, Sweden 3 AstraZeneca AB R&D, Early Clinical Biometrics, Sweden 4 Lund University, Centre for Mathematical Sciences, Sweden 5 KTH Royal Institute of Technology, School of Computer Science and Communication, Sweden 6 Karolinska Institute, Department of Neuroscience, Sweden Computational dynamical models describing intracellular signaling pathways of nerve cells are increasing in size and complexity as more information is obtained from experiments. Models describing protein interactions within neurons can contain over hundred protein reactions [1, 2]. These models are built from knowledge about the interaction topology, inferred from e.g. gene knock-out experiments, as well as from experimental quantitative data describing the input-output relationship of the observed system. The quantitative data are often sparse as compared to the size of the system, and translating the experimental information into dynamical models often results in large uncertainties in the parameter (e.g reaction rates) values. This uncertainty is partly due to the sparseness in experimental time series data (practical unidentifiability), but it is also an intrinsic feature of the modeled system (structural unidentifiability) and could reflect the possibility for biological variation [3]; in many biological systems the same function can be achieved in partly different ways due to degeneracy and redundancy. Here we investigate the extent of this parameter uncertainty, in a model describing calcium (Ca)-dependent activation of Calmodulin (CaM), Protein phosphatase 2B (PP2B) and Ca/CaM-dependent protein kinase II (CamKII) [4], which is an intracellular pathway of importance to synaptic plasticity. We presume that the structure of the model is correct and then characterize the uncertainty in the parameters based on data. This is done by sampling the subspace of parameter values which has an equally good fit to data (corresponding to the “viable space” in [5]), through a statistical approach known as Approximate Bayesian Computation (ABC) with Markov Chain Monte Carlo (MCMC) [6]. ABC has been proposed for parameter estimation in system biology models previously, albeit in slightly different settings, e.g. [7, 8], and can be applied to a wide range of problems. In order to characterize the viable space efficiently with several experimental datasets, we employ multivariate probability distributions called copulas as a part of the ABC sampling. The result is a multivariate posterior distribution for the parameters, which represents how much they are constrained by the data (i.e. the uncertainty) as illustrated in Figure 1 (pairwise plots of viable space for four parameters indicating constraints and dependencies before and after calibration to data). The next step is to look into how the parameter uncertainty is translated to uncertainty of the predictions made from the model. We also investigate the role and importance of different parameters, with respect to different model outputs including the predictions, by means of global sensitivity analysis [9]. Figure 2 illustrates the sensitivity of one of the outputs calculated by a method based on the decomposition of the variance [10, 11, 12], showing different types of sensitivity measures. Finally, we explore how the sensitivity of different subsets of parameters change as different parts of the viable space are investigated, which is shown in Figure 3, where sensitivity profiles are clustered into groups with different sensitivity characteristics. Figure 1 Figure 2 Figure 3 Acknowledgements This research was supported by funding from the Swedish e-Science Research Centre and the Human Brain Project References [1] Gutierrez-Arenas O, Eriksson O, Hellgren Kotaleski J (2014) Segregation and Crosstalk of D1 Receptor-Mediated Activation of ERK in Striatal Medium Spiny Neurons upon Acute Administration of Psychostimulants. PLoS Comput Biol 10, e1003445. [2] Nair AG, Gutierrez-Arenas O, Eriksson O, Vincent P, and Hellgren Kotaleski J (2015) Sensing Positive versus Negative Reward Signals through Adenylyl Cyclase-Coupled GPCRs in Direct and Indirect Pathway Striatal Medium Spiny Neurons. J. Neuroscience 35, 14017-14030. [3] Marder E, Goeritz ML, Otopalik AG (2015) Robust circuit rhythms in small circuits arise from variable circuit components and mechanisms. Curr Opin Neurobiol 31, 156-163. [4] Nair AG, Gutierrez-Arenas O, Eriksson O, Jauhiainen A, Blackwell KT and Kotaleski, JH (2014). Modeling intracellular signaling underlying striatal function in health and disease. Progress in molecular biology and translational science, 123, 277 [5] Zamora-Sillero E, Hafner M, Ibig A, Stelling J, and Wagner, A. (2011). Efficient characterization of high-dimensional parameter spaces for systems biology. BMC systems biology 5, 142. [6] Marjoram P, Molitor J, Plagnol V, and Tavare S (2003) Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. U.S.A. 100, 15324–15328. [7] Toni T, Welch D, Strelkowa N, Ipsen A, and Stumpf MP. Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J R Soc Interface, 6, 187–202. [8] Liepe J, Kirk P, Filippi S, Toni T, Barnes CP, and Stumpf MP. A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation. Nat Protoc, 9, 439–456. [9] Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D, Saisana M, Tarantola S Global sensitivity analysis: the primer. In Wiley 2008 New York, NY:Wiley. [10] Sobol IM (2001) Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates, Math. Comput. Simul. 55, 271–280. [11] Saltelli A (2002) Making best use of model evaluations to compute sensitivity indices. Comp Phys Comm 145, 280 – 297. [12] Halnes G, Ulfhielm E Ljunggren EE, Kotaleski JH and Rospars JP (2009) Modelling and sensitivity analysis of the reactions involving receptor, G-protein and effector in vertebrate olfactory receptor neurons. Journal of computational neuroscience 27, 471-491. Keywords: sensitivity analysis, Approximate Bayesian Computation, copulas, intracellular signalling, Dynamical Modeling, global sensitivity analysis, Markov chain Monte Carlo, variance decomposition Conference: Neuroinformatics 2016, Reading, United Kingdom, 3 Sep - 4 Sep, 2016. Presentation Type: Investigator presentations Topic: Computational neuroscience Citation: Eriksson O, Jauhiainen A, Maad Sasane S, Nair A, Sartorius C and Hellgren Kotaleski J (2016). Statistical uncertainty and sensitivity analysis of intracellular signaling models - through approximate Bayesian computation and variance based global sensitivity analysis. Front. Neuroinform. Conference Abstract: Neuroinformatics 2016. doi: 10.3389/conf.fninf.2016.20.00014 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 31 May 2016; Published Online: 18 Jul 2016. * Correspondence: PhD. Olivia Eriksson, Contributed equally to this work, Stockholm, Sweden, olivia@kth.se Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Olivia Eriksson Alexandra Jauhiainen Sara Maad Sasane Anu G Nair Carolina Sartorius Jeanette Hellgren Kotaleski Google Olivia Eriksson Alexandra Jauhiainen Sara Maad Sasane Anu G Nair Carolina Sartorius Jeanette Hellgren Kotaleski Google Scholar Olivia Eriksson Alexandra Jauhiainen Sara Maad Sasane Anu G Nair Carolina Sartorius Jeanette Hellgren Kotaleski PubMed Olivia Eriksson Alexandra Jauhiainen Sara Maad Sasane Anu G Nair Carolina Sartorius Jeanette Hellgren Kotaleski Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

This study addresses the challenge of estimating parameters for two logistic populations that share a common scale parameter but have different location parameters in the presence of fuzzy data. To handle these complexities, both Maximum Likelihood Estimation (MLE) and Bayesian methods are employed. Asymptotic confidence intervals are constructed using ML estimates. For Bayesian estimation, a conjugate prior is utilized, and Bayes estimators are approximated using Lindley’s method due to the lack of closed-form solutions. Furthermore, Approximate Bayesian Computation (ABC) and Markov Chain Monte Carlo (MCMC) techniques, including Hamiltonian Monte Carlo (HMC) and the Metropolis–Hastings (MH) algorithm, are utilized to sample from the posterior distributions and construct Highest Posterior Density (HPD) intervals. A detailed comparative analysis of MLE, Lindley’s approximation, ABC, HMC, and MH is conducted to assess their performance. The effectiveness of the proposed methodology is demonstrated using a real-world dataset under fuzzy conditions.

Numerical simulations based on Reynolds-averaged Navier Stokes (RANS) models are still the work-horse tool in engineering design involving turbulent flows. Two decades ago, when LES started gaining popularity thanks to the increasing availability of computational resources, it was widely expected that it would have gradually replaced RANS methods in industrial CFD for decades to come. In the past two decades, however, while LES-based methods gained widespread applications and the earlier hope did not diminish, the predicted time when LES would replace RANS has been significantly delayed. Most industrial users are probably decades away from any routine use of scale resolving simulations, not to mention the cost, time and user skill it take to run these computations. In brief, RANS solvers, particularly those based on standard eddy viscosity models (e.g k-e, k-ω, S-A and k-ω SST) are expected to remain the workhorse in the CFD of high Reynolds number flows for decades. However, predictions from RANS simulations are known to have large discrepancies in many flows of engineering relevance, including those with swirl, pressure gradients, or mean streamline curvature. It is a consensus that the dominant cause for such discrepancies is the RANS-modeled Reynolds stresses. In light of the long stagnation in traditional turbulence modeling, researchers explored machine learning as an alternative to improve RANS modeling by leveraging data from highfidelity simulations. The goal is to make use of vast amounts of turbulent flows data, machine learning techniques and current understanding of turbulence physics to develop models with better predictive capabilities in the context of RANS simulations. Recently, in a seminar work, Ling et al (2016) developed a neural network architecture capable of embedding invariance properties into the Reynolds stress tensor predicted in output. Such a network, named the tensor basis neural network (TBNN), was applied to a variety of flow fields with encouraging results compared to both classical turbulence models and neural networks that do not preserve Galilean invariance. Yet, as in most data driven turbulence modelling approaches, the TBNN was used as a post-processing tool to correct the Reynolds stress tensor field predicted by a RANS simulation run with standard closure models. This means that, theoretically, the network can be applied only to correct the Reynolds stress tensor for the same RANS model on which it has been trained since, in general, different turbulence models yield different results depending on the flow type. Moreover, there is no physisical insight that suggests a relation between the RANS velocity gradients - used as inputs of the machine learning model - and the true Reynolds stress tensor. Differently, in this work a network with a similar architecture to the Ling’s one was trained and tested on a database of high-fidelity data of eight different flows to learn a functional mapping between the inputs of Pope’s General Eddy Viscosity Model and the anisotropic part of the Reynolds stress tensor. Then the network was embedded into a CFD RANS solver as a replacement of the standard closure model - and therefore called at every solver’s iteration. Lasty, the RANS solver with embedded TBNN was be tested on a canonical flow case - turbulent channel flow - to evaluate its performances. As for the organization of this work: in Chapter 1 further details on the data driven turbulence modelling will be given, RANS models and equations will be introduced and also an introduction to Neural Networks will be presented. In Chapter 2, it will be given a detailedexplanation of the RANS CFD solver and the neural network’s implementation. In Chapter 3, the method will be tested on a turbulent channel flow case and the results will be discussed. Lastly, in Chapter 4, some meaningful conclusions will be drawn.

A Markov Chain Monte Carlo technique for parameter estimation and inference in pesticide fate and transport modeling

Reynolds Averaged Navier Stokes (RANS) models are widely used in industry to predict fluid flows, despite their acknowledged deficiencies. Not only do RANS models often produce inaccurate flow predictions, but there are very limited diagnostics available to assess RANS accuracy for a given flow configuration. If experimental or higher fidelity simulation results are not available for RANS validation, there is no reliable method to evaluate RANS accuracy. This paper explores the potential of utilizing machine learning algorithms to identify regions of high RANS uncertainty. Three different machine learning algorithms were evaluated: support vector machines, Adaboost decision trees, and random forests. The algorithms were trained on a database of canonical flow configurations for which validated direct numerical simulation or large eddy simulation results were available, and were used to classify RANS results on a point-by-point basis as having either high or low uncertainty, based on the breakdown of specific RANS modeling assumptions. Classifiers were developed for three different basic RANS eddy viscosity model assumptions: the isotropy of the eddy viscosity, the linearity of the Boussinesq hypothesis, and the non-negativity of the eddy viscosity. It is shown that these classifiers are able to generalize to flows substantially different from those on which they were trained. Feature selection techniques, model evaluation, and extrapolation detection are discussed in the context of turbulence modeling applications.

Next-generation sequencing of pooled samples (Pool-seq) is a popular method to assess genome-wide diversity patterns in natural and experimental populations. However, Pool-seq is associated with specific sources of noise, such as unequal individual contributions. Consequently, using Pool-seq for the reconstruction of evolutionary history has remained underexplored. Here we describe a novel Approximate Bayesian Computation (ABC) method to infer demographic history, explicitly modelling Pool-seq sources of error. By jointly modelling Pool-seq data, demographic history and the effects of selection due to barrier loci, we obtain estimates of demographic history parameters accounting for technical errors associated with Pool-seq. Our ABC approach is computationally efficient as it relies on simulating subsets of loci (rather than the whole-genome) and on using relative summary statistics and relative model parameters. Our simulation study results indicate Pool-seq data allows distinction between general scenarios of ecotype formation (single versus parallel origin) and to infer relevant demographic parameters (e.g. effective sizes and split times). We exemplify the application of our method to Pool-seq data from the rocky-shore gastropod Littorina saxatilis, sampled on a narrow geographical scale at two Swedish locations where two ecotypes (Wave and Crab) are found. Our model choice and parameter estimates show that ecotypes formed before colonization of the two locations (i.e. single origin) and are maintained despite gene flow. These results indicate that demographic modelling and inference can be successful based on pool-sequencing using ABC, contributing to the development of suitable null models that allow for a better understanding of the genetic basis of divergent adaptation.

Bayesian model updating has received considerable attention and has been extensively used in structural damage detection. It provides a rigorous statistical framework for realizing structural system identification and characterizing uncertainties associated with modeling and measurements. The Markov Chain Monte Carlo (MCMC) is a promising tool for inferring the posterior distribution of model parameters to avoid the intractable evaluation of multi-dimensional integration. However, the efficacy of most MCMC techniques suffers from the curse of parameter dimension, which restricts the application of Bayesian model updating to the damage detection of large-scale systems. In addition, there are several MCMC techniques that require users to properly choose application-specific models, based on the understanding of algorithm mechanisms and limitations. As seen in the literature, there is a lack of comprehensive work that investigates the performances of various MCMC algorithms in their application of structural damage detection. In this study, the Differential Evolutionary Adaptive Metropolis (DREAM), a multi-chain MCMC, is explored and adapted to Bayesian model updating. This paper illustrates how DREAM is used for model updating with many uncertainty parameters (i.e., 40 parameters). Furthermore, the study provides a tutorial to users who may be less experienced with Bayesian model updating and MCMC. Two advanced single-chain MCMC algorithms, namely, the Delayed Rejection Adaptive Metropolis (DRAM) and Transitional Markov Chain Monte Carlo (TMCMC), and DREAM are elaborately introduced to allow practitioners to understand better the concepts and practical implementations. Their performances in model updating and damage detection are compared through three different engineering applications with increased complexity, e.g., a forty-story shear building, a two-span continuous steel beam, and a large-scale steel pedestrian bridge.

The Ideal Observer (IO) performance has been advocated when optimizing medical imaging systems for signal detection tasks. However, analytical computation of the IO test statistic is generally intractable. To approximate the IO test statistic, sampling-based methods that employ Markov-Chain Monte Carlo (MCMC) techniques have been developed. However, current applications of MCMC techniques have been limited to several object models such as a lumpy object model and a binary texture model, and it remains unclear how MCMC methods can be implemented with other more sophisticated object models. Deep learning methods that employ generative adversarial networks (GANs) hold great promise to learn stochastic object models (SOMs) from image data. In this study, we described a method to approximate the IO by applying MCMC techniques to SOMs learned by use of GANs. The proposed method can be employed with arbitrary object models that can be learned by use of GANs, thereby the domain of applicability of MCMC techniques for approximating the IO performance is extended. In this study, both signal-known-exactly (SKE) and signal-known-statistically (SKS) binary signal detection tasks are considered. The IO performance computed by the proposed method is compared to that computed by the conventional MCMC method. The advantages of the proposed method are discussed.

In vitro studies and mathematical models are now being widely used to study the underlying mechanisms driving the expansion of cell colonies. This can improve our understanding of cancer formation and progression. Although much progress has been made in terms of developing and analysing mathematical models, far less progress has been made in terms of understanding how to estimate model parameters using experimental in vitro image-based data. To address this issue, a new approximate Bayesian computation (ABC) algorithm is proposed to estimate key parameters governing the expansion of melanoma cell (MM127) colonies, including cell diffusivity, D, cell proliferation rate, λ, and cell-to-cell adhesion, q, in two experimental scenarios, namely with and without a chemical treatment to suppress cell proliferation. Even when little prior biological knowledge about the parameters is assumed, all parameters are precisely inferred with a small posterior coefficient of variation, approximately 2–12%. The ABC analyses reveal that the posterior distributions of D and q depend on the experimental elapsed time, whereas the posterior distribution of λ does not. The posterior mean values of D and q are in the ranges 226–268 µm2h−1, 311–351 µm2h−1 and 0.23–0.39, 0.32–0.61 for the experimental periods of 0–24 h and 24–48 h, respectively. Furthermore, we found that the posterior distribution of q also depends on the initial cell density, whereas the posterior distributions of D and λ do not. The ABC approach also enables information from the two experiments to be combined, resulting in greater precision for all estimates of D and λ.

Parameter estimation and uncertainty analysis for a watershed model

Population sex ratios are of high ecological relevance, but are challenging to determine in species lacking conspicuous external cues indicating their sex. Acoustic sexing is an option if vocalizations differ between sexes, but is precluded by overlapping distributions of the values of male and female vocalizations in many species. A method allowing the inference of sex ratios despite such an overlap will therefore greatly increase the information extractable from acoustic data. To meet this demand, we developed a novel approach using Approximate Bayesian Computation (ABC) to infer the sex ratio of populations from acoustic data. Additionally, parameters characterizing the male and female distribution of acoustic values (mean and standard deviation) are inferred. This information is then used to probabilistically assign a sex to a single acoustic signal. We furthermore develop a simpler means of sex ratio estimation based on the exclusion of calls from the overlap zone. Applying our methods to simulated data demonstrates that sex ratio and acoustic parameter characteristics of males and females are reliably inferred by the ABC approach. Applying both the ABC and the exclusion method to empirical datasets (echolocation calls recorded in colonies of lesser horseshoe bats, Rhinolophus hipposideros) provides similar sex ratios as molecular sexing. Our methods aim to facilitate evidence-based conservation, and to benefit scientists investigating ecological or conservation questions related to sex- or group specific behaviour across a wide range of organisms emitting acoustic signals. The developed methodology is non-invasive, low-cost and time-efficient, thus allowing the study of many sites and individuals. We provide an R-script for the easy application of the method and discuss potential future extensions and fields of applications. The script can be easily adapted to account for numerous biological systems by adjusting the type and number of groups to be distinguished (e.g. age, social rank, cryptic species) and the acoustic parameters investigated.

This article estimates technical and allocative inefficiencies and increase in costs therefrom of individual firms using a translog cost system consisting of the cost function and the cost share equations. We call it a nonlinear random-effects system because technical and allocative inefficiencies are random (hence the term random effects) and are separated from the random noise terms appearing in each equation of the system, and because the inefficiency terms appear in the system in a highly nonlinear fashion, which helps in separating them from random errors. We use Bayesian inference procedures based on Markov chain Monte Carlo (MCMC) techniques to estimate the proposed system. Inferences on firm-specific technical inefficiency and both input-specific and firm-specific allocative inefficiencies are developed using MCMC techniques. We apply the new methods to a sample of 500 U.S. commercial banks. We focus on input allocation problem based on the assumption that banks minimize cost. Empirical results show that cost for the top (bottom) 10% of banks is increased by at least 3% (11%) due to technical inefficiency. In contrast, very few banks are found to be efficient in allocating all the inputs. Costs are increased by 13% on average due to input misallocation. Increase in costs due to both technical and allocative inefficiencies for the top (bottom) 10% of the banks is at least 11% (29%). When translated into dollar figures, this result indicates that elimination of technical and allocative inefficiencies would save the top (bottom) 10% of the banks more than $$.20 ($$3.56) million. We also find that none of the banks in our sample exceeded the efficient scale size, although most of them were operating near their optimum scale (unitary returns to scale).

Using the standard one-zone synchrotron self-Compton model and the Markov chain Monte Carlo (MCMC) technique, we systematically analyse the quasi-simultaneous multiwavelength spectral energy distributions (SEDs) of Mrk 421 and Mrk 501 during states of relatively low activity. With this model in place, a semi-analytical method is developed to examine the uncertainty in jet power estimation caused by degeneracy of the radiative models. The semi-analytical method, in combination with the MCMC technique, allows us to explore the jet properties over a wide range of the variability time-scale. Our results seem to support the following. (i) In both Mrk 421 and Mrk 501, the jets are powered by rapidly rotating black holes. The black hole spin in Mrk 501 could be lower than that in Mrk 421 or possibly they are equal under the assumption of the Blandford–Znajek mechanism. (ii) The energy losses, which could be used to form the large-scale radio structure, are important for reconciling the differences of the kinetic power derived from the observations of the large-scale structure and the SED fitting results. Moreover, the jet energy losses in the propagation are more significant for Mrk 501 than for Mrk 421.

Group Importance Sampling for particle filtering and MCMC

The problem of identifying single degree-of-freedom (SDOF) nonlinear mechanical oscillators with piecewise-linear (PWL) restoring forces is considered. PWL nonlinear systems are a class of models that specify or approximate nonlinear systems via a set of locally-linear maps, each defined over different operating regions. They are useful in modelling hybrid phenomena common in practical situations, such as, systems with different modes of operation, or systems whose dynamics change because of physical limits or thresholds. However, identifying PWL models can be a challenging task when the number of operating regions and their partitions are unknown. This paper formulates the identification of oscillators with PWL restoring forces as a task of concurrent model selection and parameter estimation, where the selection of the number of linear regions is treated as a model selection task and identifying the associated system parameters as a task of parameter estimation. In this study, PWL maps in restoring forces with up to four regions are considered, and the task of model selection and parameter estimation task is addressed in a Bayesian framework. A likelihood-free Approximate Bayesian Computation (ABC) scheme is followed, which is easy to implement and provides a simplified way of doing model selection. The proposed approach has been demonstrated using two numerical examples and an experimental study, where ABC has been used to select models and identify parameters from among four SDOF PWL systems with different number of PWL regions. The results demonstrate the flexibility of using the proposed Bayesian approach for identifying the correct model and parameters of PWL systems, in addition to furnishing uncertainty estimates of the identified parameters.