Identifiability analysis for stochastic differential equation models in systems biology.

Event Abstract Back to Event Stochastic modeling of neuronal signaling Jukka Intosalmi1*, Tiina Manninen1, Keijo Ruohonen1 and Marja-Leena Linne1 1 Tampere University of Technology, Finland The time evolution of biochemical systems in neurons is traditionally modeled using deterministic ordinary differential equations (ODEs). Chemical reactions, however, are random in nature, and the deterministic approach is valid only for a restricted class of systems. Stochastic models take random fluctuations into account and are thus more realistic. Biochemical reactions can be modeled stochastically using numerous different methods. An ideal model would have the following three important properties. First, the model should be as realistic as possible, second, the mathematical method should be easily implementable as a computer algorithm, and third, the algorithm should be computationally effective. Naturally, all these three conditions cannot be fulfilled at the same time. Some realistic modeling approaches can be derived directly from chemical kinetics without making any approximations. Such approaches are called exact. A good example of an exact modeling approach is the stochastic simulation algorithm (SSA) developed by Gillespie. The SSA is applicable when the molecular populations in the system are small, but it becomes computationally inefficient when the numbers of molecules increase. In order to construct stochastic models that can be effectively simulated, new mathematical approaches have to be explored. As an approximate method also stochastic differential equations (SDEs) have been considered a promising way to model biochemical reactions stochastically. The SDE approach is attractive especially if we consider a system for which the SSA is computationally inefficient and the traditional deterministic ODE approach cannot be used as a good approximation. SDE models treat the chemical populations as real numbers and the construction of the model is based on the law of mass action, similarly as the construction of the traditional deterministic modeling approach. The SDE model, however, takes the random fluctuations into account by describing the time evolution of the system with the stochastic Itô process instead of the deterministic set of ODEs. The Itô process is basically a set of coupled stochastic differential equations. Although SDE modeling usually leads to equations that cannot be solved analytically, the solutions can be approximated numerically using different numerical integration methods. Thus, SDEs provide a mathematically rigorous way to construct models that can be simulated effectively. In order to investigate how well the SDE approach models biochemical systems, the results have to be compared with experimental data or with the simulation results from some exact simulation procedure. In this study, we use the methodology of spectral analysis to compare the simulation results of the SDE model and the SSA. As case studies, we consider e.g. calcium binding and protein kinase C (PKC) signal transduction pathway. We investigate the nature of noise in different modeling approaches and try to interpret the meaning of the noise in real biological systems. The main goal of our study is to find the most essential components and parameters in the SDE model and to adjust them so that the model is capable of giving similar results as the SSA. The simulations are carried out using the distributed computing resources (GRID) provided by Techila Technologies Ltd. Conference: Neuroinformatics 2008, Stockholm, Sweden, 7 Sep - 9 Sep, 2008. Presentation Type: Poster Presentation Topic: Computational Neuroscience Citation: Intosalmi J, Manninen T, Ruohonen K and Linne M (2008). Stochastic modeling of neuronal signaling. Front. Neuroinform. Conference Abstract: Neuroinformatics 2008. doi: 10.3389/conf.neuro.11.2008.01.023 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 28 Jul 2008; Published Online: 28 Jul 2008. * Correspondence: Jukka Intosalmi, Tampere University of Technology, Tampere, Finland, jukka.intosalmi@tut.fi Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Jukka Intosalmi Tiina Manninen Keijo Ruohonen Marja-Leena Linne Google Jukka Intosalmi Tiina Manninen Keijo Ruohonen Marja-Leena Linne Google Scholar Jukka Intosalmi Tiina Manninen Keijo Ruohonen Marja-Leena Linne PubMed Jukka Intosalmi Tiina Manninen Keijo Ruohonen Marja-Leena Linne Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Parameter and uncertainty estimation in stochastic differential equation models with multi-rate data and nonstationary disturbances

For system identification, the ordinary differential equations (ODEs) model is popular for its accuracy and effectiveness. Consequently, the ODEs model is extended to the stochastic differential equations (SDEs) model to tackle the stochastic case intuitively. But the existence of stochastic integral is a rigid barrier. We simply transform the SDEs to their corresponding stochastic difference equations (SDCEs) to eliminate stochastic integrals and propose an easy but effective solution to stochastic system identification. In this solution, the maximum likelihood estimation can be applied and the evolutionary algorithms are used to determine structures and parameters of the unknown system.

Stochastic differential equation (SDE) models are the foundation for pricing and hedging financial derivatives. The drift and volatility functions in SDE models are typically chosen to be algebraic functions with a small number ( < 5 ) of parameters which can be calibrated to market data. A more flexible approach is to use neural networks to model the drift and volatility functions, which provides more degrees-of-freedom to match observed market data. Training of models requires optimizing over an SDE, which is computationally challenging. For European options, we develop a fast stochastic gradient descent (SGD) algorithm for training the neural network-SDE model. Our SGD algorithm uses two independent SDE paths to obtain an unbiased estimate of the direction of steepest descent. For American options, we optimize over the corresponding Kolmogorov partial differential equation (PDE). The neural network appears as coefficient functions in the PDE. Models are trained on large datasets (many contracts), requiring either large simulations (many Monte Carlo samples for the stock price paths) or large numbers of PDEs (a PDE must be solved for each contract). Numerical results are presented for real market data including S&P 500 index options, S&P 100 index options, and single-stock American options. The neural-network-based SDE models are compared against the Black-Scholes model, the Dupire's local volatility model, and the Heston model. Models are evaluated in terms of how accurate they are at pricing out-of-sample financial derivatives, which is a core task in derivative pricing at financial institutions. Specifically, we calibrate a neural network-SDE model to market data for a financial derivative on an asset with price S t with a payoff function g ( s ) , and we then evaluate its generalization accuracy for a financial derivative on the same asset S t but with a different payoff function f ( s ) . In addition to comparing out-of-sample pricing accuracy, we evaluate the hedging performance of the neural network-SDE model.

Stochastic Differential Equation (SDE) models are often used to model the dynamics of complex biological systems. The stochastic nature of these models means that some behaviors are more likely than others. It is often the case that a model's primary purpose is to study rare but interesting or important behaviors, such as the formation of a tumor, or the failure of a cyber-physical system. Unfortunately, due to the limited availability of analytic methods for SDEs, stochastic simulations are the most common means for estimating (or bounding) the probability of rare behaviors. Naturally, the cost of stochastic simulations increases with the rarity of the behavior under consideration. To address this problem, we introduce a new algorithm, RESERCHE, that is specifically designed to quantify the likelihood of rare but interesting behaviors in SDE models. Our approach relies on the use of temporal logics for specifying rare behaviors of possible interest, and on the ability of bit-vector decision procedures to reason exhaustively about fixed precision arithmetic. We also compute the probability of an observed behavior under the assumption of Gaussian noise.

In this study, we propose a new method that is useful for estimating unknown parameter values of stochastic differential equation (SDE) models, based on probability density function (PDF) data measured from random dynamical systems. As our method does not require explicit description of PDF, it can be applied to the SDE models even when their PDFs are hardly derived in explicit forms due to multiplicative-noise terms, nonlinear terms, and so on. Therefore, our method is expected to provide a versatile tool to dynamically parameterize measured PDF data. In our proposed method, it is assumed that a measured PDF is obtained from a random dynamical system whose structure is described by a known SDE model with unknown parameter values. With the help of It\^o calculus, the Fokker-Planck equation (FPE) is derived from the SDE model. The measured PDF and a candidate of parameter values are substituted into the FPE to calculate a FPE residual. Our method is applied to two random vibration systems. Their FPE residuals tend to zero as the parameter values tend to exact values, showing that our proposed FPE residual can be utilized for unknown parameter estimation of SDE models.

Background: Growth trajectories are highly variable between children, making epidemiological analyses challenging both to the identification of malnutrition interventions at the population level and also risk assessment at individual level. We introduce stochastic differential equation (SDE) models into child growth research. SDEs describe flexible dynamic processes comprising: drift - gradual smooth changes – such as physiology or gut microbiome, and diffusion - sudden perturbations, such as illness or infection. Methods: We present a case study applying SDE models to child growth trajectory data from the Haydom, Tanzania and Venda, South Africa sites within the MAL-ED cohort. These data comprise n=460 children aged 0-24 months. A comparison with classical curve fitting (linear mixed models) is also presented. Results: The SDE models offered a wide range of new flexible shapes and parameterizations compared to classical additive models, with performance as good or better than standard approaches. The predictions from the SDE models suggest distinct longitudinal clusters that form distinct ‘streams’ hidden by the large between-child variability. Conclusions: Using SDE models to predict future growth trajectories revealed new insights in the observed data, where trajectories appear to cluster together in bands, which may have a future risk assessment application. SDEs offer an attractive approach for child growth modelling and potentially offer new insights.

The problem of computing the maximum likelihood estimate of the parameters of a specific class of stochastic differential equation (SDE) models with linear drift whose sample paths are observed at discrete time points is considered. This estimate is obtained as in Cleur and Manfredi (1999) by discretizing the explicit expressions for the estimates which maximize the likelihood function in continuous time, by discretizing the likelihood function through a quadrature approximation before maximizing it, and by maximizing the likelihood function of the Euler scheme approximation to the underlying continuous process. Simulation results indicate that, for the constellation of parameter values considered, all three approaches lead to very similar results.

This work focuses on modeling of film porosity in thin film deposition using stochastic differential equations. A deposition process is modeled via kinetic Monte Carlo (kMC) simulation on a triangular lattice. The microscopic process events involve atom adsorption and migration and the film growth allows for vacancies and overhangs to develop inside the film. Appropriate definitions of film site occupancy ratio (SOR), i.e., fraction of film sites occupied by particles over total number of film sites, and its fluctuation are introduced to describe film porosity. Deterministic and stochastic ordinary differential equation (ODE) models are also derived to describe the time evolution of film SOR and its fluctuation. The coefficients of the ODE models are estimated on the basis of data obtained from the kMC simulator of the deposition process using least-square methods. Simulation results demonstrate the applicability and effectiveness of the proposed film porosity modeling methods in the context of the deposition process under consideration.

The Golgi apparatus plays a central role in processing and sorting proteins and lipids in eukaryotic cells. Golgi compartments constantly exchange material with each other and with other cellular components, allowing them to maintain and reform distinct identities despite dramatic changes in structure and size during cell division, development, and osmotic stress. We have developed three minimal models of membrane and protein exchange in the Golgi-a discrete, stochastic model, a continuous ordinary differential equation model, and a continuous stochastic differential equation model-each based on two fundamental mechanisms: vesicle-coat-mediated selective concentration of cargoes and soluble N-ethylmaleimide-sensitive factor attachment protein receptor (SNARE) proteins during vesicle formation and SNARE-mediated selective fusion of vesicles. By exploring where the models differ, we hope to discover whether the discrete, stochastic nature of vesicle-mediated transport is likely to have appreciable functional consequences for the Golgi. All three models show similar ability to restore and maintain distinct identities over broad parameter ranges. They diverge, however, in conditions corresponding to collapse and reassembly of the Golgi. The results suggest that a continuum model provides a good description of Golgi maintenance but that considering the discrete nature of vesicle-based traffic is important to understanding assembly and disassembly of the Golgi. Experimental analysis validates a prediction of the models that altering guanine nucleotide exchange factor expression levels will modulate Golgi size.

The tenth special issue of the Japanese Journal of Statistics and Data Science (JJSD) focuses on statistics for stochastic processes. With the advancement of information and communication technology and measurement technology, a vast amount of diverse time series data have been readily obtained. Consequently, the significance of stochastic processes as mathematical models for random phenomena evolving in time or space has increased dramatically, and statistics for various types of stochastic processes have been rapidly developing. In fact, a multitude of studies on statistics for stochastic processes such as ARMA, CARMA, COGARCH, diffusion processes, fractional Brownian motions, GARCH, jump-diffusion processes, Lèvy processes, Markov processes, point processes, random fields, stochastic differential equations and stochastic partial differential equations have been conducted. Nonetheless, because of the mathematical complexity of statistics for stochastic processes, numerous theoretical and computational challenges remain to be addressed. This special feature contains ten original papers contributed by leading mathematical statisticians and data scientists on current topics in statistics for stochastic processes. The first paper examines the efficient particle Markov chain Monte Carlo method. The second and third papers respectively address the estimation problems of Hawkes processes and Cox proportional excess hazard models. The fourth and fifth papers investigate parametric estimation for probability partial differential equation models. The latter five papers focus on statistical inference for stochastic differential equation models driven by various stochastic processes, including the Wiener process, fractional Brownian motion, β-stable process and Lèvy process. The ten papers in the special feature are summarized as follows: Awaya and Omori propose an efficient particle Markov chain Monte Carlo methodology for the rolling-window estimation of state space models. They provide theoretical justifications for their proposed method and demonstrate the efficiency of B Masayuki Uchida

It is shown how different but equivalent Itô stochastic differential equation models of random dynamical systems can be constructed. Advantages and disadvantages of the different models are described. Stochastic differential equation models are derived for problems in chemistry, textile engineering, and epidemiology. Computational comparisons are made between the different stochastic models.

Neurons in the brain express intrinsic dynamic behavior which is known to be stochastic in nature. A crucial question in building models of neuronal excitability is how to be able to mimic the dynamic behavior of the biological counterpart accurately and how to perform simulations in the fastest possible way. The well-established Hodgkin-Huxley formalism has formed to a large extent the basis for building biophysically and anatomically detailed models of neurons. However, the deterministic Hodgkin-Huxley formalism does not take into account the stochastic behavior of voltage-dependent ion channels. Ion channel stochasticity is shown to be important in adjusting the transmembrane voltage dynamics at or close to the threshold of action potential firing, at the very least in small neurons. In order to achieve a better understanding of the dynamic behavior of a neuron, a new modeling and simulation approach based on stochastic differential equations and Brownian motion is developed. The basis of the work is a deterministic one-compartmental multi-conductance model of the cerebellar granule cell. This model includes six different types of voltage-dependent conductances described by Hodgkin-Huxley formalism and simple calcium dynamics. A new model for the granule cell is developed by incorporating stochasticity inherently present in the ion channel function into the gating variables of conductances. With the new stochastic model, the irregular electrophysiological activity of an in vitro granule cell is reproduced accurately, with the same parameter values for which the membrane potential of the original deterministic model exhibits regular behavior. The irregular electrophysiological activity includes experimentally observed random subthreshold oscillations, occasional spontaneous spikes, and clusters of action potentials. As a conclusion, the new stochastic differential equation model of the cerebellar granule cell excitability is found to expand the range of dynamics in comparison to the original deterministic model. Inclusion of stochastic elements in the operation of voltage-dependent conductances should thus be emphasized more in modeling the dynamic behavior of small neurons. Furthermore, the presented approach is valuable in providing faster computation times compared to the Markov chain type of modeling approaches and more sophisticated theoretical analysis tools compared to previously presented stochastic modeling approaches.

We study theoretical foundation of model comparison for ergodic stochastic differential equation (SDE) models and an extension of the applicable scope of the conventional Bayesian information criterion. Different from previous studies, we suppose that the candidate models are possibly misspecified models, and we consider both Wiener and a pure‐jump Lévy noise‐driven SDE. Based on the asymptotic behavior of the marginal quasi‐log likelihood, the Schwarz‐type statistics and stepwise model selection procedure are proposed. We also prove the model selection consistency of the proposed statistics with respect to an optimal model. We conduct some numerical experiments and they support our theoretical findings.

Reliable forecasting of wind power generation is crucial to optimal control of costs in generation of electricity with respect to the electricity demand. Here, we propose and analyze stochastic wind power forecast models described by parametrized stochastic differential equations, which introduce appropriate fluctuations in numerical forecast outputs. We use an approximate maximum likelihood method to infer the model parameters taking into account the time correlated sets of data. Furthermore, we study the validity and sensitivity of the parameters for each model. We applied our models to Uruguayan wind power production as determined by historical data and corresponding numerical forecasts for the period of March 1 to May 31, 2016.