Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis

Ordinary differential equation (ODE) models are quite popular for modelling complex dynamic processes in many scientific fields, and the parameters in these models are usually unknown, and we need to estimate them using statistical methods. When some observations are contaminated, regular estimation methods, such as nonlinear least-square estimation, will bring large bias. In this paper, robust estimations of both constant and time-varying parameters in ODE models using M-estimators are proposed, and their asymptotic properties are obtained under some mild conditions. We focus on Huber M-estimator, and also provide a method to adjust the Huber parameter automatically to the observations. The proposed method is compared to existing methods in numerical simulations and CD8+ T cell data analysis. It is demonstrated that our method gain substantial efficiency as well as robust properties.

Experimental results have shown that anti-cancer therapies, such as radiotherapy and chemotherapy, can modulate the cell cycle and generate cell cycle phase-dependent responses. As a result, obtaining a detailed understanding of the cell cycle is one possible path towards improving the efficacy of many of these therapies. Here, we consider a basic structured partial differential equation (PDE) model for cell progression through the cell cycle, and derive expressions for key quantities, such as the population growth rate and cell phase proportions. These quantities are shown to be periodic and, as such, we compare the PDE model to a corresponding ordinary differential equation (ODE) model in which the parameters are linked by ensuring that the long-term ODE behaviour agrees with the average PDE behaviour. By design, we find that the ODE model does an excellent job of representing the mean dynamics of the PDE model within just a few cell cycles. However, by probing the parameter space we find cases in which this mean behaviour is not a good measure of the PDE population growth. Our analytical comparison of two caricature models (one PDE and one ODE system) provides insight into cases in which the simple ODE model is an appropriate approximation to the PDE model.

Modeling and analysis of complex systems are important aspects of understanding systemic behavior. In the lack of detailed knowledge about a system, we often choose modeling equations out of convenience and search the (high-dimensional) parameter space randomly to learn about model properties. Qualitative modeling sidesteps the issue of choosing specific modeling equations and frees the inference from specific properties of the equations. We consider classes of ordinary differential equation (ODE) models arising from interactions of species/entities, such as (bio)chemical reaction networks or ecosystems. A class is defined by imposing mild assumptions on the interaction rates. In this framework, we investigate whether there can be multiple positive steady states in some ODE models in a given class. We have developed and implemented a method to decide whether any ODE model in a given class cannot have multiple steady states. The method runs efficiently on models of moderate size. We tested the method on a large set of models for gene silencing by sRNA interference and on two publicly available databases of biological models, KEGG and Biomodels. We recommend that this method is used as (i) a pre-screening step for selecting an appropriate model and (ii) for investigating the robustness of non-existence of multiple steady state for a given ODE model with respect to variation in interaction rates. Scripts and examples in Maple are available in the Supplementary Information. wiuf@math.ku.dk Supplementary data are available at Bioinformatics online.

For system identification, the ordinary differential equations (ODEs) model is popular for its accuracy and effectiveness. Consequently, the ODEs model is extended to the stochastic differential equations (SDEs) model to tackle the stochastic case intuitively. But the existence of stochastic integral is a rigid barrier. We simply transform the SDEs to their corresponding stochastic difference equations (SDCEs) to eliminate stochastic integrals and propose an easy but effective solution to stochastic system identification. In this solution, the maximum likelihood estimation can be applied and the evolutionary algorithms are used to determine structures and parameters of the unknown system.

US high school (HS) students are lagging behind their foreign peers in math performance. It is thus necessary to design teaching modules that motivate US HS students to learn more math and to improve their math skills. Our study set out to test two different software programs designed for the development of such skills. MATLAB Simulink provides a friendly interface with which HS students can build ordinary differential equation (ODE) models. An alternative programming language, the R language is free and extensively used in the field of statistical learning and engineering. In this study, a group of HS students first got trained in both Simulink and R and then solved an ODE model of a microbial fuel cell (MFC), which can convert the organic compounds found in waste water into electricity, using both Simulink and R. Three anonymous surveys were conducted for: 1) students’ feedback on the module-based teaching approaches with R and Simulink training, 2) students’ self-evaluation in the improvement of their skills in solving ODE models following their training in R and Simulink, and 3) students’ comparison of R and Simulink in their learning and modeling experience. The instructor also evaluated students’ modeling skills with R and Simulink based on their performance in developing the MFC ODE model. Both students’ self-evaluation and the instructor’s direct evaluation indicate that students’ skills in solving ODE models with both R and Simulink were significantly improved after this R-Simulink modeling project. The surveys show that students generally enjoyed the module-based teaching format and that they would recommended this project to their peers. It was shown in the surveys that students preferred Simulink over R in solving ODE models and that they preferred learning Simulink first and then following up by learning R. Students’ comments for opening questions indicate that the major hurdle in this project was their limited background knowledge in biology and electrochemistry for microbial fuel cells.

SummaryThis study addresses control‐oriented modeling and control design of tensegrity–membrane systems. Lagrange's method is used to develop a control‐oriented model for a generic system. The equations of motion are expressed as a set of differential‐algebraic equations (DAEs). For control design, the DAEs are converted into second‐order ordinary differential equations (ODEs) based on coordinate partitioning and coordinate mapping. Because the number of inputs is less than the number of state variables, the system belongs to the class of underactuated nonlinear systems. A nonlinear adaptive controller based on the collocated partial feedback linearization (PFL) technique is designed for system deployment. The stability of the closed‐loop system for the actuated coordinates is studied using the Lyapunov stability theory. Because of system complexity, numerical tests are used to conduct stability analysis for the dynamics of the underactuated coordinates, which represents the system's zero dynamics. For the tensegrity–membrane systems studied in this work, analytical proof of zero dynamics stability remains an open theoretical problem. An H∞ controller is implemented for rapid stabilization of the system at the final deployed configuration. Simulations are conducted to test the performance of the two controllers. The simulation results are presented and discussed in detail. Copyright © 2016 John Wiley & Sons, Ltd.

Vector-borne diseases provide unique challenges to public health because the epidemiology is so closely tied to external environmental factors such as climate, landscape, and population migration, as well as the complicated biology of vector-transmitted pathogens. In particular, this close link between the epidemiology, the environment, and pathogen biology means that the traditional view that many vector-borne diseases are relatively stable in numerous regions does not provide a complete picture of their complexity. In fact, several regions exist with low levels of endemicity most of the time, punctuated by severe, often explosive, epidemics. These regions are considered unstable transmission settings. Ordinary differential equation (ODE) models have thus far dominated the study of vector-borne disease and have provided considerable insight into our understanding of transmission and effective control in stable transmission settings. To address the short-­comings of autonomous ODE models, we present a class of models, differential-delay equation (DDE) models, that have the potential to better describe unstable endemic settings for vector-borne disease. These models develop naturally out of the biology of diseases transmitted by vectors because of the extrinsic and intrinsic incubation periods and vector maturation process necessary for successful transmission of vector-transmitted pathogens. In this chapter, we introduce five examples of vector-borne diseases that span the globe, and discuss the clinical implications of unstable transmission of these diseases. Next, we present the original ODE version of the Ross-Macdonald model for vector-borne diseases, modify this model by introducing different types of naturally occurring delays, then illustrate how these models can exhibit more complex behavior such as oscillations via Hopf bifurcation and chaos via period-doubling, that the ODE model cannot produce. Finally, we explore the possibility for delay-models to contribute to our understanding of unstable transmission settings, which in turn will inform the development of effective control strategies for these epidemic-prone regions.

Kalman Filtering is a computational tool with widespread applications in robotics, financial and weather forecasting, environmental engineering and defense. Given observation and state transition models, the Kalman Filter (KF) recursively estimates the state variables of a dynamic system. However, the KF requires a cubic time matrix inversion operation at every timestep which prevents its application in domains with large numbers of state variables. We propose Relational Gaussian Models to represent and model dynamic systems with large numbers of variables efficiently. Furthermore, we devise an exact lifted Kalman Filtering algorithm which takes only linear time in the number of random variables at every timestep. We prove that our algorithm takes linear time in the number of state variables even when individual observations apply to each variable. To our knowledge, this is the first lifted (linear time) algorithm for filtering with continuous dynamic relational models.

We find concrete evidence for a recently discovered form of intermittency, referred to as in–out intermittency, in both partial differential equation (PDE) and ordinary differential equation (ODE) models of mean field dynamos. This type of intermittency [introduced in P. Ashwin, E. Covas, and R. Tavakol, Nonlinearity 9, 563 (1999)] occurs in systems with invariant submanifolds and, as opposed to on–off intermittency which can also occur in skew product systems, it requires an absence of skew product structure. By this we mean that the dynamics on the attractor intermittent to the invariant manifold cannot be expressed simply as the dynamics on the invariant subspace forcing the transverse dynamics; the transverse dynamics will alter that tangential to the invariant subspace when one is far enough away from the invariant manifold. Since general systems with invariant submanifolds are not likely to have skew product structure, this type of behavior may be of physical relevance in a variety of dynamical settings. The models employed here to demonstrate in–out intermittency are axisymmetric mean-field dynamo models which are often used to study the observed large-scale magnetic variability in the Sun and solar-type stars. The occurrence of this type of intermittency in such models may be of interest in understanding some aspects of such variabilities.

Parameter estimation and associated uncertainty quantification is an important problem in dynamical systems characterised by ordinary differential equation (ODE) models that are often nonlinear. Typically, such models have analytically intractable trajectories which result in likelihoods and posterior distributions that are similarly intractable. Bayesian inference for ODE systems via simulation methods require numerical approximations to produce inference with high accuracy at a cost of heavy computational power and slow convergence. At the same time, Artificial Neural Networks (ANN) offer tractability that can be utilized to construct an approximate but tractable likelihood and posterior distribution. In this paper we propose a hybrid approach, where Laplace-based Bayesian inference is combined with an ANN architecture for obtaining approximations to the ODE trajectories as a function of the unknown initial values and system parameters. Suitable choices of customized loss functions are proposed to fine tune the approximated ODE trajectories and the subsequent Laplace approximation procedure. The effectiveness of our proposed methods is demonstrated using an epidemiological system with non-analytical solutions—the Susceptible-Infectious-Removed (SIR) model for infectious diseases—based on simulated and real-life influenza datasets. The novelty and attractiveness of our proposed approach include (i) a new development of Bayesian inference using ANN architectures for ODE based dynamical systems, and (ii) a computationally fast posterior inference by avoiding convergence issues of benchmark Markov Chain Monte Carlo methods. These two features establish the developed approach as an accurate alternative to traditional Bayesian computational methods, with improved computational cost.

A system of recurrent neural networks for modularising, parameterising and dynamic analysis of cell signalling networks

BackgroundParameter estimation is often the bottlenecking step in biological system modeling. For ordinary differential equation (ODE) models, the challenge in this estimation has been attributed to not only the lack of parameter identifiability, but also computational issues such as finding globally optimal parameter estimates over highly multidimensional search space. Recent methods using incremental estimation approach could alleviate the computational difficulty by performing the parameter estimation one-reaction-at-a-time. However, incremental estimation strategies usually require data smoothing and are known to produce biased parameter estimates.ResultsIn this article, we presented a new parameter estimation method called integrated flux parameter estimation (IFPE). We employed the integral form of the ODE such that we could compute the integral of reaction fluxes from time-series concentration data without data smoothing. Here, we formulated the parameter estimation as a nested optimization problem. In the outer optimization, we performed a minimization of model prediction errors over parameters associated with a subset of reactions labeled as independent. The dimension of the independent reaction subset was equal to the degrees of freedom in the calculation of integrated fluxes (IF) from concentration data. We selected the independent reactions such that given their IF values, the IFs of the remaining (dependent) reactions could be uniquely determined. Meanwhile, in the inner optimization, we estimated the model parameters associated with the dependent reactions, one-reaction-at-a-time, by minimizing the dependent IF prediction errors. We demonstrated the performance of the IFPE method using two case studies: a generalized mass action model of a branched pathway and a lin-log ODE model of Lactococcus lactis glycolytic pathway.ConclusionsThe IFPE significantly outperformed standard simultaneous parameter estimation in terms of computational efficiency and scaling. In comparison to incremental parameter estimation (IPE) method, the IFPE produced parameter estimates with significantly lower bias and did not require time-series data smoothing. The advantages of IFPE over the IPE however came at the cost of a small increase in the computational time.Electronic supplementary materialThe online version of this article (doi:10.1186/s12918-014-0127-x) contains supplementary material, which is available to authorized users.

Dynamic process modeling is essential for simulating time‐evolving biochemical systems, particularly those with multistate interactions and combinatorial complexity. Traditional Ordinary Differential Equation (ODE) models offer mechanistic clarity but struggle with scalability and context‐sensitive encoding. Rule‐Based Modeling (RBM) frameworks address these limitations through modular rule abstraction, yet require manual specification and lack adaptive learning. This study introduces algorithmic innovations within the Neural Ordinary Differential Equation (Neural ODE) paradigm to bridge the gap between mechanistic interpretability and scalable expressivity. Neural ODEs can be considered as a revolutionary approach in the field of modeling dynamic biochemical interactions. They have made it possible to create models of such interactions that are flexible enough to adapt to different scenarios and do so without requiring any manual intervention in terms of rule encoding or predefined reaction schemes. This is achieved by employing differential solvers within the framework of neural networks, thus enabling a learning process that is in accordance with the behavior of the system. Using the DARPP‐32 signaling network—a benchmark system characterized by multivalent phosphorylation and dynamic perturbations—the proposed Neural ODE framework demonstrates the ability to replicate key dynamic behaviors observed in ODE and RBM models. Comparative simulations under baseline and perturbed conditions reveal that Neural ODEs maintain trajectory fidelity while offering enhanced modularity and computational efficiency. Feature importance analysis and latent space visualizations further validate the model's interpretability and robustness. Unlike ODEs and RBMs, Neural ODEs adapt to structural mutations and binding schemes through latent trajectory learning, enabling flexible simulation of biochemical variability without manual rule encoding. This work establishes Neural ODEs as a viable and scalable alternative for modeling complex biochemical systems, combining the strengths of data‐driven learning with the interpretability of differential equations.

We study the presence of social communities in mobility traces from vehicular fleets. By analyzing publicly available sets of fleet vehicle mobility traces obtained from two real-world deployments — consisting of more than 2000 taxis in Shanghai and Beijing respectively, we confirm the existence of small numbers of distinct social communities in vehicular networks, which is in direct contrast to the general belief that vehicular networks are best modeled as a relatively homogeneous system. We examine the spatio-temporal characteristics of social communities, gaining the insight that they are driven primarily by social proximity induced by geographic locality. We then develop a parsimonious multi-community ordinary differential equation (ODE) model, which uses the heterogeneous structure introduced by social communities to model information dissemination. We show through simulations that this approach dramatically outperforms the conventional homogeneous ODE model in capturing the dynamics of the dissemination process. We further demonstrate that the use of the ODE model to optimize seeding of an initial set of vehicles results in improved utility for information dissemination compared to seed-optimization using a homogeneous model.

An aggregation method is proposed that transforms the multiple DFIGs into an equivalent DFIG model that retains the major collective dynamic characteristics of a group of DFIGs. It is intended for Electromagnetic Transients Simulation (EMT). The aggregated machine can take into account different speeds and parameters of each of the individual DFIGs and the connecting impedance of individual DFIGs. Starting with a State Variable (SV) model of an individual DFIG, aggregation is carried out recursively, by combining two DFIGs at a time and then reducing the order of the aggregate to match the state variable equations of a single DFIG so that the steady state performances are identical. Validation is carried out by comparing the detailed electromagnetic transient (EMT) simulation of the unreduced system with the reduced aggregate system. It is shown that the proposed aggregation method accurately matches the steady state response and also accurately reproduces the dominant transient response of the unreduced system. As the aggregate DFIG has the same number of state variables as a single DFIG, the overall wind farms order is reduced significantly to increase the modelling and simulation efficiency.