A Hybrid Sobolev Gradient Method for Learning NODEs

Dynamic process modeling is essential for simulating time‐evolving biochemical systems, particularly those with multistate interactions and combinatorial complexity. Traditional Ordinary Differential Equation (ODE) models offer mechanistic clarity but struggle with scalability and context‐sensitive encoding. Rule‐Based Modeling (RBM) frameworks address these limitations through modular rule abstraction, yet require manual specification and lack adaptive learning. This study introduces algorithmic innovations within the Neural Ordinary Differential Equation (Neural ODE) paradigm to bridge the gap between mechanistic interpretability and scalable expressivity. Neural ODEs can be considered as a revolutionary approach in the field of modeling dynamic biochemical interactions. They have made it possible to create models of such interactions that are flexible enough to adapt to different scenarios and do so without requiring any manual intervention in terms of rule encoding or predefined reaction schemes. This is achieved by employing differential solvers within the framework of neural networks, thus enabling a learning process that is in accordance with the behavior of the system. Using the DARPP‐32 signaling network—a benchmark system characterized by multivalent phosphorylation and dynamic perturbations—the proposed Neural ODE framework demonstrates the ability to replicate key dynamic behaviors observed in ODE and RBM models. Comparative simulations under baseline and perturbed conditions reveal that Neural ODEs maintain trajectory fidelity while offering enhanced modularity and computational efficiency. Feature importance analysis and latent space visualizations further validate the model's interpretability and robustness. Unlike ODEs and RBMs, Neural ODEs adapt to structural mutations and binding schemes through latent trajectory learning, enabling flexible simulation of biochemical variability without manual rule encoding. This work establishes Neural ODEs as a viable and scalable alternative for modeling complex biochemical systems, combining the strengths of data‐driven learning with the interpretability of differential equations.

Inferring ecological interactions is hard because we often lack suitable parametric representations to portray them. Neural ordinary differential equations (NODEs) provide a way of estimating interactions non‐parametrically from time‐series data. NODEs, however, are slow to fit, and inferred interactions usually are not compared with the ground truth. We provide a fast NODE fitting method, Bayesian neural gradient matching (BNGM), which relies on interpolating time series with neural networks and fitting NODEs to the interpolated dynamics with Bayesian regularisation. We test the accuracy of the approach by inferring ecological interactions in time series generated by an ODE model with known interactions. We compare these results against three existing approaches for estimating ecological interactions, standard NODEs, ODE models and convergent cross‐mapping (CCM). We also infer interactions in experimentally replicated time series of a microcosm featuring an algae, flagellate and rotifer population, in the hare and lynx system, and the Maizuru Bay community featuring 11 species. Our BNGM approach allows us to reduce the fitting time of NODE systems to only a few seconds and provides accurate estimates of ecological interactions in the artificial system, as true ecological interactions are recovered with minimal error. Our benchmark analysis reveals that our approach is both faster and more accurate than standard NODEs and parametric ODEs, while CCM was found to be faster but less accurate. The analysis of the replicated time series reveals that only the strongest interactions are consistent across replicates, while the analysis of the Maizuru community shows the strong negative impact of the chameleon goby on most species of the community, and a potential indirect negative effect of temperature by favouring goby population growth. Overall, NODEs alleviate the need for a mechanistic understanding of interactions, and BNGM alleviates the heavy computational cost. This is a crucial step availing quick NODE fitting to larger systems, cross‐validation and uncertainty quantification, as well as more objective estimation of interactions, and complex context dependence, than parametric models.

Measurement noise is an integral part of collecting data of a physical process. Thus, noise removal is necessary to draw conclusions from these data, and it often becomes essential to construct dynamical models using these data. We discuss a methodology to learn differential equation(s) using noisy and irregularly sampled measurements. In our methodology, the main innovation can be seen in the integration of deep neural networks with the neural ordinary differential equations (ODEs) approach. Precisely, we aim at learning a neural network that provides (approximately) an implicit representation of the data and an additional neural network that models the vector fields of the dependent variables. We combine these two networks by constraints using neural ODEs. The proposed framework to learn a model describing the vector field is highly effective under noisy measurements. The approach can handle scenarios where dependent variables are unavailable at the same temporal grid. Moreover, a particular structure, e.g. second order with respect to time, can easily be incorporated. We demonstrate the effectiveness of the proposed method for learning models using data obtained from various differential equations and present a comparison with the neural ODE method that does not make any special treatment to noise. Additionally, we discuss an ensemble approach to improve the performance of the proposed approach further.

Inferring the functional shape of ecological and evolutionary processes from time‐series data can be challenging because processes are often not describable with simple equations. The dynamical coupling between variables in time series further complicates the identification of equations through model selection as the inference of a given process is contingent on the accurate depiction of all other processes. We present a novel method, neural ordinary differential equations (NODEs), for learning ecological and evolutionary processes from time‐series data by modelling dynamical systems as ordinary differential equations and dynamical functions with artificial neural networks (ANNs). Upon successful training, the ANNs converge to functional shapes that best describe the biological processes underlying the dynamics observed, in a way that is robust to mathematical misspecifications of the dynamical model. We demonstrate NODEs in a population dynamic context and show how they can be used to infer ecological interactions, dynamical causation and equilibrium points. We tested NODEs by analysing well‐understood hare and lynx time‐series data, which revealed that prey–predator oscillations were mainly driven by the interspecific interaction, as well as intraspecific densitydependence, and characterised by a single equilibrium point at the centre of the oscillation. Our approach is applicable to any system that can be modelled with differential equations, and particularly suitable for linking ecological, evolutionary and environmental dynamics where parametric approaches are too challenging to implement, opening new avenues for theoretical and empirical investigations.

Galactic systems are inherently multiphase, and understanding the roles and interactions of the various phases is key towards a more complete picture of galaxy formation and evolution. For instance, these interactions play a pivotal role in the cycling of baryons which fuels star formation. The transport and dynamics of cold clouds in their surrounding hot environment are governed by complex small-scale processes (such as the interplay of turbulence and radiative cooling) that determine how the phases exchange mass, momentum, and energy. Large-scale models thus require subgrid prescriptions in the form of models validated on small-scale simulations, which often take the form of coupled differential equations. In this work, we explore using neural ordinary differential equations (ODEs) which embed neural networks as terms in the model to capture an uncertain physical process. We then apply symbolic regression to potentially discover new insights into the physics of cloud–environment interactions. We test this on both generated mock data and actual simulation data. We also extend the neural ODE to include a secondary neural term. We show that neural ODEs in tandem with symbolic regression can be used to enhance the accuracy and efficiency of subgrid models, and/or discover the underlying equations to improve generality and scientific understanding. We highlight the potential of this scientific machine learning approach as a natural extension to the traditional modelling paradigm, both for the development of semi-analytical models and for physically interpretable equation discovery in complex non-linear systems.

Machine learning (ML) surrogate models offer a promising route to accelerate material property prediction, bypassing costly atomistic simulations. Here, we introduce IsothermODE, a neural ordinary differential equation (NODE) framework for reconstructing full uptake and heat of adsorption $$\left(\Delta {H}_{{\rm{ads}}}\right)$$ isotherms for CO2 adsorption in metal-organic frameworks (MOFs) using only sparse pressure data. Unlike traditional ML models, IsothermODE leverages the intrinsic structure of differential equations to produce smooth, physically-consistent predictions that generalize across wide pressure ranges. We demonstrate high-fidelity interpolation and extrapolation, even with only five pressure points. To address the stochasticity inherent in Grand Canonical Monte Carlo (GCMC) simulations, we integrate uncertainty quantification, yielding tight bounds on predicted enthalpy curves. We further interpret the learned latent dynamics in terms of adsorption thermodynamics and textural properties, offering insight into structure-property relationships. Finally, we demonstrate IsothermODE’s long-range interpolation and extrapolation capabilities with sparse isotherm data (5 pressure points) and large incomplete intervals featuring missing data between 4–40 (case 1) and 25–50 (case 2) bars. IsothermODE provides a fast, robust alternative to simulation-heavy workflows, enabling scalable screening and design of next-generation carbon capture materials.

With the development of data acquisition technology, a large amount of time-series data can be collected. However, handling too much data often leads to a waste of social resources. It becomes significant to determine the minimum data size required for training. In this paper, a framework for neural ordinary differential equations based on incremental learning is discussed, which can enhance learning ability and determine the minimum data size required in data modeling compared to neural ordinary differential equations. This framework continuously updates the neural ordinary differential equations with newly added data while avoiding the addition of extra parameters. Once the preset accuracy is reached, the minimum data size needed for training can be determined. Furthermore, the minimum data size required for five classic models under various sampling rates is discussed. By incorporating new data, it enhances accuracy instead of increasing the depth and width of the neural network. The close integration of data generation and training can significantly reduce the total time required. Theoretical analysis confirms convergence, while numerical results demonstrate that the framework offers superior predictive ability and reduced computation time compared to traditional neural differential equations.

Traffic flow prediction is the core problem of traffic planning management and the basic basis of traffic control. However, the existing methods often do not fully consider the differences of different road nodes when modeling spatial dependence. In this paper, we propose a traffic flow prediction framework called graph neural ordinary differential equation recurrent neural network (GODE-RNN). Our method fuses graph convolution into neural ordinary differential equation to capture the spatial correlation, while the temporal correlation is extracted by sequence-to-sequence model. Specifically, we use neural ordinary differential equation (ODE) to expand the hidden state of graph convolution network, and different weights are used to weight and sum the hidden states of ODE for different road nodes, so that the network can learn the difference of spatial correlation of different road nodes independently. On the real-world data set, our method is significantly better than the baseline.

Informative missingness is unavoidable in the digital processing of continuous time series, where the value for one or more observations at different time points are missing. Such missing observations are one of the major limitations of time series processing using deep learning. Practical applications, e.g., sensor data, healthcare, weather, generates data that is in truth continuous in time, and informative missingness is a common phenomenon in these datasets. These datasets often consist of multiple variables, and often there are missing values for one or many of these variables. This characteristic makes time series prediction more challenging, and the impact of missing input observations on the accuracy of the final output can be significant. A recent novel deep learning model called GRU-D is one early attempt to address informative missingness in time series data. On the other hand, a new family of neural networks called Neural ODEs (Ordinary Differential Equations) are natural and efficient for processing time series data which is continuous in time. In this paper, a deep learning model is proposed that leverages the effective imputation of GRU-D, and the temporal continuity of Neural ODEs. A time series classification task performed on the PhysioNet dataset demonstrates the performance of this architecture.

Deep learning methods have become attractive recently to accelerate topology optimization (TO) because of their capability to save huge computational costs with negligible sacrifice in the quality of final topologies. However, most current approaches rely on numerical mechanics platforms for training which incur significant computation costs. Further, current approaches are not suited for dynamically changing boundary conditions. This is because these deep learning frameworks once trained using results generated with a specific set of boundary conditions do not readily adapt to others. In this article, we present a novel approach to leverage the abilities of deep learning models for TO. We demonstrate that optimization can be achieved using a neural ordinary differential equation. The evolution of the design variable in each iteration of TO is then achieved by numerical integration of this neural differential equation with respect to the starting design. To improve the quality of the results, two levels of generative adversarial networks are also introduced, at the sequence, and image levels, respectively. The proposed machine learning framework is capable of generating full optimization paths relevant to TO in high resolution within seconds and can address novel unseen boundary conditions.

Most structures of deep neural networks (DNN) are with a fixed complexity of both computational cost (parameters and FLOPs) and the expressiveness. In this work, we experimentally investigate the effectiveness of using neural ordinary differential equations (NODEs) as a component to provide further depth to relatively shallower networks rather than stacked layers (depth) which achieved improvement with fewer parameters. Moreover, we construct deep neural networks with flexible complexity based on NODEs which enables the system to adjust its complexity while training. The proposed method achieved more parameter-efficient performance than stacking standard DNNs, and it alleviates the defect of the heavy cost required by NODEs.

PIDNODEs: Neural ordinary differential equations inspired by a proportional–integral–derivative controller

We propose to formulate MRI image reconstruction as an optimization problem and model the optimization trajectory as a dynamic process using ordinary differential equations (ODEs). We model the dynamics in ODE with a neural network and solve the desired ODE with the off-the-shelf (fixed) solver to obtain reconstructed images. We extend this model and incorporate the knowledge of off-the-shelf ODE solvers into the network design (learned solvers). We investigate several models based on three ODE solvers and compare models with fixed solvers and learned solvers. Our models achieve better reconstruction results and are more parameter efficient than other popular methods such as UNet and cascaded CNN. We introduce a new way of tackling the MRI reconstruction problem by modeling the continuous optimization dynamics using neural ODEs.

It is crucial to establish accurate models for the identification of genes interactions. In our study, the ensemble model (EM) of flexible neural tree (FNT) and ordinary differential equations (ODEs) is used to infer gene regulatory networks (GRNs) more precisely. During the train of FNT, probabilistic incremental program evolution (PIPE) is employed to optimize the architecture, and particle swarm optimization (PSO) is employed to evolve the parameters. Similarly, the ODEs also use this method to optimize. Finally, the experimental results show that the model can be more accurate compared with some earlier methods. The mean squared error (MSE) of gene expression time series forecasting in this paper is smaller than prior work.

Modelling of oil well systems is important for a wide range of petroleum scientific and oil industrial processes. Considering the uncertainty of the measurements and the demand for empirical knowledge, a purely first‐principle model and a black‐box model based on data are not sufficient for accurately describing an oil well system. Thus, there is a growing body of literature that recognizes the importance of data‐driven methods combined with physical knowledge. However, the application of combination methods for dynamic nonlinear systems is still challenging. In this work, we demonstrate the application of a physics‐informed neural network to a gas lifting oil well system. The neural ordinary differential equation is the main tool for the modeling and the simulation is examined in Julia programming language. The advantage and drawbacks of the physics‐informed data‐driven method are analyzed.