Inferring incompressible two-phase flow fields from the interface motion using physics-informed neural networks

Physics-informed neural networks for high-speed flows

Physics-Informed Neural Networks (PINNs) provide a powerful framework for solving partial differential equations (PDEs) across various scientific and engineering disciplines. Unlike traditional numerical methods, PINNs do not require extensive expertise in the underlying numerical techniques of the target domain, making them accessible for a broader range of users. While PINNs have shown promise for solving forward and inverse problems, their performance relative to classical methods remains a topic of active investigation, particularly in radiative transfer applications. In this study, we assess the application of PINNs to a one-dimensional radiative transfer equation. Through simple test cases, we evaluate their strengths and limitations compared to the discrete ordinate method. Our findings indicate that PINNs can achieve an error as low as 0.3 % within several minutes of computation, whereas traditional solvers are significantly faster. However, the ease of implementation and the ability of PINNs to address inverse and high-dimensional problems highlight their potential as a complementary approach, particularly for complex scenarios.

Physics-informed neural networks (PINNs) have been widely used to solve various scientific computing problems. However, large training costs limit PINNs for some real-time applications. Although some works have been proposed to improve the training efficiency of PINNs, few consider the influence of initialization. To this end, we propose a New Reptile initialization based Physics-Informed Neural Network (NRPINN). The original Reptile algorithm is a meta-learning initialization method based on labeled data. PINNs can be trained with less labeled data or even without any labeled data by adding partial differential equations (PDEs) as a penalty term into the loss function. Inspired by this idea, we propose the new Reptile initialization to sample more tasks from the parameterized PDEs and adapt the penalty term of the loss. The new Reptile initialization can acquire initialization parameters from related tasks by supervised, unsupervised, and semi-supervised learning. Then, PINNs with initialization parameters can efficiently solve PDEs. Besides, the new Reptile initialization can also be used for the variants of PINNs. Finally, we demonstrate and verify the NRPINN considering both forward problems, including solving Poisson, Burgers, and Schr\"odinger equations, as well as inverse problems, where unknown parameters in the PDEs are estimated. Experimental results show that the NRPINN training is much faster and achieves higher accuracy than PINNs with other initialization methods.

A physics-informed neural network (PINN) uses physics-augmented loss functions, e.g., incorporating the residual term from governing partial differential equations (PDEs), to ensure its output is consistent with fundamental physics laws. However, it turns out to be difficult to train an accurate PINN model for many problems in practice. In this article, we present a novel perspective of the merits of learning in sinusoidal spaces with PINNs. By analyzing behavior at model initialization, we first show that a PINN of increasing expressiveness induces an initial bias around <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">flat output functions</i> . Notably, this initial solution can be very close to satisfying many physics PDEs, i.e., falling into a local minimum of the PINN loss that only minimizes PDE residuals, while still being far from the true solution that jointly minimizes PDE residuals and the initial and/or boundary conditions. It is difficult for gradient descent optimization to escape from such a local minimum trap, often causing the training to stall. We then prove that the sinusoidal mapping of inputs—in an architecture we label as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sf</i> -PINN—is effective to increase input gradient variability, thus avoiding being trapped in such deceptive local minimum. The level of variability can be effectively modulated to match high-frequency patterns in the problem at hand. A key facet of this article is the comprehensive empirical study that demonstrates the efficacy of learning in sinusoidal spaces with PINNs for a wide range of forward and inverse modeling problems spanning multiple physics domains.

Two-dimensional soil consolidation involves coupled vertical and horizontal drainage, posing challenges in accurately predicting consolidation coefficients due to complex boundary conditions and data limitations in real-world applications. This study addresses these issues by evaluating data-driven physics-informed neural networks (PINNs) for solving forward and inverse problems in 2D consolidation with coupled consolidation coefficients. The methodology integrates hard constraints for drainage boundaries and soft constraints for both drainage and impermeable boundaries, enabling precise modelling of excess pore pressure and coefficient interactions. Forward problems were analysed with and without observed data, optimising hyperparameters and assessing sensor arrangement effects on predictions. Inverse problems utilised observed data to estimate vertical and horizontal consolidation coefficients and incorporated noise to simulate practical scenarios. Results show that PINNs accurately predict the full consolidation process, determining coefficients from initial-period data alone and robustly handling noise levels typical in field measurements. This work contributes a novel, efficient PINN-based approach for consolidation analysis with coupled coefficients, offering practical advantages for geotechnical engineering where long-term data is scarce and measurements are error-prone.

&amp;lt;p&amp;gt;LEMTA - UMR 7563 CNRS-UL - Universite de Lorraine, Nancy, France&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;&amp;lt;br&amp;gt;Gravity currents [Ungarish, 2009] are common finescale structures in coastal areas, where they are involved in the vertical transport of physical and biogeological tracers. These stratified flows frequently carry a suspended solid phase that controls the average properties of the flow, such as the geometry, velocity and apparent viscosity leading to a rich dynamic whose optimization/prediction remains a major scientific issue.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;A better understanding of the particle-flow coupling requires an accurate characterization of the particle volume fraction, fluid density, velocity and pressure fields. The experimental apparatus available at LEMTA allows high accuracy instantaneous measurements of density fields in laboratory experiments with the light attenuation technique (LAT) as described in a companion presentation, while the joint measurement of velocity and pressure fields remains beyond the reach of known metrological techniques.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;To address this issue we propose an original approach based on the Physics Informed Neural Network (PINN) technique [Raissi 2020], which extracts the complete hydrodynamic variable set from the sole observation of the density field. The accuracy of this approach is evaluated against the well-documented test case of the lock-exchange configuration in both laminar and turbulent regimes. As a validation step, the PINN technique is applied on numerical simulations outputs with the well-established Nek5000 flow solver with convincing results. Then, an application on experimental data from LAT experiments will be shown, illustrating the robustness of this promising paradigm.&amp;lt;/p&amp;gt;

Physics-informed neural networks (PINNs) represent a transformative approach to data models by incorporating known physical laws into neural network training, thereby improving model generalizability, reduce data dependency, and enhance interpretability. Like many other fields in engineering and science, the analysis of physiological signals has been influenced by PINNs in recent years. This manuscript provides a comprehensive overview of PINNs from various perspectives in the physiological signal analysis domain. After exploring the literature and screening the search results, more than 40 key studies in the related domain are selected and categorized based on both practically and theoretically significant perspectives, including input data types, applications, physics-informed models, and neural network architectures. While the advantages of PINNs in tackling forward and inverse problems in physiological signal contexts are highlighted, challenges such as noisy inputs, computational complexity, loss function types, and overall model configuration are discussed, providing insights into future research directions and improvements. This work can serve as a guiding resource for researchers exploring PINNs in biomedical and physiological signal processing, paving the way for more precise, data-efficient, and clinically relevant solutions.

Deep neural network methods for solving forward and inverse problems of time fractional diffusion equations with conformable derivative

3D concrete printing (3DCP) is an innovative construction method that extrudes cementitious materials layer-by-layer to fabricate building components based on a digital model. 3DCP has gained increasing adoption globally for projects like buildings, bridges, retaining walls, and stormwater management systems. However, 3DCP places stringent demands on the rheological properties of the printable cementitious materials. Rheology is critical for the flow and buildability of fresh concrete during extrusion. The mixture must have a low yield stress for pumping, moderate viscosity to hold its shape after deposition, minimal bleeding and segregation of aggregates, as well as responsive rheology that can transition from fluid to solid state as layers are printed sequentially. In this dissertation, we aim to solve the challenges mentioned above including: (1) accurate and efficient quantification of rheological and thixotropic properties of cementitious materials, (2) evaluation of flow behavior of fresh concrete during extrusion. Using traditional experimental methods to evaluate rheological properties of cement paste is labor-intensive and time-consuming. Numerical simulations like Finite Element Method (FEM) or Computational Fluid Dynamics (CFD) can help increase the speed of finding the most suitable material for 3DCP. However, these simulation methods are costly in computing power, making them less desirable for simulating the dynamic and complex process of 3DCP. Moreover, mesh-based approaches are not suitable for complex geometries as the mesh generation would be challenging with curved boundaries, small features, holes or thin regions. Physics-Informed Neural Network (PINN) is introduced in the work as an innovative solution to simulate and model the whole process of 3DCP. It can leverage the flexibility and computational efficiency of data-based neural networks, which can accelerate the learning and convergence speed. At the same time, Partial Differential Equations (PDEs) would be embedded in the network structure to increase the accuracy of its prediction. The integrated governing physical laws can help the neural network understanding the underlying physics of the 3DCP process, potentially increasing the accuracy and reliability of simulations. A significant part of the research is dedicated to understanding the rheological and thixotropic behavior of cementitious materials, which is critical for the material design in 3DCP. We developed a Rheology-informed Neural Network, RheologyNet, where we embedded the rheological constitutive laws into the loss function. These physical laws can help the network focus on simulating the rheological properties of cementitious materials. Moreover, a thixotropic evaluation system has been established based on the proposed model to evaluate the thixotropic property of cement paste. This study is presented in Chapter 2. Then, we proposed a Navier-Stokes Informed Neural Network (NSINN) to study the flow behavior of cement paste in the 3DCP barrel and nozzle since it involves complicated Multiscale or Multiphysics behaviors, such as anisotropy and non-uniform shear rate distribution. The Navier Stokes Equation and the rheological constitutive equations are coupled and embedded into the NSINN architecture to simulate the flow behavior in the barrel during 3D-printing extrusion process. This approach demonstrates the potential of PINNs to provide high-accuracy simulations of velocity and pressure fields in a computationally efficient manner compared to traditional mesh-based models. Also, we used the NSINN to learn the relationship between the nozzle size and the printing quality. The details of the study are presented in Chapter 3. At last, a multi-subnetwork PINN architecture (MultiSubPINN) is proposed to address the limitations of traditional PINNs. This is because traditional PINNs used Fully-connected Neural Network (FNN) as main bone. It has problems like different outputs must share parameters within the network structure. However, this parameter sharing nature might lead to potential errors and decrease the model performance. The main architecture of this network is consisted of multiple separated sub-networks and the loss function of each sub-network is calculated separately to update the parameters in each sub-network. By doing this, we can ensure that the optimization for one output doesn't negatively impact others, therefore providing a more flexible, scalable, and potentially more accurate alternative for the evaluation. Besides, using randomly distributed training points in the computational domain for PINNs is a common way. However, many physical problems exhibit non-uniform behavior, with certain regions of the domain having more complex dynamics or sharper gradients than others. Randomly distributed training points might not adequately capture these regions, leading to a model that misrepresents the underlying physics in critical areas. We proposed a Physical-based sampling strategy to optimize the training of the PINNs. This study is presented in Chapter 4. In summary, this dissertation has established a new platform that is enabled by interpretable NN to accurately quantify cement paste’s rheological properties and simulate the flow behavior of 3D concrete printing process. The findings suggest that PINNs can not only improve the efficiency and accuracy of simulations but also provide a more profound understanding of the material behaviors and process dynamics critical to 3DCP. The novel PINN structure developed from this dissertation is a powerful general platform that could be applicable to a variety of material domains for complex rheological behavior predictions.

Partial differential equations (PDEs) are core tools for characterizing physical laws and are widely used in fluid mechanics, power systems, additive manufacturing, and other fields. However, traditional numerical methods are limited by mesh partitioning, leading to a trade-off between accuracy and efficiency in complex geometric domains and high-dimensional problems. While Physically Informed Neural Networks (PINNs) achieve meshless PDE solutions by embedding prior physical knowledge, they still face challenges such as insufficient accuracy, low training efficiency, and poor stability. This study aims to address the core bottlenecks of PINN in solving PDEs by proposing a systematic optimization strategy to improve its numerical solution accuracy, efficiency, and stability. Methodologically, firstly, a composite loss function containing PDE residuals and boundary initial condition constraints is constructed based on automatic differentiation techniques to ensure that the network satisfies physical laws. Secondly, the PINN training system is optimized, including using Neural Architecture Search (NAS-PINN) to automatically match the optimal network structure, designing an adaptive data sampling strategy to improve data utilization, and introducing parallel computing and hardware acceleration techniques to reduce training time. Finally, the effectiveness of the strategy is verified through classical PDEs and engineering problems. The results show that the optimized PINN achieves high-precision numerical approximation in classical PDE solutions and can efficiently solve forward design and backward parameter inversion problems in engineering scenarios. Compared with traditional methods, it exhibits stronger adaptability in complex geometric domains and high-dimensional problems, while improving training efficiency by more than 30% and significantly enhancing stability. This research not only enriches the optimization theory of PINN and provides an efficient new path for PDE numerical solutions, but also promotes the practical application of PINN in engineering fields such as fluid mechanics, additive manufacturing, and power systems, possessing significant theoretical value and application significance.

Simulating solute transport in heterogeneous porous media poses computational challenges due to the high-resolution meshing required for traditional solvers. To overcome these challenges, this study explores a mesh-free method based on deep learning to accelerate solute transport simulation. We employ Physics-informed Neural Networks (PiNN) with a periodic activation function to solve solute transport problems in both homogeneous and heterogeneous porous media governed by the advection-dispersion equation. Unlike traditional neural networks that rely on large training datasets, PiNNs use strong-form mathematical models to constrain the network in the training phase and simultaneously solve for multiple dependent or independent field variables, such as pressure and solute concentration fields. To demonstrate the effectiveness of using PiNNs with a periodic activation function to resolve solute transport in porous media, we construct PiNNs using two activation functions, sin and tanh, for seven case studies, including 1D and 2D scenarios. The accuracy of the PiNNs’ predictions is then evaluated using absolute point error and mean square error metrics and compared to the ground truth solutions obtained analytically or numerically. Our results demonstrate that the PiNN with sin activation function, compared to tanh activation function, is up to two orders of magnitude more accurate and up to two times faster to train, especially in heterogeneous porous media. Moreover, PiNN’s simultaneous predictions of pressure and concentration fields can reduce computational expenses in terms of inference time by three orders of magnitude compared to FEM simulations for two-dimensional cases.

Wind loads can endanger the safety and stability of bridges, especially long-span cable-supported bridges. Therefore, it is important to evaluate the potential wind loads during the bridge design stage. Traditionally, wind load evaluation is performed by wind tunnel testing, which is relatively expensive. With the development of computational fluid dynamics and high-performance computing, numerical simulations are becoming more accessible for designers. However, the costs required for accurate numerical results are still high, especially for high-fidelity simulations. Under this condition, searching for a more efficient method to evaluate the wind loads in bridge wind engineering has become a new goal. It seems that flow visualization is a good entry point. Although flow visualization techniques have been developed in recent years, it remains difficult to extract velocity and pressure fields from images. To address this problem, physics-informed neural networks (PINNs) have been developed and validated. This study establishes a PINN to investigate the two-dimensional viscous incompressible fluid flow passing a generic bridge deck section. Two cases with different Reynolds numbers are tested. After careful training, it is found that the PINN can accurately extract the velocity and pressure fields from the concentration field and predict the drag and lift coefficients. The results demonstrate that PINNs are a promising method for extracting useful flow information from flow visualization data in engineering applications.

In recent years, physics-informed neural networks (PINNs) that can simultaneously utilize data and mathematical physical models have shown great potential in numerical computation. However, PINNs often struggle to provide accurate predictions in scenarios with drastic changes. To address this, we have designed a multi-head adaptive weighted structure that aims to make the network focus its attention on areas that are difficult to predict. Additionally, to prevent the network from learning erroneous information in cases where the dataset is noisy or has other issues, we have used enhanced physical information as a regularization term to constrain the network. However, complex network structures can make it difficult for optimizers to work effectively. To ensure that the network can be optimized smoothly, we have used ResNet blocks. We conducted numerical experiments using noisy forward and inverse problems, which resulted in significantly improved prediction accuracy. Furthermore, we used ablation study to verify the role of each component.

Identifying optimal architectures of physics-informed neural networks by evolutionary strategy

We present the deep random vortex network (DRVN), a novel physics-informed framework for simulating and inferring the fluid dynamics governed by the incompressible Navier–Stokes equations. Unlike the existing physics-informed neural network (PINN), which embeds physical and geometry information through the residual of equations and boundary data, DRVN automatically embeds this information into neural networks through neural random vortex dynamics equivalent to the Navier–Stokes equation. Specifically, the neural random vortex dynamics motivates a Monte Carlo-based loss function for training neural networks, which avoids the calculation of derivatives through auto-differentiation. Therefore, DRVN can efficiently solve Navier–Stokes equations with non-differentiable initial conditions and fractional operators. Furthermore, DRVN naturally embeds the boundary conditions into the kernel function of the neural random vortex dynamics and, thus, does not need additional data to obtain boundary information. We conduct experiments on forward and inverse problems with incompressible Navier–Stokes equations. The proposed method achieves accurate results when simulating and when inferring Navier–Stokes equations. For situations that include singular initial conditions and agnostic boundary data, DRVN significantly outperforms the existing PINN method. Furthermore, compared with the conventional adjoint method when solving inverse problems, DRVN achieves a 2 orders of magnitude improvement for the training time with significantly precise estimates.