A Holistic Approach to Computing First-Arrival Traveltimes Using Physics-Informed Neural Networks

Adaptive transfer learning for PINN

The eikonal equation plays an important role across multidisciplinary branches of science and engineering. In geophysics, the eikonal equation, and its characteristics, are used in addressing two fundamental questions pertaining to seismic waves: what paths do the seismic waves take (its spreading)? and how long do they take? There have been numerous attempts to solve the eikonal equation, which can be broadly categorized as finite-difference and physics informed neural network (PINN) based approaches. While the former has been developed and optimized over the years, it still inherits some numerical inaccuracies and also the cost scales exponentially with the velocity model size. More importantly, it requires upwind calculations to satisfy the viscosity solution. PINNs, on the other hand, have shown great promise due to several features allowing for higher accuracy and scalability than conventional approaches. In this paper, we demonstrate another unique feature of PINN solutions, specifically its flexibility resulting from the global nature of its NN functional optimization, allowing for functional gradients referred to as automatic differentiation. This feature allows us to overcome the inability of conventional methods to handle large areas of missing information (gap) in the velocity model. We find empirically that the PINNs interpolation-extrapolation inherent capability enables us to circumvent a scenario when traveltime modelling is performed on velocity models containing gaps. Such a capability is crucial when performing traveltime modelling using the global tomographic Earth velocity model.

Transfer learning enhanced physics informed neural network for phase-field modeling of fracture

Indoor airflow field reconstruction using physics-informed neural network

• The finite element-integrated neural network (FEINN) is enhanced by transfer learning strategy. • The impacts of the FEM mesh on FEINN are investigated and fully discussed. • Scale transfer learning, material transfer learning, load transfer learning strategies are evaluated. • FEINN is successfully accelerated in solving elastic, elastoplastic, multiple material boundary value problems. Physics informed neural networks (PINNs) have attracted increasing attention in computational solid mechanics due to their success in solving complex partial differential equations (PDEs). Nevertheless, the low efficiency and precision always hinder the application of PINNs in boundary value problems. To address this issue, this study proposed a transfer learning enhanced hybrid framework that integrates the finite element method with PINNs to accelerate the training process. The finite element-integrated neural network framework (FEINN) is first introduced, leveraging finite elements for domain discretization and the weak-form governing equation for defining the loss function. A mesh parametric study is subsequently conducted, aiming to identify the optimal discretization configuration by exploring various element sizes, element types, and orders of shape functions. Furthermore, various transfer learning strategies are proposed and fully evaluated to improve the training efficiency and precision of FEINN, including scale transfer learnings (STLs) from coarse mesh to refine mesh and from small domain to large domain, material transfer learnings (MTLs) from elastic material to elastoplastic material and from elastic material to elastic material problems, as well as load transfer learnings (LTLs) form displacement load condition to force load condition. A series of experiments are conducted to showcase the effectiveness of FEINN, identifying the most efficient discretization configuration and validating the efficacy of transfer learning strategies across elastic, elastoplastic, and multi-material scenarios. The results indicate that the element type and size, and shape function order have significant impacts on training efficiency and accuracy. Moreover, the transfer learning techniques can significantly improve the accuracy and training efficiency of FEINN.

Recently, there has been a growing interest in the development of intelligent labelfree structural damage identification methods that utilize physics-informed neural networks (PINNs). However, penalizing the governing equation of training data is computationally time-consuming since the existence of high-order partial derivatives. To address this issue, a damage identification method for isotropic and homogeneous thin plates is proposed in this paper that utilizes transfer learning physics-informed neural networks (TL-PINNs). TL-PINNs are efficient PINNs that solve inverse problems by leveraging transfer learning. Transfer learning is a machine learning technique that leverages knowledge from a source task to enhance performance on a related but different target task. It involves reusing a source model trained on a source task and then fine-tuning it to a target model with a target task. In the proposed method, the source model is trained to minimize the mismatch between training data and its predictions. Then, it is finetuned as the target model by minimizing both the mismatch between training data and its predictions as well as residuals that penalize the governing equation of isotropic and homogeneous thin plates. It is resulting in fewer iterations being required in training to penalize the governing equation than those in PINNs, which is time-consuming for highorder partial derivatives using automatic differentiation. Hence, TL-PINNs have a substantial reduction in computational time compared to PINNs for damage identification. A trained TL-PINN from a measured flexural guided wavefield is referred to as a pseudopristine model since it can generate a wavefield that approximates that governed by an isotropic and homogeneous thin plate. This unique functionality arises from penalizing the governing equation in the target model and the fact that the governing equation does not consider the existence of the damage. Any local anomalies in the measured wavefield can be isolated by comparing them with the wavefield generated by the pseudo-pristine model and then intensified using the Teager energy operator. An accumulative damage index is formulated, and the damage can be identified within neighborhoods with high index values. The effectiveness of the proposed method is demonstrated through a numerical investigation. A parameter study is also conducted to investigate the robustness of TL-PINNs with different hyper-parameters.

Physics-informed neural networks (PINNs) have been widely used to solve various scientific computing problems. However, large training costs limit PINNs for some real-time applications. Although some works have been proposed to improve the training efficiency of PINNs, few consider the influence of initialization. To this end, we propose a New Reptile initialization based Physics-Informed Neural Network (NRPINN). The original Reptile algorithm is a meta-learning initialization method based on labeled data. PINNs can be trained with less labeled data or even without any labeled data by adding partial differential equations (PDEs) as a penalty term into the loss function. Inspired by this idea, we propose the new Reptile initialization to sample more tasks from the parameterized PDEs and adapt the penalty term of the loss. The new Reptile initialization can acquire initialization parameters from related tasks by supervised, unsupervised, and semi-supervised learning. Then, PINNs with initialization parameters can efficiently solve PDEs. Besides, the new Reptile initialization can also be used for the variants of PINNs. Finally, we demonstrate and verify the NRPINN considering both forward problems, including solving Poisson, Burgers, and Schr\"odinger equations, as well as inverse problems, where unknown parameters in the PDEs are estimated. Experimental results show that the NRPINN training is much faster and achieves higher accuracy than PINNs with other initialization methods.

A novel sequential method to train physics informed neural networks for Allen Cahn and Cahn Hilliard equations

Nuclear reactor safety and efficiency can be enhanced through the development of accurate and fast methods for prediction of reactor transient (RT) states. Physics informed neural networks (PINNs) leverage deep learning methods to provide an alternative approach to RT modeling. Applications of PINNs in monitoring of RTs for operator support requires near real-time model performance. However, as with all machine learning models, development of a PINN involves time-consuming model training. Here, we show that a transfer learning (TL-PINN) approach achieves significant performance gain, as measured by reduction of the number of iterations for model training. Using point kinetic equations (PKEs) model with six neutron precursor groups, constructed with experimental parameters of the Purdue University Reactor One (PUR-1) research reactor, we generated different RTs with experimentally relevant range of variables. The RTs were characterized using Hausdorff and Fréchet distance. We have demonstrated that pre-training TL-PINN on one RT results in up to two orders of magnitude acceleration in prediction of a different RT. The mean error for conventional PINN and TL-PINN models prediction of neutron densities is smaller than 1%. We have developed a correlation between TL-PINN performance acceleration and similarity measure of RTs, which can be used as a guide for application of TL-PINNs.

We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as physics informed neural networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of data-fidelity and equation-fidelity objectives. Conflicts between objectives can arise from scale imbalances, heteroscedasticity in the data, stiffness of the physical equation, or from catastrophic interference during sequential training. We explain the training pathology arising from this and propose a simple yet effective inverse Dirichlet weighting strategy to alleviate the issue. We compare with Sobolev training of neural networks, providing the baseline of analytically ε-optimal training. We demonstrate the effectiveness of inverse Dirichlet weighting in various applications, including a multi-scale model of active turbulence, where we show orders of magnitude improvement in accuracy and convergence over conventional PINN training. For inverse modeling using sequential training, we find that inverse Dirichlet weighting protects a PINN against catastrophic forgetting.

Summary Traveltimes corresponding to both compressional and shear waves are needed for many applications in seismology ranging from seismic imaging to earthquake localization. Since the behavior of shear waves in anisotropic media is considerably more complicated than the isotropic case, accurate traveltime computation for shear waves in anisotropic media remains a challenge. Ray tracing methods are often used to compute qSV wave traveltimes but they become unstable around triplication points and, therefore, we often use the weak anisotropy approximation. Here, we employ the emerging paradigm of physics-informed neural networks to solve transversely isotropic eikonal equation for the qSV wave that otherwise are not easily solvable using conventional finite difference methods. By minimizing a loss function formed by imposing the validity of eikonal equation, we train a neural network to produce traveltime solutions that are consistent with the underlying equation. Through tests on synthetic models, we show that the method is capable of producing accurate qSV wave traveltimes even at triplication points and works for arbitrary strength of medium anisotropy.

The Burgers equation is a well-established test case in the computational modeling of several phenomena, such as fluid dynamics, gas dynamics, shock theory, cosmology and others. In this work, we present the application of physics-informed neural networks (PINNs) with a transfer learning approach using the θ-scheme to solve the Burgers' equation. The proposed approach consists of searching for a discrete solution in time through a sequence of artificial neural networks (ANNs). At each time step, the previous ANN transfers its learning to the next network model, which learns the solution in the current time by minimizing a loss function based on the θ-scheme approximation of the Burgers' equation. To test this approach, we present its application to two benchmark problems with known analytical solutions. Compared to usual PINN models, the proposed approach has the advantage of requiring smaller neural network architectures with similar accurate results and potentially decreasing computational costs.

Simulating solute transport in heterogeneous porous media poses computational challenges due to the high-resolution meshing required for traditional solvers. To overcome these challenges, this study explores a mesh-free method based on deep learning to accelerate solute transport simulation. We employ Physics-informed Neural Networks (PiNN) with a periodic activation function to solve solute transport problems in both homogeneous and heterogeneous porous media governed by the advection-dispersion equation. Unlike traditional neural networks that rely on large training datasets, PiNNs use strong-form mathematical models to constrain the network in the training phase and simultaneously solve for multiple dependent or independent field variables, such as pressure and solute concentration fields. To demonstrate the effectiveness of using PiNNs with a periodic activation function to resolve solute transport in porous media, we construct PiNNs using two activation functions, sin and tanh, for seven case studies, including 1D and 2D scenarios. The accuracy of the PiNNs’ predictions is then evaluated using absolute point error and mean square error metrics and compared to the ground truth solutions obtained analytically or numerically. Our results demonstrate that the PiNN with sin activation function, compared to tanh activation function, is up to two orders of magnitude more accurate and up to two times faster to train, especially in heterogeneous porous media. Moreover, PiNN’s simultaneous predictions of pressure and concentration fields can reduce computational expenses in terms of inference time by three orders of magnitude compared to FEM simulations for two-dimensional cases.

In this paper, we employ the emerging paradigm of physics-informed neural networks (PINNs) for the solution of representative inverse scattering problems in photonic metamaterials and nano-optics technologies. In particular, we successfully apply mesh-free PINNs to the difficult task of retrieving the effective permittivity parameters of a number of finite-size scattering systems that involve many interacting nanostructures as well as multi-component nanoparticles. Our methodology is fully validated by numerical simulations based on the finite element method (FEM). The development of physics-informed deep learning techniques for inverse scattering can enable the design of novel functional nanostructures and significantly broaden the design space of metamaterials by naturally accounting for radiation and finite-size effects beyond the limitations of traditional effective medium theories.

This article presents a novel framework for onsite estimation of lithium-ion battery electrochemical parameters by combining physics-informed neural networks (PINNs) with transfer learning (TL). Unlike conventional physics-based models that require costly offline parameterization or extensive experimental tests, the proposed two-phase strategy first trains a PINN using only the governing equations of the single particle model, and then fine-tunes selected electrochemical parameters with field voltage measurements while keeping most network weights frozen. This enables efficient adaptation to ageing and cell-to-cell variability with minimal data requirements. The approach achieves high accuracy in estimating key parameters such as diffusion coefficients and active material volume fractions under different degradation conditions. Under a simulation environment, the method reconstructs voltage responses with a root-mean-square error of 8.94 mV and estimates active material fractions with an average relative error below 2.2% across 400 study cases, significantly outperforming classical optimization-based estimators. Experimental validation on an aged NMC cell (82.09% nominal capacity) implemented on a Raspberry Pi 5, demonstrates practical feasibility, yielding a relative error of 3.89% in estimating volume fractions with a computational cost of 106 ms per iteration. These results confirm that the proposed TL-based PINN framework offers an accurate, computationally efficient, and easily deployable solution for real-time battery management systems.