Exact Dirichlet boundary Physics-informed Neural Network EPINN for solid mechanics

Exact boundary condition for time-dependent wave equation based on boundary integral

Physics-informed neural networks for mesh deformation with exact boundary enforcement

A novel sequential method to train physics informed neural networks for Allen Cahn and Cahn Hilliard equations

Simulating convective heat transfer using traditional numerical methods requires explicit definition of thermal boundary conditions on all boundaries of the domain, which is almost impossible to fulfill in real applications. Here, we address this ill-posed problem using machine learning techniques by assuming that we have some extra measurements of the temperature at a few locations in the domain, not necessarily located on the boundaries with the unknown thermal boundary condition. In particular, we employ physics-informed neural networks (PINNs) to represent the velocity and temperature fields while simultaneously enforce the Navier-Stokes and energy equations at random points in the domain. In PINNs, all differential operators are computed using automatic differentiation, hence avoiding discretization in either space or time. The loss function is composed of multiple terms, including the mismatch in the velocity and temperature data, the boundary and initial conditions, as well as the residuals of the Navier-Stokes and energy equations. Here, we develop a data-driven strategy based on PINNs to infer the temperature field in the prototypical problem of convective heat transfer in flow past a cylinder. We assume that we have just a couple of temperature measurements on the cylinder surface and a couple more temperature measurements in the wake region, but the thermal boundary condition on the cylinder surface is totally unknown. Upon training the PINN, we can discover the unknown boundary condition while simultaneously infer the temperature field everywhere in the domain with less than 5% error in the Nusselt number prediction. In order to assess the performance of PINN, we carried out a high fidelity simulation of the same heat transfer problem (with known thermal boundary conditions) by using the high-order spectral/hp-element method (SEM), and quantitatively evaluated the accuracy of PINN’s prediction with respect to SEM. We also propose a method to adaptively select the location of sensors in order to minimize the number of required temperature measurements while increasing the accuracy of the inference in heat transfer.

The nonlinear sine‐Gordon equation is a prevalent feature in numerous scientific and engineering problems. In this paper, we propose a machine learning‐based approach, physics‐informed neural networks (PINNs), to investigate and explore the solution of the generalized non‐linear sine‐Gordon equation, encompassing Dirichlet and Neumann boundary conditions. To incorporate physical information for the sine‐Gordon equation, a multiobjective loss function has been defined consisting of the residual of governing partial differential equation (PDE), initial conditions, and various boundary conditions. Using multiple densely connected independent artificial neural networks (ANNs), called feedforward deep neural networks designed to handle partial differential equations, PINNs have been trained through automatic differentiation to minimize a loss function that incorporates the given PDE that governs the physical laws of phenomena. To illustrate the effectiveness, validity, and practical implications of our proposed approach, two computational examples from the nonlinear sine‐Gordon are presented. We have developed a PINN algorithm and implemented it using Python software. Various experiments were conducted to determine an optimal neural architecture. The network training was employed by using the current state‐of‐the‐art optimization methods in machine learning known as Adam and L‐BFGS‐B minimization techniques. Additionally, the solutions from the proposed method are compared with the established analytical solutions found in the literature. The findings show that the proposed method is a computational machine learning approach that is accurate and efficient for solving nonlinear sine‐Gordon equations with a variety of boundary conditions as well as any complex nonlinear physical problems across multiple disciplines.

In recent engineering applications using deep learning, physics-informed neural network (PINN) is a new development as it can exploit the underlying physics of engineering systems. The novelty of PINN lies in the use of partial differential equations (PDE) for the loss function. Most PINNs are implemented using automatic differentiation (AD) for training the PDE loss functions. A lesser well-known study is the use of finite difference method (FDM) as an alternative. Unlike an AD based PINN, an immediate benefit of using a FDM based PINN is low implementation cost. In this paper, we propose the use of finite difference method for estimating the PDE loss functions in PINN. Our work is inspired by computational analysis in electromagnetic systems that traditionally solve Laplace’s equation using successive over-relaxation. In the case of Laplace’s equation, our PINN approach can be seen as taking the Laplacian filter response of the neural network output as the loss function. Thus, the implementation of PINN can be very simple. In our experiments, we tested PINN on Laplace’s equation and Burger’s equation. We showed that using FDM, PINN consistently outperforms non-PINN based deep learning. When comparing to AD based PINNs, we showed that our method is faster to compute as well as on par in terms of error reduction.

We present a new and very powerful theoretical method-a distorted-wave Born approximation (DWBA) with an arbitrary final-state electron-electron correlation function. This method combines the flexibility of including any theoretically desired electron-electron correlation and the physical clarity of the DWBA method. Calculations explore the use of an auxiliary final-state wave function with the exact boundary condition, in attempting to describe the absolute cross sections for ionisation of helium in coplanar symmetric kinematics and to resolve discrepancies between theory and experiment.

In recent years, Physics-Informed Neural Networks (PINNs) have introduced a novel approach to solving partial differential equations (PDEs) using deep learning techniques. Despite the promising results and rapid advancements in the field, there is a lack of more accurate studies concerning properties related to modeling choices within the deep learning framework, especially, but not exclusively, in solid mechanics problems. In this study, we aim to explore whether the neural network architecture, activation function, optimizer, and sampling method can influence the accuracy of results and training speed for solid mechanics problems. We will focus on elasticity and hyperelasticity problems in 1D, 2D, and 3D dimensions to lay the groundwork for more complex investigations.

In this paper, we propose an extrinsic approach based on physics-informed neural networks (PINNs) for solving the partial differential equations (PDEs) on surfaces embedded in high dimensional space. PINNs are one of the deep learning-based techniques. Based on the training data and physical models, PINNs introduce the standard feedforward neural networks (NNs) to approximate the solutions to the PDE systems. Using automatic differentiation, the PDEs information could be encoded into NNs and a loss function. To deal with the surface differential operators in the loss function, we combine the extrinsic approach with PINNs and then express that loss function in extrinsic form. Subsequently, the loss function could be minimized extrinsically with respect to the NN parameters. Numerical results demonstrate that the extrinsic approach based on PINNs for surface problems has good accuracy and higher efficiency compared with the embedding approach based on PINNs. In addition, the strong nonlinear mapping ability of NNs makes this approach robust in solving time-dependent nonlinear problems on more complex surfaces.

The physics‐informed neural network (PINN) method has been applied to solve water hammer equations in pipeline systems due to its ability of seamlessly integrate measurement data with conservation laws, offering advantages over traditional numerical method. However, existing PINN approaches require multiple neural networks to construct composite models for complex water distribution systems (WDS). This situation treats nodal information as boundary condition or labeled data during training, leading to a weaker robustness and a high demand for data. To address these issues, a hybrid water hammer model based on eXtended Physics‐Informed Neural Networks for WDS (WDS‐XPINN) is developed in this study. Unlike the standard PINN, WDS‐XPINN incorporates the nodal mechanistic model directly into the loss function, enabling to synchronously train a unified neural network jointly through sparse augmented measurement data for pipeline system. Additionally, an adaptive weights method is introduced to improve model robustness by balancing the contributions of flowrates and pressures. The proposed WDS‐XPINN is evaluated in two case studies: a series pipeline system with different operational events and noise perturbation, as well as a topological structure with looped and branched pipe. According to the simulation results and uncertainty analysis, the WDS‐XPINN model demonstrates its excellent capacity of modeling fluid transient accurately in pipeline system, even without exact operational conditions or true pipe parameters.

The exploration of deep learning methodologies has recently generated significant interest in the use of Physics-Informed Neural Networks (PINNs) to address complex physical problems governed by partial differential equations (PDEs). The PINN is trained using information from physical laws, including governing PDEs, boundary conditions, and initial conditions. However, achieving a well-trained PINN typically necessitates an appropriate balance between the weights of each loss function, which can considerably increase manual effort. This paper introduces a novel training approach that integrates non-dimensionalization with a multi-task learning technique, termed Automatic Weighted Loss (AWL), to autonomously achieve an optimal balance for each loss function. In the baseline PINN training for solving time-dependent PDEs, multiple weights (usually more than six) must be manually tuned for the model, whereas this method can reduce the number of scaling weights to only one. The proposed approach, referred to as the Non-dimensionalization Automatic Weighted Loss (NDAWL), is evaluated through six free-surface flow problems modelled by the Shallow Water Equations (SWEs). Furthermore, a comparative analysis is conducted between the solutions obtained using NDAWL-PINN and those from the original PINN, which relies on manual fine-tuning of loss functions. The numerical results indicate that the NDAWL-PINN method achieves comparable or superior accuracy to the original PINN, demonstrating its effectiveness in automating the balancing of loss functions.

1-D coupled surface flow and transport equations revisited via the physics-informed neural network approach

Satellite-based research on global sea surface wind is essential for understanding and monitoring the air-sea interface dynamical processes, driving the need for accurate and efficient assessment. Spaceborne scatterometers, which are among the most relevant sensors for sea surface wind observation, play a vital role in obtaining global ocean surface wind information. However, a significant challenge associated with polar-orbiting satellites lies in data gaps caused by their orbital paths, resulting in missing observations between swaths. While several satellite-derived sea surface wind products have been developed using data assimilation (DA) techniques, these existing methods are time-consuming and require large amounts of diverse data, rendering them computationally expensive. Additionally, the iterative steps of variational algorithms can only perform linear or weakly nonlinear adjustments to the governing equations, which may pose challenges given the highly non-linear nature of these equations.In this study, we leveraged physics-informed neural networks (PINNs) techniques to reconstruct large-scale sea surface wind fields with sparse scatterometer observations from different satellites, integrating observations and filling gaps. By incorporating physical constraints into the loss function, specifically the Navier-Stokes equations, we efficiently fill the data gaps and reconstruct wind fields that not only match observational data but also adhere to physical principles. Another objective of this work is to introduce the wind speed gradient and direction parallel consistent constraints into the loss function in order to enhance the detail of the reconstructed wind field and increase the accuracy of both wind speed and direction. Structurally, the PINN resembles a fully connected neural network (FCNN), offering the advantage of automatic feature extraction. Our model not only extracts valuable information from existing data but also uncovers complex patterns and correlations in data that are difficult for traditional algorithms to capture. This approach provides a novel perspective and an alternative methodology for wind field reconstruction.The results show that PINNs can reconstruct wind fields that closely resemble realistic wind patterns, capturing large-scale structures while preserving fine-scale details, thanks to the introduction of wind speed gradient and direction parallel consistent constraints. The training time for this model is about 3 hours, using only a single GPU core. This efficiency is partly due to the fact that PINNs do not rely on ensemble methods or large datasets to produce results. Unlike traditional DA methods, PINN does not depend on an initial best-guess field for assimilating observations. While we use only a small amount of scatterometer observation data, no initial field is required to complete the reconstruction. Additionally, since the PINN represents a continuous and differentiable function, it can produce outputs at any spatial or temporal point within the training domain.Recognizing their potential for forecast models and data integration, PINNs offer a promising approach for accurate sea surface wind field reconstruction and could serve as an effective alternative to current methods.

A physics-informed neural network (PINN) uses physics-augmented loss functions, e.g., incorporating the residual term from governing partial differential equations (PDEs), to ensure its output is consistent with fundamental physics laws. However, it turns out to be difficult to train an accurate PINN model for many problems in practice. In this article, we present a novel perspective of the merits of learning in sinusoidal spaces with PINNs. By analyzing behavior at model initialization, we first show that a PINN of increasing expressiveness induces an initial bias around <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">flat output functions</i> . Notably, this initial solution can be very close to satisfying many physics PDEs, i.e., falling into a local minimum of the PINN loss that only minimizes PDE residuals, while still being far from the true solution that jointly minimizes PDE residuals and the initial and/or boundary conditions. It is difficult for gradient descent optimization to escape from such a local minimum trap, often causing the training to stall. We then prove that the sinusoidal mapping of inputs—in an architecture we label as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sf</i> -PINN—is effective to increase input gradient variability, thus avoiding being trapped in such deceptive local minimum. The level of variability can be effectively modulated to match high-frequency patterns in the problem at hand. A key facet of this article is the comprehensive empirical study that demonstrates the efficacy of learning in sinusoidal spaces with PINNs for a wide range of forward and inverse modeling problems spanning multiple physics domains.

3D concrete printing (3DCP) is an innovative construction method that extrudes cementitious materials layer-by-layer to fabricate building components based on a digital model. 3DCP has gained increasing adoption globally for projects like buildings, bridges, retaining walls, and stormwater management systems. However, 3DCP places stringent demands on the rheological properties of the printable cementitious materials. Rheology is critical for the flow and buildability of fresh concrete during extrusion. The mixture must have a low yield stress for pumping, moderate viscosity to hold its shape after deposition, minimal bleeding and segregation of aggregates, as well as responsive rheology that can transition from fluid to solid state as layers are printed sequentially. In this dissertation, we aim to solve the challenges mentioned above including: (1) accurate and efficient quantification of rheological and thixotropic properties of cementitious materials, (2) evaluation of flow behavior of fresh concrete during extrusion. Using traditional experimental methods to evaluate rheological properties of cement paste is labor-intensive and time-consuming. Numerical simulations like Finite Element Method (FEM) or Computational Fluid Dynamics (CFD) can help increase the speed of finding the most suitable material for 3DCP. However, these simulation methods are costly in computing power, making them less desirable for simulating the dynamic and complex process of 3DCP. Moreover, mesh-based approaches are not suitable for complex geometries as the mesh generation would be challenging with curved boundaries, small features, holes or thin regions. Physics-Informed Neural Network (PINN) is introduced in the work as an innovative solution to simulate and model the whole process of 3DCP. It can leverage the flexibility and computational efficiency of data-based neural networks, which can accelerate the learning and convergence speed. At the same time, Partial Differential Equations (PDEs) would be embedded in the network structure to increase the accuracy of its prediction. The integrated governing physical laws can help the neural network understanding the underlying physics of the 3DCP process, potentially increasing the accuracy and reliability of simulations. A significant part of the research is dedicated to understanding the rheological and thixotropic behavior of cementitious materials, which is critical for the material design in 3DCP. We developed a Rheology-informed Neural Network, RheologyNet, where we embedded the rheological constitutive laws into the loss function. These physical laws can help the network focus on simulating the rheological properties of cementitious materials. Moreover, a thixotropic evaluation system has been established based on the proposed model to evaluate the thixotropic property of cement paste. This study is presented in Chapter 2. Then, we proposed a Navier-Stokes Informed Neural Network (NSINN) to study the flow behavior of cement paste in the 3DCP barrel and nozzle since it involves complicated Multiscale or Multiphysics behaviors, such as anisotropy and non-uniform shear rate distribution. The Navier Stokes Equation and the rheological constitutive equations are coupled and embedded into the NSINN architecture to simulate the flow behavior in the barrel during 3D-printing extrusion process. This approach demonstrates the potential of PINNs to provide high-accuracy simulations of velocity and pressure fields in a computationally efficient manner compared to traditional mesh-based models. Also, we used the NSINN to learn the relationship between the nozzle size and the printing quality. The details of the study are presented in Chapter 3. At last, a multi-subnetwork PINN architecture (MultiSubPINN) is proposed to address the limitations of traditional PINNs. This is because traditional PINNs used Fully-connected Neural Network (FNN) as main bone. It has problems like different outputs must share parameters within the network structure. However, this parameter sharing nature might lead to potential errors and decrease the model performance. The main architecture of this network is consisted of multiple separated sub-networks and the loss function of each sub-network is calculated separately to update the parameters in each sub-network. By doing this, we can ensure that the optimization for one output doesn't negatively impact others, therefore providing a more flexible, scalable, and potentially more accurate alternative for the evaluation. Besides, using randomly distributed training points in the computational domain for PINNs is a common way. However, many physical problems exhibit non-uniform behavior, with certain regions of the domain having more complex dynamics or sharper gradients than others. Randomly distributed training points might not adequately capture these regions, leading to a model that misrepresents the underlying physics in critical areas. We proposed a Physical-based sampling strategy to optimize the training of the PINNs. This study is presented in Chapter 4. In summary, this dissertation has established a new platform that is enabled by interpretable NN to accurately quantify cement paste’s rheological properties and simulate the flow behavior of 3D concrete printing process. The findings suggest that PINNs can not only improve the efficiency and accuracy of simulations but also provide a more profound understanding of the material behaviors and process dynamics critical to 3DCP. The novel PINN structure developed from this dissertation is a powerful general platform that could be applicable to a variety of material domains for complex rheological behavior predictions.