Adversarial Multi-task Learning Enhanced Physics-informed Neural Networks for Solving Partial Differential Equations

In this paper, we propose a method to construct invariant finite-difference schemes for solution of partial differential equations (PDEs) via consideration of modified forms of the underlying PDEs. The invariant schemes, which preserve Lie symmetries, are obtained based on the method of equivariant moving frames. While it is often difficult to construct invariant numerical schemes for PDEs due to complicated symmetry groups associated with cumbersome discrete variable transformations, we note that symmetries associated with more convenient transformations can often be obtained by appropriately modifying the original PDEs. In some cases, modifications to the original PDEs are also found to be useful in order to avoid trivial solutions that might arise from particular selections of moving frames. In our proposed method, modified forms of PDEs can be obtained either by addition of perturbation terms to the original PDEs or through defect correction procedures. These additional terms, whose primary purpose is to enable symmetries with more convenient transformations, are then removed from the system by considering moving frames for which these specific terms go to zero. Further, we explore selection of appropriate moving frames that result in improvement in accuracy of invariant numerical schemes based on modified PDEs. The proposed method is tested using the linear advection equation (in one- and two-dimensions) and the inviscid Burgers' equation. Results obtained for these tests cases indicate that numerical schemes derived from the proposed method perform significantly better than existing schemes not only by virtue of improvement in numerical accuracy but also due to preservation of qualitative properties or symmetries of the underlying differential equations.

SONets: Sub-operator learning enhanced neural networks for solving parametric partial differential equations

The method of Lyapunov functions is used to investigate the stability of systems with distributed and lumped parameters, described by linear partial and ordinary differential equations. The original high-order partial differential equations are represented by a first-order system of partial differential evolution equations and constraint equations; to do that, we introduce additional variables. Passing to the first-order system of partial differential equations and representing ordinary differential equations in the normal Cauchy form, we obtain a possibility to construct the Lyapunov function as a sum of integral and classical quadratic forms and develop general methods of the investigation of the stability of a broad class of systems with distributed and lumped parameters. For example, we consider the stability of the work of the wind-driven lift pump and take into account the elasticity of the shaft transmitting the torque from the wind engine to the pump.

The exploration of deep learning methodologies has recently generated significant interest in the use of Physics-Informed Neural Networks (PINNs) to address complex physical problems governed by partial differential equations (PDEs). The PINN is trained using information from physical laws, including governing PDEs, boundary conditions, and initial conditions. However, achieving a well-trained PINN typically necessitates an appropriate balance between the weights of each loss function, which can considerably increase manual effort. This paper introduces a novel training approach that integrates non-dimensionalization with a multi-task learning technique, termed Automatic Weighted Loss (AWL), to autonomously achieve an optimal balance for each loss function. In the baseline PINN training for solving time-dependent PDEs, multiple weights (usually more than six) must be manually tuned for the model, whereas this method can reduce the number of scaling weights to only one. The proposed approach, referred to as the Non-dimensionalization Automatic Weighted Loss (NDAWL), is evaluated through six free-surface flow problems modelled by the Shallow Water Equations (SWEs). Furthermore, a comparative analysis is conducted between the solutions obtained using NDAWL-PINN and those from the original PINN, which relies on manual fine-tuning of loss functions. The numerical results indicate that the NDAWL-PINN method achieves comparable or superior accuracy to the original PINN, demonstrating its effectiveness in automating the balancing of loss functions.

In recent years, successful applications of deep learning (DL) have inspired scientists to explore the possibilities of applying DL approaches to modeling scientific problems. Existing studies have revealed that to bake the physics into the DL makes a good supplement to the traditional numerical methods (e.g., finite element, finite volume method) which primarily rely on partial differential equations (PDEs). While DL models are ordinarily trained in a purely data-driven manner, integrating physics into them for simulating scientific problems has several benefits such as (i) physics constraints could regularize the over-parameterized model and hence mitigate the overfitting issue commonly seen in DL; (ii) physics information could also effectively reduce the amount of data needed for training the model; (iii) the resultant physics-informed DL models feature better interpretability and generalizability compared with the conventional black-box model. Furthermore, the powerful expressiveness of the deep network, guaranteed by the universal approximation theorem, makes it a suitable approximator for the solution to a physical system. In this dissertation, we develop two different DL architectures (or approaches), one being continuous scheme-based while the other discrete scheme-based, that leverage physics knowledge for modeling scientific problems. Through comprehensive numerical experiments, we demonstrate the proposed models can be used in solving general PDEs, establishing predictive data-driven models for dynamical systems, identifying the parameters in governing PDEs or even discovering the entire governing PDEs of dynamical systems from scarce and noisy measurements. The continuous model roots on the physics-informed neural network (PINN) which uses a fully connected neural network (FCNN) to approximate the physical fields of a system globally. This model is mesh-free as the residual of the physics (e.g., PDEs, initial/boundary values) is evaluated on a set of collocation points within the physical domain. Several applications including the forward simulations, data-driven simulations and solving inverse problems are presented to exemplify the advantages of PINN over traditional numerical methods. However, the original PINN suffers from inaccurate initial/boundary values due to the weak enforcement of the initial/boundary conditions (I/BCs). To overcome this issue, we propose an improved PINN model by utilizing multiple deep neural networks (DNNs) to construct the solution. Through a DNN pre-trained to represent the initial/boundary values, the approximated solution would obey the given I/BCs forcibly. With several numerical examples, we show that the improved PINN is characterized with much better accuracy on the I/BCs. Though the PINN shows great promise in data-driven modeling and solving inverse problems, some inherent limitations of PINN still exist, such as (i) the solution might lacks fine-scale details due to the global approximation of FCNN; (ii) high computational expense caused by the FCNN it roots on; (iii) incapability to incorporate existing PDE terms (e.g., $\Delta u$) into the network architecture. To overcome these drawbacks, this dissertation also proposes a discrete model - Physics-encoded Recurrent Convolutional Neural Network (PeRCNN) which recurrently updates the solution (or state variable) for time marching. Specifically, it utilizes convolutional neural network (CNN) to capture the spatial patterns of the solution while the recurrent network mimics the forward Euler scheme (or Runge-Kutta scheme) in numerical methods. PeRCNN is a mesh-based and discrete model due to the discretization in time and spatial dimension. The local connectivity of CNN makes PeRCNN more computationally efficient. In addition, the coercive encoding mechanism of physics in PeRCNN, fundamentally different from the PINN relying on soft penalty, ensures the network to rigorously obey given physics. The proposed PeRCNN is successfully applied to solving general PDEs, the data-driven modeling of dynamical systems and the data-driven discovery of governing PDEs from scarce and noisy measurements. Comparisons with the state-of-the-art DL models demonstrate that the proposed PeRCNN possesses excellent computational efficiency, accuracy and generalizability. --Author's abstract

Partial differential equations (PDEs) are core tools for characterizing physical laws and are widely used in fluid mechanics, power systems, additive manufacturing, and other fields. However, traditional numerical methods are limited by mesh partitioning, leading to a trade-off between accuracy and efficiency in complex geometric domains and high-dimensional problems. While Physically Informed Neural Networks (PINNs) achieve meshless PDE solutions by embedding prior physical knowledge, they still face challenges such as insufficient accuracy, low training efficiency, and poor stability. This study aims to address the core bottlenecks of PINN in solving PDEs by proposing a systematic optimization strategy to improve its numerical solution accuracy, efficiency, and stability. Methodologically, firstly, a composite loss function containing PDE residuals and boundary initial condition constraints is constructed based on automatic differentiation techniques to ensure that the network satisfies physical laws. Secondly, the PINN training system is optimized, including using Neural Architecture Search (NAS-PINN) to automatically match the optimal network structure, designing an adaptive data sampling strategy to improve data utilization, and introducing parallel computing and hardware acceleration techniques to reduce training time. Finally, the effectiveness of the strategy is verified through classical PDEs and engineering problems. The results show that the optimized PINN achieves high-precision numerical approximation in classical PDE solutions and can efficiently solve forward design and backward parameter inversion problems in engineering scenarios. Compared with traditional methods, it exhibits stronger adaptability in complex geometric domains and high-dimensional problems, while improving training efficiency by more than 30% and significantly enhancing stability. This research not only enriches the optimization theory of PINN and provides an efficient new path for PDE numerical solutions, but also promotes the practical application of PINN in engineering fields such as fluid mechanics, additive manufacturing, and power systems, possessing significant theoretical value and application significance.

Cellular Neural Network (CNN) has been used for solving Partial Differential Equations (PDE). However, the equivalence and stability of system should be considered carefully in a particular problem. In this paper, we introduce the model CNN for solving set of two PDEs describing water flow channels (called Saint Venant equation). We analyze the approximation and topological equivalence issues between Cellular Partial Difference Differential Equation (CPDDE) and its original PDEs. The stability of CNN system is also proved from discovering the equilibrium of the state and output of each cell. The paper has 4 parts. After introduction, part 2 gives a two-layered CNN 1D model for solving PDE Saint Venant equation. In the part 3 the equivalence and stability of the CNN model are proved, then simulation using FPGA. The conclusions are given in the last part.

In this paper, we propose an extrinsic approach based on physics-informed neural networks (PINNs) for solving the partial differential equations (PDEs) on surfaces embedded in high dimensional space. PINNs are one of the deep learning-based techniques. Based on the training data and physical models, PINNs introduce the standard feedforward neural networks (NNs) to approximate the solutions to the PDE systems. Using automatic differentiation, the PDEs information could be encoded into NNs and a loss function. To deal with the surface differential operators in the loss function, we combine the extrinsic approach with PINNs and then express that loss function in extrinsic form. Subsequently, the loss function could be minimized extrinsically with respect to the NN parameters. Numerical results demonstrate that the extrinsic approach based on PINNs for surface problems has good accuracy and higher efficiency compared with the embedding approach based on PINNs. In addition, the strong nonlinear mapping ability of NNs makes this approach robust in solving time-dependent nonlinear problems on more complex surfaces.

The paper considers a mixed initial-boundary value problem for a parabolic equation with nonhomogeneous boundary conditions. The classical approach to search for analytical solution of such problems in the first phase involves variable substitution, leading to a problem with homogeneous boundary conditions. Reference materials [1] give, as a rule, the simplest types of variable substitutions where new and old unknown functions differ by a term, linear in the spatial variable. The form of this additive term depends on the type of the boundary conditions, but is in no way related to the equation under consideration. Moreover, in the case of the second boundary-value problem, it is necessary to use a quadratic additive, since a linear substitution for this type of conditions may be unavailable. The courseware [2] - [4], usually, ends only with the first boundary-value problem generally formulated. The paper considers a substitution that takes into account, in principle, the form of a linear differential operator. Namely, as an additive term, it is proposed to use the parametrically time-dependent solution of the boundary value problem for an ordinary differential equation obtained from the original partial differential equation by the method of separation of the Fourier variables. The existence of the proposed substitution for boundary conditions of any type is proved by the example of a non-stationary heat-transfer equation with the heat exchange available with the surrounding medium. In this case, the additive term is a linear combination of hyperbolic functions. It is shown that, in addition to the insensitivity to the type of boundary conditions, the advantages of a new substitution in comparison with the traditional linear (or quadratic) one include a much simpler structure of the solution obtained. Just the described approach allows us to obtain a solution with a clearly distinguished stationary component, in case a stationarity occurs, for values of the time variable that exceeds relaxation time. This effect extends not only to parabolic equations, but is also observed in equations describing oscillatory processes with damping. It is also shown that in the simplest particular case of an equation with constant coefficients without heat exchange with the surrounding medium, the proposed substitution becomes appropriately linear and coincides with the classical one. The use of the proposed approach allows students majoring in engineering to expand a range of educational and research tasks while learning a course in partial differential equations.

Aiming at the heat conduction equation, the heat conduction model is a very typical example of the partial differential equation. This paper focuses on the development and preliminary knowledge of the partial differential equation, the establishment of the heat conduction model and the solution of the heat conduction model. The partial differential equation with fractional Laplace operator is a kind of typical fractional partial differential equation, which has important applications in the fields of science and engineering. Fractional Laplace operator is a kind of nonlocal quasi-differential operator, which is the infinitesimal generator of L é vy steady-state process. It is essentially different from the classical Laplace operator, which leads to the disappearance of some classical properties, which generally brings difficulties to the study of this kind of problems. It is very difficult to solve the explicit solution of partial differential equation with fractional Laplace operator. In order to improve the accuracy and speed of solving partial differential equation, a high precision method for solving partial differential equation based on deep learning network is proposed. A multilayer radial basis function neural network is constructed and its structural layers are determined. By introducing subnetworks into each layer of the multilayer radial basis function neural network, a composite multilayer radial basis function neural network is constructed to improve the real function approximation performance and operation accuracy of the multilayer radial basis function neural network. The high-precision compound multi-layer radial basis function neural network is used to solve the partial differential equation. By giving a specific example of solving the partial differential equation, the solution effect of the method is tested. The results show that the method has very high solution accuracy under the four-layer network structure, which can improve about 1.5 orders of magnitude compared with the first layer. Under different number of training samples, it has higher solution accuracy and speed, and the comprehensive performance is superior. Therefore, it is a practical but challenging work to study the existence of solutions of partial differential equations with fractional Laplace operators. This paper mainly describes the research progress and trends of the existence of solutions of several classes of partial differential equations with fractional Laplace operators, including some of the work done by the author in this field in recent years.

A novel sequential method to train physics informed neural networks for Allen Cahn and Cahn Hilliard equations

The curse-of-dimensionality taxes computational resources heavily with exponentially increasing computational cost as the dimension increases. This poses great challenges in solving high-dimensional partial differential equations (PDEs), as Richard E. Bellman first pointed out over 60 years ago. While there has been some recent success in solving numerical PDEs in high dimensions, such computations are prohibitively expensive, and true scaling of general nonlinear PDEs to high dimensions has never been achieved. We develop a new method of scaling up physics-informed neural networks (PINNs) to solve arbitrary high-dimensional PDEs. The new method, called Stochastic Dimension Gradient Descent (SDGD), decomposes a gradient of PDEs’ and PINNs’ residual into pieces corresponding to different dimensions and randomly samples a subset of these dimensional pieces in each iteration of training PINNs. We prove theoretically the convergence and other desired properties of the proposed method. We demonstrate in various diverse tests that the proposed method can solve many notoriously hard high-dimensional PDEs, including the Hamilton–Jacobi-Bellman (HJB) and the Schrödinger equations in tens of thousands of dimensions very fast on a single GPU using the PINNs mesh-free approach. Notably, we solve nonlinear PDEs with nontrivial, anisotropic, and inseparable solutions in less than one hour for 1000 dimensions and in 12 h for 100,000 dimensions on a single GPU using SDGD with PINNs. Since SDGD is a general training methodology of PINNs, it can be applied to any current and future variants of PINNs to scale them up for arbitrary high-dimensional PDEs.

Optimal gas-power flow (OGPF) is one of the most fundamental problems for the operation of integrated electric-gas systems (IEGSs), where the gas flow dynamics partial differential equations (PDEs) are computational challenging. This paper aims to propose a lightweight and accurate model for gas flow dynamics for the OGPF problem. Firstly, the second-order hyperbolic PDEs are equivalently derived based on the original PDEs. Then, the second-order hyperbolic PDEs are discretized using the tri-layer explicit difference scheme. Given some mild boundary and initial conditions, the time-domain two-port model for gas flow dynamics is established, where the state variables of the in-between segments of the pipeline are eliminated. Finally, the iterative parameter updating procedure is devised to improve the modeling accuracy. The lightweight feature of the proposed gas flow dynamics model is verified by the simulation results of several test systems.

HL-nets: Physics-informed neural networks for hydrodynamic lubrication with cavitation

<p dir="ltr"><span>PINNs (Physics-Informed Neural Networks) are neural networks designed to solve Partial Differential Equations (PDEs) by integrating physical knowledge into the learning framework. Constructing a PINN involves defining a neural network to approximate the PDE solution, with the total loss calculated as a combination of the losses associated with the PDE, boundary conditions, initial conditions, and measured data. This concept is employed in practical applications to solve various PDEs, such as the Black-Scholes and Heston equations, which are fundamental in financial option pricing. This approach enables the modelling and pricing of financial options, with the added advantage of parallelising the training process across multiple economic scenarios.</span></p><div><span><br /></span></div>