Predicting positon solutions of a family of nonlinear Schrödinger equations through deep learning algorithm

In recent engineering applications using deep learning, physics-informed neural network (PINN) is a new development as it can exploit the underlying physics of engineering systems. The novelty of PINN lies in the use of partial differential equations (PDE) for the loss function. Most PINNs are implemented using automatic differentiation (AD) for training the PDE loss functions. A lesser well-known study is the use of finite difference method (FDM) as an alternative. Unlike an AD based PINN, an immediate benefit of using a FDM based PINN is low implementation cost. In this paper, we propose the use of finite difference method for estimating the PDE loss functions in PINN. Our work is inspired by computational analysis in electromagnetic systems that traditionally solve Laplace’s equation using successive over-relaxation. In the case of Laplace’s equation, our PINN approach can be seen as taking the Laplacian filter response of the neural network output as the loss function. Thus, the implementation of PINN can be very simple. In our experiments, we tested PINN on Laplace’s equation and Burger’s equation. We showed that using FDM, PINN consistently outperforms non-PINN based deep learning. When comparing to AD based PINNs, we showed that our method is faster to compute as well as on par in terms of error reduction.

In recent years, successful applications of deep learning (DL) have inspired scientists to explore the possibilities of applying DL approaches to modeling scientific problems. Existing studies have revealed that to bake the physics into the DL makes a good supplement to the traditional numerical methods (e.g., finite element, finite volume method) which primarily rely on partial differential equations (PDEs). While DL models are ordinarily trained in a purely data-driven manner, integrating physics into them for simulating scientific problems has several benefits such as (i) physics constraints could regularize the over-parameterized model and hence mitigate the overfitting issue commonly seen in DL; (ii) physics information could also effectively reduce the amount of data needed for training the model; (iii) the resultant physics-informed DL models feature better interpretability and generalizability compared with the conventional black-box model. Furthermore, the powerful expressiveness of the deep network, guaranteed by the universal approximation theorem, makes it a suitable approximator for the solution to a physical system. In this dissertation, we develop two different DL architectures (or approaches), one being continuous scheme-based while the other discrete scheme-based, that leverage physics knowledge for modeling scientific problems. Through comprehensive numerical experiments, we demonstrate the proposed models can be used in solving general PDEs, establishing predictive data-driven models for dynamical systems, identifying the parameters in governing PDEs or even discovering the entire governing PDEs of dynamical systems from scarce and noisy measurements. The continuous model roots on the physics-informed neural network (PINN) which uses a fully connected neural network (FCNN) to approximate the physical fields of a system globally. This model is mesh-free as the residual of the physics (e.g., PDEs, initial/boundary values) is evaluated on a set of collocation points within the physical domain. Several applications including the forward simulations, data-driven simulations and solving inverse problems are presented to exemplify the advantages of PINN over traditional numerical methods. However, the original PINN suffers from inaccurate initial/boundary values due to the weak enforcement of the initial/boundary conditions (I/BCs). To overcome this issue, we propose an improved PINN model by utilizing multiple deep neural networks (DNNs) to construct the solution. Through a DNN pre-trained to represent the initial/boundary values, the approximated solution would obey the given I/BCs forcibly. With several numerical examples, we show that the improved PINN is characterized with much better accuracy on the I/BCs. Though the PINN shows great promise in data-driven modeling and solving inverse problems, some inherent limitations of PINN still exist, such as (i) the solution might lacks fine-scale details due to the global approximation of FCNN; (ii) high computational expense caused by the FCNN it roots on; (iii) incapability to incorporate existing PDE terms (e.g., $\Delta u$) into the network architecture. To overcome these drawbacks, this dissertation also proposes a discrete model - Physics-encoded Recurrent Convolutional Neural Network (PeRCNN) which recurrently updates the solution (or state variable) for time marching. Specifically, it utilizes convolutional neural network (CNN) to capture the spatial patterns of the solution while the recurrent network mimics the forward Euler scheme (or Runge-Kutta scheme) in numerical methods. PeRCNN is a mesh-based and discrete model due to the discretization in time and spatial dimension. The local connectivity of CNN makes PeRCNN more computationally efficient. In addition, the coercive encoding mechanism of physics in PeRCNN, fundamentally different from the PINN relying on soft penalty, ensures the network to rigorously obey given physics. The proposed PeRCNN is successfully applied to solving general PDEs, the data-driven modeling of dynamical systems and the data-driven discovery of governing PDEs from scarce and noisy measurements. Comparisons with the state-of-the-art DL models demonstrate that the proposed PeRCNN possesses excellent computational efficiency, accuracy and generalizability. --Author's abstract

The generalization of Physics-Informed Neural Networks (PINNs) used to solve the inhomogeneous Helmholtz equation in a simplified three-dimensional room is investigated. PINNs are appealing since they can efficiently integrate a partial differential equation and experimental data by minimizing a loss function. However, a previous study experienced limitations in acoustics regarding the source term. A challenging but realistic excitation case is a confined (e.g., single-point) excitation area, yielding a smooth spatial wave field periodically with the wavelength. Compared to studies using smooth (unrealistic) sound excitation, the network’s generalization capabilities regarding a realistic sound excitation are addressed. Different methods like hyperparameter optimization, adaptive refinement, Fourier feature engineering, and locally adaptive activation functions with slope recovery are tested to tailor the PINN’s accuracy to an experimentally validated finite element analysis reference solution computed with openCFS. The hyperparameter study and optimization are conducted regarding the network depth and width, the learning rate, the used activation functions, and the deep learning backends (PyTorch 2.5.1, TensorFlow 2.18.0 1, TensorFlow 2.18.0 2, JAX 0.4.39). A modified (feature-engineered) PINN architecture was designed using input feature engineering to include the dispersion relation of the wave in the neural network. For smoothly (unrealistic) distributed sources, it was shown that the standard PINNs and the feature-engineered PINN converge to the analytic solution, with a relative error of 0.28% and 2×10−4%, respectively. The locally adaptive activation functions with the slope lead to a relative error of 0.086% with a source sharpness of s=1 m. Similar relative errors were obtained for the case s=0.2 m using adaptive refinement. The feature-engineered PINN significantly outperformed the results of previous studies regarding accuracy. Furthermore, the trainable parameters were reduced to a fraction by Bayesian hyperparameter optimization (around 5%), and likewise, the training time (around 3%) was reduced compared to the standard PINN formulation. By narrowing this excitation towards a single point, the convergence rate and minimum errors obtained of all presented network architectures increased. The feature-engineered architecture yielded a one order of magnitude lower accuracy of 0.20% compared to 0.019% of the standard PINN formulation with a source sharpness of s=1 m. It outperformed the finite element analysis and the standard PINN in terms time needed to obtain the solution, needing 15 min and 30 s on an AMD Ryzen 7 Pro 8840HS CPU (AMD, Santa Clara, CA, USA) for the FEM, compared to about 20 min (standard PINN) and just under a minute of the feature-engineered PINN, both trained on a Tesla T4 GPU (NVIDIA, Santa Clara, CA, USA).

This thesis investigates two physics informed neural network (PINN) approaches for two mechanistic problems via the performance metrics of model parameter estimation and system state prediction. The PINNs are compared to Bayesian inference (BI) tested under similar conditions. The first PINN studied is the loss-based PINN, which embeds physics into a standard feedforward neural network within the loss function of this network. The second PINN investigated is the hybrid-recurrent physics informed neural network, which uses a recurrent neural network (RNN) that embeds physics in RNN architecture. The BI used as a point of comparison uses a Transitional Markov Chain Monte Carlo algorithm to estimate the posterior distributions. The applications studied are a compartmental model for an infectious disease, and a mass-spring-damper model. It is found that BI performs better in ideal data conditions and the PINNs perform better in weak data conditions.

Controlling nonlinear dynamics arises in various engineering fields. We present efforts to model the forced van der Pol system control using physics-informed neural networks (PINN) compared to benchmark methods, including idealized nonlinear feedforward (FF) control, linearized feedback control (FB), and feedforward-plus-feedback combined (C). The aim is to implement circular trajectories in the state space of the van der Pol system. A designed benchmark problem is used for testing the behavioral differences of the disparate controllers and then investigating controlled schemes and systems of various extents of nonlinearities. All methods exhibit a short initialization accompanying arbitrary initialization points. The feedforward control successfully converges to the desired trajectory, and PINN executes good controls with higher stochasticity observed for higher-order terms based on the phase portraits. In contrast, linearized feedback control and combined feed-forward plus feedback failed. Varying trajectory amplitudes revealed that feed-forward, linearized feedback control, and combined feed-forward plus feedback control all fail for unity nonlinear damping gain. Traditional control methods display a robust fluctuation for higher-order terms. For some various nonlinearities, PINN failed to implement the desired trajectory instead of becoming “trapped” in the phase of small radius, yet idealized nonlinear feedforward successfully implemented controls. PINN generally exhibits lower relative errors for varying targeted trajectories. However, PINN also shows evidently higher computational burden compared with traditional control theory methods, with at least more than 30 times longer control time compared with benchmark idealized nonlinear feed-forward control. This manuscript proposes a comprehensive comparative study for future controller employment considering deterministic and machine learning approaches.

In recent years, the artificial intelligence computing paradigm represented by physics-informed neural networks (PINNs) has received great attention in the field of plasma numerical simulation. However, the plasma chemical system considered in related research is relatively simplified, and the research on solving the more complex multi-particle low-temperature fluid model based on PINNs is still blank. In more complex chemical systems, the coupling relationship between particle densities and between particle densities and mean electron energy become more intricate. Therefore, the applicability of PINNs in dealing with sophisticated reaction systems needs further exploring and improving. In this work, we propose a general PINN framework (source term decoupled PINNs, Std-PINNs) for solving multi-particle low-temperature plasma fluid model. By introducing equivalent positive ions and replacing each particle transport equation with the current continuity equation as a physical constraint, Std-PINN splits the entire solution process into the training processes of two neural networks, realizing the decoupling of the source term of the heavy particle transport equation from the electron density and mean electron energy, which greatly reduces the complexity of neural network training. In this work, the application of Std-PINNs to solving multi-particle low-temperature plasma fluid models is demonstrated through two classic discharge cases with different complexity of reaction systems (low-pressure argon glow discharge and atmospheric-pressure helium glow discharge) and the performance of Std-PINN is compared with that of conventional PINN and finite element method (FEM). The results show that the training results output from the traditional PINN are completely incorrect due to the strong coupling correlation of each physical variable through the source terms of each particle transport equation, while the <i>L</i><sub>2</sub> relative error between Std-PINN and FEM results can reach up to ~10<sup>–2</sup> , thus verifying the feasibility of Std-PINN in simulating multi-particle plasma fluid model. Std-PINN expands the application of deep learning method to modeling complex physical systems and provides new ideas for conducting low-temperature plasma simulations. In addition, this study provides novel insights into the field of artificial intelligence scientific computing: the mathematical form that describes the state of a physical system is not unique. By introducing equivalent physical variables, equations suitable for neural network solutions can be derived and combined with observable data to simplify problems.

Simulating solute transport in heterogeneous porous media poses computational challenges due to the high-resolution meshing required for traditional solvers. To overcome these challenges, this study explores a mesh-free method based on deep learning to accelerate solute transport simulation. We employ Physics-informed Neural Networks (PiNN) with a periodic activation function to solve solute transport problems in both homogeneous and heterogeneous porous media governed by the advection-dispersion equation. Unlike traditional neural networks that rely on large training datasets, PiNNs use strong-form mathematical models to constrain the network in the training phase and simultaneously solve for multiple dependent or independent field variables, such as pressure and solute concentration fields. To demonstrate the effectiveness of using PiNNs with a periodic activation function to resolve solute transport in porous media, we construct PiNNs using two activation functions, sin and tanh, for seven case studies, including 1D and 2D scenarios. The accuracy of the PiNNs’ predictions is then evaluated using absolute point error and mean square error metrics and compared to the ground truth solutions obtained analytically or numerically. Our results demonstrate that the PiNN with sin activation function, compared to tanh activation function, is up to two orders of magnitude more accurate and up to two times faster to train, especially in heterogeneous porous media. Moreover, PiNN’s simultaneous predictions of pressure and concentration fields can reduce computational expenses in terms of inference time by three orders of magnitude compared to FEM simulations for two-dimensional cases.

Numerical simulation and experimental techniques are the primary methods for solving fluid dynamics problems. However, while numerical simulation approaches are sensitive when meshing a complex structure, experimental methods have difficulty simulating the physical challenges. Therefore, building an affordable model to solve the fluid dynamics problem is very important. Deep learning (DL) approaches have great abilities to handle strong nonlinearity and high dimensionality that attract much attention for solving fluid dynamics problems. In this paper, we used a deep learning-based framework, physics-informed neural networks (PINNs). The main idea of PINN approaches is to encode the underlying physical law (i.e., the partial differential equation) into the neural network as prior information. In the oil recovery process involving the injection of fluids and multiphase flow in porous media, fingering instability is observed if a fluid with low viscosity displaces a high viscosity fluid. This paper provides a deep learning framework to simulate the instability (fingering) phenomenon during secondary and enhanced oil recovery methods. We have considered two models, namely, the first model on immiscible multiphase flow from the secondary oil recovery method, which is analyzed both with and without deliberating mass flow rate. In the second model, instability arising during the enhanced oil recovery method involving miscible displacement of multiphase is examined, focusing on oil recovery using carbonated water injection. We solve the governing nonlinear partial differential equation using PINNs. Furthermore, we have compared results from PINNs with the semi-analytical solution from the literature. The results show that PINNs are very effective in fluid flow problems and deserve further research.

A nonlocal physics-informed deep learning framework using the peridynamic differential operator

The nonlinear telegraph equation appears in a variety of engineering and science problems. This paper presents a deep learning algorithm termed multi-term physics-informed neural networks to resolve initial boundary value problems of 2D and 3D hyperbolic nonlinear telegraph equations. The standard physics-informed neural networks include the gradients of residuals of the problem, initial conditions, and boundary conditions directly into the training process in addition to the standard PINNs loss function terms. Using multiple densely connected neural networks, termed feed-forward deep neural networks, the proposed scheme has been trained to minimize the total loss results from the multi-term loss function. Three computational examples are provided to demonstrate the efficacy and applications of our suggested method. For each illustrative example considered, we compared the proposed scheme with the standard physics-informed neural networks based on the network model train and test losses and other well-known performance measure error analysis including each problems with long-time domain. The results demonstrate that the proposed method outperforms the conventional deep learning approach physics-informed neural networks in predicting the solutions of both initial boundary value problems of telegraph equations. Therefore, the proposed computational approach can offer precise and stable solutions for long-time domain PDEs and is efficient in handling challenging nonlinear physical problems with a variety of boundary conditions.

This study works on how to simulate a flow field in a data-driven manner using a deep learning method called Physics-Informed Neural Networks (PINNs). This method trains a neural network, which inputs time and space and outputs the corresponding flow quantities by satisfying the Navier-Stokes equations, boundary conditions, and initial conditions. Then it can quickly reproduce a flow field in any time and space. The application of PINNs to engineering problems is an area for further investigation since PINNs need to be re-trained whenever the boundary conditions change, i.e., the training cost should be as low as possible. Hence, this study considers how to reduce the training cost of PINNs for an airfoil by adaptively arranging the collocation points. At present, this paper examines PINNs for a laminar flat plate as a preliminary step. The results show that, for qualitatively accurate PINNs, it is influential to consider the stagnation pressure at the leading edge of the plate as a constraint in the PINNs loss function.

Accurate and rapid prediction of structural responses to seismic loading is critical for ensuring structural safety. Recently, there has been active research focusing on the application of deep learning techniques, including Physics-Informed Neural Networks (PINNs) and Long Short-Term Memory (LSTM) networks, to predict the dynamic behavior of structures. While these methods have shown promise, each comes with distinct limitations. PINNs offer physical consistency but struggle with capturing long-term temporal dependencies in nonlinear systems, while LSTMs excel in learning sequential data but lack physical interpretability. To address these complementary limitations, this study proposes a hybrid LSTM-PINN model, combining the temporal learning ability of LSTMs with the physics-based constraints of PINNs. This hybrid approach allows the model to capture both nonlinear, time-dependent behaviors and maintain physical consistency. The proposed model is evaluated on both single-degree-of-freedom (SDOF) and multi-degree-of-freedom (MDOF) structural systems subjected to the El-Centro ground motion. For validation, the 1940 El-Centro NS earthquake record was used, and the ground acceleration data were normalized and discretized for numerical simulation. The proposed LSTM-PINN is trained under the same conditions as the conventional PINN models (e.g., same optimizer, learning rate, and loss structure), but with fewer training epochs, to evaluate learning efficiency. Prediction accuracy is quantitatively assessed using mean error and mean squared error (MSE) for displacement, velocity, and acceleration, and results are compared with PINN-only models (PINN-1, PINN-2). The results show that LSTM-PINN consistently achieves the most stable and precise predictions across the entire time domain. Notably, it outperforms the baseline PINNs even with fewer training epochs. Specifically, it achieved up to 50% lower MSE with only 10,000 epochs, compared to the PINN's 50,000 epochs, demonstrating improved generalization through temporal sequence learning. This study empirically validates the potential of physics-guided time-series AI models for dynamic structural response prediction. The proposed approach is expected to contribute to future applications such as real-time response estimation, structural health monitoring, and seismic performance evaluation.

An analysis was made of physics-informed neural networks used to solve partial differential equations. The prospects for the implementation of physics-informed neural networks in the MATLAB system are shown. An algorithm for solving partial differential equations in MATLAB using physics-informed neural networks has been developed. On the example of a model problem described by the Poisson equation, physics-informed neural networks were implemented and studied, which showed that MATLAB can be successfully used to implement such networks. MATLAB made it possible to solve the Poisson equation up to the mean square value of the loss function equal to 0.01. The best results were obtained by networks with a small number of layers (3–4) and a sufficiently large number of neurons in each layer (50–100). Comparison with known results showed that MATLAB was inferior to TensorFlow in terms of learning speed. The application of the Optimization Toolbox MATLAB for the implementation of the L-BFGS quasi-Newtonian learning algorithm for physics-informed neural networks was studied. The quasi-Newtonian algorithm makes it possible to increase the accuracy of solving the problem, but requires a lot of training time. As further research, it is recommended to expand the capabilities of the Deep Learning Toolbox by including quasi-Newtonian learning algorithms, in particular, the Levenberg-Marquard algorithm, and new neural network architectures, for example, networks of radial basis functions.

We devise a deep learning solver inspired by physics-informed neural networks (PINNs) to tackle the polymer self-consistent field theory (SCFT) equationsfor one-dimensional AB-diblock copolymers. The PINNs framework comprises two parallel feedforward neural networks that separately represent the segmental partition functions and self-consistent chemical potential fields. The two networks are coupled through a loss function incorporating the governing equation, initial and boundary conditions, and the incompressibility constraint. To avoid the metastable homogeneous solution, the network parameters are initialized based on known self-consistent fields obtained from the numerical pseudospectral method. For copolymers of length N at a given volume fraction of A block (f) and the reduced Flory-Huggins interaction parameter (χN), the minimization of the loss function leads to the converged network parameters that successfully capture the stable lamellar phase. The periodicity of the lamellar structure is correctly reproduced for the explored sets of [f,χN], irrespective of the presumed computational domain size for initialization. Moreover, the proposed PINNs are applicable to the inverse discovery of the interaction parameter and the embedded chemical potential fields for an observed structure. This capability of solving the inverse SCFT problem demonstrates the potential of using PINNs to accelerate the exploration of new polymeric materials.

The exploration of deep learning methodologies has recently generated significant interest in the use of Physics-Informed Neural Networks (PINNs) to address complex physical problems governed by partial differential equations (PDEs). The PINN is trained using information from physical laws, including governing PDEs, boundary conditions, and initial conditions. However, achieving a well-trained PINN typically necessitates an appropriate balance between the weights of each loss function, which can considerably increase manual effort. This paper introduces a novel training approach that integrates non-dimensionalization with a multi-task learning technique, termed Automatic Weighted Loss (AWL), to autonomously achieve an optimal balance for each loss function. In the baseline PINN training for solving time-dependent PDEs, multiple weights (usually more than six) must be manually tuned for the model, whereas this method can reduce the number of scaling weights to only one. The proposed approach, referred to as the Non-dimensionalization Automatic Weighted Loss (NDAWL), is evaluated through six free-surface flow problems modelled by the Shallow Water Equations (SWEs). Furthermore, a comparative analysis is conducted between the solutions obtained using NDAWL-PINN and those from the original PINN, which relies on manual fine-tuning of loss functions. The numerical results indicate that the NDAWL-PINN method achieves comparable or superior accuracy to the original PINN, demonstrating its effectiveness in automating the balancing of loss functions.