A data-driven tracking control framework using physics-informed neural networks and deep reinforcement learning for dynamical systems

In recent years, successful applications of deep learning (DL) have inspired scientists to explore the possibilities of applying DL approaches to modeling scientific problems. Existing studies have revealed that to bake the physics into the DL makes a good supplement to the traditional numerical methods (e.g., finite element, finite volume method) which primarily rely on partial differential equations (PDEs). While DL models are ordinarily trained in a purely data-driven manner, integrating physics into them for simulating scientific problems has several benefits such as (i) physics constraints could regularize the over-parameterized model and hence mitigate the overfitting issue commonly seen in DL; (ii) physics information could also effectively reduce the amount of data needed for training the model; (iii) the resultant physics-informed DL models feature better interpretability and generalizability compared with the conventional black-box model. Furthermore, the powerful expressiveness of the deep network, guaranteed by the universal approximation theorem, makes it a suitable approximator for the solution to a physical system. In this dissertation, we develop two different DL architectures (or approaches), one being continuous scheme-based while the other discrete scheme-based, that leverage physics knowledge for modeling scientific problems. Through comprehensive numerical experiments, we demonstrate the proposed models can be used in solving general PDEs, establishing predictive data-driven models for dynamical systems, identifying the parameters in governing PDEs or even discovering the entire governing PDEs of dynamical systems from scarce and noisy measurements. The continuous model roots on the physics-informed neural network (PINN) which uses a fully connected neural network (FCNN) to approximate the physical fields of a system globally. This model is mesh-free as the residual of the physics (e.g., PDEs, initial/boundary values) is evaluated on a set of collocation points within the physical domain. Several applications including the forward simulations, data-driven simulations and solving inverse problems are presented to exemplify the advantages of PINN over traditional numerical methods. However, the original PINN suffers from inaccurate initial/boundary values due to the weak enforcement of the initial/boundary conditions (I/BCs). To overcome this issue, we propose an improved PINN model by utilizing multiple deep neural networks (DNNs) to construct the solution. Through a DNN pre-trained to represent the initial/boundary values, the approximated solution would obey the given I/BCs forcibly. With several numerical examples, we show that the improved PINN is characterized with much better accuracy on the I/BCs. Though the PINN shows great promise in data-driven modeling and solving inverse problems, some inherent limitations of PINN still exist, such as (i) the solution might lacks fine-scale details due to the global approximation of FCNN; (ii) high computational expense caused by the FCNN it roots on; (iii) incapability to incorporate existing PDE terms (e.g., $\Delta u$) into the network architecture. To overcome these drawbacks, this dissertation also proposes a discrete model - Physics-encoded Recurrent Convolutional Neural Network (PeRCNN) which recurrently updates the solution (or state variable) for time marching. Specifically, it utilizes convolutional neural network (CNN) to capture the spatial patterns of the solution while the recurrent network mimics the forward Euler scheme (or Runge-Kutta scheme) in numerical methods. PeRCNN is a mesh-based and discrete model due to the discretization in time and spatial dimension. The local connectivity of CNN makes PeRCNN more computationally efficient. In addition, the coercive encoding mechanism of physics in PeRCNN, fundamentally different from the PINN relying on soft penalty, ensures the network to rigorously obey given physics. The proposed PeRCNN is successfully applied to solving general PDEs, the data-driven modeling of dynamical systems and the data-driven discovery of governing PDEs from scarce and noisy measurements. Comparisons with the state-of-the-art DL models demonstrate that the proposed PeRCNN possesses excellent computational efficiency, accuracy and generalizability. --Author's abstract

In recent engineering applications using deep learning, physics-informed neural network (PINN) is a new development as it can exploit the underlying physics of engineering systems. The novelty of PINN lies in the use of partial differential equations (PDE) for the loss function. Most PINNs are implemented using automatic differentiation (AD) for training the PDE loss functions. A lesser well-known study is the use of finite difference method (FDM) as an alternative. Unlike an AD based PINN, an immediate benefit of using a FDM based PINN is low implementation cost. In this paper, we propose the use of finite difference method for estimating the PDE loss functions in PINN. Our work is inspired by computational analysis in electromagnetic systems that traditionally solve Laplace’s equation using successive over-relaxation. In the case of Laplace’s equation, our PINN approach can be seen as taking the Laplacian filter response of the neural network output as the loss function. Thus, the implementation of PINN can be very simple. In our experiments, we tested PINN on Laplace’s equation and Burger’s equation. We showed that using FDM, PINN consistently outperforms non-PINN based deep learning. When comparing to AD based PINNs, we showed that our method is faster to compute as well as on par in terms of error reduction.

Predicting positon solutions of a family of nonlinear Schrödinger equations through deep learning algorithm

The generalization of Physics-Informed Neural Networks (PINNs) used to solve the inhomogeneous Helmholtz equation in a simplified three-dimensional room is investigated. PINNs are appealing since they can efficiently integrate a partial differential equation and experimental data by minimizing a loss function. However, a previous study experienced limitations in acoustics regarding the source term. A challenging but realistic excitation case is a confined (e.g., single-point) excitation area, yielding a smooth spatial wave field periodically with the wavelength. Compared to studies using smooth (unrealistic) sound excitation, the network’s generalization capabilities regarding a realistic sound excitation are addressed. Different methods like hyperparameter optimization, adaptive refinement, Fourier feature engineering, and locally adaptive activation functions with slope recovery are tested to tailor the PINN’s accuracy to an experimentally validated finite element analysis reference solution computed with openCFS. The hyperparameter study and optimization are conducted regarding the network depth and width, the learning rate, the used activation functions, and the deep learning backends (PyTorch 2.5.1, TensorFlow 2.18.0 1, TensorFlow 2.18.0 2, JAX 0.4.39). A modified (feature-engineered) PINN architecture was designed using input feature engineering to include the dispersion relation of the wave in the neural network. For smoothly (unrealistic) distributed sources, it was shown that the standard PINNs and the feature-engineered PINN converge to the analytic solution, with a relative error of 0.28% and 2×10−4%, respectively. The locally adaptive activation functions with the slope lead to a relative error of 0.086% with a source sharpness of s=1 m. Similar relative errors were obtained for the case s=0.2 m using adaptive refinement. The feature-engineered PINN significantly outperformed the results of previous studies regarding accuracy. Furthermore, the trainable parameters were reduced to a fraction by Bayesian hyperparameter optimization (around 5%), and likewise, the training time (around 3%) was reduced compared to the standard PINN formulation. By narrowing this excitation towards a single point, the convergence rate and minimum errors obtained of all presented network architectures increased. The feature-engineered architecture yielded a one order of magnitude lower accuracy of 0.20% compared to 0.019% of the standard PINN formulation with a source sharpness of s=1 m. It outperformed the finite element analysis and the standard PINN in terms time needed to obtain the solution, needing 15 min and 30 s on an AMD Ryzen 7 Pro 8840HS CPU (AMD, Santa Clara, CA, USA) for the FEM, compared to about 20 min (standard PINN) and just under a minute of the feature-engineered PINN, both trained on a Tesla T4 GPU (NVIDIA, Santa Clara, CA, USA).

Numerical simulation and experimental techniques are the primary methods for solving fluid dynamics problems. However, while numerical simulation approaches are sensitive when meshing a complex structure, experimental methods have difficulty simulating the physical challenges. Therefore, building an affordable model to solve the fluid dynamics problem is very important. Deep learning (DL) approaches have great abilities to handle strong nonlinearity and high dimensionality that attract much attention for solving fluid dynamics problems. In this paper, we used a deep learning-based framework, physics-informed neural networks (PINNs). The main idea of PINN approaches is to encode the underlying physical law (i.e., the partial differential equation) into the neural network as prior information. In the oil recovery process involving the injection of fluids and multiphase flow in porous media, fingering instability is observed if a fluid with low viscosity displaces a high viscosity fluid. This paper provides a deep learning framework to simulate the instability (fingering) phenomenon during secondary and enhanced oil recovery methods. We have considered two models, namely, the first model on immiscible multiphase flow from the secondary oil recovery method, which is analyzed both with and without deliberating mass flow rate. In the second model, instability arising during the enhanced oil recovery method involving miscible displacement of multiphase is examined, focusing on oil recovery using carbonated water injection. We solve the governing nonlinear partial differential equation using PINNs. Furthermore, we have compared results from PINNs with the semi-analytical solution from the literature. The results show that PINNs are very effective in fluid flow problems and deserve further research.

In recent years, the artificial intelligence computing paradigm represented by physics-informed neural networks (PINNs) has received great attention in the field of plasma numerical simulation. However, the plasma chemical system considered in related research is relatively simplified, and the research on solving the more complex multi-particle low-temperature fluid model based on PINNs is still blank. In more complex chemical systems, the coupling relationship between particle densities and between particle densities and mean electron energy become more intricate. Therefore, the applicability of PINNs in dealing with sophisticated reaction systems needs further exploring and improving. In this work, we propose a general PINN framework (source term decoupled PINNs, Std-PINNs) for solving multi-particle low-temperature plasma fluid model. By introducing equivalent positive ions and replacing each particle transport equation with the current continuity equation as a physical constraint, Std-PINN splits the entire solution process into the training processes of two neural networks, realizing the decoupling of the source term of the heavy particle transport equation from the electron density and mean electron energy, which greatly reduces the complexity of neural network training. In this work, the application of Std-PINNs to solving multi-particle low-temperature plasma fluid models is demonstrated through two classic discharge cases with different complexity of reaction systems (low-pressure argon glow discharge and atmospheric-pressure helium glow discharge) and the performance of Std-PINN is compared with that of conventional PINN and finite element method (FEM). The results show that the training results output from the traditional PINN are completely incorrect due to the strong coupling correlation of each physical variable through the source terms of each particle transport equation, while the <i>L</i><sub>2</sub> relative error between Std-PINN and FEM results can reach up to ~10<sup>–2</sup> , thus verifying the feasibility of Std-PINN in simulating multi-particle plasma fluid model. Std-PINN expands the application of deep learning method to modeling complex physical systems and provides new ideas for conducting low-temperature plasma simulations. In addition, this study provides novel insights into the field of artificial intelligence scientific computing: the mathematical form that describes the state of a physical system is not unique. By introducing equivalent physical variables, equations suitable for neural network solutions can be derived and combined with observable data to simplify problems.

Simulating solute transport in heterogeneous porous media poses computational challenges due to the high-resolution meshing required for traditional solvers. To overcome these challenges, this study explores a mesh-free method based on deep learning to accelerate solute transport simulation. We employ Physics-informed Neural Networks (PiNN) with a periodic activation function to solve solute transport problems in both homogeneous and heterogeneous porous media governed by the advection-dispersion equation. Unlike traditional neural networks that rely on large training datasets, PiNNs use strong-form mathematical models to constrain the network in the training phase and simultaneously solve for multiple dependent or independent field variables, such as pressure and solute concentration fields. To demonstrate the effectiveness of using PiNNs with a periodic activation function to resolve solute transport in porous media, we construct PiNNs using two activation functions, sin and tanh, for seven case studies, including 1D and 2D scenarios. The accuracy of the PiNNs’ predictions is then evaluated using absolute point error and mean square error metrics and compared to the ground truth solutions obtained analytically or numerically. Our results demonstrate that the PiNN with sin activation function, compared to tanh activation function, is up to two orders of magnitude more accurate and up to two times faster to train, especially in heterogeneous porous media. Moreover, PiNN’s simultaneous predictions of pressure and concentration fields can reduce computational expenses in terms of inference time by three orders of magnitude compared to FEM simulations for two-dimensional cases.

A nonlocal physics-informed deep learning framework using the peridynamic differential operator

Design automation is undergoing a new generation of changes caused by artificial intelligence technologies represented by deep learning and reinforcement learning. Notably, the advantages of deep reinforcement learning in addressing solution optimisation and decision-making tasks with cognitive automation functionality have garnered attention in design. In the context of surrogate model-driven engineering design optimisation, this paper addresses current research challenges such as reliance on domain knowledge for local development, shortcomings in the self-learning and adaptive capabilities of optimisation algorithms for global exploration, etc. Centred around the deep reinforcement learning model, Deep Q-learning, and complemented by self-organising maps and neural network technologies, we propose a methodology considering multi-fidelity simulation data for design space exploration. This approach effectively reduces sampling costs and enables the optimisation model to learn the optimal direction for high-precision predictions and achieve rapid, accurate optimisation. Finally, the effectiveness of the proposed method is comprehensively validated through four typical optimisation scenarios and a case study involving the optimisation of a wheeled robot's suspension swing arm structure. This work will be a crucial reference for applying deep reinforcement learning in simulation-driven engineering design optimisation.

The nonlinear telegraph equation appears in a variety of engineering and science problems. This paper presents a deep learning algorithm termed multi-term physics-informed neural networks to resolve initial boundary value problems of 2D and 3D hyperbolic nonlinear telegraph equations. The standard physics-informed neural networks include the gradients of residuals of the problem, initial conditions, and boundary conditions directly into the training process in addition to the standard PINNs loss function terms. Using multiple densely connected neural networks, termed feed-forward deep neural networks, the proposed scheme has been trained to minimize the total loss results from the multi-term loss function. Three computational examples are provided to demonstrate the efficacy and applications of our suggested method. For each illustrative example considered, we compared the proposed scheme with the standard physics-informed neural networks based on the network model train and test losses and other well-known performance measure error analysis including each problems with long-time domain. The results demonstrate that the proposed method outperforms the conventional deep learning approach physics-informed neural networks in predicting the solutions of both initial boundary value problems of telegraph equations. Therefore, the proposed computational approach can offer precise and stable solutions for long-time domain PDEs and is efficient in handling challenging nonlinear physical problems with a variety of boundary conditions.

This study works on how to simulate a flow field in a data-driven manner using a deep learning method called Physics-Informed Neural Networks (PINNs). This method trains a neural network, which inputs time and space and outputs the corresponding flow quantities by satisfying the Navier-Stokes equations, boundary conditions, and initial conditions. Then it can quickly reproduce a flow field in any time and space. The application of PINNs to engineering problems is an area for further investigation since PINNs need to be re-trained whenever the boundary conditions change, i.e., the training cost should be as low as possible. Hence, this study considers how to reduce the training cost of PINNs for an airfoil by adaptively arranging the collocation points. At present, this paper examines PINNs for a laminar flat plate as a preliminary step. The results show that, for qualitatively accurate PINNs, it is influential to consider the stagnation pressure at the leading edge of the plate as a constraint in the PINNs loss function.

Accurate and rapid prediction of structural responses to seismic loading is critical for ensuring structural safety. Recently, there has been active research focusing on the application of deep learning techniques, including Physics-Informed Neural Networks (PINNs) and Long Short-Term Memory (LSTM) networks, to predict the dynamic behavior of structures. While these methods have shown promise, each comes with distinct limitations. PINNs offer physical consistency but struggle with capturing long-term temporal dependencies in nonlinear systems, while LSTMs excel in learning sequential data but lack physical interpretability. To address these complementary limitations, this study proposes a hybrid LSTM-PINN model, combining the temporal learning ability of LSTMs with the physics-based constraints of PINNs. This hybrid approach allows the model to capture both nonlinear, time-dependent behaviors and maintain physical consistency. The proposed model is evaluated on both single-degree-of-freedom (SDOF) and multi-degree-of-freedom (MDOF) structural systems subjected to the El-Centro ground motion. For validation, the 1940 El-Centro NS earthquake record was used, and the ground acceleration data were normalized and discretized for numerical simulation. The proposed LSTM-PINN is trained under the same conditions as the conventional PINN models (e.g., same optimizer, learning rate, and loss structure), but with fewer training epochs, to evaluate learning efficiency. Prediction accuracy is quantitatively assessed using mean error and mean squared error (MSE) for displacement, velocity, and acceleration, and results are compared with PINN-only models (PINN-1, PINN-2). The results show that LSTM-PINN consistently achieves the most stable and precise predictions across the entire time domain. Notably, it outperforms the baseline PINNs even with fewer training epochs. Specifically, it achieved up to 50% lower MSE with only 10,000 epochs, compared to the PINN's 50,000 epochs, demonstrating improved generalization through temporal sequence learning. This study empirically validates the potential of physics-guided time-series AI models for dynamic structural response prediction. The proposed approach is expected to contribute to future applications such as real-time response estimation, structural health monitoring, and seismic performance evaluation.

Deep reinforcement learning (DRL) is one of the emerging areas of machine learning which focuses on maximized rewards. DRL is a type of machine learning that combines reinforcement learning and deep learning. It uses a series of algorithms to enable an agent to learn how to make decisions in a complex environment. DRL is a subset of artificial intelligence that focuses on making decisions based on the environment and the rewards associated with each action.The goal of DRL is to maximize the long-term reward of an agent. In order to do this, the agent must use a combination of deep learning, reinforcement learning and other AI techniques to learn which actions will lead to the highest reward. DRL is used to solve a variety of problems, from playing video games to controlling robots. It is also used in autonomous driving and robotics, as well as for financial trading. DRL is a powerful tool for solving complex problems and has been used in a variety of research projects. DRL has the potential to revolutionize the way we interact with machines and the environment.

Learning ability is the basic characteristic of human intelligence. The July 1, 2005 issue of Science published a list of 125 important questions in science. Among them, the question 94 “What are the limits of learning by machines?”. The annotation “Computers can already beat the world’s best chess players, and they have a wealth of information on the Web to draw on. But abstract reasoning is still beyond any machine”. In recent artificial intelligence has made great progresses. In 1997, the rise of the man-machine war, IBM Supercomputer Deep Blue defeated the chess master Garry Kasparov. On February 14, 2011, IBM’s Watson supercomputer won a practice round against Jeopardy champions Ken Jennings and Brad Rutter. In March 2016, Google DeepMind’s AlphaGo sealed a 4-1 victory over a South Korean Go grandmaster Lee Se-dol. This paper focuses on the machine learning methods of AlphaGo, including reinforcement learning, deep learning, deep reinforcement learning, analysis of the existing problems and the latest research progress. Deep reinforcement learning is the combination of deep learning and reinforcement learning, which can realize the learning algorithm from the perception to action. Simply said, this is the same as human behavior, input sensing information such as vision, and then, direct output action through the deep neural network. Deep reinforcement learning has the potential to learn a variety of skills for the robot to achieve full autonomy. Even though reinforcement learning is practiced successfully, but feature states need to manually set, for complex scene is a difficult thing, especially easy to cause the dimension disaster, and expression is not good. In 2010, Sascha Lange and Martin Riedmiller proposed deep auto-encoder neural networks in reinforcement learning to extract feature, which is used to control the visual correlation. In 2013, DeepMind proposed deep Q-network (DQN) in NIPS 2013, using convolution neural network to extract features, and then applied in reinforcement learning. They continue to improve and published an improved version of DQN on Nature in 2015, which has aroused widespread concern. In order to break through the limits of learning by machines, cognitive machine learning is proposed, which is the combination of machine learning and brain cognition, so that the machine intelligence is constantly evolving, and gradually reaches the human level of artificial intelligence. A cognitive model entitled Consciousness And Memory (CAM) is proposed by author, which consists of memory, consciousness, high-level cognitive functions, perception and motor. High-level cognitive functions of the brain include learning, language, thinking, decision making, emotion, and so on. Learning is a course to accept the stimulus through the nervous system and obtain new behavior, habits and accumulation experience. According to the current research progress of brain science and cognitive science, cognitive machine learning may be interested in learning emergence, procedural memory knowledge learning, learning evolution and so on. For intelligence, so-called evolution is refers to the learning of learning and the structure also follows the change. It is important to record the learning result by structure changing and improve the learning method.

Path planning refers to that the mobile robot can obtain the surrounding environment information and its own state information through the sensor carried by itself, which can avoid obstacles and move towards the target point. Deep reinforcement learning consists of two parts: reinforcement learning and deep learning, mainly used to deal with perception and decision-making problems, has become an important research branch in the field of artificial intelligence. This paper first introduces the basic knowledge of deep learning and reinforcement learning. Then, the research status of deep reinforcement learning algorithm based on value function and strategy gradient in path planning is described, and the application research of deep reinforcement learning in computer game, video game and autonomous navigation is described. Finally, I made a brief summary and outlook on the algorithms and applications of deep reinforcement learning.