Enhancing Temperature and Rainfall Prediction Accuracy Through Deep Learning Frameworks

In recent years, successful applications of deep learning (DL) have inspired scientists to explore the possibilities of applying DL approaches to modeling scientific problems. Existing studies have revealed that to bake the physics into the DL makes a good supplement to the traditional numerical methods (e.g., finite element, finite volume method) which primarily rely on partial differential equations (PDEs). While DL models are ordinarily trained in a purely data-driven manner, integrating physics into them for simulating scientific problems has several benefits such as (i) physics constraints could regularize the over-parameterized model and hence mitigate the overfitting issue commonly seen in DL; (ii) physics information could also effectively reduce the amount of data needed for training the model; (iii) the resultant physics-informed DL models feature better interpretability and generalizability compared with the conventional black-box model. Furthermore, the powerful expressiveness of the deep network, guaranteed by the universal approximation theorem, makes it a suitable approximator for the solution to a physical system. In this dissertation, we develop two different DL architectures (or approaches), one being continuous scheme-based while the other discrete scheme-based, that leverage physics knowledge for modeling scientific problems. Through comprehensive numerical experiments, we demonstrate the proposed models can be used in solving general PDEs, establishing predictive data-driven models for dynamical systems, identifying the parameters in governing PDEs or even discovering the entire governing PDEs of dynamical systems from scarce and noisy measurements. The continuous model roots on the physics-informed neural network (PINN) which uses a fully connected neural network (FCNN) to approximate the physical fields of a system globally. This model is mesh-free as the residual of the physics (e.g., PDEs, initial/boundary values) is evaluated on a set of collocation points within the physical domain. Several applications including the forward simulations, data-driven simulations and solving inverse problems are presented to exemplify the advantages of PINN over traditional numerical methods. However, the original PINN suffers from inaccurate initial/boundary values due to the weak enforcement of the initial/boundary conditions (I/BCs). To overcome this issue, we propose an improved PINN model by utilizing multiple deep neural networks (DNNs) to construct the solution. Through a DNN pre-trained to represent the initial/boundary values, the approximated solution would obey the given I/BCs forcibly. With several numerical examples, we show that the improved PINN is characterized with much better accuracy on the I/BCs. Though the PINN shows great promise in data-driven modeling and solving inverse problems, some inherent limitations of PINN still exist, such as (i) the solution might lacks fine-scale details due to the global approximation of FCNN; (ii) high computational expense caused by the FCNN it roots on; (iii) incapability to incorporate existing PDE terms (e.g., $\Delta u$) into the network architecture. To overcome these drawbacks, this dissertation also proposes a discrete model - Physics-encoded Recurrent Convolutional Neural Network (PeRCNN) which recurrently updates the solution (or state variable) for time marching. Specifically, it utilizes convolutional neural network (CNN) to capture the spatial patterns of the solution while the recurrent network mimics the forward Euler scheme (or Runge-Kutta scheme) in numerical methods. PeRCNN is a mesh-based and discrete model due to the discretization in time and spatial dimension. The local connectivity of CNN makes PeRCNN more computationally efficient. In addition, the coercive encoding mechanism of physics in PeRCNN, fundamentally different from the PINN relying on soft penalty, ensures the network to rigorously obey given physics. The proposed PeRCNN is successfully applied to solving general PDEs, the data-driven modeling of dynamical systems and the data-driven discovery of governing PDEs from scarce and noisy measurements. Comparisons with the state-of-the-art DL models demonstrate that the proposed PeRCNN possesses excellent computational efficiency, accuracy and generalizability. --Author's abstract

In recent engineering applications using deep learning, physics-informed neural network (PINN) is a new development as it can exploit the underlying physics of engineering systems. The novelty of PINN lies in the use of partial differential equations (PDE) for the loss function. Most PINNs are implemented using automatic differentiation (AD) for training the PDE loss functions. A lesser well-known study is the use of finite difference method (FDM) as an alternative. Unlike an AD based PINN, an immediate benefit of using a FDM based PINN is low implementation cost. In this paper, we propose the use of finite difference method for estimating the PDE loss functions in PINN. Our work is inspired by computational analysis in electromagnetic systems that traditionally solve Laplace’s equation using successive over-relaxation. In the case of Laplace’s equation, our PINN approach can be seen as taking the Laplacian filter response of the neural network output as the loss function. Thus, the implementation of PINN can be very simple. In our experiments, we tested PINN on Laplace’s equation and Burger’s equation. We showed that using FDM, PINN consistently outperforms non-PINN based deep learning. When comparing to AD based PINNs, we showed that our method is faster to compute as well as on par in terms of error reduction.

In recent years, Physics-Informed Neural Networks (PINNs) have introduced a novel approach to solving partial differential equations (PDEs) using deep learning techniques. Despite the promising results and rapid advancements in the field, there is a lack of more accurate studies concerning properties related to modeling choices within the deep learning framework, especially, but not exclusively, in solid mechanics problems. In this study, we aim to explore whether the neural network architecture, activation function, optimizer, and sampling method can influence the accuracy of results and training speed for solid mechanics problems. We will focus on elasticity and hyperelasticity problems in 1D, 2D, and 3D dimensions to lay the groundwork for more complex investigations.

Predicting positon solutions of a family of nonlinear Schrödinger equations through deep learning algorithm

Physics-Informed Neural Network for modeling and predicting temperature fluctuations in proton exchange membrane electrolysis

Surrogate modeling of parameterized multi-dimensional premixed combustion with physics-informed neural networks for rapid exploration of design space

Summary Classical reservoir engineering studies require building geological models and solving complex fluid flow transport equations that require high-quality data, numerous computational resources, time and workflows. For large and mature fields data-driven models can be used to get faster answer and to perform production analysis more efficiently. Capacitance Resistive Models (CRM) are a class of methods based on material balance that can be used to estimate production wells liquid rates as a function of injected water and Bottom Hole Pressure (BHP) variations. CRM methods quantify the connectivity between producers and injectors using only dynamic data. An important drawback of CRM is that they can suffer from parameter identification problems. Moreover, the analytical solution can be only obtained in specific conditions: linear variations of BHP and fixed injection rate between two consecutive time steps. In this work we present a new approach combining CRM material balance equations with neural networks in order to obtain more robust and reliable estimation of the CRM parameters (i.e. well connectivity, productivity indices and time constants). This proposal is also interesting since it is not based on any assumption on BHP and injection rates. To this end, we use a recent approach called Physics Informed Neural Networks (PINNs). In this approach neural networks are trained on observed data with additional physics constraints traduced in appropriate loss functions. The parameters of this physical equation are evaluated at the same time as the neural network weights. The introduction of PINNs in our approach raised after testing classical machine learning (ML) models (SVMs, Random Forests …) and deep learning models (MLP, LSTM, RNNs…). Indeed, such models can perform well in some specific cases but usually struggle to produce robust results (i.e. forecasting) in the long term. Unfortunately, such systems do not natively integrate physics constraints. Our aim is to impose physic constraints in neural networks. Thus, we may obtain more stable and reliable results. On the same time, we should be able to account for more behaviors that are not explained by simplified physic equations such as material balance. We performed a full comparison between our approach using PINNs, other standard ML and DL approaches and a given framework of CRMs on two data-sets: a simple but realistic model build using a commercial reservoir simulator, and a real data set. We show that our approach gives more robust results (in terms of MSE) while not suffering from parameter identification issue.

A nonlocal physics-informed deep learning framework using the peridynamic differential operator

Numerical simulation and experimental techniques are the primary methods for solving fluid dynamics problems. However, while numerical simulation approaches are sensitive when meshing a complex structure, experimental methods have difficulty simulating the physical challenges. Therefore, building an affordable model to solve the fluid dynamics problem is very important. Deep learning (DL) approaches have great abilities to handle strong nonlinearity and high dimensionality that attract much attention for solving fluid dynamics problems. In this paper, we used a deep learning-based framework, physics-informed neural networks (PINNs). The main idea of PINN approaches is to encode the underlying physical law (i.e., the partial differential equation) into the neural network as prior information. In the oil recovery process involving the injection of fluids and multiphase flow in porous media, fingering instability is observed if a fluid with low viscosity displaces a high viscosity fluid. This paper provides a deep learning framework to simulate the instability (fingering) phenomenon during secondary and enhanced oil recovery methods. We have considered two models, namely, the first model on immiscible multiphase flow from the secondary oil recovery method, which is analyzed both with and without deliberating mass flow rate. In the second model, instability arising during the enhanced oil recovery method involving miscible displacement of multiphase is examined, focusing on oil recovery using carbonated water injection. We solve the governing nonlinear partial differential equation using PINNs. Furthermore, we have compared results from PINNs with the semi-analytical solution from the literature. The results show that PINNs are very effective in fluid flow problems and deserve further research.

Controlling nonlinear dynamics arises in various engineering fields. We present efforts to model the forced van der Pol system control using physics-informed neural networks (PINN) compared to benchmark methods, including idealized nonlinear feedforward (FF) control, linearized feedback control (FB), and feedforward-plus-feedback combined (C). The aim is to implement circular trajectories in the state space of the van der Pol system. A designed benchmark problem is used for testing the behavioral differences of the disparate controllers and then investigating controlled schemes and systems of various extents of nonlinearities. All methods exhibit a short initialization accompanying arbitrary initialization points. The feedforward control successfully converges to the desired trajectory, and PINN executes good controls with higher stochasticity observed for higher-order terms based on the phase portraits. In contrast, linearized feedback control and combined feed-forward plus feedback failed. Varying trajectory amplitudes revealed that feed-forward, linearized feedback control, and combined feed-forward plus feedback control all fail for unity nonlinear damping gain. Traditional control methods display a robust fluctuation for higher-order terms. For some various nonlinearities, PINN failed to implement the desired trajectory instead of becoming “trapped” in the phase of small radius, yet idealized nonlinear feedforward successfully implemented controls. PINN generally exhibits lower relative errors for varying targeted trajectories. However, PINN also shows evidently higher computational burden compared with traditional control theory methods, with at least more than 30 times longer control time compared with benchmark idealized nonlinear feed-forward control. This manuscript proposes a comprehensive comparative study for future controller employment considering deterministic and machine learning approaches.

Maintaining consistent part quality remains a critical challenge in industrial additive manufacturing, particularly in laser-based powder bed fusion of polymers (PBF-LB/P), where crystallization-driven thermal instabilities, governed by isothermal crystallization within a narrow sintering window, precipitate defects such as curling, warping, and delamination. In contrast to metal-based systems dominated by melt-pool hydrodynamics, polymer PBF-LB/P requires monitoring strategies capable of resolving subtle spatio-temporal thermal deviations under realistic industrial operating conditions. Although machine learning, particularly convolutional neural networks (CNNs), has demonstrated efficacy in defect detection, a structured evaluation of heterogeneous modeling paradigms and their deployment feasibility in polymer PBF-LB/P remains limited. This study presents a systematic cross-paradigm assessment of unsupervised anomaly detection (autoencoders and generative adversarial networks), supervised CNN classifiers (VGG-16, ResNet50, and Xception), hybrid CNN-LSTM architectures, and physics-informed neural networks (PINNs) using 76,450 synchronized thermal and RGB images acquired from a commercial industrial system operating under closed control constraints. CNN-based models enable frame- and sequence-level defect classification, whereas the PINN component complements detection by providing physically consistent thermal-field regression. The results reveal quantifiable trade-offs between detection performance, temporal robustness, physical consistency, and algorithmic complexity. Pre-trained CNNs achieve up to 99.09% frame-level accuracy but impose a substantial computational burden for edge deployment. The PINN model attains an RMSE of approximately 27 K under quasi-isothermal process conditions, supporting trend-level thermal monitoring. A lightweight hybrid CNN achieves 99.7% validation accuracy with 1860 parameters and a CPU-benchmarked forward-pass inference time of 1.6 ms (excluding sensor acquisition latency). Collectively, this study establishes a rigorously benchmarked, scalable, and resource-efficient deep-learning framework tailored to crystallization-dominated polymer PBF-LB/P, providing a technically grounded basis for real-time industrial quality monitoring.

Several numerical experiments were performed to investigate the dynamic and thermodynamic effects of sea surface temperature (SST) on tropical cyclone (TC) intensity. The results reveal that the relative SST within a radius of 2–3 times the radius of maximum wind contributes positively and greatly to TC intensity, while the remote SST far away from the TC center could reduce storm intensity. The change of air-sea temperature and moisture differences may be the reason why TC intensity is more sensitive to the relative rather than the absolute SST. As the inflow air moves toward the eyewall, warmer (colder) remote SST can gradually increase (decrease) the underlying surface air temperature and moisture, and thus decrease (increase) the air-sea temperature and moisture differences, which lead to less (more) energy fluxes entering the eyewall and then decrease (increase) the TC intensity and make it less sensitive to the absolute SST change. Finally, with all the related dynamic and thermodynamic processes being taken into account, a schematic diagram for the effects of relative SST and absolute SST on TC intensity is proposed.

This paper proposes a solution to the problem of automatically predicting hysteresis and backbone curves of bridge piers under seismic loads. The proposed solution utilizes a stacked hybrid Convolutional Neural Network-bidirectional Cuda Deep Neural Network Long Short Term Memory layer benefiting from the skip connections technique and incorporates a custom task-specific attention layer to enhance its performance. The proposed framework borrows the functional API provided by the Keras library in Python to construct a model taking into account horizontal and vertical ground accelerations, actuator loads in both horizontal and vertical directions, the effective pier height, the second moment of area, and the superstructure mass as input features. The deep learning model demands a substantial amount of data for effective training, validation, and testing. An error-sensitive analysis suggests that a comprehensive dataset should consist of a minimum of 12 sets of pier data for real-time hybrid simulations and 17 sets for cyclic experiments (10 for high-speed and seven for low-speed scenarios). This extensive dataset is deemed essential for the optimal performance of the model. The same deep learning framework and optimization of hyperparameters apply when training real-time hybrid simulations and conducting cyclic tests. After 5000 epochs, the proposed hybrid loss function, combining mean square and mean absolute errors, exhibits a steady and gradual decrease toward near-zero values within the datasets used for training and validation. Additionally, over 93% correlation exists between the predicted unseen time series responses and those derived from empirical measurements. Overall, the proposed deep learning model offers significant advantages, notably in terms of time and cost savings associated with experimental endeavors for new tests. By providing a rapid and accurate understanding of the hysteretic behavior of bridge piers, this model contributes to more efficient bridge design processes. Ultimately, it facilitates precision in design decisions, leading to enhanced accuracy and effectiveness in bridge engineering.

Objective: To design a deep transfer learning framework for modeling fluence map predictions for stereotactic body radiation therapy (SBRT) of adrenal cancer and similar sites that usually have a small number of cases. Approach: We developed a transfer learning framework for adrenal SBRT planning that leverages knowledge in a pancreas SBRT planning model. Treatment plans from the two sites had different dose prescriptions and beam settings but both prioritized gastrointestinal sparing. A base framework was first trained with 100 pancreas cases. This framework consists of two convolutional neural networks (CNN), which predict individual beam doses (BD-CNN) and fluence maps (FM-CNN) sequentially for 9-beam intensity-modulated radiation therapy (IMRT) plans. Forty-five adrenal plans were split into training/validation/test sets with the ratio of 20/10/15. The base BD-CNN was re-trained with transfer learning using 5/10/15/20 adrenal training cases to produce multiple candidate adrenal BD-CNN models. The base FM-CNN was directly used for adrenal cases. The deep learning (DL) plans were evaluated by several clinically relevant dosimetric endpoints, producing a percentage score relative to the clinical plans. Main results: Transfer learning significantly reduced the number of training cases and training time needed to train such a DL framework. The adrenal transfer learning model trained with 5/10/15/20 cases achieved validation plan scores of 85.4/91.2/90.7/89.4, suggesting that model performance saturated with 10 training cases. Meanwhile, a model using all 20 adrenal training cases without transfer learning only scored 80.5. For the final test set, the 5/10/15/20-case models achieved scores of 73.5/75.3/78.9/83.3. Significance: It is feasible to use deep transfer learning to train an IMRT fluence prediction framework. This technique could adapt to different dose prescriptions and beam configurations. This framework potentially enables DL modeling for clinical sites that have a limited dataset, either due to few cases or due to rapid technology evolution.

A comprehensive empirical study on bug characteristics of deep learning frameworks