Physical information-fused deep learning model ensembled with a subregion-specific sampling method for predicting flood dynamics

1-D coupled surface flow and transport equations revisited via the physics-informed neural network approach

The real-time forecasting of flood dynamics is a long-standing challenge traditionally addressed through numerical solutions of the Shallow Water Equations (SWEs). Numerical solutions of realistic flow problems using numerical schemes are often hindered by high computational costs, particularly due to the need for fine spatial and temporal discretization, complex boundary conditions, and the resolution of non-linearities inherent in the governing equations. In this study, we investigate the use of Physics-Informed Neural Networks (PINNs) to solve 1D and 2D SWEs in dam-break scenarios. The proposed PINN framework incorporates the governing partial differential equations along with the initial and boundary conditions directly within the training process of the network, ensuring physically consistent solutions. We conduct a systematic comparison of the solutions of SWE using the classical numerical scheme (Lax-Wendroff) with estimates of physics informed neural networks. For 1D SWE, a neural network is trained and validated on a dam-break problem, revealing that physics-informed models produce smoother but still acceptable estimates of wave propagation compared to standard numerical results. For 2D SWE, we consider various configurations of dam geometries along with varying initial profiles for water heights. Across all scenarios, reproduce the numerical baselines, albeit with limited accuracy, while avoiding spurious oscillations and numerical artifacts. Further tuning, achieved by incorporating numerical solutions into the PINN training, improved accuracy. This proof of concept demonstrates the potential of hybridized PINNs as a mesh-free, scalable, and generalizable framework for approximating solutions to nonlinear hyperbolic systems. Our results indicate that pre-trained, physics-informed models could serve as a viable alternative for real-time flood forecasting in complex domains.

The application of numerical models for flood and inundation modelling has become widespread in the past decades as a result of significant improvements in computational capabilities. Computational approaches to flood forecasting have significant benefits compared to empirical approaches which estimate statistical patterns of hydrological variables from observed data. However, there is still a significant computational cost associated with numerical flood modelling at high spatio-temporal resolutions. This limitation of numerical modelling has led to the development of statistical emulator models, machine learning (ML) models designed to learn the underlying generating process of the numerical model. The data-driven approach to ML involves relying entirely upon a set of training data to inform decisions about model selection and parameterisations. Deep learning models have leveraged data-driven learning methods with improvements in hardware and an increasing abundance of data to obtain breakthroughs in various fields such as computer vision, natural language processing and autonomous driving. In many scientific and engineering problems however, the cost of obtaining data is high and so there is a need for ML models that are able to generalise in the ‘small-data’ regime common to many complex problems. In this study, to overcome extrapolation and over-fitting issues of data-driven emulators, a Physics-Informed Neural Network model is adopted for the emulation of all two-dimensional hydrodynamic models which model fluid according the shallow water equations. This study introduces a novel approach to encoding the conservation of mass into a deep learning model, with additional terms included in the optimisation criterion, acting to regularise the model, avoid over-fitting and produce more physically consistent predictions by the emulator.

In recent years, successful applications of deep learning (DL) have inspired scientists to explore the possibilities of applying DL approaches to modeling scientific problems. Existing studies have revealed that to bake the physics into the DL makes a good supplement to the traditional numerical methods (e.g., finite element, finite volume method) which primarily rely on partial differential equations (PDEs). While DL models are ordinarily trained in a purely data-driven manner, integrating physics into them for simulating scientific problems has several benefits such as (i) physics constraints could regularize the over-parameterized model and hence mitigate the overfitting issue commonly seen in DL; (ii) physics information could also effectively reduce the amount of data needed for training the model; (iii) the resultant physics-informed DL models feature better interpretability and generalizability compared with the conventional black-box model. Furthermore, the powerful expressiveness of the deep network, guaranteed by the universal approximation theorem, makes it a suitable approximator for the solution to a physical system. In this dissertation, we develop two different DL architectures (or approaches), one being continuous scheme-based while the other discrete scheme-based, that leverage physics knowledge for modeling scientific problems. Through comprehensive numerical experiments, we demonstrate the proposed models can be used in solving general PDEs, establishing predictive data-driven models for dynamical systems, identifying the parameters in governing PDEs or even discovering the entire governing PDEs of dynamical systems from scarce and noisy measurements. The continuous model roots on the physics-informed neural network (PINN) which uses a fully connected neural network (FCNN) to approximate the physical fields of a system globally. This model is mesh-free as the residual of the physics (e.g., PDEs, initial/boundary values) is evaluated on a set of collocation points within the physical domain. Several applications including the forward simulations, data-driven simulations and solving inverse problems are presented to exemplify the advantages of PINN over traditional numerical methods. However, the original PINN suffers from inaccurate initial/boundary values due to the weak enforcement of the initial/boundary conditions (I/BCs). To overcome this issue, we propose an improved PINN model by utilizing multiple deep neural networks (DNNs) to construct the solution. Through a DNN pre-trained to represent the initial/boundary values, the approximated solution would obey the given I/BCs forcibly. With several numerical examples, we show that the improved PINN is characterized with much better accuracy on the I/BCs. Though the PINN shows great promise in data-driven modeling and solving inverse problems, some inherent limitations of PINN still exist, such as (i) the solution might lacks fine-scale details due to the global approximation of FCNN; (ii) high computational expense caused by the FCNN it roots on; (iii) incapability to incorporate existing PDE terms (e.g., $\Delta u$) into the network architecture. To overcome these drawbacks, this dissertation also proposes a discrete model - Physics-encoded Recurrent Convolutional Neural Network (PeRCNN) which recurrently updates the solution (or state variable) for time marching. Specifically, it utilizes convolutional neural network (CNN) to capture the spatial patterns of the solution while the recurrent network mimics the forward Euler scheme (or Runge-Kutta scheme) in numerical methods. PeRCNN is a mesh-based and discrete model due to the discretization in time and spatial dimension. The local connectivity of CNN makes PeRCNN more computationally efficient. In addition, the coercive encoding mechanism of physics in PeRCNN, fundamentally different from the PINN relying on soft penalty, ensures the network to rigorously obey given physics. The proposed PeRCNN is successfully applied to solving general PDEs, the data-driven modeling of dynamical systems and the data-driven discovery of governing PDEs from scarce and noisy measurements. Comparisons with the state-of-the-art DL models demonstrate that the proposed PeRCNN possesses excellent computational efficiency, accuracy and generalizability. --Author's abstract

Solving the two‐dimensional Shallow Water Equations (SWE) is a fundamental problem in flood simulation technology. In recent years, physics‐informed neural networks (PINNs) have emerged as a novel methodology for addressing this problem. Given their advantages in parallel computing, potential for data assimilation and parameter calibration, and the rapid advancement of artificial intelligence, it is crucial to investigate both the capabilities and limitations of PINNs. While current research has demonstrated the significant potential of PINNs, many aspects of this new approach remain to be explored. In this paper, we employ PINNs enhanced by dimensional transformation and Neuron‐wise Local Adaptive Activation Function techniques to validate its effectiveness in solving two‐dimensional free surface flow with rainfall on terrain topography. The SWE primarily exist in two forms: the primitive form and the conservative form. Through theoretical analysis and experimental validation, we demonstrate that a hybrid primitive‐conservative form offers superior performance. Additionally, in our cases, we find that incorporating the energy conservation law, specifically the entropy condition, does not improve accuracy and sometimes destabilizes training due to an over‐weighting of the mass conservation law. Furthermore, we have developed an open‐source module on the PINNacle platform for solving SWE using PINNs, which includes over 10 case studies and various equation forms, to promote research and application in this field.

Flood forecasting is usually performed by means of physically based models or conceptual models. Data-driven methods such as Artificial Neural Networks (ANN) are a promising solution as they can extract complex non-linear dependencies between inputs and outputs with robustness, generalization, and computational efficiency. However, the ANNs still rank below the classical method of numerical solvers for Saint Venant Equations in terms of accuracy. To bridge the accuracy gap, this work focuses on devising a Physics Informed Neural Network (PINN) model for spatial-temporal scale flood forecasting based on the Saint Venant Equations. The ANN architecture is extended to PINN with the implementation of the custom Partial Differential Equation (PDE) loss. This procedure allows to quickly steer the learning algorithm toward a true solution. A synthetic dataset was developed by performing numerical simulations of the Saint Venant Equations for training and evaluation of the PINN model. The results demonstrate that the PINN model performs better than the ANN Models and is suitable for water depth forecasting.

The exploration of deep learning methodologies has recently generated significant interest in the use of Physics-Informed Neural Networks (PINNs) to address complex physical problems governed by partial differential equations (PDEs). The PINN is trained using information from physical laws, including governing PDEs, boundary conditions, and initial conditions. However, achieving a well-trained PINN typically necessitates an appropriate balance between the weights of each loss function, which can considerably increase manual effort. This paper introduces a novel training approach that integrates non-dimensionalization with a multi-task learning technique, termed Automatic Weighted Loss (AWL), to autonomously achieve an optimal balance for each loss function. In the baseline PINN training for solving time-dependent PDEs, multiple weights (usually more than six) must be manually tuned for the model, whereas this method can reduce the number of scaling weights to only one. The proposed approach, referred to as the Non-dimensionalization Automatic Weighted Loss (NDAWL), is evaluated through six free-surface flow problems modelled by the Shallow Water Equations (SWEs). Furthermore, a comparative analysis is conducted between the solutions obtained using NDAWL-PINN and those from the original PINN, which relies on manual fine-tuning of loss functions. The numerical results indicate that the NDAWL-PINN method achieves comparable or superior accuracy to the original PINN, demonstrating its effectiveness in automating the balancing of loss functions.

In an open source software development environment, it is hard to decide the number of group members required for resolving software issues. Developers generally reply to issues based totally on their domain knowledge and interest, and there are no predetermined groups. The developers openly collaborate on resolving the issues based on many factors, such as their interest, domain expertise, and availability. This study compares eight different algorithms employing machine learning and deep learning, namely—Convolutional Neural Network, Multilayer Perceptron, Classification and Regression Trees, Generalized Linear Model, Bayesian Additive Regression Trees, Gaussian Process, Random Forest and Conditional Inference Tree for predicting group size in five open source software projects developed and managed using an open source development framework GitHub. The social information foraging model has also been extended to predict group size in software issues, and its results compared to those obtained using machine learning and deep learning algorithms. The prediction results suggest that deep learning and machine learning models predict better than the extended social information foraging model, while the best-ranked model is a deep multilayer perceptron((R.M.S.E. sequelize—1.21, opencv—1.17, bitcoin—1.05, aseprite—1.01, electron—1.16). Also it was observed that issue labels helped improve the prediction performance of the machine learning and deep learning models. The prediction results of these models have been used to build an Issue Group Recommendation System as an Internet of Things application that recommends and alerts additional developers to help resolve an open issue.

Accurate and rapid prediction of structural responses to seismic loading is critical for ensuring structural safety. Recently, there has been active research focusing on the application of deep learning techniques, including Physics-Informed Neural Networks (PINNs) and Long Short-Term Memory (LSTM) networks, to predict the dynamic behavior of structures. While these methods have shown promise, each comes with distinct limitations. PINNs offer physical consistency but struggle with capturing long-term temporal dependencies in nonlinear systems, while LSTMs excel in learning sequential data but lack physical interpretability. To address these complementary limitations, this study proposes a hybrid LSTM-PINN model, combining the temporal learning ability of LSTMs with the physics-based constraints of PINNs. This hybrid approach allows the model to capture both nonlinear, time-dependent behaviors and maintain physical consistency. The proposed model is evaluated on both single-degree-of-freedom (SDOF) and multi-degree-of-freedom (MDOF) structural systems subjected to the El-Centro ground motion. For validation, the 1940 El-Centro NS earthquake record was used, and the ground acceleration data were normalized and discretized for numerical simulation. The proposed LSTM-PINN is trained under the same conditions as the conventional PINN models (e.g., same optimizer, learning rate, and loss structure), but with fewer training epochs, to evaluate learning efficiency. Prediction accuracy is quantitatively assessed using mean error and mean squared error (MSE) for displacement, velocity, and acceleration, and results are compared with PINN-only models (PINN-1, PINN-2). The results show that LSTM-PINN consistently achieves the most stable and precise predictions across the entire time domain. Notably, it outperforms the baseline PINNs even with fewer training epochs. Specifically, it achieved up to 50% lower MSE with only 10,000 epochs, compared to the PINN's 50,000 epochs, demonstrating improved generalization through temporal sequence learning. This study empirically validates the potential of physics-guided time-series AI models for dynamic structural response prediction. The proposed approach is expected to contribute to future applications such as real-time response estimation, structural health monitoring, and seismic performance evaluation.

This paper investigates the estimation of R-values from transient time series using regression outputs of different neural network architectures, specifically a naive multilayer perceptron (MLP) and a physics-informed neural network (PINN). The PINN incorporates a bias correction for steady-state R-value calculations. The ground truth is derived from transient simulations of wall assemblies. A parameter study examines the prediction accuracy of various MLP architectures within the training process. Further evaluations assess the MLP performances on both validation and independent test datasets. The results indicate that neural networks can effectively enhance R-value estimation. However, in the proposed setup, PINNs did not yield significantly better predictions than conventional MLPs. Future research should explore alternative PINN architectures incorporating more temporal context of the transient input data. Additional studies should also investigate the applicability of more context-aware neural networks, such as convolutional or recurrent neural networks.

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

This article investigates the identification of hate speech on social media using machine learning and deep learning techniques. The research uses metrics such as F-measure, AUC-ROC, precision, accuracy, and recall assessing the effectiveness of various tactics. The findings indicate that deep learning models, particularly the bidirectional long short-term memory (BiLSTM) architecture, consistently outperform other methods in categorization tasks. The research emphasizes the importance of sophisticated neural network designs in identifying the intricacies of hostile and offensive content online. The study offers insights for promoting early identification and prevention of cyberbullying, improving secure and inclusive online environments. Future research may explore real-time detection systems, hybrid approaches, or the integration of complementary components to enhance and improve innovative technology in tackling this significant social issue. A sample tweet was annotated by specialists who categorize tweets as hate speech, offensive language, or neutral. The researchers applied shallow learning methodologies and integrated word embeddings like Word2Vec and GloVe to enhance the efficacy of deep learning models. The results indicate that BiLSTM surpasses shallow learning methods in detecting hate speech on Twitter, highlighting the efficacy of deep learning approaches in recognizing and tracking hate speech on social media platforms. When comparing different deep learning and machine learning models on different datasets, the results reveal that deep learning techniques are usually more effective. A reasonably high level of accuracy is achieved by KNN and SVM among classical algorithms, whereas Naïve Bayes performs the poorest. While deep learning approaches provide better results, tree-based models such as Random Forest and Decision Trees offer more consistent accuracy. Models based on neural networks, such as LSTM, CNN, and BI-LSTM, perform well, with LSTM-based methods excelling in particular. The most successful strategy for classification problems is the model presented, which obtains the greatest accuracy, precision, recall, F1-score of 95%. The research aids in the development of advanced tools and methodologies to mitigate hate speech on social media and foster positive online interactions. Future research may investigate alternative deep learning architectures, such as transformers, to enhance hate speech detection efficacy. The advancement of interpretable AI methodologies for identifying hate speech and delivering transparent forecasts might enhance user confidence and facilitate better content moderation decisions.

PurposeThe first transmission of coronavirus to humans started in Wuhan city of China, took the shape of a pandemic called Corona Virus Disease 2019 (COVID-19), and posed a principal threat to the entire world. The researchers are trying to inculcate artificial intelligence (Machine learning or deep learning models) for the efficient detection of COVID-19. This research explores all the existing machine learning (ML) or deep learning (DL) models, used for COVID-19 detection which may help the researcher to explore in different directions. The main purpose of this review article is to present a compact overview of the application of artificial intelligence to the research experts, helping them to explore the future scopes of improvement.MethodsThe researchers have used various machine learning, deep learning, and a combination of machine and deep learning models for extracting significant features and classifying various health conditions in COVID-19 patients. For this purpose, the researchers have utilized different image modalities such as CT-Scan, X-Ray, etc. This study has collected over 200 research papers from various repositories like Google Scholar, PubMed, Web of Science, etc. These research papers were passed through various levels of scrutiny and finally, 50 research articles were selected.ResultsIn those listed articles, the ML / DL models showed an accuracy of 99% and above while performing the classification of COVID-19. This study has also presented various clinical applications of various research. This study specifies the importance of various machine and deep learning models in the field of medical diagnosis and research.ConclusionIn conclusion, it is evident that ML/DL models have made significant progress in recent years, but there are still limitations that need to be addressed. Overfitting is one such limitation that can lead to incorrect predictions and overburdening of the models. The research community must continue to work towards finding ways to overcome these limitations and make machine and deep learning models even more effective and efficient. Through this ongoing research and development, we can expect even greater advances in the future.

Summary Classical reservoir engineering studies require building geological models and solving complex fluid flow transport equations that require high-quality data, numerous computational resources, time and workflows. For large and mature fields data-driven models can be used to get faster answer and to perform production analysis more efficiently. Capacitance Resistive Models (CRM) are a class of methods based on material balance that can be used to estimate production wells liquid rates as a function of injected water and Bottom Hole Pressure (BHP) variations. CRM methods quantify the connectivity between producers and injectors using only dynamic data. An important drawback of CRM is that they can suffer from parameter identification problems. Moreover, the analytical solution can be only obtained in specific conditions: linear variations of BHP and fixed injection rate between two consecutive time steps. In this work we present a new approach combining CRM material balance equations with neural networks in order to obtain more robust and reliable estimation of the CRM parameters (i.e. well connectivity, productivity indices and time constants). This proposal is also interesting since it is not based on any assumption on BHP and injection rates. To this end, we use a recent approach called Physics Informed Neural Networks (PINNs). In this approach neural networks are trained on observed data with additional physics constraints traduced in appropriate loss functions. The parameters of this physical equation are evaluated at the same time as the neural network weights. The introduction of PINNs in our approach raised after testing classical machine learning (ML) models (SVMs, Random Forests …) and deep learning models (MLP, LSTM, RNNs…). Indeed, such models can perform well in some specific cases but usually struggle to produce robust results (i.e. forecasting) in the long term. Unfortunately, such systems do not natively integrate physics constraints. Our aim is to impose physic constraints in neural networks. Thus, we may obtain more stable and reliable results. On the same time, we should be able to account for more behaviors that are not explained by simplified physic equations such as material balance. We performed a full comparison between our approach using PINNs, other standard ML and DL approaches and a given framework of CRMs on two data-sets: a simple but realistic model build using a commercial reservoir simulator, and a real data set. We show that our approach gives more robust results (in terms of MSE) while not suffering from parameter identification issue.

Deep learning techniques have found diverse applications, including solving partial differential equations (PDEs). This chapter observes the utilization of physics-informed neural networks (PINNs) for solving a range of PDEs related to fluid dynamics applications. PINNs harness the inherent physical insights embedded within PDEs by integrating them as regularization terms, thereby optimizing the performance of neural networks. Notably, PINNs prove effective in handling higher-dimensional differential equations, offering substantial reductions in computational costs. The current chapter focuses on the application of PINNs to solve specific instances of fluid dynamics problems, including second-order porous medium and third-order Korteweg-de Vriesequations. The proposed methodology incorporates the Adam optimizer and employs the “tanh” activation function. The achieved mean squared error (MSE) losses are reported as https://www.w3.org/1998/Math/MathML" display="inline">4.972656×10−6, 16.58, https://www.w3.org/1998/Math/MathML" display="inline">2.73382×10−5, and https://www.w3.org/1998/Math/MathML" display="inline">9.083319×10−5. To validate the efficacy of our approach, a comparative analysis with ground truth solutions is presented. This contribution offers insights into the cutting-edge applications of machine learning in solving differential equations that represent fluid dynamics phenomena.