Physics-Based Self-Learning Recurrent Neural Network enhanced time integration scheme for computing viscoplastic structural finite element response

Physics-Informed Neural Network for modeling and predicting temperature fluctuations in proton exchange membrane electrolysis

3D concrete printing (3DCP) is an innovative construction method that extrudes cementitious materials layer-by-layer to fabricate building components based on a digital model. 3DCP has gained increasing adoption globally for projects like buildings, bridges, retaining walls, and stormwater management systems. However, 3DCP places stringent demands on the rheological properties of the printable cementitious materials. Rheology is critical for the flow and buildability of fresh concrete during extrusion. The mixture must have a low yield stress for pumping, moderate viscosity to hold its shape after deposition, minimal bleeding and segregation of aggregates, as well as responsive rheology that can transition from fluid to solid state as layers are printed sequentially. In this dissertation, we aim to solve the challenges mentioned above including: (1) accurate and efficient quantification of rheological and thixotropic properties of cementitious materials, (2) evaluation of flow behavior of fresh concrete during extrusion. Using traditional experimental methods to evaluate rheological properties of cement paste is labor-intensive and time-consuming. Numerical simulations like Finite Element Method (FEM) or Computational Fluid Dynamics (CFD) can help increase the speed of finding the most suitable material for 3DCP. However, these simulation methods are costly in computing power, making them less desirable for simulating the dynamic and complex process of 3DCP. Moreover, mesh-based approaches are not suitable for complex geometries as the mesh generation would be challenging with curved boundaries, small features, holes or thin regions. Physics-Informed Neural Network (PINN) is introduced in the work as an innovative solution to simulate and model the whole process of 3DCP. It can leverage the flexibility and computational efficiency of data-based neural networks, which can accelerate the learning and convergence speed. At the same time, Partial Differential Equations (PDEs) would be embedded in the network structure to increase the accuracy of its prediction. The integrated governing physical laws can help the neural network understanding the underlying physics of the 3DCP process, potentially increasing the accuracy and reliability of simulations. A significant part of the research is dedicated to understanding the rheological and thixotropic behavior of cementitious materials, which is critical for the material design in 3DCP. We developed a Rheology-informed Neural Network, RheologyNet, where we embedded the rheological constitutive laws into the loss function. These physical laws can help the network focus on simulating the rheological properties of cementitious materials. Moreover, a thixotropic evaluation system has been established based on the proposed model to evaluate the thixotropic property of cement paste. This study is presented in Chapter 2. Then, we proposed a Navier-Stokes Informed Neural Network (NSINN) to study the flow behavior of cement paste in the 3DCP barrel and nozzle since it involves complicated Multiscale or Multiphysics behaviors, such as anisotropy and non-uniform shear rate distribution. The Navier Stokes Equation and the rheological constitutive equations are coupled and embedded into the NSINN architecture to simulate the flow behavior in the barrel during 3D-printing extrusion process. This approach demonstrates the potential of PINNs to provide high-accuracy simulations of velocity and pressure fields in a computationally efficient manner compared to traditional mesh-based models. Also, we used the NSINN to learn the relationship between the nozzle size and the printing quality. The details of the study are presented in Chapter 3. At last, a multi-subnetwork PINN architecture (MultiSubPINN) is proposed to address the limitations of traditional PINNs. This is because traditional PINNs used Fully-connected Neural Network (FNN) as main bone. It has problems like different outputs must share parameters within the network structure. However, this parameter sharing nature might lead to potential errors and decrease the model performance. The main architecture of this network is consisted of multiple separated sub-networks and the loss function of each sub-network is calculated separately to update the parameters in each sub-network. By doing this, we can ensure that the optimization for one output doesn't negatively impact others, therefore providing a more flexible, scalable, and potentially more accurate alternative for the evaluation. Besides, using randomly distributed training points in the computational domain for PINNs is a common way. However, many physical problems exhibit non-uniform behavior, with certain regions of the domain having more complex dynamics or sharper gradients than others. Randomly distributed training points might not adequately capture these regions, leading to a model that misrepresents the underlying physics in critical areas. We proposed a Physical-based sampling strategy to optimize the training of the PINNs. This study is presented in Chapter 4. In summary, this dissertation has established a new platform that is enabled by interpretable NN to accurately quantify cement paste’s rheological properties and simulate the flow behavior of 3D concrete printing process. The findings suggest that PINNs can not only improve the efficiency and accuracy of simulations but also provide a more profound understanding of the material behaviors and process dynamics critical to 3DCP. The novel PINN structure developed from this dissertation is a powerful general platform that could be applicable to a variety of material domains for complex rheological behavior predictions.

This study explores the efficacy of supervised machine learning regression models in precisely forecasting the Global Maximum Power Point (GMPP) in Photovoltaic (PV) systems under Partial Shading Conditions (PSC). Various supervised machine learning models are examined for the suitability of GMPP tracking during PSC. The study evaluated the effectiveness of Neural Network architectures, including Long Short Term Memory (LSTM), Bidirectional Long Short Term Memory (Bi LSTM), Recurrent Neural Networks (RNN), Support Vector Machines (SVM) and Artificial Neural Networks (ANN) in predicting GMPP. The data set for training was obtained from the PV system, modelled in MATLAB Simulink. All regression models underwent rigorous training in the MATLAB Simulink environment, followed by an analytical review of their performance, during partial shading scenarios. Comparative analysis conducted across different regression models to evaluate their effectiveness in predicting the GMPP in PV systems under PSC suggests that Neural Networks exhibit the capability to forecast the GMPP during PSC scenario. This finding underscores the potential of Feedforward Neural Network (FNN), particularly its deep variant, demonstrated superior performer in terms of predicting the GMPP.

This thesis investigates two physics informed neural network (PINN) approaches for two mechanistic problems via the performance metrics of model parameter estimation and system state prediction. The PINNs are compared to Bayesian inference (BI) tested under similar conditions. The first PINN studied is the loss-based PINN, which embeds physics into a standard feedforward neural network within the loss function of this network. The second PINN investigated is the hybrid-recurrent physics informed neural network, which uses a recurrent neural network (RNN) that embeds physics in RNN architecture. The BI used as a point of comparison uses a Transitional Markov Chain Monte Carlo algorithm to estimate the posterior distributions. The applications studied are a compartmental model for an infectious disease, and a mass-spring-damper model. It is found that BI performs better in ideal data conditions and the PINNs perform better in weak data conditions.

Accurate and rapid prediction of structural responses to seismic loading is critical for ensuring structural safety. Recently, there has been active research focusing on the application of deep learning techniques, including Physics-Informed Neural Networks (PINNs) and Long Short-Term Memory (LSTM) networks, to predict the dynamic behavior of structures. While these methods have shown promise, each comes with distinct limitations. PINNs offer physical consistency but struggle with capturing long-term temporal dependencies in nonlinear systems, while LSTMs excel in learning sequential data but lack physical interpretability. To address these complementary limitations, this study proposes a hybrid LSTM-PINN model, combining the temporal learning ability of LSTMs with the physics-based constraints of PINNs. This hybrid approach allows the model to capture both nonlinear, time-dependent behaviors and maintain physical consistency. The proposed model is evaluated on both single-degree-of-freedom (SDOF) and multi-degree-of-freedom (MDOF) structural systems subjected to the El-Centro ground motion. For validation, the 1940 El-Centro NS earthquake record was used, and the ground acceleration data were normalized and discretized for numerical simulation. The proposed LSTM-PINN is trained under the same conditions as the conventional PINN models (e.g., same optimizer, learning rate, and loss structure), but with fewer training epochs, to evaluate learning efficiency. Prediction accuracy is quantitatively assessed using mean error and mean squared error (MSE) for displacement, velocity, and acceleration, and results are compared with PINN-only models (PINN-1, PINN-2). The results show that LSTM-PINN consistently achieves the most stable and precise predictions across the entire time domain. Notably, it outperforms the baseline PINNs even with fewer training epochs. Specifically, it achieved up to 50% lower MSE with only 10,000 epochs, compared to the PINN's 50,000 epochs, demonstrating improved generalization through temporal sequence learning. This study empirically validates the potential of physics-guided time-series AI models for dynamic structural response prediction. The proposed approach is expected to contribute to future applications such as real-time response estimation, structural health monitoring, and seismic performance evaluation.

A Physics-Informed Neural Network (PINN) was developed to solve the complex Navier-Stokes (N-S) equations for multiphase flow measurement including flowrates of individual phases in the multiphase mixture, and phase ratios. The results were compared against a mathematical based multiphase flow measurement approach and validated through a controlled loop test using oil, water, and gas. The PINN embeds fundamental fluid dynamics principles into its learning process, ensuring that the neural network adheres to the governing physical laws of motion, continuity, and conservation. Unlike traditional neural networks that rely solely on data-driven learning, PINNs incorporate fluid dynamics driven differential equations, to enhance model accuracy and reliability. This study evaluates the accuracy, efficiency, and repeatability of PINNs by comparing their results against traditional mathematical-based multiphase flow measurement (MPFM) technology and controlled test cases. The developed PINN successfully solved the Navier- Stokes (N-S) equations in their closest approximation to the original form, with minimal assumptions and simplifications, to estimate multiphase flow rates, integrating fundamental fluid dynamics principles into its learning process. The results demonstrated that the PINN-based approach achieved accuracy comparable to traditional mathematical multiphase flow measurement (MPFM) technologies, while significantly improving computational efficiency. During controlled loop tests with oil, water, and gas, the PINN model consistently captured flow dynamics with high precision, validating its effectiveness under real-world conditions. Moreover, the PINN method exhibited strong generalization capabilities, adapting to various flow regimes without the need for extensive empirical calibration. Compared to conventional MPFM technologies, PINN reduced reliance on predefined correlations and offered greater adaptability to changing fluid properties. The repeatability of results confirmed the model's reliability, positioning PINNs as a promising alternative for complex multiphase flow measurement applications. These findings highlight the potential of PINNs as an innovative, physics-guided, and data-efficient solution, effectively bridging the gap between traditional physics-based methods and modern machine learning techniques. This paper presents an innovative application of Physics-Informed Neural Networks (PINNs) for multiphase flow measurement, offering a data-driven yet physics-guided alternative to traditional mathematical MPFM technologies. By embedding fundamental physical laws into the neural network training process, the proposed approach enhances accuracy, computational efficiency, and adaptability, while minimizing reliance on extensive empirical calibration. This advancement significantly improves real-time flow monitoring, equipping practicing engineers with a more reliable, data-efficient, and scalable solution for complex multiphase flow measurement in industrial applications.

Prediction concept needs human-like thinking and ideas. ANN helps make a program for describing the solutions to the prediction problems. To gain accurate prediction, there is a need for deep thinking in the knowledge system discovered. Deep learning helps form such next-level type of ANN thinking. Deep learning models do not require humans for programming the problems. It can determine the prediction by itself by learning the dataset. This paper describes the prediction by considering sequential model for continuous accessing of data with the help of multi-layers. Deep learning is very helpful for predicting larger dataset. This paper proposes an algorithmic model known as a Comprehensive Deep Recurrent Artificial Neural Networks (CDRANN) by hybridizing LSTM (long short-term memory) with RNN (recurrent neural network) backtrack solver for better future stock prediction. This paper also explains the need for hybridizing evolutionary model CDRANN to create a novel change in future prediction.KeywordsCDRANN (Comprehensive Deep Recurrent Artificial Neural Networks)LSTM (long short-term memory)RNN (recurrent neural network)Stock marketPrediction

The adoption of wind energy has grown significantly in recent years. New, cost-effective technologies have been developed, led by customer awareness of green technologies and a legal framework proposed at the European Union level. The stochastic nature of wind speed is transferred to wind turbine output, making wind energy difficult to predict. The main scope of predicting wind energy production is to be proactive in balancing and reserving energy to meet demand. When the prediction identifies a potential gap between supply and demand, additional energy from other sources must be generated and supplied. Creating a synergy of physical devices through advanced sensing capabilities, software, storage and analytics capabilities, the Industrial Internet of Things is enabling the effective transition to wind energy through automation by removing many of the disadvantages in a way that has recently become accessible. This research focuses on the data analytics, proposing a fast univariate network-based approach for wind energy prediction, using Feed Forward Neural Networks, Recurrent Neural Networks, Long-Short Term Memory, Gated Recurrent Unit, and Convolutional Neural Networks. Moreover, by introducing the theoretical fundamentals, the implementation method and the hyperparameters of the final models, this article becomes unique in the context of wind energy. At the time of this study, no prior research studies have presented a direct comparison between feedforward, recurrent, and convolutional neural networks ? these being the most important in the field of supervised learning.

Stock price returns forecasting is challenging task for day traders to yield more returns. In the past, most of the literature was focused on machine learning algorithm to predict the stock returns. In this work, the recurrent neural network (RNN) with long short term memory (LSTM) is studied to forecast future stock returns. It has the ability to keep the memory of historical stock returns in order to forecast future stock return output. RNN with LSTM is used to store recent stock information than old related stock information. We have considered a recurrent dropout in RNN layers to avoid overfitting in the model. To accomplish the task we have calculated stock return based on stock closing prices. These stock returns are given as input to the recurrent neural network. The objective function of the prediction model is to minimize the error in the model. To conduct the experiment, data is collected from the National Stock Exchange, India (NSE). The proposed RNN with LSTM model outperforms compared to an feed forward artificial neural network.

In recent years, the artificial intelligence computing paradigm represented by physics-informed neural networks (PINNs) has received great attention in the field of plasma numerical simulation. However, the plasma chemical system considered in related research is relatively simplified, and the research on solving the more complex multi-particle low-temperature fluid model based on PINNs is still blank. In more complex chemical systems, the coupling relationship between particle densities and between particle densities and mean electron energy become more intricate. Therefore, the applicability of PINNs in dealing with sophisticated reaction systems needs further exploring and improving. In this work, we propose a general PINN framework (source term decoupled PINNs, Std-PINNs) for solving multi-particle low-temperature plasma fluid model. By introducing equivalent positive ions and replacing each particle transport equation with the current continuity equation as a physical constraint, Std-PINN splits the entire solution process into the training processes of two neural networks, realizing the decoupling of the source term of the heavy particle transport equation from the electron density and mean electron energy, which greatly reduces the complexity of neural network training. In this work, the application of Std-PINNs to solving multi-particle low-temperature plasma fluid models is demonstrated through two classic discharge cases with different complexity of reaction systems (low-pressure argon glow discharge and atmospheric-pressure helium glow discharge) and the performance of Std-PINN is compared with that of conventional PINN and finite element method (FEM). The results show that the training results output from the traditional PINN are completely incorrect due to the strong coupling correlation of each physical variable through the source terms of each particle transport equation, while the <i>L</i><sub>2</sub> relative error between Std-PINN and FEM results can reach up to ~10<sup>–2</sup> , thus verifying the feasibility of Std-PINN in simulating multi-particle plasma fluid model. Std-PINN expands the application of deep learning method to modeling complex physical systems and provides new ideas for conducting low-temperature plasma simulations. In addition, this study provides novel insights into the field of artificial intelligence scientific computing: the mathematical form that describes the state of a physical system is not unique. By introducing equivalent physical variables, equations suitable for neural network solutions can be derived and combined with observable data to simplify problems.

The main neural network architectures applied for product demand forecasting are examined. Particular attention is given to Recurrent Neural Networks (RNN), which demonstrate high efficiency when working with time series and detecting dependencies within data. A detailed analysis is provided for the Long Short-Term Memory (LSTM) architecture, an advanced variant of RNN that effectively addresses the vanishing gradient problem, a common issue in traditional RNN. Due to its capability to retain information over extended periods, LSTM is ideal for tasks requiring the analysis of long-term dependencies, such as seasonal fluctuations in product demand. The application of Convolutional Neural Networks (CNN) is analyzed, which show strong performance when processing structured data, such as images or matrices. With their ability to effectively detect spatial dependencies, CNN can be useful for demand forecasting when complex relationships between various data features need to be considered, particularly for demand evaluations dependent on multiple factors. The architectures Feedforward Neural Networks (FNN), Gated Recurrent Units (GRU), Attention-based models (ABM) and Autoencoders (AE) are considered, which can also be applied to solve demand forecasting problems. A comparative analysis of the architectures is made in terms of forecast accuracy and ability to work with time series. By evaluating the training speed, data processing time, and reliability of these architectures in terms of the stability of results and forecast reliability, it has been determined that LSTM architectures are the most effective for product demand forecasting. These networks provide high accuracy and reliability of results and perform well with time series, which is crucial for forecasting demand under dynamic conditions.

Induction motors play a critical role in industrial operations and electric vehicles, yet their reliability is often compromised by stator winding inter-turn short-circuit (ITSC) faults. To mitigate costly downtimes, this study investigates machine learning-based fault detection methods, comparing Artificial Neural Networks (ANN), Long Short-Term Memory (LSTM) networks, and Physics-Informed Neural Networks (PINNs). A benchmark dataset comprising stator current signals, leakage flux, and varying load conditions was employed to train and evaluate the models. While classification across seven fault categories remained challenging, the proposed framework achieved outstanding results in binary classification (healthy vs. faulty). In particular, the LSTM model attained a near-perfect accuracy of 100%, and the PINN model achieved 99.32%, both surpassing the baseline ANN performance of 99.01%. These findings highlight the effectiveness of temporal modeling with LSTM and the added value of incorporating motor physics into PINNs, especially in scenarios with limited data. The results confirm that advanced machine learning models can significantly improve early fault detection, enabling more reliable predictive maintenance and reduced operational costs in industrial systems.

This paper presents a new technique for wheezes detection in respiratory sounds using a Long Short-Term Memory (LSTM), a specific type of Recurrent Neural Networks (RNNs). The purpose of this work is to develop an LSTM-based system and compare its classification performances to those obtained by a Multilayer Perceptron (MLP) feed-forward network. The MLP is a widely used neural network that has proven its efficiency in respiratory sound classification. Feed-forward networks do not consider time dependencies, while RNNs reach their limit in detecting dependencies when they occur at long time intervals. Because wheezing occurs over several consecutive intervals, we assume that LSTM takes into account the changing characteristics better than MLP and provides better results. Pulmonary sounds are characterized using the Mel-Frequency Cepstral Coefficients (MFCC) method before applying the LSTM-based classifier. As expected, the experimental tests show that LSTM takes advantage of the long-term dependencies observed in wheezing sounds to lead to better classification performances (accuracy of 91%).

Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mechanism concept can help RNNs remember and forget previous information. Representing the hidden layers of an RNN with more expressive operations (i.e., tensor products) helps it learn a more complex relationship between the current input and the previous hidden layer information. These ideas can generally improve RNN performances. In this paper, we proposed a novel RNN architecture that combine the concepts of gating mechanism and the tensor product into a single model. By combining these two concepts into a single RNN, our proposed models learn long-term dependencies by modeling with gating units and obtain more expressive and direct interaction between input and hidden layers using a tensor product on 3-dimensional array (tensor) weight parameters. We use Long Short Term Memory (LSTM) RNN and Gated Recurrent Unit (GRU) RNN and combine them with a tensor product inside their formulations. Our proposed RNNs, which are called a Long-Short Term Memory Recurrent Neural Tensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor Network (GRURNTN), are made by combining the LSTM and GRU RNN models with the tensor product. We conducted experiments with our proposed models on word-level and character-level language modeling tasks and revealed that our proposed models significantly improved their performance compared to our baseline models.

In this paper, we present a novel approach for driving style classification based on time series data. Instead of automatically learning the embedding vector for temporal representation of the input data with Recurrent Neural Networks, we propose a combination of Hyperdimensional Computing (HDC) for data representation in high-dimensional vectors and much simpler feed-forward neural networks. This approach provides three key advantages: first, instead of having a “black box” of Recurrent Neural Networks learning the temporal representation of the data, our approach allows to encode this temporal structure in high-dimensional vectors in a human-comprehensible way using the algebraic operations of HDC while only relying on feed-forward neural networks for the classification task. Second, we show that this combination is able to achieve at least similar and even slightly superior classification accuracy compared to state-of-the-art Long Short-Term Memory (LSTM)-based networks while significantly reducing training time and the necessary amount of data for successful learning. Third, our HDC-based data representation as well as the feed-forward neural network, allow implementation in the substrate of Spiking Neural Networks (SNNs). SNNs show promise to be orders of magnitude more energy-efficient than their rate-based counterparts while maintaining comparable prediction accuracy when being deployed on dedicated neuromorphic computing hardware, which could be an energy-efficient addition in future intelligent vehicles with tight restrictions regarding on-board computing and energy resources. We present a thorough analysis of our approach on a publicly available data set including a comparison with state-of-the-art reference models.