Towards Predicting Vehicular Data Consumption

Recurring wildfires pose a critical global issue as they undermine social and economic stability and jeopardize human lives. To effectively manage disasters and bolster community resilience, the development of wildfire susceptibility maps (WFSMs) has emerged as a crucial undertaking in recent years. In this research endeavor, two deep learning algorithms were leveraged to generate WFSMs using two distinct remote sensing datasets. Specifically, the Moderate-Resolution Imaging Spectroradiometer (MODIS) and Landsat-8 images were utilized to monitor wildfires that transpired during the year 2021. To develop an effective WFSM, two datasets were created by incorporating 599 wildfire locations with Landsat-8 images and 232 sites with MODIS images, as well as twelve factors influencing wildfires. Deep learning algorithms, namely the long short-term memory (LSTM) and recurrent neural network (RNN), were utilized to model wildfire susceptibility using the two datasets. Subsequently, four WFSMs were generated using the LSTM (MODIS), LSTM (Landsat-8), RNN (MODIS), and RNN (Landsat-8) algorithms. The evaluation of the WFSMs was performed using the area under the receiver operating characteristic (ROC) curve (AUC) index. The results revealed that the RNN (MODIS) (AUC = 0.971), RNN (Landsat-8) (AUC = 0.966), LSTM (MODIS) (AUC = 0.964), and LSTM (Landsat-8) (AUC = 0.941) algorithms demonstrated the highest modeling accuracy, respectively. Moreover, the Gini index was employed to assess the impact of the twelve factors on wildfires in the study area. The results of the random forest (RF) algorithm indicated that temperature, wind speed, slope, and topographic wetness index (TWI) parameters had a significant effect on wildfires in the study region. These findings are instrumental in facilitating efficient wildfire management and enhancing community resilience against the detrimental effects of wildfires.

Prediction concept needs human-like thinking and ideas. ANN helps make a program for describing the solutions to the prediction problems. To gain accurate prediction, there is a need for deep thinking in the knowledge system discovered. Deep learning helps form such next-level type of ANN thinking. Deep learning models do not require humans for programming the problems. It can determine the prediction by itself by learning the dataset. This paper describes the prediction by considering sequential model for continuous accessing of data with the help of multi-layers. Deep learning is very helpful for predicting larger dataset. This paper proposes an algorithmic model known as a Comprehensive Deep Recurrent Artificial Neural Networks (CDRANN) by hybridizing LSTM (long short-term memory) with RNN (recurrent neural network) backtrack solver for better future stock prediction. This paper also explains the need for hybridizing evolutionary model CDRANN to create a novel change in future prediction.KeywordsCDRANN (Comprehensive Deep Recurrent Artificial Neural Networks)LSTM (long short-term memory)RNN (recurrent neural network)Stock marketPrediction

The failure of water chiller will cause a series of problems such as increasing energy consumption, decreasing comfort of users and decreasing life of equipment. The fault diagnosis of the chiller has an important role in the stable operation of the air conditioner. In order to improve the fault diagnosis level of water chillers and solve the problems of difficulty in determining the parameters of neural network and great randomness, this paper presents a fault diagnosis method of long short-term memory (LSTM) network based on whale optimization algorithm (WOA) optimization. The fault diagnosis performance was verified by RP-1043 dataset, and compared with the results of recurrent neural network (RNN) and BP neural network. The results show that the WOA-LSTM method can effectively identify five typical chillers, and the diagnostic accuracy is as high as 99.27%. Compared with RNN and BP neural network, the proposed method is helpful to detect faults as early as possible and reduce the loss. It is especially obvious to improve the detection efficiency of small faults and complex faults.

Hydrate plugging and pipeline leak can impair the normal operation of natural gas pipeline and may lead to serious accidents. Since natural gas pipeline safety monitoring based on active acoustic excitation can detect and locate not only the two abnormal events but also normal components such as valves and pipeline elbows, recognition and classification of these events are of great importance to provide maintenance guidance for the pipeline operators and avoid false alarm. In this article, long short-term memory (LSTM) network is introduced and applied to classify detection signals of hydrate plugging, pipeline leak, and elbow. Adaptive moment estimation (Adam) algorithm is introduced and utilized to accelerate the long short-term memory network convergence in training. Experimental results demonstrate that the network with three layers and 64 units per cell performs the best. The cross-entropy loss in training is 0.0005, and classification accuracies are all 100% in training, validation, and testing which verify the validity of the long short-term memory network. Therefore, the method based on the long short-term memory network and adaptive moment estimation algorithm can work efficiently on pipeline events classification and has great guiding significance for safety assurance of natural gas transmission.

Recurrent neural networks (RNNs) such as Long Short Term Memory (LSTM) networks have become popular in a variety of applications such as image processing, data classification, speech recognition, and as controllers in autonomous systems. In practical settings, there is often a need to deploy such RNNs on resource-constrained platforms such as mobile phones or embedded devices. As the memory footprint and energy consumption of such components become a bottleneck, there is interest in compressing and optimizing such networks using a range of heuristic techniques. However, these techniques do not guarantee the safety of the optimized network, e.g., against adversarial inputs, or equivalence of the optimized and original networks. To address this problem, we propose DiffRNN, the first differential verification method for RNNs to certify the equivalence of two structurally similar neural networks. Existing work on differential verification for ReLU-based feed-forward neural networks does not apply to RNNs where nonlinear activation functions such as Sigmoid and Tanh cannot be avoided. RNNs also pose unique challenges such as handling sequential inputs, complex feedback structures, and interactions between the gates and states. In DiffRNN, we overcome these challenges by bounding nonlinear activation functions with linear constraints and then solving constrained optimization problems to compute tight bounding boxes on non-linear surfaces in a high-dimensional space. The soundness of these bounding boxes is then proved using the dReal SMT solver. We demonstrate the practical efficacy of our technique on a variety of benchmarks and show that DiffRNN outperforms state-of-the-art RNN verification tools such as Popqorn.

The developed research includes an Long Short-Term Memory (LSTM) network with IoT sensor data for the management of rice crop irrigation. Optimizing the amount of water required for rice crops is very critical, based on real-time predictions of the actual environmental and soil parameters including moisture, temperature, humidity, and rainfall as well as all other possible related factors. IoT sensors installed in agricultural fields gathered data, which was transmitted to a controller and then uploaded to the cloud for further processing. The training dataset contained 5,600 records. In this research, 70% was used for training and the rest 30% for testing. Three LSTM models were used for analyzing sensor data and predicting the need for irrigation and controlling the irrigation pump accordingly. These were Vanilla LSTM, Stacked LSTM, and Bidirectional LSTM. It was evaluated the performance of the models developed using precision, recall, F1 score, accuracy, and other performance measures such as Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Score. Bidirectional LSTM was determined to have the best performance, with MAE at 0.15, RMSE at 0.24, and an R2 score of 0.93, exhibiting high accuracy and predictive power. The Stacked LSTM and Vanilla LSTM also gave fair results, in terms of having R2 values of 0.89 and 0.84, respectively. The number of training and memory used for the computation were also given, with Vanilla LSTM taking least, followed by Stacked LSTM and Bidirectional LSTM. These results suggest that the LSTM networks can potentially be capable of improving the irrigation management such that more water is conserved and, subsequently, more suitable for agricultural productivity.

Most Graph Neural Networks (GNNs) follow the message-passing scheme. Residual connection is an effective strategy to tackle GNNs' over-smoothing issue and performance reduction issue on non-homophilic networks. Unfortunately, the coarse-grained residual connection still suffers from class-imbalanced over-smoothing issue, due to the fixed and linear combination of topology and attribute in node representation learning. To make the combination flexible to capture complicated relationship, this paper reveals that the residual connection needs to be node-dependent, layer-dependent, and related to both topology and attribute. To alleviate the difficulty in specifying complicated relationship, this paper presents a novel perspective on GNNs, i.e., the representations of one node in different layers can be seen as a sequence of states. From this perspective, existing residual connections are not flexible enough for sequence modeling. Therefore, a novel node-dependent residual connection, i.e., Long Short-Term Graph Memory Network (LSTGM) is proposed to employ Long Short-Term Memory (LSTM), to model the sequence of node representation. To make the graph topology fully employed, LSTGM innovatively enhances the updated memory and three gates with graph topology. A speedup version is also proposed for effective training. Experimental evaluations on real-world datasets demonstrate their effectiveness in preventing over-smoothing issue and handling networks with heterophily.

This paper aims to predict the occurrence of pests and diseases for cotton based on long short term memory (LSTM) network. First, the problem of occurrence of pests and diseases was formulated as time series prediction. Then LSTM was adopted to solve the problem. LSTM is a special kind of recurrent neutral network (RNN), which introduces gate mechanism to prevent the vanished or exploding gradient problem. It has been shown good performance in solving time series problem and can handle the long-term dependency problem, as mentioned in many literatures. The experimental results showed that LSTM performed good on the prediction of occurrence of pests and diseases in cotton fields, and yielded an Area Under the Curve (AUC) of 0.97. The paper further verified that the weather factors indeed have strong impact on the occurrence of pests and diseases, and the LSTM network has great advantage on solving the long-term dependency problem.

Temperature sensing is one of the most common measurements of a nuclear reactor monitoring system. The coolant fluid flow in a reactor core depends on the reactor power state. We investigated the monitoring and estimation of the thermocouple time series using machine learning for a range of flow regimes. Measurement data were obtained, in two separate experiments, in a flow loop filled with water and with liquid metal Galinstan. We developed long short-term memory (LSTM) recurrent neural networks (RNNs) for sensor predictions by training on the sensor’s own prior history, and transfer learning LSTM (TL-LSTM) by training on a correlated sensor’s prior history. Sensor cross-correlations were identified by calculating the Pearson correlation coefficient of the time series. The accuracy of LSTM and TL-LSTM predictions of temperature was studied as a function of Reynolds number (Re). The root-mean-square error (RMSE) for the test segment of time series of each sensor was shown to linearly increase with Re for both water and Galinstan fluids. Using linear correlations, we estimated the range of values of Re for which RMSE is smaller than the thermocouple measurement uncertainty. For both water and Galinstan fluids, we showed that both LSTM and TL-LSTM provide reliable estimations of temperature for typical flow regimes in a nuclear reactor. The LSTM runtime was shown to be substantially smaller than the data acquisition rate, which allows for performing estimation and validation of sensor measurements in real time.

Modern decision-making in fixed income asset management benefits from intelligent systems, which involve the use of state-of-the-art machine learning models and appropriate methodologies. We conduct the first study of bond yield forecasting using long short-term memory (LSTM) networks, validating its potential and identifying its memory advantage. Specifically, we model the 10-year bond yield using univariate LSTMs with three input sequences and five forecasting horizons. We compare those with multilayer perceptrons (MLP), univariate and with the most relevant features. To demystify the notion of black box associated with LSTMs, we conduct the first internal study of the model. To this end, we calculate the LSTM signals through time, at selected locations in the memory cell, using sequence-to-sequence architectures, uni and multivariate. We then proceed to explain the states’ signals using exogenous information, for what we develop the LSTM-LagLasso methodology. The results show that the univariate LSTM model with additional memory is capable of achieving similar results as the multivariate MLP using macroeconomic and market information. Furthermore, shorter forecasting horizons require smaller input sequences and vice-versa. The most remarkable property found consistently in the LSTM signals, is the activation/deactivation of units through time, and the specialisation of units by yield range or feature. Those signals are complex but can be explained by exogenous variables. Additionally, some of the relevant features identified via LSTM-LagLasso are not commonly used in forecasting models. In conclusion, our work validates the potential of LSTMs and methodologies for bonds, providing additional tools for financial practitioners.

Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mechanism concept can help RNNs remember and forget previous information. Representing the hidden layers of an RNN with more expressive operations (i.e., tensor products) helps it learn a more complex relationship between the current input and the previous hidden layer information. These ideas can generally improve RNN performances. In this paper, we proposed a novel RNN architecture that combine the concepts of gating mechanism and the tensor product into a single model. By combining these two concepts into a single RNN, our proposed models learn long-term dependencies by modeling with gating units and obtain more expressive and direct interaction between input and hidden layers using a tensor product on 3-dimensional array (tensor) weight parameters. We use Long Short Term Memory (LSTM) RNN and Gated Recurrent Unit (GRU) RNN and combine them with a tensor product inside their formulations. Our proposed RNNs, which are called a Long-Short Term Memory Recurrent Neural Tensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor Network (GRURNTN), are made by combining the LSTM and GRU RNN models with the tensor product. We conducted experiments with our proposed models on word-level and character-level language modeling tasks and revealed that our proposed models significantly improved their performance compared to our baseline models.

Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, and Gated Recurrent Units (GRUs) have gained significant popularity in time series forecasting across diverse domains including healthcare, astronomy, and engineering. However, the inherent variability in model performance due to random weight initialization raises questions about the reliability and consistency of these architectures for time series analysis. This study addresses this concern by conducting a comprehensive benchmark evaluation of nine neural network architectures: vanilla RNN, LSTM, GRU, and six hybrid configurations (RNN-LSTM, RNN-GRU, LSTM-RNN, GRU-RNN, LSTM-GRU, and GRU-LSTM). Performance evaluation was conducted using Monte Carlo simulation with 100 iterations across three real-world datasets: sunspot activity, Indonesian COVID-19 cases, and dissolved oxygen concentration measurements. Statistical analysis employed the Friedman test to assess performance differences across architectures. Results showed no statistically significant differences among the nine architectures. Despite the lack of statistical significance, consistent performance patterns emerged favoring LSTM-based hybrid architectures. The LSTM-GRU and LSTM-RNN configurations demonstrated superior performance across multiple evaluation metrics, with LSTM-RNN excelling in sunspot and dissolved oxygen forecasting, while standalone LSTM showed optimal performance for COVID-19 prediction. These findings provide evidence-based guidance for architecture selection in time series forecasting applications, suggesting that while statistical equivalence exists among architectures, LSTM-based hybrids offer practical advantages in terms of consistency and robustness across diverse temporal patterns.

We explore the architecture of recurrent neural networks (RNNs) by studying the complexity of string sequences it is able to memorize. Symbolic sequences of different complexity are generated to simulate RNN training and study parameter configurations with a view to the network's capability of learning and inference. We compare Long Short-Term Memory (LSTM) networks and gated recurrent units (GRUs). We find that an increase in RNN depth does not necessarily result in better memorization capability when the training time is constrained. Our results also indicate that the learning rate and the number of units per layer are among the most important hyper-parameters to be tuned. Generally, GRUs outperform LSTM networks on low-complexity sequences while on high-complexity sequences LSTMs perform better.

Synchronous multi-parameter prediction of battery systems on electric vehicles using long short-term memory networks

Photovoltaic power generation is one of the most efficient renewable energy generation techniques. To ensure photovoltaic (PV) safe operation and cost-effective integration into smart grids. As a result, a realistic PV power forecast method can aid in mitigating PV power generation’s drawbacks, which is critical for power plant maintenance and repair. In this research, we compared two Deep Learning models which are both highly advantageous, namely a long short-term memory (LSTM) and a recurrent neural network (RNN). A case study employing an actual dataset obtained from 1B DKASC, Alice Springs, Australia, is used to demonstrate the performance of LSTM and RNN. The forecasts generated by the LSTM are compared with the results of RNN using MAE, MSE, MBE, R\(^2\) and RMSE for all the forecast horizons. The results of the study demonstrate that the LSTM based forecast outperforms RNN and has the potential to improve the accuracy and stability of the prediction.KeywordsDeep LearningLong Short-Term Memory (LSTM)Recurrent neural network (RNN)