Comprehensive system based on a DNN and LSTM for predicting sinter composition

Adaptive Cruise Control (ACC) is an important part of automotive autonomy. To have a robust Adaptive Cruise Control system, a well-defined sensor fusion system is required. This paper focuses on feature-level sensor fusion system where three fusion techniques are explored: independent Kalman filtering, interacting multiple model (IMM) filtering, and a novel hybrid technique using a combination of both an IMM Filter and deep neural network (DNN). Kalman Filtering is a popular approach for automotive sensor fusion, but heavily relies on the given kinematic models to make state estimations. We explore the IMM filter to capture multi-model motion as well as a DNN approach to build a data-driven model of the system. Our IMM filter is built with several Kalman Filters featuring different kinematic models. The proposed DNN used is a Long Short-Term Memory (LSTM) network trained on radar and camera data to forecast a vehicle's state. To overcome the weaknesses in both IMM filtering and LSTM networks, we propose a hybrid technique of consisting of both the IMM filter and the LSTM network. The proposed hybrid system uses a single filter for each sensor. The filtered sensor data is synchronized and used as the input for a trained LSTM network. Our LSTM network was trained on over 100 simulated highway driving scenarios that might occur in an ACC application. Simulations and code were developed using MATLAB and the Autonomous Driving Toolbox. Our hybrid IMM-LSTM system outperformed the independent Kalman Filtering approach in longitudinal and lateral tracking accuracy (RMSE) by 23 percent and had promising results compared to the IMM filter where the tracking error was decreased by nearly 50 percent in certain driving scenarios.

The developed research includes an Long Short-Term Memory (LSTM) network with IoT sensor data for the management of rice crop irrigation. Optimizing the amount of water required for rice crops is very critical, based on real-time predictions of the actual environmental and soil parameters including moisture, temperature, humidity, and rainfall as well as all other possible related factors. IoT sensors installed in agricultural fields gathered data, which was transmitted to a controller and then uploaded to the cloud for further processing. The training dataset contained 5,600 records. In this research, 70% was used for training and the rest 30% for testing. Three LSTM models were used for analyzing sensor data and predicting the need for irrigation and controlling the irrigation pump accordingly. These were Vanilla LSTM, Stacked LSTM, and Bidirectional LSTM. It was evaluated the performance of the models developed using precision, recall, F1 score, accuracy, and other performance measures such as Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Score. Bidirectional LSTM was determined to have the best performance, with MAE at 0.15, RMSE at 0.24, and an R2 score of 0.93, exhibiting high accuracy and predictive power. The Stacked LSTM and Vanilla LSTM also gave fair results, in terms of having R2 values of 0.89 and 0.84, respectively. The number of training and memory used for the computation were also given, with Vanilla LSTM taking least, followed by Stacked LSTM and Bidirectional LSTM. These results suggest that the LSTM networks can potentially be capable of improving the irrigation management such that more water is conserved and, subsequently, more suitable for agricultural productivity.

Modern decision-making in fixed income asset management benefits from intelligent systems, which involve the use of state-of-the-art machine learning models and appropriate methodologies. We conduct the first study of bond yield forecasting using long short-term memory (LSTM) networks, validating its potential and identifying its memory advantage. Specifically, we model the 10-year bond yield using univariate LSTMs with three input sequences and five forecasting horizons. We compare those with multilayer perceptrons (MLP), univariate and with the most relevant features. To demystify the notion of black box associated with LSTMs, we conduct the first internal study of the model. To this end, we calculate the LSTM signals through time, at selected locations in the memory cell, using sequence-to-sequence architectures, uni and multivariate. We then proceed to explain the states’ signals using exogenous information, for what we develop the LSTM-LagLasso methodology. The results show that the univariate LSTM model with additional memory is capable of achieving similar results as the multivariate MLP using macroeconomic and market information. Furthermore, shorter forecasting horizons require smaller input sequences and vice-versa. The most remarkable property found consistently in the LSTM signals, is the activation/deactivation of units through time, and the specialisation of units by yield range or feature. Those signals are complex but can be explained by exogenous variables. Additionally, some of the relevant features identified via LSTM-LagLasso are not commonly used in forecasting models. In conclusion, our work validates the potential of LSTMs and methodologies for bonds, providing additional tools for financial practitioners.

This paper aims to predict the occurrence of pests and diseases for cotton based on long short term memory (LSTM) network. First, the problem of occurrence of pests and diseases was formulated as time series prediction. Then LSTM was adopted to solve the problem. LSTM is a special kind of recurrent neutral network (RNN), which introduces gate mechanism to prevent the vanished or exploding gradient problem. It has been shown good performance in solving time series problem and can handle the long-term dependency problem, as mentioned in many literatures. The experimental results showed that LSTM performed good on the prediction of occurrence of pests and diseases in cotton fields, and yielded an Area Under the Curve (AUC) of 0.97. The paper further verified that the weather factors indeed have strong impact on the occurrence of pests and diseases, and the LSTM network has great advantage on solving the long-term dependency problem.

Accurate electrical load forecasting is of great significance to help power companies in better scheduling and efficient management. Since high levels of uncertainties exist in the load time series, it is a challenging task to make accurate short-term load forecast (STLF). In recent years, deep learning approaches provide better performance to predict electrical load in real world cases. The convolutional neural network (CNN) can extract the local trend and capture the same pattern, and the long short-term memory (LSTM) is proposed to learn the relationship in time steps. In this paper, a new deep neural network framework that integrates the hidden feature of the CNN model and the LSTM model is proposed to improve the forecasting accuracy. The proposed model was tested in a real-world case, and detailed experiments were conducted to validate its practicality and stability. The forecasting performance of the proposed model was compared with the LSTM model and the CNN model. The Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE) were used as the evaluation indexes. The experimental results demonstrate that the proposed model can achieve better and stable performance in STLF.

Flash floods occur frequently and distribute widely in mountainous areas because of complex geographic and geomorphic conditions and various climate types. Effective flash flood forecasting with useful lead times remains a challenge due to its high burstiness and short response time. Recently, machine learning has led to substantial changes across many areas of study. In hydrology, the advent of novel machine learning methods has started to encourage novel applications or substantially improve old ones. This study aims to establish a discharge forecasting model based on Long Short-Term Memory (LSTM) networks for flash flood forecasting in mountainous catchments. The proposed LSTM flood forecasting (LSTM-FF) model is composed of T multivariate single-step LSTM networks and takes spatial and temporal dynamics information of observed and forecast rainfall and early discharge as inputs. The case study in Anhe revealed that the proposed models can effectively predict flash floods, especially the qualified rates (the ratio of the number of qualified events to the total number of flood events) of large flood events are above 94.7% at 1–5 h lead time and range from 84.2% to 89.5% at 6–10 h lead-time. For the large flood simulation, the small flood events can help the LSTM-FF model to explore a better rainfall-runoff relationship. The impact analysis of weights in the LSTM network structures shows that the discharge input plays a more obvious role in the 1-h LSTM network and the effect decreases with the lead-time. Meanwhile, in the adjacent lead-time, the LSTM networks explored a similar relationship between input and output. The study provides a new approach for flash flood forecasting and the highly accurate forecast contributes to prepare for and mitigate disasters.

Utilization of the Long Short-Term Memory network for predicting streamflow in ungauged basins in Korea

&lt;p dir="ltr"&gt;Recurrent Neural Networks (RNNs) are a major class of Artificial Neural Networks (ANNs). Suitable network architecture is vital for RNNs to achieve high performance. However, designing the architecture of deep RNNs (DRNNs) is a complicated and time-consuming process. Each layer of a DRNN contains numerous hyper-parameters. It is challenging to optimize the hyper-parameters of all layers jointly to achieve the best possible learning performance. Trial-and-error methods for designing DRNNs have been proven to be laborious and highly costly in practice. Therefore, an efficient automatic architecture search technique is needed to design DRNNs.&lt;/p&gt;&lt;p dir="ltr"&gt;Genetic Algorithm (GA) is a popular evolutionary computation based approach. Existing research works already explored the use of GAs in designing ANNs. But, most of the existing approaches focus on the design of fixed-depth ANNs. However, it is not easy to fix the depth of DRNNs in advance, since different problems need DRNNs of different depths. Hence, the potential of GA has to be further explored to evolve the architecture of DRNNs of varying depths.&lt;/p&gt;&lt;p dir="ltr"&gt;The primary goal of this thesis is to develop advanced GA approaches for designing Long Short Term Memory (LSTM) based DRNN architectures of varying depths effectively and efficiently.&lt;/p&gt;&lt;p dir="ltr"&gt;Firstly, this thesis proposes a GA-based algorithm called Two-Stage Surrogate-Assisted GA (TS-SA-GA) with a progressive incremental strategy to design the architecture of LSTM networks of varying depths. The progressive approach can effectively extend well-designed shallow networks to high performing deep networks. Moreover, the new GA-based algorithm adopts newly designed knowledge-driven crossover and mutation operators to identify and repair LSTM network designs affected by inappropriate use of activation functions, thereby significantly reducing the chances for GA to evolve hard-to-train LSTM network architectures. Furthermore, this thesis proposes a two-stage surrogate method to predict the trainability and fitness of LSTM networks. This improves the efficiency and effectiveness of the GA-based algorithm to evolve LSTM networks.&lt;/p&gt;&lt;p dir="ltr"&gt;Secondly, this thesis proposes a new algorithm called Evolutionary Design of LSTM Ensembles (ED-LSTM-Ensemble) based on multi-objective optimization techniques to evolve LSTM networks and ensembles simultaneously to directly utilize the ensemble performance to drive the evolution of base LSTM networks. Additionally, this thesis proposes a connection weight inheritance strategy to evolve LSTM networks and ensembles efficiently and effectively.&lt;/p&gt;&lt;p dir="ltr"&gt;Thirdly, this thesis proposes a new algorithm called Skip Connections through Evolutionary and Differential Architecture search (SCEDA) to design LSTM networks together with appropriate skip connections automatically. Designing LSTM networks together with skip connections is highly challenging. This thesis hybridize gradient-based DAS and evolutionary architecture search to optimize the architecture of LSTM networks with suitable skip connections in a single evolutionary process to improve the efficiency and effectiveness of the search process.&lt;/p&gt;&lt;p dir="ltr"&gt;Finally, this thesis conducts an in-depth empirical analysis of the impact of activation functions on the trainability of LSTM networks based on the concept of Edge Of Chaos. The analysis reveals the strong interrelation between the activation functions across multiple layers of an LSTM network and the trainability of the network. The analysis also highlights the importance of controlling activation functions to avoid generating untrainable LSTM networks. On the basis of the analysis, this thesis proposes a machine learning model to guide the use of activation functions in LSTM networks.&lt;/p&gt;

Crop growth monitoring is an important component of agricultural information, and suitable soil temperature (ST), soil moisture content (SMC) and soil electrical conductivity (SEC) play a key role in crop growth. Real-time monitoring of the three soil parameters to predict the growth of tea plantation helps tea trees grow healthily and to accurately grasp the growth trend of tea trees. In this paper, five different models based on the polynomial model and power model were used to construct the soil temperature, soil water content and soil conductivity and tea plantation growth monitoring models. Experiments proved that tea plantation growth were positively correlated with ST and negatively correlated with SMC and SEC, and among the constructed models, the ternary cubic polynomial model was the best, and R square (R2) of the constructed models were 0.6369, 0.4510 and 0.5784, respectively, indicating that SEC was the most relevant to tea plantation growth maximum. To improve the prediction accuracy, a model based on sum of soil temperature (SST), sum of soil water content (SSMC) and sum of soil conductivity (SSEC) was proposed, and the experiments also showed that the ternary cubic polynomial model was the best, with 0.9638, 0.9733 and 0.9660, respectively. At the same time, a model incorporating three parameters such as soil temperature, soil water content and soil conductivity was also suggested, with 0.6605 and 0.9761, respectively, which effectively improved the prediction accuracy. Validation experiments were conducted. Twelve data sets were utilized to verify the performance of the model. The experiments showed that the regressions in the polynomial models achieved a better prediction effect. Finally, a long short-term memory (LSTM) network prediction model optimized by the bald eagle search algorithm (BES) was also constructed, and R2, root mean square error (RMSE), mean squared error (MSE), mean absolute error (MAE) and mean absolute percentage error (MAPE) of prediction were 0.8666, 0.0629, 0.0040, 0.0436 and 10.5257, respectively, which significantly outperformed the LSTM network and achieved better performance. The model proposed in this paper can be used to predict the actual situation during the growing period of tea leaves, which can improve the production management of tea plantations and also provide a scientific basis for accurate tea planting and a decision basis for agricultural policy formulation, as well as provide technical support for the realization of agricultural modernization.

Railway turnout is one of the weakest elements in railway infrastructure, whose normal operation is directly related to the safety of the passing trains. The railway switch gap is an important part of the indication mechanism of the turnout. Once the size of the switch gap exceeds the standard, the indication mechanism of the turnout will fail, causing the close of the railway route and endangering the safety of the passing trains. However, at present, the maintenance of the switch gap size still follows the traditional failure based maintenance (FBM) strategy. Therefore, this paper first analyzes the causes of the change of the switch gap size, and constructs a regression and autoregressive integrated moving average (RegARIMA) model to describe the change law of the switch gap size. Then, a size prediction method for the switch gap is proposed by utilizing this model and a long short-term memory (LSTM) network. By referring to the prediction results of this method, the maintenance personnel can make maintenance plan before the switch gap size exceeds the standard, so as to avoid the failure of the turnout. Finally, the experimental results based on condition monitoring data from a turnout show that the root mean square error, the mean absolute error and the mean absolute percentage error of the proposed prediction method are 0.0732mm, 0.0568mm and 2.8670% respectively, which means the proposed model can effectively describe the change law of the switch gap size, and the proposed prediction method has good prediction performance.

The prediction of wind energy generation is important to enhance the performance and dependability of renewable energy systems due to the rising demand for wind-generated electricity and advancements in wind energy technology competitiveness. This study leverages advanced machine learning (ML) and some other statistical and deep learning based time series forecasting models to enhance the accuracy of wind energy predictions. This comprehensive analysis includes nine ML models—Linear Regression, Random Forests (RF), Gradient Boosting Machines (GBM), Support Vector Machines (SVM), K-Nearest Neighbors (KNN), AdaBoost, XGBoost, Support Vector Regression (SVR), and Neural Networks—as well as Four time-series forecasting models—ARIMA, Temporal Convolutional Networks (TCNs), Long Short-Term Memory (LSTM) networks and GRU. Each ML model underwent rigorous cross-validation to ensure optimal performance. The assessment criteria utilized here comprised the Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and the R² Score. It was found that among the nine ML models, Random Forests, GBM and KNN consistently provided superior accuracy and robustness, making them the top choices for wind energy prediction whereas the performance of linear regression, SVM and SVR were very poor for the considered dataset. From the experiment, Random Forest, GBM, and KNN showed the best performance with low MSE values of 0.77, 1.95, and 1.51 respectively, while other models had MSEs above 7.5, with AdaBoost reaching 30. Their RMSEs (0.88, 1.40, 1.23) and MAEs (0.093, 0.73, 0.10) also indicate strong predictive accuracy compared to the rest.In this paper, time series forecasting, TCNs, LSTM and GRU networks showed strong capabilities in capturing temporal dependencies and trends within the wind energy data. Visualization techniques were employed to compare model performances comprehensively, providing clear insights into their predictive power. Therefore, this present study offers a robust framework for researchers and practitioners aiming to leverage machine learning and time series forecasting in the realm of renewable energy prediction.

Offshore wind turbines (OWTs) have complex operating conditions and are prone to accidents. Timely and accurate vibration prediction can reduce the probability of failure of OWTs. To address this issue, this study proposed a vibration prediction model based on long short-term memory (LSTM) network. The proposed prediction model was applied to an OWT in an offshore wind farm in China, and three evaluation indicators, i.e. mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE), were used to verify the vibration prediction performance of LSTM. The results demonstrated that the prediction accuracy of LSTM was better than that of back propagation (BP) neural network, and LSTM had optimum engineering practicability. The prediction accuracy of LSTM and BP neural networks decreased with an increase in prediction durations. However, the prediction accuracy of LSTM was higher. This study improves the online safety monitoring of OWTs.

Soft sensors have been extensively utilized to approximate real-time power prediction in wind power generation, which is challenging to measure instantaneously. The short-term forecast of wind power aims at providing a reference for the dispatch of the intraday power grid. This study proposes a soft sensor model based on the Long Short-Term Memory (LSTM) network by combining data preprocessing with Variational Modal Decomposition (VMD) to improve wind power prediction accuracy. It does so by adopting the isolation forest algorithm for anomaly detection of the original wind power series and processing the missing data by multiple imputation. Based on the process data samples, VMD technology is used to achieve power data decomposition and noise reduction. The LSTM network is introduced to predict each modal component separately, and further sum reconstructs the prediction results of each component to complete the wind power prediction. From the experimental results, it can be seen that the LSTM network which uses an Adam optimizing algorithm has better convergence accuracy. The VMD method exhibited superior decomposition outcomes due to its inherent Wiener filter capabilities, which effectively mitigate noise and forestall modal aliasing. The Mean Absolute Percentage Error (MAPE) was reduced by 9.3508%, which indicates that the LSTM network combined with the VMD method has better prediction accuracy.

As a kind of continuous time series, plantar pressure data contains rich contact of time information which has not been fully utilized in existing gait recognition methods. In this paper, we proposed a new gait recognition method based on plantar pressure data with a Long Short-Term Memory (LSTM) network. By normalization and dimensionality reduction, the raw pressure data was converted to feature tensor. Then we feed the LSTM network with the feature tensors and implement classification recognition. We collected data from 93 subjects of different age groups, and each subjects was collected 10 sets of pressure data. The experiment results turn out that our LSTM network can get high classification accuracy and performs better than CNN model and many traditional methods.

Seismic response prediction of RC bridge piers through stacked long short-term memory network