Forecasting Solar Power Using Long-Short Term Memory and Convolutional Neural Networks

Utilization of the Long Short-Term Memory network for predicting streamflow in ungauged basins in Korea

A general-purpose no-reference video quality assessment algorithm based on a long short-term memory (LSTM) network and a pretrained convolutional neural network (CNN) is introduced. Considering video sequences as a time series of deep features extracted with the help of a CNN, an LSTM network is trained to predict subjective quality scores. In contrast to previous methods, the resulting algorithm was trained on the recently published Konstanz Natural Video Quality Database (KoNViD-1k), which is the only publicly available database that contains sequences with authentic distortions. The results of experiments on KoNViD-1k demonstrate that the proposed method outperforms other state-of-the-art algorithms. Furthermore, these results are also confirmed using tests on the LIVE Video Quality Assessment Database, which consists of artificially distorted videos.

Modern decision-making in fixed income asset management benefits from intelligent systems, which involve the use of state-of-the-art machine learning models and appropriate methodologies. We conduct the first study of bond yield forecasting using long short-term memory (LSTM) networks, validating its potential and identifying its memory advantage. Specifically, we model the 10-year bond yield using univariate LSTMs with three input sequences and five forecasting horizons. We compare those with multilayer perceptrons (MLP), univariate and with the most relevant features. To demystify the notion of black box associated with LSTMs, we conduct the first internal study of the model. To this end, we calculate the LSTM signals through time, at selected locations in the memory cell, using sequence-to-sequence architectures, uni and multivariate. We then proceed to explain the states’ signals using exogenous information, for what we develop the LSTM-LagLasso methodology. The results show that the univariate LSTM model with additional memory is capable of achieving similar results as the multivariate MLP using macroeconomic and market information. Furthermore, shorter forecasting horizons require smaller input sequences and vice-versa. The most remarkable property found consistently in the LSTM signals, is the activation/deactivation of units through time, and the specialisation of units by yield range or feature. Those signals are complex but can be explained by exogenous variables. Additionally, some of the relevant features identified via LSTM-LagLasso are not commonly used in forecasting models. In conclusion, our work validates the potential of LSTMs and methodologies for bonds, providing additional tools for financial practitioners.

Adaptive Cruise Control (ACC) is an important part of automotive autonomy. To have a robust Adaptive Cruise Control system, a well-defined sensor fusion system is required. This paper focuses on feature-level sensor fusion system where three fusion techniques are explored: independent Kalman filtering, interacting multiple model (IMM) filtering, and a novel hybrid technique using a combination of both an IMM Filter and deep neural network (DNN). Kalman Filtering is a popular approach for automotive sensor fusion, but heavily relies on the given kinematic models to make state estimations. We explore the IMM filter to capture multi-model motion as well as a DNN approach to build a data-driven model of the system. Our IMM filter is built with several Kalman Filters featuring different kinematic models. The proposed DNN used is a Long Short-Term Memory (LSTM) network trained on radar and camera data to forecast a vehicle's state. To overcome the weaknesses in both IMM filtering and LSTM networks, we propose a hybrid technique of consisting of both the IMM filter and the LSTM network. The proposed hybrid system uses a single filter for each sensor. The filtered sensor data is synchronized and used as the input for a trained LSTM network. Our LSTM network was trained on over 100 simulated highway driving scenarios that might occur in an ACC application. Simulations and code were developed using MATLAB and the Autonomous Driving Toolbox. Our hybrid IMM-LSTM system outperformed the independent Kalman Filtering approach in longitudinal and lateral tracking accuracy (RMSE) by 23 percent and had promising results compared to the IMM filter where the tracking error was decreased by nearly 50 percent in certain driving scenarios.

Flash floods occur frequently and distribute widely in mountainous areas because of complex geographic and geomorphic conditions and various climate types. Effective flash flood forecasting with useful lead times remains a challenge due to its high burstiness and short response time. Recently, machine learning has led to substantial changes across many areas of study. In hydrology, the advent of novel machine learning methods has started to encourage novel applications or substantially improve old ones. This study aims to establish a discharge forecasting model based on Long Short-Term Memory (LSTM) networks for flash flood forecasting in mountainous catchments. The proposed LSTM flood forecasting (LSTM-FF) model is composed of T multivariate single-step LSTM networks and takes spatial and temporal dynamics information of observed and forecast rainfall and early discharge as inputs. The case study in Anhe revealed that the proposed models can effectively predict flash floods, especially the qualified rates (the ratio of the number of qualified events to the total number of flood events) of large flood events are above 94.7% at 1–5 h lead time and range from 84.2% to 89.5% at 6–10 h lead-time. For the large flood simulation, the small flood events can help the LSTM-FF model to explore a better rainfall-runoff relationship. The impact analysis of weights in the LSTM network structures shows that the discharge input plays a more obvious role in the 1-h LSTM network and the effect decreases with the lead-time. Meanwhile, in the adjacent lead-time, the LSTM networks explored a similar relationship between input and output. The study provides a new approach for flash flood forecasting and the highly accurate forecast contributes to prepare for and mitigate disasters.

With the rapid increase in bridge spans, the mitigation of risk to flutter (aeroelastic instability) is of critical importance in the design of long-span bridges, especially considering the more frequent intense hurricanes under climate change. Although the strong nonlinearities of the aeroelastic (self-excited) forces in wind–bridge interactions can be well captured through either numerical simulations or experimental tests, both are expensive and time consuming. Hence, it is important to develop an efficient reduced-order model for the simulations of nonlinear aeroelastic forces on the bridge decks. This study proposes a reduced-order model based on the long short-term memory (LSTM) network to simulate the nonlinear aeroelastic forces on bridge decks with various leading edges, and thus rapidly predict the corresponding post-flutter behaviors of long-span bridges. To generate the training datasets, computational fluid dynamics (CFD) was employed to simulate the nonlinear aeroelasticities of bridge decks with a wide range of leading-edge configurations and wind speeds. Trained on the high-fidelity CFD datasets, the LSTM network takes the motion of a bridge deck, leading-edge angles and wind speeds as inputs and outputs the nonlinear aeroelastic forces on the bridge decks. A hybrid loss function utilizing the prediction errors of both aeroelastic forces simulated by the LSTM network and the bridge deck responses calculated by the Newmark-β algorithm was introduced into the training process to improve the network performance. The prediction results of the trained LSTM model were compared with the CFD simulations, which demonstrated that the nonlinear aeroelastic forces of the bridge deck with various leading edges can be accurately and efficiently acquired by the proposed LSTM model.

In order to enhance the accuracy of thermal response estimation, this study proposes the utilization of a deep learning-driven method, specifically the Long Short-Term Memory (LSTM) network, for predicting temperature-induced strains and deflections in the Queensferry Crossing, UK. Initially, an examination of the temperature fields of the bridge is conducted based on monitored temperature data. Subsequently, the LSTM network is introduced as a tool to estimate thermal response utilizing temperature field information. The study proceeds to discuss the results of temperature-induced response estimation, including strains and deflections, obtained through the LSTM network, with a comparative analysis against a linear regression model. It is observed that the temperature discrepancies among steel components within the bridge are lesser in comparison to those of concrete components. The LSTM network exhibits higher estimation accuracy than the linear model, manifesting in an improvement of mean square error from 4.45 to 0.89 for strain estimation. In addition, the LSTM network can partially address time-lag effects. However, the estimation performance of the LSTM network is influenced by the structural response type and the resolution of employed sensors. Specifically, the correlation coefficient (R) associated with strain estimation is measured at 0.980 using the LSTM network, whereas R for deflection estimation stands at 0.825. Generally, local structural responses, such as strains, outperform global responses like deflections. Furthermore, sensors with higher resolutions yield enhanced estimation accuracy.

To address the issues of slow convergence speed, poor dynamic adaptability, and path redundancy in the Double Deep Q Network (DDQN) within complex obstacle environments, this paper proposes an enhanced algorithm within the deep reinforcement learning framework. This algorithm, termed LPDDQN, integrates Prioritized Experience Replay (PER) and the Long Short Term Memory (LSTM) network to improve upon the DDQN algorithm. First, Prioritized Experience Replay (PER) is utilized to prioritize experience data and optimize storage and sampling operations through the SumTree structure, rather than the conventional experience queue. Second, the LSTM network is introduced to enhance the dynamic adaptability of the DDQN algorithm. Owing to the introduction of the LSTM model, the experience samples must be sliced and populated. The performance of the proposed LPDDQN algorithm is compared with five other path planning algorithms in both static and dynamic environments. Simulation analysis shows that in a static environment, LPDDQN demonstrates significant improvements over traditional DDQN in terms of convergence, number of moving steps, success rate, and number of turns, with respective improvements of 24.07%, 17.49%, 37.73%, and 61.54%. In dynamic and complex environments, the success rates of all algorithms, except TLD3 and the LPDDQN, decreased significantly. Further analysis reveals that the LPDDQN outperforms the TLD3 by 18.87%, 2.41%, and 39.02% in terms of moving steps, success rate, and number of turns, respectively.

To explore the potential relationship between the leading vehicle and the following vehicle during car-following, we proposed a novel car-following model combining a convolutional neural network (CNN) with a long short-term memory (LSTM) network. Firstly, 400 car-following periods were extracted from the natural driving database and the OpenACC car-following experiment database. Then, we developed a CNN-LSTM car-following model, and the CNN is employed to analyze the potential relationship between the vehicle’s dynamic parameters and to extract the features of car-following behavior to generate the feature vector. The LSTM network is adopted to save the feature vector and predict the speed of the following vehicle. Finally, the CNN-LSTM model is trained and tested with the extracted car-following trajectories data and compared with the classical car-following models (LSTM model, intelligent driver model). The results show that the accuracy and the ability to learn the heterogeneity of the proposed model are better than the other two. Furthermore, the CNN-LSTM model can accurately reproduce the hysteresis phenomenon of congested traffic flow and apply to heterogeneous traffic flow mixed with adaptive cruise control vehicles on the freeway, which indicates that it has strong generalization ability.

This paper aims to predict the occurrence of pests and diseases for cotton based on long short term memory (LSTM) network. First, the problem of occurrence of pests and diseases was formulated as time series prediction. Then LSTM was adopted to solve the problem. LSTM is a special kind of recurrent neutral network (RNN), which introduces gate mechanism to prevent the vanished or exploding gradient problem. It has been shown good performance in solving time series problem and can handle the long-term dependency problem, as mentioned in many literatures. The experimental results showed that LSTM performed good on the prediction of occurrence of pests and diseases in cotton fields, and yielded an Area Under the Curve (AUC) of 0.97. The paper further verified that the weather factors indeed have strong impact on the occurrence of pests and diseases, and the LSTM network has great advantage on solving the long-term dependency problem.

The study explores the field of weather prediction with attention to its real‑world use and the quality that can be enhanced with the help of the most advanced methods of machine learning. It particularly studies the application of long short‑ term memory (LSTM) networks to improve forecasting. The paper relies on the data collected in Delhi, India, to train and evaluate the LSTM model. The paper identifies critical gaps in weather forecasting studies and investigates the effects of the gaps on business and lives. Accurate weather prediction can be used in such industries as agriculture, transport and disaster management, where a small development may have tremendous consequences. The paper provides a clear overview of LSTM model, the architecture, validation policy and evaluation. LSTM networks are the most appropriate networks to make the weather forecasts given that they are capable of repeating the patterns using the time series data. The paper will reveal that LSTM networks are efficient to improve the accuracy of weather prediction as well as how they may revolutionize industries that are reliant on the accuracy of predictions. This model was trained in this study based on an effective evaluation of MSE 0.034.

Electrocardiography (ECG) is a technique for observing and recording the electrical activity of the human heart. The usage of an ECG signal is common among clinical professionals in the collection of time data for the examination of any rhythmic conditions associated with a subject. The investigation was carried out in order to computerize the assignment by exhibiting the issue using encoder-decoder techniques, creating the information that was simply typical of it, and utilising misfortune appropriation to anticipate standard or anomalous information. On a broad variety of applications such as voice recognition and prediction, the long short-term memory (LSTM) fully connected layer (FCL) and the two convolutional neural networks (CNNs) have shown superior performance over deep learning networks (DLNs). DNNs are suitable for making high points for a more divisible region and CNNs are suitable for reducing recurrence types, LSTMs are appropriate for temporary displays, in the same way as CNNs are appropriate for reducing recurrence types. The CNN, LSTM, and DNN algorithms are acceptable for viewing. The complementarity of DNNs, CNNs, and LSTMs was investigated in this research by bringing them all together under the single architectural company. The researchers got the ECG data from the MIT-BIH arrhythmia database as a result of the investigation. Our results demonstrate that the approach proposed may expressively describe ECG series and identify abnormalities via scores that outperform existing supervised and unsupervised methods in both the short term and long term. The LSTM network and FCL additionally demonstrated that the unbalanced datasets associated with the ECG beat detection problem could be consistently resolved and that they were not susceptible to the accuracy of ECG signals. It is recommended that cardiologists employ the unique technique to aid them in performing reliable and impartial interpretation of ECG data in telemedicine settings.

This project focuses on improving the detection of network attacks by using a machine learning technique known as Long Short-Term Memory (LSTM) networks. LSTM networks are a type of neural network that excels at analyzing sequences of data, making them well-suited for identifying patterns associated with network intrusions. To enhance the LSTM model's effectiveness, we introduce new probability features that help the model better distinguish between normal and malicious activities. Our approach includes collecting network data, preprocessing it to make it suitable for training, and then using this data to train the LSTM model. We evaluate the model's performance using a range of metrics to ensure its accuracy and reliability. The results indicate that our method significantly improves the detection rate of network attacks while also reducing the number of false alarms. This means that our LSTM-based model not only catches more real threats but also makes fewer mistakes in identifying normal activities as attacks. Overall, this project showcases the potential of advanced machine learning techniques, like LSTM networks, to enhance cyber security measures and protect against network threats more effectively.

Time-history responses of the bridge induced by the moving vehicle provide crucial information for bridge design, operation, maintenance, etc. As inspired by this, this work attempts to provide a new paradigm for vehicle–bridge interaction (VBI) by highlighting the comparison of different deep learning algorithms applied to the prediction of time-history responses of the bridge under vehicular loads. Particularly, three deep learning architectures with few and measurable input features developed by using fully-connected feedforward neural network, long short-term memory (LSTM) network, and convolutional neural network (CNN) are proposed on the basis of the governing equation of bridge vibrations. Three VBI systems with various vehicle models are developed and further validated to produce reliable training data. To examine the accuracy of the predictive models, two advanced metrics are exploited for time-history estimate. Moreover, the proposed deep learning models are comprehensively investigated through a parametric study on the influential factors associated with the VBI system and network architecture. The results show that deep feedforward neural network (DFNN), LSTM network, and CNN can be applied in VBI analysis to estimate the bridge time-history response. The three neural networks have comparable prediction accuracies. When considering the irregularity excitation, CNN is found to be the most efficient predictive model, while DFNN needs the least training time under perfect bridge surface condition.

Accurate and timely short-term traffic flow forecasting plays a key role in intelligent transportation systems, especially for prospective traffic control. For the past decade, a series of methods have been developed for short-term traffic flow forecasting. However, due to the intrinsic stochastic and evolutionary trend, accurate forecasting remains challenging. In this paper, we propose a noise-immune long short-term memory (NiLSTM) network for short-term traffic flow forecasting, which embeds a noise-immune loss function deduced by maximum correntropy into the long short-term memory (LSTM) network. Different from the conventional LSTM network equipped with the mean square error loss, the maximum correntropy induced loss is a local similar metric, which is immunized to non-Gaussian noises. Extensive experiments on four benchmark datasets demonstrate the superior performance of our NiLSTM network by comparing it with the frequently used models and state-of-the-art models.