Missing data filling based on the spectral analysis and the Long Short- Term Memory network

Modern decision-making in fixed income asset management benefits from intelligent systems, which involve the use of state-of-the-art machine learning models and appropriate methodologies. We conduct the first study of bond yield forecasting using long short-term memory (LSTM) networks, validating its potential and identifying its memory advantage. Specifically, we model the 10-year bond yield using univariate LSTMs with three input sequences and five forecasting horizons. We compare those with multilayer perceptrons (MLP), univariate and with the most relevant features. To demystify the notion of black box associated with LSTMs, we conduct the first internal study of the model. To this end, we calculate the LSTM signals through time, at selected locations in the memory cell, using sequence-to-sequence architectures, uni and multivariate. We then proceed to explain the states’ signals using exogenous information, for what we develop the LSTM-LagLasso methodology. The results show that the univariate LSTM model with additional memory is capable of achieving similar results as the multivariate MLP using macroeconomic and market information. Furthermore, shorter forecasting horizons require smaller input sequences and vice-versa. The most remarkable property found consistently in the LSTM signals, is the activation/deactivation of units through time, and the specialisation of units by yield range or feature. Those signals are complex but can be explained by exogenous variables. Additionally, some of the relevant features identified via LSTM-LagLasso are not commonly used in forecasting models. In conclusion, our work validates the potential of LSTMs and methodologies for bonds, providing additional tools for financial practitioners.

This paper aims to predict the occurrence of pests and diseases for cotton based on long short term memory (LSTM) network. First, the problem of occurrence of pests and diseases was formulated as time series prediction. Then LSTM was adopted to solve the problem. LSTM is a special kind of recurrent neutral network (RNN), which introduces gate mechanism to prevent the vanished or exploding gradient problem. It has been shown good performance in solving time series problem and can handle the long-term dependency problem, as mentioned in many literatures. The experimental results showed that LSTM performed good on the prediction of occurrence of pests and diseases in cotton fields, and yielded an Area Under the Curve (AUC) of 0.97. The paper further verified that the weather factors indeed have strong impact on the occurrence of pests and diseases, and the LSTM network has great advantage on solving the long-term dependency problem.

Flash floods occur frequently and distribute widely in mountainous areas because of complex geographic and geomorphic conditions and various climate types. Effective flash flood forecasting with useful lead times remains a challenge due to its high burstiness and short response time. Recently, machine learning has led to substantial changes across many areas of study. In hydrology, the advent of novel machine learning methods has started to encourage novel applications or substantially improve old ones. This study aims to establish a discharge forecasting model based on Long Short-Term Memory (LSTM) networks for flash flood forecasting in mountainous catchments. The proposed LSTM flood forecasting (LSTM-FF) model is composed of T multivariate single-step LSTM networks and takes spatial and temporal dynamics information of observed and forecast rainfall and early discharge as inputs. The case study in Anhe revealed that the proposed models can effectively predict flash floods, especially the qualified rates (the ratio of the number of qualified events to the total number of flood events) of large flood events are above 94.7% at 1–5 h lead time and range from 84.2% to 89.5% at 6–10 h lead-time. For the large flood simulation, the small flood events can help the LSTM-FF model to explore a better rainfall-runoff relationship. The impact analysis of weights in the LSTM network structures shows that the discharge input plays a more obvious role in the 1-h LSTM network and the effect decreases with the lead-time. Meanwhile, in the adjacent lead-time, the LSTM networks explored a similar relationship between input and output. The study provides a new approach for flash flood forecasting and the highly accurate forecast contributes to prepare for and mitigate disasters.

As a kind of continuous time series, plantar pressure data contains rich contact of time information which has not been fully utilized in existing gait recognition methods. In this paper, we proposed a new gait recognition method based on plantar pressure data with a Long Short-Term Memory (LSTM) network. By normalization and dimensionality reduction, the raw pressure data was converted to feature tensor. Then we feed the LSTM network with the feature tensors and implement classification recognition. We collected data from 93 subjects of different age groups, and each subjects was collected 10 sets of pressure data. The experiment results turn out that our LSTM network can get high classification accuracy and performs better than CNN model and many traditional methods.

Seismic response prediction of RC bridge piers through stacked long short-term memory network

The developed research includes an Long Short-Term Memory (LSTM) network with IoT sensor data for the management of rice crop irrigation. Optimizing the amount of water required for rice crops is very critical, based on real-time predictions of the actual environmental and soil parameters including moisture, temperature, humidity, and rainfall as well as all other possible related factors. IoT sensors installed in agricultural fields gathered data, which was transmitted to a controller and then uploaded to the cloud for further processing. The training dataset contained 5,600 records. In this research, 70% was used for training and the rest 30% for testing. Three LSTM models were used for analyzing sensor data and predicting the need for irrigation and controlling the irrigation pump accordingly. These were Vanilla LSTM, Stacked LSTM, and Bidirectional LSTM. It was evaluated the performance of the models developed using precision, recall, F1 score, accuracy, and other performance measures such as Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Score. Bidirectional LSTM was determined to have the best performance, with MAE at 0.15, RMSE at 0.24, and an R2 score of 0.93, exhibiting high accuracy and predictive power. The Stacked LSTM and Vanilla LSTM also gave fair results, in terms of having R2 values of 0.89 and 0.84, respectively. The number of training and memory used for the computation were also given, with Vanilla LSTM taking least, followed by Stacked LSTM and Bidirectional LSTM. These results suggest that the LSTM networks can potentially be capable of improving the irrigation management such that more water is conserved and, subsequently, more suitable for agricultural productivity.

With the rapid increase in bridge spans, the mitigation of risk to flutter (aeroelastic instability) is of critical importance in the design of long-span bridges, especially considering the more frequent intense hurricanes under climate change. Although the strong nonlinearities of the aeroelastic (self-excited) forces in wind–bridge interactions can be well captured through either numerical simulations or experimental tests, both are expensive and time consuming. Hence, it is important to develop an efficient reduced-order model for the simulations of nonlinear aeroelastic forces on the bridge decks. This study proposes a reduced-order model based on the long short-term memory (LSTM) network to simulate the nonlinear aeroelastic forces on bridge decks with various leading edges, and thus rapidly predict the corresponding post-flutter behaviors of long-span bridges. To generate the training datasets, computational fluid dynamics (CFD) was employed to simulate the nonlinear aeroelasticities of bridge decks with a wide range of leading-edge configurations and wind speeds. Trained on the high-fidelity CFD datasets, the LSTM network takes the motion of a bridge deck, leading-edge angles and wind speeds as inputs and outputs the nonlinear aeroelastic forces on the bridge decks. A hybrid loss function utilizing the prediction errors of both aeroelastic forces simulated by the LSTM network and the bridge deck responses calculated by the Newmark-β algorithm was introduced into the training process to improve the network performance. The prediction results of the trained LSTM model were compared with the CFD simulations, which demonstrated that the nonlinear aeroelastic forces of the bridge deck with various leading edges can be accurately and efficiently acquired by the proposed LSTM model.

Utilization of the Long Short-Term Memory network for predicting streamflow in ungauged basins in Korea

In recent years, with the advance of Artificial Intelligence, automatic music composition has been demonstrated. However, there are many music genres and music instruments. For a same piece of music, different music instruments would produce different effects. Invented some 2500 years ago, Guzheng is one of the oldest music instruments in China and the world. It has distinct timbres and patterns that cannot be duplicated by other music instruments. Therefore, it is interesting to see whether AI can compose Guzheng music or alike. In this paper we present a method that can automatically compose and play Guzheng music. First, we collect a number of existing Guzheng music pieces and convert them into Music Instrument Digital Interface format. Second, we use these data to train a Long Short-Term Memory (LSTM) network and use the trained network to generate new Guzheng music pieces. Next, we use the Reinforcement Learning to optimize the LSTM network by adding special Guzheng playing techniques. Comparing to the existing AI methods, such as LSTM and Generative Adversary Network, our new method is more effective in capturing the characteristics of Guzheng music. According to the evaluations from skilled Guzheng players and general audiences, our Guzheng music is very close to the real Guzheng music. The presented method can also be used to automatically compose the music of other Chinese music instruments.

Accurate and timely short-term traffic flow forecasting plays a key role in intelligent transportation systems, especially for prospective traffic control. For the past decade, a series of methods have been developed for short-term traffic flow forecasting. However, due to the intrinsic stochastic and evolutionary trend, accurate forecasting remains challenging. In this paper, we propose a noise-immune long short-term memory (NiLSTM) network for short-term traffic flow forecasting, which embeds a noise-immune loss function deduced by maximum correntropy into the long short-term memory (LSTM) network. Different from the conventional LSTM network equipped with the mean square error loss, the maximum correntropy induced loss is a local similar metric, which is immunized to non-Gaussian noises. Extensive experiments on four benchmark datasets demonstrate the superior performance of our NiLSTM network by comparing it with the frequently used models and state-of-the-art models.

Adaptive Cruise Control (ACC) is an important part of automotive autonomy. To have a robust Adaptive Cruise Control system, a well-defined sensor fusion system is required. This paper focuses on feature-level sensor fusion system where three fusion techniques are explored: independent Kalman filtering, interacting multiple model (IMM) filtering, and a novel hybrid technique using a combination of both an IMM Filter and deep neural network (DNN). Kalman Filtering is a popular approach for automotive sensor fusion, but heavily relies on the given kinematic models to make state estimations. We explore the IMM filter to capture multi-model motion as well as a DNN approach to build a data-driven model of the system. Our IMM filter is built with several Kalman Filters featuring different kinematic models. The proposed DNN used is a Long Short-Term Memory (LSTM) network trained on radar and camera data to forecast a vehicle's state. To overcome the weaknesses in both IMM filtering and LSTM networks, we propose a hybrid technique of consisting of both the IMM filter and the LSTM network. The proposed hybrid system uses a single filter for each sensor. The filtered sensor data is synchronized and used as the input for a trained LSTM network. Our LSTM network was trained on over 100 simulated highway driving scenarios that might occur in an ACC application. Simulations and code were developed using MATLAB and the Autonomous Driving Toolbox. Our hybrid IMM-LSTM system outperformed the independent Kalman Filtering approach in longitudinal and lateral tracking accuracy (RMSE) by 23 percent and had promising results compared to the IMM filter where the tracking error was decreased by nearly 50 percent in certain driving scenarios.

Hydrate plugging and pipeline leak can impair the normal operation of natural gas pipeline and may lead to serious accidents. Since natural gas pipeline safety monitoring based on active acoustic excitation can detect and locate not only the two abnormal events but also normal components such as valves and pipeline elbows, recognition and classification of these events are of great importance to provide maintenance guidance for the pipeline operators and avoid false alarm. In this article, long short-term memory (LSTM) network is introduced and applied to classify detection signals of hydrate plugging, pipeline leak, and elbow. Adaptive moment estimation (Adam) algorithm is introduced and utilized to accelerate the long short-term memory network convergence in training. Experimental results demonstrate that the network with three layers and 64 units per cell performs the best. The cross-entropy loss in training is 0.0005, and classification accuracies are all 100% in training, validation, and testing which verify the validity of the long short-term memory network. Therefore, the method based on the long short-term memory network and adaptive moment estimation algorithm can work efficiently on pipeline events classification and has great guiding significance for safety assurance of natural gas transmission.

Maglev is a developing transportation mode in the recent years, and its safety and stability are required to be ensured by structural health monitoring system. However, the data measured by some sensors may be partially or totally unavailable due to the external electromagnetic interference or sensor failure. While, in some cases, those missing data are important to identify and analyze damage of maglev bridges. Since the structural dynamic characteristics of maglev bridge are complicated, the data reconstruction of those sensors becomes challenging. Therefore, this study proposes a bidirectional long short-term memory (BiLSTM) network to accurately reconstruct the maglev bridge acceleration data. The design of long short-term memory helps to sufficiently exploit the time series data and learn the hidden features of maglev bridge acceleration data between sensors. The bidirectional architecture enables the network to simultaneously learn the time series data from past and future, which is beneficial to extract more hidden features. A dataset collected from an in-site experiment for maglev bridge is used to verify the feasibility of the proposed method. The lost acceleration data from abnormal sensors is predicted by the acceleration data recorded from normally operating sensors. The results shows that the difference between predicted and true acceleration data is at a very low magnitude. Consequently, the proposed method can be applied for the high-performance reconstruction of maglev bridge acceleration data.

Multi-fidelity surrogate modeling using long short-term memory networks

With the large-scale wind integration into power grid, the intermittent and stochastic nature of wind power possess a threat to the safety of power grid, and wind power prediction has become one of the most important solutions to the current grid connection problem. However, the prediction error in point prediction of wind power cannot be ignored, so it is necessary to make probability prediction of wind power and improve reliable information to the power grid dispatch department. In this paper, by combing with the wavelet decomposition technology and the long short-term memory (LSTM) network, an ultra short-term probability model is proposed. Firstly, the original time series are smoothed by wavelet decomposition technology, then the LSTM network prediction model of each sub-series sample is developed. The Gaussian distribution function of the prediction error is obtained by using the maximum likelihood estimation method, and finally the probability interval prediction of the future wind power in four hours could be realized. Finally, by using the data collected from a wind farm in northeast china, a numerical example is presented to illustrate the usefulness of the proposed method, which show that combining wavelet decomposition with deep learning method can improve the accuracy of prediction, improve the interval reliability of probability prediction, and enhance the generalization ability of the model.