LSTM Based Adaptive Filtering for Reduced Prediction Errors of Hyperspectral Images

In order to improve the detection and recognition ability of multiscale hyperspectral images with weak edges, a lossless compression algorithm of multiscale hyperspectral images with weak edges based on improved deep learning is proposed. Using deep-level feature information extraction method to extract fine-grained features of cancer cells from multiscale hyperspectral images with weak edges, constructing 3D surface structure reconstruction model of multiscale hyperspectral images with weak edges, combining with variational Bayesian filter detection method to decompose multiscale color structure features of multiscale hyperspectral images with weak edges, and constructing gradually refined output structure model of multiscale hyperspectral images with weak edges, Fuzzy image coarse-scale neural network learning method is used to realize block matching and edge detection of multi-scale hyperspectral images with weak edges. Ultra-fine-grained features of multi-scale hyperspectral images with weak edges are extracted. Deep learning algorithm is used to adaptively optimize the lossless compression process of multi-scale hyperspectral images with weak edges, and the optimal design of lossless compression algorithm for multi-scale hyperspectral images with weak edges is realized. The simulation results show that the adaptive convergence of lossless compression of multi-scale weak edge hyperspectral images with this method is good, the feature resolution is strong, and the hyperspectral feature detection and recognition ability is good.

Utilization of the Long Short-Term Memory network for predicting streamflow in ungauged basins in Korea

With the rapid increase in bridge spans, the mitigation of risk to flutter (aeroelastic instability) is of critical importance in the design of long-span bridges, especially considering the more frequent intense hurricanes under climate change. Although the strong nonlinearities of the aeroelastic (self-excited) forces in wind–bridge interactions can be well captured through either numerical simulations or experimental tests, both are expensive and time consuming. Hence, it is important to develop an efficient reduced-order model for the simulations of nonlinear aeroelastic forces on the bridge decks. This study proposes a reduced-order model based on the long short-term memory (LSTM) network to simulate the nonlinear aeroelastic forces on bridge decks with various leading edges, and thus rapidly predict the corresponding post-flutter behaviors of long-span bridges. To generate the training datasets, computational fluid dynamics (CFD) was employed to simulate the nonlinear aeroelasticities of bridge decks with a wide range of leading-edge configurations and wind speeds. Trained on the high-fidelity CFD datasets, the LSTM network takes the motion of a bridge deck, leading-edge angles and wind speeds as inputs and outputs the nonlinear aeroelastic forces on the bridge decks. A hybrid loss function utilizing the prediction errors of both aeroelastic forces simulated by the LSTM network and the bridge deck responses calculated by the Newmark-β algorithm was introduced into the training process to improve the network performance. The prediction results of the trained LSTM model were compared with the CFD simulations, which demonstrated that the nonlinear aeroelastic forces of the bridge deck with various leading edges can be accurately and efficiently acquired by the proposed LSTM model.

<p dir="ltr">Recurrent Neural Networks (RNNs) are a major class of Artificial Neural Networks (ANNs). Suitable network architecture is vital for RNNs to achieve high performance. However, designing the architecture of deep RNNs (DRNNs) is a complicated and time-consuming process. Each layer of a DRNN contains numerous hyper-parameters. It is challenging to optimize the hyper-parameters of all layers jointly to achieve the best possible learning performance. Trial-and-error methods for designing DRNNs have been proven to be laborious and highly costly in practice. Therefore, an efficient automatic architecture search technique is needed to design DRNNs.</p><p dir="ltr">Genetic Algorithm (GA) is a popular evolutionary computation based approach. Existing research works already explored the use of GAs in designing ANNs. But, most of the existing approaches focus on the design of fixed-depth ANNs. However, it is not easy to fix the depth of DRNNs in advance, since different problems need DRNNs of different depths. Hence, the potential of GA has to be further explored to evolve the architecture of DRNNs of varying depths.</p><p dir="ltr">The primary goal of this thesis is to develop advanced GA approaches for designing Long Short Term Memory (LSTM) based DRNN architectures of varying depths effectively and efficiently.</p><p dir="ltr">Firstly, this thesis proposes a GA-based algorithm called Two-Stage Surrogate-Assisted GA (TS-SA-GA) with a progressive incremental strategy to design the architecture of LSTM networks of varying depths. The progressive approach can effectively extend well-designed shallow networks to high performing deep networks. Moreover, the new GA-based algorithm adopts newly designed knowledge-driven crossover and mutation operators to identify and repair LSTM network designs affected by inappropriate use of activation functions, thereby significantly reducing the chances for GA to evolve hard-to-train LSTM network architectures. Furthermore, this thesis proposes a two-stage surrogate method to predict the trainability and fitness of LSTM networks. This improves the efficiency and effectiveness of the GA-based algorithm to evolve LSTM networks.</p><p dir="ltr">Secondly, this thesis proposes a new algorithm called Evolutionary Design of LSTM Ensembles (ED-LSTM-Ensemble) based on multi-objective optimization techniques to evolve LSTM networks and ensembles simultaneously to directly utilize the ensemble performance to drive the evolution of base LSTM networks. Additionally, this thesis proposes a connection weight inheritance strategy to evolve LSTM networks and ensembles efficiently and effectively.</p><p dir="ltr">Thirdly, this thesis proposes a new algorithm called Skip Connections through Evolutionary and Differential Architecture search (SCEDA) to design LSTM networks together with appropriate skip connections automatically. Designing LSTM networks together with skip connections is highly challenging. This thesis hybridize gradient-based DAS and evolutionary architecture search to optimize the architecture of LSTM networks with suitable skip connections in a single evolutionary process to improve the efficiency and effectiveness of the search process.</p><p dir="ltr">Finally, this thesis conducts an in-depth empirical analysis of the impact of activation functions on the trainability of LSTM networks based on the concept of Edge Of Chaos. The analysis reveals the strong interrelation between the activation functions across multiple layers of an LSTM network and the trainability of the network. The analysis also highlights the importance of controlling activation functions to avoid generating untrainable LSTM networks. On the basis of the analysis, this thesis proposes a machine learning model to guide the use of activation functions in LSTM networks.</p>

Modern decision-making in fixed income asset management benefits from intelligent systems, which involve the use of state-of-the-art machine learning models and appropriate methodologies. We conduct the first study of bond yield forecasting using long short-term memory (LSTM) networks, validating its potential and identifying its memory advantage. Specifically, we model the 10-year bond yield using univariate LSTMs with three input sequences and five forecasting horizons. We compare those with multilayer perceptrons (MLP), univariate and with the most relevant features. To demystify the notion of black box associated with LSTMs, we conduct the first internal study of the model. To this end, we calculate the LSTM signals through time, at selected locations in the memory cell, using sequence-to-sequence architectures, uni and multivariate. We then proceed to explain the states’ signals using exogenous information, for what we develop the LSTM-LagLasso methodology. The results show that the univariate LSTM model with additional memory is capable of achieving similar results as the multivariate MLP using macroeconomic and market information. Furthermore, shorter forecasting horizons require smaller input sequences and vice-versa. The most remarkable property found consistently in the LSTM signals, is the activation/deactivation of units through time, and the specialisation of units by yield range or feature. Those signals are complex but can be explained by exogenous variables. Additionally, some of the relevant features identified via LSTM-LagLasso are not commonly used in forecasting models. In conclusion, our work validates the potential of LSTMs and methodologies for bonds, providing additional tools for financial practitioners.

This paper aims to predict the occurrence of pests and diseases for cotton based on long short term memory (LSTM) network. First, the problem of occurrence of pests and diseases was formulated as time series prediction. Then LSTM was adopted to solve the problem. LSTM is a special kind of recurrent neutral network (RNN), which introduces gate mechanism to prevent the vanished or exploding gradient problem. It has been shown good performance in solving time series problem and can handle the long-term dependency problem, as mentioned in many literatures. The experimental results showed that LSTM performed good on the prediction of occurrence of pests and diseases in cotton fields, and yielded an Area Under the Curve (AUC) of 0.97. The paper further verified that the weather factors indeed have strong impact on the occurrence of pests and diseases, and the LSTM network has great advantage on solving the long-term dependency problem.

The application of deep learning in stock price prediction has actually obtained boosting importance because of its capacity to capture complex patterns in financial information. This study uses Long Short-Term Memory (LSTM) networks to anticipate stock costs utilizing both day-to-day trading information and high-frequency minute-level information. For the day-to-day trading data, 42 stock datasets were analyzed, with 37 utilized for training and 5 booked for testing. High-frequency data, characterized by minute-by-minute changes, called for the development of specific designs for each stock. Each design incorporated differing configurations of LSTM devices and technical indicators to enhance predictive accuracy. The experimental outcomes demonstrated that the inclusion of technical indicators led to a reduction in prediction error, as measured by Root Mean Squared Error (RMSE). This highlights the capacity of LSTM networks, especially when boosted with technical indicators, for improving stock price forecasting accuracy throughout various timeframes.

To address the issues of slow convergence speed, poor dynamic adaptability, and path redundancy in the Double Deep Q Network (DDQN) within complex obstacle environments, this paper proposes an enhanced algorithm within the deep reinforcement learning framework. This algorithm, termed LPDDQN, integrates Prioritized Experience Replay (PER) and the Long Short Term Memory (LSTM) network to improve upon the DDQN algorithm. First, Prioritized Experience Replay (PER) is utilized to prioritize experience data and optimize storage and sampling operations through the SumTree structure, rather than the conventional experience queue. Second, the LSTM network is introduced to enhance the dynamic adaptability of the DDQN algorithm. Owing to the introduction of the LSTM model, the experience samples must be sliced and populated. The performance of the proposed LPDDQN algorithm is compared with five other path planning algorithms in both static and dynamic environments. Simulation analysis shows that in a static environment, LPDDQN demonstrates significant improvements over traditional DDQN in terms of convergence, number of moving steps, success rate, and number of turns, with respective improvements of 24.07%, 17.49%, 37.73%, and 61.54%. In dynamic and complex environments, the success rates of all algorithms, except TLD3 and the LPDDQN, decreased significantly. Further analysis reveals that the LPDDQN outperforms the TLD3 by 18.87%, 2.41%, and 39.02% in terms of moving steps, success rate, and number of turns, respectively.

This study utilizes Long Short-Term Memory (LSTM) networks to predict stock prices in the Nepal Stock Exchange (NEPSE), aiming to support investment decision-making. LSTM’s capability to model= temporal dependencies enables it to outperform traditional models in forecasting accuracy. The research uses historical data from insurance companies listed in NEPSE between 2020 and 2024. Evaluation metrics including MAE, MSE, and RMSE demonstrate the effectiveness of the LSTM model. The results are valuable for investors seeking to minimize risk and optimize returns through data-driven strategies. This research employs a Long Short-Term Memory (LSTM) model to predict stock prices in Nepal Stock Exchange (NEPSE). It aims to evaluate accuracy and effectiveness of LSTM in stock price forecasting. LSTM networks, known for their ability to capture complex temporal dependencies, are applied to analyze historical stock price trends. The findings suggest that LSTM models can significantly improve prediction accuracy and offer valuable insights for investors and financial analysts. This paper outlines the methodologies used, presents detailed results, and discusses the implications for investment decision-making.

This project focuses on improving the detection of network attacks by using a machine learning technique known as Long Short-Term Memory (LSTM) networks. LSTM networks are a type of neural network that excels at analyzing sequences of data, making them well-suited for identifying patterns associated with network intrusions. To enhance the LSTM model's effectiveness, we introduce new probability features that help the model better distinguish between normal and malicious activities. Our approach includes collecting network data, preprocessing it to make it suitable for training, and then using this data to train the LSTM model. We evaluate the model's performance using a range of metrics to ensure its accuracy and reliability. The results indicate that our method significantly improves the detection rate of network attacks while also reducing the number of false alarms. This means that our LSTM-based model not only catches more real threats but also makes fewer mistakes in identifying normal activities as attacks. Overall, this project showcases the potential of advanced machine learning techniques, like LSTM networks, to enhance cyber security measures and protect against network threats more effectively.

Phishing is an evolving form of cybercrime that targets users' sensitive information through URL manipulation. Conventional detection methods such as blacklists and signature-based approaches have become increasingly inadequate in addressing the dynamic variations of modern phishing attacks. This study evaluates the effectiveness of Recurrent Neural Network (RNN) variants, such Long Short-Term Memory (LSTM), Bidirectional LSTM (Bi-LSTM), and Gated Recurrent Unit (GRU), in detecting phishing threats based on URL data. The methodology involves a Systematic Literature Review (SLR) of scholarly publications from the past ten years, complemented by experimental implementation of the models using a public dataset from Kaggle. Literature findings show that Bi-LSTM consistently achieves the highest accuracy, up to 99%, while GRU stands out for its computational efficiency. Experimental results support these findings, with Bi-LSTM achieving an accuracy of 96.22%, GRU 96.29%, and LSTM 95.43%. Classification metrics indicate that RNN-based models perform very well in detecting benign and defacement URLs, although their performance on phishing URLs remains challenged, particularly in terms of recall. These results confirm that RNNs remain a promising approach for phishing detection systems, especially when integrated into hybrid models with complementary architectures. This study is expected to provide a foundation for developing precise and adaptive AI systems to combat increasingly sophisticated phishing threats.

The ever-increasing importance of accelerated information processing, communica-tion, and storing are major requirements within the big-data era revolution. With the extensive rise in data availability, handy information acquisition, and growing data rate, a critical challenge emerges in efficient handling. Even with advanced technical hardware developments and multiple Graphics Processing Units (GPUs) availability, this demand is still highly promoted to utilise these technologies effectively. Health-care systems are one of the domains yielding explosive data growth. Especially when considering their modern scanners abilities, which annually produce higher-resolution and more densely sampled medical images, with increasing requirements for massive storage capacity. The bottleneck in data transmission and storage would essentially be handled with an effective compression method. Since medical information is critical and imposes an influential role in diagnosis accuracy, it is strongly encouraged to guarantee exact reconstruction with no loss in quality, which is the main objective of any lossless compression algorithm. Given the revolutionary impact of Deep Learning (DL) methods in solving many tasks while achieving the state of the art results, includ-ing data compression, this opens tremendous opportunities for contributions. While considerable efforts have been made to address lossy performance using learning-based approaches, less attention was paid to address lossless compression. This PhD thesis investigates and proposes novel learning-based approaches for compressing 3D medical images losslessly.Firstly, we formulate the lossless compression task as a supervised sequential prediction problem, whereby a model learns a projection function to predict a target voxel given sequence of samples from its spatially surrounding voxels. Using such 3D local sampling information efficiently exploits spatial similarities and redundancies in a volumetric medical context by utilising such a prediction paradigm. The proposed NN-based data predictor is trained to minimise the differences with the original data values while the residual errors are encoded using arithmetic coding to allow lossless reconstruction.Following this, we explore the effectiveness of Recurrent Neural Networks (RNNs) as a 3D predictor for learning the mapping function from the spatial medical domain (16 bit-depths). We analyse Long Short-Term Memory (LSTM) models’ generalisabil-ity and robustness in capturing the 3D spatial dependencies of a voxel’s neighbourhood while utilising samples taken from various scanning settings. We evaluate our proposed MedZip models in compressing unseen Computerized Tomography (CT) and Magnetic Resonance Imaging (MRI) modalities losslessly, compared to other state-of-the-art lossless compression standards.This work investigates input configurations and sampling schemes for a many-to-one sequence prediction model, specifically for compressing 3D medical images (16 bit-depths) losslessly. The main objective is to determine the optimal practice for enabling the proposed LSTM model to achieve a high compression ratio and fast encoding-decoding performance. A solution for a non-deterministic environments problem was also proposed, allowing models to run in parallel form without much compression performance drop. Compared to well-known lossless codecs, experimental evaluations were carried out on datasets acquired by different hospitals, representing different body segments, and have distinct scanning modalities (i.e. CT and MRI).To conclude, we present a novel data-driven sampling scheme utilising weighted gradient scores for training LSTM prediction-based models. The objective is to determine whether some training samples are significantly more informative than others, specifically in medical domains where samples are available on a scale of billions. The effectiveness of models trained on the presented importance sampling scheme was evaluated compared to alternative strategies such as uniform, Gaussian, and sliced-based sampling.

Accurate and timely short-term traffic flow forecasting plays a key role in intelligent transportation systems, especially for prospective traffic control. For the past decade, a series of methods have been developed for short-term traffic flow forecasting. However, due to the intrinsic stochastic and evolutionary trend, accurate forecasting remains challenging. In this paper, we propose a noise-immune long short-term memory (NiLSTM) network for short-term traffic flow forecasting, which embeds a noise-immune loss function deduced by maximum correntropy into the long short-term memory (LSTM) network. Different from the conventional LSTM network equipped with the mean square error loss, the maximum correntropy induced loss is a local similar metric, which is immunized to non-Gaussian noises. Extensive experiments on four benchmark datasets demonstrate the superior performance of our NiLSTM network by comparing it with the frequently used models and state-of-the-art models.

Adaptive Cruise Control (ACC) is an important part of automotive autonomy. To have a robust Adaptive Cruise Control system, a well-defined sensor fusion system is required. This paper focuses on feature-level sensor fusion system where three fusion techniques are explored: independent Kalman filtering, interacting multiple model (IMM) filtering, and a novel hybrid technique using a combination of both an IMM Filter and deep neural network (DNN). Kalman Filtering is a popular approach for automotive sensor fusion, but heavily relies on the given kinematic models to make state estimations. We explore the IMM filter to capture multi-model motion as well as a DNN approach to build a data-driven model of the system. Our IMM filter is built with several Kalman Filters featuring different kinematic models. The proposed DNN used is a Long Short-Term Memory (LSTM) network trained on radar and camera data to forecast a vehicle's state. To overcome the weaknesses in both IMM filtering and LSTM networks, we propose a hybrid technique of consisting of both the IMM filter and the LSTM network. The proposed hybrid system uses a single filter for each sensor. The filtered sensor data is synchronized and used as the input for a trained LSTM network. Our LSTM network was trained on over 100 simulated highway driving scenarios that might occur in an ACC application. Simulations and code were developed using MATLAB and the Autonomous Driving Toolbox. Our hybrid IMM-LSTM system outperformed the independent Kalman Filtering approach in longitudinal and lateral tracking accuracy (RMSE) by 23 percent and had promising results compared to the IMM filter where the tracking error was decreased by nearly 50 percent in certain driving scenarios.

Flash floods occur frequently and distribute widely in mountainous areas because of complex geographic and geomorphic conditions and various climate types. Effective flash flood forecasting with useful lead times remains a challenge due to its high burstiness and short response time. Recently, machine learning has led to substantial changes across many areas of study. In hydrology, the advent of novel machine learning methods has started to encourage novel applications or substantially improve old ones. This study aims to establish a discharge forecasting model based on Long Short-Term Memory (LSTM) networks for flash flood forecasting in mountainous catchments. The proposed LSTM flood forecasting (LSTM-FF) model is composed of T multivariate single-step LSTM networks and takes spatial and temporal dynamics information of observed and forecast rainfall and early discharge as inputs. The case study in Anhe revealed that the proposed models can effectively predict flash floods, especially the qualified rates (the ratio of the number of qualified events to the total number of flood events) of large flood events are above 94.7% at 1–5 h lead time and range from 84.2% to 89.5% at 6–10 h lead-time. For the large flood simulation, the small flood events can help the LSTM-FF model to explore a better rainfall-runoff relationship. The impact analysis of weights in the LSTM network structures shows that the discharge input plays a more obvious role in the 1-h LSTM network and the effect decreases with the lead-time. Meanwhile, in the adjacent lead-time, the LSTM networks explored a similar relationship between input and output. The study provides a new approach for flash flood forecasting and the highly accurate forecast contributes to prepare for and mitigate disasters.