Comparative Analysis of Recurrent Neural Network Models Performance in Predicting Bitcoin Prices

Advances in information technology have given rise to online hotel reservation options. The user review feature is an important factor during the online booking of hotels. Generally, most online hotel booking service providers provide review and rating features for assessing hotels. However, not all service providers provide rating features or recap reviews for every aspect of the hotel services offered. Therefore, we propose a method to summarise reviews based on multiple aspects, including food, room, service, and location. This method uses long short-term memory (LSTM), together with hidden layers and automation of the optimal number of hidden neurons. The F1-measure value of 75.28% for the best model was based on the fact that (i) the size of the first hidden layer is 1,200 neurons with the tanh activation function, and (ii) the size of the second hidden layer is 600 neurons with the ReLU activation function. The proposed model outperforms the baseline model (also known as standard LSTM) by 10.16%. It is anticipated that the model developed through this study can be accessed by users of online hotel booking services to acquire a review recap on more specific aspects of services offered by hotels

Artificial neural networks (ANN) have been used in many applications such like handwriting recognition and speech recognition. It is well-known that learning rate is a crucial value in the training procedure for artificial neural networks. It is shown that the initial value of learning rate can confoundedly affect the final result and this value is always set manually in practice. A new parameter called beta stabilizer has been introduced to reduce the sensitivity of the initial learning rate. But this method has only been proposed for deep neural network (DNN) with sigmoid activation function. In this paper we extended beta stabilizer to long short-term memory (LSTM) and investigated the effects of beta stabilizer parameters on different models, including LSTM and DNN with relu activation function. It is concluded that beta stabilizer parameters can reduce the sensitivity of learning rate with almost the same performance on DNN with relu activation function and LSTM. However, it is shown that the effects of beta stabilizer on DNN with relu activation function and LSTM are fewer than the effects on DNN with sigmoid activation function.

Extreme weather modeling is a challenge for modeling experts in Indonesia and the world. Extreme weather prediction is a complex problem because the chances of it happening are very small, so the developed models often have a low level of accuracy. The purpose of this research is to combine the classic model, Autoregressive Integrated Moving Average (ARIMA), recurrent neural network (RNN) model using Adam and SGD estimation (RNN-Adam and RNN-SGD) with the reLU, tanh, sigmoid and gaussian activation functions. In addition, the ARIMA-RNN mix model was also demonstrated in this study. These models are applied to monthly period extreme weather data obtained from the Meteorology, Climatology and Geophysics Agency (BMKG) of West Sulawesi Province which are converted into training data and test data. The RMSE value is used to see the level of prediction accuracy in both training data and test data. Based on the research results, the best model obtained for modeling Indonesia’s extreme weather is the ARIMA-RNN-Adam mix model with the reLU activation function based on the RMSE value on the training and test data. At n = 50, the smallest RMSE and MSE values of the third model are the ARIMA-RNN-Adam model which is 0.23212 using the reLU activation function, then the ARIMA-RNN-SGD model which is 0.25432 with the same activation function, while the ARIMA value is 0.3270. At n=100 it can be seen that the smallest RMSE and MSE values of the three models are the ARIMA-RNN-Adam model which is equal to 0.25149 using the reLU activation function, then the ARIMA-RNN-SGD model which is equal to 0.25256 with the same activation function, while the ARIMA value is 0.2644.

Stock market is a dynamic and volatile market that is considered as time series data. The growth of financial data exposed the computational efficiency of the conventional systems. This paper proposed a hybrid deep learning model based on Long Short- Term Memory (LSTM) and Artificial Bee Colony (ABC) algorithm. ABC is best fit for hyper parameter selection for deep LSTM models and maintains the equilibrium of exploitation and exploration issues. Handling a large volume of multidimensional reviews from social media is a major challenge. This paper evolves the multiple aspects of market sentiments and uses the reliable Big data platform Hadoop ecosystem and its services to compute sentiment polarity index. The ABC-LSTM hybrid model is validated with other core and hybrid models with evolutionary algorithms as Differential Evolution (DE) and Genetic Algorithm (GA). For the experiments, 10 years of historical datasets and social media reviews of IT sector funds Apple Inc. (AAPL), Microsoft corporation (MSFT) and Intel corporation (INTL) from NASDAQ GS, an American stock exchange are considered to validate hybrid forecasting models. Proposed algorithm ABC-LSTM is used to tune the hyperparameters (window size, LSTM units, dropout probability, epochs, batch size and learning rate) and evaluated through Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE) as loss function. Performance analysis proves that with sentiment polarity, ABC optimized LSTM obtains improved forecasting accuracy over its counterpart models.

Agriculture plays a vital role in the Indian economy. The growth of agriculture sector is based on the type of gift we have got from the nature. It varies state to state, district to district, taluka to taluka, block to block and even village to village. This study is confined to Bhimtal block of Nainital district. The main purpose of agriculture is growing crops and raising livestock. In order to grow the crops several types of agri-inputs are required, among them fertile lands have the great significance in crop cultivation. As far as fertile land is concerned it solely depends on the quality of the soil in terms of producing the nutrients for the crops. The available nutrients present in soil can be evaluated and measured by soil testing tools. The appropriate quantity of soil nutrients supplied to the soil can also be determined by this tool. The quantity of supplied nutrients is based on soil fertility and crop needs. In this study we have classified different soil features such as OC (Organic Carbon), P (Phosphorus), K (Potassium), Mn (Magnesium) and B (Boron). In order to make meaningful inferences and estimates, machine learning techniques especially ANN network with two activation functions relu and tanh are used in this study. For categorizations and predictions we have used village wise soil test report values. This kind of practice will not only help stakeholders to mitigate the expenditure of continuously supplying fertilizers to soil but it would also be cost effective, less time consuming and more profitable for stakeholders. In this regard data was complied, classified, tabulated, presented, analyzed and it can be seen that relu activation function has ensured higher accuracy over tanh activation function. It is expedient and necessary to mention here that out of the five classified soil nutrient parameters relu activation function has shown better performance in respect of four classified soil nutrient parameters while tanh gave better performance in only one classified soil nutrient parameter.

Time series analysis plays an important role in many fields. Autoregressive Integrated Moving Average (ARIMA) and Long Short-Term Memory (LSTM) methods are the most common tools to forecast sequential and time series data. However, both of them have obvious defects. The ARIMA model cannot capture non-linear information in sequences. Although the LSTM network is good at learning dynamic variations, it is prone to overfitting and requires a mass of long-term data. In this study, a hybrid technology blending the ARIMA and the LSTM algorithms was utilized to forecast the number of cases of influenza in China. This hybrid method leveraged the advantages of the ARIMA model and the LSTM network. Firstly, the ARIMA model was used to analyze the linear relationship within the time series. Then, the residuals of the ARIMA were taken as the input values to train the LSTM networks. A new hybrid ARIMA-LSTM model obtained, combining a SARIMA (0,1,0) (1,0,0)52 model and a LSTM model with 50 epochs and 32 batch size. This model successfully addressed the previously mentioned issues and enhanced the precision of the predictions. It managed to reduce 4.6% of RMSE, 8.9% of MSE, and 13.9% of MAE. In addition, this new algorithm was found that it didn’t have a high requirement like the individual LSTM model. Since there were not very many observations in the dataset, the performance of the individual LSTM model was not good. However, the integrated model improved this problem and obtained a more precise prediction. Even though the hybrid model had a better performance on prediction, it still has the risk of overfitting the data. The future work will be to improve the hybrid model to decrease this risk by adding more variables and modifying the structure of the LSTM model. Meanwhile, applying this method to another field further proves its feasibility and provides more effective prediction models.

The desire to provide robust guarantees on neural networks has never been more important, as their prevalence in society is increasing. One popular method that has seen a large amount of success is to use bounds on the activation functions within these networks to provide such guarantees. However, due to the large number of possible ways to bound the activation functions, there is a trade-off between conservativeness and complexity. We approach the problem from a different perspective, using polynomial optimisation and real algebraic geometry (the Positivstellensatz) to assert the emptiness of a semi-algebraic set. We show that by using the Positivstellen- satz, bounds on the robustness guarantees can be tightened significantly over other popular methods, at the expense of computational resource. We demonstrate the effectiveness of this approach on networks that use the ReLU, sigmoid and tanh activation functions. This method can be extended to more activation functions, and combined with recent sparsity- exploiting methods can result in a computationally acceptable method for verifying neural networks.

SummaryThe present existing weather prediction systems deployed by weather forecasting agencies predict the anomalous events using numerical simulation models developed based on geophysics and oceanographic theories. The major disadvantage with existing models is that are explicit simulation models. So, we try to build a real time prediction system that will automatically predict the intensity of any marine natural disaster events and alert the authorities to take necessary actions using a satellite and observational real time data set including moored met ocean data. We propose a novel hybrid model which is a combination of Convolutional Neural Network (CNN) and Long Short‐Term Memory (LSTM). The paper analyzes various other models such as Gated Recurrent Unit (GRU), Bidirectional LSTM (Bi‐LSTM) Recurrent Neural Network (RNN), and Extreme Learning Machine. In addition the other standard CNN architectures are also analyzed. Since the residual CNN network is the most effective architecture, the paper proposes a modification of the residual CNN network by combining the residual CNN network with LSTM/GRU RNN network along with a proposed new activation function which was experimentally proved to be more effective than the deep learning community standard activation function ReLU. The proposed new activation function overcomes the dead neuron propagation problem in ReLU activation function and hence the model's accuracy is increased by more than 1%. The proposed CNN + LSTM/GRU model is benchmarked with UCR time series benchmark dataset and it is experimentally seen that the model performs better than all existing algorithms.

Reading books is one of the most effective ways to reduce stress. In today's digital era, access to finding and buying books is getting easier, so reader reviews are important in choosing books that match interests. However, with a large number of reviews, Natural Language Processing (NLP) with the Long Short-Term Memory (LSTM) method is used to help analyze positive and negative sentiments from many book reviews. This sentiment analysis is useful for readers to evaluate the quality of books, as well as for authors and sellers to find out the opinions of readers and improve the quality of their work. In this study, the book review dataset "Vera Wong's Unsolicited Advice for Murderers" from the Goodreads website is used, which is then divided into training data and validation data with a ratio of 75%: 25%. The Long Short-Term Memory (LSTM) method is used to analyze the sentiment of the reviews. The model architecture built consists of Embedding Layer, LSTM Layer with 128 neuron units, 3 Dense Layer with ReLU activation function, 3 Dropout Layer, and Fully Connected Layer with and Sigmoid activation function, Binary Cross Entropy loss function, and RMSprop optimizer. The model training process was conducted with 30 epochs. The evaluation results show that the model achieved an accuracy of 90%, indicating the model performs relatively well in correctly classifying positive sentiments.

Air pollution prediction using LSTM deep learning and metaheuristics algorithms

Xuliujing section, Yangtze River Estuary, China. Predicting discharge in tidal rivers is challenging due to complex flow dynamics influenced by topography, tides, runoff, and weather. Traditional forecasting methods struggle with fixed parameters, limiting their adaptability and accuracy over time. To address this, we propose an enhanced deep learning model, A KAN-augmented LSTM framework, which integrates a Kolmogorov-Arnold network (KAN) with a long short-term memory (LSTM) network. This model retains LSTM's ability to handle long-term dependencies while replacing the fully connected layer with a KAN layer. A learnable B-spline activation function in the KAN layer improves the model's capacity to capture nonlinear dynamics and long-term dependencies in time series data, enhancing forecasting accuracy. This paper applies the LSTM-KAN model to the Xuliujing section of the Yangtze River Estuary and compares its performance with traditional harmonic analysis (HA) and four neural network models: LSTM, XGBoost, DLinear, and Informer. The results demonstrate that the LSTM-KAN model significantly enhances discharge forecasting accuracy, outperforming all comparative methods across short-term (6 h), medium-term (12–24 h), and long-term (36–48 h) forecasts. Specifically, it achieved relative accuracy improvements of 12.1 %–35.2 % over HA and 7 %–52.8 % over the traditional LSTM model. These findings suggest that the complex interplay of tidal forcing, runoff, and weather in the Yangtze Estuary is better represented by the adaptive, function-learning paradigm of KAN than by models with fixed nonlinearities. The model's superior performance offers new insights for studying complex flow dynamics, indicating that deep learning techniques with learnable activation functions provide a more powerful and accurate tool for operational forecasting in highly dynamic tidal river environments. • This study developed an LSTM-KAN model for more accurate discharge forecasting in tidal rivers. • KAN leverages adaptive edge-weight activation to capture tidal-hydrodynamic nonlinearities with fewer parameters. • This study first embeds the architecture in hydrologic forecasting, using the Yangtze Xuliujing reach as the pilot. • The proposed model significantly outperforms traditional approaches in streamflow-forecasting accuracy.

Landslide displacement prediction has garnered significant recognition as a pivotal component in realizing successful early warnings and implementing effective control measures. This task remains challenging as landslide deformation involves not only temporal dependency within time series data but also spatial dependence across various regions within landslides. The present study proposes a landslide spatiotemporal displacement forecasting model by introducing attention-based deep learning algorithms based on spatiotemporal analysis. The Maximal Information Coefficient (MIC) approach is employed to quantify the spatial and temporal correlations within the daily data of Global Navigation Satellite System (GNSS) observations. Based on the quantitative spatiotemporal analysis, the proposed prediction model combines a convolutional neural network (CNN) and long short-term memory (LSTM) network to capture spatial and temporal dependencies individually. Spatial–temporal attention mechanisms are implemented to optimize the model. Additionally, we develop a single-point prediction model using LSTM and a multiple-point prediction model using the CNN-LSTM without an attention mechanism to compare the forecasting capabilities of the attention-based CNN-LSTM model. The Outang landslide in the Three Gorges Reservoir Area (TGRA), characterized by a large and active landslide equipped with an advanced monitoring system, is taken as a studied case. The temporal MIC results shed light on the response times of monitored daily displacement to external factors, showing a lagging duration of between 10 and 50 days. The spatial MIC results indicate mutual influence among different locations within the landslide, particularly in the case of nearby sites experiencing significant deformation. The attention-based CNN-LSTM model demonstrates an impressive predictive performance across six monitoring stations within the Outang landslide area. Notably, it achieves a remarkable maximum coefficient of determination (R2) value of 0.9989, accompanied by minimum values for root mean squared error (RMSE), absolute mean error (MAE), and mean absolute percentage error (MAPE), specifically, 1.18 mm, 0.99 mm, and 0.33%, respectively. The proposed model excels in predicting displacements at all six monitoring points, whereas other models demonstrate strong performance at specific individual stations but lack consistent performance across all stations. This study, involving quantitative deformation characteristics analysis and spatiotemporal displacement prediction, holds promising potential for a more profound understanding of landslide evolution and a significant contribution to reducing landslide risk.

Accurate rainfall forecasting is crucial for effective urban planning and disaster management in Dhaka, the capital of Bangladesh, a city highly vulnerable to urban flooding and extreme weather events. Traditional forecasting methods often struggle to capture the region's complex rainfall patterns, resulting in inaccurate rainfall forecasts. This study evaluates the performance of two traditional machine learning algorithms, Random Forest Regression and Multi-layer Perceptron (MLP), alongside one deep learning algorithm, the Long Short-Term Memory (LSTM) network. These models are trained and tested to forecast rainfall over 1 to 5-day lead times, emphasizing their ability to handle temporal dependencies in time series data. Atmospheric and hydrologic variables, including temperature, surface pressure, evaporation, solar surface radiation, total column rainwater, large-scale precipitation, and total cloud cover, from the ECMWF (European Centre for Medium-Range Weather Forecasts) Reanalysis v5 (ERA5) dataset, were used as model inputs. Model forecasts were validated against ERA5 rainfall data and compared with the forecasts from the Global Forecast System (GFS) model. Results indicate that the Random Forest model outperforms all others, achieving an RMSE of 6.11 mm and Pearson’s correlation coefficient (R) of 0.74 for a 1-day lead time. The LSTM model achieved an RMSE of 7.46 mm, while the MLP performed less effectively than both RF and LSTM, with an RMSE of 7.61 mm. In comparison, the GFS forecasts displayed an RMSE of 9.16 mm. The RF model outperformed the other models at all lead times; however, its accuracy decreased as the lead time increased. This study highlights the potential of machine learning to improve short to medium range rainfall forecasts, contributing to timely decision-making for urban resilience and resource management.

Improving the performance of Deep Learning (DL) algorithms is a challenging problem. However, DL is applied to different types of Deep Neural Networks, and Long Short-Term Memory (LSTM) is one of them that deals with time series or sequential data. This paper attempts to overcome this problem by optimizing LSTM parameters using the Bees Algorithm (BA), which is a nature-inspired algorithm that mimics the foraging behavior of honey bees. In particular, it was used to optimize the adjustment factors of the learning rate in the forget, input, and output gates, in addition to cell candidate, in both forward and backward sides. Furthermore, the BA was used to optimize the learning rate factor in the fully connected layer. In this study, artificial porosity images were used for testing the algorithms; since the input data were images, a Convolutional Neural Network (CNN) was added in order to extract the features in the images to feed into the LSTM for predicting the percentage of porosity in the sequential layers of artificial porosity images that mimic real CT scan images of products manufactured by the Selective Laser Melting (SLM) process. Applying a Convolutional Neural Network Long Short-Term Memory (CNN-LSTM) yielded a porosity prediction accuracy of 93.17%. Although using Bayesian Optimization (BO) to optimize the LSTM parameters mentioned previously did not improve the performance of the LSTM, as the prediction accuracy was 93%, adding the BA to optimize the same LSTM parameters did improve its performance in predicting the porosity, with an accuracy of 95.17% where a hybrid Bees Algorithm Convolutional Neural Network Long Short-Term Memory (BA-CNN-LSTM) was used. Furthermore, the hybrid BA-CNN-LSTM algorithm was capable of dealing with classification problems as well. This was shown by applying it to Electrocardiogram (ECG) benchmark images, which improved the test set classification accuracy, which was 92.50% for the CNN-LSTM algorithm and 95% for both the BO-CNN-LSTM and BA-CNN-LSTM algorithms. In addition, the turbofan engine degradation simulation numerical dataset was used to predict the Remaining Useful Life (RUL) of the engines using the LSTM network. A CNN was not needed in this case, as there was no feature extraction for the images. However, adding the BA to optimize the LSTM parameters improved the prediction accuracy in the testing set for the LSTM and BO-LSTM, which increased from 74% to 77% for the hybrid BA-LSTM algorithm.

In this paper, we develop a method based on a deep learning method in financial market prediction, which includes BRICS economies as the test cases. Financial markets are rife with volatility that is affected by a "bed of complexity," coddled by local and distal factors. To leverage these vast datasets both deep learning models such as Convolutional Neural Networks (CNNs), Long Short Term Memory (LSTM) networks as well as hybrid architectures are used in this study. The paper evaluates the predictive accuracy of the models, and by so doing, identifies their strengths in predicting temporal dependencies and intricate market patterns. In particular, deep learning techniques are applied to case studies of individual countries in the BRICS to highlight the application of deep learning to disparate country specific problems, such as liquidity crises and market shocks. These findings show that classical statistical methods are outperformed by deep learning systems in a precise and reliable financial forecasting. This research highlights the ability of AI driven systems to change financial decision making processes, improving investor confidence and improving economic stability in BRICS nations. This study also readers the value of deep learning in financial market analysis, especially in economies in the developing countries. Application of techniques and architectures e.g. Convolutional Neural Networks (CNNs) that excel at identifying spatial patterns, and Long Short-Term Memory (LSTM) networks renowned for their prowess on sequential and time series data, for real world market prediction are explained. In addition, the research discusses hybrid architectures which extend knowledge, fusing strengths of both architectures to improve prediction accuracy and how deep learning develops to solve particular financial challenges. Through reading these notes readers get exposed to data preprocessing techniques such as normalization and feature selection which are important for boosting deep learning performance. The paper also includes an introduction to the evaluation of models using MSE and R-squared values for validating them in terms of reliable outputs. This research combines deep learning theory and practical case study to offer a useful educational resource for students, researchers, and practitioners who want to apply AI in financial forecasting in complex and dynamic global markets.