Stochastic Markov Recurrent Neural Network for Source Separation

Prediction concept needs human-like thinking and ideas. ANN helps make a program for describing the solutions to the prediction problems. To gain accurate prediction, there is a need for deep thinking in the knowledge system discovered. Deep learning helps form such next-level type of ANN thinking. Deep learning models do not require humans for programming the problems. It can determine the prediction by itself by learning the dataset. This paper describes the prediction by considering sequential model for continuous accessing of data with the help of multi-layers. Deep learning is very helpful for predicting larger dataset. This paper proposes an algorithmic model known as a Comprehensive Deep Recurrent Artificial Neural Networks (CDRANN) by hybridizing LSTM (long short-term memory) with RNN (recurrent neural network) backtrack solver for better future stock prediction. This paper also explains the need for hybridizing evolutionary model CDRANN to create a novel change in future prediction.KeywordsCDRANN (Comprehensive Deep Recurrent Artificial Neural Networks)LSTM (long short-term memory)RNN (recurrent neural network)Stock marketPrediction

Recurring wildfires pose a critical global issue as they undermine social and economic stability and jeopardize human lives. To effectively manage disasters and bolster community resilience, the development of wildfire susceptibility maps (WFSMs) has emerged as a crucial undertaking in recent years. In this research endeavor, two deep learning algorithms were leveraged to generate WFSMs using two distinct remote sensing datasets. Specifically, the Moderate-Resolution Imaging Spectroradiometer (MODIS) and Landsat-8 images were utilized to monitor wildfires that transpired during the year 2021. To develop an effective WFSM, two datasets were created by incorporating 599 wildfire locations with Landsat-8 images and 232 sites with MODIS images, as well as twelve factors influencing wildfires. Deep learning algorithms, namely the long short-term memory (LSTM) and recurrent neural network (RNN), were utilized to model wildfire susceptibility using the two datasets. Subsequently, four WFSMs were generated using the LSTM (MODIS), LSTM (Landsat-8), RNN (MODIS), and RNN (Landsat-8) algorithms. The evaluation of the WFSMs was performed using the area under the receiver operating characteristic (ROC) curve (AUC) index. The results revealed that the RNN (MODIS) (AUC = 0.971), RNN (Landsat-8) (AUC = 0.966), LSTM (MODIS) (AUC = 0.964), and LSTM (Landsat-8) (AUC = 0.941) algorithms demonstrated the highest modeling accuracy, respectively. Moreover, the Gini index was employed to assess the impact of the twelve factors on wildfires in the study area. The results of the random forest (RF) algorithm indicated that temperature, wind speed, slope, and topographic wetness index (TWI) parameters had a significant effect on wildfires in the study region. These findings are instrumental in facilitating efficient wildfire management and enhancing community resilience against the detrimental effects of wildfires.

Photovoltaic power generation is one of the most efficient renewable energy generation techniques. To ensure photovoltaic (PV) safe operation and cost-effective integration into smart grids. As a result, a realistic PV power forecast method can aid in mitigating PV power generation’s drawbacks, which is critical for power plant maintenance and repair. In this research, we compared two Deep Learning models which are both highly advantageous, namely a long short-term memory (LSTM) and a recurrent neural network (RNN). A case study employing an actual dataset obtained from 1B DKASC, Alice Springs, Australia, is used to demonstrate the performance of LSTM and RNN. The forecasts generated by the LSTM are compared with the results of RNN using MAE, MSE, MBE, R\(^2\) and RMSE for all the forecast horizons. The results of the study demonstrate that the LSTM based forecast outperforms RNN and has the potential to improve the accuracy and stability of the prediction.KeywordsDeep LearningLong Short-Term Memory (LSTM)Recurrent neural network (RNN)

Abstract— Stocks are investments that have dynamic movements. Stock price changes move every day even hourly. With very fast changes, stock prices require predictions to be able to determine stock market projections. Predictions are used to reduce risk when making transactions. In this study, predictions of stock price trends were made using the Recurrent Neural Network (RNN). The approach taken is to perform a time series analysis using the RNN variance, namely Long Short Term Memory (LSTM). Hyperparameter construction in the LSTM model testing simulation can estimate stock prices with maximum percentage accuracy. The results showed that the prediction model produced a loss function of 0.0012 and a training time of 73 m/step. The evaluation was carried out with the RMSE which resulted in a score of 17.13325. Predictions are obtained after doing machine learning using 1239 data. The RMSE and LSTM models are calculated by changing the number of epochs, the variation between the predicted stock price and the current stock price. Computations are carried out using a stock market dataset that includes open, high, low, close, adj prices, closes, and volumes. The main objective of this study is to determine the extent to which the LSTM algorithm anticipates stock market prices with better accuracy. Code can be seen at iranihoeronis/RNN-LSTM (github.com) Keywords— Stock Prediction, Time Series, Recurrent Neural Network (RNN), Long Short Term Memory (LSTM).

Stock price returns forecasting is challenging task for day traders to yield more returns. In the past, most of the literature was focused on machine learning algorithm to predict the stock returns. In this work, the recurrent neural network (RNN) with long short term memory (LSTM) is studied to forecast future stock returns. It has the ability to keep the memory of historical stock returns in order to forecast future stock return output. RNN with LSTM is used to store recent stock information than old related stock information. We have considered a recurrent dropout in RNN layers to avoid overfitting in the model. To accomplish the task we have calculated stock return based on stock closing prices. These stock returns are given as input to the recurrent neural network. The objective function of the prediction model is to minimize the error in the model. To conduct the experiment, data is collected from the National Stock Exchange, India (NSE). The proposed RNN with LSTM model outperforms compared to an feed forward artificial neural network.

Sequence Classification is one of the on-demand research projects in the field of Natural Language Processing (NLP). Classifying a set of images or text into an appropriate category or class is a complex task that a lot of Machine Learning (ML) models fail to accomplish accurately and end up under-fitting the given dataset. Some of the ML algorithms used in text classification are KNN, Naïve Bayes, Support Vector Machines, Convolutional Neural Networks (CNNs), Recursive CNNs, Recurrent Neural Networks (RNNs), Long Short Term Memory (LSTM), etc. For this experimental study, LSTM and a few other algorithms were chosen for a more comparative study. The dataset used is the SMS Spam Collection Dataset from Kaggle and 150 more entries were additionally added from different sources. Two possible class labels for the data points are spam and ham. Each entry consists of the class label, a few sentences of text followed by a few useless features that are eliminated. After converting the text to the required format, the models are run and then evaluated using various metrics. In experimental studies, the LSTM gives much better classification accuracy than the other machine learning models. F1-Scores in the high nineties were achieved using LSTM for classifying the text. The other models showed very low F1-Scores and Cosine Similarities indicating that they had underperformed on the dataset. Another interesting observation is that the LSTM had reduced the number of false positives and false negatives than any other model.

Recurrent neural networks (RNN) are at the core of modern automatic speech recognition (ASR) systems. In particular, long short-term memory (LSTM) recurrent neural networks have achieved state-of-the-art results in many speech recognition tasks, due to their efficient representation of long and short term dependencies in sequences of inter-dependent features. Nonetheless, internal dependencies within the element composing multidimensional features are weakly considered by traditional real-valued representations. We propose a novel quaternion long short-term memory (QL-STM) recurrent neural network that takes into account both the external relations between the features composing a sequence, and these internal latent structural dependencies with the quaternion algebra. QLSTMs are compared to LSTMs during a memory copy-task and a realistic application of speech recognition on the Wall Street Journal (WSJ) dataset. QLSTM reaches better performances during the two experiments with up to 2.8 times less learning parameters, leading to a more expressive representation of the information.

Most Graph Neural Networks (GNNs) follow the message-passing scheme. Residual connection is an effective strategy to tackle GNNs' over-smoothing issue and performance reduction issue on non-homophilic networks. Unfortunately, the coarse-grained residual connection still suffers from class-imbalanced over-smoothing issue, due to the fixed and linear combination of topology and attribute in node representation learning. To make the combination flexible to capture complicated relationship, this paper reveals that the residual connection needs to be node-dependent, layer-dependent, and related to both topology and attribute. To alleviate the difficulty in specifying complicated relationship, this paper presents a novel perspective on GNNs, i.e., the representations of one node in different layers can be seen as a sequence of states. From this perspective, existing residual connections are not flexible enough for sequence modeling. Therefore, a novel node-dependent residual connection, i.e., Long Short-Term Graph Memory Network (LSTGM) is proposed to employ Long Short-Term Memory (LSTM), to model the sequence of node representation. To make the graph topology fully employed, LSTGM innovatively enhances the updated memory and three gates with graph topology. A speedup version is also proposed for effective training. Experimental evaluations on real-world datasets demonstrate their effectiveness in preventing over-smoothing issue and handling networks with heterophily.

While the growing accessibility of technology improves usability, it also creates more opportunities for cybercriminals to exploit vulnerabilities, significantly accelerating the proliferation of cybersecurity attacks. Deep learning (DL) approaches present significant advancements over conventional machine learning (ML) techniques by automating feature selection and extraction while minimizing external dependencies. This study proposes a deep learning-based model to enhance cyberattack detection and ensure that data security goals are achieved. This study employs a quantitative research design, utilizing simulation and modeling as the primary analytical tools. The dataset used is the Canadian Institute for Cybersecurity Intrusion Detection System (CIC-IDS-2017) dataset. Three distinct DL algorithms are used to design the detection models, namely, Deep Neural Network (DNN), Long Short-Term Memory (LSTM), and Recurrent Neural Network (RNN). The performance comparison metrics are F1-score, accuracy, sensitivity, false positive rate (FPR), specificity, positive predictive value (PPV), false negative rate (FNR), and negative predictive value (NPV). Optimization concepts are integrated to enhance the detection efficiency in web-based systems, including loss functions, gradient-based optimization, and efficient model generalization techniques. The results of k-fold cross-validation show LSTM’s higher scores for F1-score (94.6%), recall (94.7%), accuracy (94.8%), and precision (94.6%). LSTM outperformed RNN and DNN, achieving the highest accuracy, precision, specificity, and sensitivity at 94.7%, 94.3%, 98.9%, and 94.7% respectively, validating LSTM's superior generalizability for cyberattack detection tasks.

Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mechanism concept can help RNNs remember and forget previous information. Representing the hidden layers of an RNN with more expressive operations (i.e., tensor products) helps it learn a more complex relationship between the current input and the previous hidden layer information. These ideas can generally improve RNN performances. In this paper, we proposed a novel RNN architecture that combine the concepts of gating mechanism and the tensor product into a single model. By combining these two concepts into a single RNN, our proposed models learn long-term dependencies by modeling with gating units and obtain more expressive and direct interaction between input and hidden layers using a tensor product on 3-dimensional array (tensor) weight parameters. We use Long Short Term Memory (LSTM) RNN and Gated Recurrent Unit (GRU) RNN and combine them with a tensor product inside their formulations. Our proposed RNNs, which are called a Long-Short Term Memory Recurrent Neural Tensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor Network (GRURNTN), are made by combining the LSTM and GRU RNN models with the tensor product. We conducted experiments with our proposed models on word-level and character-level language modeling tasks and revealed that our proposed models significantly improved their performance compared to our baseline models.

Cryptocurrency price prediction is most wanted by investors nowadays to get more money in cryptocurrency investment. All existing methods depicted in the survey for Cryptocurrencies price prediction are not suitable for real-time investment price prediction. To handle the above-mentioned issues, Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) is anticipated for Cryptocurrency price prediction. The proposed method depends on machine learning technique, mostly in monetary fields for forecasting stock prices. Min-Max Scaler is used for pre-processing, changing the numeric values to the common scale in the dataset. LSTM is an Artificial Recurrent Neural Network (RNN) model employed in the deep learning field, and here it is used for cryptocurrency price prediction. Recurrent Neural Network (RNN) using LS TM can be accomplished in the proposed model, which proceeds with a set of working out sequences by using an optimization procedure like gradient descent with back transmission through time to calculate the gradients required through the progression of optimization in order to change each weight of the LSTM network to perform error calculation at the output layer of LSTM with respect to the corresponding weight. The proposed strategy involves the result from the model, which is considered as the another contribution for a similar model.

One of the most promising tools in recent years for the analysis and prediction of time series data, which includes financial market data has been the use of neural networks. To ensure the accuracy of the outcomes of these systems, it is critical to overcome the vanishing gradient and exploding gradient problems that often occur when recurrent neural networks (RNN) process data. Long Short-Term Memory (LSTM) has been shown to provide good performance when dealing with time series datasets. This paper will explore the feasibility of using an RNN with LSTM as a predictive tool for use with futures trading data. Using a dataset comprised of all futures trading occurring on the Bourse de MontrA©al (TMX) during a 9-month period from January to September 2015, we assessed the predictive effectiveness of an RNN in predicting the price of front-end contracts for the futures symbol BAX. We found that while an RNN provided a degree of short-term predictive capability, this capability did not extend beyond a couple of days. Although it failed as a trading instrument to predict futures prices, the RNN could detect, identify, and reflect underlying trends in the data, indicating the tool may hold promise in the detection of trading patterns.Â

This paper develops a general framework for learning interpretable data representation via Long Short-Term Memory (LSTM) recurrent neural networks over hierarchal graph structures. Instead of learning LSTM models over the pre-fixed structures, we propose to further learn the intermediate interpretable multi-level graph structures in a progressive and stochastic way from data during the LSTM network optimization. We thus call this model the structure-evolving LSTM. In particular, starting with an initial element-level graph representation where each node is a small data element, the structure-evolving LSTM gradually evolves the multi-level graph representations by stochastically merging the graph nodes with high compatibilities along the stacked LSTM layers. In each LSTM layer, we estimate the compatibility of two connected nodes from their corresponding LSTM gate outputs, which is used to generate a merging probability. The candidate graph structures are accordingly generated where the nodes are grouped into cliques with their merging probabilities. We then produce the new graph structure with a Metropolis-Hasting algorithm, which alleviates the risk of getting stuck in local optimums by stochastic sampling with an acceptance probability. Once a graph structure is accepted, a higher-level graph is then constructed by taking the partitioned cliques as its nodes. During the evolving process, representation becomes more abstracted in higher-levels where redundant information is filtered out, allowing more efficient propagation of long-range data dependencies. We evaluate the effectiveness of structure-evolving LSTM in the application of semantic object parsing and demonstrate its advantage over state-of-the-art LSTM models on standard benchmarks.

With the development of China's Belt and Road Initiative (BRI), the port plays a significant role and its operation management faces some pressure. In this regard, prediction of daily container volumes will provide the manager with data support for better plan of a storage yard. In this work, by deep learning the historical dataset, the long short-term memory (LSTM) recurrent neural network (RNN) is trained and used to predict daily volumes of containers which will enter the storage yard. The raw dataset of a certain port from 2013 to 2016 is chosen as the training set and the dataset of 2017 is used as the test set to evaluate the performance of the proposed prediction model. Then the LSTM model is established with Python and Tensorflow framework. The structure parameters are adjusted to find the optimal LSTM network, so as to improve the prediction accuracy. It appears that the LSTM model with two hidden layers and 30 hidden layer units has less prediction error between the real data and predicted data of 2017. The prediction error of daily container volumes between predicted value and real data of 2017 is about 12.39%, which is less than the people-predicted error. It is promising that the proposed LSTM RNN model can be applied to predict the daily volumes of containers and have higher prediction accuracy.

Prediction of dissolved oxygen in a fishery pond based on gated recurrent unit (GRU)