Automatic Intrusion Detection System Using Deep Recurrent Neural Network Paradigm

A recurrent neural network (RNN) model is developed for simulating and predicting shear behavior of both a fine-grained residual soil and a dune sand. The RNN model with one hidden layer of 20 nodes appears very effective in modeling complex soil behavior, due to its feedback connections from a hidden layer to an input layer. A dynamic gradient descent learning algorithm is used to train the network. By training part of the experimental data, which include strain-controlled undrained tests and stress-controlled drained tests performed on a residual Hawaiian volcanic soil, the network is able to capture significant variability of shear behavior existing in the residual soil. The unusual characteristics that the denser soil samples dilate under a higher stress level and the looser soil samples contract under a lower stress level are well represented by the RNN model. The RNN model also shows encouraging results in simulation and prediction of behavior of a dune sand which experienced loading-unloading-reloading conditions. Excellent agreements between the measured data and the modeling results are observed in both stress-strain behavior and volumetric-change characteristics. As compared with a traditional model, the RNN model shows more effectiveness and less effort.Key words: neural network, modeling, soil behavior, shear tests, simulation, prediction.

Accurate production forecasting is crucial for effective reservoir management and field development planning. Traditional methods; such as numerical simulation, material balance equations coupled with well models, and advanced decline curve analysis (DCA) require extensive data and involve tradeoffs between accuracy and complexity. Recently, deep learning techniques, specifically artificial neural network (ANN) and recurrent neural network (RNN) models have demonstrated promising performance in this domain. This study applies ANN and RNN for production forecasting and compares their effectiveness with traditional DCA models.An actual production dataset for a gas field containing 13 wells is used to compare ANN and RNN models to traditional decline curve analysis models. After thorough data cleaning and preparation, the dataset was divided into training (historical) and test data sets. The training data was used to build ANN and RNN models in addition to estimating the model parameters for traditional DCA models. Traditional DCA models that were investigated include Arps (exponential, hyperbolic, and harmonic), Duong, Power Law Exponential Decline (PLE), and Stretched Exponential Decline (SEPD). Finally, the prediction of different models was evaluated and compared using the testing dataset. The comparison also included long, medium, and short-term prediction.Based on the comparison of daily gas production rate forecast, the ANN model demonstrated notable improvements over Arps models for the majority of wells, with a mean absolute percent error (MAPE) of 3.9% compared to 9.1%. However, across all wells, the average MAPE was slightly lower at 6.3% versus 8.1%. Conversely, the RNN model exhibited varied performance compared to Arps models. It showed poorer long-term performance (13% versus 4.5%), similar performance in the medium term (4.4% versus 4.2%), and superior performance in the short term (2.7% versus 4.2%). Furthermore, among DCA models specifically developed for unconventional wells, SEPD exhibited nearly identical performance to Arps (8.3% compared to 8.1%). In contrast, Duong model demonstrated the poorest performance with MAPE of 21.9%.This study provides insights into model analysis and introduces guidelines in using artificial and recurrent neural networks for predicting production rates. The ANN and RNN models were developed through careful testing with different designs and techniques. All the models in this research were created using Python scripts and powerful open-source libraries like SciPy and TensorFlow.

One of the most investigated issues in health informatics is breast cancer diagnostics. Cancer treatment has also been widely researched, showing the importance of early cancer detection. Health records are integrated into a Deep Learning Model to provide an enhanced prognosis. The goal of the work is to create a system that automated those leverages with deep learning and recurrent neural network (RNN) models. The proposed work used the technique for cancer prediction as RNN and a Gated Recurrent Unit (GRU). In the simulation, Python and the cancer dataset have been used. The performance of the system represents that cancer predictions using GRU and RNN models gives a significant role.

The object of the study is methods based on neural networks for analyzing the tonality of a corpus of texts. To achieve the goal set in the work, it is necessary to solve the following tasks: study the theoretical material for learning deep neural networks and their features in relation to natural language processing; study the documentation of the Tensorflow library; develop models of convolutional and recurrent neural networks; to develop the implementation of linear and non-linear classification methods on bag of words and Word2Vec models; to compare the accuracy and other quality indicators of implemented neural network models with classical methods. Tensorboard is used for learning visualization. The work shows the superiority of classifiers based on deep neural networks over classical classification methods, even if the Word2Vec model is used for vector representations of words. The model of recurrent neural network with LSTM blocks has the highest accuracy for this corpus of texts.

Artificial intelligence is a promising futuristic concept in the field of science and technology, and is widely used in new industries. The deep-learning technology leads to performance enhancement and generalization of artificial intelligence technology. The global leader in the field of information technology has declared its intention to utilize the deep-learning technology to solve environmental problems such as climate change, but few environmental applications have so far been developed. This study uses deep-learning technologies in the environmental field to predict the status of pro-environmental consumption. We predicted the pro-environmental consumption index based on Google search query data, using a recurrent neural network (RNN) model. To verify the accuracy of the index, we compared the prediction accuracy of the RNN model with that of the ordinary least square and artificial neural network models. The RNN model predicts the pro-environmental consumption index better than any other model. We expect the RNN model to perform still better in a big data environment because the deep-learning technologies would be increasingly sophisticated as the volume of data grows. Moreover, the framework of this study could be useful in environmental forecasting to prevent damage caused by climate change.

Deep learning models, including recurrent neural network (RNN) and gated recurrent unit (GRU), were used to construct the clinical prognostic prediction models for peritoneal dialysis (PD) patients based on routine clinical data. The performance of the RNN and GRU were compared with logistic regression (LR), which is commonly used in medical researches. The possible underlining clinical implications based on the result from the GRU model were also investigated. We used the clinical data from the PD center of Peking University Third Hospital as the data source. Both the baseline data at the beginning of dialysis, and the follow-up and prognostic data of the patients were used by the RNN and GRU prediction models. The hyper-parameters were tuned based on the 10-fold cross-validation. The risk prediction performance of each model was evaluated via area under the receiver operation characteristic curve (AUROC), recall rate and F1-score on the testset. A total of 656 patients with the 261 occurrences of death were included in the experiment. The total number of all diagnostic records were 13 091. The results on the testset showed that the AUROC of the LR model, RNN model, and GRU model was 0.701 4, 0.786 0, and 0.814 7, respectively. The predictive performances of the GRU and RNN models were significantly better than that of the LR model. The performances of the GRU and RNN models assessed by recall rate and F1-score were also significantly better than that of the LR model, in which the GRU model reached the best performance. In addition, the recall rates were different among different causes of death or by different prediction time windows. The recurrent neural network model, especially the GRU model, is more effective in predicting PD patients' prognosis as compared with the LR model. This new model may be helpful for clinicians to provide timely intervention, thus improving the quality of care of PD.

Recurrent Neural Network (RNN) functions can be used to model highly nonlinear driver circuits. In this paper, RNN modeling technique is extended to multiple ports to model both the power supply noise and the ground noise accurately. RNN driver models can be extended to multiple ports to capture sensitive effects like Simultaneous Switching Noise (SSN) accurately when multiple driver circuits are switching. A comparison study is performed on test cases between RNN models and transistor level driver circuits for accuracy and computational speed-up is performed on few test cases. Results show that RNN driver models have huge computational speed-up over transistor level driver circuits.

Deep unfolding methods design deep neural networks as learned variations of optimization algorithms through the unrolling of their iterations. These networks have been shown to achieve faster convergence and higher accuracy than the original optimization methods. In this line of research, this paper presents novel interpretable deep recurrent neural networks (RNNs), designed by the unfolding of iterative algorithms that solve the task of sequential signal reconstruction (in particular, video reconstruction). The proposed networks are designed by accounting that video frames' patches have a sparse representation and the temporal difference between consecutive representations is also sparse. Specifically, we design an interpretable deep RNN (coined reweighted-RNN) by unrolling the iterations of a proximal method that solves a reweighted version of the l1 - l1 minimization problem. Due to the underlying minimization model, our reweighted-RNN has a different thresholding function (alias, different activation function) for each hidden unit in each layer. In this way, it has higher network expressivity than existing deep unfolding RNN models. We also present the derivative l1 - l1 -RNN model, which is obtained by unfolding a proximal method for the l1 - l1 minimization problem. We apply the proposed interpretable RNNs to the task of video frame reconstruction from low-dimensional measurements, that is, sequential video frame reconstruction. The experimental results on various datasets demonstrate that the proposed deep RNNs outperform various RNN models.

The redundant manipulator as a ubiquitous and essential component of robots, acts as a momentous role in mechanized production. The motion control of a redundant manipulator is an essential problem that must be handled. Consequently, for the repetitive motion control of redundant manipulators, a recurrent neural network (RNN) model for the double-index (DI) scheme is researched in this paper. The DI scheme sets the minimum kinetic energy (MKE) and minimum joint-angle offset (MJAO) as optimization indexes, for which the time-dependent weights are designed, and takes the dynamic equations of redundant manipulators and joint limit constraints into consideration as well. Then, the DI scheme is reformulated as a quadratic programming (QP) problem. Besides, the RNN model based on the iteration of related parameters is derived to solve the QP problem, through which joint data (i.e., joint angles, joint velocities, and joint accelerations) are obtained to drive the motion of the redundant manipulator. To verify the effectiveness of the DI scheme for motion control of redundant manipulator, simulation results of the DI scheme solved by the RNN model, the minimum acceleration norm (MAN) scheme and MKE scheme are compared.

Recurrent neural networks (RNNs) have been widely used in text similarity modeling for text semantic representation learning. However, referring to the classical topic models, a text contains many different latent topics, and the complete semantic information of the text is described by all the latent topics. Previous RNN based models usually learn the text representation with the separated words in the text instead of topics, which will bring noises and loss hierarchical structure information for text representation. In this paper, we proposed a novel fractional latent topic based RNN (FraLT-RNN) model, which focuses on the text representation in topic-level and largely preserve the whole semantic information of a text. To be specific, we first adopt the fractional calculus to generate latent topics for a text with the hidden states learned by a RNN model. Then, we propose a topic-wise attention gating mechanism and embed it into our model to generate the topic-level attentive vector for each topic. Finally, we reward the topic perspective with the topic-level attention for text representation. Experiments on four benchmark datasets, namely TREC-QA and WikiQA for answer selection, MSRP for paraphrase identification, and MultiNLI for textual entailment, show the great advantages of our proposed model.

Indonesia consists of various tribes with their respective regional languages, one of which is the Besemah tribe in South Sumatra Province with its language culture, namely the Besemah Language. Until now, the Besemah Language is still used by the Besemah tribe, but over time the number of Besemah Language speakers has decreased, plus most of the wider community does not know what the Besemah Language is. Machine Learning is a part of artificial intelligence that is often used to solve various problems. Machine Learning involves the use of computers and mathematical algorithms that use data to make predictions in the future. Machine translation is a tool that can convert one language to another. This study aims to collect datasets in the form of sentences and words from the Besemah Language, then create a Besemah Language translation machine to Indonesian and vice versa. In the research conducted, the approach used is Experimental Research in Machine Learning. Experimental research in machine learning for language translation is a research approach that involves designing and implementing a series of experiments to test and validate the performance of the language translation model. In this study, Neural Machine Translation (NMT) technology was applied with the Recurrent Neural Network (RNN) approach. The results of the study showed that the val_accuracy value for the Besemah-Indonesian translation was 0.8469 and for Indonesia-Besemah was 0.8492, in the translation trial conducted using the RNN model, 100 epochs, batch size 64, and validation split of 0.2.

Analysis of TEC variations and prediction of TEC by RNN during Indonesian earthquakes occurred from 2004 to 2024 and comparison with IRI-2020 model

In this paper, we propose new listwise learning-to-rank models that mitigate the shortcomings of existing ones. Existing listwise learning-to-rank models are generally derived from the classical Plackett-Luce model, which has three major limitations. (1) Its permutation probabilities overlook ties, i.e., a situation when more than one document has the same rating with respect to a query. This can lead to imprecise permutation probabilities and inefficient training because of selecting documents one by one. (2) It does not favor documents having high relevance. (3) It has a loose assumption that sampling documents at different steps is independent. To overcome the first two limitations, we model ranking as selecting documents from a candidate set based on unique rating levels in decreasing order. The number of steps in training is determined by the number of unique rating levels. We propose a new loss function and associated four models for the entire sequence of weighted classification tasks by assigning high weights to the selected documents with high ratings for optimizing Normalized Discounted Cumulative Gain (NDCG). To overcome the final limitation, we further propose a novel and efficient way of refining prediction scores by combining an adapted Vanilla Recurrent Neural Network (RNN) model with pooling given selected documents at previous steps. We encode all of the documents already selected by an RNN model. In a single step, we rank all of the documents with the same ratings using the last cell of the RNN multiple times. We have implemented our models using three settings: neural networks, neural networks with gradient boosting, and regression trees with gradient boosting. We have conducted experiments on four public datasets. The experiments demonstrate that the models notably outperform state-of-the-art learning-to-rank models.

Accurate temperature and rainfall (T&R) forecasting is vital for the climate-sensitive regions of Northern India, particularly Jammu, Kashmir, and Ladakh, where volatile weather patterns significantly affect livelihoods, socio-economic development, and disaster management efforts. Despite their importance, traditional forecasting methods often fall short due to their high computational demands and inability to provide localized, real-time predictions, leaving a critical research gap in addressing these challenges. This study addresses the need for precise and efficient T&R forecasting using deep learning-based framework tailored to the unique climatic conditions of these regions. The major research focus is to develop and evaluate a model capable of capturing complex temporal dependencies in localized time-series weather data. Utilizing data from the Indian Meteorological Department (IMD) for Jammu, Srinagar, and Ladakh stations covering the period from January 1, 2000, to December 31, 2023, the proposed framework employs recurrent neural networks (RNN) and long short-term memory (LSTM) architectures, both optimized for time-series forecasting. Key findings reveal that while both RNN and LSTM models exhibit robust performance in single input single output (SISO) setups, RNN model consistently outperforms the LSTM in capturing intricate temporal relationships. The RNN model in MIMO configuration achieved significantly lower mean absolute error (MAE), root mean squared error (RMSE), and mean squared error (MSE) for Jammu, Srinagar, and Ladakh, with respective values of [0.0636, 0.1011, 0.0401] for Jammu, [0.1048, 0.1555, 0.0455] for Srinagar, and [0.0854, 0.1344, 0.0411] for Ladakh. These results underscore the RNN model’s precision, making it a practical tool for real-time weather forecasting. By enhancing the accuracy of T&R predictions in regions with challenging meteorological conditions, this study contributes to improved climate adaptation strategies, disaster preparedness, and sustainable development. Its findings hold broader implications for advancing localized forecasting technologies in other regions with similar climatic complexities.

The predictive power of environmental factors such as CO2 levels and Brent oil price was evaluated on the stock market (S&P 500) and foreign exchange (FX) rates for market currency pairs (EUR/USD) and emerging market currency pairs (MXN/USD) with recurrent neural networks (RNN). The S&P 500 is a collection of well-established companies in the United States representing the macroeconomic health of the nation and the global economy. The FX market is a global decentralized over-the-counter (OTC) market used to determine the spot price of currency pairs. A highly leveraged market usually trades within a specific price range. Since stocks and FX pairs are highly correlated to macroeconomic factors, it was hypothesized that environmental factors such as CO2 levels and oil prices also have predictive power due to their close causal relationship with anthropological economic activities. To verify the predictive power, an RNN model was built, and a bi-directional neural network with an internal state was used to process data sequences. The performance of RNN was quantified by measuring the residual prediction from the true value. Although the study at its current state might need further statistical rigor, it concluded that environmental factors increased the predictive power for the S&P 500 while decreasing it for the DM FX pair and the EM FX pair showed mixed results.