ECG-based biometrics using recurrent neural networks

Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mechanism concept can help RNNs remember and forget previous information. Representing the hidden layers of an RNN with more expressive operations (i.e., tensor products) helps it learn a more complex relationship between the current input and the previous hidden layer information. These ideas can generally improve RNN performances. In this paper, we proposed a novel RNN architecture that combine the concepts of gating mechanism and the tensor product into a single model. By combining these two concepts into a single RNN, our proposed models learn long-term dependencies by modeling with gating units and obtain more expressive and direct interaction between input and hidden layers using a tensor product on 3-dimensional array (tensor) weight parameters. We use Long Short Term Memory (LSTM) RNN and Gated Recurrent Unit (GRU) RNN and combine them with a tensor product inside their formulations. Our proposed RNNs, which are called a Long-Short Term Memory Recurrent Neural Tensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor Network (GRURNTN), are made by combining the LSTM and GRU RNN models with the tensor product. We conducted experiments with our proposed models on word-level and character-level language modeling tasks and revealed that our proposed models significantly improved their performance compared to our baseline models.

Stock price forecasting using PSO hypertuned neural nets and ensembling

This paper examines and evaluates the effectiveness of recurrent neural network (RNN) architectures for natural language knowledge. The paper starts by imparting historical data on recurrent neural networks and their software to various responsibilities. It then goes on to talk about diverse RNN architectures, such as lengthy quick-time period reminiscence (LSTM), Gated Recurrent Unit (GRU) and Bonior Recurrent Neural network (BONN). The paper then focuses on the efficacy of each RNN structure for textual content expertise obligations. A sequence of experiments is carried out to degree the overall performance of the one-of-a-kind fashions, the use of 3 preferred datasets-Wiki text, Squad and IMDB, as well as two newly created datasets. The paper analyses the empirical consequences of the various RNN architectures and finds that LSTM fashions are normally satisfactory performers, although BONN architectures can also be aggressive in some cases. The paper additionally looks at numerous techniques to enhance performance, which include pre-processing and hyper-parameter tuning. The paper concludes that, whilst RNN architectures are powerful for herbal language know-how responsibilities, the exact architecture and parameters can also need to be chosen cautiously, relying on the project and dataset to hand.

MOOCs (Massive Open Online Courses) are definitely one of the best approach to support the international agenda about inclusive and equitable education and lifelong learning opportunities for all (SDG4) [1]. A great deal universities and institutions offer valuable free courses to their numerous students and to people around the word through MOOC platforms. However, because of huge number of learners and data generated, learner’s behaviour in those platforms remain a kind of black box for learners themselves and for courses instructors who are supposed to tutor or monitor learners in the learning process. Therefore, learner do not receive sufficient support from instructors and from their peers, during the learning process [2]. This is one the main reasons that lead to high dropout, low completion and success rate observed in the MOOCs. Many research work have suggested descriptive, predictive and prescriptive models to address this issue, but most of these models focus on predicting dropout, completion and/or success, and do not generally pay enough attention to one of the key step (learner behaviour), that comes before, and can explain dropping out and failure. Our research aims to develop a deep learning model to predict learner behaviour (learner interactions) in the learning process, in order to equip learners and course instructors with insight understanding of the learner behaviour in the learning process. This specific paper will focus on analysing relevant algorithms to develop such model. For this analysis, we used data from UNESCO-IICBA (International Institute for Capacity Building in Africa) MOOC platform, designed for teacher training in Africa, and then we examine many types of features: geographical, social behavioural and learning behavioural features. Learner’s behaviour being a time series Big data, we built the predictive model using Deep Learning algorithms for better performance and accuracy (Thanks to the power of deep learning) compared to baseline Machine learning algorithms. Time series data is best handled by recurrent neural networks (RNN) [3], so, we choose RNN and implemented/tested the three main architectures of RNN: Simple RNNs, GRU (Gated Recurrent Unit) RNNs and LSTM (Long short-term memory) RNNs. The models were trained using L2 Regularization, based on the predictions results, we unexpectedly found model with simple RNNs produced the best performance and accuracy on the dataset used than the other RNN architectures. We had couple of observations, example: we saw a correlation between video viewing and quiz behaviour and the participation of the learner to the learning process. This observation could allow teachers to provide meaningful support and guidance to at risk or less active students. We also observed that, the shorter the video or the quiz, the more the viewer. We conclude that we could use learner video or quiz viewing behaviour to predict his behaviour concerning other MOOC contents. These results suggest the need of deeper research on educational video and educational quiz designing for MOOCs.

We explore the architecture of recurrent neural networks (RNNs) by studying the complexity of string sequences it is able to memorize. Symbolic sequences of different complexity are generated to simulate RNN training and study parameter configurations with a view to the network's capability of learning and inference. We compare Long Short-Term Memory (LSTM) networks and gated recurrent units (GRUs). We find that an increase in RNN depth does not necessarily result in better memorization capability when the training time is constrained. Our results also indicate that the learning rate and the number of units per layer are among the most important hyper-parameters to be tuned. Generally, GRUs outperform LSTM networks on low-complexity sequences while on high-complexity sequences LSTMs perform better.

Forecasting longevity is essential in multiple research and policy areas, including the pricing of life insurance contracts, the valuation of capital market solutions for longevity risk management, and pension policy. This paper empirically investigates the predictive performance of Recurrent Neural Networks (RNN) with Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM) architectures in jointly modeling and multivariate time series forecasting of age-specific mortality rates at all ages. We fine-tune the three hidden layers GRU and LSTM model’s hyperparameters for time series forecasting and compare the model’s forecasting accuracy with that produced by traditional Generalised Age-Period-Cohort (GAPC) stochastic mortality models. The empirical results suggest that the two RNN architectures generally outperform the GAPC models investigated in both the male and female populations, but the results are sensitive to the accuracy criteria. The empirical results also show that the RNN-GRU network slightly outperforms the RNN with an LSTM architecture and can produce mortality schedules that capture relatively well the dynamics of mortality rates across age and time. Further investigations considering other RNN architectures, calibration procedures, and sample datasets are necessary to confirm the superiority of RNN in forecasting longevity.

Quantifying Gross Primary Production (GPP) is fundamental for understanding terrestrial carbon dynamics, particularly in forests. The overarching question we address here is whether integrating remote sensing (RS) with deep learning (DL) methodologies can enhance the estimation of daily forest GPP on a European scale.The Eddy Covariance (EC) method, although widely used to infer ecosystem-scale estimates of GPP from in situ CO2 exchange measurements, suffers from limited global coverage. When EC data are not available, RS data are often employed to estimate GPP by establishing statistical relationships with in situ observations. Recently, Machine Learning (ML) strategies, particularly involving RS and meteorological inputs, have been used for estimating GPP continuously in space and time.However, the potential of DL techniques, particularly those exploiting the sequential characteristics of time series data (i.e. recurrent neural network architectures) in estimating daily forest GPP has not been comprehensively explored. This includes their performance during photosynthetic downregulation (e.g. during climate extremes such as droughts) and an in-depth examination of the importance of the utilised features.This study presents a comparative analysis of three recurrent neural network architectures—Recurrent Neural Networks (RNNs), Gated Recurrent Units (GRUs), and Long-Short Term Memory (LSTMs)—for daily forest GPP estimation across ICOS ecosystem stations. We assess their performance throughout the entire year, during the growing season, and under photosynthetic downregulation events. This assessment utilises RS inputs —optical data from Sentinel-2, Land Surface Temperature (LST) from MODIS, and radar data from Sentinel-1—, combined with potential radiation data. Furthermore, we analysed the importance of these features to provide insights into the complex interactions and dependencies when estimating GPP.Our results indicate that all three architectures yield similar accuracy for full period and growing season GPP estimations, with mean NRMSE values of 0.136 and 0.170, respectively. All models exhibit increased errors while estimating GPP during photosynthetic downregulation events and their performance varies notably, with LSTMs showing the best results (NRMSE=0.202), followed by RNNs (NRMSE=0.214), and GRUs exhibiting the least efficacy (NRMSE=0.276). Additionally, our study underscores the importance of potential radiation as a critical feature influencing the GPP response, with LST data proving particularly valuable when estimating GPP during photosynthetic downregulation events, more so than optical or radar data.These findings suggest that recurrent neural network architectures, especially LSTMs, are effective in daily GPP estimations, with slight performance decreases during climate extreme conditions. The study also reveals the efficacy of combining RS data with potential radiation for accurate forest GPP estimation, with LST data being especially crucial when estimating GPP during photosynthetic downregulation events.Our research paves the way for further exploration into recurrent models and innovative architectures, with an emphasis on overcoming the challenges in modelling climate-induced GPP extremes.

Prediction of dissolved oxygen in a fishery pond based on gated recurrent unit (GRU)

To quantitatively evaluate the performance of two types of recurrent neural networks (RNNs), long short-term memory (LSTM) and gated recurrent units (GRU), using Monte Carlo dropout (MCD) to predict pharmacokinetic (PK) parameters from dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data. DCE-MRI data for simulation studies were synthesized using the extended Tofts model and a population-averaged arterial input function (AIF). The ranges of PK parameters for training the RNNs were determined from data of patients with brain tumors. The effects of the number of training samples, number of hidden units, dropout rate (DR), and bolus arrival time delay and dispersion in AIF on the accuracy of the PK parameters were investigated, and the uncertainties for different DRs and peak signal-to-noise ratios (PSNRs) were quantified. For comparison, PK parameters were estimated using the nonlinear least-squares method. In the clinical studies, the PK parameter and uncertainty images were generated by applying the trained RNNs to DCE-MRI data. Compared with GRU, the computational cost for training the LSTM was significantly higher. The prediction accuracy of GRU decreased with decreasing numbers of training samples and hidden units, whereas the performance of LSTM remained stable. Despite an increased computational cost, MCD reduced the prediction error at low PSNR and improved the quality of PK parameter images. The simulation results recommended using a DR of 0.25-0.5 at low PSNR and ≤ 0.25 for other PSNRs. The clinical studies recommended using a DR of 0.25 and 0.5 for LSTM and GRU, respectively. MCD is effective in quantifying uncertainty in PK parameter prediction from DCE-MRI data and improves their performance, particularly at low PSNR; however, at the expense of increased computational cost. This study helps deepen our understanding of RNNs with MCD and select suitable hyperparameters for creating an RNN architecture for DCE-MRI studies.

Emotion classification using EEG (Electroencephalogram) brain signals has recently garnered significant interest due to its potential applications in fields such as mental health, human-computer interaction, and affective computing. The three prominent recurrent neural network (RNN) architectures GRU (Gated Recurrent Unit), LSTM (Long Short-Term Memory), and BiLSTM (Bidirectional Long Short-Term Memory) are thoroughly compared in this study with a focus on classifying emotions. The study examines how well these RNN variations capture temporal dependencies and patterns within EEG data to precisely discriminate between various emotional states. Based on classification accuracy and training effectiveness, these model's performance will be assessed. The results will reveal the most suitable architecture in terms of accuracy and robustness for EEG-based emotion classification. The findings provide valuable insights into choosing the appropriate RNN architecture for emotion classification tasks, advancing the understanding of AI's potential in interpreting human emotions through EEG signals.

Recently, emotion recognition began to be implemented in the industry and human resource field. In the time we can perceive the emotional state of the employee, the employer could gain benefits from it as they could improve the quality of decision makings regarding their employee. Hence, this subject would become an embryo for emotion recognition tasks in the human resource field. In a fact, emotion recognition has become an important topic of research, especially one based on physiological signals, such as EEG. One of the reasons is due to the availability of EEG datasets that can be widely used by researchers. Moreover, the development of many machine learning methods has been significantly contributed to this research topic over time. Here, we investigated the classification method for emotion and propose two models to address this task, which are a hybrid of two deep learning architectures: One-Dimensional Convolutional Neural Network (CNN-1D) and Recurrent Neural Network (RNN). We implement Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM) in the RNN architecture, that specifically designed to address the vanishing gradient problem which usually becomes an issue in the time-series dataset. We use this model to classify four emotional regions from the valence-arousal plane: High Valence High Arousal (HVHA), High Valence Low Arousal (HVLA), Low Valence High Arousal (LVHA), and Low Valence Low Arousal (LVLA). This experiment was implemented on the well-known DEAP dataset. Experimental results show that proposed methods achieve a training accuracy of 96.3% and 97.8% in the 1DCNN-GRU model and 1DCNN-LSTM model, respectively. Therefore, both models are quite robust to perform this emotion classification task.

Purpose: This study aims to predict the closing price of the EUR/JPY currency pair in the forex market using recurrent neural network (RNN) architectures, namely Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), with the incorporation of Bidirectional layers. Methods: The dataset comprises hourly price data obtained from Yahoo Finance and pre-processed accordingly. The data is divided into training and testing sets, and time series sequences are constructed for input into the models. The RNN, LSTM, and GRU models are trained using the Adam optimization algorithm with the mean squared error (MSE) loss metric. Results: Results indicate that the LSTM model, particularly when coupled with Bidirectional layers, exhibits superior predictive performance compared to the other models, as evidenced by lower MSE values. Conclusions: Therefore, the LSTM model with Bidirectional layers is the most effective in predicting the EUR/JPY currency pair's closing price in the forex market. These findings offer valuable insights for practitioners and researchers involved in financial market prediction and neural network modelling

A new approach for quantitative precipitation estimation from radar reflectivity using a gated recurrent unit network

Bitcoin's decentralized nature has made it a popular mode of payment for buyers and sellers, but its highly volatile nature poses a challenge for investors. This study aims to predict future Bitcoin prices using a combination of Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) algorithms. The non-stationary nature of Bitcoin prices is addressed using RNN, which is particularly useful for analyzing sequential data.The dataset used in this research is sourced from the Kraken exchange and includes various factors that are believed to influence Bitcoin prices, such as transaction volume, hash rate, and Google search trends. The data is preprocessed and cleaned to ensure accuracy, and then fed into the RNN and LSTM models for training and testing.The study's use of RNN and LSTM algorithms demonstrates the effectiveness of these methods in predicting Bitcoin prices, particularly in the context of sequential data. The results of the study, provide insights into potential future trends in Bitcoin prices and identify key indicators that significantly influence Bitcoin prices. The findings of this research have important implications for investors and traders looking to make informed decisions in the cryptocurrency market, as well as for researchers seeking to improve our understanding of Bitcoin's price dynamics. By predicting future prices, the study provides insights that can mitigate the risks associated with Bitcoin's volatility, making it a more viable investment option.

Coalbed methane (CBM) has emerged as one of the clean unconventional resources to supplement the rising demand of conventional hydrocarbons. Analyzing and predicting CBM production performance is critical in choosing the optimal completion methods and parameters. However, the conventional numerical simulation has challenges of complicated gridding issues and expensive computational costs. The huge amount of available production data that has been collected in the field site opens up a new opportunity to develop data-driven approaches in predicting the production rate. Here, we proposed a novel physics-constrained data-driven workflow to effectively forecast the CBM productivity based on a Gated Recurrent Unit (GRU) and Multi-Layer Perceptron (MLP) combined neural network (GRU-MLP model). The model architecture is optimized by the multiobjective algorithm: nondominated sorting genetic algorithm Ⅱ (NSGA Ⅱ). The proposed framework was used to predict synthetic cases with various fracture-network-complexities and two multistage-fractured wells in field sites located at Qinshui basin and Ordos basin, China. The results indicated that the proposed GRU-MLP combined neural network was able to accurately and stably predict the production performance of multi-fractured horizontal CBM wells in a fast manner. Compared with Simple Recurrent Neural Network (RNN), Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM), the proposed GRU-MLP had the highest accuracy and stability especially for gas production in late-time. Consequently, a physics-constrained data-driven approach performed better than a pure data-driven method. Moreover, the optimum GRU-MLP model architecture was a group of optimized solutions, rather than a single solution. Engineers can evaluate the tradeoffs within this set according to the field-site requirements. This study provides a novel machine learning approach based on a GRU-MLP combined neural network model to estimate production performances in CBM wells. The method is simple and gridless, but is capable of predicting the productivity in a computational cost-effective way. The key findings of this work are expected to provide a theoretical guidance for the intelligent development in oil and gas industry.