Emotion Classification using 1D-CNN and RNN based On DEAP Dataset

Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mechanism concept can help RNNs remember and forget previous information. Representing the hidden layers of an RNN with more expressive operations (i.e., tensor products) helps it learn a more complex relationship between the current input and the previous hidden layer information. These ideas can generally improve RNN performances. In this paper, we proposed a novel RNN architecture that combine the concepts of gating mechanism and the tensor product into a single model. By combining these two concepts into a single RNN, our proposed models learn long-term dependencies by modeling with gating units and obtain more expressive and direct interaction between input and hidden layers using a tensor product on 3-dimensional array (tensor) weight parameters. We use Long Short Term Memory (LSTM) RNN and Gated Recurrent Unit (GRU) RNN and combine them with a tensor product inside their formulations. Our proposed RNNs, which are called a Long-Short Term Memory Recurrent Neural Tensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor Network (GRURNTN), are made by combining the LSTM and GRU RNN models with the tensor product. We conducted experiments with our proposed models on word-level and character-level language modeling tasks and revealed that our proposed models significantly improved their performance compared to our baseline models.

Quantifying Gross Primary Production (GPP) is fundamental for understanding terrestrial carbon dynamics, particularly in forests. The overarching question we address here is whether integrating remote sensing (RS) with deep learning (DL) methodologies can enhance the estimation of daily forest GPP on a European scale.The Eddy Covariance (EC) method, although widely used to infer ecosystem-scale estimates of GPP from in situ CO2 exchange measurements, suffers from limited global coverage. When EC data are not available, RS data are often employed to estimate GPP by establishing statistical relationships with in situ observations. Recently, Machine Learning (ML) strategies, particularly involving RS and meteorological inputs, have been used for estimating GPP continuously in space and time.However, the potential of DL techniques, particularly those exploiting the sequential characteristics of time series data (i.e. recurrent neural network architectures) in estimating daily forest GPP has not been comprehensively explored. This includes their performance during photosynthetic downregulation (e.g. during climate extremes such as droughts) and an in-depth examination of the importance of the utilised features.This study presents a comparative analysis of three recurrent neural network architectures—Recurrent Neural Networks (RNNs), Gated Recurrent Units (GRUs), and Long-Short Term Memory (LSTMs)—for daily forest GPP estimation across ICOS ecosystem stations. We assess their performance throughout the entire year, during the growing season, and under photosynthetic downregulation events. This assessment utilises RS inputs —optical data from Sentinel-2, Land Surface Temperature (LST) from MODIS, and radar data from Sentinel-1—, combined with potential radiation data. Furthermore, we analysed the importance of these features to provide insights into the complex interactions and dependencies when estimating GPP.Our results indicate that all three architectures yield similar accuracy for full period and growing season GPP estimations, with mean NRMSE values of 0.136 and 0.170, respectively. All models exhibit increased errors while estimating GPP during photosynthetic downregulation events and their performance varies notably, with LSTMs showing the best results (NRMSE=0.202), followed by RNNs (NRMSE=0.214), and GRUs exhibiting the least efficacy (NRMSE=0.276). Additionally, our study underscores the importance of potential radiation as a critical feature influencing the GPP response, with LST data proving particularly valuable when estimating GPP during photosynthetic downregulation events, more so than optical or radar data.These findings suggest that recurrent neural network architectures, especially LSTMs, are effective in daily GPP estimations, with slight performance decreases during climate extreme conditions. The study also reveals the efficacy of combining RS data with potential radiation for accurate forest GPP estimation, with LST data being especially crucial when estimating GPP during photosynthetic downregulation events.Our research paves the way for further exploration into recurrent models and innovative architectures, with an emphasis on overcoming the challenges in modelling climate-induced GPP extremes.

Emotions play a significant role in human-computer interaction and entertainment consumption behavior, which young adults commonly use. The main challenge is the lack of a publicly available dataset for young adults with emotion labeling of physiological signals. This article presents a multi-modal data set of Electrocardiograms (ECG) and Galvanic Skin Response (GSR) signals for the emotion classification of young adults. Signal acquisition was performed through Shimmer3 ECG and Shimmer3 GSR units wearable to the chest and palm of the participants. The ECG signals were acquired from 25 participants, while GSR signals were acquired from 12 participants while watching 21 emotional stimulus videos divided into three sessions. The data was self-annotated for seven emotions: happy, sad, fear, surprise, anger, disgust, and neutral. These emotional states were further self-annotated with five very low, low, moderate, high, and very high-intensity levels of felt emotion. The participant also annotated valence, arousal, and dominance scores through Google form against each provided stimulus. The base experimental results for classifying four classes of high valence high arousal (HVHA), high valence low arousal (HVLA), low valence high arousal (LVHA), and low valence low arousal for ECG data is reported with an accuracy of 69.66%. Our baseline method for the proposed dataset achieved 66.64% accuracy for the eight-class classification of categorical emotions. The significance of data lies in the more emotional classes and less intrusive sensors to mimic real-world applications. Young adult's affective data (YAAD) is made publicly available, and it is valuable for researchers to develop behavioral assessments based on physiological signals.

MOOCs (Massive Open Online Courses) are definitely one of the best approach to support the international agenda about inclusive and equitable education and lifelong learning opportunities for all (SDG4) [1]. A great deal universities and institutions offer valuable free courses to their numerous students and to people around the word through MOOC platforms. However, because of huge number of learners and data generated, learner’s behaviour in those platforms remain a kind of black box for learners themselves and for courses instructors who are supposed to tutor or monitor learners in the learning process. Therefore, learner do not receive sufficient support from instructors and from their peers, during the learning process [2]. This is one the main reasons that lead to high dropout, low completion and success rate observed in the MOOCs. Many research work have suggested descriptive, predictive and prescriptive models to address this issue, but most of these models focus on predicting dropout, completion and/or success, and do not generally pay enough attention to one of the key step (learner behaviour), that comes before, and can explain dropping out and failure. Our research aims to develop a deep learning model to predict learner behaviour (learner interactions) in the learning process, in order to equip learners and course instructors with insight understanding of the learner behaviour in the learning process. This specific paper will focus on analysing relevant algorithms to develop such model. For this analysis, we used data from UNESCO-IICBA (International Institute for Capacity Building in Africa) MOOC platform, designed for teacher training in Africa, and then we examine many types of features: geographical, social behavioural and learning behavioural features. Learner’s behaviour being a time series Big data, we built the predictive model using Deep Learning algorithms for better performance and accuracy (Thanks to the power of deep learning) compared to baseline Machine learning algorithms. Time series data is best handled by recurrent neural networks (RNN) [3], so, we choose RNN and implemented/tested the three main architectures of RNN: Simple RNNs, GRU (Gated Recurrent Unit) RNNs and LSTM (Long short-term memory) RNNs. The models were trained using L2 Regularization, based on the predictions results, we unexpectedly found model with simple RNNs produced the best performance and accuracy on the dataset used than the other RNN architectures. We had couple of observations, example: we saw a correlation between video viewing and quiz behaviour and the participation of the learner to the learning process. This observation could allow teachers to provide meaningful support and guidance to at risk or less active students. We also observed that, the shorter the video or the quiz, the more the viewer. We conclude that we could use learner video or quiz viewing behaviour to predict his behaviour concerning other MOOC contents. These results suggest the need of deeper research on educational video and educational quiz designing for MOOCs.

Deep learning (DL) has emerged as a powerful tool for intelligent cyberattack detection, especially Distributed Denial-of-Service (DDoS) in Software-Defined Networking (SDN), where rapid and accurate traffic classification is essential for ensuring security. This paper presents a comprehensive evaluation of six deep learning models (Multilayer Perceptron (MLP), one-dimensional Convolutional Neural Network (1D-CNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Recurrent Neural Network (RNN), and a proposed hybrid CNN-GRU model) for binary classification of network traffic into benign or attack classes. The experiments were conducted on an SDN traffic dataset initially exhibiting class imbalance. To address this, Synthetic Minority Over-sampling Technique (SMOTE) was applied, resulting in a balanced dataset of 24,500 samples (12,250 benign and 12,250 attacks). A robust preprocessing pipeline followed, including missing value verification (no missing values were found), feature normalization using StandardScaler to standardize numerical values, reshaping the data into 3D format to fit temporal models like CNN and GRU, and stratified train-test split (80% training, 20% testing) to maintain class distribution. The CNN-GRU model integrates a 1D convolutional layer for spatial pattern extraction and a GRU layer for temporal sequence learning, followed by dense layers with dropout regularization. The model was trained using the Adam optimizer with early stopping to prevent overfitting. Among all models, the CNN-GRU hybrid achieved perfect test performance, with 100% accuracy, 1.0000 precision, recall, and F1-score, and an ROC AUC of 1.0000. It also demonstrated exceptional generalization, achieving a mean cross-validation (CV) accuracy of 99.70% ± 0.09% and a mean AUC of 1.0000 ± 0.0000 across 5-fold stratified cross-validation. While individual models such as GRU, 1D-CNN, and LSTM also showed strong performance, the CNN-GRU hybrid consistently outperformed them in both accuracy and stability. These results validate the effectiveness of combining convolutional and recurrent architectures, augmented with data balancing via SMOTE, for highly accurate SDN-based intrusion detection.Supplementary InformationThe online version contains supplementary material available at 10.1038/s41598-025-13754-1.

This paper examines and evaluates the effectiveness of recurrent neural network (RNN) architectures for natural language knowledge. The paper starts by imparting historical data on recurrent neural networks and their software to various responsibilities. It then goes on to talk about diverse RNN architectures, such as lengthy quick-time period reminiscence (LSTM), Gated Recurrent Unit (GRU) and Bonior Recurrent Neural network (BONN). The paper then focuses on the efficacy of each RNN structure for textual content expertise obligations. A sequence of experiments is carried out to degree the overall performance of the one-of-a-kind fashions, the use of 3 preferred datasets-Wiki text, Squad and IMDB, as well as two newly created datasets. The paper analyses the empirical consequences of the various RNN architectures and finds that LSTM fashions are normally satisfactory performers, although BONN architectures can also be aggressive in some cases. The paper additionally looks at numerous techniques to enhance performance, which include pre-processing and hyper-parameter tuning. The paper concludes that, whilst RNN architectures are powerful for herbal language know-how responsibilities, the exact architecture and parameters can also need to be chosen cautiously, relying on the project and dataset to hand.

We explore the architecture of recurrent neural networks (RNNs) by studying the complexity of string sequences it is able to memorize. Symbolic sequences of different complexity are generated to simulate RNN training and study parameter configurations with a view to the network's capability of learning and inference. We compare Long Short-Term Memory (LSTM) networks and gated recurrent units (GRUs). We find that an increase in RNN depth does not necessarily result in better memorization capability when the training time is constrained. Our results also indicate that the learning rate and the number of units per layer are among the most important hyper-parameters to be tuned. Generally, GRUs outperform LSTM networks on low-complexity sequences while on high-complexity sequences LSTMs perform better.

Forecasting longevity is essential in multiple research and policy areas, including the pricing of life insurance contracts, the valuation of capital market solutions for longevity risk management, and pension policy. This paper empirically investigates the predictive performance of Recurrent Neural Networks (RNN) with Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM) architectures in jointly modeling and multivariate time series forecasting of age-specific mortality rates at all ages. We fine-tune the three hidden layers GRU and LSTM model’s hyperparameters for time series forecasting and compare the model’s forecasting accuracy with that produced by traditional Generalised Age-Period-Cohort (GAPC) stochastic mortality models. The empirical results suggest that the two RNN architectures generally outperform the GAPC models investigated in both the male and female populations, but the results are sensitive to the accuracy criteria. The empirical results also show that the RNN-GRU network slightly outperforms the RNN with an LSTM architecture and can produce mortality schedules that capture relatively well the dynamics of mortality rates across age and time. Further investigations considering other RNN architectures, calibration procedures, and sample datasets are necessary to confirm the superiority of RNN in forecasting longevity.

Prediction of dissolved oxygen in a fishery pond based on gated recurrent unit (GRU)

To quantitatively evaluate the performance of two types of recurrent neural networks (RNNs), long short-term memory (LSTM) and gated recurrent units (GRU), using Monte Carlo dropout (MCD) to predict pharmacokinetic (PK) parameters from dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data. DCE-MRI data for simulation studies were synthesized using the extended Tofts model and a population-averaged arterial input function (AIF). The ranges of PK parameters for training the RNNs were determined from data of patients with brain tumors. The effects of the number of training samples, number of hidden units, dropout rate (DR), and bolus arrival time delay and dispersion in AIF on the accuracy of the PK parameters were investigated, and the uncertainties for different DRs and peak signal-to-noise ratios (PSNRs) were quantified. For comparison, PK parameters were estimated using the nonlinear least-squares method. In the clinical studies, the PK parameter and uncertainty images were generated by applying the trained RNNs to DCE-MRI data. Compared with GRU, the computational cost for training the LSTM was significantly higher. The prediction accuracy of GRU decreased with decreasing numbers of training samples and hidden units, whereas the performance of LSTM remained stable. Despite an increased computational cost, MCD reduced the prediction error at low PSNR and improved the quality of PK parameter images. The simulation results recommended using a DR of 0.25-0.5 at low PSNR and ≤ 0.25 for other PSNRs. The clinical studies recommended using a DR of 0.25 and 0.5 for LSTM and GRU, respectively. MCD is effective in quantifying uncertainty in PK parameter prediction from DCE-MRI data and improves their performance, particularly at low PSNR; however, at the expense of increased computational cost. This study helps deepen our understanding of RNNs with MCD and select suitable hyperparameters for creating an RNN architecture for DCE-MRI studies.

Recurrent neural networks (RNN) are ubiquitous computing systems for sequences and multivariate time-series data. While several robust RNN architectures are known, it is unclear how to relate RNN initialization, architecture, and other hyperparameters with accuracy for a given task. In this work, we propose treating RNN as dynamical systems and correlating hyperparameters with accuracy through Lyapunov spectral analysis, a methodology designed explicitly for nonlinear dynamical systems. To address the fact that RNN features go beyond the existing Lyapunov spectral analysis, we propose to infer relevant features from the Lyapunov spectrum with an Autoencoder and an embedding of its Latent representation (AeLLE). Our studies of various RNN architectures show that AeLLE successfully correlates RNN Lyapunov spectrum with accuracy. Furthermore, the Latent representation learned by AeLLE is generalizable to novel inputs from the same task and is formed early in the process of RNN training. The latter property allows for predicting the accuracy to which RNN would converge when training is complete. We conclude that the representation of RNN through the Lyapunov spectrum, along with AeLLE, provides a novel method for the organization and interpretation of variants of RNN architectures.

In this paper, we propose the use of recurrent neural networks (RNNs) to develop an effective solution to two problems in electrocardiogram (ECG)-based biometrics: identification/classification and authentication. Different RNN architectures with various parameter settings were evaluated, including traditional, long short-term memory (LSTM), gated recurrent unit (GRU), unidirectional, and bidirectional networks. Unlike many existing methods, the RNN-based method does not require any feature extraction. The method was evaluated using two publicly available datasets: ECG-ID and MIT-BIH Arrhythmia (MITDB). For the identification problem, nearly 100% classification accuracy on the ECG-ID dataset was achieved, and similar results were observed for the MITDB dataset. For the authentication problem, an RNN was trained and the hidden state at the final time step was extracted to make a decision. We evaluated the effect of the training size on the equal error rate (EER), and showed that the EER drops from approximately 3.5% to nearly 0% as we increased the percentage of subjects used for training from approximately 15% to 80%.

Emotion classification using EEG (Electroencephalogram) brain signals has recently garnered significant interest due to its potential applications in fields such as mental health, human-computer interaction, and affective computing. The three prominent recurrent neural network (RNN) architectures GRU (Gated Recurrent Unit), LSTM (Long Short-Term Memory), and BiLSTM (Bidirectional Long Short-Term Memory) are thoroughly compared in this study with a focus on classifying emotions. The study examines how well these RNN variations capture temporal dependencies and patterns within EEG data to precisely discriminate between various emotional states. Based on classification accuracy and training effectiveness, these model's performance will be assessed. The results will reveal the most suitable architecture in terms of accuracy and robustness for EEG-based emotion classification. The findings provide valuable insights into choosing the appropriate RNN architecture for emotion classification tasks, advancing the understanding of AI's potential in interpreting human emotions through EEG signals.

Purpose: This study aims to predict the closing price of the EUR/JPY currency pair in the forex market using recurrent neural network (RNN) architectures, namely Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), with the incorporation of Bidirectional layers. Methods: The dataset comprises hourly price data obtained from Yahoo Finance and pre-processed accordingly. The data is divided into training and testing sets, and time series sequences are constructed for input into the models. The RNN, LSTM, and GRU models are trained using the Adam optimization algorithm with the mean squared error (MSE) loss metric. Results: Results indicate that the LSTM model, particularly when coupled with Bidirectional layers, exhibits superior predictive performance compared to the other models, as evidenced by lower MSE values. Conclusions: Therefore, the LSTM model with Bidirectional layers is the most effective in predicting the EUR/JPY currency pair's closing price in the forex market. These findings offer valuable insights for practitioners and researchers involved in financial market prediction and neural network modelling

In recent years, Deep Learning (DL) models, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTM), have been widely used for Human Activities Recognition (HAR). They have achieved considerable performance improvements over classical Machine Learning (ML) approaches due to their excellent feature representation capabilities. Simple human activities are performed in sequential order with no overlaps or concurrent actions. The paucity of labeled training activity samples, high computational cost, and system resource requirements of deep learning architectures compared to lightweight models are some of the research challenges confronting the HAR community. We framed the lightweight DL-based CNNs, RNNs, and LSTM model for HAR to tackle these research challenges. The RNNs and LSTM have single layers with softmax activation functions, whereas the one-dimensional CNNs have two convolutional and single max-pooling layers. To evaluate the performance of our models, we used the publicly available benchmark WISDM experimental dataset, which includes the reading of six activities (walking, jogging, upstairs, downstairs, sitting, and standing) performed by 36 participants using a single accelerometer sensor. The experimental result illustrates the efficacy of our models in activity recognition, demonstrating that they attain higher accuracy while being computationally efficient.