Comparison of Pre-Trained Word Vectors for Arabic Text Classification Using Deep Learning Approach

A study of the performance of embedding methods for Arabic short-text sentiment analysis using deep learning approaches

This article explores the suitability of a long short-term memory recurrent neural network (LSTM-RNN) and artificial intelligence (AI) method for low-flow time series forecasting. The long short-term memory works on the sequential framework which considers all of the predecessor data. This forecasting method used daily discharged data collected from the Basantapur gauging station located on the Mahanadi River basin, India. Different metrics [root-mean-square error (RMSE), Nash–Sutcliffe efficiency (ENS), correlation coefficient (R) and mean absolute error] were selected to assess the performance of the model. Additionally, recurrent neural network (RNN) model is also used to compare the adaptability of LSTM-RNN over RNN and naive method. The results conclude that the LSTM-RNN model (R = 0.943, ENS = 0.878, RMSE = 0.487) outperformed RNN model (R = 0.935, ENS = 0.843, RMSE = 0.516) and naive method (R = 0.866, ENS = 0.704, RMSE = 0.793). The finding of this research concludes that LSTM-RNN can be used as new reliable AI technique for low-flow forecasting.

Remaining useful life (RUL) prognosis is one of the most important techniques in concrete structure health management. This technique evaluates the concrete structure strength through determining the advent of failure, which is very helpful to reduce maintenance costs and extend structure life. Degradation information with the capability of reflecting structure health can be considered as a principal factor to achieve better prognosis performance. In traditional data-driven RUL prognosis, there are drawbacks in which features are manually extracted and threshold is defined to mark the specimen’s breakdown. To overcome these limitations, this paper presents an innovative SAE-DNN structure capable of automatic health indicator (HI) construction from raw signals. HI curves constructed by SAE-DNN have much better fitness metrics than HI curves constructed from statistical parameters such as RMS, Kurtosis, Sknewness, etc. In the next stage, HI curves constructed from training degradation data are then used to train a long short-term memory recurrent neural network (LSTM-RNN). The LSTM-RNN is utilized as a RUL predictor since its special gates allow it to learn long-term dependencies even when the training data is limited. Model construction, verification, and comparison are performed on experimental reinforced concrete (RC) beam data. Experimental results indicates that LSTM-RNN generally estimates more accurate RULs of concrete beams than GRU-RNN and simple RNN with the average prediction error cycles was less than half compared to those of the simple RNN.

Remaining useful life prediction of PEMFC based on long short-term memory recurrent neural networks

Abstract: The rapid expansion of digital transactions has intensified the need for real-time fraud detection systems capable of identifying anomalous patterns with high accuracy. Traditional rule-based and classical machine learning approaches often fall short in handling the complex, high-dimensional, and sequential nature of transaction data. This study investigates the application of deep learning models—specifically Convolutional Neural Networks (CNNs) and Long Short-Term Memory Recurrent Neural Networks (LSTM-RNNs)—to detect fraudulent behavior in real-time transaction networks. Using the IEEE-CIS Fraud Detection dataset, comprising over 500,000 records, the models were evaluated based on accuracy, precision, recall, and F1-score. Data preprocessing included normalization, categorical encoding, and sequence transformation. The LSTM-RNN model demonstrated superior performance, achieving 96% accuracy and an F1-score of 0.945, significantly outperforming baseline models such as logistic regression and random forest. Regression analysis further validated the stability of predictions across various transaction features. The results highlight the effectiveness of sequence-aware deep learning models in capturing subtle fraud indicators and reducing false negatives. The study underscores the importance of combining high model accuracy with low latency and interpretability for deployment in financial security systems. Keywords: deep learning, fraud detection, real-time transactions, LSTM-RNN, CNN, financial security, anomaly detection, predictive modeling, sequential data, IEEE-CIS dataset

Non-intrusive load monitoring system (NILMS) have become a hot research topic. However, the majority of existing methods for NILM are unreliability, especially, the generalization of model and the accuracy of identification in the multi-load with the similar load energy consumption values. This paper proposes a reliable algorithm architecture fused deep learning and transfer learning to improve the reliability of load combination identification model. In the method, a load event outlier detection way based on adaptive sliding-window (ASW) is presented to make better use of experimental data; a feature extractor based on deep convolutional neural networks (DNN) is used to complete the high-quality (Hi-Q) representation of on-line multi-load features; the long short-term memory recurrent neural network (LSTM-RNN) with transfer learning is used to address the generalization of model and the inaccuracy of load combination identification problems. The three data sets from the different domain are employed for training, testing and verifying of the proposed model. The experimental results show that the proposed method can significantly improve the performance of model in NILMS.

Lithium battery packs are the main driving energy source for electric vehicles. A battery pack equalization charging solution using a constant current source for variable rate charging is presented in this paper. The charging system consists of a main constant current source and independent auxiliary constant current sources. Auxiliary constant current sources are controlled by the battery management system (BMS), which can change the current rate of the corresponding single battery, and achieve full charging of each single cell in the series battery pack. At the same time, the state of charge (SOC) is regarded as time series data to establish a long short-term memory recurrent neural network (LSTM-RNN) model, and it is possible to obtain the single battery with lower capacity, so that the charging efficiency and battery pack consistency can be improved. The experimental results show that the open circuit voltage difference between the single cells is less than 50 mV after the charging of 20 strings of lithium battery packs by using this method, which achieve the purpose of equalization charging.

Automatic recognition of frustration, by analysing facial and vocal expressions, can help user experience designers to identify interaction obstacles. To encourage the development of automated systems such as these, we present a novel audiovisual database: the Multimodal Game Frustration Database (MGFD), consisting of ca. 5 hours of audiovisual data, collected from 67 Chinese students speaking in English. For data collection, we developed ‘Crazy Trophy’, a Wizard-of-Oz voice activated web-game designed with a variety of usability problems and aimed to induce increasing amounts of frustration. We also present a baseline for binary multimodal frustration classification (frustration vs no-frustration). For this, we compare the performance of a conventional method, Support Vector Machine classifier, and a state-of-the-art method utilising Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN), extracting both audio (Mel-frequency Cepstral Coefficients) and video (facial action units) features. Using LSTM-RNN and a feature-based multi-model fusion strategy, the best result acheived for the baseline was 60.3 % UAR. To enable further research in this area, the game (‘Crazy Trophy’), the database (MGFD), and the partitioning considered in the presented baseline, are made accessible to the research community.

With the evolution of network threat, identifying attack from both external and internal is getting more and more difficult. To detect both known and unknown malicious attacks, several machine learning algorithms are utilized. However, these algorithms have still some limitations such as high false positive and false alarm rate. To overcome above challenge, we propose a threat analysis method combined with kernel principal component analysis (PCA) and long short-term memory recurrent neural network (LSTM-RNN). To achieve high accurate detection rate, data preprocessing, feature extraction, attack detection is seamlessly integrated into an end-to-end detection method. To assess the method, the well-known NSL-KDD dataset has been used. Experimental results show that the proposed threat analysis method greatly outperforms several attack detection methods that use SVM, neural network or Bayesian methods.

In this paper, we propose a Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) and multiple kernel learning (MKL) based multi-modal affect recognition scheme (LSTM-MKL). It takes the LSTM-RNN advantage to model the long range dependencies between successive observations, and uses the MKL power to model the non-linear correlations between the inputs and outputs. For each of the affect dimensions (arousal, valence, expectancy, and power), two LSTM-RNN models are trained, one for each modality. In the recognition phase, the audio and visual features are input to the corresponding learned LSTM models, which in turn produce initial estimates of the affect dimensions. The LSTM outputs are further input into a multi-kernel support vector regression (MK-SVR) for the final recognition. Experimental results carried out on the AVEC2012 database, show that compared to the traditional SVR-LLR (Support Vector Machine - local linear regression) or MK-SVR fusion scheme, the proposed LSTM-MKL fusion scheme obtains higher recognition results, with an correlation coefficient (COR) of 0.354, compared to a COR of 0.124 for SVR-LLR, and 0.168 for MK-SVR, respectively.

Intrusion Detection System (IDS) is built to detect both known and unknown malicious attacks. Several machine learning algorithms are used widely in IDS such as neural network, SVM, KNN etc. However, these algorithms have still some limitations such as high false positive and false alarm rate. To overcome above challenge, we propose a novel effective attack detection method based on kernel principal component analysis (PCA) and long short-term memory recurrent neural network (LSTM-RNN). To achieve high accurate detection rate, data preprocessing, feature extraction, attack detection is seamlessly integrated into an end-to-end detection method. To assess the method, the well-known NSL-KDD dataset has been used. The results of experimental show that the proposed attack detection strategy greatly outperforms several attack detection strategy that use SVM, neural network or Bayesian methods.

Sentiment analysis is one of the major fields in natural language processing whose main task is to extract sentiments, opinions, attitudes, and emotions from a subjective text. And for its importance in decision making and in people's trust with reviews on web sites, there are many academic researches to address sentiment analysis problems. Deep Learning (DL) is a powerful Machine Learning (ML) technique that has emerged with its ability of feature representation and differentiating data, leading to state-of-the-art prediction results. In recent years, DL has been widely used in sentiment analysis, however, there is scarce in its implementation in the Arabic language field. Most of the previous researches address other languages like English. The proposed model tackles Arabic Sentiment Analysis (ASA) by using a DL approach. ASA is a challenging field where Arabic language has a rich morphological structure more than other languages. In this work, Long Short-Term Memory (LSTM) as a deep neural network has been used for training the model combined with word embedding as a first hidden layer for features extracting. The results show an accuracy of about 82% is achievable using DL method.

With more and more frequency, users communicate with each other on social media. Many users start on Twitter or Facebook to find friends who have the same hobby. Our study proposes a method to estimate the users’ interests (hobby) based on tweets on Twitter. One tweet does not, in and of itself, contain a lot of information, and some tweets are not related to the user’s hobby. Therefore, we propose a reliable hobby estimation method by extracting features from multiple, sequential tweets. The proposed method uses Recurrent Neural Networks (RNN) which can accommodate time-series information. We also used a Convolutional Neural Networks (CNN) which can treat contextual information. We used an averaged vector of word distributed representation as a feature. Using the proposed method based on Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN), we obtained a 23.72% improvement as compared with a baseline method using a Random Forest (RF) regression as a machine learning algorithm.

The present study aimed to describe and evaluate the current assessment practices prevalent in the different translation courses offered at the College of Languages and Translation (COLT). A sample of specialized translation final exams in 18 translation subject areas was collected. Each final exam was analyzed in terms of the following: (1) # of English and Arabic source texts included on each final exam for each course (2) readability and difficulty level of texts included in the translation exams, (3) # of exams with a terminology subtest, (4) English and Arabic text length in words, (5) reliability, validity and discriminating power of final exams. Data analysis showed that 50% of the exams included one English text, 32% included 2 texts and 18% included 3 texts. Arabic texts were included in 73% of the exams. However, 59% of the exams included one Arabic text, 9% included two texts and 5% included 3 texts. In Addition to English and Arabic texts, 56% of the exams included a vocabulary subtest. 41% do not have any Arabic texts. The English text length ranged between 66-430 words with a median length of 181 words. The Arabic text length ranged between 26-180 with a median length of 97. The typical Flesch Reading Ease of English texts was 40 and the typical Flesch-Kincaid Grade level score was 11. There were no significant differences among the different college levels nor different subject areas in text length or text difficulty level. Translation exams currently used at COLT lack validity, reliability, and discriminating power. Some reasons for lack of reliability and validity are given. Students’ views on translation exams are also reported. A model for more valid, reliable, and discriminating translation exams is given with students views of it as well.

Since Web 2.0 and the freedom to share information, perspectives, and opinions on global events, services, goods, etc., most of the content on social media platforms comes from users. Social media data includes user sentiment-related subjects. Due to its complexity, ambiguity, dialects, shortage of resources, and morphological diversity, little work has been done on Arabic sentiment analysis. This study introduces a novel word embedding model, AraWord2Vec, based on Wor2Vec trained on Wikipedia and the Sentiment Benchmark dataset. We also examined how Wor2Vec embedding models improved classification tasks. Second, a novel CM_BiLSTM deep learning model called Convolutional Max Pooling and Bidirectional LSTM is presented to enhance Arabic sentiment analysis. Our experimental findings revealed the performance of all models on diverse datasets with an accuracy of 98.47% in binary classification; in multi-classes classification, we obtained a high performance of 98.92%.