Arabic Sentiment Analysis (ASA) Using Deep Learning Approach

Sentiment analysis also referred to as opinion mining, plays a significant role in automating the identification of negative, positive, or neutral sentiments expressed in textual data. The proliferation of social networks, review sites, and blogs has rendered these platforms valuable resources for mining opinions. Sentiment analysis finds applications in various domains and languages, including English and Arabic. However, Arabic presents unique challenges due to its complex morphology characterized by inflectional and derivation patterns. To effectively analyze sentiment in Arabic text, sentiment analysis techniques must account for this intricacy. This paper proposes a model designed using the transformer model and deep learning (DL) techniques. The word embedding is represented by Transformer-based Model for Arabic Language Understanding (ArabBert), and then passed to the AraBERT model. The output of AraBERT is subsequently fed into a Long Short-Term Memory (LSTM) model, followed by feedforward neural networks and an output layer. AraBERT is used to capture rich contextual information and LSTM to enhance sequence modeling and retain long-term dependencies within the text data. We compared the proposed model with machine learning (ML) algorithms and DL algorithms, as well as different vectorization techniques: term frequency-inverse document frequency (TF-IDF), ArabBert, Continuous Bag-of-Words (CBOW), and skipGrams using four Arabic benchmark datasets. Through extensive experimentation and evaluation of Arabic sentiment analysis datasets, we showcase the effectiveness of our approach. The results underscore significant improvements in sentiment analysis accuracy, highlighting the potential of leveraging transformer models for Arabic Sentiment Analysis. The outcomes of this research contribute to advancing Arabic sentiment analysis, enabling more accurate and reliable sentiment analysis in Arabic text. The findings reveal that the proposed framework exhibits exceptional performance in sentiment classification, achieving an impressive accuracy rate of over 97%.

With the development and emergence of the concept of the Web, as well as the advancement of social networks, forums, blogs and E-commerce a growing number of Arabs express their views and opinions using their native language on those platforms where we find a wealth of information that could be exploited. This information, whose size is expanding and which describes opinions, has become the subject of several research fields particularly: “Sentiment Analysis” or “Opinion Mining”. Sentiment Analysis considered a subfield of natural language processing (NLP), information retrieval and text mining. Currently, sentiment analysis is commonly used for English, while it is in its infancy in the Arabic language. The most important and critical step in feeling analysis is to select an appropriate technique for classifying feelings. In this paper, we will be reviewing the published work on the application of Deep Learning (DL) techniques to the problems of sentiment analysis. We focus on the Arabic language because of its importance, the large number of its speakers, the scarcity of Arabic resources on it and the challenges related to its work. We explain, categorize, summarize and compare the techniques offered in this area.

The rapid growth of social networks allows users to express their views on various topics, significantly impacting Sentiment Analysis (SA) in Natural Language Processing (NLP). SA extracts valuable insights and aids decision-making based on public sentiment in various domains, particularly education. In other words, Arabic Sentiment Analysis (ASA) improves educational outcomes by assessing students’ learning progress and monitoring their performance. SA faces challenges with the Arabic language due to its dialects and complex morphology. Deep Learning (DL) models, especially Convolutional Neural Networks (CNNs), have advanced SA by effectively capturing relevant features, outperforming traditional Machine Learning (ML) algorithms. This paper assesses FastText word embedding with a CNN-based model for Arabic Sentiment Analysis (ASA). Our results show that the CNN model with FastText achieves strong performance in both datasets, with higher classification accuracy in the first dataset compared to the second. Which confirms the effectiveness of this proposed model to evaluate website and MOOC content by measuring learner satisfaction through forum interactions and module evaluations.

Deep learning has been very successful in the past decades, especially in Computer Vision and Speech Recognition fields. It has been also used successfully in the Natural Language Processing field because of the availability of an enormous amount of online text data, such as social networks and reviews websites, which have gained a lot of popularity and success in the past years. Sentiment Analysis is one of the hottest applications of Natural Language Processing (NLP). Many researchers have done excellent work on Sentiment Analysis for English language. However, the amount of work on Sentiment Analysis for Arabic language is, in comparison, very limited due to the complexity of the Arabic language's morphology and orthography. Unlike the English language, Arabic has many different dialects which makes Sentiment Analysis for Arabic more difficult and challenging, especially when working on data collected from social networks, which is known to be unstructured and noisy. Most of the work that has been done on Sentiment Analysis of Arabic language, focused on using lexicons and basic machine learning models. In addition, most of the work has been done on small datasets because of the limited number of the available annotated datasets for Arabic language. This paper proposes state-of-the-art research for Sentiment Analysis of Arabic microblogging using new techniques, and a sophisticated Arabic text data preprocessing.

Historical information in the Canadian Maritime Judiciary increases with time because of the need to archive data to be utilized in case references and for later application when determining verdicts for similar cases. However, such data are typically stored in multiple systems, making its reachability technical. Utilizing technologies like deep learning and sentiment analysis provides chances to facilitate faster access to court records. Such practice enhances impartial verdicts, minimizing workloads for court employees, and decreases the time used in legal proceedings for claims during maritime contracts such as shipping disputes between parties. This paper seeks to develop a sentiment analysis framework that uses deep learning, distributed learning, and machine learning to improve access to statutes, laws, and cases used by maritime judges in making judgments to back their claims. The suggested approach uses deep learning models, including convolutional neural networks (CNNs), deep neural networks, long short-term memory (LSTM), and recurrent neural networks. It extracts court records having crucial sentiments or statements for maritime court verdicts. The suggested approach has been used successfully during sentiment analysis by emphasizing feature selection from a legal repository. The LSTM + CNN model has shown promising results in obtaining sentiments and records from multiple devices and sufficiently proposing practical guidance to judicial personnel regarding the regulations applicable to various situations.

In recent years, domain generation algorithms (DGAs) are the foundational mechanisms for many malware families. Mainly, due to the fact that DGA can generate immense number of pseudo random domain names to associate to a command and control (C2) infrastructures. This paper focuses on to detect and classify the pseudo random domain names without relying on the feature engineering or any other linguistic, contextual or semantics and statistical information by adopting deep learning approaches. A deep learning approach is a complex model of traditional machine learning mechanism that has received renewed interest by solving the long-standing tasks in artificial intelligence (AI) related to the field of natural language processing, image recognition, speech processing and many others. They have immense capability to extract optimal feature representations by taking input as in the form of raw input texts. To leverage this and to transfer the performance enhancement in aforementioned areas towards characterize, detect and classify the DGA generated domain names to a specific malware family, this paper adopts deep learning mechanisms with a known one million benign domain names from Alexa, OpenDNS and a corpus of malicious domain names generated from 17 DGA malware families in real time for training in character and bigram level and a trained model has been evaluated on the OSNIT data set in real-time. Specifically, to understand the effectiveness of various deep learning mechanisms, we used recurrent neural network (RNN), identity-recurrent neural network (I-RNN), long short-term memory (LSTM), convolution neural network (CNN), and convolutional neural network-long short-term memory (CNN-LSTM) architectures. Additionally, to find out an optimal architecture, experiments are done with various configurations of network parameters and network structures. All experiments run up to 1000 epochs with a learning rate set in the range [0.01-0.5]. Overall, deep learning approaches, particularly family of recurrent neural network and a hybrid network (where the first layer is CNN and a subsequent layer is LSTM) have showed significant performance with a highest detection rate 0.9945 and 0.9879 respectively. The main reason is deep learning approaches have inherent mechanisms to capture hierarchical feature extraction and long range-dependencies in sequence inputs.

Sentiment analysis is a subfield of Natural Language Processing (NLP) which tries to process a text to extract opinions or attitudes towards topics or entities. Recently, the use of deep learning methods for sentiment analysis has received noticeable attention from researchers. Generally, different deep learning methods have shown superb performance in sentiment analysis problem. However, deep learning models are different in nature and have different strengths and limitations. For example, convolutional neural networks are useful for extracting local structures from data, while recurrent models are able to learn order dependence in sequential data. In order to combine the advantages of different deep models, in this paper we have proposed a novel approach for aspect-based sentiment analysis which utilizes deep ensemble learning. In the proposed method, we first build four deep learning models, namely CNN, LSTM, BiLSTM and GRU. Then the outputs of these models are combined using stacking ensemble approach where we have used logistic regression as meta-learner. The results of applying the proposed method on the real datasets show that our method has increased the accuracy of aspect-based prediction by 5% to 20% compared to the basic deep learning methods.

In the field of natural language processing, word embeddings and bag of words are two objects used to represent initial features for many machine learning models. Bag of words is known to be widely used and suitable for traditional models such as SVM, logistic regression, random forest. Meanwhile, word embeddings are used suitable for deep learning models that designed based on Convolutional neural network (CNN) and Long short-term memory (LSTM). Many sentiment analysis studies have effectively used word embeddings. However, they have ignored the role of bag of words in their proposed model, this paper proposes a CNN model that simultaneously exploits the effectiveness of word embeddings and bag of words for vietnamese sentiment analysis. In the experiment, we used 4009 actual textual reviews in the data domain of mobile phone products, the experimental results have demonstrated the ability of the proposed model as well as the role of bag of words.KeywordsWord embeddingBag of wordsVietnamese sentiment analysisDeep learningConvolutional neural networkLSTM

<span lang="EN-US">In the realm of natural language processing (NLP), a diverse array of language models has emerged, catering to a wide spectrum of tasks, ranging from speaker recognition and auto-correction to sentiment analysis and stock prediction. The significance of language models in enabling the execution of these NLP tasks cannot be overstated. This study proposes an approach to enhance accuracy by leveraging a hybrid language model, combining the strengths of long short-term memory (LSTM) and gated recurrent unit (GRU). LSTM excels in preserving long-term dependencies in data, while GRU's simpler gating mechanism expedites the training process. The research endeavors to evaluate four variations of this hybrid model: LSTM, GRU, <a name="_Hlk158992630"></a>bidirectional long short-term memory (Bi-LSTM), and a combination of LSTM with GRU. These models are subjected to rigorous testing on two distinct datasets: one focused on IBM stock price prediction, and the other on Jigsaw toxic comment classification (sentiment analysis). This work represents a significant stride towards democratizing NLP capabilities, ensuring that even in resource-constrained settings, NLP models can exhibit improved performance. The anticipated implications of these findings span a wide spectrum of real-world applications and hold the potential to stimulate further research in the field of NLP. </span>

Sentiment analysis of product reviews plays a crucial role in understanding consumer feedback, improving customer experience and making informed business decisions. This paper explores the application of machine learning and deep learning algorithms to effectively classify and analyse the sentiment of product reviews. Traditional machine learning techniques, such as Naïve Bayes, Support Vector Machines (SVM) and Random Forests are employed for sentiment classification based on manually engineered features. Simultaneously, deep learning approaches like Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks are leveraged to automatically learn complex representations from raw text data. The study compares the performance of these methods in terms of accuracy, precision, recall and F1-score. Additionally, pre-trained language models such as BERT are incorporated to enhance contextual understanding. Experimental results demonstrate that deep learning models particularly LSTM and BERT, outperform traditional machine learning techniques in capturing sentiments. This analysis provides valuable insights into the effectiveness of different algorithms in sentiment analysis tasks, paving the way for more advanced applications in natural language processing and customer sentiment evaluation.

Social media are considered an excellent source of information and can provide opinions, thoughts and insights toward various important topics. Sentiment analysis becomes a hot topic in research due to its importance in making decisions based on opinions derived from analyzing the user’s contents on social media. Although the Arabic language is one of the widely spoken languages used for content sharing across the social media, the sentiment analysis on Arabic contents is limited due to several challenges including the morphological structures of the language, the varieties of dialects and the lack of the appropriate corpora. Hence, the rapid increase in research in Arabic sentiment analysis is grown slowly in contrast to other languages such as English. The contribution of this paper is twofold: First, we introduce a corpus of forty thousand labeled Arabic tweets spanning several topics. Second, we present three deep learning models, namely CNN, LSTM and RCNN, for Arabic sentiment analysis. With the help of word embedding, we validate the performance of the three models on the proposed corpus. The experimental results indicate that LSTM with an average accuracy of 81.31% outperforms CNN and RCNN. Also, applying data augmentation on the corpus increases LSTM accuracy by 8.3%.

Morphologically complex languages such as Arabic pose several challenges in Natural Language Processing (NLP) due to their complexity and token sparsity. Most techniques approach the problem by transforming the words of the language from their sparse surface form representation to a less sparse form before processing. The transformation usually takes the form of a morphological analysis or a morphological segmentation. This dissertation addresses two tasks in Arabic NLP: Statistical Machine Translation(SMT) and Sentiment Analysis. To improve English-Arabic SMT, we apply segmentation on Arabic to decrease token sparsity and enhance the correspondence between tokens of the English and Arabic language. However, due to this segmentation, the translation system is limited to extracting features based on morphemes (partial words) and only outputting morphemes during decoding. Such a system lacks knowledge of the original form of the words. We further improve translation from English to Arabic by combining both segmented and desegmented views of the target language. The system can benefit from segmentation's sparsity reduction and verifies its generation of correct words. We present a language-independent technique to desegmentation that approaches the problem as a string transduction task. We propose a new algorithm that desegments the decoder's search space encoded as a lattice, thus allowing the system to use features from the desegmented view of the search space. We extend the phrase-based statistical machine translation system to allow desegmentation during the decoding process on the fly. In addition, we conduct an experimental study to verify what matters most in morphologically segmented SMT models. Our second task is sentiment analysis, where we resort to Arabic lemmatization to improve sentiment analysis of Arabic tweets and blog posts. We explore translation in the opposite direction, from Arabic into English in order to evaluate the loss of sentiment predictability when Arabic social media posts are translated to English, manually or using an SMT system. We use state-of-the-art Arabic and English sentiment Analysis systems and develop automatically generated Arabic lexicons from lemmatized tweets to improve this task.

In the era of big data, natural language processing becomes an important research discipline, owing to the immense quantity of text documents and the progresses in machine learning. Natural language processing has been succesfully employed in many different areas, including machine translation, search engines, virtual assistants, spam filtering, question answering and sentiment analysis. Recent studies in the field of natural language processing indicate that word embedding based representation, in which words have been represented in dense spaces through fixed length vectors, can yield promising results. In this study, we evaluate the predictive performance of 36 word embedding based representation obtained by three word embedding methods (i.e., word2vec, fastText and DOC2vec), two basic weighting functions (i.e., inverse document frequency and smooth inverse document frequency) and three vector pooling schemes (namely, weighted sum, center based approach and delta rule). Experimental analysis indicates that word2vec based representation in conjunction with inverse document frequency based weighting and center based pooling, yields promising results for sentiment analysis in Turkish.

Natural Language Processing (NLP) is a valuable technology and business topic as it helps turn data into useful information with the spread of digital information. Nevertheless, there are some difficulties in its use, including the language’s complexity and the data quality. To address these challenges, in this study, the researchers first performed a series of ablation experiments on 14 models derived from various variations in Deep Learning (DL) methods, including A Lite BERT (ALBERT) together with Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), Max Pooling layer, and attention mechanism. Subsequently, they proposed an ALBERT-cascaded CNN hybrid model as an effective method to overcome the related challenges by evaluating the performance results obtained from these models. In the proposed model, a transformer architecture with parallel processing capability for both word and subword tokenization is used in addition to creating contextualized word embeddings. Local and global feature extraction was also performed using two 1-D CNN blocks before classification to improve the model performance. The model was optimized using an advanced hyperparameter optimization tool called OPTUNA. The findings of the experiment conducted with the proposed model were obtained based on Amazon Fashion 2023 data under 5-fold cross-validation conditions. The experimental results demonstrate that the proposed hybrid model exhibits good performance with average scores of 0.9308 (accuracy), 0.9296 (F1 score), 0.9412 (precision), 0.9182 (recall), and 0.9797 (AUC) in the validation dataset, and scores of 0.9313, 0.9305, 0.9414, 0.9199, and 0.9800 in the test dataset. In addition, comparisons of the model with models in studies using similar datasets support the experimental results and reveal that it can be used as a competitive approach for solving the problems encountered in the NLP field.

With the increase in E-Commerce businesses in the last decade,the sentiment analysis of product reviews has gained a lot of attention in linguistic research. In literature, the survey depicts the majority of the research done emphasizes on mere polarity identification of the reviews. The proposed system emphasized on classifying the sentiment polarity and the product aspect identification from the reviews. Proposed work experimented with traditional machine learning techniques as well as deep neural networks such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and Long Short Term Memory(LSTM) Networks. The proposed system gives a better understanding of these algorithms by comparing the outcomes. The Deep Learning approach in the proposed work successfully provides a mechanism which identifies the review polarity and intensity of the reviews and also analyses the short form words used by people in the reviews. The experimental results in this work, applied on amazon product dataset, shows that the LSTM model works the best for sentiment analysis and intensity of reviews with 93% accuracy. This research work also predicts polarity for short-form word reviews which is the common trend these days while writing the reviews.