Applications of Deep Learning in Arabic Sentiment Analysis: Research Perspective

Graph theory and the fields of natural language processing and information retrieval are well-studied disciplines. Traditionally, these areas have been perceived as distinct, with different algorithms, different applications and different potential end-users. However, recent research has shown that these disciplines are intimately connected, with a large variety of natural language processing and information retrieval applications finding efficient solutions within graph-theoretical frameworks. This book extensively covers the use of graph-based algorithms for natural language processing and information retrieval. It brings together topics as diverse as lexical semantics, text summarization, text mining, ontology construction, text classification and information retrieval, which are connected by the common underlying theme of the use of graph-theoretical methods for text and information processing tasks. Readers will come away with a firm understanding of the major methods and applications in natural language processing and information retrieval that rely on graph-based representations and algorithms.

Sentiment analysis is one of the major fields in natural language processing whose main task is to extract sentiments, opinions, attitudes, and emotions from a subjective text. And for its importance in decision making and in people's trust with reviews on web sites, there are many academic researches to address sentiment analysis problems. Deep Learning (DL) is a powerful Machine Learning (ML) technique that has emerged with its ability of feature representation and differentiating data, leading to state-of-the-art prediction results. In recent years, DL has been widely used in sentiment analysis, however, there is scarce in its implementation in the Arabic language field. Most of the previous researches address other languages like English. The proposed model tackles Arabic Sentiment Analysis (ASA) by using a DL approach. ASA is a challenging field where Arabic language has a rich morphological structure more than other languages. In this work, Long Short-Term Memory (LSTM) as a deep neural network has been used for training the model combined with word embedding as a first hidden layer for features extracting. The results show an accuracy of about 82% is achievable using DL method.

Opinion mining is considered as a subfield of natural language processing, information retrieval and text mining. Opinion mining is the process of extracting human thoughts and perceptions from unstructured texts, which with regard to the emergence of online social media and mass volume of users’ comments, has become to a useful, attractive and also challenging issue. There are varieties of researches with different trends and approaches in this area, but the lack of a comprehensive study to investigate them from all aspects is tangible. In this paper we represent a complete, multilateral and systematic review of opinion mining and sentiment analysis to classify available methods and compare their advantages and drawbacks, in order to have better understanding of available challenges and solutions to clarify the future direction. For this purpose, we present a proper framework of opinion mining accompanying with its steps and levels and then we completely monitor, classify, summarize and compare proposed techniques for aspect extraction, opinion classification, summary production and evaluation, based on the major validated scientific works. In order to have a better comparison, we also propose some factors in each category, which help to have a better understanding of advantages and disadvantages of different methods.

In the past decades, sentiment analysis has become one of the most active research areas in natural language processing, data mining, web mining, and information retrieval. The great demand in everyday life and the factor of novelty coupled with the availability of data from social networks have served as strong motivation for research on sentiment-analysis. A number of technical problems, most of which had not been attempted before, either in the NLP or linguistics communities have also generated strong research interests in academia. Sentiment analysis, also called opin-ion mining, is the field of study that analyzes people’s opinions, sentiments, apprais-als, attitudes, and emotions toward entities and their attributes expressed in written text. The entities can be products, services, organizations, individuals, events, issues, or topics. The field represents a large problem space. It improves not only the field of natural language processing but also management, political science, economics, and sociology because all these areas are related to the thoughts of consumers and public. User-generated content is full of opinions, because the main reason why people post messages on social media platforms is to express their views and opinions, and therefore sentiment analysis is at the centre of social media analysis. It turned out that user messages often contain plenty of sarcastic expressions and ambiguous words. Within one opinion both positive and negative sentiments can be present. This also applies to negative particles, which do not always indicate a negative tone. This article investigates four challenges faced by researchers while conducting sentiment analysis, namely: sarcasm, negation, word ambiguity, and multipolarity. These aspects significantly affect the accuracy of the results when we determine a sentiment. Modern approaches to solving the problem are also covered. These are mainly machine learning methods, such as convolutional neural networks (CNN), deep neural networks (DNN), long short-term memory (LTSM), recurrent neural network (RNN), support vector machines (SVM), etc.

The rapid growth of social networks allows users to express their views on various topics, significantly impacting Sentiment Analysis (SA) in Natural Language Processing (NLP). SA extracts valuable insights and aids decision-making based on public sentiment in various domains, particularly education. In other words, Arabic Sentiment Analysis (ASA) improves educational outcomes by assessing students’ learning progress and monitoring their performance. SA faces challenges with the Arabic language due to its dialects and complex morphology. Deep Learning (DL) models, especially Convolutional Neural Networks (CNNs), have advanced SA by effectively capturing relevant features, outperforming traditional Machine Learning (ML) algorithms. This paper assesses FastText word embedding with a CNN-based model for Arabic Sentiment Analysis (ASA). Our results show that the CNN model with FastText achieves strong performance in both datasets, with higher classification accuracy in the first dataset compared to the second. Which confirms the effectiveness of this proposed model to evaluate website and MOOC content by measuring learner satisfaction through forum interactions and module evaluations.

Morphologically complex languages such as Arabic pose several challenges in Natural Language Processing (NLP) due to their complexity and token sparsity. Most techniques approach the problem by transforming the words of the language from their sparse surface form representation to a less sparse form before processing. The transformation usually takes the form of a morphological analysis or a morphological segmentation. This dissertation addresses two tasks in Arabic NLP: Statistical Machine Translation(SMT) and Sentiment Analysis. To improve English-Arabic SMT, we apply segmentation on Arabic to decrease token sparsity and enhance the correspondence between tokens of the English and Arabic language. However, due to this segmentation, the translation system is limited to extracting features based on morphemes (partial words) and only outputting morphemes during decoding. Such a system lacks knowledge of the original form of the words. We further improve translation from English to Arabic by combining both segmented and desegmented views of the target language. The system can benefit from segmentation's sparsity reduction and verifies its generation of correct words. We present a language-independent technique to desegmentation that approaches the problem as a string transduction task. We propose a new algorithm that desegments the decoder's search space encoded as a lattice, thus allowing the system to use features from the desegmented view of the search space. We extend the phrase-based statistical machine translation system to allow desegmentation during the decoding process on the fly. In addition, we conduct an experimental study to verify what matters most in morphologically segmented SMT models. Our second task is sentiment analysis, where we resort to Arabic lemmatization to improve sentiment analysis of Arabic tweets and blog posts. We explore translation in the opposite direction, from Arabic into English in order to evaluate the loss of sentiment predictability when Arabic social media posts are translated to English, manually or using an SMT system. We use state-of-the-art Arabic and English sentiment Analysis systems and develop automatically generated Arabic lexicons from lemmatized tweets to improve this task.

Sentiment analysis also referred to as opinion mining, plays a significant role in automating the identification of negative, positive, or neutral sentiments expressed in textual data. The proliferation of social networks, review sites, and blogs has rendered these platforms valuable resources for mining opinions. Sentiment analysis finds applications in various domains and languages, including English and Arabic. However, Arabic presents unique challenges due to its complex morphology characterized by inflectional and derivation patterns. To effectively analyze sentiment in Arabic text, sentiment analysis techniques must account for this intricacy. This paper proposes a model designed using the transformer model and deep learning (DL) techniques. The word embedding is represented by Transformer-based Model for Arabic Language Understanding (ArabBert), and then passed to the AraBERT model. The output of AraBERT is subsequently fed into a Long Short-Term Memory (LSTM) model, followed by feedforward neural networks and an output layer. AraBERT is used to capture rich contextual information and LSTM to enhance sequence modeling and retain long-term dependencies within the text data. We compared the proposed model with machine learning (ML) algorithms and DL algorithms, as well as different vectorization techniques: term frequency-inverse document frequency (TF-IDF), ArabBert, Continuous Bag-of-Words (CBOW), and skipGrams using four Arabic benchmark datasets. Through extensive experimentation and evaluation of Arabic sentiment analysis datasets, we showcase the effectiveness of our approach. The results underscore significant improvements in sentiment analysis accuracy, highlighting the potential of leveraging transformer models for Arabic Sentiment Analysis. The outcomes of this research contribute to advancing Arabic sentiment analysis, enabling more accurate and reliable sentiment analysis in Arabic text. The findings reveal that the proposed framework exhibits exceptional performance in sentiment classification, achieving an impressive accuracy rate of over 97%.

In recent years, the ever-increasing quantities of entities in large knowledge bases on the Web, such as DBpedia, Freebase and YAGO, pose new challenges but at the same time open up new opportunities for intelligent information access. These knowledge bases (KBs) have become valuable resources in many research areas, such as natural language processing (NLP) and information retrieval (IR). Recently, almost every major commercial Web search engine has incorporated entities into their search process, including Google’s Knowledge Graph, Yahoo!’s Web of Objects and Microsoft’s Satori Graph/Bing Snapshots. The goal is to bridge the semantic gap between natural language text and formalized knowledge. Within the context of globalization, multilingual and cross-lingual access to information has emerged as an issue of major interest. Nowadays, more and more people from different countries are connecting to the Internet, in particular the Web, and many users can understand more than one language. While the diversity of languages on the Web has been growing, for most people there is still very little content in their native language. As a consequence of the ability to understand more than one language, users are also interested in Web content in other languages than their mother tongue. There is an impending need for technologies that can help in overcoming the language barrier for multilingual and cross-lingual information access. In this thesis, we face the overall research question of how to allow for semantic-aware and cross-lingual processing of Web documents and user queries by leveraging knowledge bases. With the goal of addressing this complex problem, we provide the following solutions: (1) semantic annotation for addressing the semantic gap between Web documents and knowledge; (2) semantic search for coping with the semantic gap between keyword queries and knowledge; (3) the exploitation of cross-lingual semantics for overcoming the language barrier between natural language expressions (i.e., keyword queries and Web documents) and knowledge for enabling cross-lingual semantic annotation and search. We evaluated these solutions and the results showed advances beyond the state-of-the-art. In addition, we implemented a framework of cross-lingual semantic annotation and search, which has been widely used for cross-lingual processing of media content in the context of our research projects.

The plethora of online content has paved the way for the development of sophisticated and advanced Natural Language Processing (NLP) and Information Retrieval (IR) tools. However, such tools are only available for English and other high-resource languages while being unavailable for low-resource languages such as Urdu. In this regard, generally, cross-lingual transfer learning techniques are adopted to utilize tools developed for the English language for low resource languages. In this paper, we evaluate the performance of three word-level transfer learning methods: OrthoMap, VecMap-supervised, and VecMap unsupervised for Urdu text. We further test these transfer learning methods for three tasks: propaganda identification, topic classification, and sentiment analysis. For this purpose, we augment an English-Urdu word dictionary and three datasets of Ur-En Propaganda, Ur-En News Dataset, and Ur-En Sentiment Corpus. Our analysis shows that the transfer learning methods optimize better for the short-text of Ur-En Sentiment Corpus with a precision of 40.1%. While for propaganda detection, the classifier attained an accuracy of 83% after transfer learning which is competitive with the 87% accuracy achieved after training the model on Urdu text data. We believe that this work will be beneficial for NLP, IR, and computational linguistic researchers working on Urdu language content.

This study examines the interdisciplinary connections that result from deep learning applications in artificial intelligence (AI) across several fields. A division of machine learning recognized as "deep learning" has proven to be remarkably effective at drawing out intricate patterns and representations from enormous datasets. Deep learning algorithms have made important improvements in fields as well as computer vision, natural language processing, robotics, finance, healthcare, advertising, and more by utilizing deep neural networks. This study’s goal is to evaluate the impact of deep learning in AI across disciplines and to draw attention to its revolutionary potential. The study examines the specific uses of deep learning in several fields and talk about the potential and problems that come with its deployment. The study provides a look at how precise object recognition, scene comprehension, and image production made deep learning transform the computer vision. Deep learning approaches have improved machine translation, sentiment analysis, and dialogue systems in natural language processing. This research study offers insights into the possibility of interdisciplinary cooperation by looking at the uses, difficulties, and future prospects of deep learning in AI. It demonstrates the transformative impact of deep learning in fostering creativity and advancement across a variety of sectors, opening the door for the creation of more intelligent and advanced AI systems.

The origin of route of text mining is the process of stemming. It is usually used in several types of applications such as Natural Language Processing (NLP), Information Retrieval (IR) and Text Mining (TM) including Text Categorization (TC), Text Summarization (TS). Establish a stemmer effective for the language of Gujarati has been always a search domain hot since the Gujarati has a very different structure and difficult that the other language due to the rich morphology.

IBM Research has over 200 people working on Unstructured Information Management (UIM) technologies with a strong focus on Natural Language Processing (NLP). These researchers are engaged in activities ranging from natural language dialog, information retrieval, topic-tracking, named-entity detection, document classification and machine translation to bioinformatics and open-domain question answering. An analysis of these activities strongly suggested that improving the organization's ability to quickly discover each other's results and rapidly combine different technologies and approaches would accelerate scientific advance. Furthermore, the ability to reuse and combine results through a common architecture and a robust software framework would accelerate the transfer of research results in NLP into IBM's product platforms. Market analyses indicating a growing need to process unstructured information, specifically multilingual, natural language text, coupled with IBM Research's investment in NLP, led to the development of middleware architecture for processing unstructured information dubbed UIMA. At the heart of UIMA are powerful search capabilities and a data-driven framework for the development, composition and distributed deployment of analysis engines. In this paper we give a general introduction to UIMA focusing on the design points of its analysis engine architecture and we discuss how UIMA is helping to accelerate research and technology transfer.

Recent years have observed a significant progress in information retrieval and natural language processing with deep learning technologies being successfully applied into almost all of their major tasks. The key to the success of deep learning is its capability of accurately learning distributed representations (vector representations or structured arrangement of them) of natural language expressions such as sentences, and effectively utilizing the representations in the tasks. This tutorial aims at summarizing and introducing the results of recent research on deep learning for information retrieval, in order to stimulate and foster more significant research and development work on the topic in the future. The tutorial mainly consists of three parts. In the first part, we introduce the fundamental techniques of deep learning for natural language processing and information retrieval, such as word embedding, recurrent neural networks, and convolutional neural networks. In the second part, we explain how deep learning, particularly representation learning techniques, can be utilized in fundamental NLP and IR problems, including matching, translation, classification, and structured prediction. In the third part, we describe how deep learning can be used in specific application tasks in details. The tasks are search, question answering (from either documents, database, or knowledge base), and image retrieval.

Events play an important role in natural language processing and information retrieval due to numerous event-oriented texts and information needs. Many natural language processing and information retrieval applications could benefit from a structured event-oriented document representation. In this paper, we proposeevent graphsas a novel way of structuring event-based information from text. Nodes in event graphs represent the individual mentions of events, whereas edges represent the temporal and coreference relations between mentions. Contrary to previous natural language processing research, which has mainly focused on individual event extraction tasks, we describe a complete end-to-end system for event graph extraction from text. Our system is a three-stage pipeline that performs anchor extraction, argument extraction, and relation extraction (temporal relation extraction and event coreference resolution), each at a performance level comparable with the state of the art. We presentEvExtra, a large newspaper corpus annotated with event mentions and event graphs, on which we train and evaluate our models. To measure the overall quality of the constructed event graphs, we propose two metrics based on the tensor product between automatically and manually constructed graphs. Finally, we evaluate the overall quality of event graphs with the proposed evaluation metrics and perform a headroom analysis of the system.

Deep learning has been very successful in the past decades, especially in Computer Vision and Speech Recognition fields. It has been also used successfully in the Natural Language Processing field because of the availability of an enormous amount of online text data, such as social networks and reviews websites, which have gained a lot of popularity and success in the past years. Sentiment Analysis is one of the hottest applications of Natural Language Processing (NLP). Many researchers have done excellent work on Sentiment Analysis for English language. However, the amount of work on Sentiment Analysis for Arabic language is, in comparison, very limited due to the complexity of the Arabic language's morphology and orthography. Unlike the English language, Arabic has many different dialects which makes Sentiment Analysis for Arabic more difficult and challenging, especially when working on data collected from social networks, which is known to be unstructured and noisy. Most of the work that has been done on Sentiment Analysis of Arabic language, focused on using lexicons and basic machine learning models. In addition, most of the work has been done on small datasets because of the limited number of the available annotated datasets for Arabic language. This paper proposes state-of-the-art research for Sentiment Analysis of Arabic microblogging using new techniques, and a sophisticated Arabic text data preprocessing.