Towards a robust deep learning framework for Arabic sentiment analysis

As a discipline of Natural Language Processing, Sentiment Analysis is used to extract and analyze subjective information present in natural language data. The task of Sentiment Analysis has acquired wide commercial uses including social media monitoring tasks, survey responses, review systems, etc. Languages like English have several resources which aid in the task of Sentiment Analysis. SentiWordNet and Subjectivity WordList are examples of such tools and resources. With more data being available in native vernacular, language-specific SentiWordNet(s) have become essential. For resource poor languages, creating such SentiWordNet(s) is a difficult task to achieve. One solution is to use available resources in English and translate the final source lexicon to target lexicon via machine translation. Machine translation systems for the English-Odia language pair have not yet been developed. In this paper, we discuss a method to create a SentiWordNet for Odia, which is resource-poor, by only using resources which are currently available for Indian languages. The lexicon created, would serve as a tool for Sentiment Analysis related task specific to Odia data.

Sentiment analysis, one of the most trending natural language processing tasks, is used to mine opinions or sentiments from a given text. Two significant challenges of sentiment analysis are 1) complexity in data pre-processing caused by the high dimensionality of textual data; 2) uncertainty in classifying sentiment polarities due to the ambiguity of natural languages. To address these issues, we propose a model using part-of-speech-based feature extraction to reduce dimensionality and game-theoretic rough sets (GTRS) to establish a balance between the accuracy and coverage trade-off. We evaluate this model with three different sizes of datasets (Yelp reviews, IMDB movie reviews, and Amazon product reviews). The experiment results show that the proposed model outperforms Pawlak’s rough set model and 0.5-probabilistic rough set model. In comparison with four traditional binary classification models (i.e., SVM, naïve Bayes, decision tree, and KNN), the proposed model also achieves higher accuracy rates. This research suggests that the proposed model is promising to deal with the complexity and uncertainty in sentiment analysis tasks.

Sentiment analysis has always been a major research area for natural language processing, which aims at computing people’s opinions, attitudes, and emotions toward an entity. Benefiting from the development of deep convolution neural networks, the performance of sentiment analysis has made a breakthrough. However, existing sentiment analysis methods often fluctuate greatly in different sentiment analysis tasks. In this paper, to solve this problem, we introduce ensemble learning on a variant of deep learning models to gain a better score on sentiment analysis tasks. Ensemble learning is one of the most efficient methods for improving representation ability on many tasks. Comprehensive experiments on benchmark dataset demonstrate that the ensemble method greatly improves the performance of the model through combining the advantages of the different previous and current state-of-the-art. Also, we carried out experiment to compare different type of ensemble methods to further enhance model performance.

Deep learning has become most prominent in solving various Natural Language Processing (NLP) tasks including sentiment analysis. However, these techniques require a considerably large amount of annotated corpus, which is not easy to obtain for most of the languages, especially under the scenario of low-resource settings. In this article, we propose a deep multi-task multi-lingual adversarial framework to solve the resource-scarcity problem of sentiment analysis by leveraging the useful and relevant knowledge from a high-resource language. To transfer the knowledge between the different languages, both the languages are mapped to the shared semantic space using cross-lingual word embeddings. We evaluate our proposed architecture on a low-resource language, Hindi, using English as the high-resource language. Experiments show that our proposed model achieves an accuracy of 60.09% for the movie review dataset and 72.14% for the product review dataset. The effectiveness of our proposed approach is demonstrated with significant performance gains over the state-of-the-art systems and translation-based baselines.

With the development and emergence of the concept of the Web, as well as the advancement of social networks, forums, blogs and E-commerce a growing number of Arabs express their views and opinions using their native language on those platforms where we find a wealth of information that could be exploited. This information, whose size is expanding and which describes opinions, has become the subject of several research fields particularly: “Sentiment Analysis” or “Opinion Mining”. Sentiment Analysis considered a subfield of natural language processing (NLP), information retrieval and text mining. Currently, sentiment analysis is commonly used for English, while it is in its infancy in the Arabic language. The most important and critical step in feeling analysis is to select an appropriate technique for classifying feelings. In this paper, we will be reviewing the published work on the application of Deep Learning (DL) techniques to the problems of sentiment analysis. We focus on the Arabic language because of its importance, the large number of its speakers, the scarcity of Arabic resources on it and the challenges related to its work. We explain, categorize, summarize and compare the techniques offered in this area.

Speech recognition and text summarization, plagiarism detection, machine translation, chatbots, sentiment analysis (SA), question answering (QA), and dialogue systems are all products of natural language processing (NLP), a branch of AI concerned with modeling natural languages for the purpose of developing relevant applications. NLP draws on several disciplines, not only computer science and linguistics, for its research and development. These include cognitive science, psychology, mathematics, and more. More than 1.5 billion Muslims throughout the world depend on the Arabic language for their daily five-times-prayer practice, and Arabic is one of six official languages used by more than 422 million people in the Arab world, according to UNESCO. Dialectal Arabic is the slang language spoken informally in everyday life and varies from country to country; Classical Arabic is a reflection of the language spoken by the Arabs more than fourteen centuries ago. Modern Standard Arabic is an evolving variety of Arabic that borrows and innovates regularly to meet the changing needs of its speakers. Complexity is added to the Arabic language by the fact that it encompasses not one but three different varieties of spoken language: classical, contemporary, and colloquial. However, Arabic language processing on computers remains difficult for a variety of reasons. Because of the extensive inflection and derivational processes that occur in Arabic, a single lemma may be utilized to produce several words with distinct meanings.

Currently, sentiment analysis has become a hot research topic in the natural language processing (NLP) field as it is highly valuable for many practical usages and theoretical studies. As a basic task in sentiment analysis, construction of sentiment lexicon aims to classify one word into positive, neutral or negative according to its sentiment orientation. However, when constructing a sentiment lexicon in Chinese, there are two major problems: 1) Chinese words are very ambiguous, which makes it hard to compute the sentiment orientation of a word; 2) Given the related research on sentiment analysis, available resource for constructing Chinese sentiment lexicons remains weak. Note that there are several corpus and lexicons in English sentiment analysis. In this study, we first use machine translation system with bilingual resources, i.e., English and Chinese information, then we get the sentiment orientation of Chinese words by computing the point-wise mutual information (PMI) values with English seed words. Experiment results from three domains demonstrate that the lexicon generated with our approach reaches an excellent precision and could cover domain information effectively.

Sentiment analysis (or opinion mining), is a natural language processing (NLP) technique that harnesses Information from text data that is naturally subjective. Opinions, emotions, and attitudes are things that this information can contain. In recent years, there has been an increasing focus on sentiment analysis because of the explosive growth of unstructured data on the internet. Sentiment analysis tasks have seen state of the art performance using deep learning models. However, these models can be computationally intensive and high dimensional can produce problems of overfitting when working with relatively limited data. In order to overcome these limitations, a new hybrid model is proposed using a combination of LSTM network for feature extraction and Random Forest classifier for the final sentiment classification improving the accuracy of the sentiment analysis, while reducing computational cost. In this paper, the proposed hybrid model is designed and implemented, and evaluated on a publicly available dataset of movie reviews.

The desire for comprehending user behavior is significant, since social media data is growing quickly owing to user contributions, especially in light of the recent coronavirus outbreak. In the literature works, several semantic analysis methodologies are used for examining the user behavior and emotions from the corpus data. They continue to deal with the main issues of poor accuracy, a complex system, inadequate emotion recognition, and ineffectiveness. The main objective of this paper is to build a new model for sentiment analysis and opinion mining. When compared to the conventional works, the novel concept behind this work is, it adopts some unique computational methodologies for accurate sentiment prediction. Here, a new Cycle Generative Adversarial Network (GAN) based Term Memory integrated Sentiment Analysis (CyG-TMSA) framework is developed for analyzing and classifying user emotions from corpus data. The most accurate identification of emotions is made possible by the efficient generation of augmented data using the Cycle Generative Adversarial Network (GAN) model. Moreover, a classification technique based on Term Memory Allocation (TMA) is applied to accurately predict opinions derived from the data supplementation. The best way to approximate the weight value for classifying data is to use the Elite Golden Jackle Optimization (EGJO) technique, which enhances the classifier’s decision-making ability. To verify and contrast the effectiveness and results of the suggested CyG-TAMS model, multiple corpus data sets extracted from public repositories are employed. The proposed CyG-TMSA framework demonstrated great effectiveness in sentiment analysis via CycleGAN-based data augmentation, TMA for precise classification, and optimization of parameter tuning through GJO. In experimental evaluations conducted on datasets extracted from Amazon and Twitter, the model obtained high accuracy at 98% within a very short processing time of 10.12 s, significantly outclassing other conventional methods in terms of accuracy and computational efficiency.

Over the past few years, Large Language Models of Code (Code LLMs) have started to have a significant impact on programming practice. Code LLMs are also emerging as building blocks for research in programming languages and software engineering. However, the quality of code produced by a Code LLM varies significantly by programming language. Code LLMs produce impressive results on high-resource programming languages that are well represented in their training data (e.g., Java, Python, or JavaScript), but struggle with low-resource languages that have limited training data available (e.g., OCaml, Racket, and several others). This paper presents an effective approach for boosting the performance of Code LLMs on low-resource languages using semi-synthetic data. Our approach, called M ulti PL-T, generates high-quality datasets for low-resource languages, which can then be used to fine-tune any pretrained Code LLM. M ulti PL-T translates training data from high-resource languages into training data for low-resource languages in the following way. 1) We use a Code LLM to synthesize unit tests for commented code from a high-resource source language, filtering out faulty tests and code with low test coverage. 2) We use a Code LLM to translate the code from the high-resource source language to a target low-resource language. This gives us a corpus of candidate training data in the target language, but many of these translations are wrong. 3) We use a lightweight compiler to compile the test cases generated in (1) from the source language to the target language, which allows us to filter our obviously wrong translations. The result is a training corpus in the target low-resource language where all items have been validated with test cases. We apply this approach to generate tens of thousands of new, validated training items for five low-resource languages: Julia, Lua, OCaml, R, and Racket, using Python as the source high-resource language. Furthermore, we use an open Code LLM (StarCoderBase) with open training data (The Stack), which allows us to decontaminate benchmarks, train models without violating licenses, and run experiments that could not otherwise be done. Using datasets generated with M ulti PL-T, we present fine-tuned versions of StarCoderBase and Code Llama for Julia, Lua, OCaml, R, and Racket that outperform other fine-tunes of these base models on the natural language to code task. We also present Racket fine-tunes for two very recent models, DeepSeek Coder and StarCoder2, to show that M ulti PL-T continues to outperform other fine-tuning approaches for low-resource languages. The M ulti PL-T approach is easy to apply to new languages, and is significantly more efficient and effective than alternatives such as training longer.

A multi-aspect framework for explainable sentiment analysis

Twitter is an open, freely available and widely used social media platform for sharing emotions, political and social opinions without hesitation. With the passage of time data generated by Twitter is increasing tremendously and due to availability of the such a huge amount of data, there is an opportunity for industry giants to analyze their current product reviews and made a better business making for the future and to conformance needs of customers. Similarly, there is an opportunity for research aspirants to explore the data to provide a better solution for modern problems. Therefore, in this research paper, we perform a detailed sentiment analysis of the three beverage giants (Pepsi Vs Fanta Vs Coca-cola). Specifically, we perform the classification of extracted tweeter data into three classes (positive, negative and neutral) then the visualization of the data is performed and at the end, naïve Bayes, decision tree and LSTM algorithm is applied to perform the prediction of tweets. Particularly, we use machine learning libraries and natural language toolkits for data extraction, data visualization, sentiment analysis and prediction task. The maximum accuracy obtained on Pepsi, Fanta and Coca-Cola is 89%, 92% and 91% respectively by applying the LSTM algorithm. The experimental result reveals that LSTM is a better technique for continuous data

Aspects extraction is the key task in the sentiment analysis problem, which includes extraction of both explicit and implicit aspects. Identifying implicit aspects is not a new task in sentiment analysis, but it still presents many challenges. Numerous studies have addressed the issue and offered different approaches, but there are still fundamental challenges as follows: solving the context word problem; the domain-specific problem; and the implicit aspects represented by sentiment words that do not appear explicitly in the text. This paper proposes a method for identifying implicit aspects of sentiment words. To address these above challenges, the system is built on the foundation of deep learning with context vectors, dependency grammar, anaphora coreference resolution, and sentiment ontology. The dependency parser and the entity coreference resolver are adopted to filter out the sentiment–aspect pairs data generated from embedding context words. Then, a fine-tuned technique is applied, and the classifier corresponding to each sentiment word is built to identify implicit aspects. This combination has produced positive results, reaching almost 90% accuracy.

Sentiment analysis (SA) is one of the most active and rapidly developing research areas in both information retrieval and text mining fields of computer science. SA is the process of analysing, handling, generalizing, and reasoning about user sentiments, opinions, and emotions hidden within the text by using sentiment. There are various categories of levels of SA: (1) document-level, (2) sentence-level, and (3) aspect-level. This paper is focused on sorting out some Aspect-based sentiment analysis (ABSA) works. ABSA is one of the primary tasks in sentiment analysis, which involves identifying the expression of an opinion or sensation of an object in a particular dimension or feature. ABSA involves two primary subtasks: (1) aspect extraction and (2) aspect-based sentiment classification. In contrast to document-and sentence-based sentiment analysis, ABSA simultaneously considers sentiment and target information. Generally, the ABSA methods can be categorized into Transformer based ABSA and Graph Neural Network based ABSA. This paper introduces some popular frameworks, datasets, and evaluation metrics of ABSA tasks based on the above-proposed classification and methodology.

Morphologically complex languages such as Arabic pose several challenges in Natural Language Processing (NLP) due to their complexity and token sparsity. Most techniques approach the problem by transforming the words of the language from their sparse surface form representation to a less sparse form before processing. The transformation usually takes the form of a morphological analysis or a morphological segmentation. This dissertation addresses two tasks in Arabic NLP: Statistical Machine Translation(SMT) and Sentiment Analysis. To improve English-Arabic SMT, we apply segmentation on Arabic to decrease token sparsity and enhance the correspondence between tokens of the English and Arabic language. However, due to this segmentation, the translation system is limited to extracting features based on morphemes (partial words) and only outputting morphemes during decoding. Such a system lacks knowledge of the original form of the words. We further improve translation from English to Arabic by combining both segmented and desegmented views of the target language. The system can benefit from segmentation's sparsity reduction and verifies its generation of correct words. We present a language-independent technique to desegmentation that approaches the problem as a string transduction task. We propose a new algorithm that desegments the decoder's search space encoded as a lattice, thus allowing the system to use features from the desegmented view of the search space. We extend the phrase-based statistical machine translation system to allow desegmentation during the decoding process on the fly. In addition, we conduct an experimental study to verify what matters most in morphologically segmented SMT models. Our second task is sentiment analysis, where we resort to Arabic lemmatization to improve sentiment analysis of Arabic tweets and blog posts. We explore translation in the opposite direction, from Arabic into English in order to evaluate the loss of sentiment predictability when Arabic social media posts are translated to English, manually or using an SMT system. We use state-of-the-art Arabic and English sentiment Analysis systems and develop automatically generated Arabic lexicons from lemmatized tweets to improve this task.