Identifying comparative opinions in Arabic text in social media using machine learning techniques

The Impact of Arabic Part of Speech Tagging on Sentiment Analysis: A New Corpus and Deep Learning Approach

Developing Resources For Sentiment Analysis Of Informal Arabic Text In Social Media

Sentiment analysis is the process of computationally recognizing and classifying the attitudes conveyed in each text towards a particular topic, product, etc. which is either positive or negative. Sentiment analysis is one of the interesting applications of natural language processing (NLP) and which is used to analyze the social media. Text in social media is casual and it can be written either in code-switch or monolingual text. Several researchers have implemented sentiment analysis on monolingual text, though sentiments can be expressed in code-switch text. Sentiment analysis can be applied through deep learning (DL), machine learning (ML), or a Lexicon-based approach. Machine learning (ML) and deep learning (DL) methods are time-consuming, computationally expensive, and need training data for analysis. Lexicon-based method does not require training data and requires less time to find the sentiments in comparison with ML and DL. In this paper, we propose the Lexicon-based approach (NBLex) to analyze the sentiments expressed in Kannada-English code-switch text. This is the first effort that targets to perform sentiment analysis in Kannada-English code-switch text using the Lexicon-based approach. The proposed approach performed with better accuracy of 83.2% and 83% of F1-score.

This thesis examines the issue of ill-formed text (microtext) and shows its impact on sentiment analysis. The source of microtext is Twitter, WhatsApp, Facebook and other social media platforms. Words or phrases not in their standard language format like “lol” (laugh out loud), “c u 2nite” (see you tonight) and “plz” (please) are called out of vocabulary (OOV) terms. On the other hand, words or phrases in their standard forms, for example, “tonight”, “love”, or “talk to you later”, are called In Vocabulary (IV) terms. The motivation behind microtext normalization is its growing presence on both online and offline platforms. Microtext might appear to be an ill-formed text, which requires find-and-replace methods. Traditionally, in a Natural Language Processing (NLP) context, microtexts such as abbreviations and length shortening or extending are normalized, but phonemic and graphemic variation is often ignored. However, the features of microtext can be traced back to social, cognitive and geographic influences. To understand social media text, it is essential to develop a microtext normalization module for traditional and modern NLP models. The first section of this thesis focuses on combining unsupervised learning methods in 2 different chapters, 4 and 5. The first method includes lexicon creation to transform the most frequent microtexts and emoticons on social media and applies them to sentiment analysis. This method shows an accuracy increase of 4% when microtext normalization is applied before sentiment analysis. This method is extended with an application to a pre-trained chatbot to improve the chatbot’s understanding of microtext (social media slangs). This method shows a mean BLEU score of 0.8 for the SMS and Tweet dataset. The second method incorporates a phonetic based approach to transform any microtext into its phonetic equivalent before 1normalizing it. This method shows an accuracy improvement of 6% for sentiment analysis on SenticNet. The second part of the unsupervised learning is in Chapter 5, introducing the IPA-based method for microtext normalization and sentiment analysis. Epitran was used to transform a word to its International Phonetic Alphabet (IPA) equivalent. The International Phonetic Alphabet (IPA) is an alphabetic system of phonetic notation based primarily on the Latin script. There is a significant improvement in microtext normalization task and sentiment analysis over Soundex. The results also show that there is very little redundancy when words are transformed to phonetics in IPA. In addition, the accuracy of polarity detection using this method has an improvement over baseline by 5%. Chapter 6 discusses a simple sequence-to-sequence based Deep Learning method. As the language is dynamic, it is not easy to maintain the lexicon. Deep Learning alleviates the static lexicon with a probabilistic model to transform microtext into its standard form. The findings suggest that a semiotic class of microtext was easy to transform, whereas the phonetic class of microtext was not handled easily. The proposed model improves the pre-trained sentiment analysis model by 4%. We also introduce a corpus for English microtext normalization. The corpus contains microtext-containing sentences along with their correctly spelt sentences and their correct polarity. This dataset can tackle two NLP problems, i.e., microtext normalization and sentiment analysis. This corpus consists of 38% positive, 39.9% negative and 22.1% neutral. A pre-trained model shows an accuracy of 46.4 % on OOV text for sentiment analysis, showing considerable room for improvement. In summary, this thesis introduces several approaches to microtext normalization analysis and focuses on sentiment analysis application. Microtext cannot be thought of as a one-dimensional problem to NLP. It encapsulates many factors like regional (geographical), ethnic (national and racial), and social (class, age, gender, socioeconomic status and education), which cannot be ignored given the growing importance of social media in our daily life.

Abstract: Automobile reviews provide a valuable opportunity for sentiment analysis in order to gain a better understanding of consumer opinions and improve product quality. As a result of the complexity of natural language and differences in user opinions, accurate and robust sentiment analysis by computer models remains challenging. Traditional machine learning algorithms such as Support Vector Machines (SVM) and Naive Bayes (NB) have been used for sentiment analysis on textual data. They can, however, be limited by the heterogeneity of user opinions and language variations. In contrast, deep learning models and natural language processing have shown great promise in sentiment analysis. In particular, the VADER (Valence Aware Dictionary and sentiment Reasoner) algorithm has been proposed as a powerful tool for sentiment analysis on social media texts, and has shown promising results in other domains as well. In this work, The aim is to achieve accurate sentiment analysis on automobile reviews by comparing the accuracy performance of SVM, NB, and the VADER algorithm.

Navigating Social Media in #Ophthalmology

Natural Language Processing (NLP) is an essential element of computational linguistics and artificial ‎intelligence, enabling fluid interactions between humans and machines. Social networking networks ‎produce substantial volumes of user-generated text daily, offering both opportunities and challenges for ‎NLP researchers. Social media discourse's informal, dynamic, and context-dependent characteristics ‎necessitate specific NLP techniques for precise processing and analysis. This study thoroughly examines NLP applications in social media, including essential tasks such as sentiment analysis, topic ‎modeling, misinformation detection, and hate speech identification. It examines the influence of machine ‎learning and deep learning methodologies, particularly transformer models, on the advancement of NLP ‎capabilities. This study also emphasizes the ethical issues related to NLP-driven social media apps, ‎including data privacy, algorithmic bias, and the regulation of misinformation. The paper continues by ‎discussing emerging research paths, highlighting the necessity for adaptable and ethical NLP solutions in ‎the changing social media environment‎.

Modern Standard Arabic (MSA) is the formal language in most Arabic countries. Arabic Dialects (AD) or daily language differs from MSA especially in social media communication. However, most Arabic social media texts have mixed forms and many variations especially between MSA and AD. This paper aims to bridge the gap between MSA and AD by providing a framework for the translation of texts of social media. More precisely, this paper focuses on the Tunisian Dialect of Arabic (TAD) with an application on automatic machine translation for a social media text into MSA and any other target language. Linguistic tools such as a bilingual TAD-MSA lexicon and a set of grammatical mapping rules are collaboratively constructed and exploited in addition to a language model to produce MSA sentences of Tunisian dialectal sentences. This work is a first-step towards collaboratively constructed semantic and lexical resources for Arabic Social Media within the ASMAT (Arabic Social Media Analysis Tools) project.

Sentiment analysis is the computational study of attitudes, opinions, and sentiments towards certain issues, products, individuals, and organizations. Companies and customers are making decisions by seeking opinions from social media text. Sentiment analysis is getting intelligent with the advancement of artificial intelligence and natural language processing. With a stunning increase in the use of social media, a huge volume of text available on these platforms is in imperfect and informal languages like Roman Urdu mixed with the English language. Present sentiment analysis techniques do not perform precisely on these code-mixed imperfect, informal, and poorly resourced languages. A promising solution is the use of deep learning models on these code-mixed Roman Urdu and English text. Therefore, the objective of this paper is to perform a sentiment analysis of code-mixed Roman Urdu and English social media text using state-of-the-art deep learning models. Our work is independent of lexical normalization, language dictionary, and code transfer indication. We perform sentiment analysis using Multilingual BERT (mBERT) and XLM-RoBERTa (XLM-R) models. The results reveal that performance of XLM-R model with tuned hyperparameters for code-mixed Roman Urdu and English social media text is better than the mBERT model with F1 score of 71%.

Sentiment Analysis (SA) is an increasingly important field of study, also known as opinion mining, as it has the ability to identify the emotional tone of a source material as being either positive, negative, or neutral. Simply, SA digests natural languages and extracts insights, making it extremely useful to gain an overview of the public opinion in a certain issue, topic, or even a product. With the vast amount of information over social network platforms, shared views, website reviews, and blogs comes the importance of SA. Arabic is a rich language with extremely complex inflectional and derivational morphology making sentiment analysis in Arabic text more challenging. In this paper, we propose a novel approach to enhance the accuracy of Arabic Sentiment Analysis (ASA). In this aspect, nine supervised machine learning algorithms have been implemented for ASA. Three of these classifiers have never been used before in ASA classification, namely, they are Ridge, Gradient Boosting, and Multi-layer Perceptron. The performance of all nine classifiers is tested using our constructed dataset. The dataset contains 6318 reviews written in different forms of Arabic language and prepared manually by gathering hotel reviews from Booking.com website.

Text emanated from users’ posts and comments on social media constitutes important piece of information for wide ranging Natural Language Processing (NLP) applications, such as Sentiment Analysis, Sarcasm Detection, Named Entity Identification, Question Answering and Information Retrieval (IR). Part–of–Speech (POS) tagging, a prerequisite for all such applications, augments tag information to the raw text. However, an inherent tendency of social media users to include multilingual contents in their posts, called code-mixing, poses challenge to POS tagging. Besides, intricate and free style writing add to the complexity of problem. To cope with the issue, a Hidden Markov Model (HMM) based supervised algorithm has been introduced for POS tagging of code–mixed Indian social media text. Publicly available social media text of Indian Languages (ILs), particularly English, Hindi, Bengali and Telugu, have been used to train and test the proposed system. Correctness of system annotated tags has been evaluated on ground of F-measure.

Over 420 million people speak Arabic, and it is the official language of 22 countries. Its complex morphology and dialectal diversity present unique challenges for natural language processing (NLP) models like ChatGPT. This systematic review investigates the application of ChatGPT in Arabic language text processing, examining its potential uses, accuracy, and limitations. Covering literature published between 2021 and 2024, this review synthesizes findings from 21 articles, addressing four key research questions: ChatGPT’s applications in Arabic text processing, its performance in terms of accuracy and reliability, the challenges and limitations encountered, and future directions to enhance its utilization. Results indicate that ChatGPT has potential in several applications, including educational tools, machine translation, text generation, and sentiment analysis. Despite current limitations, ChatGPT's potential in Arabic text processing is promising. While it shows high accuracy in structured tasks, it struggles with dialectal variations and cultural nuances, especially in complex text types. Primary limitations include a lack of high-quality Arabic datasets, difficulty handling dialects, and a need for more nuanced contextual understanding. Future research should focus on improving data quality, expanding dialectal coverage, fine-tuning models for specific linguistic tasks, and integrating AI with human teaching methods. Addressing these areas will enhance ChatGPT's accuracy and reliability for Arabic NLP.

A Comprehensive Methodology for Extracting Signal from Social Media Text Using Natural Language Processing and Machine Learning

Sentiment analysis is a methodology used to analyse the emotion or view of an individual to a situation or topic. In present scenario, Social media is the source for the collection of individual's feedbacks, user's emotions, reviews and personal experiences which lead to a need for efficient mining of the text to derive knowledge. An optimal classification of text based on emotion is an unsolved problem in text mining. To extract knowledge from text many machine learning tools and techniques were proposed. An onto-based process is proposed to analyse the customer's emotion in this paper. The input emotional text that needs to be classified is given as input to the NLP and processed and an emotional ontology is created for better understanding of the semantics and relationships. When adding new instances, Ontology can be automatically classify them based on emotional relationship. The Emowords from ontology can be further classified using any of the standard machine learning techniques which definitively gives a better performance. This paper is a review of all the machine learning techniques that can be applied on the semantic analysis of sentiments.

Adverse drug events (ADEs) remain a significant burden to public health and a persistent challenge for pharmacovigilance. The proliferation of patient-generated discourse on social media offers a complementary, real-time signal for ADE surveillance. This article provides a concise yet comprehensive review of recent natural language processing (NLP) research on identifying ADEs in social media text. We systematically reviewed 100 peer-reviewed studies (2017-2025) on NLP/AI for detecting or analysing ADEs in social media. Searches in Google Scholar targeted English-language journal and conference papers; patents and protocols were excluded. Of 130 records screened, 6 were protocols and 24 were excluded because the full text could not be located or the item was a conference abstract lacking methodological detail (i.e., no description of approaches or experiments), yielding a final sample of 100 studies. One reviewer performed screening, with full-text eligibility verified by a second. We extracted objectives, data sources/languages, preprocessing and annotation practices, datasets, model families, evaluation metrics, and stated limitations. Studies were grouped into five task categories-classification, extraction, normalization, corpus creation, and broader analytical work-with evidence tables summarizing contributions, toolchains, datasets, and performance. Recurrent challenges include noisy/imbalanced data, multilingual and code-mixed content, and variability in annotation standards. Twitter remains the primary data source: 60% of studies analyse Twitter alone and a further 18% combine Twitter with other platforms (78% in total). English overwhelmingly dominates; only about 5% of studies draw on non-English sources (e.g., French, Chinese, Arabic). Standard pre-processing-URL removal, tokenisation, and lowercasing-is near-universal. Transformer-based models predominate, with BERT and its biomedical or "tweet" variants (e.g., RoBERTa, BioBERT, BERTweet) used in more than 60% of approaches. Persistent obstacles include severe class imbalance and ambiguous or implicit drug-event expressions. Although shared tasks such as SMM4H provide widely used benchmarks, comprehensive annotation guidelines remain uncommon (12% of papers). Recent work increasingly incorporates multimodal inputs and integrates structured biomedical knowledge, yet gaps persist in multilingual coverage, temporal/longitudinal modelling, and real-world deployment. To our knowledge, this is the first review to synthesise findings from a corpus of 100 peer-reviewed studies on ADE detection in social media using NLP. By organising the literature by task type and tracing methodological trends and limitations, it provides practical guidance for researchers and practitioners. The review also outlines actionable directions for future work, including model explainability, support for low-resource languages, and closer collaboration with regulatory authorities to enable real-world deployment.