Sentiment Analysis Methods and Applications–A Review

The rapid growth of Internet-based applications, such as social media platforms and blogs, has resulted in comments and reviews concerning day-to-day activities. Sentiment analysis is the process of gathering and analyzing people’s opinions, thoughts, and impressions regarding various topics, products, subjects, and services. People’s opinions can be beneficial to corporations, governments, and individuals for collecting information and making decisions based on opinion. However, the sentiment analysis and evaluation procedure face numerous challenges. These challenges create impediments to accurately interpreting sentiments and determining the appropriate sentiment polarity. Sentiment analysis identifies and extracts subjective information from the text using natural language processing and text mining. This article discusses a complete overview of the method for completing this task as well as the applications of sentiment analysis. Then, it evaluates, compares, and investigates the approaches used to gain a comprehensive understanding of their advantages and disadvantages. Finally, the challenges of sentiment analysis are examined in order to define future directions.

This thesis examines the issue of ill-formed text (microtext) and shows its impact on sentiment analysis. The source of microtext is Twitter, WhatsApp, Facebook and other social media platforms. Words or phrases not in their standard language format like “lol” (laugh out loud), “c u 2nite” (see you tonight) and “plz” (please) are called out of vocabulary (OOV) terms. On the other hand, words or phrases in their standard forms, for example, “tonight”, “love”, or “talk to you later”, are called In Vocabulary (IV) terms. The motivation behind microtext normalization is its growing presence on both online and offline platforms. Microtext might appear to be an ill-formed text, which requires find-and-replace methods. Traditionally, in a Natural Language Processing (NLP) context, microtexts such as abbreviations and length shortening or extending are normalized, but phonemic and graphemic variation is often ignored. However, the features of microtext can be traced back to social, cognitive and geographic influences. To understand social media text, it is essential to develop a microtext normalization module for traditional and modern NLP models. The first section of this thesis focuses on combining unsupervised learning methods in 2 different chapters, 4 and 5. The first method includes lexicon creation to transform the most frequent microtexts and emoticons on social media and applies them to sentiment analysis. This method shows an accuracy increase of 4% when microtext normalization is applied before sentiment analysis. This method is extended with an application to a pre-trained chatbot to improve the chatbot’s understanding of microtext (social media slangs). This method shows a mean BLEU score of 0.8 for the SMS and Tweet dataset. The second method incorporates a phonetic based approach to transform any microtext into its phonetic equivalent before 1normalizing it. This method shows an accuracy improvement of 6% for sentiment analysis on SenticNet. The second part of the unsupervised learning is in Chapter 5, introducing the IPA-based method for microtext normalization and sentiment analysis. Epitran was used to transform a word to its International Phonetic Alphabet (IPA) equivalent. The International Phonetic Alphabet (IPA) is an alphabetic system of phonetic notation based primarily on the Latin script. There is a significant improvement in microtext normalization task and sentiment analysis over Soundex. The results also show that there is very little redundancy when words are transformed to phonetics in IPA. In addition, the accuracy of polarity detection using this method has an improvement over baseline by 5%. Chapter 6 discusses a simple sequence-to-sequence based Deep Learning method. As the language is dynamic, it is not easy to maintain the lexicon. Deep Learning alleviates the static lexicon with a probabilistic model to transform microtext into its standard form. The findings suggest that a semiotic class of microtext was easy to transform, whereas the phonetic class of microtext was not handled easily. The proposed model improves the pre-trained sentiment analysis model by 4%. We also introduce a corpus for English microtext normalization. The corpus contains microtext-containing sentences along with their correctly spelt sentences and their correct polarity. This dataset can tackle two NLP problems, i.e., microtext normalization and sentiment analysis. This corpus consists of 38% positive, 39.9% negative and 22.1% neutral. A pre-trained model shows an accuracy of 46.4 % on OOV text for sentiment analysis, showing considerable room for improvement. In summary, this thesis introduces several approaches to microtext normalization analysis and focuses on sentiment analysis application. Microtext cannot be thought of as a one-dimensional problem to NLP. It encapsulates many factors like regional (geographical), ethnic (national and racial), and social (class, age, gender, socioeconomic status and education), which cannot be ignored given the growing importance of social media in our daily life.

Recognizing and categorizing user’s sentiments from a part of text into different sentiments is known as sentiment analysis. For instance, emotions such as happy, sad, angry or positive, negative or neutral to determine the users attitude concerning a certain subject or object. Sentiment analysis is one of the utmost active research areas in natural language processing, web mining and text mining. It plays an important role in many fields like management sciences and social sciences including education, where student feedback is essential to assess the effectiveness of learning technologies. With increase in educational organizations, online learning portals have fascinated by many students by offering free courses with no fee. Heaps of learners enroll in these massive online courses every year and further review their sentiments about the course content and quality of education. Also, provide suggestions in blogs in order to improve the quality of teaching by giving positive or negative sentiments. This paper proposes a model based on Deep Learning approach to perform sentiment analysis on Educational data. In this paper we focused on the accuracy and performance of the training data set to predict the best model. MLP and SVM are recognized as the outperforming models.

With the development and emergence of the concept of the Web, as well as the advancement of social networks, forums, blogs and E-commerce a growing number of Arabs express their views and opinions using their native language on those platforms where we find a wealth of information that could be exploited. This information, whose size is expanding and which describes opinions, has become the subject of several research fields particularly: “Sentiment Analysis” or “Opinion Mining”. Sentiment Analysis considered a subfield of natural language processing (NLP), information retrieval and text mining. Currently, sentiment analysis is commonly used for English, while it is in its infancy in the Arabic language. The most important and critical step in feeling analysis is to select an appropriate technique for classifying feelings. In this paper, we will be reviewing the published work on the application of Deep Learning (DL) techniques to the problems of sentiment analysis. We focus on the Arabic language because of its importance, the large number of its speakers, the scarcity of Arabic resources on it and the challenges related to its work. We explain, categorize, summarize and compare the techniques offered in this area.

The sentiment analysis process that gives this work its name is the main theme of the work. Since the beginning of 2000, sentiment analysis has become one of the most active research areas by researchers working on natural language processing and social networking analysis. In addition, data mining, web mining, and text mining are also studied extensively. Moreover, the method of sentiment analysis has spread over many fields, from computer science to management science, from social science to economics, due to the importance given to the business world as a whole and the collectivity. In this study, Konstanz Information Miner (KNIME), which is a powerful data mining tool with its richest features and many visualization tools, was used on twitter data. Ten thousand Twitter data were used in this study. The sentiment analysis study, which is in fact a classification study, was conducted using machine learning algorithms on Twitter data. The results of the study were interpreted by carrying out an accuracy analysis. It is anticipated that the use of the KNIME, which has rich visualization tools, will be widespread in sentiment analysis studies to make these works both easier and more reliable.

IntroductionSince the Coronavirus disease 2019 (COVID-19), public safety measures, including social distancing and school closures, have been implemented, precipitating psychological difficulties and heightened online activities for adolescents. However, studies examining the impact of the pandemic on adolescent mental health and their coping strategies in Asian countries are limited. Further, most studies have used survey measures to capture mental health challenges so far. Accordingly, this study aimed to examine the psychological challenges South Korean adolescents experienced and their coping strategies during the pandemic using the Natural Language Processing (NLP) and Text mining (TM) technique on adolescents’ social media texts/posts.MethodsThe data were gathered from social media texts/posts such as online communities, Twitter, and personal blogs from January 1, 2019, to October 31, 2021. The 12,520,250 texts containing keywords related to adolescents’ common psychological difficulties reported during the pandemic, including self-harm, Attention-Deficit/Hyperactivity Disorders (ADHD), and insomnia, were analyzed by TM, NLP using information extraction, co-occurrence and sentiment analysis. The monthly frequency of the keywords and their associated words was also analyzed to understand the time trend.ResultsAdolescents used the word “self-harm” in their social media texts more frequently during the second wave of COVID-19 (August to September 2020). “Friends” was the most associated word with “self-harm.” While the frequency of texts with “Insomnia” stayed constant throughout the pandemic, the word “ADHD” was increasingly mentioned in social media. ADHD and insomnia were most frequently associated with ADHD medications and sleeping pills, respectively. Friends were generally associated with positive words, while parents were associated with negative words.ConclusionDuring COVID-19, Korean adolescents often expressed their psychological challenges on social media platforms. However, their coping strategies seemed less efficient to help with their difficulties, warranting strategies to support them in the prolonged pandemic era. For example, Korean adolescents shared psychological challenges such as self-harm with friends rather than their parents. They considered using medicine (e.g., sleeping pills and ADHD medication) as coping strategies for sleep and attention problems.

This paper presents an application of sentiment analysis on customer feedback data in the area of heavy equipment repair services. Sentiment analysis is used as a part of a framework for text mining-based Customer Loyalty Improvement Recommender System (CLIRS). In order to provide business users of the system with accurate predictions for customer satisfaction metrics, the original algorithm for the opinion mining needed to be improved. The paper presents the background of the proposed approach, the current techniques used to mine text data and existing applications of sentiment analysis. We propose an aspect-based, taxonomy-driven approach for customized sentiment analysis. The contribution of this paper is the implementation and evaluation of the proposed methods that improve the accuracy and coverage of the opinion mining algorithm. The improvements are illustrated with examples covered by the algorithm in the customer dataset. The application of the proposed methods resulted in increasing the algorithm’s accuracy from 92% to 96%, and coverage from 36% to 48%. This research is an attempt to handle well-known issues in natural language processing that are currently not handled by text mining algorithms, such as ambiguity and context, opinionated verbs/nouns, subject recognition from pronouns. This is significant because the proposed techniques are generalizable to any application that uses sentiment analysis algorithm.

Sentiment analysis has become a powerful tool in processing and analysing expressed opinions on a large scale. While the application of sentiment analysis on English-language content has been widely examined, the applications on the Russian language remains not as well-studied. In this survey, we comprehensively reviewed the applications of sentiment analysis of Russian-language content and identified current challenges and future research directions. In contrast with previous surveys, we targeted the applications of sentiment analysis rather than existing sentiment analysis approaches and their classification quality. We synthesised and systematically characterised existing applied sentiment analysis studies by their source of analysed data, purpose, employed sentiment analysis approach, and primary outcomes and limitations. We presented a research agenda to improve the quality of the applied sentiment analysis studies and to expand the existing research base to new directions. Additionally, to help scholars selecting an appropriate training dataset, we performed an additional literature review and identified publicly available sentiment datasets of Russian-language texts.

In recent years, text mining has become a branch of Natural Language Processing. This chapter focuses on the analysis of textual data, illustrated by examples treated with the software R, which will be used to manipulate texts and strings, to transform them by word embedding methods and to analyse them by supervised and unsupervised statistical and machine learning methods. It discusses methods of topic modeling and sentiment analysis. There are two types of methods in textual data analysis: descriptive methods and predictive methods. The chapter describes the various lexical analyses that are often performed to categorize documents or to detect the topics contained in a set of documents. The analysis of textual data is most often done on vector representations of the data, which are stored in matrices containing the documents in rows and the terms in columns. A recent application of natural language processing is sentiment analysis.

Sentiment analysis is an important task in natural language processing, which has promises great value to areas of interests such as business, politics and other fields. The prevalence of the internet has caused people to prefer expressing their opinion and sentiment on the Internet via methods such as tweeting on social media and commenting on products. However, the discourse of users on social media are usually short either due to limitations on post size or time limitations of the users. What emerges from these features is that the grammar used in these posts and the word meanings are flexible, which makes sentiment analysis difficult. Therefore, sentiment analysis is an important yet challenging task. In this paper, we propose a context-based regularization classification method for short text sentiment analysis. Specifically, we use contextual knowledge obtained from the data to improve performance of the sentiment classification. In this paper, the contextual knowledge includes two parts: word-sentiment knowledge and word-similarity knowledge. Moreover, we propose methods to calculate the sentiment of the words and the similarity between words on semantic level. Specifically, on the one side, we use a TRSR method based on the TextRank algorithm to rank words in each sentiment sample to determine the sentiment polarity of each word. On the other, we calculate the similarity between words according to word-embedding. In this way, we can determine the similarity between words and the sentiment polarity of a word. We then incorporate the contextual knowledge as a regularization into a supervised classification framework, which then converts into an optimization problem to train a more accurate model. Experiments on both Chinese and English datasets outperform than other baseline approaches, which demonstrates our method to be stable and effective.

Arabic language suffers from the lack of available large datasets for machine learning and sentiment analysis applications. This work adds to the recently reported large dataset BRAD, which is the largest Book Reviews in Arabic Dataset. In this paper, we introduce HARD (Hotel Arabic-Reviews Dataset), the largest Book Reviews in Arabic Dataset for subjective sentiment analysis and machine language applications. HARD comprises of 490587 hotel reviews collected from the Booking.com website. Each record contains the review text in the Arabic language, the reviewer’s rating on a scale of 1 to 10 stars, and other attributes about the hotel/reviewer. We make available the full unbalanced dataset as well as a balanced subset. To examine the datasets, we implement six popular classifiers using Modern Standard Arabic (MSA) as well as Dialectal Arabic (DA). We test the sentiment analyzers for polarity and rating classifications. Furthermore, we implement a polarity lexicon-based sentiment analyzer. The findings confirm the effectiveness of the classifiers and the datasets. Our core contribution is to make this benchmark-dataset available and accessible to the research community on Arabic language.

Opinion mining is considered as a subfield of natural language processing, information retrieval and text mining. Opinion mining is the process of extracting human thoughts and perceptions from unstructured texts, which with regard to the emergence of online social media and mass volume of users’ comments, has become to a useful, attractive and also challenging issue. There are varieties of researches with different trends and approaches in this area, but the lack of a comprehensive study to investigate them from all aspects is tangible. In this paper we represent a complete, multilateral and systematic review of opinion mining and sentiment analysis to classify available methods and compare their advantages and drawbacks, in order to have better understanding of available challenges and solutions to clarify the future direction. For this purpose, we present a proper framework of opinion mining accompanying with its steps and levels and then we completely monitor, classify, summarize and compare proposed techniques for aspect extraction, opinion classification, summary production and evaluation, based on the major validated scientific works. In order to have a better comparison, we also propose some factors in each category, which help to have a better understanding of advantages and disadvantages of different methods.

Sentiment analysis is a technique of mining data for its sentiment content. It is one of the areas of text mining and natural language processing whose importance and scope is increasing day by day. Sentiment analysis helps us to extract sentiment of consumers from data. The data can be in any form. In this paper, we have used tweets as our data set. A large amount of data are generated everyday in social media platforms like Facebook, Twitter and others due to increased use of social media. Such data can be in the form of some status update or tweets. These data often reflect the sentiment of the users. Thus, we can harvest this large amount of data to extract sentiment of the consumers of any industry. The airline industry is rapidly growing with the number of passengers increasing day by day. The passengers often put forward their feedback or experience with a particular flight company on social media platform Twitter. Twitter has become a platform to present such reviews from the consumers of the airline industry. In this paper, we mine tweets from Twitter to present a case study of the Indian airline industry. An in-depth study of the sentiments has been presented. We have analysed the emotion content of the tweets of the airline passengers. The percentage of positive tweets was found highest for Vistara Airlines at 60.8 per cent. We use the Syuzhet (Extract Sentiment and Plot Arcs from Text, Matthew L. Jockers, 2015. https://github.com/mjockers/syuzhet) package available in R. We consider the major airline companies in India, namely IndiGo, Jet Airways, Air India, GoAir, SpiceJet, AirAsia and Vistara.

In recent years, vocabulary emotion processing has become immensely popular and the requirements for language emotion analysis mining and processing have become significantly abundant. The sentiment extraction and analysis work has always been very challenging; especially, the Chinese word segmentation operation is difficult to deal with effectively, the multiple combinations of implicit and explicit words make the task of sentiment analysis mining more difficult, and, in particular, the efficiency of machine analysis of language sentiment is feeble. We use some expressions and sentiment vocabulary dictionaries combined with hybrid structures and use information synergy methods to get in touch with sentiment analysis methods. We use the relevant sentiment to evaluate the explicit or implicit emotional association of the emotional connection of the vocabulary and add the unique emotional word matrix to analyze the related clustering results of the emotional words to continuously optimize and upgrade the performance, so that our sentiment analysis results are systematic in terms of efficiency and significantly improved.

<sec> <title>BACKGROUND</title> The COVID-19 pandemic caused many changes in primary health care systems in Europe. The fast adoption of telemedicine, the shift of health care resources to COVID-19–related tasks, and the tendency of patients to cancel their nonurgent appointments are some examples of these changes. Patient satisfaction is an important outcome of health care services, and the changes caused by COVID-19 in the system may have affected it. Google Maps reviews provide an important channel for patients to communicate about their experiences regarding the primary health care system. </sec> <sec> <title>OBJECTIVE</title> Drawing from research on social media data analytics and text mining, this study set out to investigate the changes in public sentiment regarding primary health care in Finland and Andalusia (Spain) before, during, and after the COVID-19 pandemic. </sec> <sec> <title>METHODS</title> We collected 55,043 Google Maps reviews from primary health care locations in Finland and Andalusia from January 1, 2013, to May 15, 2024. There are 604 primary health care locations in Finland and 1016 in Andalusia. The total number of Google Maps reviews collected was 12,247 for Finland and 42,796 for Andalusia. First, lexicon-based sentiment analysis using the open-source software AFINN was conducted for the Finnish- and Spanish-language datasets. Thereafter, transformer-based deep learning models for sentiment analysis were applied for both languages. The numeric user ratings and the results of the sentiment analysis were then analyzed. In addition, we conducted a word frequency analysis of the reviews. </sec> <sec> <title>RESULTS</title> There were important changes in the ratings and sentiment in the data for Andalusia. The ratings shifted from median 4 (IQR 3) before the COVID-19 pandemic to median 1 (IQR 2) during and median 1 (IQR 3) after the COVID-19 pandemic, on a scale from 1 to 5. The median of the sentiment values of the review texts shifted from neutral before the COVID-19 pandemic to –2 (IQR 0.055) or –1 (IQR 1) during and after the COVID-19 pandemic, depending on which sentiment analysis method was used. Interestingly, changes in numeric ratings and sentiment of the review texts in Finland were only minor, and the median values were the same during all 3 periods. Lexical analysis revealed changes in word frequencies across the periods, reflecting shifts in primary health care experiences during the pandemic, especially among the Spanish-language reviews. </sec> <sec> <title>CONCLUSIONS</title> The change toward a more negative public discussion on primary health care in Andalusia during the COVID-19 pandemic was considerable. This can be observed both in the numeric user ratings and in the sentiment analysis of the review texts. However, the data for Finland show that the public discourse stayed mostly neutral or slightly positive. The findings have implications on the quality management procedures in primary health care and on the use of user-generated content as an additional information source. </sec>