A survey on sentiment analysis methods, applications, and challenges

Sentiment analysis is the most vital research field in the area of natural language processing and text mining. It is a powerful tool for entrepreneurs that introduces products upon online selling websites where customers can post their reviews. In this paper, we have taken the state-of-the-art research papers which are highly cited and analyse them to understand the different techniques and methods to perform sentiment analysis. As sentiment analysis is related to natural language processing it is having several issues of computation. We have discussed many different techniques and tried to suggest the best way to work with them. We have also discussed the different applications of sentiment analysis.

This thesis examines the issue of ill-formed text (microtext) and shows its impact on sentiment analysis. The source of microtext is Twitter, WhatsApp, Facebook and other social media platforms. Words or phrases not in their standard language format like “lol” (laugh out loud), “c u 2nite” (see you tonight) and “plz” (please) are called out of vocabulary (OOV) terms. On the other hand, words or phrases in their standard forms, for example, “tonight”, “love”, or “talk to you later”, are called In Vocabulary (IV) terms. The motivation behind microtext normalization is its growing presence on both online and offline platforms. Microtext might appear to be an ill-formed text, which requires find-and-replace methods. Traditionally, in a Natural Language Processing (NLP) context, microtexts such as abbreviations and length shortening or extending are normalized, but phonemic and graphemic variation is often ignored. However, the features of microtext can be traced back to social, cognitive and geographic influences. To understand social media text, it is essential to develop a microtext normalization module for traditional and modern NLP models. The first section of this thesis focuses on combining unsupervised learning methods in 2 different chapters, 4 and 5. The first method includes lexicon creation to transform the most frequent microtexts and emoticons on social media and applies them to sentiment analysis. This method shows an accuracy increase of 4% when microtext normalization is applied before sentiment analysis. This method is extended with an application to a pre-trained chatbot to improve the chatbot’s understanding of microtext (social media slangs). This method shows a mean BLEU score of 0.8 for the SMS and Tweet dataset. The second method incorporates a phonetic based approach to transform any microtext into its phonetic equivalent before 1normalizing it. This method shows an accuracy improvement of 6% for sentiment analysis on SenticNet. The second part of the unsupervised learning is in Chapter 5, introducing the IPA-based method for microtext normalization and sentiment analysis. Epitran was used to transform a word to its International Phonetic Alphabet (IPA) equivalent. The International Phonetic Alphabet (IPA) is an alphabetic system of phonetic notation based primarily on the Latin script. There is a significant improvement in microtext normalization task and sentiment analysis over Soundex. The results also show that there is very little redundancy when words are transformed to phonetics in IPA. In addition, the accuracy of polarity detection using this method has an improvement over baseline by 5%. Chapter 6 discusses a simple sequence-to-sequence based Deep Learning method. As the language is dynamic, it is not easy to maintain the lexicon. Deep Learning alleviates the static lexicon with a probabilistic model to transform microtext into its standard form. The findings suggest that a semiotic class of microtext was easy to transform, whereas the phonetic class of microtext was not handled easily. The proposed model improves the pre-trained sentiment analysis model by 4%. We also introduce a corpus for English microtext normalization. The corpus contains microtext-containing sentences along with their correctly spelt sentences and their correct polarity. This dataset can tackle two NLP problems, i.e., microtext normalization and sentiment analysis. This corpus consists of 38% positive, 39.9% negative and 22.1% neutral. A pre-trained model shows an accuracy of 46.4 % on OOV text for sentiment analysis, showing considerable room for improvement. In summary, this thesis introduces several approaches to microtext normalization analysis and focuses on sentiment analysis application. Microtext cannot be thought of as a one-dimensional problem to NLP. It encapsulates many factors like regional (geographical), ethnic (national and racial), and social (class, age, gender, socioeconomic status and education), which cannot be ignored given the growing importance of social media in our daily life.

IntroductionSince the Coronavirus disease 2019 (COVID-19), public safety measures, including social distancing and school closures, have been implemented, precipitating psychological difficulties and heightened online activities for adolescents. However, studies examining the impact of the pandemic on adolescent mental health and their coping strategies in Asian countries are limited. Further, most studies have used survey measures to capture mental health challenges so far. Accordingly, this study aimed to examine the psychological challenges South Korean adolescents experienced and their coping strategies during the pandemic using the Natural Language Processing (NLP) and Text mining (TM) technique on adolescents’ social media texts/posts.MethodsThe data were gathered from social media texts/posts such as online communities, Twitter, and personal blogs from January 1, 2019, to October 31, 2021. The 12,520,250 texts containing keywords related to adolescents’ common psychological difficulties reported during the pandemic, including self-harm, Attention-Deficit/Hyperactivity Disorders (ADHD), and insomnia, were analyzed by TM, NLP using information extraction, co-occurrence and sentiment analysis. The monthly frequency of the keywords and their associated words was also analyzed to understand the time trend.ResultsAdolescents used the word “self-harm” in their social media texts more frequently during the second wave of COVID-19 (August to September 2020). “Friends” was the most associated word with “self-harm.” While the frequency of texts with “Insomnia” stayed constant throughout the pandemic, the word “ADHD” was increasingly mentioned in social media. ADHD and insomnia were most frequently associated with ADHD medications and sleeping pills, respectively. Friends were generally associated with positive words, while parents were associated with negative words.ConclusionDuring COVID-19, Korean adolescents often expressed their psychological challenges on social media platforms. However, their coping strategies seemed less efficient to help with their difficulties, warranting strategies to support them in the prolonged pandemic era. For example, Korean adolescents shared psychological challenges such as self-harm with friends rather than their parents. They considered using medicine (e.g., sleeping pills and ADHD medication) as coping strategies for sleep and attention problems.

Everyone in this rapid evolution generation makes use of online resources. There is a huge amount of textual content available online from several sources, including social media apps, e-commerce sites, movie recommendation system, etc. Everyone in the modern society provides a variety of reviews for a specific product or item. An effective and smart technique is required that can analyze and provide the polarity of this textual data. since it is impossible to manually review such a vast volume of user-generated data. For this user-generated data, automatic sentiment classification is possible using a variety of tools and methods. Opinion mining and sentiment classification benefit from the dependable and accuracy of machine learning. Moreover, the sentiment analysis and evaluation process are fraught with difficulties. These complexities make it difficult to interpret sentiments accurately and figure out the right sentiment polarity. This article describes an overview on sentiment analysis(SA) applications, challenges, and issues encountered by researchers, with the goal of providing a systematic review on sentiment analysis and related areas for future research.

With the development and emergence of the concept of the Web, as well as the advancement of social networks, forums, blogs and E-commerce a growing number of Arabs express their views and opinions using their native language on those platforms where we find a wealth of information that could be exploited. This information, whose size is expanding and which describes opinions, has become the subject of several research fields particularly: “Sentiment Analysis” or “Opinion Mining”. Sentiment Analysis considered a subfield of natural language processing (NLP), information retrieval and text mining. Currently, sentiment analysis is commonly used for English, while it is in its infancy in the Arabic language. The most important and critical step in feeling analysis is to select an appropriate technique for classifying feelings. In this paper, we will be reviewing the published work on the application of Deep Learning (DL) techniques to the problems of sentiment analysis. We focus on the Arabic language because of its importance, the large number of its speakers, the scarcity of Arabic resources on it and the challenges related to its work. We explain, categorize, summarize and compare the techniques offered in this area.

Audio sentiment analysis is a rapidly evolving field that seeks to uncover emotional insights from audio signals, with broad applications in technology, healthcare, and user experience design. This paper presents a detailed review of recent advancements, highlighting key techniques, ongoing challenges, and real-world applications that illustrate the potential of this technology. The Literature Review provides an integrated analysis of significant findings across various studies, identifying core trends and unique contributions. Finally, the Conclusion explores promising future research directions, laying the groundwork for continued innovation in extracting sentiment from sound. Keywords Audio Sentiment Analysis, Sentiment Classification, Deep Learning Techniques, Feature Extraction, Transfer Learning Ensemble Methods, Multi-Task Learning, Challenges in Sentiment Analysis, Applications of Sentiment Analysis, Literature Review of Audio Sentiment Analysis

The sentiment analysis process that gives this work its name is the main theme of the work. Since the beginning of 2000, sentiment analysis has become one of the most active research areas by researchers working on natural language processing and social networking analysis. In addition, data mining, web mining, and text mining are also studied extensively. Moreover, the method of sentiment analysis has spread over many fields, from computer science to management science, from social science to economics, due to the importance given to the business world as a whole and the collectivity. In this study, Konstanz Information Miner (KNIME), which is a powerful data mining tool with its richest features and many visualization tools, was used on twitter data. Ten thousand Twitter data were used in this study. The sentiment analysis study, which is in fact a classification study, was conducted using machine learning algorithms on Twitter data. The results of the study were interpreted by carrying out an accuracy analysis. It is anticipated that the use of the KNIME, which has rich visualization tools, will be widespread in sentiment analysis studies to make these works both easier and more reliable.

Nowadays, with the increasing number and use of social media platforms, people now share their experiences about a product they have bought or a place they have been to on social media platforms more frequently. Considering the volume of data on social media platforms, it is considered that there is some meaningful information for institutions or companies in the reviews and experiences shared on social media platforms. As such, it is important to improve the methods of extracting meaningful information from the reviews and experiences shared on social media and to know which method is better. In this study, the classification successes of the bag of words and the fastText word representation methods, which are among the word representation methods in sentiment analysis methods mentioned above, were compared by using Turkish reviews performed for touristic places. Besides, while performing the comparison process, it was measured whether the process of separating the words into their roots and negation of the words, which is the preliminary stage of the sentiment analysis process, contributed to the classification success. In the study, both two-class (positive, negative) sentiment analysis and three-class (positive, negative, neutral) sentiment analysis were performed. Six data sets were created to carry out the mentioned comparison operations. The data sets were first classified using the Naive Bayes (NB), Multinomial Naive Bayes (MNB), k-Nearest Neighbor (k-NN) and Support Vector Machines (SVM) algorithms, which are frequently used in text mining, and based on bag of words word representation method, they were classified with WEKA program. After the test results of all data sets were obtained according to the bag of words word representation method, the tests of the fastText word representation method were carried out using the fastText library of the Python programming language. Classification procedures were carried out with 10-fold cross-validation methods, and f-score values of the classification processes were obtained. Finally, it was determined that bag of words word representation method performed a more successful classification than the fastText word representation method in two-class emotion analysis, while the fastText word representation method performed a more successful classification process than bag of words word representation method in three-class emotional analysis. It was observed that the process of separating the words into their roots and negating the words, which are the preliminary processes of sentiment analysis, did not contribute positively or negatively to the classification processes performed with the fastText word representation method. However, it was determined that it had a minor contribution to sentiment analysis processes performed by using bag of words word representation method. In the two-class sentiment analysis, the most successful classification result was achieved by using the machine learning model created with the SVM algorithm with the value of 0.91 f-score employing bag of words word representation method. In the three-class sentiment analysis, the most successful classification result was achieved with the machine learning model created using the fastText word representation method with the value of 0.78 f-score.

Sentiment analysis, a critical task in natural language processing, aims to automatically identify and classify the sentiment expressed in textual data. Aspect-level sentiment analysis focuses on determining sentiment at a more granular level, targeting specific aspects or features within a piece of text. In this paper, we explore various techniques for sentiment analysis, including traditional machine learning approaches and state-of-the-art deep learning models. Additionally, deep learning techniques has been utilized to identifying and extracting specific aspects from text, addressing aspect-level ambiguity, and capturing nuanced sentiments for each aspect. These datasets are valuable for conducting aspect-level sentiment analysis. In this article, we explore a language model based on pre-trained deep neural networks. This model can analyze sequences of text to classify sentiments as positive, negative, or neutral without explicit human labeling. To evaluate these models, data from Twitter's US airlines sentiment database was utilized. Experiments on this dataset reveal that the BERT, RoBERTA and DistilBERT model outperforms than the ML based model in accuracy and is more efficient in terms of training time. Notably, our findings showcase significant advancements over previous state-of-the-art methods that rely on supervised feature learning, bridging existing gaps in sentiment analysis methodologies. Our findings shed light on the advancements and challenges in sentiment analysis, offering insights for future research directions and practical applications in areas such as customer feedback analysis, social media monitoring, and opinion mining.

Even though sociolinguistic and psycholinguistic studies demonstrate that the qualities of the speaker and the intended audience have an impact on the message that is transmitted, the majority of NLP models used today assume language as universal. The difficulty of sentiment analysis, often known as opinion mining, in natural language processing (NLP) is one that is currently being faced. It’s also challenging for people to share their ideas on websites like Facebook, Twitter, and Yelp since innovation has grown dramatically. Social media’s widespread use has resulted in a huge volume of data generation, including evaluations, comments, and opinions. Researchers claim that the phrase, document, aspect, and user levels are the foundations for sentiment analysis. This data requires more time to analyse and is challenging to process. In order to categorize or determine positive, negative, and neutral opinions, an intelligent system must be developed.

This paper presents an application of sentiment analysis on customer feedback data in the area of heavy equipment repair services. Sentiment analysis is used as a part of a framework for text mining-based Customer Loyalty Improvement Recommender System (CLIRS). In order to provide business users of the system with accurate predictions for customer satisfaction metrics, the original algorithm for the opinion mining needed to be improved. The paper presents the background of the proposed approach, the current techniques used to mine text data and existing applications of sentiment analysis. We propose an aspect-based, taxonomy-driven approach for customized sentiment analysis. The contribution of this paper is the implementation and evaluation of the proposed methods that improve the accuracy and coverage of the opinion mining algorithm. The improvements are illustrated with examples covered by the algorithm in the customer dataset. The application of the proposed methods resulted in increasing the algorithm’s accuracy from 92% to 96%, and coverage from 36% to 48%. This research is an attempt to handle well-known issues in natural language processing that are currently not handled by text mining algorithms, such as ambiguity and context, opinionated verbs/nouns, subject recognition from pronouns. This is significant because the proposed techniques are generalizable to any application that uses sentiment analysis algorithm.

Sentiment analysis has become a powerful tool in processing and analysing expressed opinions on a large scale. While the application of sentiment analysis on English-language content has been widely examined, the applications on the Russian language remains not as well-studied. In this survey, we comprehensively reviewed the applications of sentiment analysis of Russian-language content and identified current challenges and future research directions. In contrast with previous surveys, we targeted the applications of sentiment analysis rather than existing sentiment analysis approaches and their classification quality. We synthesised and systematically characterised existing applied sentiment analysis studies by their source of analysed data, purpose, employed sentiment analysis approach, and primary outcomes and limitations. We presented a research agenda to improve the quality of the applied sentiment analysis studies and to expand the existing research base to new directions. Additionally, to help scholars selecting an appropriate training dataset, we performed an additional literature review and identified publicly available sentiment datasets of Russian-language texts.

Sentiment analysis is an important task in natural language processing, which has promises great value to areas of interests such as business, politics and other fields. The prevalence of the internet has caused people to prefer expressing their opinion and sentiment on the Internet via methods such as tweeting on social media and commenting on products. However, the discourse of users on social media are usually short either due to limitations on post size or time limitations of the users. What emerges from these features is that the grammar used in these posts and the word meanings are flexible, which makes sentiment analysis difficult. Therefore, sentiment analysis is an important yet challenging task. In this paper, we propose a context-based regularization classification method for short text sentiment analysis. Specifically, we use contextual knowledge obtained from the data to improve performance of the sentiment classification. In this paper, the contextual knowledge includes two parts: word-sentiment knowledge and word-similarity knowledge. Moreover, we propose methods to calculate the sentiment of the words and the similarity between words on semantic level. Specifically, on the one side, we use a TRSR method based on the TextRank algorithm to rank words in each sentiment sample to determine the sentiment polarity of each word. On the other, we calculate the similarity between words according to word-embedding. In this way, we can determine the similarity between words and the sentiment polarity of a word. We then incorporate the contextual knowledge as a regularization into a supervised classification framework, which then converts into an optimization problem to train a more accurate model. Experiments on both Chinese and English datasets outperform than other baseline approaches, which demonstrates our method to be stable and effective.

People have more channels to express their opinions and feelings about events, products, and celebrities because of the development of social networks. They are becoming rich data sources, gaining attention for many practical applications and in the field of research. Sentiment analysis (SA) is one of the most common uses of this data source. Of the currently available SA datasets, most are only suitable for use in SA corresponding to a specific level, such as document, sentence, or aspect levels. This renders it difficult to develop practical systems that require a combination of sentiment analyzes at all three levels. Additionally, the previous datasets included opinions on only a single domain, although many people often mention multiple domains when expressing their views. This study introduces a new dataset called multi-level and multi-domain (M2SA) for SA. Each sample in M2SA contains a short text with at least two sentences and two aspects with different domains and sentiment polarities. The release of the M2SA dataset will contribute to the promotion of research in the field of SA, primarily by promoting the development and improvement of methods for multi-level SA or multi-aspect, multi-domain SA. The M2SA dataset was tested using state-of-the-art SA methods and was compared with other standard datasets. The results demonstrate that the M2SA dataset is better than the previous datasets in supporting to improve of the performance of SA methods.

Recognizing and categorizing user’s sentiments from a part of text into different sentiments is known as sentiment analysis. For instance, emotions such as happy, sad, angry or positive, negative or neutral to determine the users attitude concerning a certain subject or object. Sentiment analysis is one of the utmost active research areas in natural language processing, web mining and text mining. It plays an important role in many fields like management sciences and social sciences including education, where student feedback is essential to assess the effectiveness of learning technologies. With increase in educational organizations, online learning portals have fascinated by many students by offering free courses with no fee. Heaps of learners enroll in these massive online courses every year and further review their sentiments about the course content and quality of education. Also, provide suggestions in blogs in order to improve the quality of teaching by giving positive or negative sentiments. This paper proposes a model based on Deep Learning approach to perform sentiment analysis on Educational data. In this paper we focused on the accuracy and performance of the training data set to predict the best model. MLP and SVM are recognized as the outperforming models.