Study of Twitter Sentiment Analysis using Machine Learning Algorithms on Python

This thesis examines the issue of ill-formed text (microtext) and shows its impact on sentiment analysis. The source of microtext is Twitter, WhatsApp, Facebook and other social media platforms. Words or phrases not in their standard language format like “lol” (laugh out loud), “c u 2nite” (see you tonight) and “plz” (please) are called out of vocabulary (OOV) terms. On the other hand, words or phrases in their standard forms, for example, “tonight”, “love”, or “talk to you later”, are called In Vocabulary (IV) terms. The motivation behind microtext normalization is its growing presence on both online and offline platforms. Microtext might appear to be an ill-formed text, which requires find-and-replace methods. Traditionally, in a Natural Language Processing (NLP) context, microtexts such as abbreviations and length shortening or extending are normalized, but phonemic and graphemic variation is often ignored. However, the features of microtext can be traced back to social, cognitive and geographic influences. To understand social media text, it is essential to develop a microtext normalization module for traditional and modern NLP models. The first section of this thesis focuses on combining unsupervised learning methods in 2 different chapters, 4 and 5. The first method includes lexicon creation to transform the most frequent microtexts and emoticons on social media and applies them to sentiment analysis. This method shows an accuracy increase of 4% when microtext normalization is applied before sentiment analysis. This method is extended with an application to a pre-trained chatbot to improve the chatbot’s understanding of microtext (social media slangs). This method shows a mean BLEU score of 0.8 for the SMS and Tweet dataset. The second method incorporates a phonetic based approach to transform any microtext into its phonetic equivalent before 1normalizing it. This method shows an accuracy improvement of 6% for sentiment analysis on SenticNet. The second part of the unsupervised learning is in Chapter 5, introducing the IPA-based method for microtext normalization and sentiment analysis. Epitran was used to transform a word to its International Phonetic Alphabet (IPA) equivalent. The International Phonetic Alphabet (IPA) is an alphabetic system of phonetic notation based primarily on the Latin script. There is a significant improvement in microtext normalization task and sentiment analysis over Soundex. The results also show that there is very little redundancy when words are transformed to phonetics in IPA. In addition, the accuracy of polarity detection using this method has an improvement over baseline by 5%. Chapter 6 discusses a simple sequence-to-sequence based Deep Learning method. As the language is dynamic, it is not easy to maintain the lexicon. Deep Learning alleviates the static lexicon with a probabilistic model to transform microtext into its standard form. The findings suggest that a semiotic class of microtext was easy to transform, whereas the phonetic class of microtext was not handled easily. The proposed model improves the pre-trained sentiment analysis model by 4%. We also introduce a corpus for English microtext normalization. The corpus contains microtext-containing sentences along with their correctly spelt sentences and their correct polarity. This dataset can tackle two NLP problems, i.e., microtext normalization and sentiment analysis. This corpus consists of 38% positive, 39.9% negative and 22.1% neutral. A pre-trained model shows an accuracy of 46.4 % on OOV text for sentiment analysis, showing considerable room for improvement. In summary, this thesis introduces several approaches to microtext normalization analysis and focuses on sentiment analysis application. Microtext cannot be thought of as a one-dimensional problem to NLP. It encapsulates many factors like regional (geographical), ethnic (national and racial), and social (class, age, gender, socioeconomic status and education), which cannot be ignored given the growing importance of social media in our daily life.

In the digital world, social media has become one of the most popular communication mediums that allow users to share their views on various topics in their social network. For example, Twitter users are allowed to share their thoughts on various topic by sending tweets with a maximum length of 140 characters. Hence, social media driven information contains opinions and sentiments on various topics of interest which are extremely useful for companies to design marketing strategies. Sentiment Analysis is widely used to assist people to understand the massive amount of data available online and identify the polarity of the topical based social media opinions. However, social media platforms’ users come from all over the world and have variation in terms of informal language and short notation used on social media platforms. Therefore, the identification on the polarity of topical social media has become more challenging and the accuracy on the polarity of topical social media opinions might be influenced. This paper investigates the effectiveness of applying different spelling correction algorithms, such as Levenshtein distance and Peter Norvig’s algorithm for spelling correction of unknown words found in social media such as Twitter, before carrying out sentiment analysis. The evaluation of spelling correction algorithms on sentiment analysis is carried out by comparing the polarities of manually annotated tweets with the polarities obtained from the sentiment analysis algorithm. Based on the results obtained, there are slight improvements in term of percentage of matched polarity, where 1.6% improvement by using the Levenshtein distance-based algorithm and 2.0% improvement by using the Peter Norvig’s algorithm.

Navigating Social Media in #Ophthalmology

Sentiment analysis is a popular technique to identify the sentiment of a piece of text. Several different domains have been targeted by sentiment analysis research, such as Twitter, movie reviews, and mobile app reviews. Although several techniques have been proposed, the performance of current sentiment analysis techniques is still far from acceptable, mainly when applied in domains on which they were not trained. In addition, the causes of wrong classifications are not clear. In this article, we study how sentiment analysis performs on game reviews. We first report the results of a large-scale empirical study on the performance of widely used sentiment classifiers on game reviews. Then, we investigate the root causes for the wrong classifications and quantify the impact of each cause on the overall performance. We study three existing classifiers: <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Stanford CoreNLP</monospace> , <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NLTK</monospace> , and <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SentiStrength</monospace> . Our results show that most classifiers do not perform well on game reviews, with the best one being <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NLTK</monospace> (with an AUC of 0.70). We also identified four main causes for wrong classifications, such as reviews that point out advantages and disadvantages of the game, which might confuse the classifier. The identified causes are not trivial to be resolved and we call upon sentiment analysis and game researchers and developers to prioritize a research agenda that investigates how the performance of sentiment analysis of game reviews can be improved, for instance by developing techniques that can automatically deal with specific game-related issues of reviews (e.g., reviews with advantages and disadvantages). Finally, we show that training sentiment classifiers on reviews that are stratified by the game genre is effective.

Nowadays, with the increasing number and use of social media platforms, people now share their experiences about a product they have bought or a place they have been to on social media platforms more frequently. Considering the volume of data on social media platforms, it is considered that there is some meaningful information for institutions or companies in the reviews and experiences shared on social media platforms. As such, it is important to improve the methods of extracting meaningful information from the reviews and experiences shared on social media and to know which method is better. In this study, the classification successes of the bag of words and the fastText word representation methods, which are among the word representation methods in sentiment analysis methods mentioned above, were compared by using Turkish reviews performed for touristic places. Besides, while performing the comparison process, it was measured whether the process of separating the words into their roots and negation of the words, which is the preliminary stage of the sentiment analysis process, contributed to the classification success. In the study, both two-class (positive, negative) sentiment analysis and three-class (positive, negative, neutral) sentiment analysis were performed. Six data sets were created to carry out the mentioned comparison operations. The data sets were first classified using the Naive Bayes (NB), Multinomial Naive Bayes (MNB), k-Nearest Neighbor (k-NN) and Support Vector Machines (SVM) algorithms, which are frequently used in text mining, and based on bag of words word representation method, they were classified with WEKA program. After the test results of all data sets were obtained according to the bag of words word representation method, the tests of the fastText word representation method were carried out using the fastText library of the Python programming language. Classification procedures were carried out with 10-fold cross-validation methods, and f-score values of the classification processes were obtained. Finally, it was determined that bag of words word representation method performed a more successful classification than the fastText word representation method in two-class emotion analysis, while the fastText word representation method performed a more successful classification process than bag of words word representation method in three-class emotional analysis. It was observed that the process of separating the words into their roots and negating the words, which are the preliminary processes of sentiment analysis, did not contribute positively or negatively to the classification processes performed with the fastText word representation method. However, it was determined that it had a minor contribution to sentiment analysis processes performed by using bag of words word representation method. In the two-class sentiment analysis, the most successful classification result was achieved by using the machine learning model created with the SVM algorithm with the value of 0.91 f-score employing bag of words word representation method. In the three-class sentiment analysis, the most successful classification result was achieved with the machine learning model created using the fastText word representation method with the value of 0.78 f-score.

A Review on Reddit News Headlines with NLTK tool

Sentiment Analysis in Social Media and Its Application: Systematic Literature Review

The rapid growth of financial technology (fintech) services in Indonesia has significantly transformed digital transaction behavior, with digital wallets such as DANA becoming widely adopted. Despite high usage rates, public discourse on social media frequently highlights both positive experiences and recurring concerns related to system reliability and data security. Twitter, as a real-time public communication platform, provides a rich source of user-generated content that reflects public perception toward fintech services. This study investigates public sentiment toward DANA fintech services in Indonesia through sentiment analysis of Twitter data using machine learning approaches. Purpose:This study aims to analyze public perception of DANA fintech services based on sentiment expressed on Twitter and to compare the performance of several machine learning classification algorithms in identifying positive, neutral, and negative sentiments in Indonesian-language social media texts. Methods/Study design/approach: The study employs a quantitative descriptive approach using text mining and sentiment analysis techniques. Twitter data were collected via the Twitter API over a specified period and processed through preprocessing stages including text cleaning, tokenization, stopword removal, and stemming. Feature extraction was performed using the Term Frequency–Inverse Document Frequency (TF-IDF) method. Sentiment classification was conducted using Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Random Forest algorithms. Model performance was evaluated using accuracy, precision, recall, and F1-score metrics. Result/Findings:The results indicate that public sentiment toward DANA is predominantly positive, driven by perceptions of transaction convenience and speed, followed by neutral sentiment, while negative sentiment is mainly associated with system disruptions and data security issues. Among the evaluated algorithms, SVM achieved the highest classification accuracy (91.4%), outperforming Random Forest and KNN. These findings demonstrate that SVM is more effective for sentiment classification of short Indonesian-language texts on social media platforms. Novelty/Originality/Value:This study contributes to the growing body of sentiment analysis research in Indonesia by providing empirical evidence on public perception of fintech services using machine learning techniques. By systematically comparing multiple classification algorithms in the context of Indonesian Twitter data, the study offers methodological insights and practical implications for fintech service providers in improving service quality, communication strategies, and user trust.

&lt;p&gt;Sentiment Analysis (SA) or Opinion Mining is the process of analysing natural language texts to detect an emotion or a pattern of emotions towards a certain product to make a decision about that product. SA is a topic of text mining, Natural Language Processing (NLP) and web mining disciplines. Research in SA is currently at its peak given the amount of data generated from social media networks. The concept is that consumers are expressing exactly what they need, want and expect from a product but on the other hand the companies don’t have the tools to analyse and understand these feelings to satisfy these consumers accordingly. &lt;/p&gt;&lt;p&gt;One of the applications that generate a high rate of reactions and sentiments in social networks is Instagram. This study focuses on analysing the reactions generated by the top 50 fashion houses on Instagram given their top 20 images with the highest number of likes. The approach taken in this study is to qualify the visual aesthetics of fashion images and to establish why some succeed on social media more than others. &lt;/p&gt;&lt;p class="Els-Abstract-text"&gt;The basic question asked in this paper is whether there are certain visual aesthetics that appeal more to the user and are therefore more successful on social media than others as determined by a measure we introduce, ‘Social Value’. To do so, a sentiment analysis tool is developed to measure the proposed social value of each image. An input of comments from each image will be processed. Each comment will go through a pre-processing phase; each word will be placed through a lexicon to identify if it is positive or negative. The output of the lexicon is a score value assigned to each comment to identify its degree of positivity, negativity, or it has no effect on the social value. Adding to these results, the number of likes and shares would also be taken into consideration quantifying the image’s value. A cumulative result is then produced to determine the social value of an image.&lt;/p&gt;

Understanding and comprehending humans' views, beliefs, attitudes, or opinions toward a particular entity is sentiment analysis (SA). Advancements in e‐commerce platforms has led to an abundance of the real‐time and free forms of opinions floating on social media platforms. This real‐world data are imprecise and vague hence fuzzy logic is required to deal with such subjective data. Since opinions can be fuzzy in nature and definitions of opinion words can be elucidated differently; fuzzy logic has witnessed itself as an effective method to capture the expression of opinions. The study presents an elaborate review of the around 170 published research works for SA using fuzzy logic. The primary emphasis is focused on text‐based SA, audio‐based SA, and fusion of text‐audio features‐based SA. This article discusses the various novel ways of classifying fuzzy logic‐based SA research articles, which have not been accomplished by any other review article till date. The article puts forward the importance of SA tasks and identifies how fuzzy logic adds to this importance. Finally, the article outlines a taxonomy for sentiment classification based on the technique‐supervised and unsupervised in the SA models and comprehensively reviews the SA approaches specific to their task. Prominently, this study highlights the suitability of fuzzy‐based SA approaches into five different classes vis‐a‐vis (a) Sentiment Cognition from Words using fuzzy logic, (b) Sentiment Cognition from Phrases using fuzzy logic, (c) Fuzzy‐rule based SA, (d) Neuro‐fuzzy network‐based SA, and (e) Fuzzy Emotion Recognition.This article is categorized under:Algorithmic Development &gt; Text MiningFundamental Concepts of Data and Knowledge &gt; Motivation and Emergence of Data Mining

BackgroundLoneliness is a global public health issue affecting a considerable number of people as well as burdening the public health system and increasing the risk of other life-threatening and life-damaging conditions. In USA an estimated 17% adults aged 18–70 report loneliness. The monetary loss as result of loneliness is estimated to be between USD 8074.80 and USD 12,0777.70 per person per year in the United Kingdom. But the dynamics of loneliness are not understood. Social media platforms have become a valuable source of data to study this phenomenon. ObjectivesThis paper aims to visualize the frequency of loneliness-related themes and topics in Twitter data. By using natural language (NLP) processing, sentiment analysis, and topic modeling, we seek to understand prevalent sentiments and concerns. Through interactive tree maps and radar plots, we present an engaging view of loneliness dimensions, allowing users to explore and gain insights into this issue on social media. We focus on comparative analysis of USA and India through analyzing tweets from both countries on loneliness. These two countries are the biggest countries population-wise where access to Twitter is legally allowed. MethodsThis study consists of two parts. In the first part, we employ NLP techniques and machine learning algorithms to extract and analyze tweets containing keywords related to loneliness. Through sentiment analysis and topic modeling, we discern linguistic patterns and contextual information to categorize the recurring themes and topics. Advanced text analytics is used to gain nuanced insights into the experiences, emotions, and challenges connected with loneliness. In the second part, interactive visualizations are developed to present the findings in an engaging and intuitive manner. Techniques such as tree maps and radar plots are utilized to transform the analyzed data into visually appealing representations. ResultsThe analysis of Twitter data yields valuable knowledge about the prevalence and nature of themes and topics associated with loneliness. The interactive visualizations present a comprehensive view of the sentiments and concerns expressed by Twitter users. These interactive plots provide a holistic view of the distribution of themes and topics associated with loneliness, allowing experts to explore and interact with the data, gaining deeper insights into the complexities surrounding this issue. ConclusionThis paper successfully explores themes and topics related to loneliness on Twitter by employing NLP, sentiment analysis, and topic modeling. The interactive visualizations enhance the accessibility and usability of the findings, providing valuable insights for various stakeholders. The study contributes to a deeper comprehension of loneliness in the context of social media.

The exponential growth of user-generated content on social media platforms (e.g., Twitter, Weibo, Facebook) has created massive datasets rich in public sentiment. While sentiment analysis proves vital for understanding social trends, brand perception, and political developments, traditional methods struggle to handle the informal nature, noise, and context-dependent characteristics of social media language. This necessitates advanced computational technologies to extract meaningful insights. This paper summarizes the application of Python in social media text sentiment analysis, covering various methods supported by its NLP library (NLTK, TextBlob), machine learning/deep learning framework (TensorFlow, PyTorch), and discusses cross-platform comparative analysis and specific domain adaptation. The results indicate that Python-based models achieve high levels of accuracy. For example, the RoBERTa-BiLSTM-MHA model achieved an accuracy of 93.44%, and these models outperform conventional tools by 612% in terms of F1 scores. Key findings reveal cultural and platform-based disparities in sentiment expression. Specifically, Chinese social media platforms like Weibo emphasize economic sentiment, while Western platforms such as Twitter focus more on technical and ethical implications. The integration of sentiment dictionaries and multimodal data, including emojis, further enhances the robustness of these sentiment analysis models. Overall, this review underscores Pythons versatility in enabling scalable and real-time sentiment analysis, which in turn drives innovations in NLP research and practical applications.

This project aims to develop a machine learning model for sentiment analysis on social media platforms such as Twitter and YouTube and E-commerce websites like Amazon. Social media plays a pivotal role in shaping people’s views and ideas, making it essential to understand the sentiments expressed on these platforms. The primary challenge lies in the vast amounts of unstructured data, the ambiguity in human language, and the diverse range of opinions expressed. Manual analysis of such data is impractical, necessitating the use of machine learning techniques to extract meaningful insights. For businesses, it provides valuable consumer insights that can enhance marketing strategies. Policymakers can benefit by understanding public opinion on welfare schemes and identifying areas for improvement. Individuals can stay informed about emerging trends and issues. The project methodology encompasses several key steps. First, data is collected from diverse sources, including social media posts, comments, and reviews. The data is then preprocessed, involving cleaning, structuring, and labeling for machine learning purposes. Feature extraction transforms text data into numerical representations suitable for modeling. Multiple machine learning models are experimented with to select the one with the highest accuracy. Subsequently, the selected model is trained and evaluated. Finally, a user-friendly web interface is deployed, allowing users to input social media content for sentiment analysis. Deliverables include a trained machine learning model for sentiment analysis, a well-documented codebase, and a user-friendly web application for end-users. This project aims to provide a valuable tool for sentiment analysis on social media platforms, facilitating informed decision-making and enhancing our understanding of public sentiment in the digital age.

Social media data are rapidly evolving and accessible, which presents opportunities for research. Data science techniques, such as sentiment or emotion analysis which analyse textual emotion, provide an opportunity to gather insight from social media. This paper describes a systematic scoping review of interdisciplinary evidence to explore how sentiment or emotion analysis methods alongside other data science methods have been used to examine nutrition, food and cooking social media content. A PRISMA search strategy was used to search nine electronic databases in November 2020 and January 2022. Of 7325 studies identified, thirty-six studies were selected from seventeen countries, and content was analysed thematically and summarised in an evidence table. Studies were published between 2014 and 2022 and used data from seven different social media platforms (Twitter, YouTube, Instagram, Reddit, Pinterest, Sina Weibo and mixed platforms). Five themes of research were identified: dietary patterns, cooking and recipes, diet and health, public health and nutrition and food in general. Papers developed a sentiment or emotion analysis tool or used available open-source tools. Accuracy to predict sentiment ranged from 33·33% (open-source engine) to 98·53% (engine developed for the study). The average proportion of sentiment was 38·8% positive, 46·6% neutral and 28·0% negative. Additional data science techniques used included topic modelling and network analysis. Future research requires optimising data extraction processes from social media platforms, the use of interdisciplinary teams to develop suitable and accurate methods for the subject and the use of complementary methods to gather deeper insights into these complex data.

Social media platforms have become indispensable channels for communication and interaction, offering a wealth of data that can provide valuable insights into public sentiments. However, analyzing sentiment on these platforms poses significant challenges due to the diverse and unstructured nature of user-generated content. Traditional Natural Language Processing (NLP) techniques struggle to accurately classify sentiments expressed through text, images, emoticons, and multimedia elements. Moreover, the informal and nuanced language used in Electronic Word of Mouth (eWOM) further complicates sentiment analysis. In response, this paper explores the role of Artificial Intelligence (AI) in improving sentiment analysis on social media. By leveraging Machine Learning (ML) algorithms trained on large datasets, AI can enhance the accuracy and efficiency of sentiment classification, providing decision-makers with actionable insights into the sentiment landscape of social media. Keywords: Social media, Sentiment analysis, Artificial Intelligence, Machine Learning, Natural Language Processing, Electronic Word of Mouth.