EXPLORING SENTIMENT ANALYSIS RESEARCH: A SOCIAL MEDIA DATA PERSPECTIVE

In recent years, the utilization of Internet has turned out to be one of the everyday activities in our life. Social networks constitute a noteworthy segment of the Web and made an upheaval. It incorporates social media, forum conversations, blogs and micro-blogs like twitter. Due to this, large numbers of comments are produced on daily basis. So, nowadays most of the researchers or analyzers are concentrating on extracting significant data from social networks in order to understand the public viewpoint. This research has been reached out outside the computer science to cover other areas like business, political and social science. Hence, Sentiment analysis and Opinion mining are popular field of research in Data mining. This paper delineates various aspects of sentiment analysis in detail inclusive of important concepts, classification, process, importance, challenges and applications. The following paper presents experiment on sentiment analysis of public opinion on demonetization in India. Sentiment analysis is performed on tweets related to demonetization in India extracted from twitter. Polarity of the opinion is observed through the experimental analysis. Through the outcome of this analysis, the sentiments of the citizens that are determined help the government in improving their decisions and work for the welfare of the citizens.

The rapid growth of the video game industry and its reliance on digital distribution have created new opportunities for data-driven sales forecasting. Social media platforms serve as influential environments where consumer sentiment, trends, and discussions impact purchasing behaviors. This study examines the potential of using sentiment analysis of social media data to predict video game sales. While traditional sales forecasting models mainly depend on historical sales data and statistical techniques, sentiment analysis offers real-time insights into consumer interest and market demand. This paper reviews existing research on video game sales prediction, the application of sentiment analysis in the gaming industry, and sentiment-based forecasting models in other domains. The findings highlight a significant research gap in applying sentiment analysis to video game sales forecasting, despite its demonstrated efficacy in related fields. The study emphasizes the advantages and challenges of integrating sentiment analysis with traditional forecasting methods and proposes future research directions to enhance predictive accuracy.

This thesis examines the issue of ill-formed text (microtext) and shows its impact on sentiment analysis. The source of microtext is Twitter, WhatsApp, Facebook and other social media platforms. Words or phrases not in their standard language format like “lol” (laugh out loud), “c u 2nite” (see you tonight) and “plz” (please) are called out of vocabulary (OOV) terms. On the other hand, words or phrases in their standard forms, for example, “tonight”, “love”, or “talk to you later”, are called In Vocabulary (IV) terms. The motivation behind microtext normalization is its growing presence on both online and offline platforms. Microtext might appear to be an ill-formed text, which requires find-and-replace methods. Traditionally, in a Natural Language Processing (NLP) context, microtexts such as abbreviations and length shortening or extending are normalized, but phonemic and graphemic variation is often ignored. However, the features of microtext can be traced back to social, cognitive and geographic influences. To understand social media text, it is essential to develop a microtext normalization module for traditional and modern NLP models. The first section of this thesis focuses on combining unsupervised learning methods in 2 different chapters, 4 and 5. The first method includes lexicon creation to transform the most frequent microtexts and emoticons on social media and applies them to sentiment analysis. This method shows an accuracy increase of 4% when microtext normalization is applied before sentiment analysis. This method is extended with an application to a pre-trained chatbot to improve the chatbot’s understanding of microtext (social media slangs). This method shows a mean BLEU score of 0.8 for the SMS and Tweet dataset. The second method incorporates a phonetic based approach to transform any microtext into its phonetic equivalent before 1normalizing it. This method shows an accuracy improvement of 6% for sentiment analysis on SenticNet. The second part of the unsupervised learning is in Chapter 5, introducing the IPA-based method for microtext normalization and sentiment analysis. Epitran was used to transform a word to its International Phonetic Alphabet (IPA) equivalent. The International Phonetic Alphabet (IPA) is an alphabetic system of phonetic notation based primarily on the Latin script. There is a significant improvement in microtext normalization task and sentiment analysis over Soundex. The results also show that there is very little redundancy when words are transformed to phonetics in IPA. In addition, the accuracy of polarity detection using this method has an improvement over baseline by 5%. Chapter 6 discusses a simple sequence-to-sequence based Deep Learning method. As the language is dynamic, it is not easy to maintain the lexicon. Deep Learning alleviates the static lexicon with a probabilistic model to transform microtext into its standard form. The findings suggest that a semiotic class of microtext was easy to transform, whereas the phonetic class of microtext was not handled easily. The proposed model improves the pre-trained sentiment analysis model by 4%. We also introduce a corpus for English microtext normalization. The corpus contains microtext-containing sentences along with their correctly spelt sentences and their correct polarity. This dataset can tackle two NLP problems, i.e., microtext normalization and sentiment analysis. This corpus consists of 38% positive, 39.9% negative and 22.1% neutral. A pre-trained model shows an accuracy of 46.4 % on OOV text for sentiment analysis, showing considerable room for improvement. In summary, this thesis introduces several approaches to microtext normalization analysis and focuses on sentiment analysis application. Microtext cannot be thought of as a one-dimensional problem to NLP. It encapsulates many factors like regional (geographical), ethnic (national and racial), and social (class, age, gender, socioeconomic status and education), which cannot be ignored given the growing importance of social media in our daily life.

Sentiment analysis is a new area of research in data mining that concerns the detection of opinions and/or sentiments in texts. This work focuses on the application and the comparison of three classification techniques over a text corpus composed of reviews of commercial products in order to detect opinions about them. The chosen domain is about "perfumes", and user opinions composing the corpus are written in Italian language. The proposed approach is completely data-driven: a Term Frequency / Inverse Document Frequency (TFIDF) terms selection procedure has been applied in order to make computation more efficient, to improve the classification results and to manage some issues related to the specific classification procedure adopted.

Sentiment analysis has become a powerful tool in processing and analysing expressed opinions on a large scale. While the application of sentiment analysis on English-language content has been widely examined, the applications on the Russian language remains not as well-studied. In this survey, we comprehensively reviewed the applications of sentiment analysis of Russian-language content and identified current challenges and future research directions. In contrast with previous surveys, we targeted the applications of sentiment analysis rather than existing sentiment analysis approaches and their classification quality. We synthesised and systematically characterised existing applied sentiment analysis studies by their source of analysed data, purpose, employed sentiment analysis approach, and primary outcomes and limitations. We presented a research agenda to improve the quality of the applied sentiment analysis studies and to expand the existing research base to new directions. Additionally, to help scholars selecting an appropriate training dataset, we performed an additional literature review and identified publicly available sentiment datasets of Russian-language texts.

Data mining (DM) research has successfully developed advanced DM techniques and algorithms over the last few decades, and many organisations have great expectations to take more benefit of their data warehouses in decision making. Currently, the strong focus of most DM-researchers is still only on technology-oriented topics. Commonly the DM research has several stakeholders, the major of which can be divided into internal and external ones each having their own point of view, and which are at least partly conflicting. The most important internal groups of stakeholders are the DM research community and academics in other disciplines. The most important external stakeholder groups are managers and domain experts who have their own utility-based interests to DM and DM research results. In this paper we discuss these practice-oriented points of view towards DM research and suggest broader discussions inside the DM research community about who should do that kind of research. We bring in the discussion several topics developed in the information systems (IS) discipline and show some similarities between IS and DM systems. DM systems have also their own peculiarities and we conclude that researchers who take into account human and organisational aspects related to DM systems need to have also some understanding about DM. This makes us suggest that the research area inside the DM community should be made broader than the current heavily technology-oriented one.

Arabic language suffers from the lack of available large datasets for machine learning and sentiment analysis applications. This work adds to the recently reported large dataset BRAD, which is the largest Book Reviews in Arabic Dataset. In this paper, we introduce HARD (Hotel Arabic-Reviews Dataset), the largest Book Reviews in Arabic Dataset for subjective sentiment analysis and machine language applications. HARD comprises of 490587 hotel reviews collected from the Booking.com website. Each record contains the review text in the Arabic language, the reviewer’s rating on a scale of 1 to 10 stars, and other attributes about the hotel/reviewer. We make available the full unbalanced dataset as well as a balanced subset. To examine the datasets, we implement six popular classifiers using Modern Standard Arabic (MSA) as well as Dialectal Arabic (DA). We test the sentiment analyzers for polarity and rating classifications. Furthermore, we implement a polarity lexicon-based sentiment analyzer. The findings confirm the effectiveness of the classifiers and the datasets. Our core contribution is to make this benchmark-dataset available and accessible to the research community on Arabic language.

Data mining is one of the most important steps of the knowledge discovery in databases process and is considered as significant subfield in knowledge management. Research in data mining continues growing in business and in learning organization over coming decades. This review paper explores the applications of data mining techniques which have been developed to support knowledge management process. The journal articles indexed in ScienceDirect Database from 2007 to 2012 are analyzed and classified. The discussion on the findings is divided into 4 topics: (i) knowledge resource; (ii) knowledge types and/or knowledge datasets; (iii) data mining tasks; and (iv) data mining techniques and applications used in knowledge management. The article first briefly describes the definition of data mining and data mining functionality. Then the knowledge management rationale and major knowledge management tools integrated in knowledge management cycle are described. Finally, the applications of data mining techniques in the process of knowledge management are summarized and discussed.

The purpose of this study is to learn how people who speak different languages interpret the same issues, and to compare the results obtained and show the difference between their perspectives. To learn this point of view, we must first turn to open source intelligence. In this execution, a sentiment analysis application was designed using the Python programming language and the Natural Language Processing algorithms in the texts, which were taken as a data set of comments in Azerbaijani, Turkish, Russian and English languages from social media. As the data set, the comments made on 4 subjects: the declaration of Hagia Sophia as a mosque, the objection events that started with the natural gas hike in Kazakhstan, the natural disasters in Turkey, the Ukraine crisis. After loading the texts in four languages from the network environment, after preprocessing, the text was divided into 8 different categories (neutral, fear, joy, anger, sadness, surprise, disgust, shame) by means of the application written in Python programming language based on Data Mining and Machine Learning topics. In the study, precision, sensitivity, accuracy and F1 score were obtained by using Random Decision Forests, K - Near Neighbor Algorithm, Decision Trees, Support Vector Machine, Naive Bayes Algorithm, Logistic Regression, which are machine learning methods. By comparing the results, it was determined that the Logistic Regression method obtained the highest result. A sentiment analysis model was created using the Logistic Regression method, and sentiment analysis was performed for each subject at separation and the results were compared.

Sentiment Analysis in Social Media and Its Application: Systematic Literature Review

The Outlier detection is very active area of research in data mining where outlier is a mismatched data in dataset with respect to the other available data.In existing approaches the outlier detection done only on numeric dataset.For outlier detection if we use clustering method , then they mainly focus on those elements as outliers which are lying outside the clusters but it may possible that some of the unknown elements with any possible reasons became the part of the cluster so we have to concentrate on that also.The Proposed method uses hybrid approach to reduce the number of outliers.The number of outlier can only reduce by improving the cluster formulation method.The proposed method uses two data mining techniques for cluster formulation i.e. weighted k-means and neural network where weighted kmeans is the clustering technique that can apply on text and date data set as well as numeric data set.Weighted kmeans assign the weights to each element in dataset.The output of weighted k-means becomes the input for neural network where the neural network is the classification and clustering technique of data mining.Training is provided to the neural network and according to that neurons performed the testing.The neural network test the cluster formulated by weighted k-means to ensure that the clusters formulated by weighted k-means are group accordingly.There is lots of outlier detection methods present in data mining.The proposed method use Integrating Semantic Knowledge (SOF) for outlier detection.This method detects the semantic outlier where the semantic outlier is a data point that behaves differently with other data points in the same class or cluster.The main motive of this research work is to reduce the number of outliers by improving the cluster formulation methods so that outlier rate reduces and also to decrease the mean square error and improve the accuracy.The simulation result clearly shows that proposed method works pretty well as it significantly reduces the outlier.

PurposeDespite the extensive academic interest in social media sentiment for financial fields, multimodal data in the stock market has been neglected. The purpose of this paper is to explore the influence of multimodal social media data on stock performance, and investigate the underlying mechanism of two forms of social media data, i.e. text and pictures.Design/methodology/approachThis research employs panel vector autoregressive models to quantify the effect of the sentiment derived from two modalities in social media, i.e. text information and picture information. Through the models, the authors examine the short-term and long-term associations between social media sentiment and stock performance, measured by three metrics. Specifically, the authors design an enhanced sentiment analysis method, integrating random walk and word embeddings through Global Vectors for Word Representation (GloVe), to construct a domain-specific lexicon and apply it to textual sentiment analysis. Secondly, the authors exploit a deep learning framework based on convolutional neural networks to analyze the sentiment in picture data.FindingsThe empirical results derived from vector autoregressive models reveal that both measures of the sentiment extracted from textual information and pictorial information in social media are significant leading indicators of stock performance. Moreover, pictorial information and textual information have similar relationships with stock performance.Originality/valueTo the best of the authors’ knowledge, this is the first study that incorporates multimodal social media data for sentiment analysis, which is valuable in understanding pictures of social media data. The study offers significant implications for researchers and practitioners. This research informs researchers on the attention of multimodal social media data. The study’s findings provide some managerial recommendations, e.g. watching not only words but also pictures in social media.

Noting that several rule discovery algorithms in data mining can produce a large number of irrelevant or obvious rules from data, there has been substantial research in data mining that addressed the issue of what makes rules truly 'interesting'. This resulted in the development of a number of interestingness measures and algorithms that find all interesting rules from data. However, these approaches have the drawback that many of the discovered rules, while supposed to be interesting by definition, may actually (1) be obvious in that they logically follow from other discovered rules or (2) be expected given some of the other discovered rules and some simple distributional assumptions. In this paper we argue that this is a paradox since rules that are supposed to be interesting, in reality are uninteresting for the above reason. We show that this paradox exists for various popular interestingness measures and present an abstract characterization of an approach to alleviate the paradox. We finally discuss existing work in data mining that addresses this issue and show how these approaches can be viewed with respect to the characterization presented here.

Evidence-based medicine is an important initiative of the U.S. National Institutes of Health. However, published data are limited. Medical knowledge can be advanced through pharmacoepidemiology and data mining research of very large databases, data warehouses. Before Vioxx (rofecoxib) and Celebrex (celecoxib) were publicly suspected of being associated with acute myocardial infarction and stroke, etodolac was evaluated for gastrointestinal safety using a data warehouse constructed from several data sources and data mining tools embedded with pharmacoepidemiologic methods. Results were published in a top tier medical journal, and established gastrointestinal safety of a generic COX-2 analgesic medication. This project demonstrates that the knowledge needed for evidence-based medicine can be augmented through pharmacoepidemiology research using data mining and data warehousing.

The internet’s prevalence has increased the use of social media platforms for sharing peoples’ views, reviews, opinions, and sentiments. Consequently, sentiment analysis has become valuable to marketing, brand management, public policy, healthcare, and e-commerce. Panic detection systems refer to a system that utilizes sentiment analysis techniques to identify, quantify, and potentially track expressions of panic, fear, or related strong negative emotions within digital content. Sentiment analysis application can indicate a person’s attitude or cultural influences. The rise of Arabic digital content on social platforms presents unique challenges and opportunities for sentiment analysis. There are fewer Arabic sentiment lexicons, emotional connotations, and annotated corpora than in English. Therefore, this study applies machine learning and deep learning to identify sentiments within the IRAQI-Arabic panic detection systems. The proposed model is based on Residual Network (ResNet), MobileNet, and Convolutional Neural Network (CNN). A web scraping API collected COVID-19-related news from Google, Facebook, Twitter, and BBC News. The model used was retrained on the images and comments based on training, public, and private sub-sets. Bidirectional Encoder Representations from Transformers (BERT) were applied across SMOTE, Random oversampling, five-folds, and ten folds. The results show that the accuracy of BERT is 91%. The proposed MobileNet model had an accuracy of 0.98, recall of 0.98, and MAE of 0.032. The model’s performance confirms that it is suitable for critical applications that require high-level precision. The study offers a novel model for Arabic panic systems. Also, it presents an Iraqi-Arabic dataset tailored for panic detection.