Enhancing Opinion Mining with BERT: A Comparative Study Across Multidomain App Review Data

The rapid growth of financial technology (fintech) services in Indonesia has significantly transformed digital transaction behavior, with digital wallets such as DANA becoming widely adopted. Despite high usage rates, public discourse on social media frequently highlights both positive experiences and recurring concerns related to system reliability and data security. Twitter, as a real-time public communication platform, provides a rich source of user-generated content that reflects public perception toward fintech services. This study investigates public sentiment toward DANA fintech services in Indonesia through sentiment analysis of Twitter data using machine learning approaches. Purpose:This study aims to analyze public perception of DANA fintech services based on sentiment expressed on Twitter and to compare the performance of several machine learning classification algorithms in identifying positive, neutral, and negative sentiments in Indonesian-language social media texts. Methods/Study design/approach: The study employs a quantitative descriptive approach using text mining and sentiment analysis techniques. Twitter data were collected via the Twitter API over a specified period and processed through preprocessing stages including text cleaning, tokenization, stopword removal, and stemming. Feature extraction was performed using the Term Frequency–Inverse Document Frequency (TF-IDF) method. Sentiment classification was conducted using Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Random Forest algorithms. Model performance was evaluated using accuracy, precision, recall, and F1-score metrics. Result/Findings:The results indicate that public sentiment toward DANA is predominantly positive, driven by perceptions of transaction convenience and speed, followed by neutral sentiment, while negative sentiment is mainly associated with system disruptions and data security issues. Among the evaluated algorithms, SVM achieved the highest classification accuracy (91.4%), outperforming Random Forest and KNN. These findings demonstrate that SVM is more effective for sentiment classification of short Indonesian-language texts on social media platforms. Novelty/Originality/Value:This study contributes to the growing body of sentiment analysis research in Indonesia by providing empirical evidence on public perception of fintech services using machine learning techniques. By systematically comparing multiple classification algorithms in the context of Indonesian Twitter data, the study offers methodological insights and practical implications for fintech service providers in improving service quality, communication strategies, and user trust.

Sentiment analysis has become increasingly important in understanding user perceptions of digital platforms. This study focuses on analyzing TikTok application reviews from the Google Play Store in Indonesia using machine learning techniques. The research aims to investigate sentiment distribution and compare the performance of three popular machine learning models: Random Forest, Support Vector Machine (SVM), and Naive Bayes. The study employed a comprehensive methodology involving data collection, preprocessing, feature extraction, and model evaluation. A dataset of 10,000 TikTok reviews was collected and preprocessed using techniques such as case folding, tokenization, and stopword removal. The sentiment labeling process categorizes reviews into positive, negative, and neutral sentiments based on user ratings. The TF-IDF algorithm was used for feature extraction, and the SMOTE technique addressed class imbalance. Results revealed a predominance of negative sentiment (53.5%), followed by neutral (32.1%) and positive (14.4%) sentiment. Model performance comparisons at different data sharing ratios (80/20 and 70/30) demonstrated that Random Forest and SVM consistently outperformed Naive Bayes. At the 80/20 ratio, Random Forest achieved the highest accuracy of 83.73%, highlighting its effectiveness in sentiment classification. The research contributes to the field of sentiment analysis and natural language processing by providing insights into user experiences with the TikTok application in Indonesia. The findings can guide application developers in understanding user perceptions and improving user experience.

This study explores the role of sentiment analysis as a predictive tool for understanding and forecasting product launch success in the digital market. Sentiment analysis involves the classification of consumer sentiment expressed on social media platforms such as Twitter and Instagram, and it can significantly impact businesses by predicting consumer behavior and product performance. The research highlights the relationship between social media sentiment and product success, demonstrating that positive sentiment is strongly correlated with higher sales and consumer engagement, while negative sentiment can lead to declines. Machine learning models, including Support Vector Machines (SVM) and Random Forest, were employed to classify sentiment from large volumes of social media data and correlate it with product performance indicators such as sales volume and consumer interaction. The study found that sentiment analysis models were highly effective in predicting product success, with positive sentiment generally driving product profitability and negative sentiment posing a potential threat to brand reputation. Moreover, the analysis showed that social media sentiment provides real-time insights into consumer perceptions, enabling businesses to quickly adjust marketing strategies and product development plans. These findings underscore the importance of integrating sentiment analysis into product launch evaluations and strategic decision-making. Future research should explore the integration of sentiment analysis with other predictive market models and investigate the effects of fake reviews and post-purchase consumer behaviors on product success.

Sentiment analysis is a crucial part of natural language processing (NLP) that categorizes text into positive, negative, or neutral sentiments. This technology helps businesses automatically interpret customer emotions, enabling informed decisions in marketing, product development, and customer support. Traditional machine learning models like Random Forest, Naive Bayes, and Support Vector Machine (SVM) have been commonly used for sentiment classification, especially effective for short and concise texts such as brief customer comments. SVM, in particular, performs well in classifying sentiments in short texts. However, the emergence of advanced large language models (LLMs) like GPT-4 has revolutionized sentiment analysis by capturing more nuanced emotional expressions. GPT-4 excels in understanding complex context, sarcasm, and subtle emotional tones within longer texts, offering multi-dimensional sentiment insights such as satisfaction, frustration, and uncertainty. This makes it more capable than traditional models when analyzing detailed customer reviews or conversations. Comparative studies reveal that while traditional models handle short texts efficiently, GPT-4 surpasses them in analyzing in-depth content with greater precision, recall, and F1 scores. GPT-4 also uniquely identifies mixed sentiments that simpler models often miss. In conclusion, although traditional machine learning approaches remain useful for straightforward sentiment analysis tasks, GPT-4 provides a more sophisticated and comprehensive understanding of customer emotions in complex texts. Integrating GPT-4 into sentiment analysis workflows can significantly enhance the accuracy and richness of insights derived from customer feedback, supporting better business decisions.

The use of Machine Learning (ML) and Sentiment Analysis (SA) on data from microblogging sites has become a popular method for stock market prediction. In this work, we developed a model for predicting stock movement utilizing SA on Twitter and StockTwits data. Stock movement and sentiment data were used to evaluate this approach and validate it on Microsoft stock. We gathered tweets from Twitter and StockTwits, as well as financial data from Finance Yahoo. SA was applied to tweets, and seven ML classification models were implemented: K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Logistic Regression (LR), Naïve Bayes (NB), Decision Tree (DT), Random Forest (RF) and Multilayer Perceptron (MLP). The main novelty of this work is that it integrates multiple SA and ML methods, emphasizing the retrieval of extra features from social media (i.e., public sentiment), for improving stock prediction accuracy. The best results were obtained when tweets were analyzed using Valence Aware Dictionary and sEntiment Reasoner (VADER) and SVM. The top F-score was 76.3%, while the top Area Under Curve (AUC) value was 67%.

This study evaluates the performance of four machine learning models: Support Vector Machine (SVM), Random Forest, K-Nearest Neighbors (KNN), and Naive Bayes in analyzing visitor reviews of the Lokawisata Baturaden tourist attraction. Using 5-fold cross-validation, the study aims to determine which machine learning model best suits sentiment analysis on the Baturaden review data. This study was conducted through several stages, including data preprocessing, feature extraction, and the data training process. Case folding, text cleaning, tokenization, stopword removal, and stemming were performed during the data preprocessing stage. The feature extraction method used was TF-IDF. SMOTE was applied to increase data variation and address the data imbalance in the dataset. The results show that SVM provides the best performance with an accuracy of 0.937, an F1-score of 0.937, a precision of 0.943, and a recall of 0.937. Random Forest also performs well with an accuracy of 0.918 and an F1-score of 0.918, though slightly below SVM. KNN shows the lowest performance with an accuracy of 0.651 and an F1-score of 0.544, while Naive Bayes performs adequately with an accuracy of 0.845 and an F1-score of 0.841. Based on this evaluation, SVM is recommended as the best model for sentiment analysis of reviews, followed by Random Forest as a good alternative. The KNN model is not recommended due to its lower performance, while Naive Bayes can be considered for its speed and simplicity, although its results are not as good as SVM and Random Forest. These conclusions guide the selection of the optimal model to enhance understanding and visitor experience at the Baturaden tourist attraction.

Long before the elections were held, the topic related to elections was widely discussed on news portals and social media, including Twitter. A few studies related to the Indonesian election have tried to predict candidates who will run for the presidential election, but there has been no research that examines public sentiment on social media towards each of the potential candidates. The main objective of this study is to analyze the public sentiment in Twitter towards potential candidates for the 2024 Indonesian presidential election. This research seeks to fill the gaps in previous research and become a reference for further research regarding sentiment analysis for election prediction using Twitter. The presidential candidates used in the research are the top 3 candidates based on the Poltracking survey, namely Ganjar Pranowo, Prabowo Subianto, and Anies Baswedan. The data were taken from January until October 2022, more than a year before the general election began. To predict the sentiment, four different machine-learning methods were used and compared to each other. There are Naïve Bayes, Support Vector Machines, Random Forests, and Neural Networks. Based on the sentiment results of each candidate, the highest sentiment towards Prabowo is neutral (55.49%), the highest sentiment towards Ganjar is positive (61.34%), and the highest sentiment towards Anies is neutral (44.84%). Results from the study also show that Anies was the presidential candidate who received more negative sentiment than the other two (56.63%). Meanwhile, Ganjar Pranowo got the most positive sentiment of all (42,69%). For the neutral sentiment, Anies Baswedan also got the most results (39,87%), followed by Prabowo (38.99%) and Ganjar Pranowo (21.14%). The result of the study also discovered that random forest and neural networks have the best performance for sentiment analysis.

Abstract—This paper presents a hybrid sentiment analysis system that integrates lexicon-based, traditional machine learning, and deep learning techniques to classify textual data into positive, negative, or neutral sentiments. The system leverages Python-based libraries such as Scikit-learn, NLTK, and Transformers to pre- process text, extract features, and apply models in- cluding Support Vector Machine (SVM), Bidirectional Encoder Representations from Transformers (BERT), Linear Regression, and Valence Aware Dictionary and sEntiment Reasoner (VADER). The framework pro- cesses customer reviews from e-commerce platforms (Amazon, Flipkart) and social media (Instagram) using web scraping and provides actionable insights through sentiment summarization and visualizations (bar and pie charts). Experimental results demonstrate BERT’s superior performance with 92.3% accuracy, followed by SVM (85.6%), Linear Regression (81.2%), and VADER (76.8%). The system addresses challenges like sarcasm, class imbalance, and scalability, offering a scalable, user-friendly solution for real-world applications in e- commerce, social media analytics, and brand reputation management. Index Terms—Sentiment Analysis, Machine Learning, Deep Learning, Natural Language Processing, BERT, SVM, VADER, Text Classification, Data Visualization.

In recent times, when we fiddle with innovation particularly with social media, it leads to an increasing change in human conduct. The day-by-day utilization of web-based media by individuals has expanded a lot that it is gradually infusing an expression into our conduct. Particularly, Twitter is generally utilized authoritatively by numerous individuals and is one of only a handful few social media platforms where individuals search for reliable information. The intent of this work is to perform sentiment analysis of the tweets using different Machine Learning algorithms and the BERT model to prevent offensive use of words and give an insight into the emotion of the words in the tweet. The dataset utilized is obtained from Kaggle. The Machine Learning models employed are Support Vector Machine (SVM), Stochastic Gradient Descent (SGD), Logistic Regression, Decision-Tree., Random Forest Method, and XGBoost along with a deep learning architecture BERT for the analysis. A comparative analysis of all the models was carried out and it can be concluded that logistic Regression obtained the highest training and validation accuracy of 82.65% and 81.74%, respectively, and the F1 score of this model was 79.14%. BERT model has an exceptional performance in Natural Language Processing, so BERT was also implemented for the sentimental analysis of the Twitter tweets and it achieved the highest training accuracy, validation accuracy, and F1 score of 94%., 93.63%, and 92.34% respectively.

Opinion mining is the computational study of people's opinions, emotions and attitudes which is one of the key research field in Natural Language Processing (NLP). To cope with the competitive world, owners of business need to extract exact opinion of people about his/her business. Recently, people in Bangladesh are more interested to express their opinion in Bangla and most importantly in Phonetic Bangla rather than English. Since no specific work of Opinion mining introduced this criteria, in this paper, we have developed review analysis system on Bangla and Phonetic Bangla where we have used Restaurant reviews as case study and the dataset is created manually by us without using translator. Our approach starts by preprocessing raw data and then feature extraction with different N-gram techniques. Then vectorization is applied on that data with HashingVectorizer, CountVectorizer and TF-IDF vectorizer. Later machine learning based approaches namely Support Vector Machine (SVM), Decision Tree (DT) and Logistic Regression (LR) are applied to classify reviews. We have classified the reviews in three different classes, i.e. bad, good and excellent. Finally a comparison is shown between vectorizers in accordance with different classifiers where SVM provides better accuracy with 75.58%.

Stock news is one of the information sources that can used to monitor stock prices. The information from stock news usually contains positive and negative sentiments that can affect stock prices. Therefore, sentiment analysis is needed to process the sentiment of stock news. The stock news dataset is taken from Kaggle. From these data, there is an imbalanced class between positive and negative sentiment. This research proposed a method to solve the imbalance dataset with random oversampling which worked by randomly replicating several minority classes. This research presents several scenarios of pre-processing text with different stages, intending to get high accuracy. The classification method used in this paper is a robustly optimized Bidirectional Transformer Encoder Representation (RoBERTa). Besides that, this paper also compared with baseline of Machine Learning (ML) such as Multinomial Naïve Bayes, Bernoulli Naïve Bayes, Support Vector Machine, Random Forest Classifier, Logistic Regression and used two different text representation such as TF-IDF and Word2Vec. The best result in this research is obtained using RoBERTa method with the fourth scenario of pre-processing text, in which the stage of pre-processing in this scenario only removing hashtag, without removing punctuation, removing the number, converting number, stop word removal, and lemmatization. The performance result is 0.85 precision, 0,84 recall, 0,84 F1-score, and 86% for accuracy result.

Online social media platforms allow people to express their ideas and opinions about anything as products, education, or politics, this produce a huge data requires appropriate tools to analyze it accurately, and that’s why Sentiment Analysis (SA) has become more used in recent years. This paper review the SA and classification methods of machine learning especially supervised techniques like Support Vector Machine (SVM), Neural Networks (NN), Naïve Bayes (NB), Rough Set Theory (RST), Random Forest (RF), Decision Trees (DT), and other technology of SA is lexicon techniques and hybrid methods (which include the lexicon and machine learning techniques). Machine learning consist of two group supervised (Classification and Regression) and unsupervised (clustering and association). The SA Affected from one system to another based on its polarity as “positive” or “negative”, and the language of data that processed.

With the rapid growth of digital technology, social media has become a key platform for sharing information and opinions. Twitter, one of the most popular platforms in Indonesia, enables users to interact directly with public figures such as Najwa Shihab. This study aims to analyze public sentiment toward Najwa Shihab on Twitter using sentiment analysis, specifically employing the Naïve Bayes and Support Vector Machine (SVM) algorithms. Sentiment analysis is essential to understanding public opinion, as it classifies text into categories like positive, negative, or neutral, providing valuable insights into societal perspectives on public figures. In this study, 10,000 tweets related to Najwa Shihab were collected from January 1, 2023, to January 31, 2023. Data preprocessing steps such as data cleaning, tokenization, stopwords removal, and filtering were conducted to ensure high-quality data for analysis. The Naïve Bayes and SVM algorithms were applied using RapidMiner to classify the sentiment of the tweets. The performance of both algorithms was evaluated based on accuracy, precision, recall, and F1-score.The results revealed that SVM outperformed Naïve Bayes in all metrics, demonstrating its superior ability to classify sentiments correctly. The sentiment distribution indicated a majority of positive opinions toward Najwa Shihab, with fluctuations in negative sentiment during specific events. This study provides insights into public sentiment analysis and contributes to understanding social media opinions on public figures.

All groups of people felt the impact of the COVID-19 pandemic. This situation triggers anxiety, which is bad for everyone. The government's role is very influential in solving these problems with its work program. It also has many pros and cons that cause public anxiety. For that, it is necessary to detect anxiety to improve government programs that can increase public expectations. This study applies machine learning to detecting anxiety based on social media comments regarding government programs to deal with this pandemic. This concept will adopt a sentiment analysis in detecting anxiety based on positive and negative comments from netizens. The machine learning methods implemented include K-NN, Bernoulli, Decision Tree Classifier, Support Vector Classifier, Random Forest, and XG-boost. The data sample used is the result of crawling YouTube comments. The data used amounted to 4862 comments consisting of negative and positive data with 3211 and 1651. Negative data identify anxiety, while positive data identifies hope (not anxious). Machine learning is processed based on feature extraction of count-vectorization and TF-IDF. The results showed that the sentiment data amounted to 3889 and 973 in testing, and training with the greatest accuracy was the random forest with feature extraction of vectorization count and TF-IDF of 84.99% and 82.63%, respectively. The best precision test is K-NN, while the best recall is XG-Boost. Thus, Random Forest is the best accurate to detect someone's anxiety based-on data from social media.

Nowadays, with the increasing number and use of social media platforms, people now share their experiences about a product they have bought or a place they have been to on social media platforms more frequently. Considering the volume of data on social media platforms, it is considered that there is some meaningful information for institutions or companies in the reviews and experiences shared on social media platforms. As such, it is important to improve the methods of extracting meaningful information from the reviews and experiences shared on social media and to know which method is better. In this study, the classification successes of the bag of words and the fastText word representation methods, which are among the word representation methods in sentiment analysis methods mentioned above, were compared by using Turkish reviews performed for touristic places. Besides, while performing the comparison process, it was measured whether the process of separating the words into their roots and negation of the words, which is the preliminary stage of the sentiment analysis process, contributed to the classification success. In the study, both two-class (positive, negative) sentiment analysis and three-class (positive, negative, neutral) sentiment analysis were performed. Six data sets were created to carry out the mentioned comparison operations. The data sets were first classified using the Naive Bayes (NB), Multinomial Naive Bayes (MNB), k-Nearest Neighbor (k-NN) and Support Vector Machines (SVM) algorithms, which are frequently used in text mining, and based on bag of words word representation method, they were classified with WEKA program. After the test results of all data sets were obtained according to the bag of words word representation method, the tests of the fastText word representation method were carried out using the fastText library of the Python programming language. Classification procedures were carried out with 10-fold cross-validation methods, and f-score values of the classification processes were obtained. Finally, it was determined that bag of words word representation method performed a more successful classification than the fastText word representation method in two-class emotion analysis, while the fastText word representation method performed a more successful classification process than bag of words word representation method in three-class emotional analysis. It was observed that the process of separating the words into their roots and negating the words, which are the preliminary processes of sentiment analysis, did not contribute positively or negatively to the classification processes performed with the fastText word representation method. However, it was determined that it had a minor contribution to sentiment analysis processes performed by using bag of words word representation method. In the two-class sentiment analysis, the most successful classification result was achieved by using the machine learning model created with the SVM algorithm with the value of 0.91 f-score employing bag of words word representation method. In the three-class sentiment analysis, the most successful classification result was achieved with the machine learning model created using the fastText word representation method with the value of 0.78 f-score.