Implementation of Feature Extraction Using BERT in Aspect Based Sentiment Analysis

In the ever-expanding digital landscape, the abundance of user-generated content on consumer platforms such as Booking and TripAdvisor offers a rich source of information for both travellers and hoteliers. Sentiment analysis, a fundamental research task of natural language processing (NLP) is used for mining sentiments and opinions within this vast reservoir of text reviews. A more specific type of sentiment analysis, i.e., aspect-based sentiment analysis (ABSA), is used when processing customer reviews is required. In ABSA, we aim to capture aspect-level sentiments and intricate relationships between various aspects within reviews. This article proposes a novel approach to ABSA by introducing a novel technique of word sense disambiguation (WSD) and integrating it with the Transformer architecture bidirectional encoder representations from Transformers (BERT) and graph convolutional networks (GCNs). The proposed approach resolves the intriguing ambiguities of the words and represents the review data as a complex graph structure, facilitating the modeling of intricate relationships between different aspects. The combination of bidirectional long short-term memory (BiLSTM) and GCN proves effective in capturing inter-dependencies among various aspects, providing a nuanced understanding of customer sentiments. The experiments are conducted on the RABSA dataset (an enhanced and richer hotel review data collection), and results demonstrate that our approach outperforms previous baselines, showcasing the effectiveness of integrating WSD in ABSA. Furthermore, an ablation study confirms the significant contribution of the WSD module to the overall performance. Moreover, we explore different similarity measures and find that cosine similarity yields the best results when identifying the real sense of a word in a given sentence using WordNet. The findings of our work and future work related to our work create lots of interest for people in the tourism and hospitality industry. This research gives another boost to the concept of the potential of NLP techniques in sentiment analysis. It emphasizes that if we combine the potential of NLP techniques along with state-of-the-art machine learning frameworks, we can shape the future of this field.

Sentiment analysis is becoming rapidly important for exploring social media Bangla text. The lack of sufficient resources is considered to be an important challenge for Aspect Based Sentiment Analysis (ABSA) of the Bangla language. The ABSA is a technique that divides the text and defines its sentiment based on its aspects. In this paper, we developed a high-quality Bangla ABSA annotated dataset namely BANGLA_ABSA. The datasets are labelled with aspects category and their respective sentiment polarity to do the ABSA task in Bangla. Four open domains namely Restaurant, Movie, Mobile phone, and Car are considered to make the dataset. The datasets are called Restaurant_ABSA, Movie_ABSA, Mobile_phone_ABSA, and Car_ABSA respectively that contain 801, 800, 975, and 1149 comments. All the comments are either complex or compound sentences. We created the dataset manually and annotated the same by exerting opinions. We organized the dataset as three tuples in Excel format namely 〈Id, Comment, {Aspect category, Sentiment Polarity}〉. These data are very important that facilitate the efficient handling of sentiment for any machine learning and deep learning research, especially for Bangla text.

Consumer opinions on purchased products significantly influence the number of purchases on ecommerce platforms. Sentiment analysis of these opinions assists both consumers and companies in making informed decisions. Currently, consumers judiciously evaluate different aspects of a product to guarantee it meets their requirements. Consequently, performing sentiment analysis on various product aspects can aid consumers select the precise product and support companies in meeting on areas that prerequisite upgrading. This study recommends a systematic and expandable method to aspect-based sentiment analysis of user reviews for electrical devices like computers, mobile phones, and headphones. Through an amalgamation of advanced preprocessing, sentence-level segmentation by means of transformer-based models, and similarity-based aspect detection, the system is competent of extracting fine-grained sentiment for each product aspect. A hybrid sentiment classification strategy employing both BERT and VADER certifies robustness and accurateness in sentiment detection, while the structured transformation of data into a multi-label format permits for efficient model training. The proposed Bi-LSTM architecture with an integrated attention mechanism boosts sentiment prediction by concentrating on the utmost pertinent parts of each sentence, directing to better-quality performance across evaluation metrics, including accuracy, precision, recall, and F1-score. The exploration offers a comparative study of the performance of aspect-based sentiment analysis using a standard Bi-LSTM model versus a Bi-LSTM model with an attention mechanism, applied to analyse the reviews of three diverse products. The outcomes explains that the Bidirectional Long Short-Term Memory (Bi-LSTM) model with an attention mechanism outpaces the Bidirectional Long Short-Term Memory (Bi-LSTM) in terms of accuracy, precision, recall, and F1-score. The investigation validates the efficiency of merging NLP techniques and deep learning to deliver understandings from product reviews, presenting beneficial assistance to consumers by aiding to make informed decisions and focus on companies directing to improve their offerings based on targeted feedback.

Semantics perception and refinement network for aspect-based sentiment analysis

Sentiment analysis (SA) is one of the most active and rapidly developing research areas in both information retrieval and text mining fields of computer science. SA is the process of analysing, handling, generalizing, and reasoning about user sentiments, opinions, and emotions hidden within the text by using sentiment. There are various categories of levels of SA: (1) document-level, (2) sentence-level, and (3) aspect-level. This paper is focused on sorting out some Aspect-based sentiment analysis (ABSA) works. ABSA is one of the primary tasks in sentiment analysis, which involves identifying the expression of an opinion or sensation of an object in a particular dimension or feature. ABSA involves two primary subtasks: (1) aspect extraction and (2) aspect-based sentiment classification. In contrast to document-and sentence-based sentiment analysis, ABSA simultaneously considers sentiment and target information. Generally, the ABSA methods can be categorized into Transformer based ABSA and Graph Neural Network based ABSA. This paper introduces some popular frameworks, datasets, and evaluation metrics of ABSA tasks based on the above-proposed classification and methodology.

Sentiment analysis also referred to as opinion mining, plays a significant role in automating the identification of negative, positive, or neutral sentiments expressed in textual data. The proliferation of social networks, review sites, and blogs has rendered these platforms valuable resources for mining opinions. Sentiment analysis finds applications in various domains and languages, including English and Arabic. However, Arabic presents unique challenges due to its complex morphology characterized by inflectional and derivation patterns. To effectively analyze sentiment in Arabic text, sentiment analysis techniques must account for this intricacy. This paper proposes a model designed using the transformer model and deep learning (DL) techniques. The word embedding is represented by Transformer-based Model for Arabic Language Understanding (ArabBert), and then passed to the AraBERT model. The output of AraBERT is subsequently fed into a Long Short-Term Memory (LSTM) model, followed by feedforward neural networks and an output layer. AraBERT is used to capture rich contextual information and LSTM to enhance sequence modeling and retain long-term dependencies within the text data. We compared the proposed model with machine learning (ML) algorithms and DL algorithms, as well as different vectorization techniques: term frequency-inverse document frequency (TF-IDF), ArabBert, Continuous Bag-of-Words (CBOW), and skipGrams using four Arabic benchmark datasets. Through extensive experimentation and evaluation of Arabic sentiment analysis datasets, we showcase the effectiveness of our approach. The results underscore significant improvements in sentiment analysis accuracy, highlighting the potential of leveraging transformer models for Arabic Sentiment Analysis. The outcomes of this research contribute to advancing Arabic sentiment analysis, enabling more accurate and reliable sentiment analysis in Arabic text. The findings reveal that the proposed framework exhibits exceptional performance in sentiment classification, achieving an impressive accuracy rate of over 97%.

Reading massive amounts of user-generated text and pulling out the relevant aspects and opinions is a complicated process. Summaries, on the other hand, help busy people who only have a little bit of time to read get the gist of the information quickly. Text summarization is the process of taking the original text and making a shorter version of it that still has all of its informational value and main idea. Humans have a hard time summarizing long texts by hand. Different ways to summarize a text can be put into groups based on the more general techniques of extractive and abstractive summarization. The research paper discusses the need for generating aspect-based summaries and Sentiment analysis. A framework is proposed based on extracting coherent aspects from the reviews and applying the extractive summarization method to generate summaries. In addition, providing insights into the reviews of tourist attractions by using aspect-based sentiment analysis. The results are evaluated using crowdsourcing, Fairsumm, and Centroid method. The crowdsourcing method gives the best result on aspect-based summaries.

Sentiment analysis or opinion mining used to capture the community's attitude who have experienced the specific service/product. Sentiment analysis usually concentrates to classify the opinion of whole document or sentence. However, in most comments, users often express their opinions on different aspects of the mentioned entity rather than express general sentiments on entire document. In this case, using aspect-based sentiment analysis (ABSA) is a solution. ABSA emphases on extracting and synthesizing sentiments on particular aspects of entities in opinion text. The previous studies have difficulty working with aspect extraction and sentiment polarity classification in multiple domains of review. We offer an innovative deep learning approach with the integrated construction of bidirectional Long Short Term Memory (BiLSTM) and Convolutional Neural Network (CNN) for multidomain ABSA in this article. Our system finished the following tasks: domain classification, aspect extraction and opinion determination of aspect in the document. Besides applying GloVe word embedding for input sentences from mixed Laptop_Restaurant domain of the SemEval 2016 dataset, we also use the additional layer of POS to pick out the word morphological attributes before feeding to the CNN_BiLSTM architecture to enhance the flexibility and precision of our suggested model. Through experiment, we found that our proposed model has performed the above mentioned tasks of domain classification, aspect and sentiment extraction concurrently on a mixed domain dataset and achieved the positive results compared to previous models that were performed only on separated domain dataset.

Aspect-based sentiment analysis is a fine-grained sentiment analysis that focuses on the sentiment polarity of different aspects of text, and most current research methods use a combination of dependent syntactic analysis and graphical neural networks. In this paper, a graph attention network aspect-based sentiment analysis model based on the weighting of dependencies (WGAT) is designed to address the problem in that traditional models do not sufficiently analyse the types of syntactic dependencies; in the proposed model, graph attention networks can be weighted and averaged according to the importance of different nodes when aggregating information. The model first transforms the input text into a low-dimensional word vector through pretraining, while generating a dependency syntax graph by analysing the dependency syntax of the input text and constructing a dependency weighted adjacency matrix according to the importance of different dependencies in the graph. The word vector and the dependency weighted adjacency matrix are then fed into a graph attention network for feature extraction, and sentiment polarity is predicted through the classification layer. The model can focus on syntactic dependencies that are more important for sentiment classification during training, and the results of the comparison experiments on the Semeval-2014 laptop and restaurant datasets and the ACL-14 Twitter social comment dataset show that the WGAT model has significantly improved accuracy and F1 values compared to other baseline models, validating its effectiveness in aspect-level sentiment analysis tasks.

ReMemNN: A novel memory neural network for powerful interaction in aspect-based sentiment analysis

Sentiment Analysis (SA) task is to label people's opinions as different categories such as positive and negative from a given piece of text. Another task is to decide whether a given text is subjective, expressing the writer's opinions, or objective, expressing. These tasks were performed at different levels of analysis ranging from the document level, to the sentence and phrase level. Another task is aspect extraction which originated from aspect-based sentiment analysis in phrase level. All these tasks are under the umbrella of SA. In recent years a large number of methods, techniques and enhancements have been proposed for the problem of SA in different tasks at different levels. This survey aims to categorize SA techniques in general, without focusing on specific level or task. And also to review the main research problems in recent articles presented in this field. We found that machine learning-based techniques including supervised learning, unsupervised learning and semi-supervised learning techniques, Lexicon-based techniques and hybrid techniques are the most frequent techniques used. The open problems are that recent techniques are still unable to work well in different domain; sentiment classification based on insufficient labeled data is still a challenging problem; there is lack of SA research in languages other than English; and existing techniques are still unable to deal with complex sentences that requires more than sentiment words and simple parsing.

Previous studies usually divided aspect-based sentiment analysis into several subtasks in pipeline, i.e., first aspect term and/or opinion term extraction, then aspect-based sentiment prediction, resulting in error propagation and external resources dependency. To overcome the problems mentioned above, in this work we present a novel one-stop solution on aspect-based sentiment analysis. Specifically, we propose a novel multi-task neural learning framework to jointly tackle aspect extraction and sentiment prediction subtasks at the same time, and leverage attention mechanisms to learn the joint representation of aspect-sentiment relationship. We have conducted extensive comparative experiments on two benchmark datasets from SemEva1–2014. The experiment results demonstrate the effectiveness of our proposed solution. Especially, our multi-task model outperforms the state-of-the-art systems on aspect extraction subtask.

Aspect-based sentiment analysis (ABSA) aims to analyze the sentiment polarity of an input sentence in a certain aspect. Many existing methods of ABSA employ long short-term memory (LSTM) networks and attention mechanism. However, the attention mechanism only models the local certain dependencies of the input information, which fails to capture the global dependence of the inputs. Simply improving the attention mechanism fails to solve the issue of target-sensitive sentiment expression, which has been proven to degrade the prediction effectiveness. In this work, we propose the multi-head self-attention transformation (MSAT) networks for ABSA tasks, which conducts more effective sentiment analysis with target specific self-attention and dynamic target representation. Given a set of review sentences, MSAT applies multi-head target specific self-attention to better capture the global dependence and introduces target-sensitive transformation to effectively tackle the problem of target-sensitive sentiment at first. Second, the part-of-speech (POS) features are integrated into MSAT to capture the grammatical features of sentences. A series of experiments carried on the SemEval 2014 and Twitter datasets show that the proposed model achieves better effectiveness compared with several state-of-the-art methods.

Background: Mobile app usage is increasing in the digital age, with Ride-Hailing app becoming the primary example of this trend. To obtain valuable understanding of how people perceive and interact with mobile app, user reviews on platforms such as Google Play are usually analyzed. This analysis can assist developers to identify areas for improvement in both Ride-hailing and Google Play App. A promising method that can be used to analyze user perception in this instance is Aspect-Based Sentiment Analysis (ABSA). Objective: This research aimed to apply ABSA to user reviews using Bidirectional Encoder Representations from Transformers (BERT) models. In this context, aspect identification and topic modeling were performed by using Latent Dirichlet Allocation (LDA). The model extracted topics from the reviews and used Generative Artificial Intelligence (GenAI) to define the aspects of the topics to further enhance the analysis. For consistency and accuracy, the method included sentiment annotation by a human annotator. Methods: A total of two datasets were used in this research, with the first collected by scraping user reviews of Ride-Hailing App while the second was obtained from Kaggle, and to identify relevant topics, modeling was performed using LDA. These topics were then categorized into aspects using GenAI, covering areas, such as customer experience, service, payment, app features, task management, and event management. Subsequently, sentiment labeling was conducted using human annotators to provide a reliable baseline. BERT model was then used to classify sentiment with aspect hints, and the evaluation included calculations of accuracy, precision, recall, and F1-score. Results: The results showed that BERT model achieved the highest accuracy of 97% in sentiment analysis across all datasets. Conclusion: This research provided valuable understanding of user experience and established a strong ABSA framework for analyzing user reviews using LDA, Aspect Annotation, GenAI, and BERT sentiment models. Future research could expand this method to other app categories and incorporate real-time ABSA for continuous monitoring and dynamic feedback. Keywords: User Reviews, Aspect-Based Sentiment Analysis (ABSA), Sentiment Analysis, Topic Modeling, Generative Artificial Intelligence (GenAI)

The emergence of various e-commerce sites has led to an increase in review sites for various services and products. People nowadays easily get information about products and services that will be used through reviews. Here sentiment analysis plays an important role in classifying the polarity of product reviews. However, with a large number of reviews, a sentiment analysis that only gives overall polarity is not sufficient. This will make it difficult to find the reviews of certain aspects (features) of the product. Aspect-based sentiment analysis as fine-grained sentiment analysis is able to provide specific polarity for each aspect contained in a sentence. Various kinds of development methods have been carried out to provide accurate results in aspect-based sentiment analysis. This paper will discuss the various deep learning methods that have been carried out and provide the possibility of research that can be carried out from Aspect-Based Sentiment Analysis.