Question-Driven Span Labeling Model for Aspect–Opinion Pair Extraction

In aspect-based sentiment analysis, a fundamental task is extracting aspect terms from opinionated sentences. Aspect term extraction (ATE) has been found to play a critical role among several scenarios, such as service quality improvement and recommendation systems. While deep learning-based methods have achieved great progress in ATE, they mainly consider sequential semantic information and generally ignore the utilisation of syntactic relations of the whole sentence on overall meanings. Furthermore, performances of these methods may also be diminished by poor handling of relation and text noises. To address these issues, we propose a fused sequential and hierarchical representation (FSHR) model, wherein both sequential and hierarchical representations are generated, which facilitates not only the capture of linear semantic information for predicting meaning-related aspect terms but also the utilisation of syntactic relations over the entire sentence to better identify structure-related aspect terms. Moreover, to refine the aspect representation, we incorporate relation-gate mechanism which selectively activates meaningful syntactic dependency paths and design the multi-way aspect attention which prompts the model to focus on relevant text segments about particular aspects. Eventually, sequential and hierarchical representations are adaptively fused for aspect prediction. Experiment results on four datasets demonstrate that FSHR outperforms competitive baselines, and further extensive analyses reveal the effectiveness of our model.

Aspect Sentiment Triplet Extraction (ASTE) is the most recent subtask of ABSA which outputs triplets of an aspect target, its associated sentiment, and the corresponding opinion term. Recent models perform the triplet extraction in an end-to-end manner but heavily rely on the interactions between each target word and opinion word. Thereby, they cannot perform well on targets and opinions which contain multiple words. Our proposed span-level approach explicitly considers the interaction between the whole spans of targets and opinions when predicting their sentiment relation. Thus, it can make predictions with the semantics of whole spans, ensuring better sentiment consistency. To ease the high computational cost caused by span enumeration, we propose a dual-channel span pruning strategy by incorporating supervision from the Aspect Term Extraction (ATE) and Opinion Term Extraction (OTE) tasks. This strategy not only improves computational efficiency but also distinguishes the opinion and target spans more properly. Our framework simultaneously achieves strong performance for the ASTE as well as ATE and OTE tasks. In particular, our analysis shows that our span-level approach achieves more significant improvements over the baselines on triplets with multi-word targets or opinions.

Aspect term extraction is a key task in aspect-level sentiment analysis, aiming at extracting aspect term from text. However, most of the existing aspect-level sentiment analysis methods focus on the matching of aspect term and sentiment polarity, and neglect the research of aspect term extraction. In this paper, an aspect term extraction method based on BiLSTM-CRF with BERT embedding is proposed. Firstly, the embedded layer in Bert model is used to represent the vector, then it is sent to BiLSTM layer for context feature learning, and finally, the aspect term are extracted by CRF. Therefore, the model based on BiLSTM-CRF with BERT embedding is constructed. The experimental results show that this method has high precision and recall rate in Restaurants and Laptop of the SemEval-2014 dataset. The average value of F1 is 85.74%, and the performance of this method has the best comprehensive performance and has more outstanding performance than other models.KeywordsDeep learningSentiment analysisAspect-level sentiment analysisAspect term extraction

This paper describes our system used in the Aspect Based Sentiment Analysis Task 4 at the SemEval-2014. Our system consists of two components to address two of the subtasks respectively: a Conditional Random Field (CRF) based classifier for Aspect Term Extraction (ATE) and a linear classifier for Aspect Term Polarity Classification (ATP). For the ATE subtask, we implement a variety of lexicon, syntactic and semantic features, as well as cluster features induced from unlabeled data. Our system achieves state-of-the-art performances in ATE, ranking 1st (among 28 submissions) and 2rd (among 27 submissions) for the restaurant and laptop domain respectively.

Fine-grained aspect term extraction is an essential subtask in aspect-based opinion analysis. It aims to identify the aspect terms (also known as opinion targets) of a product or service in each sentence. To learn a good aspect extraction model, an expensive annotation process is usually involved to acquire sufficient token-level labels for each domain, which is not realistic. To address this limitation, some previous works propose domain adaptation strategies to transfer knowledge from a sufficiently labeled source domain to unlabeled target domains. However, due to both the difficulty of fine-grained prediction problems and the large domain gap between different domains, the performance is still far from satisfactory. In this work, we conduct a pioneer study on leveraging sentence-level aspect category labels that can be usually available in commercial services, such as review sites or social media to promote token-level transfer for extraction purpose. Specifically, the aspect category information can be used to construct pivot knowledge for transfer with the assumption that the interactions between the sentence-level aspect category and the token-level aspect terms are invariant across domains. To this end, we propose a novel multilevel reconstruction mechanism that aligns both the fine- and coarse-grained information in multiple levels of abstractions. Comprehensive experiments over several benchmark data sets clearly demonstrate that our approach can fully utilize the sentence-level aspect category labels to improve cross-domain aspect term extraction with a large performance gain.

Mining opinions from online reviews is a fundamental step in obtaining the overall sentiment of a product. Detection of opinion relations among the words play an important role in the opinion target (OT) and opinion word (OW) extraction. In this paper, Partially Supervised Word Alignment Model is used to find opinion relations among words. Graph based co-ranking algorithm is used in estimating the confidence of each OT and OW. Candidates having confidence value higher than the threshold are extracted as final OT and OW. We propose a hybrid method that considers semantic relations along with opinion relations that results in fine grained opinion target (OT) and opinion word (OW) extraction. This semantic relations and opinion relations affects the confidence calculation of the OT and OW and improves the precision of extraction.

Previous studies usually divided aspect-based sentiment analysis into several subtasks in pipeline, i.e., first aspect term and/or opinion term extraction, then aspect-based sentiment prediction, resulting in error propagation and external resources dependency. To overcome the problems mentioned above, in this work we present a novel one-stop solution on aspect-based sentiment analysis. Specifically, we propose a novel multi-task neural learning framework to jointly tackle aspect extraction and sentiment prediction subtasks at the same time, and leverage attention mechanisms to learn the joint representation of aspect-sentiment relationship. We have conducted extensive comparative experiments on two benchmark datasets from SemEva1–2014. The experiment results demonstrate the effectiveness of our proposed solution. Especially, our multi-task model outperforms the state-of-the-art systems on aspect extraction subtask.

The aspect term extraction (ATE) task aims to extract aspect terms describing a part or an attribute of a product from review sentences. Most existing works rely on either general or domain embedding to address this problem. Despite the promising results, the importance of general and domain embeddings is still ignored by most methods, resulting in degraded performances. Besides, word embedding is also related to downstream tasks, and how to regularize word embeddings to capture context-aware information is an unresolved problem. To solve these issues, we first propose context-aware dynamic word embedding (CDWE), which could simultaneously consider general meanings, domain-specific meanings, and the context information of words. Based on CDWE, we propose an attention-based convolution neural network, called ADWE-CNN for ATE, which could adaptively capture the previous meanings of words by utilizing an attention mechanism to assign different importance to the respective embeddings. The experimental results show that ADWE-CNN achieves a comparable performance with the state-of-the-art approaches. Various ablation studies have been conducted to explore the benefit of each component. Our code is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">http://github.com/xiejiajia2018/ADWE-CNN</uri> .

Deep learning models and datasets for aspect term sentiment classification: Implementing holistic recurrent attention on target-dependent memories

With proliferation of user-generated reviews, new opportunities and challenges arise. The advance of Web technologies allows people to access a large amount of reviews of products and services online. Knowing what others like and dislike becomes increasingly important for their decision making in online shopping. The retailers also care more than ever about online reviews, because a vast pool of reviews enables them to monitor reputations and collect feedbacks efficiently. However, people often find difficult times in identifying and summarizing fine-grained sentiments buried in the opinion-rich resources. The traditional sentiment analysis, which focuses on the overall sentiments, fails to uncover the sentiments with regard to the aspects of the reviewed entities. This dissertation studied the research problem of Aspect Based Sentiment Analysis (ABSA), which is to reveal the aspect-dependent sentiment information of review text. ABSA consists of several subtasks: 1) aspect extraction, 2) aspect term extraction, 3) aspect category classification, and 4) sentiment polarity classification at aspect level. We focused on the approach of topic models and neural networks for ABSA. First, to extract the aspects from a collection of reviews and to detect the sentiment polarity regarding the aspects in each review, we proposed a few probabilistic graphical models, which can model words distribution in reviews and aspect ratings at the same time. Second, we presented a multi-task learning model based on long-short term memory and convolutional neural network for aspect category classification and aspect term extraction. Third, for aspect-level sentiment polarity classification, we developed a gated convolution neural network, which can be applied to aspect category sentiment analysis as well as aspect target sentiment analysis.

A fundamental task of fine-grained sentiment analysis is aspect and opinion terms extraction. Supervised-learning approaches have shown good results for this task; however, they fail to scale across domains where labeled data is lacking. Non pre-trained unsupervised domain adaptation methods that incorporate external linguistic knowledge have proven effective in transferring aspect and opinion knowledge from a labeled source domain to unlabeled target domains; however, pre-trained transformer-based models like BERT and RoBERTa already exhibit substantial syntactic knowledge. In this paper, we propose a method for incorporating external linguistic information into a self-attention mechanism coupled with the BERT model. This enables leveraging the intrinsic knowledge existing within BERT together with externally introduced syntactic information, to bridge the gap across domains. Finally, we demonstrate enhanced results on three benchmark datasets.

Aspect-based sentiment triple extraction (ASTE) is a demanding and emerging subtask of aspect-based sentiment analysis (ABSA). The primary objective of ASTE is to extract aspect and opinion terms and their corresponding sentiment polarities from a sentence. Existing methods stack additional graph convolutional networks (GCNs) on pretrained language models (PLMs) by propagating the representation according to the dependency relationship. Nevertheless, PLMs are pretrained on general language understanding tasks, introducing unnecessary relations in full connections in the Transformer encoder and disregarding both the local n-gram and syntactic dependency information. Moreover, the graph-based layers introduce additional computational loading when model convergence suffers from incorrect initialization. This study presents a lightweight PLM incorporating local n-gram and syntactic information for ASTE. Instead of stacking extra graph-based layers, we prune self-attention with local and syntactic dependency on the Transformer of the upper PLM layer to avoid inappropriate propagation between unrelated word pairs. A refining strategy is introduced to increase the final prediction distribution from the contextual prediction distribution. The experimental results on four benchmarks show that the proposed model outperforms the previously proposed methods.

Aspect-based sentiment analysis (ABSA) has become an essential field in Natural Language Processing (NLP) in recent years.ABSA not only categorizes sentiment as positive, negative, or neutral but also understands the specific aspects or topics discussed in the text review.This study focuses on two important elements of ABSA: Aspect Term Extraction (ATE) and Aspect Sentiment Classification (ASC).This study uses a combination of Sentence Embedding (SBERT) techniques, Part-of-Speech tagging, cosine-similarity calculation to assess words with their respective aspect labels, and the sparse attention mechanism (BIGBIRD) method, which has been proven to increase accuracy effectively and is effective in terms of time and memory usage.By applying this method to two hotel review datasets, Traveloka Review and Semeval 2016 dataset, it is proven to work well on two ABSA tasks, namely ATE and ASC.The results of the ATE test obtained an accuracy of 0.99, and the ASC test obtained an accuracy of 0.89.This study contributes to the advancement of ABSA by introducing a new methodology that improves the accuracy of aspect term extraction and sentiment classification.Additionally, it identifies avenues for future research, including exploring additional techniques to improve model performance and address potential limitations.

Aspect-based sentiment analysis refers to the task of determining the sentiment polarity associated with particular aspects mentioned in a sentence or document. Previous studies have used attention-based neural network models to connect aspect terms with context words, but these models often perform poorly due to limited interaction between aspect terms and opinion words. Furthermore, these models typically focus only on explicitly stated aspect objects, which can be overly restrictive in certain scenarios. Current sentiment analysis methods that rely on aspect categories also often fail to consider the implicit placement of aspect-category information within the context. While existing models may produce strong results, they often lack domain knowledge. To address these issues, this study proposes an Aspect-position and Entity-oriented Knowledge Convolutional Graph (APEKCG) consisting of two modules: the Aspect position-aware module (APA) and the Entity oriented Knowledge Dependency Convolutional Graph (EKDCG). The APA module is designed to integrate aspect-specific sentiment features for sentiment classification by incorporating information about aspect categories into different parts of the context. The EKDCG module incorporates entity-oriented knowledge, dependency labels, and syntactic path using a dependence graph. Experimental results on five benchmarks Natural Language Processing (NLP) datasets of the English language demonstrate the effectiveness of the proposed APEKCG framework. Furthermore, the APEKCG outperformed previous state-of-the-art models with its accuracy, achieving 89.13%, 84.32%, 89.02%, 79.64%, and 90.22% on the MAMS, Laptop, Restaurant, AWARE, and SemEval-15&16 datasets, respectively.

In recent years, mining opinions from customer reviews has been widely explored. Aspect-level sentiment analysis is a fine-grained subtask, which aims to detect the sentiment polarity towards a particular target in a sentence. While most previous works focus on sentiment polarity classification, opinion words towards the target are also very important for that they provide details about target and contribute to judging polarity. To this end, we propose a hierarchical network for jointly modeling aspect-level sentiment classification and word-level opinion words extraction. Our joint model acquires superior performance in opinion words extraction and achieves comparable results in sentiment polarity classification on two datasets from SemEval 2014.