Weakly Supervised Domain Adaptation for Aspect Extraction via Multilevel Interaction Transfer.

A fundamental task of fine-grained sentiment analysis is aspect and opinion terms extraction. Supervised-learning approaches have shown good results for this task; however, they fail to scale across domains where labeled data is lacking. Non pre-trained unsupervised domain adaptation methods that incorporate external linguistic knowledge have proven effective in transferring aspect and opinion knowledge from a labeled source domain to unlabeled target domains; however, pre-trained transformer-based models like BERT and RoBERTa already exhibit substantial syntactic knowledge. In this paper, we propose a method for incorporating external linguistic information into a self-attention mechanism coupled with the BERT model. This enables leveraging the intrinsic knowledge existing within BERT together with externally introduced syntactic information, to bridge the gap across domains. Finally, we demonstrate enhanced results on three benchmark datasets.

Aspect term extraction and opinion word extraction are two fundamental subtasks of aspect-based sentiment analysis. The internal relationship between aspect terms and opinion words is typically ignored, and information for the decision-making of buyers and sellers is insufficient. In this paper, we explore an aspect–opinion pair extraction (AOPE) task and propose a Question-Driven Span Labeling (QDSL) model to extract all the aspect–opinion pairs from user-generated reviews. Specifically, we divide the AOPE task into aspect term extraction (ATE) and aspect-specified opinion extraction (ASOE) subtasks; we first extract all the candidate aspect terms and then the corresponding opinion words given the aspect term. Unlike existing approaches that use the BIO-based tagging scheme for extraction, the QDSL model adopts a span-based tagging scheme and builds a question–answer-based machine-reading comprehension task for an effective aspect–opinion pair extraction. Extensive experiments conducted on three tasks (ATE, ASOE, and AOPE) on four benchmark datasets demonstrate that the proposed method significantly outperforms state-of-the-art approaches.

In aspect-based sentiment analysis, a fundamental task is extracting aspect terms from opinionated sentences. Aspect term extraction (ATE) has been found to play a critical role among several scenarios, such as service quality improvement and recommendation systems. While deep learning-based methods have achieved great progress in ATE, they mainly consider sequential semantic information and generally ignore the utilisation of syntactic relations of the whole sentence on overall meanings. Furthermore, performances of these methods may also be diminished by poor handling of relation and text noises. To address these issues, we propose a fused sequential and hierarchical representation (FSHR) model, wherein both sequential and hierarchical representations are generated, which facilitates not only the capture of linear semantic information for predicting meaning-related aspect terms but also the utilisation of syntactic relations over the entire sentence to better identify structure-related aspect terms. Moreover, to refine the aspect representation, we incorporate relation-gate mechanism which selectively activates meaningful syntactic dependency paths and design the multi-way aspect attention which prompts the model to focus on relevant text segments about particular aspects. Eventually, sequential and hierarchical representations are adaptively fused for aspect prediction. Experiment results on four datasets demonstrate that FSHR outperforms competitive baselines, and further extensive analyses reveal the effectiveness of our model.

With proliferation of user-generated reviews, new opportunities and challenges arise. The advance of Web technologies allows people to access a large amount of reviews of products and services online. Knowing what others like and dislike becomes increasingly important for their decision making in online shopping. The retailers also care more than ever about online reviews, because a vast pool of reviews enables them to monitor reputations and collect feedbacks efficiently. However, people often find difficult times in identifying and summarizing fine-grained sentiments buried in the opinion-rich resources. The traditional sentiment analysis, which focuses on the overall sentiments, fails to uncover the sentiments with regard to the aspects of the reviewed entities. This dissertation studied the research problem of Aspect Based Sentiment Analysis (ABSA), which is to reveal the aspect-dependent sentiment information of review text. ABSA consists of several subtasks: 1) aspect extraction, 2) aspect term extraction, 3) aspect category classification, and 4) sentiment polarity classification at aspect level. We focused on the approach of topic models and neural networks for ABSA. First, to extract the aspects from a collection of reviews and to detect the sentiment polarity regarding the aspects in each review, we proposed a few probabilistic graphical models, which can model words distribution in reviews and aspect ratings at the same time. Second, we presented a multi-task learning model based on long-short term memory and convolutional neural network for aspect category classification and aspect term extraction. Third, for aspect-level sentiment polarity classification, we developed a gated convolution neural network, which can be applied to aspect category sentiment analysis as well as aspect target sentiment analysis.

Previous studies usually divided aspect-based sentiment analysis into several subtasks in pipeline, i.e., first aspect term and/or opinion term extraction, then aspect-based sentiment prediction, resulting in error propagation and external resources dependency. To overcome the problems mentioned above, in this work we present a novel one-stop solution on aspect-based sentiment analysis. Specifically, we propose a novel multi-task neural learning framework to jointly tackle aspect extraction and sentiment prediction subtasks at the same time, and leverage attention mechanisms to learn the joint representation of aspect-sentiment relationship. We have conducted extensive comparative experiments on two benchmark datasets from SemEva1–2014. The experiment results demonstrate the effectiveness of our proposed solution. Especially, our multi-task model outperforms the state-of-the-art systems on aspect extraction subtask.

Aspect term extraction is a key task in aspect-level sentiment analysis, aiming at extracting aspect term from text. However, most of the existing aspect-level sentiment analysis methods focus on the matching of aspect term and sentiment polarity, and neglect the research of aspect term extraction. In this paper, an aspect term extraction method based on BiLSTM-CRF with BERT embedding is proposed. Firstly, the embedded layer in Bert model is used to represent the vector, then it is sent to BiLSTM layer for context feature learning, and finally, the aspect term are extracted by CRF. Therefore, the model based on BiLSTM-CRF with BERT embedding is constructed. The experimental results show that this method has high precision and recall rate in Restaurants and Laptop of the SemEval-2014 dataset. The average value of F1 is 85.74%, and the performance of this method has the best comprehensive performance and has more outstanding performance than other models.KeywordsDeep learningSentiment analysisAspect-level sentiment analysisAspect term extraction

Sentiment Analysis (SA) is the classification and interpretation of the opinions or emotions within the text data using text analysis methods. Aspect Level Sentiment Analysis (ALSA) is a type of SA technique which uses different types of algorithms to extract the aspects of the entities from the text data and determining the sentiment of the aspects. ALSA is divided into two subtasks such as Aspect Category Detection (ACD) and Aspect Category Polarity Detection (ACPD). ACD determines the aspect categories discussed in each sentence from a set of review sentences and a given predefined set of aspect categories. Aspect categories need not occur as terms in the sentences and are characteristically coarser than the aspect terms. For every review sentence the aspect categories are provided to identify the polarity of aspect category. The objective of ACPD is to identify the polarity such as negative, positive, neutral or conflict of every aspect category mentioned in each sentence. In this paper, the task aspect category polarity detection is addressed. The three types of features such as linguistic features, Lexicons features and Vector Space features are extracted from the reviews. The classifier is trained with these features. The learned model is used to detect the polarities of the categories that are present in the test review sentences. The combination of proposed features with the existing features improves the accuracy of the system. The accuracy of the proposed system has enhanced by 1.3 % compared with best performing system.

This paper describes our system used in the Aspect Based Sentiment Analysis Task 4 at the SemEval-2014. Our system consists of two components to address two of the subtasks respectively: a Conditional Random Field (CRF) based classifier for Aspect Term Extraction (ATE) and a linear classifier for Aspect Term Polarity Classification (ATP). For the ATE subtask, we implement a variety of lexicon, syntactic and semantic features, as well as cluster features induced from unlabeled data. Our system achieves state-of-the-art performances in ATE, ranking 1st (among 28 submissions) and 2rd (among 27 submissions) for the restaurant and laptop domain respectively.

Aspect-based sentiment analysis refers to the task of determining the sentiment polarity associated with particular aspects mentioned in a sentence or document. Previous studies have used attention-based neural network models to connect aspect terms with context words, but these models often perform poorly due to limited interaction between aspect terms and opinion words. Furthermore, these models typically focus only on explicitly stated aspect objects, which can be overly restrictive in certain scenarios. Current sentiment analysis methods that rely on aspect categories also often fail to consider the implicit placement of aspect-category information within the context. While existing models may produce strong results, they often lack domain knowledge. To address these issues, this study proposes an Aspect-position and Entity-oriented Knowledge Convolutional Graph (APEKCG) consisting of two modules: the Aspect position-aware module (APA) and the Entity oriented Knowledge Dependency Convolutional Graph (EKDCG). The APA module is designed to integrate aspect-specific sentiment features for sentiment classification by incorporating information about aspect categories into different parts of the context. The EKDCG module incorporates entity-oriented knowledge, dependency labels, and syntactic path using a dependence graph. Experimental results on five benchmarks Natural Language Processing (NLP) datasets of the English language demonstrate the effectiveness of the proposed APEKCG framework. Furthermore, the APEKCG outperformed previous state-of-the-art models with its accuracy, achieving 89.13%, 84.32%, 89.02%, 79.64%, and 90.22% on the MAMS, Laptop, Restaurant, AWARE, and SemEval-15&16 datasets, respectively.

End-to-end aspect-based sentiment analysis (EASA) consists of two sub-tasks: the first extracts the aspect terms in a sentence and the second predicts the sentiment polarities for such terms. For EASA, compared to pipeline and multi-task approaches, joint aspect extraction and sentiment analysis provides a one-step solution to predict both aspect terms and their sentiment polarities through a single decoding process, which avoid the mismatches in between the results of aspect terms and sentiment polarities, as well as error propagation. Previous studies, especially recent ones, for this task focus on using powerful encoders (e.g., Bi-LSTM and BERT) to model contextual information from the input, with limited efforts paid to using advanced neural architectures (such as attentions and graph convolutional networks) or leveraging extra knowledge (such as syntactic information). To extend such efforts, in this paper, we propose directional graph convolutional networks (D-GCN) to jointly perform aspect extraction and sentiment analysis with encoding syntactic information, where dependency among words are integrated in our model to enhance its ability of representing input sentences and help EASA accordingly. Experimental results on three benchmark datasets demonstrate the effectiveness of our approach, where D-GCN achieves state-of-the-art performance on all datasets.

The aspect term extraction (ATE) task aims to extract aspect terms describing a part or an attribute of a product from review sentences. Most existing works rely on either general or domain embedding to address this problem. Despite the promising results, the importance of general and domain embeddings is still ignored by most methods, resulting in degraded performances. Besides, word embedding is also related to downstream tasks, and how to regularize word embeddings to capture context-aware information is an unresolved problem. To solve these issues, we first propose context-aware dynamic word embedding (CDWE), which could simultaneously consider general meanings, domain-specific meanings, and the context information of words. Based on CDWE, we propose an attention-based convolution neural network, called ADWE-CNN for ATE, which could adaptively capture the previous meanings of words by utilizing an attention mechanism to assign different importance to the respective embeddings. The experimental results show that ADWE-CNN achieves a comparable performance with the state-of-the-art approaches. Various ablation studies have been conducted to explore the benefit of each component. Our code is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">http://github.com/xiejiajia2018/ADWE-CNN</uri> .

Sentiment analysis (SA) is the process by which information can be extracted from customer reviews to be analyzed and categorized as positive, negative or neutral opinions according to three levels of SA: document, sentence and aspect levels. In aspect level, the aim is to determinate the target of sentiment or opinion represented in customer reviews, known as aspect term identification. In this paper, we address the aspect identification task involving implicit aspect implied by adjectives and verbs. Our approach considers the training data enhancement for Naive Bayes (NB) classifier through the use of synonym and definition relations from WordNet (WN). In our method, we use the two semantic relations differently. NB classifier was used with five unique subsets of synonym and definition words and applied on two benchmark datasets, products reviews and restaurant. The obtained results show that: (1) The use of synonym and definition relations as represented in our work supports NB classifier and (2) Synonym and Definition relation must be appropriately combined the used to better support each implicit aspect identification task involving adjectives and verbs.

In this article, an intensive sentiment analysis approach termed Hierarchical attention‐convolutional neural network (HAN‐CNN) has been proposed using product reviews. Firstly, the input product review is subjected to Bidirectional Encoder Representation from Transformers (BERT) tokenization, where the input data of each sentence are partitioned into little bits of words. Thereafter, Aspect Term Extraction (ATE) is carried out and feature extraction is completed utilizing some features. Finally, sentiment analysis is accomplished by the developed HAN‐CNN, which is formed by combining a Hierarchical Attention Network (HAN) and a Convolutional Neural Network (CNN). Moreover, the proposed HAN‐CNN achieved a greater performance with maximum accuracy, recall and F1‐Score of 91.70%, 90.60% and 91.20%, respectively.

Aspect-based sentiment analysis (ABSA) tries to predict the polarity of a given document with respect to a given aspect entity. While neural network architectures have been successful in predicting the overall polarity of sentences, aspect-specific sentiment analysis still remains as an open problem. In this paper, we propose a novel method for integrating aspect information into the neural model. More specifically, we incorporate aspect information into the neural model by modeling word-aspect relationships. Our novel model, Aspect Fusion LSTM (AF-LSTM) learns to attend based on associative relationships between sentence words and aspect which allows our model to adaptively focus on the correct words given an aspect term. This ameliorates the flaws of other state-of-the-art models that utilize naive concatenations to model word-aspect similarity. Instead, our model adopts circular convolution and circular correlation to model the similarity between aspect and words and elegantly incorporates this within a differentiable neural attention framework. Finally, our model is end-to-end differentiable and highly related to convolution-correlation (holographic like) memories. Our proposed neural model achieves state-of-the-art performance on benchmark datasets, outperforming ATAE-LSTM by 4%-5% on average across multiple datasets.

Aspect-based sentiment analysis (ABSA) aims to detect the sentiment polarities of a sentence with a given aspect. Aspect-term sentiment analysis (ATSA) is a subtask of ABSA, in ATSA, the aspect is given by aspect term: a word or a phrase in sentence. In previous work, most models apply attention mechanism or gating mechanism to capture the key part of the sentence and detect the sentiment polarity by classifying the weighted sum vector, which are regardless of the aspect information during the classification. However, same contexts may show different sentiment polarity with differnet aspects. For example, in sentence “The scene hunky waiters dub diner darling and it sounds like they mean it.” the context “dub diner dearling” shows negative polarity towards the aspect term “waiters”. But in sentence “The diner's husband dub diner darling.”, the same context would show neural polarity towards “husband”. The absent of aspect information would make model get a wrong result in some sentence. To solve this problem, we propose a model called Trasfomer-based context-aspect Relation Detect Model (TRDM), and add two special tokens “[CLSA]” and “[SEPA]” to specify the aspect term in the sentence. TRDM uses the embedding of two tyes of tokens (i.e. “[CLS]” and “[CLSA]”) for classification where “[CLS]” is used to represent the entire sentence in BERT. This scheme enable TRDM to combine the sentence information and aspect information in the step of sentiment polarity classification. We evaluate the performance of our model on two datasets: Restaurant dataset from SemEval2014 and MAMS from NLPCC 2020. Experiment results show that our model obtain noticeable improvement compared with state-of-art transfromer-based models.