Syntactically Aware Cross-Domain Aspect and Opinion Terms Extraction

Previous studies usually divided aspect-based sentiment analysis into several subtasks in pipeline, i.e., first aspect term and/or opinion term extraction, then aspect-based sentiment prediction, resulting in error propagation and external resources dependency. To overcome the problems mentioned above, in this work we present a novel one-stop solution on aspect-based sentiment analysis. Specifically, we propose a novel multi-task neural learning framework to jointly tackle aspect extraction and sentiment prediction subtasks at the same time, and leverage attention mechanisms to learn the joint representation of aspect-sentiment relationship. We have conducted extensive comparative experiments on two benchmark datasets from SemEva1–2014. The experiment results demonstrate the effectiveness of our proposed solution. Especially, our multi-task model outperforms the state-of-the-art systems on aspect extraction subtask.

Aspect term extraction is a key task in aspect-level sentiment analysis, aiming at extracting aspect term from text. However, most of the existing aspect-level sentiment analysis methods focus on the matching of aspect term and sentiment polarity, and neglect the research of aspect term extraction. In this paper, an aspect term extraction method based on BiLSTM-CRF with BERT embedding is proposed. Firstly, the embedded layer in Bert model is used to represent the vector, then it is sent to BiLSTM layer for context feature learning, and finally, the aspect term are extracted by CRF. Therefore, the model based on BiLSTM-CRF with BERT embedding is constructed. The experimental results show that this method has high precision and recall rate in Restaurants and Laptop of the SemEval-2014 dataset. The average value of F1 is 85.74%, and the performance of this method has the best comprehensive performance and has more outstanding performance than other models.KeywordsDeep learningSentiment analysisAspect-level sentiment analysisAspect term extraction

Fine-grained aspect term extraction is an essential subtask in aspect-based opinion analysis. It aims to identify the aspect terms (also known as opinion targets) of a product or service in each sentence. To learn a good aspect extraction model, an expensive annotation process is usually involved to acquire sufficient token-level labels for each domain, which is not realistic. To address this limitation, some previous works propose domain adaptation strategies to transfer knowledge from a sufficiently labeled source domain to unlabeled target domains. However, due to both the difficulty of fine-grained prediction problems and the large domain gap between different domains, the performance is still far from satisfactory. In this work, we conduct a pioneer study on leveraging sentence-level aspect category labels that can be usually available in commercial services, such as review sites or social media to promote token-level transfer for extraction purpose. Specifically, the aspect category information can be used to construct pivot knowledge for transfer with the assumption that the interactions between the sentence-level aspect category and the token-level aspect terms are invariant across domains. To this end, we propose a novel multilevel reconstruction mechanism that aligns both the fine- and coarse-grained information in multiple levels of abstractions. Comprehensive experiments over several benchmark data sets clearly demonstrate that our approach can fully utilize the sentence-level aspect category labels to improve cross-domain aspect term extraction with a large performance gain.

Aspect Sentiment Triplet Extraction (ASTE) is the most recent subtask of ABSA which outputs triplets of an aspect target, its associated sentiment, and the corresponding opinion term. Recent models perform the triplet extraction in an end-to-end manner but heavily rely on the interactions between each target word and opinion word. Thereby, they cannot perform well on targets and opinions which contain multiple words. Our proposed span-level approach explicitly considers the interaction between the whole spans of targets and opinions when predicting their sentiment relation. Thus, it can make predictions with the semantics of whole spans, ensuring better sentiment consistency. To ease the high computational cost caused by span enumeration, we propose a dual-channel span pruning strategy by incorporating supervision from the Aspect Term Extraction (ATE) and Opinion Term Extraction (OTE) tasks. This strategy not only improves computational efficiency but also distinguishes the opinion and target spans more properly. Our framework simultaneously achieves strong performance for the ASTE as well as ATE and OTE tasks. In particular, our analysis shows that our span-level approach achieves more significant improvements over the baselines on triplets with multi-word targets or opinions.

Aspect term extraction and opinion word extraction are two fundamental subtasks of aspect-based sentiment analysis. The internal relationship between aspect terms and opinion words is typically ignored, and information for the decision-making of buyers and sellers is insufficient. In this paper, we explore an aspect–opinion pair extraction (AOPE) task and propose a Question-Driven Span Labeling (QDSL) model to extract all the aspect–opinion pairs from user-generated reviews. Specifically, we divide the AOPE task into aspect term extraction (ATE) and aspect-specified opinion extraction (ASOE) subtasks; we first extract all the candidate aspect terms and then the corresponding opinion words given the aspect term. Unlike existing approaches that use the BIO-based tagging scheme for extraction, the QDSL model adopts a span-based tagging scheme and builds a question–answer-based machine-reading comprehension task for an effective aspect–opinion pair extraction. Extensive experiments conducted on three tasks (ATE, ASOE, and AOPE) on four benchmark datasets demonstrate that the proposed method significantly outperforms state-of-the-art approaches.

In aspect-based sentiment analysis, a fundamental task is extracting aspect terms from opinionated sentences. Aspect term extraction (ATE) has been found to play a critical role among several scenarios, such as service quality improvement and recommendation systems. While deep learning-based methods have achieved great progress in ATE, they mainly consider sequential semantic information and generally ignore the utilisation of syntactic relations of the whole sentence on overall meanings. Furthermore, performances of these methods may also be diminished by poor handling of relation and text noises. To address these issues, we propose a fused sequential and hierarchical representation (FSHR) model, wherein both sequential and hierarchical representations are generated, which facilitates not only the capture of linear semantic information for predicting meaning-related aspect terms but also the utilisation of syntactic relations over the entire sentence to better identify structure-related aspect terms. Moreover, to refine the aspect representation, we incorporate relation-gate mechanism which selectively activates meaningful syntactic dependency paths and design the multi-way aspect attention which prompts the model to focus on relevant text segments about particular aspects. Eventually, sequential and hierarchical representations are adaptively fused for aspect prediction. Experiment results on four datasets demonstrate that FSHR outperforms competitive baselines, and further extensive analyses reveal the effectiveness of our model.

This paper describes our system used in the Aspect Based Sentiment Analysis Task 4 at the SemEval-2014. Our system consists of two components to address two of the subtasks respectively: a Conditional Random Field (CRF) based classifier for Aspect Term Extraction (ATE) and a linear classifier for Aspect Term Polarity Classification (ATP). For the ATE subtask, we implement a variety of lexicon, syntactic and semantic features, as well as cluster features induced from unlabeled data. Our system achieves state-of-the-art performances in ATE, ranking 1st (among 28 submissions) and 2rd (among 27 submissions) for the restaurant and laptop domain respectively.

With proliferation of user-generated reviews, new opportunities and challenges arise. The advance of Web technologies allows people to access a large amount of reviews of products and services online. Knowing what others like and dislike becomes increasingly important for their decision making in online shopping. The retailers also care more than ever about online reviews, because a vast pool of reviews enables them to monitor reputations and collect feedbacks efficiently. However, people often find difficult times in identifying and summarizing fine-grained sentiments buried in the opinion-rich resources. The traditional sentiment analysis, which focuses on the overall sentiments, fails to uncover the sentiments with regard to the aspects of the reviewed entities. This dissertation studied the research problem of Aspect Based Sentiment Analysis (ABSA), which is to reveal the aspect-dependent sentiment information of review text. ABSA consists of several subtasks: 1) aspect extraction, 2) aspect term extraction, 3) aspect category classification, and 4) sentiment polarity classification at aspect level. We focused on the approach of topic models and neural networks for ABSA. First, to extract the aspects from a collection of reviews and to detect the sentiment polarity regarding the aspects in each review, we proposed a few probabilistic graphical models, which can model words distribution in reviews and aspect ratings at the same time. Second, we presented a multi-task learning model based on long-short term memory and convolutional neural network for aspect category classification and aspect term extraction. Third, for aspect-level sentiment polarity classification, we developed a gated convolution neural network, which can be applied to aspect category sentiment analysis as well as aspect target sentiment analysis.

The advent of pre-trained language models has directed a new era of Natural Language Processing (NLP), enabling us to create powerful language models. Among these models, Transformer-based models like BERT have grown in popularity due to their cutting-edge effectiveness. However, these models heavily rely on resource-intensive languages, forcing other languages into multilingual models(mBERT). The two fundamental challenges with mBERT become significantly more challenging in a resource-constrained language like Bangla. It was trained on a limited and organized dataset and contained weights for all other languages. Besides, current research on other languages suggests that a language-specific BERT model will exceed multilingual ones. This paper introduces Bangla-BERT^a, a monolingual BERT model for the Bangla language. Despite the limited data available for NLP tasks in Bangla, we perform pre-training on the largest Bangla language model dataset, BanglaLM, which we constructed using 40 GB of text data. Bangla-BERT achieves the highest results in all datasets and vastly improves the state-of-the-art performance in binary linguistic classification, multilabel extraction, and named entity recognition, outperforming multilingual BERT and other previous research. The pre-trained model is assessed against several non-contextual models such as Bangla fasttext and word2vec the downstream tasks. Finally, this model is evaluated by transfer learning based on hybrid deep learning models such as LSTM, CNN, and CRF in NER, and it is observed that Bangla-BERT outperforms state-of-the-art methods. The proposed Bangla-BERT model is assessed by using benchmark datasets, including Banfakenews, Sentiment Analysis on Bengali News Comments, and Cross-lingual Sentiment Analysis in Bengali. Finally, it is concluded that Bangla-BERT surpasses all prior state-of-the-art results by 3.52%, 2.2%, and 5.3%.

Extensive research has been conducted on sentiment analysis for software engineering (SA4SE). Researchers have invested much effort in developing customized tools (e.g., SentiStrength-SE, SentiCR) to classify the sentiment polarity for Software Engineering (SE) specific contents (e.g., discussions in Stack Overflow and code review comments). Even so, there is still much room for improvement. Recently, pre-trained Transformer-based models (e.g., BERT, XLNet) have brought considerable breakthroughs in the field of natural language processing (NLP). In this work, we conducted a systematic evaluation of five existing SA4SE tools and variants of four state-of-the-art pre-trained Transformer-based models on six SE datasets. Our work is the first to fine-tune pre-trained Transformer-based models for the SA4SE task. Empirically, across all six datasets, our fine-tuned pre-trained Transformer-based models outperform the existing SA4SE tools by 6.5-35.6% in terms of macro/micro-averaged F1 scores.

In this article, an intensive sentiment analysis approach termed Hierarchical attention‐convolutional neural network (HAN‐CNN) has been proposed using product reviews. Firstly, the input product review is subjected to Bidirectional Encoder Representation from Transformers (BERT) tokenization, where the input data of each sentence are partitioned into little bits of words. Thereafter, Aspect Term Extraction (ATE) is carried out and feature extraction is completed utilizing some features. Finally, sentiment analysis is accomplished by the developed HAN‐CNN, which is formed by combining a Hierarchical Attention Network (HAN) and a Convolutional Neural Network (CNN). Moreover, the proposed HAN‐CNN achieved a greater performance with maximum accuracy, recall and F1‐Score of 91.70%, 90.60% and 91.20%, respectively.

Aspect-based sentiment classification (ABSC) (also called fine-grained sentiment analysis) is an essential and challenging task among sentiment analysis tasks. Aspect level sentiment analysis overcomes the limitation of the document and sentence level when multiple aspects appear in a review. Recently, neural network approaches like LSTM has achieved better results in ABSC than traditional machine learning algorithms. Aspect extraction and polarity detection for specific aspects in the review becoming an important task in aspect level sentiment analysis. Therefore, the paper aims to predict the sentiment polarity of each aspect in the review into 3 classes by developing the multi-aspect attention (MAA) model and combine it with the BiLSTM neural network, called MAA-BLSTM. Unlike unidirectional LSTM, the BiLSTM network runs the input in two directions: forward and backward, therefore, it can understand the context better than LSTM. In this paper, the hyperparameter tuning approach is also considered to be the high-performing model. Experiments are tested on restaurant and laptop data from SemEval (2014, 2015, and 2016) datasets and compare the result with other LSTM-based methods. Finally, the result proves that the accuracy of the proposed sentiment model reaches 89.9 on the restaurant dataset and 82.1 on the laptop dataset which are higher than other LSTM approaches.

The aspect term extraction (ATE) task aims to extract aspect terms describing a part or an attribute of a product from review sentences. Most existing works rely on either general or domain embedding to address this problem. Despite the promising results, the importance of general and domain embeddings is still ignored by most methods, resulting in degraded performances. Besides, word embedding is also related to downstream tasks, and how to regularize word embeddings to capture context-aware information is an unresolved problem. To solve these issues, we first propose context-aware dynamic word embedding (CDWE), which could simultaneously consider general meanings, domain-specific meanings, and the context information of words. Based on CDWE, we propose an attention-based convolution neural network, called ADWE-CNN for ATE, which could adaptively capture the previous meanings of words by utilizing an attention mechanism to assign different importance to the respective embeddings. The experimental results show that ADWE-CNN achieves a comparable performance with the state-of-the-art approaches. Various ablation studies have been conducted to explore the benefit of each component. Our code is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">http://github.com/xiejiajia2018/ADWE-CNN</uri> .

End-to-end aspect-based sentiment analysis (EASA) consists of two sub-tasks: the first extracts the aspect terms in a sentence and the second predicts the sentiment polarities for such terms. For EASA, compared to pipeline and multi-task approaches, joint aspect extraction and sentiment analysis provides a one-step solution to predict both aspect terms and their sentiment polarities through a single decoding process, which avoid the mismatches in between the results of aspect terms and sentiment polarities, as well as error propagation. Previous studies, especially recent ones, for this task focus on using powerful encoders (e.g., Bi-LSTM and BERT) to model contextual information from the input, with limited efforts paid to using advanced neural architectures (such as attentions and graph convolutional networks) or leveraging extra knowledge (such as syntactic information). To extend such efforts, in this paper, we propose directional graph convolutional networks (D-GCN) to jointly perform aspect extraction and sentiment analysis with encoding syntactic information, where dependency among words are integrated in our model to enhance its ability of representing input sentences and help EASA accordingly. Experimental results on three benchmark datasets demonstrate the effectiveness of our approach, where D-GCN achieves state-of-the-art performance on all datasets.

As a discipline of Natural Language Processing, Sentiment Analysis is used to extract and analyze subjective information present in natural language data. The task of Sentiment Analysis has acquired wide commercial uses including social media monitoring tasks, survey responses, review systems, etc. Languages like English have several resources which aid in the task of Sentiment Analysis. SentiWordNet and Subjectivity WordList are examples of such tools and resources. With more data being available in native vernacular, language-specific SentiWordNet(s) have become essential. For resource poor languages, creating such SentiWordNet(s) is a difficult task to achieve. One solution is to use available resources in English and translate the final source lexicon to target lexicon via machine translation. Machine translation systems for the English-Odia language pair have not yet been developed. In this paper, we discuss a method to create a SentiWordNet for Odia, which is resource-poor, by only using resources which are currently available for Indian languages. The lexicon created, would serve as a tool for Sentiment Analysis related task specific to Odia data.