Triplet Contrastive Learning for Aspect Level Sentiment Classification

ALSC (Aspect-level Sentiment Classification) is a fine-grained task in the field of NLP (Natural Language Processing) which aims to identify the sentiment toward a given aspect. In addition to exploiting the sentence semantics and syntax, current ALSC methods focus on introducing external knowledge as a supplementary to the sentence information. However, the integration of the three categories of information is still challenging. In this paper, a novel method is devised to effectively combine sufficient semantic and syntactic information as well as use of external knowledge. The proposed model contains a sentence encoder, a semantic learning module, a syntax learning module, a knowledge enhancement module, an information fusion module and a sentiment classifier. The semantic information and syntactic information are respectively extracted via a self-attention network and a graphical convolutional network. Specifically, the KGE (Knowledge Graph Embedding) is employed to enhance the feature representation of the aspect. Then, the attention-based gate mechanism is taken to fuse three types of information. We evaluated the proposed model on three benchmark datasets and the experimental results establish strong evidence of high accuracy.

In this article, we present a novel Transformer-Based Aspect-Level Sentiment Classification with Intent (TASCI) model, designed to enhance sentiment analysis by integrating aspect-level sentiment classification with intent analysis. Traditional sentiment analysis methods often overlook the nuanced relationship between the intent behind a statement and the sentiment expressed toward specific aspects of an entity. TASCI addresses this gap by first extracting aspects using a self-attention mechanism and then employing a Transformer-based model to infer the speaker's intent from preceding sentences. This dual approach allows TASCI to contextualize sentiment analysis, providing a more accurate reflection of user opinions. We validate TASCI's performance on three benchmark datasets: Restaurant, Laptop, and Twitter, achieving state-of-the-art results with an accuracy of 89.10% and a macro-F1 score of 83.38% on the Restaurant dataset, 84.81% accuracy and 78.63% macro-F1 score on the Laptop dataset, and 79.08% accuracy and 77.27% macro-F1 score on the Twitter dataset. These results demonstrate that incorporating intent analysis significantly enhances the model's ability to capture complex sentiment expressions across different domains, thereby setting a new standard for aspect-level sentiment classification.

SK-GCN: Modeling Syntax and Knowledge via Graph Convolutional Network for aspect-level sentiment classification

Aspect-level sentiment classification is a fine-grained sentiment analysis task, which aims to predict the sentiment of a text in different aspects. One key point of this task is to allocate the appropriate sentiment words for the given aspect.Recent work exploits attention neural networks to allocate sentiment words and achieves the state-of-the-art performance. However, the prior work only attends to the sentiment information and ignores the aspect-related information in the text, which may cause mismatching between the sentiment words and the aspects when an unrelated sentiment word is semantically meaningful for the given aspect. To solve this problem, we propose a HiErarchical ATtention (HEAT) network for aspect-level sentiment classification. The HEAT network contains a hierarchical attention module, consisting of aspect attention and sentiment attention. The aspect attention extracts the aspect-related information to guide the sentiment attention to better allocate aspect-specific sentiment words of the text. Moreover, the HEAT network supports to extract the aspect terms together with aspect-level sentiment classification by introducing the Bernoulli attention mechanism. To verify the proposed method, we conduct experiments on restaurant and laptop review data sets from SemEval at both the sentence level and the review level. The experimental results show that our model better allocates appropriate sentiment expressions for a given aspect benefiting from the guidance of aspect terms. Moreover, our method achieves better performance on aspect-level sentiment classification than state-of-the-art models.

Aspect level sentiment classification aims to recognize the sentiment polarity of each aspect term in a sentence. However, most of the existing methods usually applied the attention mechanism over position-weighted memory and did not consider inter-aspect information. To address these issues, we propose a novel framework for aspect level sentiment classification, Deep Selective Memory Network (DSMN), which selects the context memory dynamically for better guiding the multi-hop attention mechanism and integrates inter-aspect information with deep memory network. By designing a selective attention mechanism based on the distance information between an aspect and its context, DSMN focuses on different parts of the context memory in different memory network layers to capture abundant aspect-aware context information. Besides, to make full use of the inter-aspect information, we also design effective inter-aspect modeling modules to generate both semantic and relation information of the nearby aspects for the desired aspect. We evaluate the advantages of our framework on three benchmark datasets, and experiment results show that our framework achieves state-of-the-art performance.

Aspect sentiment learning for Aspect-Level Sentiment Classification.

Semantic information and syntactic information are two essential dimensions of language. In recent years, most researches focus on extracting semantic information or how to extract useful information from syntactic dependency tree. As the semantic representations are closely related to syntactic ones, how to effectively integrate semantic and syntactic information is still a subject worth studying. We propose attention mechanism based on graph convolutional networks. The model integrates syntactic dependence information into semantic features and automatically achieve a balance between semantic information and syntactic information. We conducted experiments on the task of sentence-level relation extraction, and the results show that our model performs better than some previous methods.

Aspect-level sentiment classification aims to determine sentiment polarities of various aspects in reviews, where each review typically contains multiple aspects, that may correspond to different polarities. Aspect-level sentiment classification, unlike document-level sentiment classification, requires different context representations for different aspects. Existing methods normally use Long Short-Term Memory (LSTM) network to model aspects and contexts separately, and they combine attention mechanisms to extract features of a specific aspect in its context. Attention mechanisms are not used for sequence modeling, so aspects are not considered when generating context sequence representations. This study proposes a novel aspect-context interactive representation structure that only relies on an attention mechanism for generating sequence-to-sequence representations in both context and aspect. It is capable of extracting features related to the specific aspect during the process of its context sequence modeling, and generating a high quality aspect representation simultaneously. We have conducted comprehensive experiments to compare with thirteen existing methods. Our experimental results show that the proposed model is able to achieve significantly better performance on Restaurant dataset, as well as very competitive results on Laptop and Twitter datasets.

For aspect level sentiment classification, the attention mechanisms of memory network can capture the importance of each context word when inferring the sentiment polarity of an aspect. However, the existing attention mechanisms of memory network are either content-based or location-based while the part of speech (POS) of word is not used. Accordingly, this paper proposes a new attention mechanism called AM-PPOSC which uses the location, POS and content information. In addition, the common-used algorithms for learning word embedding typically only model the syntactic context of words but ignore the sentiment polarities of words. So, this paper proposes a method that based on TD-LSTM and SentiWordNet 3.0 to capture the sentiment polarities of words which represented as word sentiment vectors. Then, AM-PPOSC and word sentiment vectors are applied to memory network. Experiments on two datasets demonstrate that not only AM-PPOSC outperforms the original attention mechanisms of memory network when inferring the sentiment polarity of an aspect, but also word sentiment vectors can improve the performance of memory network. Furthermore, compared with feature-based SVM, LSTM and TD-LSTM, memory network with AM-PPOSC and word sentiment vectors achieves higher accuracies on two datasets.

AELA-DLSTMs: Attention-Enabled and Location-Aware Double LSTMs for aspect-level sentiment classification

Aspect-level sentiment classification aims to predict sentiment polarities for different aspect terms within the same sentence or document. However, existing methods rely heavily on modeling the semantic relevance of an aspect term and its context words, and ignore the importance of syntax analysis to a certain extent. Consequently, this may cause the model to pay attention to the context word which is used to describe other aspect terms. In this paper, we propose a model which analyze sentences both syntactically and semantically. At the same time, we propose a simple and effective fusion mechanism to make the integration of aspect information and context information more adequately. We conduct extensive experiments on the SemEval 2014 benchmark datasets, and the results show that our model achieves a new state- of-the-art performance.

A Three-Level Information Architecture containing Syntactic Information, Semantic Information and Pragmatic Information is put forward in Comprehensive Information Theory (CIT). From this point of view, spatial data analysis is in cooperation with semantic information and Spatial Data Preprocessing (SDP) is corresponding to syntactic information. However, in many practical applications, SDP based only on syntactic information can not get a good effect. Semantics-based preprocessing may be an effective scheme. RS images mosaic is a typical SDP where optimal mosaic line extraction is the crux. Lots of researches based on syntactic information are effective just for orthophoto maps. In this paper, an overall optimal mosaic line extraction scheme has been addressed for non-Ortho RS images. It is argued that there is no projection error in the projection datum fitted by Ground Control Points (GCPs), or regional main height surface which can be recognized in medium resolution RS images. Based on above reasons, the method suggests that GCPs collecting for precise geometrical correction should be on the main height surface, as well as the mosaic line extracting for RS images mosaic. Three sheets of CBERS CCD images of Taiyuan are taken as the experimental data. According to the afore-mentioned method, by collecting GCPs in wide riverbeds, all three sheets are rectified to an existing ETM+ mosaic image. And then, the central lines of wide riverbeds in the overlapping areas are extracted as the mosaic line. The experimental result indicates that this method can extract an overall optimal mosaic line and eliminate the visual texture seam-line effectively, even for non-Ortho RS images. It concludes that SDP based on semantic information can play a good role in spatial data applications.

Sentence classification is a significant task in natural language processing (NLP) and is applied in many fields. The syntactic and semantic properties of words and phrases often determine the success of sentence classification. Previous approaches based on sequential modeling mainly ignored the explicit syntactic structures in a sentence. In this paper, we propose a Syntax-Aware Transformer (SA-Trans), which integrates syntactic information in the transformer and obtains sentence embeddings by combining syntactic and semantic information. We evaluate our SA-Trans on four benchmark classification datasets (i.e., AG’News, DBpedia, ARP, ARF), and the experimental results manifest that our SA-Trans model achieves competitive performance compared to the baseline models. Finally, the case study further demonstrates the importance of syntactic information for the classification task.KeywordsSyntaxTransformerSentence classification

Prior literature has debated whether syntax is separable from semantics in the brain. Using functional magnetic resonance imaging and multi-voxel pattern analysis, our previous studies investigated brain activity during morpho-syntactic versus semantic processing. These studies only detected semantic specialization in activation patterns and no syntactic specialization in 5- to 6-year-old and 7- to 8-year-old children. To examine if older children who have mastered morpho-syntactic skills would show specialization for syntax, the current study examined 64 9- to 10-year-old children using the same design and analyses. We observed that only the left IFG pars opercularis was sensitive to syntactic but not semantic information, supporting the hypothesis that this region serves as a core region for syntax. In addition, the left STG which has been implicated in the integration of semantics and syntax, as well as the left MTG and IFG pars triangularis which have been implicated in semantics, were sensitive to both semantic and syntactic information with no evidence of specialization. These findings suggest a lexicalized view of syntax, which argues that semantically sensitive regions are also critical regions for syntactic processing during language comprehension.

A large part of the development effort of compute-intensive applications is devoted to optimization, i.e., achieving the computation within a finite budget of time, space or energy. Given the complexity of modern architectures, writing simulation applications is often a two-step workflow. Firstly, developers design a sequential program for algorithmic tuning and debugging purposes. Secondly, experts optimize and exploit possible approximations of the original program to scale to the actual problem size. This second step is a tedious, time-consuming and error-prone task. In this paper we investigate language extensions and compiler tools to achieve that task semi-automatically in the context of approximate computing. We identified the semantic and syntactic information necessary for a compiler to automatically handle approximation and adaptive techniques for a particular class of programs. We propose a set of language extensions generic enough to provide the compiler with the useful semantic information when approximation is beneficial. We implemented the compiler infrastructure to exploit these extensions and to automatically generate the adaptively approximated version of a program. We provide an experimental study of the impact and expressiveness of our language extension set on various applications.