A Short-Text Similarity Model Combining Semantic and Syntactic Information

Semantic information and syntactic information are two essential dimensions of language. In recent years, most researches focus on extracting semantic information or how to extract useful information from syntactic dependency tree. As the semantic representations are closely related to syntactic ones, how to effectively integrate semantic and syntactic information is still a subject worth studying. We propose attention mechanism based on graph convolutional networks. The model integrates syntactic dependence information into semantic features and automatically achieve a balance between semantic information and syntactic information. We conducted experiments on the task of sentence-level relation extraction, and the results show that our model performs better than some previous methods.

ALSC (Aspect-level Sentiment Classification) is a fine-grained task in the field of NLP (Natural Language Processing) which aims to identify the sentiment toward a given aspect. In addition to exploiting the sentence semantics and syntax, current ALSC methods focus on introducing external knowledge as a supplementary to the sentence information. However, the integration of the three categories of information is still challenging. In this paper, a novel method is devised to effectively combine sufficient semantic and syntactic information as well as use of external knowledge. The proposed model contains a sentence encoder, a semantic learning module, a syntax learning module, a knowledge enhancement module, an information fusion module and a sentiment classifier. The semantic information and syntactic information are respectively extracted via a self-attention network and a graphical convolutional network. Specifically, the KGE (Knowledge Graph Embedding) is employed to enhance the feature representation of the aspect. Then, the attention-based gate mechanism is taken to fuse three types of information. We evaluated the proposed model on three benchmark datasets and the experimental results establish strong evidence of high accuracy.

Abstractive single document summarization is considered as a challenging problem in the field of artificial intelligence and natural language processing. Meanwhile and specifically in the last two years, several deep learning summarization approaches were proposed that once again attracted the attention of researchers to this field.It is a well-known issue that deep learning approaches do not work well with small amounts of data. With some exceptions, this is, unfortunately, the case for most of the datasets available for the summarization task. Besides this problem, it should be considered that phonetic, morphological, semantic and syntactic features of the language are constantly changing over the time and unfortunately most of the summarization corpora are constructed from old resources. Another problem is the language of the corpora. Not only in the summarization field, but also in other fields of natural language processing, most of the corpora are only available in English. In addition to the above problems, license terms, and fees of the corpora are obstacles that prevent many academics and specifically non-academics from accessing these data.This work describes an open source framework to create an extendable multilingual corpus for abstractive single document summarization that addresses the above-mentioned problems. We describe a tool consisted of a scalable crawler and a centralized key-value store database to construct a corpus of an arbitrary size using a news aggregator service.

In the last years, the need to de-identify privacy-sensitive information within Electronic Health Records (EHRs) has become increasingly felt and extremely relevant to encourage the sharing and publication of their content in accordance with the restrictions imposed by both national and supranational privacy authorities. In the field of Natural Language Processing (NLP), several deep learning techniques for Named Entity Recognition (NER) have been applied to face this issue, significantly improving the effectiveness in identifying sensitive information in EHRs written in English. However, the lack of data sets in other languages has strongly limited their applicability and performance evaluation. To this aim, a new de-identification data set in Italian has been developed in this work, starting from the 115 COVID-19 EHRs provided by the Italian Society of Radiology (SIRM): 65 were used for training and development, the remaining 50 were used for testing. The data set was labelled following the guidelines of the i2b2 2014 de-identification track. As additional contribution, combined with the best performing Bi-LSTM + CRF sequence labeling architecture, a stacked word representation form, not yet experimented for the Italian clinical de-identification scenario, has been tested, based both on a contextualized linguistic model to manage word polysemy and its morpho-syntactic variations and on sub-word embeddings to better capture latent syntactic and semantic similarities. Finally, other cutting-edge approaches were compared with the proposed model, which achieved the best performance highlighting the goodness of the promoted approach.

Sentence representations play an important role in the field of natural language processing. While word representation has been applied to many natural language processing tasks, sentence representation has not been applied as widely due to the more complex structure and richer syntactic information of sentences. To learn sentence representations with structural and syntactic information, we propose a new model called the part-of-speech-based Long Short-Term Memory network (pos-LSTM) model. The pos-LSTM model generates a structural representation using the standard LSTM model and a syntactic representation using the part of speech; then, the pos-LSTM model obtains the final sentence representation by combining these two representations. The experimental results from 20 sentence similarity tasks and an entailment classification task show that the pos-LSTM model can better capture the syntactic information of sentences and generate higher quality sentence representations than traditional models.

Fine-grained entity typing, which aims to assign specific types to entity mentions in text, is attracting increasing attention in the field of natural language processing (NLP). However, it is quite a challenging problem due to the highly ambiguous nature of many entity mentions. Most existing entity typing methods based on attention mechanism generally extract the salient features separately from the entity mention and the contextual words. However, these approaches suffer from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">two main limitations</i> : (1) They ignore the rich information contained by entity mentions when applying the attention mechanisms. (2) They do not consider the diversity of attention processes which can be beneficial in finding the discriminative features. To address these issues, we propose the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">diversified semantic attention model (DSAM)</i> for fine-grained entity typing, and the main novelties are: (1) It explicitly pursues the diversity of attention and is able to maximally gather discriminative information. (2) It integrates two level attentions—the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mention-level attention</i> and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">context-level attention—</i> to jointly capture the rich information from mentions and contexts to enhance their mutual promotions. (3) It combines the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">attention maps constraint</i> and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">attention segments constrain</i> to exploit the subtle semantic differences for distinguishing the subtypes. Importantly, the proposed DSAM approach can be trained end-to-end without employing ad-hoc features or post-processing. Extensive experiments using three benchmark datasets demonstrated that our DSAM approach achieves competitive performance compared to the current state-of-the-art methods used for fine-grained entity typing.

Aspect Sentiment Triplet Extraction (ASTE) is a complex and challenging task in Natural Language Processing (NLP). It aims to extract the triplet of aspect term, opinion term, and their associated sentiment polarity, which is a more fine-grained study in Aspect Based Sentiment Analysis. Furthermore, there have been a large number of approaches being proposed to handle this relevant task. However, existing methods for ASTE suffer from powerless interactions between different sources of textual features, and they usually exert an equal impact on each type of feature, which is quite unreasonable while building contextual representation. Therefore, in this paper, we propose a novel Multi-Branch GCN (MBGCN)-based ASTE model to solve this problem. Specifically, our model first generates the enhanced semantic features via the structure-biased BERT, which takes the position of tokens into the transformation of self-attention. Then, a biaffine attention module is utilized to further obtain the specific semantic feature maps. In addition, to enhance the dependency among words in the sentence, four types of linguistic relations are defined, namely part-of-speech combination, syntactic dependency type, tree-based distance, and relative position distance of each word pair, which are further embedded as adjacent matrices. Then, the widely used Graph Convolutional Network (GCN) module is utilized to complete the work of integrating the semantic feature and linguistic feature, which is operated on four types of dependency relations repeatedly. Additionally, an effective refining strategy is employed to detect whether word pairs match or not, which is conducted after the operation of each branch GCN. At last, a shallow interaction layer is designed to achieve the final textual representation by fusing the four branch features with different weights. To validate the effectiveness of MBGCNs, extensive experiments have been conducted on four public and available datasets. Furthermore, the results demonstrate the effectiveness and robustness of MBGCNs, which obviously outperform state-of-the-art approaches.

In the era of information explosion, named entity recognition (NER) has attracted widespread attention in the field of natural language processing, as it is fundamental to information extraction. Recently, methods of NER based on representation learning, e.g., character embedding and word embedding, have demonstrated promising recognition results. However, existing models only consider partial features derived from words or characters while failing to integrate semantic and syntactic information, e.g., capitalization, inter-word relations, keywords, and lexical phrases, from multilevel perspectives. Intuitively, multilevel features can be helpful when recognizing named entities from complex sentences. In this study, we propose a novel attentive multilevel feature fusion (AMFF) model for NER, which captures the multilevel features in the current context from various perspectives. It consists of four components to, respectively, capture the local character-level (CL), global character-level (CG), local word-level (WL), and global word-level (WG) features in the current context. In addition, we further define document-level features crafted from other sentences to enhance the representation learning of the current context. To this end, we introduce a novel context-aware attentive multilevel feature fusion (CAMFF) model based on AMFF, to fully leverage document-level features from all the previous inputs. The obtained multilevel features are then fused and fed into a bidirectional long short-term memory (BiLSTM)-conditional random field (CRF) network for the final sequence labeling. Extensive experiments on four benchmark datasets demonstrate that our proposed AMFF and CAMFF models outperform a set of state-of-the-art baseline methods and the features learned from multiple levels are complementary.

Sentence classification is a significant task in natural language processing (NLP) and is applied in many fields. The syntactic and semantic properties of words and phrases often determine the success of sentence classification. Previous approaches based on sequential modeling mainly ignored the explicit syntactic structures in a sentence. In this paper, we propose a Syntax-Aware Transformer (SA-Trans), which integrates syntactic information in the transformer and obtains sentence embeddings by combining syntactic and semantic information. We evaluate our SA-Trans on four benchmark classification datasets (i.e., AG’News, DBpedia, ARP, ARF), and the experimental results manifest that our SA-Trans model achieves competitive performance compared to the baseline models. Finally, the case study further demonstrates the importance of syntactic information for the classification task.KeywordsSyntaxTransformerSentence classification

Sentiment analysis is an important branch of the field of natural language processing, and opinion sentiment analysis is the current research hotspot. With the development of artificial intelligence, it is very challenging to effectively extract important emotional information from a large amount of text information and analyzes the emotional tendency of views in response to the increasing number of emotional opinions with rich connotations. Today’s methods mostly use shallow emotional factors, while ignoring deeper text semantic information, and cannot explore the semantic connection between words. To make up for this deficiency, this paper proposes a triple-channel feature-mixed sentiment analysis model Tri-BiGRU-Atten based on attention mechanism. This model combines different semantic feature mixed modeling to enable it to mine deeper emotional information in the context. Compared with the traditional attention mechanism, LSTM, Bi-LSTM, and other models, the emotion classification effect is more effective.

The rich semantic information contained in images is an important clue to explore visual emotions. Therefore, exploring the correlation between visual emotion and the semantic relationship of objects, and extracting more effective semantic features through explicit or implicit modeling is very important for visual emotion analysis. In this paper, a novel Graph-based Object Semantic Refinement (GOSR) model is proposed to extract multi-level semantic features for visual emotion classification, in which graph structures is used to represent the object semantics and their position relationships of an image, and Graph Convolutional Networks (GCN) is used to refine object information by the aggregating neighbor object with their position relationships. The different convolutional layer’s features from GCN are further fused by Gated Recurrent Units (GRU) networks to achieve high-level semantic features. Then a framework with two branches to leverage visual and semantic information for visual sentiment analysis is proposed, which uses convolutional neural networks to extract visual features from images, and collaborates with semantic features from GOSR model to achieve better emotion recognition results. Besides, for alleviating the potentially unreasonable predictions and promote models collaboration, a novel tendency loss function based on the correlations among emotion labels is proposed to adjust the output activation value other than the target label. Extensive experiments on four widely used benchmark datasets show that our proposed method can achieve competitive performance and outperform most of the state-of-the-art methods on visual emotion recognition.

In recent years, due to the vigorous development of social network media, a large amount of social data can be obtained through social media. This trend has made it easier for researchers in natural language processing to obtain textual research materials, and has further stimulated the development of natural language processing. Emotion recognition is an important task in the field of natural language processing, and emotion recognition helps improve interaction in social. In this study, the emotion recognition of the content of twitter users’ posts and comments is based on the sentence level. We use large datasets to collect users’ posts and comments on Twitter for comparative analysis, and we attempt to understand the emotions expressed by users. Most of the recent studies focus on the analysis of vocabulary and syntactic features in sentences to learn the emotion of the sentence. However, the property of multi-feature may be conducive for promoting the performance of the model and may also be a disturbance of the training of the model. Our study is based on the latest of attention mechanism and XLNet to improve the recognition of emotions. We propose a novel emotion recognition model called Dynamic Weighted Attention with Multichannel Convolutional Neural Network (DACNN), which combines the multi-channel convolutional neural network and the attention mechanism of automatic weight adjustment to effectively improve the outcome of emotion recognition. In addition, we use the latest word embedding technology XLNet to obtain high-quality feature vectors of sentences. In the experiments, we compare the DACNN with the baselines on various datasets, and the results show that DACNN is better than the state-of-the-art methods in accuracy, precision, recall, and fl-score. The experimental results also prove that our attention mechanism and dynamic weighting can well promote the accuracy of emotion recognition.

This paper introduces a new, graph-based view of the data of the FrameNet project, which we hope will make it easier to understand the mixture of semantic and syntactic information contained in FrameNet annotation. We show how English FrameNet and other Frame Semantic resources can be represented as sets of interconnected graphs of frames, frame elements, semantic types, and annotated instances of them in text. We display examples of the new graphical representation based on the annotations, which combine Frame Semantics and Construction Grammar, thus capturing most of the syntax and semantics of each sentence. We consider how graph theory could help researchers to make better use of FrameNet data for tasks such as automatic Frame Semantic role labeling, paraphrasing, and translation. Finally, we describe the development of FrameNet-like lexical resources for other languages in the current Multilingual FrameNet project. which seeks to discover cross-lingual alignments, both in the lexicon (for frames and lexical units within frames) and across parallel or comparable texts. We conclude with an example showing graphically the semantic and syntactic similarities and differences between parallel sentences in English and Japanese. We will release software for displaying such graphs from the current data releases.

Aspect-based sentiment analysis (ABSA) aims to identify aspect terms in texts and determine their sentiment polarities. The in-context learning (ICL) paradigm, powered by large language models (LLMs), has proven effective in low-resource scenarios, where the retrieval of effective demonstration examples is crucial. Existing retrieval methods prioritize semantic and syntactic similarities, overlooking the fact that sentiment is often driven by its underlying causes. Recognizing that similar causes tend to yield similar sentiments, we propose the Semantic-Causal Contextual Demonstration Retrieval (SCCDR), a demonstration retriever that integrates semantic and syntactic information while explicitly modeling sentiment causes. SCCDR was trained using contrastive learning based on rich contextual signals, including semantics, aspect-sentiment relationships, syntactic structures, and sentiment causes. Experiments on four datasets show that SCCDR outperforms other retrieval methods, thereby effectively improving ABSA performance under the ICL paradigm.

The field of natural language processing (NLP) is being increasingly applied to the realm of healthcare. This abstract delves into the intersection of health care and NLP, highlighting how this technology is being used to enhance various aspects of the healthcare industry. From analyzing medical records and extracting relevant information to aiding in diagnosis and treatment recommendations, NLP is playing a pivotal role in transforming how healthcare professionals interact with data and make informed decisions. This abstract provides an overview of the key applications and benefits of employing NLP in healthcare, showcasing its potential to improve patient outcomes and streamline medical processes. This research seeks to harness the power of advanced computational linguistics to revolutionize various aspects of the healthcare industry. By enabling computers to understand and interpret human language, NLP has the potential to drive transformative changes NLP-powered systems can analyze vast amounts of medical literature, patient records, and research findings to provide healthcare practitioners with timely and relevant information. This assists in making well-informed decisions regarding diagnoses, treatment plans, and prognoses. NLP techniques facilitate the extraction of pertinent information from unstructured medical records, including patient histories and doctor's notes. This structured data can then be used for research, trend analysis, and epidemiological studies, contributing to evidence-based medicine. Versatile with unique alternatives a new method for optimization is proposed MOORA. This method is objective denotes the matrix of responses of the alternatives, however, proposing better policies, which rates are used. Well established. Multi-objective another method for optimization is used for comparison, reference point method. Then, various competitions this proved to be the best choice among the methods. From the result BC2GM is in 1st rank whereas BC5- Disease is in lowest rank.