Measuring semantic similarity within sentences

Semantic information and syntactic information are two essential dimensions of language. In recent years, most researches focus on extracting semantic information or how to extract useful information from syntactic dependency tree. As the semantic representations are closely related to syntactic ones, how to effectively integrate semantic and syntactic information is still a subject worth studying. We propose attention mechanism based on graph convolutional networks. The model integrates syntactic dependence information into semantic features and automatically achieve a balance between semantic information and syntactic information. We conducted experiments on the task of sentence-level relation extraction, and the results show that our model performs better than some previous methods.

ALSC (Aspect-level Sentiment Classification) is a fine-grained task in the field of NLP (Natural Language Processing) which aims to identify the sentiment toward a given aspect. In addition to exploiting the sentence semantics and syntax, current ALSC methods focus on introducing external knowledge as a supplementary to the sentence information. However, the integration of the three categories of information is still challenging. In this paper, a novel method is devised to effectively combine sufficient semantic and syntactic information as well as use of external knowledge. The proposed model contains a sentence encoder, a semantic learning module, a syntax learning module, a knowledge enhancement module, an information fusion module and a sentiment classifier. The semantic information and syntactic information are respectively extracted via a self-attention network and a graphical convolutional network. Specifically, the KGE (Knowledge Graph Embedding) is employed to enhance the feature representation of the aspect. Then, the attention-based gate mechanism is taken to fuse three types of information. We evaluated the proposed model on three benchmark datasets and the experimental results establish strong evidence of high accuracy.

A Three-Level Information Architecture containing Syntactic Information, Semantic Information and Pragmatic Information is put forward in Comprehensive Information Theory (CIT). From this point of view, spatial data analysis is in cooperation with semantic information and Spatial Data Preprocessing (SDP) is corresponding to syntactic information. However, in many practical applications, SDP based only on syntactic information can not get a good effect. Semantics-based preprocessing may be an effective scheme. RS images mosaic is a typical SDP where optimal mosaic line extraction is the crux. Lots of researches based on syntactic information are effective just for orthophoto maps. In this paper, an overall optimal mosaic line extraction scheme has been addressed for non-Ortho RS images. It is argued that there is no projection error in the projection datum fitted by Ground Control Points (GCPs), or regional main height surface which can be recognized in medium resolution RS images. Based on above reasons, the method suggests that GCPs collecting for precise geometrical correction should be on the main height surface, as well as the mosaic line extracting for RS images mosaic. Three sheets of CBERS CCD images of Taiyuan are taken as the experimental data. According to the afore-mentioned method, by collecting GCPs in wide riverbeds, all three sheets are rectified to an existing ETM+ mosaic image. And then, the central lines of wide riverbeds in the overlapping areas are extracted as the mosaic line. The experimental result indicates that this method can extract an overall optimal mosaic line and eliminate the visual texture seam-line effectively, even for non-Ortho RS images. It concludes that SDP based on semantic information can play a good role in spatial data applications.

Because of the complexity and uncertainty of sentences, ambiguous information often appears in sentence segmentation. To the sentence ambiguity division problem, we proposed a method that can solve the semantic information in the sentence and can select the breakpoint of word dynamically. This study proposed and designed a single-objective discrete optimization model based on the ontology technology of lexeme semantic information. This model analyzed the semantic information of the sentences through the ontology data analysis, combined the case data to calculate the relevant semantic word frequency information; then proposed an optimized artificial bee colony algorithm to solve the model, by introduced an adaptive competition strategy in the selection and elimination of the population to accelerate the convergence speed of the algorithm. Finally, this study experimented on the word segmentation module of the library automatic question-answer consultation system. The experimental results showed that this method can meet the user’s requirements, improve the question segmentation accuracy, accelerate the convergence speed of the bee colony, which verified the effectiveness of the proposed method.KeywordsChinese semantic segmentationAmbiguous informationArtificial bee colony algorithmDynamic optimization

Prior literature has debated whether syntax is separable from semantics in the brain. Using functional magnetic resonance imaging and multi-voxel pattern analysis, our previous studies investigated brain activity during morpho-syntactic versus semantic processing. These studies only detected semantic specialization in activation patterns and no syntactic specialization in 5- to 6-year-old and 7- to 8-year-old children. To examine if older children who have mastered morpho-syntactic skills would show specialization for syntax, the current study examined 64 9- to 10-year-old children using the same design and analyses. We observed that only the left IFG pars opercularis was sensitive to syntactic but not semantic information, supporting the hypothesis that this region serves as a core region for syntax. In addition, the left STG which has been implicated in the integration of semantics and syntax, as well as the left MTG and IFG pars triangularis which have been implicated in semantics, were sensitive to both semantic and syntactic information with no evidence of specialization. These findings suggest a lexicalized view of syntax, which argues that semantically sensitive regions are also critical regions for syntactic processing during language comprehension.

A large part of the development effort of compute-intensive applications is devoted to optimization, i.e., achieving the computation within a finite budget of time, space or energy. Given the complexity of modern architectures, writing simulation applications is often a two-step workflow. Firstly, developers design a sequential program for algorithmic tuning and debugging purposes. Secondly, experts optimize and exploit possible approximations of the original program to scale to the actual problem size. This second step is a tedious, time-consuming and error-prone task. In this paper we investigate language extensions and compiler tools to achieve that task semi-automatically in the context of approximate computing. We identified the semantic and syntactic information necessary for a compiler to automatically handle approximation and adaptive techniques for a particular class of programs. We propose a set of language extensions generic enough to provide the compiler with the useful semantic information when approximation is beneficial. We implemented the compiler infrastructure to exploit these extensions and to automatically generate the adaptively approximated version of a program. We provide an experimental study of the impact and expressiveness of our language extension set on various applications.

Sentence similarity methods are used to assess the degree of likelihood between phrases. Many natural language applications such as text summarization, information retrieval, text categorization, and machine translation employ measures of sentence similarity. The existing approaches for this problem represent sentences as vectors of bag of words or the syntactic information of the words in the phrase. The likelihood between phrases is calculated by composing the similarity between the words in the sentences. Such schemes do not address two important concerns in the area, however: the semantic problem and the word order. This paper proposes a new sentence similarity measure that attempts to address such problems by taking into account the lexical, syntactic, and semantic analysis of sentences. The new similarity measure proposed outperforms the state of the art systems in around 6%, when tested using a standard and publically available dataset.

Sentence classification is a significant task in natural language processing (NLP) and is applied in many fields. The syntactic and semantic properties of words and phrases often determine the success of sentence classification. Previous approaches based on sequential modeling mainly ignored the explicit syntactic structures in a sentence. In this paper, we propose a Syntax-Aware Transformer (SA-Trans), which integrates syntactic information in the transformer and obtains sentence embeddings by combining syntactic and semantic information. We evaluate our SA-Trans on four benchmark classification datasets (i.e., AG’News, DBpedia, ARP, ARF), and the experimental results manifest that our SA-Trans model achieves competitive performance compared to the baseline models. Finally, the case study further demonstrates the importance of syntactic information for the classification task.KeywordsSyntaxTransformerSentence classification

This study is twofold: (i) whether there is a significant difference between Chinese learners’ first language (L1) proficiency and second language (L2) performance in comprehension of certain multiply self-embedded complex English sentences such as (1) Tom knows that the fact that taking good care of himself is necessary surprises the boy; (ii) whether there is a relationship between Chinese learners’ L1 proficiency and L2 processing strategies in the use of syntactic and/or semantic information in the comprehension performance. Two groups of advanced Chinese learners of English differing in Chinese proficiency (i.e. fifteen high Chinese proficiency (HCP) and fifteen low Chinese proficiency (LCP) learners of English) and one control group participated in the off-line experiment to answer questions to English sentences based on comprehension accuracy. The results show that there are no significant accuracy differences between HCP learners and LCP learners in the complex English sentence comprehension, suggesting that L1 proficiency play no significant role in Chinese learners’ complex English sentence comprehension. When doing a complex-rating task where participants rated complexity of the self-embedded complex English sentences based on a five-point scale, the results show that the LCP Chinese learners of English, like the control native English speakers, were able to use both syntactic and semantic information. However, the HCP Chinese learners of English relied more on semantic information than syntactic cues, indicating that L1 proficiency may influence Chinese learners’ processing strategy in comprehension of the complex English sentences.

In this paper, an improved long short-term memory (LSTM)-based deep neural network structure is proposed for learning variable-length Chinese sentence semantic similarities. Siamese LSTM, a sequence-insensitive deep neural network model, has a limited ability to capture the semantics of natural language because it has difficulty explaining semantic differences based on the differences in syntactic structures or word order in a sentence. Therefore, the proposed model integrates the syntactic component features of the words in the sentence into a word vector representation layer to express the syntactic structure information of the sentence and the interdependence between words. Moreover, a relative position embedding layer is introduced into the model, and the relative position of the words in the sentence is mapped to a high-dimensional space to capture the local position information of the words. With this model, a parallel structure is used to map two sentences into the same high-dimensional space to obtain a fixed-length sentence vector representation. After aggregation, the sentence similarity is computed in the output layer. Experiments with Chinese sentences show that the model can achieve good results in the calculation of the semantic similarity.

Parallel-fusion LSTM with synchronous semantic and visual information for image captioning

Recently pre-trained models have achieved state-of-the-art results in various language understanding tasks. Current pre-training procedures usually focus on training the model with several simple tasks to grasp the co-occurrence of words or sentences. However, besides co-occurring information, there exists other valuable lexical, syntactic and semantic information in training corpora, such as named entities, semantic closeness and discourse relations. In order to extract the lexical, syntactic and semantic information from training corpora, we propose a continual pre-training framework named ERNIE 2.0 which incrementally builds pre-training tasks and then learn pre-trained models on these constructed tasks via continual multi-task learning. Based on this framework, we construct several tasks and train the ERNIE 2.0 model to capture lexical, syntactic and semantic aspects of information in the training data. Experimental results demonstrate that ERNIE 2.0 model outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several similar tasks in Chinese. The source codes and pre-trained models have been released at https://github.com/PaddlePaddle/ERNIE.

In the traditional task of aspect sentiment triplet extraction, existing approaches typically focus on either syntactic or semantic features independently, failing to leverage the complementary integration of these two types of information. Although graph convolutional network-based approaches have demonstrated impressive performance in triplet extraction tasks, they often ignore distance features and semantic information when capturing sentence information. As a result, the integration of syntactic and semantic information remains suboptimal, negatively impacting sentiment analysis performance. To address this limitation, we propose a novel Syntax-Semantics Graph Convolutional Network for aspect sentiment triplet extraction. Our method first extracts syntactic structural information using the probability matrix of dependency trees, from which a mask matrix is constructed based on the varying distances between words. Next, semantic information is captured via a self-attention mechanism and an aspect-attention mechanism, utilizing an attention score matrix. Finally, an interaction module is introduced to effectively integrate syntactic and semantic features. Extensive experiments on several benchmark datasets demonstrate that our approach significantly outperforms existing baselines, achieving an average F1-score improvement of at least 1.083%.

Traditional approaches to simultaneous localization and mapping (SLAM) are unable to extract semantic information from the scene or meet the high-level task requirements of robots, and the efficiency of 3D map construction is low. To solve this problem, a 3D semantic map construction system is proposed in the paper to build a 3D semantic map. Firstly, the current position of the camera is estimated and optimized based on ORB-SLAM algorithm, and the globally consistent trajectory and pose are obtained. Then the semantic segmentation network is designed to predict semantic category of every pixel. The 3D semantic point cloud information is generated by combing the semantic information and the object point cloud. The global consistent camera pose estimated by visual SLAM algorithm is integrated with the semantic point cloud information to generate a 3D semantic map. Finally, we use an Octomap which is applied to navigation projects for map storage to reduce the storage capacity of the map. The experimental result verifies the accuracy and efficiency of this method.

The information really useful to humans must be the trinity of its three components, the form termed syntactic information, the meaning termed semantic information, and the utility termed pragmatic information. But the theory of information set up by Shannon in 1948 is a statistical theory of syntactic information. Thus, the trinity of information theories needs be established as urgently as possible. Such a theory of semantic information will be presented in the paper and it will also be proved that it is the semantic information that is the unique representative of the trinity. This is why the title of the paper is set to “a theory of semantic information” without mentioning the pragmatic information.