A language extension set to generate adaptive versions automatically

Semantic information and syntactic information are two essential dimensions of language. In recent years, most researches focus on extracting semantic information or how to extract useful information from syntactic dependency tree. As the semantic representations are closely related to syntactic ones, how to effectively integrate semantic and syntactic information is still a subject worth studying. We propose attention mechanism based on graph convolutional networks. The model integrates syntactic dependence information into semantic features and automatically achieve a balance between semantic information and syntactic information. We conducted experiments on the task of sentence-level relation extraction, and the results show that our model performs better than some previous methods.

ALSC (Aspect-level Sentiment Classification) is a fine-grained task in the field of NLP (Natural Language Processing) which aims to identify the sentiment toward a given aspect. In addition to exploiting the sentence semantics and syntax, current ALSC methods focus on introducing external knowledge as a supplementary to the sentence information. However, the integration of the three categories of information is still challenging. In this paper, a novel method is devised to effectively combine sufficient semantic and syntactic information as well as use of external knowledge. The proposed model contains a sentence encoder, a semantic learning module, a syntax learning module, a knowledge enhancement module, an information fusion module and a sentiment classifier. The semantic information and syntactic information are respectively extracted via a self-attention network and a graphical convolutional network. Specifically, the KGE (Knowledge Graph Embedding) is employed to enhance the feature representation of the aspect. Then, the attention-based gate mechanism is taken to fuse three types of information. We evaluated the proposed model on three benchmark datasets and the experimental results establish strong evidence of high accuracy.

A Three-Level Information Architecture containing Syntactic Information, Semantic Information and Pragmatic Information is put forward in Comprehensive Information Theory (CIT). From this point of view, spatial data analysis is in cooperation with semantic information and Spatial Data Preprocessing (SDP) is corresponding to syntactic information. However, in many practical applications, SDP based only on syntactic information can not get a good effect. Semantics-based preprocessing may be an effective scheme. RS images mosaic is a typical SDP where optimal mosaic line extraction is the crux. Lots of researches based on syntactic information are effective just for orthophoto maps. In this paper, an overall optimal mosaic line extraction scheme has been addressed for non-Ortho RS images. It is argued that there is no projection error in the projection datum fitted by Ground Control Points (GCPs), or regional main height surface which can be recognized in medium resolution RS images. Based on above reasons, the method suggests that GCPs collecting for precise geometrical correction should be on the main height surface, as well as the mosaic line extracting for RS images mosaic. Three sheets of CBERS CCD images of Taiyuan are taken as the experimental data. According to the afore-mentioned method, by collecting GCPs in wide riverbeds, all three sheets are rectified to an existing ETM+ mosaic image. And then, the central lines of wide riverbeds in the overlapping areas are extracted as the mosaic line. The experimental result indicates that this method can extract an overall optimal mosaic line and eliminate the visual texture seam-line effectively, even for non-Ortho RS images. It concludes that SDP based on semantic information can play a good role in spatial data applications.

Prior literature has debated whether syntax is separable from semantics in the brain. Using functional magnetic resonance imaging and multi-voxel pattern analysis, our previous studies investigated brain activity during morpho-syntactic versus semantic processing. These studies only detected semantic specialization in activation patterns and no syntactic specialization in 5- to 6-year-old and 7- to 8-year-old children. To examine if older children who have mastered morpho-syntactic skills would show specialization for syntax, the current study examined 64 9- to 10-year-old children using the same design and analyses. We observed that only the left IFG pars opercularis was sensitive to syntactic but not semantic information, supporting the hypothesis that this region serves as a core region for syntax. In addition, the left STG which has been implicated in the integration of semantics and syntax, as well as the left MTG and IFG pars triangularis which have been implicated in semantics, were sensitive to both semantic and syntactic information with no evidence of specialization. These findings suggest a lexicalized view of syntax, which argues that semantically sensitive regions are also critical regions for syntactic processing during language comprehension.

Sentence similarity measures have taken an increasingly important role in a variety of applications of text knowledge presentation and discovery. Current methods utilize only semantic or syntactic information. We propose a novel method to calculate sentence similarity, which takes into account both semantic information and word order. The experimental results show that our method outperforms existing methods.

Senses are the extensions of language and interrelations between word meanings within language and the basis of flexible language use. Computers are basically made to process symbols. Computational linguistics is to simulate part of, if not all of, the natural language competence. From the perspective of computational linguistics, this paper sheds some light on semantic formalization, and proposes tagging of senses along with tagging weight values when processing word meanings. Also, this paper makes attempts to apply sense weight values to sentence processing in computational linguistics. Keywords-semantics, sense, weight, formalization, computational linguistics I. SEMANTIC KNOWLEDGE IN LANGUAGE PROCESSING According to prototype theory, semantic knowledge is part of human’s cognitive and belief systems. Semantics is also the critical research area since its initiation in 1893 by Michael Breal, and then by Trier and Lakoff who contributes to semantic research from both the descriptive and generative perspective. In linguistic semantics, rules and principles explain how literal meaning is coded in language and how to understand the “senses” of an expression or sense-relations that hold between one word and other expressions within the same language (Ceusters et al, 1999). Language is the carrier of both thoughts and forms. Thoughts are to induce, deduce, infer and judge, while senses are the contents and basis of thoughts. Senses are not the repeated addition of forms, rather a psychological process. Semantics is the mapping of cognitive structure from language expressions. Based on concepts, which are the abstract and general categorization of objects, people can apply semantic principles, situational contexts and encyclopedic knowledge to construct cognitive schema of the outside world. With frequent interactions and cognitive strategies, limited forms can be used to express unlimited connotation of thoughts (Tesser and Leone, 1977). For example, same meanings can be expressed with different semantic forms. a) 他这个人真有意思 (funny); b) 于是人们以为他们有了意思 (wish); c) 你向他意思意思 (express)如何; d) .我根本没有那个意思 (thought); e) 他常说人生真没有意思 (nonsense); f) 你们这么说是什么意思 (intention) These sentences are easy to comprehend. People not only use the grammatical structures but also the sense weights to process the words. The sense weight schema are stored in memory, and activated to represent and distill other sense weights. In this chain activation, sense weights are restructured and created to reflect prototypes. For example, “你好?”, “你好么?”, “好么你?”. Different structures are used in the above examples, but their essential meaning remains the same. So, semantic knowledge is of utmost importance in language comprehension. II. THE STATUS QUO OF COMPUTATIONAL LINGUISTICS Three domains and three corresponding levels of study are involved in building natural language processing systems, i.e. theoretical linguistics and grammar level, computational linguistics and formalization level, computer technology and realization level. Among them, grammar level concerns the innate characteristics of language and paves the way for the latter modules. Realization level focuses on how to control computer operation and provides the efficient development tools and environment. Formalization level is the interface of grammar level and realization level, with the task to readjust the general grammar modules to form grammar modules for convenient computer processing. Computational linguists’ mission is to establish formal grammars (Kaufmann and Pfister, 2012). Computational linguistics is also to establish formalized mathematical modules to analyze and process natural language, and develop computer programs to make it possible that machines can simulate part of, if not all of, human behavior and competence. Computers can only be used to process symbols and all those processed by computers are symbolized with no exception of languages (Yushi, 2003). Meanwhile, there have been breakthroughs in processing Chinese characters and phrases, so sentence level processing has become critical. Basic researches in syntactic, semantic and pragmatic knowledge are the frontier studies in Chinese processing. And how to acquire syntactic and semantic knowledge is the key, while the latter constitutes the major difficulty in processing the word level senses. As aforementioned, semantic formalization can take on many impossible missions in artificial intelligence and computational linguistics, and face challenges at the same time. Thus, the researcher will use semantic field to investigate formalization issues. Semantic fields are the sememe systems, which include the semantemes to represent similarities and differences, and are classified by the subordinates, i.e. degree and number of abstraction of Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013) Published by Atlantis Press, Paris, France.

Sentence classification is a significant task in natural language processing (NLP) and is applied in many fields. The syntactic and semantic properties of words and phrases often determine the success of sentence classification. Previous approaches based on sequential modeling mainly ignored the explicit syntactic structures in a sentence. In this paper, we propose a Syntax-Aware Transformer (SA-Trans), which integrates syntactic information in the transformer and obtains sentence embeddings by combining syntactic and semantic information. We evaluate our SA-Trans on four benchmark classification datasets (i.e., AG’News, DBpedia, ARP, ARF), and the experimental results manifest that our SA-Trans model achieves competitive performance compared to the baseline models. Finally, the case study further demonstrates the importance of syntactic information for the classification task.KeywordsSyntaxTransformerSentence classification

This study is twofold: (i) whether there is a significant difference between Chinese learners’ first language (L1) proficiency and second language (L2) performance in comprehension of certain multiply self-embedded complex English sentences such as (1) Tom knows that the fact that taking good care of himself is necessary surprises the boy; (ii) whether there is a relationship between Chinese learners’ L1 proficiency and L2 processing strategies in the use of syntactic and/or semantic information in the comprehension performance. Two groups of advanced Chinese learners of English differing in Chinese proficiency (i.e. fifteen high Chinese proficiency (HCP) and fifteen low Chinese proficiency (LCP) learners of English) and one control group participated in the off-line experiment to answer questions to English sentences based on comprehension accuracy. The results show that there are no significant accuracy differences between HCP learners and LCP learners in the complex English sentence comprehension, suggesting that L1 proficiency play no significant role in Chinese learners’ complex English sentence comprehension. When doing a complex-rating task where participants rated complexity of the self-embedded complex English sentences based on a five-point scale, the results show that the LCP Chinese learners of English, like the control native English speakers, were able to use both syntactic and semantic information. However, the HCP Chinese learners of English relied more on semantic information than syntactic cues, indicating that L1 proficiency may influence Chinese learners’ processing strategy in comprehension of the complex English sentences.

Recently pre-trained models have achieved state-of-the-art results in various language understanding tasks. Current pre-training procedures usually focus on training the model with several simple tasks to grasp the co-occurrence of words or sentences. However, besides co-occurring information, there exists other valuable lexical, syntactic and semantic information in training corpora, such as named entities, semantic closeness and discourse relations. In order to extract the lexical, syntactic and semantic information from training corpora, we propose a continual pre-training framework named ERNIE 2.0 which incrementally builds pre-training tasks and then learn pre-trained models on these constructed tasks via continual multi-task learning. Based on this framework, we construct several tasks and train the ERNIE 2.0 model to capture lexical, syntactic and semantic aspects of information in the training data. Experimental results demonstrate that ERNIE 2.0 model outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several similar tasks in Chinese. The source codes and pre-trained models have been released at https://github.com/PaddlePaddle/ERNIE.

In the traditional task of aspect sentiment triplet extraction, existing approaches typically focus on either syntactic or semantic features independently, failing to leverage the complementary integration of these two types of information. Although graph convolutional network-based approaches have demonstrated impressive performance in triplet extraction tasks, they often ignore distance features and semantic information when capturing sentence information. As a result, the integration of syntactic and semantic information remains suboptimal, negatively impacting sentiment analysis performance. To address this limitation, we propose a novel Syntax-Semantics Graph Convolutional Network for aspect sentiment triplet extraction. Our method first extracts syntactic structural information using the probability matrix of dependency trees, from which a mask matrix is constructed based on the varying distances between words. Next, semantic information is captured via a self-attention mechanism and an aspect-attention mechanism, utilizing an attention score matrix. Finally, an interaction module is introduced to effectively integrate syntactic and semantic features. Extensive experiments on several benchmark datasets demonstrate that our approach significantly outperforms existing baselines, achieving an average F1-score improvement of at least 1.083%.

The information really useful to humans must be the trinity of its three components, the form termed syntactic information, the meaning termed semantic information, and the utility termed pragmatic information. But the theory of information set up by Shannon in 1948 is a statistical theory of syntactic information. Thus, the trinity of information theories needs be established as urgently as possible. Such a theory of semantic information will be presented in the paper and it will also be proved that it is the semantic information that is the unique representative of the trinity. This is why the title of the paper is set to “a theory of semantic information” without mentioning the pragmatic information.

The domain of Aspect Level Sentiment Classification, in which the sentiment toward a given aspect is analyzed, attracts much attention in NLP. Recently, the state-of-the-art Aspect Level Sentiment Classification methods are devised by using the Graph Convolutional Networks to deal with both the semantics and the syntax of the sentence. Generally, the parsing of syntactic structure inevitably incorporates irrelevant information toward the aspect. Besides, the syntactic and semantic alignment and uniformity that contribute to the sentiment delivery is currently neglected during processing. In this work, a Triplet Contrastive Learning Network is developed to coordinate the syntactic information and the semantic information. To start with, the aspect-oriented sub-tree is constructed to replace the syntactic adjacency matrix. Further, a sentence-level contrastive learning scheme is proposed to highlight the features of sentiment words. Based on The Triple Contrastive Learning, the syntactic information and the semantic information are thoroughly interacted and coordinated whilst the global semantics and syntax can be exploited. Extensive experiments are performed on three benchmark datasets and achieve accuracies (BERT-based) of 87.40, 82.80, 77.55 on Rest14, Lap14, and Twitter datasets, which demonstrate that our approach achieves state-of-the-art results in Aspect Level Sentiment Classification task.

Cross-modal fine-grained alignment and fusion network for multimodal aspect-based sentiment analysis

Electrophysiological estimates of semantic and syntactic information access during tacit picture naming and listening to words.

An ERP study of continuous speech processing: I. Segmentation, semantics, and syntax in native speakers