From Treebank Conversion to Automatic Dependency Parsing for Vietnamese

Dependency and constituent trees are widely used by many artificial intelligence applications for representing the syntactic structure of human languages. Typically, these structures are separately produced by either dependency or constituent parsers. In this article, we propose a transition-based approach that, by training a single model, can efficiently parse any input sentence with both constituent and dependency trees, supporting both continuous/projective and discontinuous/non-projective syntactic structures. To that end, we develop a Pointer Network architecture with two separate task-specific decoders and a common encoder, and follow a multitask learning strategy to jointly train them. The resulting quadratic system, not only becomes the first parser that can jointly produce both unrestricted constituent and dependency trees from a single model, but also proves that both syntactic formalisms can benefit from each other during training, achieving state-of-the-art accuracies in several widely-used benchmarks such as the continuous English and Chinese Penn Treebanks, as well as the discontinuous German NEGRA and TIGER datasets.

This paper proposes a novel transition-based algorithm for character-level Chinese dependency parsing that straightforwardly models the dependency tree in a top-down manner. Based on the stack-pointer parser, we joint Chinese word segmentation, part-of-speech tagging, and dependency parsing in a new way. We recursively build the character-based dependency tree from root to leaf in a depth-first fashion, by searching for candidate dependents through the sentence and predicting relation type at each step. We introduce intra-word dependencies into the relation types for word segmentation, and the inter-word dependencies with POS tags for part-of-speech tagging. Since the top-down model provides a global view of an input sentences, the information of the whole sentence and all previously generated arcs are available for action decisions, and all characters of the sentence are considered as candidate dependencies. Experimental results on the Penn Chinese Treebank (CTB) show that the proposed model outperformed existing neural joint parsers by 0.81% on dependency parsing, and achieved the F1-scores of 95.97%, 91.72%, 80.25% for Chinese word segmentation, part-of-speech tagging, and dependency parsing.

It is widely accepted that part-of-speech (POS) tagging and dependency parsing are highly related. Most state-of-the-art dependency parsing methods still rely on the results of POS tagging, though the tagger is not perfect yet. Inevitably, dependency parsing model will encounter performance degradation due to the error propagation problems. And it still remains uncertain about how important POS tagging is to dependency parsing. In this work, we propose a method to jointly learn POS tagging and dependency parsing so as to alleviate the error propagation problems. Our proposed method is based on transition system, which is capable to produce dependency tree efficiently and accurately. The results reported in the experiments support our idea that POS tagging is a crucial syntactic component for dependency parsing.

Dependency parsing in Myanmar face many challenges due to agglutinative morphology, flexible word order, and lack of explicit word boundaries. This paper addresses these challenges by investigating the performance of Myanmar dependency parsing based three pretrained-transformer models: XLM-RoBERTa-Large, XLM-RoBERTa-base-Longformer-4096, and language-specific model, MyanBERTa. The proposed methodology utilizes a parameter-efficient model with adapter layers and a biaffine parsing mechanism by optimizing multi-task learning objective across six linguistic prediction tasks. The experimental results demonstrated that XLM-RoBERTa-Large achieves the highest Labeled Attachment Score (LAS) and Unlabeled Attachment Score (UAS) on both development set and test set. Despite XLM-RoBERTa-base-Longformer-4096 model can extended its input capacity and MyanBERTa is pretrained on a language capacity, their performance drop compared to XLM-RoBERTa-Large. These findings suggest that the scale and architectural strength of models are more critical for high-performance dependency parsing than models with an extended input capacity or those trained on a language-specific corpus.

Statistical systems with high accuracy are very useful in real-world applications. If these systems can capture basic linguistic information, then the usefulness of these statistical systems improves a lot. This paper is an attempt at incorporating linguistic constraints in statistical dependency parsing. We consider a simple linguistic constraint that a verb should not have multiple subjects or direct objects as its children in the dependency tree. We first describe the importance of this constraint considering machine translation systems which use dependency parser output, as an example application. We then show how the current state-of-the-art dependency parsers violate this constraint. We describe two methods to handle this constraint. We evaluate our methods on the state-of-the-art dependency Telugu parser. Our results show that we can build a statistical parser which handles linguistic constraints and thus be more useful in real-world applications without compromising accuracy.

meaning representations (AMRs) represent sentence semantics as rooted labeled directed acyclic graphs. Though there is a strong correlation between the AMR graph of a sentence and its corresponding dependency tree, the recent neural network AMR parsers do neglect the exploitation of dependency structure information. In this paper, we explore a novel approach to exploiting dependency structures for AMR parsing. Unlike traditional pipeline models, we treat dependency parsing as an auxiliary task for AMR parsing under the multi-task learning framework by sharing neural network parameters and selectively extracting syntactic representation by the attention mechanism. Particularly, to balance the gradients and focus on the AMR parsing task, we present a new dynamical weighting scheme in the loss function. The experimental results on the LDC2015E86 and LDC2017T10 dataset show that our dependency-auxiliary AMR parser significantly outperforms the baseline and its pipeline counterpart, and demonstrate that the neural AMR parsers can be greatly boosted with the help of effective methods of integrating syntax.

We present a two-stage framework to parse a sentence into its Abstract Meaning Representation (AMR). We first use a dependency parser to generate a dependency tree for the sentence. In the second stage, we design a novel transition-based algorithm that transforms the dependency tree to an AMR graph. There are several advantages with this approach. First, the dependency parser can be trained on a training set much larger than the training set for the tree-to-graph algorithm, resulting in a more accurate AMR parser overall. Our parser yields an improvement of 5% absolute in F-measure over the best previous result. Second, the actions that we design are linguistically intuitive and capture the regularities in the mapping between the dependency structure and the AMR of a sentence. Third, our parser runs in nearly linear time in practice in spite of a worst-case complexity ofO(n 2 ).

Few-Shot Relation Extraction Towards Special Interests

Aspect-based sentiment analysis (ABSA) aims to identify the sentiment polarity associated with specific aspect terms within sentences. Existing studies have primarily focused on constructing graphs from dependency trees of sentences to extract syntactic features. However, given that public datasets are often derived from online reviews, the syntactic structures of these sentences frequently exhibit irregularities. As a result, the performance of syntactic-based Graph Convolution Network (GCN) models is adversely impacted by the noise introduced during dependency parsing. Moreover, the interaction between syntactic and semantic information in these approaches is often insufficient, which significantly impairs the model's ability to accurately detect sentiment.To address these challenges, we propose a novel approach called Syntactic Denoising with Multi-strategy Auxiliary Enhancement (SDMAE) for the ABSA task. Specifically, we prune the original dependency tree by focusing on context words with specific part-of-speech features that are critical for conveying the sentiment of aspect terms, and then construct the graph. We introduce a Multi-channel Adaptive Aggregation Module (MAAM), a feature aggregation system that employs a multi-head attention mechanism to integrate semantic and syntactic GCN output representations. Furthermore, we design a multi-strategy task learning framework that incorporates sentiment lexicons and supervised contrastive learning to enhance the model's performance in aspect sentiment recognition.Comprehensive experiments conducted on four benchmark datasets demonstrate that our approach achieves significant performance improvements compared to several state-of-the-art methods across all evaluated datasets.

Head-driven phrase structure grammar (HPSG) enjoys a uniform formalism representing rich contextual syntactic and even semantic meanings. This paper makes the first attempt to formulate a simplified HPSG by integrating constituent and dependency formal representations into head-driven phrase structure. Then two parsing algorithms are respectively proposed for two converted tree representations, division span and joint span. As HPSG encodes both constituent and dependency structure information, the proposed HPSG parsers may be regarded as a sort of joint decoder for both types of structures and thus are evaluated in terms of extracted or converted constituent and dependency parsing trees. Our parser achieves new state-of-the-art performance for both parsing tasks on Penn Treebank (PTB) and Chinese Penn Treebank, verifying the effectiveness of joint learning constituent and dependency structures. In details, we report 95.84 F1 of constituent parsing and 97.00% UAS of dependency parsing on PTB.

The aim of the EVALITA Parsing Task (EPT) is at defining and extending Italian state-of-the-art parsing by encouraging the application of existing models and approaches, comparing paradigms and annotation formats. Therefore, in all the editions, held respectively in 2007, 2009 and 2011, the Task has been organized around two tracks, namely Dependency Parsing and Constituency Parsing, exploiting the same data sets made available by the organizers in two different formats.

Dependency parsing, which is a fundamental task in Natural Language Processing (NLP), has attracted a lot of interest in recent years. In general, it is a module in an NLP pipeline together with word segmentation and Part-Of-Speech (POS) tagging in real Chinese NLP application. The NLP pipeline, which is a cascade system, will lead to error propagation for the parsing. This paper proposes a global discriminative re-ranking model using non-local features from word segmentation, POS tagging and dependency parsing to re-rank the parse trees produced by an N-best enhanced NLP pipeline. Experimental results indicate that the proposed model can improve the performance of dependency parsing as well as word segmentation and POS tagging in an NLP pipeline.

In this paper we describe the implementation of Russian language pipeline in LIMA multilingual analyzer and the results obtained in GramEval-2020 shared task. LIMA is a modular pipeline that implements rule-based and machine learning analysis components. Russian language pipeline includes deep neural networks based modules for tokenization, sentence segmentation, part of speech tagging, lemmatization and dependency parsing. Part of speech tags, feature tags and dependency trees conform to Universal Dependencies rules.

Cross-lingual transfer is an effective way to build syntactic analysis tools in low-resource languages. However, transfer is difficult when transferring to typologically distant languages, especially when neither annotated target data nor parallel corpora are available. In this paper, we focus on methods for cross-lingual transfer to distant languages and propose to learn a generative model with a structured prior that utilizes labeled source data and unlabeled target data jointly. The parameters of source model and target model are softly shared through a regularized log likelihood objective. An invertible projection is employed to learn a new interlingual latent embedding space that compensates for imperfect cross-lingual word embedding input. We evaluate our method on two syntactic tasks: part-of-speech (POS) tagging and dependency parsing. On the Universal Dependency Treebanks, we use English as the only source corpus and transfer to a wide range of target languages. On the 10 languages in this dataset that are distant from English, our method yields an average of 5.2% absolute improvement on POS tagging and 8.3% absolute improvement on dependency parsing over a direct transfer method using state-of-the-art discriminative models.

Automatic dependency parsing of Thai sentences has been underexplored, as evidenced by the lack of large Thai dependency treebanks with complete dependency structures and the lack of a published evaluation of state-of-the-art models, especially transformer-based parsers. In this work, we addressed these gaps by introducing the Thai Universal Dependency Treebank (TUD), a new Thai treebank consisting of 3,627 trees annotated according to the Universal Dependencies (UD) framework. We then benchmarked 92 dependency parsing models that incorporate pretrained transformers on Thai-PUD and our TUD, achieving state-of-the-art results and shedding light on the optimal model components for Thai dependency parsing. Our error analysis of the models also reveals that polyfunctional words, serial verb construction, and lack of rich morphosyntactic features present main challenges for Thai dependency parsing.