DEPENDENCY TREE PARSING USING TRANSFORMER-BASED LANGUAGE REPRESENTATIONS

Natural language processing is the area of study that focuses on how computers and human languages interact. Machine translation, sentiment analysis, semantic analysis, and text analysis are a few of them. The key natural language processing component is morphological analysis, which breaks words into their corresponding morphemes to determine their structure and meaning. Dependency parsing algorithms use morphological information to determine the syntactic structure of a sentence. This study evaluates the performance of various parsers, including Turbo parser, Lys-FASTPARSE, UU parser, and neural network based parser, to analyse dependency parsing methodologies used in the Malayalam language. The study evaluates the performance of these parsers in handling the difficulties and effectiveness of extensive morphological and syntactic features of Malayalam. Among these parsers, Lys-FASTPARSE performs better in LAS F1 score, MLAS score, and BLEX score, maintaining values of 56.60 and 48.58 before and after optimization. The neural network parser shows minor improvements in unlabelled attachment scores from 0.72 to 0.73 and labelled attachment scores from 0.46 to 0.47. With an LAS of 66.89% and UAS of 87.12%, the Turbo parser shows better results for baseline performance. The precision of 98.81% and recall of 88.42% in binned HEAD directions of the UU parser shows its performance in managing right direction dependencies. While lower, the parser's performance in managing left-direction and root dependencies still reflects its ability to navigate complex syntactic structures effectively. The results underscore the significance of tailored parsing techniques for morphologically rich languages like Malayalam and provide insights into optimizing parser performance for improved syntactic analysis.

Dependency parsing, which is a fundamental task in Natural Language Processing (NLP), has attracted a lot of interest in recent years. In general, it is a module in an NLP pipeline together with word segmentation and Part-Of-Speech (POS) tagging in real Chinese NLP application. The NLP pipeline, which is a cascade system, will lead to error propagation for the parsing. This paper proposes a global discriminative re-ranking model using non-local features from word segmentation, POS tagging and dependency parsing to re-rank the parse trees produced by an N-best enhanced NLP pipeline. Experimental results indicate that the proposed model can improve the performance of dependency parsing as well as word segmentation and POS tagging in an NLP pipeline.

Locality and expectation effects in Hindi preverbal constituent ordering

This paper presents a new conversion method to automatically transform a constituent-based Vietnamese Treebank into dependency trees. On a dependency Treebank created according to our new approach, we examine two state-of-the-art dependency parsers: the MSTParser and the MaltParser. Experiments show that the MSTParser outperforms the MaltParser. To the best of our knowledge, we report the highest performances published to date in the task of dependency parsing for Vietnamese. Particularly, on gold standard POS tags, we get an unlabeled attachment score of 79.08% and a labeled attachment score of 71.66%.

Sentence completeness classification plays a crucial role in various natural language processing (NLP) applications, including grammar checking, text auto-completion, and language assessment. This task becomes particularly challenging in Thai due to the language’s unique characteristics such as flexible word order, implicit subject omission, and the absence of explicit word boundaries. These linguistic properties make traditional rule-based and statistical approaches prone to errors when applied to Thai. To address these challenges, this research applies modern deep learning techniques, specifically leveraging pre-trained transformer models fine-tuned for Thai sentence completeness classification. This study introduces the use of WangchanBERTa, a Thai-specific adaptation of RoBERTa, pre-trained. Two thousand Thai sentences are created. There are one thousand complete sentences and one thousand incomplete sentences. Each sentence was manually labeled to ensure high data quality. Experimental results show that WangchanBERTa achieves an average accuracy of 99.65%, significantly outperforming mBERT, a popular multilingual baseline, which achieved only 95.82%. Notably, with a Tesla T4 GPU, WangchanBERTa required just 1 hour and 15 minutes to train across all folds, compared to mBERT’s 2 hours and 59 minutes. Additionally, WangchanBERTa’s performance was compared with XLM-R, a state-of-the-art multilingual model, which achieved a slightly higher accuracy of 99.90% but at the cost of higher computational requirements. The results emphasize the advantage of language-specific pretraining in capturing the linguistic nuances of Thai. This research highlights the importance of tailored transformer models for low-resource languages. By demonstrating that WangchanBERTa achieves near state-of- the-art performance with lower computational cost, this work provides a strong foundation for future Thai NLP research.

This paper presents a deterministic parsing algorithm for projective dependency grammar. In a bottom-up way the algorithm finds the local optimum dynamically. A constraint procedure is made to use more structure information. The algorithm parses sentences in linear time and labeling is integrated with the parsing. This parser achieves 63.29% labeled attachment score on the average in CoNLL-X Shared Task.

Most evaluations for vector space models are semantically motivated, e.g. by measuring how well they capture word similarity.If one is interested in syntax-related downstream applications such as dependency parsing, a syntactically motivated evaluation seems preferable.As we show, the choice of embeddings has a noticeable impact on parser performance.Since evaluating embeddings directly in a parser is costly, we analyze the correlation between the full parsing task and a simple linear classification task as a potential proxy.

While the predictive performance of modern statistical dependency parsers relies heavily on the availability of expensive expert-annotated treebank data, not all annotations contribute equally to the training of the parsers. In this paper, we attempt to reduce the number of labeled examples needed to train a strong dependency parser using batch active learning (AL). In particular, we investigate whether enforcing diversity in the sampled batches, using determinantal point processes (DPPs), can improve over their diversity-agnostic counterparts. Simulation experiments on an English newswire corpus show that selecting diverse batches with DPPs is superior to strong selection strategies that do not enforce batch diversity, especially during the initial stages of the learning process. Additionally, our diversity-aware strategy is robust under a corpus duplication setting, where diversity-agnostic sampling strategies exhibit significant degradation.

In the past few years, performance of dependency parsing has been improved by large margin on closed-domain benchmark datasets. However, when processing real-life texts, parsing performance degrades dramatically. Besides the domain adaptation technique, which has made slow progress due to its intrinsic difficulty, one straightforward way is to annotate a certain scale of syntactic data given a new source of texts. However, it is well known that annotating data is time and effort consuming, especially for the complex syntactic annotation. Inspired by the progress in crowdsourcing, this paper proposes to annotate noisy multi-annotation syntactic data with non-experts annotators. Each sentence is independently annotated by multiple annotators and the inconsistencies are retained. In this way, we can annotate data very rapidly since we can recruit many ordinary annotators. Then we construct and release three multi-annotation datasets from different sources. Finally, we propose and compare several benchmark approaches to training dependency parsers on such multi-annotation data. We will release our code and data at http://hlt.suda.edu.cn/~zhli/.

We present the approach taken by the TurkuNLP group in the CRAFT Structural Annotation task, a shared task on dependency parsing. Our approach builds primarily on the Turku neural parser, a native dependency parser that ranked among the best in the recent CoNLL tasks on parsing Universal Dependencies. To adapt the parser to the biomedical domain, we considered and evaluated a number of approaches, including the generation of custom word embeddings, combination with other in-domain resources, and the incorporation of information from named entity recognition. We achieved a labeled attachment score of 89.7%, the best result among task participants.

Neural dependency parsing has achieved remarkable performance for Low-Resource Morphologically-Rich languages.It has also been well-studied that Morphologically-Rich languages exhibit relatively free-word-order.This prompts a fundamental investigation: Is there a way to enhance dependency parsing performance, making the model robust to word order variations utilizing the relatively freeword-order nature of Morphologically-Rich languages?In this work, we examine the robustness of graph-based parsing architectures on 7 relatively free-word-order languages.We focus on scrutinizing essential modifications such as data augmentation and the removal of position encoding required to adapt these architectures accordingly.To this end, we propose a contrastive self-supervised learning method to make the model robust to word order variations.Furthermore, our proposed modification demonstrates a substantial average gain of 3.03/2.95points in 7 relatively free-word-order languages, as measured by the UAS/LAS Score metric when compared to the best performing baseline.

The current state-of-the-art dependency parsing approaches employ BiLSTMs to encode input sentences.Motivated by the success of the transformer-based machine translation, this work for the first time applies the self-attention mechanism to dependency parsing as the replacement of the BiLSTM-based encoders, leading to competitive performance on both English and Chinese benchmark data. Based on the detailed error analysis, we then combine the power of both BiLSTM and self-attention via model ensembles, demonstrating their complementary capability of capturing contextual information. Finally, we explore the recently proposed contextualized word representations as extra input features, and further improve the parsing performance.

Spoken monologues feature greater sentence length and structural complexity than do spoken dialogues. To achieve high parsing performance for spoken monologues, it could prove effective to simplify the structure by dividing a sentence into suitable language units. This paper proposes a method for dependency parsing of Japanese monologues based on sentence segmentation. In this method, the dependency parsing is executed in two stages: at the clause level and the sentence level. First, the dependencies within a clause are identified by dividing a sentence into clauses and executing stochastic dependency parsing for each clause. Next, the dependencies over clause boundaries are identified stochastically, and the dependency structure of the entire sentence is thus completed. An experiment using a spoken monologue corpus shows this method to be effective for efficient dependency parsing of Japanese monologue sentences.

This paper proposes a generic method for the comparative evaluation of system outputs. The approach is able to quantify the pairwise differences between two outputs and to unravel in detail what the differences consist of. We apply our approach to three tasks in Computational Linguistics, i.e. POS tagging, dependency parsing, and coreference resolution. We find that system outputs are more distinct than the (often) small differences in evaluation scores seem to suggest.

Free text written in natural languages is lack of definite structures, and semantic analysis of such text is critically important to many Natural Language Processing applications. In this paper, based on dependency parsing, we present a semantic analytic technique to automatically assign clinical ICD-9-CM codes to complex medical patient records written in English. Our semantics analysis method includes dependency parsing of clinical records obtained from testing and training data sets and the calculation of semantic matching scores. Our ultimate goal is to develop an automated disease diagnosis tool through accurate clinical code assignments. We evaluated our technique with a real-world corpus utilized in the 2007 International Natural Language Processing Challenge administered by the Computational Medicine Center at the Cincinnati Children's Hospital, and obtained promising results.