Development of Punjabi WordNet

The advances in deep learning have led to great achievements in many Natural Language Processing (NLP) tasks. With the nature of language, i.e., sequential data, most NLP tasks can be framed into the sequence learning framework, such as text generation. As one of the most important foundations for modern NLP techniques, autoregressive generation models have achieved dominant performance in a great deal of NLP tasks. Therefore, this thesis emphasizes improving the autoregressive generation model for different NLP tasks. While many tasks can naturally fit into the sequence learning framework, some of them, e.g., building discourse parsing tree, require sophisticated designs to fit into neural models. Therefore, this thesis firstly emphasizes a novel unified framework for discourse parsing, which builds a discourse tree in a top-down depth-first manner, and it frames the task as an autoregressive generation task with the goal of each step being the prediction of the node position given a piece of text. The proposed approach is proven effective with extensive empirical experiments. In addition, I extend the above framework by proposing a hierarchical decoder, which leverages the information from parents and siblings of the nodes that are currently processed. The proposed decoder utilizes the nature of the tree structure and further improves the experiment performance on both discourse parsing and dependency parsing tasks. On the other hand, the de facto strategies, i.e., cross entropy loss and teacher forcing, for training the autoregressive generation models have been shown problematic in certain aspects. For example, cross entropy loss, which is one of the widely leveraged training objective functions, often leads to text degeneration in text generation, and teacher forcing suffers from the exposure bias problem, where there exists a mismatch between the training and testing setup. For text degeneration, I introduce a class of diminishing attentions, which enforces the submodularity of the coverage calculated by cross attention in the sequence-to-sequence model. The proposed diminishing attentions achieve notable improvement on several neural text generation tasks, including text summarization, machine translation, and image paragraph generation. Further, I propose a novel training objective, ScaleGrad, to replace cross entropy, which significantly reduces the degeneration problem in different text generation tasks. In fact, ScaleGrad can be extended to problems beyond text degeneration. It provides wide flexibility to inject different inductive biases into the text generation model by directly modifying the gradient information in the output layer. Next, for the exposure bias problem, this thesis introduces a novel type of scheduled sampling based on training accuracy, which requires only minimal hyper-parameter tuning compared to existing scheduled sampling methods. Additionally, a novel imitation loss is proposed to further enforce the model’s generative behavior to match the teacher-forced behavior. Moreover, this thesis demonstrates that reducing exposure bias can improve the robustness of language models against repetition and toxic errors.

The identification of the correct sense of a word is necessary for many tasks in automatic natural language processing like machine translation, information retrieval, speech and text processing. Automatic Word Sense Disambiguation (WSD) is difficult and accuracies with state-of-the art methods are substantially lower than in other areas of text understanding like part-of-speech tagging. One shortcoming of these methods is that they do not utilize substantial sources of background knowledge, such as semantic taxonomies and dictionaries, which are now available in electronic form (the methods largely use shallow syntactic features). Empirical results from the use of Inductive Logic Programming (ILP) have repeatedly shown the ability of ILP systems to use diverse sources of background knowledge. In this paper we investigate the use of ILP for WSD in two different ways: (a) as a stand-alone constructor of models for WSD; and (b) to build interesting features, which can then be used by standard model-builders such as SVM. In our experiments we examine a monolingual WSD task using the 32 English verbs contained in the SENSEVAL-3 benchmark data; and a bilingual WSD task using 7 highly ambiguous verbs in machine translation from English to Portuguese. Background knowledge available is from eight sources that provide a wide range of syntactic and semantic information. For both WSD tasks, experimental results show that ILP-constructed models and models built using ILP-generated features have higher accuracies than those obtained using a state-of-the art feature-based technique equipped with shallow syntactic features. This suggests that the use of ILP with diverse sources of background knowledge can provide one way for making substantial progress in the field of automatic WSD.

Word Sense Disambiguation (WSD) and Word Sense Induction (WSI) are two fundamental tasks in Natural Language Processing (NLP), i.e., those of, respectively, automatically assigning meaning to words in context from a predefined sense inventory and discovering senses from text for a given input word. The two tasks have generally been hard to perform with high accuracy. However, today innovations in approach to WSD and WSI are promising to open up many interesting new horizons in NLP and Information Retrieval applications. This paper is a quick tour on how to start doing research in this exciting field and suggests the hottest topics to focus on.Keywordscomputational lexical semanticsWord Sense DisambiguationWord Sense Inductiontext understanding

Text extraction is an important part of natural language processing (NLP) tasks. Most NLP tasks like text classification, machine translation, text-to-speech, text-based language identification, text summarization, and named-entity recognition involve the use of textual data. Such data is limited for low-resourced languages making it difficult to experiment advanced NLP techniques on these languages. This paper presents a Python-based toolkit for text analysis and text extraction from different types of images, documents, and audio files. The toolkit is built as a library that has functions that can be imported and utilized for text extraction.

WordNet-like lexical databases are used in many natural language processing tasks, such as word sense disambiguation, information extraction and sentiment analysis. The paper discusses the problem of querying such databases. The types of queries specific to WordNet-like databases are analyzed and previous approaches that were undertaken to query wordnets are discussed. A query language which incorporates data types and syntactic constructs based on concepts that form the core of a WordNet-like database (synsets, word senses, semantic relations, etc.) is proposed as a new solution to the problem of querying wordnets.

WordNet-like lexical databases are used in many natural language processing tasks, such as word sense disambiguation, information extraction and sentiment analysis. The paper discusses the problem of querying such databases. The types of queries specific to WordNet-like databases are analysed and previous approaches that were undertaken to query wordnets are discussed. A query language which incorporates data types and syntactic constructs based on concepts that form the core of a WordNet-like database synsets, word senses, semantic relations, etc. is proposed as a new solution to the problem of querying wordnets.

Word Sense Disambiguation (WSD) aims to identify the correct meaning of polysemous words in the particular context. Lexical resources like WordNet which are proved to be of great help for WSD in the knowledge-based methods. However, previous neural networks for WSD always rely on massive labeled data (context), ignoring lexical resources like glosses (sense definitions). In this paper, we integrate the context and glosses of the target word into a unified framework in order to make full use of both labeled data and lexical knowledge. Therefore, we propose GAS: a gloss-augmented WSD neural network which jointly encodes the context and glosses of the target word. GAS models the semantic relationship between the context and the gloss in an improved memory network framework, which breaks the barriers of the previous supervised methods and knowledge-based methods. We further extend the original gloss of word sense via its semantic relations in WordNet to enrich the gloss information. The experimental results show that our model outperforms the state-of-the-art systems on several English all-words WSD datasets.

Frame semantics is a theory of meaning in natural language, which defines the structure of the lexical semantic resources known as framenets. Both framenets and frame semantics have proved useful for a number of natural language processing (NLP) tasks. However, in this connection framenets have often been criticized for their limited coverage. A proposed reasonable-effort solution to this problem is to develop domain-specific (sublanguage) framenets to complement the corresponding general-language framenets for particular NLP tasks, and in the literature we find such initiatives covering domains such as medicine, soccer, and tourism. In this paper, we report on building a framenet to cover the terms and concepts encountered in descriptive linguistic grammars (written in English) i.e. a framenet for the linguistic domain (LingFN) to complement the general-language BFN.KeywordsFrame semanticsFramenetsSemantics

<title>Abstract</title> The Tenyidie language, a.k.a Angami language, is a Tibeto-Burman Language of Nagaland in the north-eastern part of India. Tenyidie is a low-resource language spoken by the Tenyimia Community and is considered a major language in Nagaland. Tenyidie is a tonal, Subject-Object-Verb, and highly agglutinative language. Among the many Natural Language Processing (NLP) tasks, part-of-speech tagging is one of the primary tasks which is used in building many other NLP tasks, such as dependency parsing, named entity recognition, machine translation, etc. The main aim of this paper is to develop the Part-of-speech (POS) Tagset and Tagger for the Tenyidie Language. To the best of our knowledge, there is no reported work on POS Tagset and POS Tagger for the Tenyidie language. The main contributions of this research are: creation of POS tagset for the Tenyidielanguage, creation of POS annotated dataset (11,540 sentences), and the verification of the proposed POS tagset and annotated dataset using the Machine Learning/Deep Learning techniques such as CRF, BLSTM, etc., including the state-of-the-art model such as BERT.

Word Sense Disambiguation is a long-standing task in Natural Language Processing, lying at the core of human language understanding. However, the evaluation of automatic systems has been problematic, mainly due to the lack of a reliable evaluation framework. In this paper we develop a unified evaluation framework and analyze the performance of various Word Sense Disambiguation systems in a fair setup. The results show that supervised systems clearly outperform knowledge-based models. Among the supervised systems, a linear classifier trained on conventional local features still proves to be a hard baseline to beat. Nonetheless, recent approaches exploiting neural networks on unlabeled corpora achieve promising results, surpassing this hard baseline in most test sets.

Measuring the similarity between implicit semantic relations is an important task in information retrieval and natural language processing. For example, consider the situation where you know an entity-pair (e.g. Google, YouTube), between which a particular relation holds (e.g. acquisition), and you are interested in retrieving other entity-pairs for which the same relation holds (e.g. Yahoo, Inktomi). Existing keyword-based search engines cannot be directly applied in this case because in keyword-based search, the goal is to retrieve documents that are relevant to the words used in the query -- not necessarily to the relations implied by a pair of words. Accurate measurement of relational similarity is an important step in numerous natural language processing tasks such as identification of word analogies, and classification of noun-modifier pairs. We propose a method that uses Web search engines to efficiently compute the relational similarity between two pairs of words. Our method consists of three components: representing the various semantic relations that exist between a pair of words using automatically extracted lexical patterns, clustering the extracted lexical patterns to identify the different semantic relations implied by them, and measuring the similarity between different semantic relations using an inter-cluster correlation matrix. We propose a pattern extraction algorithm to extract a large number of lexical patterns that express numerous semantic relations. We then present an efficient clustering algorithm to cluster the extracted lexical patterns. Finally, we measure the relational similarity between word-pairs using inter-cluster correlation. We evaluate the proposed method in a relation classification task. Experimental results on a dataset covering multiple relation types show a statistically significant improvement over the current state-of-the-art relational similarity measures.

Introduction. Donald Walker: a Remembrance. Section 1: The Task of Natural Language Processing. Natural Language Processing: an Historical Review K. Sparck Jones. On Getting a Computer to Listen J. Robinson. Utterance and Objective: Issues in Natural Language Communication B. Grosz. On the Proper Place of Semantics in Machine Translation M. King. Developing a Natural Language Interface to Complex Data G.G. Hendrix, E.D. Sacerdoti. User-Needs Analysis and Design Methodology for an Automated Document Generator K. Kukich, K. McKeown, J. Shaw, J. Robin, J. Lim, N. Morgan, J. Philips. Section 2: Building Computational Lexicons. Machine-Readable Dictionaries and Computational Linguistics Research B. Boguraev. Research Toward the Development of a Lexical Knowledge Base for Natural Language Processing R.A. Amsler. Discovering Relationships Among Word Senses R.J. Byrd. Machine Readable Dictionary as a Source of Grammatical Information E. Hajicova, A. Rosen. The ITT Lexical Database: Dream and Reality S. Pin-Ngern Conlon. Visions of the Digital Library: Views on Using Computational Linguistics and Semantic Nets in Information Retrieval J.L. Klavans. Anatomy of a Verb Entry: from Linguistic Theory to Lexicographic Practice B.T. Atkins, J. Kegl, B. Levin. Issues for Lexicon Building N. Calzolari. Outline of a Model for Lexical Databases N. Ide, J. le Maitre, J. Veronts. Construction-Based MT Lexicons L. Levin, S. Nirenburg. Dependency-Based Grammatical Information in the Lexicon P. Sgall. Semantics in the Brain's Lexicon -- Some Preliminary Remarks on its Epistemology H. Schnelle. Section 3: The Acquisition and Use of LargeCorpora. The Ecology of Language D.E. Walker. Representativeness in Corpus Design D. Biber. The Text Encoding Initiative C.M. Sperberg-McQueen. Discrimination Decisions for 100,000 Dimensional W.A. Gale, K.W. Church, D. Yarowsky. Acquisition and Exploitation of Textual Resources for NLP S. Armstrong-Warwick. The Center for Electronic Texts in the Humanities S. Hockey. Design Principles for Electronic Textual Resources: Investigating Users and Uses of Scholarly Information N.J. Belkin. Section 4: Topics, Methods and Formalisms in Syntax, Semantics and Pragmatics. Evaluating English Sentences in a Logical Model J. Friedman, D.B. Moran, D.S. Warren. Recovering Implicit Information M.S. Palmer, D.A. Dahl, R.J. Schiffman, L. Hirschman, M. Linebarger, J. Downing. Flexible Generation: Taking the User into Account C.L. Paris, V.O. Mittal. Two Principles of Parse Preference J.R. Hobbs, J. Bear. UD, yet Another Unification Device R. Johnson, M. Rosner. Varieties of Heuristics in Sentence Parsing M. Nagao. Some Recent Trends in Natural Language Processing A.K. Joshi. Stone Soup and the French Room Y. Wilks.

Developing a complete and well specified natural language processing (NLP) models for any language with limited electronic resources is always a challenging and demanding task. Morphological Analyzer and Generator (MAG) is an indispensable tool used extensively in natural language processing applications such as Machine Translation (MT), spell checker, Information Retrieval (IR), lexicography etc. A Morphological Analyser describes the root word and its various morphemes ordering structure of the word form in use. While a Morphological Generator processes the given root word and its morpheme structure to derive the required word form. This paper presents the development of a Morphological Analyzer and Generator for an agglutinative Dravidian language called Tulu. Even though such a Morphological Analyzer and Generator tool is mandatory for many Natural Language Processing (NLP) tasks but is not yet available for Tulu language. The proposed rule based Morphological Analyzer and Generator system is implemented using AT & T Open FST (Finite state transducer). From the experiment it is observed that, the performance of the proposed system is encouraging and can be substantially improved by increasing the number of rules.

Setswana is an under-resourced Bantu African language that is morphologically rich with the disjunctive writing system. Developing NLP pipeline tools for such a language could be challenging, due to the need to balance the linguistics semantics robustness of the tool with computational parsimony. A Part-of-Speech (POS) tagger is one such NLP tool for assigning lexical categories like noun, verb, pronoun, and so on, to each word in a text corpus. POS tagging is an important task in Natural Language Processing (NLP) applications such as information extraction, Machine Translation, Word prediction, etc. Developing a POS tagger for a morphologically rich language such as Setswana has computational linguistics challenges that could affect the effectiveness of the entire NLP system. This is due to some contextual semantics features of the language, that demand a fine-grained granularity level for the required POS tagset, with the need to balance tool semantic robustness with computational parsimony. In this paper, a context-driven corpus-based model for text segmentation and POS tagging for the language is presented. The tagger is developed using the Apache OpenNLP tool and returns the accuracy of 96.73%.

The measurement of semantic similarity is a fundamental task in natural language processing. In the settings of a community question answering (cQA) system, it is essentially a classification problem: given a pair of questions, label it similar, relevant, or irrelevant. Traditional methods, either those at word level or at sentence level, typically require many lexical and syntactic resources, which are not available in languages other than English. In addition, there does not exist a finely annotated dataset for our purpose. In this paper, we constructed a dataset containing 4,322 labelled question pairs in Chinese, which is, to the best of our knowledge, the first open Chinese dataset for question similarity classification. We propose a novel framework for measuring the semantic similarity between sentences based on the architecture of a recurrent neural network (RNN) encoderdecoder, which does not require lexical or syntactic resources. We solve the problem of lacking labelled data by first training the RNN using a larger dataset of question pairs that are automatically labelled with heuristic scores, and then fine-tuning it with our smaller, manually labelled dataset. The two-step training scheme improves the accuracy of classification compared to single-step training, and also outperforms other traditional models. The proposed model is capable of both classification and candidate ranking.