Clustering and Bootstrapping Based Framework for News Knowledge Base Completion

With the development of Semantic Web, the automatic construction of large scale knowledge bases (KBs) has been receiving increasing attention in recent years. Although these KBs are very large, they are still often incomplete. Many existing approaches to KB completion focus on performing inference over a single KB and suffer from the feature sparsity problem. Moreover, traditional KB completion methods ignore complementarity which exists in various KBs implicitly. In this paper, we treat KBs completion as a large matrix completion task and integrate different KBs to infer new facts simultaneously. We present two improvements to the quality of inference over KBs. First, in order to reduce the data sparsity, we utilize the type consistency constraints between relations and entities to initialize negative data in the matrix. Secondly, we incorporate the similarity of relations between different KBs into matrix factorization model to take full advantage of the complementarity of various KBs. Experimental results show that our approach performs better than methods that consider only existing facts or only a single knowledge base, achieving significant accuracy improvements in binary relation prediction.

POS tagging is a basic Natural Language Processing (NLP) task that tags the words in an input text according to its grammatical values. Although POS Tagging is a fundamental application for very resourced languages, such as Limbu, is still unknown due to only few tagged datasets and linguistic resources. This research project uses deep learning techniques, transfer learning, and the BiLSTM-CRF model to develop an accurate POS-tagging system for the Limbu language. Using annotated and unannotated language data, we progress in achieving a small yet informative dataset of Limbu text. Skilled multilingual tutoring was modified to enhance success on low-resource language tests. The model as propose attains 90% accuracy, which is very much better than traditional rule-based and machine learning methods for Limbu POS tagging. The results indicate that deep learning methods can address linguistic issues facing low-resource languages even with limited data. In turn, this study provides a cornerstone for follow up NLP-based applications of Limbu and similar low-resource languages, demonstrating how deep learning can fill the gap where data is scarce.

Structured knowledge bases (KBs) are a foundation of many intelligent applications, yet are notoriously incomplete. Language models (LMs) have recently been proposed for unsupervised knowledge base completion (KBC), yet, despite encouraging initial results, questions regarding their suitability remain open. Existing evaluations often fall short because they only evaluate on popular subjects, or sample already existing facts from KBs. In this work, we introduce a novel, more challenging benchmark dataset, and a methodology tailored for a realistic assessment of the KBC potential of LMs. For automated assessment, we curate a dataset called WD-Known, which provides an unbiased random sample of Wikidata, containing over 3.9 million facts. In a second step, we perform a human evaluation on predictions that are not yet in the KB, as only this provides real insights into the added value over existing KBs. Our key finding is that biases in dataset conception of previous benchmarks lead to a systematic overestimate of LM performance for KBC. However, our results also reveal strong areas of LMs. We could, for example, perform a significant completion of Wikidata on the relations nativeLanguage, by a factor of $$\sim $$ 21 (from 260k to 5.8M) at $$82\%$$ precision, and citizenOf by a factor of $$\sim $$ 0.3 (from 4.2M to 5.3M) at 90% precision. Moreover, we find that LMs possess surprisingly strong generalization capabilities: even on relations where most facts were not directly observed in LM training, prediction quality can be high. We open-source the benchmark dataset and code. ( https://github.com/bveseli/LMsForKBC ).

BackgroundRecent developments in the domain of biomedical knowledge bases (KBs) open up new ways to exploit biomedical knowledge that is available in the form of KBs. Significant work has been done in the direction of biomedical KB creation and KB completion, specifically, those having gene-disease associations and other related entities. However, the use of such biomedical KBs in combination with patients’ temporal clinical data still largely remains unexplored, but has the potential to immensely benefit medical diagnostic decision support systems.ResultsWe propose two new algorithms, LOADDx and SCADDx, to combine a patient’s gene expression data with gene-disease association and other related information available in the form of a KB, to assist personalized disease diagnosis. We have tested both of the algorithms on two KBs and on four real-world gene expression datasets of respiratory viral infection caused by Influenza-like viruses of 19 subtypes. We also compare the performance of proposed algorithms with that of five existing state-of-the-art machine learning algorithms (k-NN, Random Forest, XGBoost, Linear SVM, and SVM with RBF Kernel) using two validation approaches: LOOCV and a single internal validation set. Both SCADDx and LOADDx outperform the existing algorithms when evaluated with both validation approaches. SCADDx is able to detect infections with up to 100% accuracy in the cases of Datasets 2 and 3. Overall, SCADDx and LOADDx are able to detect an infection within 72 h of infection with 91.38% and 92.66% average accuracy respectively considering all four datasets, whereas XGBoost, which performed best among the existing machine learning algorithms, can detect the infection with only 86.43% accuracy on an average.ConclusionsWe demonstrate how our novel idea of using the most and least differentially expressed genes in combination with a KB can enable identification of the diseases that a patient is most likely to have at a particular time, from a KB with thousands of diseases. Moreover, the proposed algorithms can provide a short ranked list of the most likely diseases for each patient along with their most affected genes, and other entities linked with them in the KB, which can support health care professionals in their decision-making.

Fake news detection in low-resource languages: A novel hybrid summarization approach

During the past few decades, knowledge bases (KBs) have experienced rapid growth. Nevertheless, most KBs still suffer from serious incompletion. Researchers proposed many tasks such as knowledge base completion and relation prediction to help build the representation of KBs. However, there are some issues unsettled towards enriching the KBs. Knowledge base completion and relation prediction assume that we know two elements of the fact triples and we are going to predict the missing one. This assumption is too restricted in practice and prevents it from discovering new facts directly. To address this issue, we propose a new task, namely, fact discovery from knowledge base. This task only requires that we know the head entity and the goal is to discover facts associated with the head entity. To tackle this new problem, we propose a novel framework that decomposes the discovery problem into several facet discovery components. We also propose a novel auto-encoder based facet component to estimate some facets of the fact. Besides, we propose a feedback learning component to share the information between each facet. We evaluate our framework using a benchmark dataset and the experimental results show that our framework achieves promising results. We also conduct extensive analysis of our framework in discovering different kinds of facts. The source code of this paper can be obtained from https://github.com/thunlp/FFD.

This study aims to increase the performance of word embeddings by proposing a new weighting scheme for co-occurrence counting. The idea behind this new family of weights is to overcome the disadvantage of distant appearing word pairs, which are indeed semantically close, while representing them in the co-occurrence counting. For high-resource languages, this disadvantage might not be effective due to the high frequency of co-occurrence. However, when there are not enough available resources, such pairs suffer from being distant. To favour such pairs, a weighting scheme based on a polynomial fitting procedure is proposed to shift the weights up for distant words while the weights of nearby words are left almost unchanged. The parameter optimization for new weights and the effects of the weighting scheme are analysed for the English, Italian, and Turkish languages. A small portion of English resources and a quarter of Italian resources are utilized for demonstration purposes, as if these languages are low-resource languages. Performance increase is observed in analogy tests when the proposed weighting scheme is applied to relatively small corpora (i.e., mimicking low-resource languages) of both English and Italian. To show the effectiveness of the proposed scheme in small corpora, it is also shown for a large English corpus that the performance of the proposed weighting scheme cannot outperform the original weights. Since Turkish is relatively a low-resource language, it is demonstrated that the proposed weighting scheme can increase the performance of both analogy and similarity tests when all Turkish Wikipedia pages are utilized as a corpus. The positive effect of the proposed scheme has also been demonstrated in a standard sentiment analysis task for the Turkish language.

The rise of social media has led to vast amounts of user-generated content, with emotions ranging from joy to anger. Negative comments often target individuals, communities, or brands, prompting successful efforts to detect harmful speech such as hate speech, cyberbullying, and abuse. Recently, another type of speech referred to as ‘Hope Speech’ has gained attention from the research community. Hope speech consists of positive affirmations or words of reassurance, encouragement, consolation or motivation offered to the affected individual/ community during the lean periods of life. However, there has been relatively less research focused on the detection of hope speech, more particularly in low-resource languages. This paper, therefore, attempts to develop an ensemble model for detecting hope speech in some low-resource languages. Data for four different languages, namely English, Kannada, Malayalam and Tamil are obtained and experimented with different deep learning-based models. An ensemble model is proposed to combine the advantages of the better performing models. Experimental results demonstrate the superior performance of the proposed Ensemble (LSTM, mBERT, XLM-RoBERTa) model compared to individual models based on data from all four languages (weighted average F1-score for English is 0.93; for Kannada is 0.74; for Malayalam is 0.82; and for Tamil is 0.60). Thus, the proposed ensemble model proves to be a suitable approach for hope speech detection in the given low resource languages.

One of the major unsolved problems in knowledge acquisition is reconciling knowledge originating from different sources. This paper proposes a technique for reconciling knowledge in two independent knowledge bases, describes a working program built to implement that technique, and discusses an exploratory study for validating the technique. The technique is based on the use of heuristics for identifying and resolving discrepancies between the knowledge bases. Each heuristic developed provides detection and resolution procedures for a distinct variety of discrepancy in the knowledge bases. Sample discrepancies include using synonyms for the same term, conflicting rules, and extra reasoning steps. Discrepancies are detected and resolved through the use of circumstantial evidence available from the knowledge bases themselves and by asking sharply focussed questions to the experts responsible for the knowledge bases. The technique was tested on two independently developed knowledge bases designed to aid novice statisticians in diagnosing problems in linear regression models. The heuristics located a significant number of the discrepancies between the knowledge bases and assisted the experts in creating a consensus knowledge base for diagnosing multicollinearity problems. We argue that the task of identifying discrepancies between independent bodies of knowledge is an inevitable part of any large knowledge acquisition effort. Hence the heuristics developed in this work are applicable even when knowledge acquisition is not done by reconciling two complete knowledge bases. We also suggest that our approach can be extended to other knowledge representations such as frames and database schemas, and speculate about its potential application to other domains involving the reconciliation of knowledge, such as requirements determination, negotiation, and design.

Equipping machines with knowledge, through the construction of machinereadable knowledge bases, presents a key asset for semantic search, machine translation, question answering, and other formidable challenges in artificial intelligence. However, human knowledge predominantly resides in books and other natural language text forms. This means that knowledge bases must be extracted and synthesized from natural language text. When the source of text is the Web, extraction methods must cope with ambiguity, noise, scale, and updates. The goal of this dissertation is to develop knowledge base population methods that address the afore mentioned characteristics of Web text. The dissertation makes three contributions. The first contribution is a method for mining high-quality facts at scale, through distributed constraint reasoning and a pattern representation model that is robust against noisy patterns. The second contribution is a method for mining a large comprehensive collection of relation types beyond those commonly found in existing knowledge bases. The third contribution is a method for extracting facts from dynamic Web sources such as news articles and social media where one of the key challenges is the constant emergence of new entities. All methods have been evaluated through experiments involving Web-scale text collections.

In today's computerized and information-based society, individuals are constantly presented with vast amounts of text data, ranging from news articles, scientific publications, product reviews, to a wide range of textual information from social media. To extract value from these large, multi-domain pools of text, it is of great importance to gain an understanding of entities and their relationships. In this tutorial, we introduce data-driven methods to recognize typed entities of interest in massive, domain-specific text corpora. These methods can automatically identify token spans as entity mentions in documents and label their fine-grained types (e.g., people, product and food) in a scalable way. Since these methods do not rely on annotated data, predefined typing schema or hand-crafted features, they can be quickly adapted to a new domain, genre and language. We demonstrate on real datasets including various genres (e.g., news articles, discussion forum posts, and tweets), domains (general vs. bio-medical domains) and languages (e.g., English, Chinese, Arabic, and even low-resource languages like Hausa and Yoruba) how these typed entities aid in knowledge discovery and management.

Knowledge base completion (KBC) is the task of predicting and filling missing information based on the current data in that knowledge base. Recently, one of the most feasible approaches introduced by V. Kocijan and T. Lukasiewicz (2021) is to transfer knowledge from one collection of information to another without the need for entity or relation matching. Still, this work has not scaled pre-training to larger models and datasets and investigated the impact of the encoder architecture. In this work, we propose a method that can combine the benefits of Bidirectional Encoder Representations from Transformer (BERT), fastText, Gated Recurrent Unit (GRU), and Fully Connected (FC) layer to improve the KBC task in Kocijan and Lukasiewicz’s model. The experimental results show the effectiveness of our proposed model in several popular datasets like ReVerb20K, ReVerb45K, FB15K237, and WN18RR.

Knowledge base completion is an important research problem in knowledge bases, which play important roles in question answering, information retrieval, and other applications. A number of relational learning algorithms have been proposed to solve this problem. However, despite their success in modeling the entity relations, they are not well founded in a Bayesian manner and thus are hard to model the prior information of the entity and relation factors. Furthermore, they under-represent the interaction between entity and relation factors. In order to avoid these disadvantages, we provide a neural-inspired approach, namely Bayesian Neural Tensor Decomposition approach for knowledge base completion based on the Stochastic Gradient Variational Bayesian framework. We employ a multivariate Bernoulli likelihood function to represent the existence of facts in knowledge graphs. We further employ a Multi-layered Perceptrons to represent more complex interactions between the latent subject, predicate, and object factors. The SGVB framework can enable us to make efficient approximate variational inference for the proposed nonlinear probabilistic tensor decomposition by a novel local reparameterization trick. This way avoids the need of expensive iterative inference schemes such as MCMC and does not make any over-simplified assumptions about the posterior distributions, in contrary to the common variational inference. In order to evaluate the proposed model, we have conducted experiments on real-world knowledge bases, i.e., FreeBase and WordNet. Experimental results have indicated the promising performance of the proposed method.

The challenging task of sentiment analysis on Punjabi and other languages with limited resources and technology is investigated in this work. Though sentiment analysis is a prominent topic in NLP, low-resource languages like Punjabi lack digital resources, annotated datasets, and language-specific tools, so research in them is ignored. This paper addresses such information gaps by means of Punjabi text sentiment analysis. Internet news and social media posts were compiled into an annotated Punjabi dataset with sentiment polarity.

Automatic text summarization is a popular area in Natural Language Processing and Machine Learning. In this work, we adopt a graph-based text summarization approach, using PageRank algorithm, for automatically summarizing Konkani text documents. Konkani is an Indo-Aryan language spoken primarily in the state of Goa, which is on the west coast of India. It is a low-resource language with limited language processing tools. Such tools are readily available in other popular languages of choice for automatic text summarization, like English. The Konkani language dataset used for this purpose is based on Konkani folktales. We examine the impact of various language-independent and language-dependent similarity measures on the construction of the graph. The language-dependent similarity measures use pre-trained fastText word embeddings. A fully connected undirected graph is constructed for each document with the sentences represented as the graph's vertices. The vertices are connected to each other based on how strongly they are related to one another. Thereafter, PageRank algorithm is used for ranking the scores of the vertices. The top-ranking sentences are used to generate the summary. ROUGE toolkit was used for evaluating the quality of these system-generated summaries, and the performance was evaluated against human generated “gold-standard” abstracts and also compared with baselines and benchmark systems. The experimental results show that language-independent similarity measures performed well compared to language-dependent similarity measures despite not using language-specific tools, such as stop-words list, stemming, and word embeddings.