PEJL: A path-enhanced joint learning approach for knowledge graph completion

Knowledge graph completion is an important research issue in knowledge graph construction, knowledge engineering, and natural language processing. A knowledge graph is a knowledge support for realizing accurate knowledge services in general and professional fields. It is also an important breakthrough foundation in information retrieval, question-and-answer interactions, and information recommendation. The low quality and small scale of the knowledge graph are the main bottlenecks that hinder its wide applications. The purpose of knowledge graph completion is to build a large-scale and high-quality knowledge graph for continuously updating and expanding the knowledge graph. Aiming at the difficulty of knowledge graph completion methods to extract deep semantic features from auxiliary information, such as unstructured texts, this study proposes a knowledge graph completion method based on parsing graph embedding and weighted graph convolutional network. This method uses the weighted graph convolutional network to model the semantic dependency parsing of the entity description and construct the semantic dependency parsing graph embedding. Furthermore, it introduces a multi-grained sentence-embedding generation method of the entity description, which is intended to build entity representation learning that can capture multi-grained semantics and deep-level semantic features. The experimental results on two public datasets show that the proposed knowledge graph completion approach outperforms the existing methods, thereby demonstrating its effectiveness and superiority.

Knowledge graphs have shown increasing value in semantic search, intelligent Q&A, data analysis, natural language processing, visual understanding, IoT devices, etc. It is undeniable that knowledge graphs have become the mandatory path for the artificial intelligence fields development. However, the information contained in existing knowledge graphs is incomplete, which attracts a large number of researchers to enhance the completeness of knowledge graphs utilizing knowledge graph completion methods. Most traditional embedding-based knowledge graph completion models use structural information within data-rich triples from the knowledge graph is limited by the long-tail distribution of the relations in the triples. To address this problem, we propose a Bert-based knowledge graph completion algorithm for few-shot knowledge graphs, with the main goal of implementing the knowledge graph completion task with only a few sample triples of training instances. We improve the baseline model GMatching for handling few-shot knowledge graphs by introducing the Bert pre-trained linguistic representation model to enhance the semantic representation of entities and relations in the triples. Through experiments, we demonstrate that our improved model B-GMatching achieves good results.

As research on utilizing human knowledge in natural language processing has attracted considerable attention in recent years, knowledge graph (KG) completion has come into the spotlight. Recently, a new knowledge graph completion method using a pre-trained language model, such as KG-BERT, is presented and showed high performance. However, its scores in ranking metrics such as Hits@k are still behind state-of-the-art models. We claim that there are two main reasons: 1) failure in sufficiently learning relational information in knowledge graphs, and 2) difficulty in picking out the correct answer from lexically similar candidates. In this paper, we propose an effective multi-task learning method to overcome the limitations of previous works. By combining relation prediction and relevance ranking tasks with our target link prediction, the proposed model can learn more relational properties in KGs and properly perform even when lexical similarity occurs. Experimental results show that we not only largely improve the ranking performances compared to KG-BERT but also achieve the state-of-the-art performances in Mean Rank and Hits@10 on the WN18RR dataset.

Knowledge Graph Completion (KGC) aims at complementing missing relationships between entities in a Knowledge Graph (KG). While closed-world KGC approaches utilizing the knowledge within KG could only complement very limited number of missing relations, more and more approaches tend to get knowledge from open-world resources such as online encyclopedias and newswire corpus. For instance, a recent proposed open-world KGC model called ConMask learns embeddings of the entity’s name and parts of its text-description to connect unseen entities to the KG. However, this model does not make full use of the rich feature information in the text descriptions, besides, the proposed relationship-dependent content masking method may easily miss to find the target-words. In this paper, we propose to use a Multiple Interaction Attention (MIA) mechanism to model the interactions between the head entity description, head entity name, the relationship name, and the candidate tail entity descriptions, to form the enriched representations. Our empirical study conducted on two real-world data collections shows that our approach achieves significant improvements comparing to state-of-the-art KGC methods.

Scholarly knowledge graph is a knowledge graph that is used to represent knowledge contained in scientific publication documents. The information we can find in a scientific publication document is as follows: author, institution, name of journal/conference, and research topic. A knowledge graph that has been built is usually still not perfect. Some incomplete information may be found. To add the missing information, we can use knowledge graph completion, which is a method for finding missing or incorrect relationships to improve the quality of a knowledge graph. Knowledge graph completion can be carried out on a scholarly knowledge graph by adding new entities and relationships to produce further information in the scholarly knowledge graph. The data added to the scholarly knowledge graph are only other papers of first author entity, the research field of first author entity, and a description of the conference/journal entity. The result shows that the scholarly knowledge graph was completed by adding 81% correct data for other papers of first author entity, 80.3% correct data for the research field of first author entity, and 53.9% correct data for the description of the conference/journal.

AttenE: a knowledge graph completion model based on entity-relation attention embedding

Knowledge graphs (KGs) are important resources for a variety of natural language processing tasks but suffer from incompleteness. To address this challenge, a number of knowledge graph completion (KGC) methods have been developed using low-dimensional graph embeddings. Most existing methods focus on the structured information of triples in encyclopaedia KG and maximize the likelihood of them. However, they neglect semantic information contained in lexical KG. To overcome this drawback, we propose a novel KGC method (named as TransC), that integrates the structured information in encyclopaedia KG and the entity concepts in lexical KG, which describe the categories of entities. Since all entities appearing in the head (or tail) position with the same relation have some common concepts, we introduce a novel semantic similarity to measure the distinction of entity semantics with the concept information. And then TransC utilizes concept-based semantic similarity of the related entities and relations to capture prior distributions of entities and relations. With the concept-based prior distributions, TransC generates multiple embedding representations of each entity in different contexts and estimates the posterior probability of entity and relation prediction. Experimental results demonstrate the efficiency of the proposed method on two benchmark datasets.

Sparsity is a challenge for knowledge graph completion. The negative sample space, which is much larger than the positive sample space, has many negative samples similar to the positive samples or potential positive samples, which affects the performance of the knowledge graph completion model. We propose a novel knowledge graph completion method, hard negative examples mining bilinear model (HNEMB). This model uses two components to model entities and relations, namely models relations into relation matrices and inverse relation matrices to learn latent inverse semantics of relations, and models entities into head vectors and tail vectors to learn different representations of entity in different positions. Then we calculate the bilinear scores of triples, we choose correct candidates based on the scores. To address the problem of sparsity of the knowledge graph, we first propose a method for mining hard negative examples for knowledge graph completion, to make model effectively capture the associations between head entities, relations and tail entities. Finally, experiments on multiple knowledge graph completion datasets shows that, as compare with baselines, our model achieves significant improvements and shows its advantages to model sparse knowledge graph.

Knowledge Graph Completion (KGC) is an essential task aimed at mitigating the issue of incompleteness in knowledge graphs, thereby enhancing their utility for various downstream applications. Existing KGC models predominantly fall into two categories: structure-based and semantic-based approaches. Structure-based methods often encounter challenges with long-tail entities due to the scarcity of structural information and imbalanced entity distributions. Conversely, semantic-based methods, while addressing those limitations, necessitate extensive training of language models and specific finetuning for each knowledge graph, thus constraining their practical efficiency. To alleviate those limitations in both approaches, in this paper, we propose KICGPTv2, an innovative framework that synergizes a large language model (LLM) with traditional KGC methods. This integration effectively mitigates the long-tail entity problem without incurring significant additional training overhead. Central to the KICGPTv2 model is a novel in-context learning strategy, termed Knowledge Prompt, which encodes structural knowledge into demonstrations to effectively guide the LLM. Comprehensive evaluations on various KGC tasks, including link prediction, relation prediction, and triple classification, underscore the efficacy of the KICGPTv2 model, highlighting its ability to achieve competitive performance with reduced training demands and without the need for finetuning

Industrial knowledge graphs (IKGs) have received widespread attention from researchers in recent years; they are intuitive to humans and can be understood and processed by machines. However, how to update the entity triples in the graph based on the continuous production data to cover as much knowledge as possible, while applying a KG to meet the needs of different industrial tasks, are two difficulties. This paper proposes a two-stage model construction strategy to benefit both knowledge graph completion and industrial tasks. Firstly, this paper summarizes the specific forms of multi-source data in industry and provides processing methods for each type of data. The core is to vectorize the data and align it conceptually, thereby achieving the fusion modeling of multi-source data. Secondly, this paper defines two interrelated subtasks to construct a pretrained language–knowledge graph model based on multi-task learning. At the same time, considering the dynamic characteristics of the production process, a dynamic expert network structure is adopted for different tasks combined with the pretrained model. In the knowledge completion task, the proposed model achieved an accuracy of 91.25%, while in the self-healing control task of a blast furnace, the proposed model reduced the incorrect actions rate to 0 and completed self-healing control for low stockline fault in 278 min. The proposed framework has achieved satisfactory results in experiments, which verifies the effectiveness of introducing knowledge into industry.

MRGAT: Multi-Relational Graph Attention Network for knowledge graph completion

We propose a general joint representation learning framework for knowledge acquisition (KA) on two tasks, knowledge graph completion (KGC) and relation extraction (RE) from text. In this framework, we learn representations of knowledge graphs (KGs) and text within a unified parameter sharing semantic space. To achieve better fusion, we propose an effective mutual attention between KGs and text. The reciprocal attention mechanism enables us to highlight important features and perform better KGC and RE. Different from conventional joint models, no complicated linguistic analysis or strict alignments between KGs and text are required to train our models. Experiments on relation extraction and entity link prediction show that models trained under our joint framework are significantly improved in comparison with other baselines. Most existing methods for KGC and RE can be easily integrated into our framework due to its flexible architectures. The source code of this paper can be obtained from https://github.com/thunlp/JointNRE.

Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors. These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities. For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding. Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning.

The accurate identification of disease-associated genes is crucial for understanding the molecular mechanisms underlying various diseases. Most current methods focus on constructing biological networks and utilizing machine learning, particularly deep learning, to identify disease genes. However, these methods overlook complex relations among entities in biological knowledge graphs. Such information has been successfully applied in other areas of life science research, demonstrating their effectiveness. Knowledge graph embedding methods can learn the semantic information of different relations within the knowledge graphs. Nonetheless, the performance of existing representation learning techniques, when applied to domain-specific biological data, remains suboptimal. To solve these problems, we construct a biological knowledge graph centered on diseases and genes, and develop an end-to-end knowledge graph completion framework for disease gene prediction using interactional tensor decomposition named KDGene. KDGene incorporates an interaction module that bridges entity and relation embeddings within tensor decomposition, aiming to improve the representation of semantically similar concepts in specific domains and enhance the ability to accurately predict disease genes. Experimental results show that KDGene significantly outperforms state-of-the-art algorithms, whether existing disease gene prediction methods or knowledge graph embedding methods for general domains. Moreover, the comprehensive biological analysis of the predicted results further validates KDGene's capability to accurately identify new candidate genes. This work proposes a scalable knowledge graph completion framework to identify disease candidate genes, from which the results are promising to provide valuable references for further wet experiments. Data and source codes are available at https://github.com/2020MEAI/KDGene.

Knowledge Graph Embeddings (KGE) aim to map entities and relations to low dimensional spaces and have become the de-facto standard for knowledge graph completion. Most existing KGE methods suffer from the sparsity challenge, where it is harder to predict entities that appear less frequently in knowledge graphs. In this work, we propose a novel framework KRACL1 to alleviate the widespread sparsity in KGs with graph context and contrastive learning. Firstly, we propose the Knowledge Relational Attention Network (KRAT) to leverage the graph context by simultaneously projecting neighboring triples to different latent spaces and jointly aggregating messages with the attention mechanism. KRAT is capable of capturing the subtle semantic information and importance of different context triples as well as leveraging multi-hop information in knowledge graphs. Secondly, we propose the knowledge contrastive loss by combining the contrastive loss with cross entropy loss, which introduces more negative samples and thus enriches the feedback to sparse entities. Our experiments demonstrate that KRACL achieves superior results across various standard knowledge graph benchmarks, especially on WN18RR and NELL-995 which have large numbers of low in-degree entities. Extensive experiments also bear out KRACL’s effectiveness in handling sparse knowledge graphs and robustness against noisy triples.