Knowledge Graph Completion for High-Speed Railway Turnout Switch Machine Maintenance Based on the Multi-Level KBGC Model

Knowledge graph completion is an important research issue in knowledge graph construction, knowledge engineering, and natural language processing. A knowledge graph is a knowledge support for realizing accurate knowledge services in general and professional fields. It is also an important breakthrough foundation in information retrieval, question-and-answer interactions, and information recommendation. The low quality and small scale of the knowledge graph are the main bottlenecks that hinder its wide applications. The purpose of knowledge graph completion is to build a large-scale and high-quality knowledge graph for continuously updating and expanding the knowledge graph. Aiming at the difficulty of knowledge graph completion methods to extract deep semantic features from auxiliary information, such as unstructured texts, this study proposes a knowledge graph completion method based on parsing graph embedding and weighted graph convolutional network. This method uses the weighted graph convolutional network to model the semantic dependency parsing of the entity description and construct the semantic dependency parsing graph embedding. Furthermore, it introduces a multi-grained sentence-embedding generation method of the entity description, which is intended to build entity representation learning that can capture multi-grained semantics and deep-level semantic features. The experimental results on two public datasets show that the proposed knowledge graph completion approach outperforms the existing methods, thereby demonstrating its effectiveness and superiority.

AttenE: a knowledge graph completion model based on entity-relation attention embedding

Graph neural networks have been proven to be very effective for representation learning of knowledge graphs. Recent methods such as SACN and CompGCN, have achieved the most advanced results in knowledge graph completion. However, previous efforts mostly rely on localized first-order approximations of spectral graph convolutions or first-order neighborhoods, ignoring the abundant local structures like cycles and stars. Therefore, the diverse semantic information beneath these structures is not well-captured, leaving opportunities for better knowledge representation which will finally help KGC. In this work, we propose LSA-GAT, a graph attention network with a novel neighborhood aggregation strategy for knowledge graph completion. The model can take special local structures into account, and derive a sophisticated representation covering both the semantic and structural information. Moreover, the LSA-GAT model is combined with a CNN-based decoder to form an encoder-decoder framework with a carefully designed training process. The experimental results show significant improvement of the proposed LSA-GAT compared to current state-of-the-art methods on FB15k-237 and WN18RR datasets.

Knowledge Graph Completion (KGC) aims at complementing missing relationships between entities in a Knowledge Graph (KG). While closed-world KGC approaches utilizing the knowledge within KG could only complement very limited number of missing relations, more and more approaches tend to get knowledge from open-world resources such as online encyclopedias and newswire corpus. For instance, a recent proposed open-world KGC model called ConMask learns embeddings of the entity’s name and parts of its text-description to connect unseen entities to the KG. However, this model does not make full use of the rich feature information in the text descriptions, besides, the proposed relationship-dependent content masking method may easily miss to find the target-words. In this paper, we propose to use a Multiple Interaction Attention (MIA) mechanism to model the interactions between the head entity description, head entity name, the relationship name, and the candidate tail entity descriptions, to form the enriched representations. Our empirical study conducted on two real-world data collections shows that our approach achieves significant improvements comparing to state-of-the-art KGC methods.

Knowledge graphs have shown increasing value in semantic search, intelligent Q&A, data analysis, natural language processing, visual understanding, IoT devices, etc. It is undeniable that knowledge graphs have become the mandatory path for the artificial intelligence fields development. However, the information contained in existing knowledge graphs is incomplete, which attracts a large number of researchers to enhance the completeness of knowledge graphs utilizing knowledge graph completion methods. Most traditional embedding-based knowledge graph completion models use structural information within data-rich triples from the knowledge graph is limited by the long-tail distribution of the relations in the triples. To address this problem, we propose a Bert-based knowledge graph completion algorithm for few-shot knowledge graphs, with the main goal of implementing the knowledge graph completion task with only a few sample triples of training instances. We improve the baseline model GMatching for handling few-shot knowledge graphs by introducing the Bert pre-trained linguistic representation model to enhance the semantic representation of entities and relations in the triples. Through experiments, we demonstrate that our improved model B-GMatching achieves good results.

As research on utilizing human knowledge in natural language processing has attracted considerable attention in recent years, knowledge graph (KG) completion has come into the spotlight. Recently, a new knowledge graph completion method using a pre-trained language model, such as KG-BERT, is presented and showed high performance. However, its scores in ranking metrics such as Hits@k are still behind state-of-the-art models. We claim that there are two main reasons: 1) failure in sufficiently learning relational information in knowledge graphs, and 2) difficulty in picking out the correct answer from lexically similar candidates. In this paper, we propose an effective multi-task learning method to overcome the limitations of previous works. By combining relation prediction and relevance ranking tasks with our target link prediction, the proposed model can learn more relational properties in KGs and properly perform even when lexical similarity occurs. Experimental results show that we not only largely improve the ranking performances compared to KG-BERT but also achieve the state-of-the-art performances in Mean Rank and Hits@10 on the WN18RR dataset.

<abstract><p>Knowledge graphs (KGs) often suffer from incompleteness. Knowledge graph completion (KGC) is proposed to complete missing components in a KG. Most KGC methods focus on direct relations and fail to leverage rich semantic information in multi-hop paths. In contrast, path-based embedding methods can capture path information and utilize extra semantics to improve KGC. However, most path-based methods cannot take advantage of full multi-hop information and neglect to capture multiple semantic associations between single and multi-hop triples. To bridge the gap, we propose a novel path-enhanced joint learning approach called PEJL for KGC. Rather than learning multi-hop representations, PEJL can recover multi-hop embeddings by encoding full multi-hop components. Meanwhile, PEJL extends the definition of translation energy functions and generates new semantic representations for each multi-hop component, which is rarely considered in path-based methods. Specifically, we first use the path constraint resource allocation (PCRA) algorithm to extract multi-hop triples. Then we use an embedding recovering module consisting of a bidirectional gated recurrent unit (GRU) layer and a fully connected layer to obtain multi-hop embeddings. Next, we employ a KG modeling module to leverage various semantic information and model the whole knowledge graph based on translation methods. Finally, we define a joint learning approach to train our proposed PEJL. We evaluate our model on two KGC datasets: FB15K-237 and NELL-995. Experiments show the effectiveness and superiority of PEJL.</p></abstract>

Knowledge Graph Completion (KGC) is a popular topic in knowledge graph construction and related applications, aiming to complete the structure of knowledge graphs by predicting missing entities or relations and mining unknown facts in the knowledge graph. In the KGC task, Graph Neural Network (GNN)-based methods have achieved remarkable results due to their advantage of effectively capturing complex relations among entities and generating more accurate and rich entity representations by aggregating information from neighbouring nodes. These methods mainly focus on the representation of entities, and the representation of relations is obtained using simple dimensional transformations or initial embeddings. This treatment ignores the diversity and complex semantics of relations and restricts the efficiency of the model in utilizing relational information in the reasoning process. In this work, we propose the Relational Representation Augmented Graph Attention Network (RRA-GAT), which effectively identifies and weights neighbouring relations that actually contribute to the target relation by filtering out irrelevant information through an attention function based on the information and spatial domain. Furthermore, we capture complex patterns and features in the relational embedding by means of a feed-forward network consisting of a series of linear transformations and nonlinear activation functions. Experiments demonstrate the very advanced performance of RRA-GAT on the link prediction task on standard datasets FB15k-237 and WN18RR (e.g., improved the MRR metric on the WN18RR dataset by 7.8% relative improvement).

Knowledge Graph Embeddings (KGE) aim to map entities and relations to low dimensional spaces and have become the de-facto standard for knowledge graph completion. Most existing KGE methods suffer from the sparsity challenge, where it is harder to predict entities that appear less frequently in knowledge graphs. In this work, we propose a novel framework KRACL1 to alleviate the widespread sparsity in KGs with graph context and contrastive learning. Firstly, we propose the Knowledge Relational Attention Network (KRAT) to leverage the graph context by simultaneously projecting neighboring triples to different latent spaces and jointly aggregating messages with the attention mechanism. KRAT is capable of capturing the subtle semantic information and importance of different context triples as well as leveraging multi-hop information in knowledge graphs. Secondly, we propose the knowledge contrastive loss by combining the contrastive loss with cross entropy loss, which introduces more negative samples and thus enriches the feedback to sparse entities. Our experiments demonstrate that KRACL achieves superior results across various standard knowledge graph benchmarks, especially on WN18RR and NELL-995 which have large numbers of low in-degree entities. Extensive experiments also bear out KRACL’s effectiveness in handling sparse knowledge graphs and robustness against noisy triples.

MRGAT: Multi-Relational Graph Attention Network for knowledge graph completion

Scholarly knowledge graph is a knowledge graph that is used to represent knowledge contained in scientific publication documents. The information we can find in a scientific publication document is as follows: author, institution, name of journal/conference, and research topic. A knowledge graph that has been built is usually still not perfect. Some incomplete information may be found. To add the missing information, we can use knowledge graph completion, which is a method for finding missing or incorrect relationships to improve the quality of a knowledge graph. Knowledge graph completion can be carried out on a scholarly knowledge graph by adding new entities and relationships to produce further information in the scholarly knowledge graph. The data added to the scholarly knowledge graph are only other papers of first author entity, the research field of first author entity, and a description of the conference/journal entity. The result shows that the scholarly knowledge graph was completed by adding 81% correct data for other papers of first author entity, 80.3% correct data for the research field of first author entity, and 53.9% correct data for the description of the conference/journal.

Graph attention network with dynamic representation of relations for knowledge graph completion

Sparsity is a challenge for knowledge graph completion. The negative sample space, which is much larger than the positive sample space, has many negative samples similar to the positive samples or potential positive samples, which affects the performance of the knowledge graph completion model. We propose a novel knowledge graph completion method, hard negative examples mining bilinear model (HNEMB). This model uses two components to model entities and relations, namely models relations into relation matrices and inverse relation matrices to learn latent inverse semantics of relations, and models entities into head vectors and tail vectors to learn different representations of entity in different positions. Then we calculate the bilinear scores of triples, we choose correct candidates based on the scores. To address the problem of sparsity of the knowledge graph, we first propose a method for mining hard negative examples for knowledge graph completion, to make model effectively capture the associations between head entities, relations and tail entities. Finally, experiments on multiple knowledge graph completion datasets shows that, as compare with baselines, our model achieves significant improvements and shows its advantages to model sparse knowledge graph.

We propose a general joint representation learning framework for knowledge acquisition (KA) on two tasks, knowledge graph completion (KGC) and relation extraction (RE) from text. In this framework, we learn representations of knowledge graphs (KGs) and text within a unified parameter sharing semantic space. To achieve better fusion, we propose an effective mutual attention between KGs and text. The reciprocal attention mechanism enables us to highlight important features and perform better KGC and RE. Different from conventional joint models, no complicated linguistic analysis or strict alignments between KGs and text are required to train our models. Experiments on relation extraction and entity link prediction show that models trained under our joint framework are significantly improved in comparison with other baselines. Most existing methods for KGC and RE can be easily integrated into our framework due to its flexible architectures. The source code of this paper can be obtained from https://github.com/thunlp/JointNRE.

Knowledge Graph Completion (KGC) is an essential task aimed at mitigating the issue of incompleteness in knowledge graphs, thereby enhancing their utility for various downstream applications. Existing KGC models predominantly fall into two categories: structure-based and semantic-based approaches. Structure-based methods often encounter challenges with long-tail entities due to the scarcity of structural information and imbalanced entity distributions. Conversely, semantic-based methods, while addressing those limitations, necessitate extensive training of language models and specific finetuning for each knowledge graph, thus constraining their practical efficiency. To alleviate those limitations in both approaches, in this paper, we propose KICGPTv2, an innovative framework that synergizes a large language model (LLM) with traditional KGC methods. This integration effectively mitigates the long-tail entity problem without incurring significant additional training overhead. Central to the KICGPTv2 model is a novel in-context learning strategy, termed Knowledge Prompt, which encodes structural knowledge into demonstrations to effectively guide the LLM. Comprehensive evaluations on various KGC tasks, including link prediction, relation prediction, and triple classification, underscore the efficacy of the KICGPTv2 model, highlighting its ability to achieve competitive performance with reduced training demands and without the need for finetuning