Representation learning with complete semantic description of knowledge graphs

The representation learning of knowledge graph refers to embedding entities and relations in knowledge graph into a low-dimensional dense vector space. Existing knowledge graph embedding models mostly chose Euclidean Space as their vector space and consider each fact triple in knowledge graph independently. However, Euclidean Space is unable to represent knowledge effectively due to its strict constraints and mathematical expression. Besides, entities in knowledge graph are not isolated, while these models ignore the association between entities. To solve the problems above, we proposed a text-enhanced knowledge graph representation learning model in Hyperbolic Space. We utilize rich semantic information of entity description by using Transformer Encoder to enhance the ability of knowledge representation. Besides, we embed entities and relations into Hyperbolic Space, which can better capture hierarchical information of the knowledge graph. Experiments on benchmark dataset show that our method achieves better performance compared with other state-of-art methods.KeywordsKnowledge graphRepresentation learningEmbedding modelHyperbolic space

IoT devices generate substantial amounts of measurements and use many different formats to store this data. In order to make IoT devices interoperable, a number of ontologies have been developed, such as the Smart Applications REFerence (SAREF) ontology, to create knowledge graphs that can represent all kinds of measurements from IoT devices. We call the resulting knowledge graphs: IoT measurement knowledge graphs. In this thesis, we set out to investigate what differentiates IoT measurement knowledge graphs from regular knowledge graphs. In the first half, we describe in detail how to create IoT measurement knowledge graphs from real-world measurement data. In the second half, we investigate what happens when we use the graphs in different scenarios, with different methods and applications. In order to generate an IoT measurement knowledge graph, we create a mapping to transform the measurement data coming from IoT devices. We explore multiple ways to create this mapping and evaluate the resulting IoT measurement knowledge graph with competency questions. Using our mapping, we create OfficeGraph, by transforming IoT measurement data recorded over a year from 444 IoT devices located in an office building. This IoT measurement knowledge graph is validated by answering competency questions created with the support of the building owners, showing that we can answer questions we were not able to answer without it. Besides the entities, literals, and relations in knowledge graphs, the combination of these, the context of entities, provides additional knowledge. Therefore, if we want to use entities in knowledge graphs to train machine learning models, it would be a waste to take only the entities from the graph, because this would leave out information. Representation models, such as RDF2Vec and Graph Convolutional Networks (GCNs), can be used to learn (embedding) representations for entities that take context into account, creating a representation based on which relations, entities, and literals occur near the entity in the graph. We learn over IoT measurement knowledge graphs using multiple representation models and experiment with the effect of making more knowledge available to the representation models through semantic enrichment. This is done by making implicit information, such as (e.g.) consecutive measurements, explicitly available, by adding a relation between two measurements. Results show that the semantic enrichment has a positive effect on the learnability, based on the evaluation score, of the entity representations. Due to the dynamic nature of IoT measurement knowledge graphs, it is to be expected that new measurements are made after the initial IoT measurement knowledge graphs are created. Therefore, we investigate the possibility of estimating representations for new entities that are based on the existing entity representations. We introduce the embedding estimation method, which used the numerical attributes of entities, the measurement values, to find entities similar to the new entity. Then by averaging the embedding representations of those entities, it creates a new, estimated, embedding for the new entity. We perform experiments to test the embedding estimation method, and show that there is a trade-off between the amount of time saved by not re-training the entire pipeline, but also a decrease in accuracy, with the estimated embeddings. This thesis offers a focused research into IoT measurement knowledge graph. We provide insight into how IoT measurement knowledge graph can be created and how they differ from regular knowledge graphs, specifically in being numerical, dynamic, and shallow knowledge graphs. We suggest and validate remedies to the shallowness of IoT measurement knowledge graphs, through the semantic enrichments. Furthermore we show how the numerical aspect can be a strength, through the embedding estimation method, which can help with the dynamic nature of IoT measurement knowledge graph.

To conquer incompleteness of knowledge graphs (KGs), we focus on a relation prediction task that completes KGs by ranking all relationships between entities and selecting the top one. Most existing methods for relation prediction are representation-based models, which learn the structural information of KGs to obtain the embeddings of entities and relationships but fail to effectively use relation paths to predict potential relationships between two entities. The traditional path-based logical models consider the path characteristics but ignore the global semantics of entities in KGs, such as surrounding entities and relationships. In this paper, we propose path embedded hybrid reasoning model (PE-HRM), a novel reasoning mode based on logical reasoning, representation learning and neural network. PE-HRM considers structural characteristics of entities and path features between them and effectively fuses these features. Finally, we evaluate PE-HRM on datasets FB15K237 and NELL995. Experimental results show that PE-HRM significantly outperforms baseline models.

Knowledge graph conflict resolution is a method to solve the knowledge conflict problem in constructing knowledge graphs. The existing methods ignore the time attributes of facts and the dynamic changes of the relationships between entities in knowledge graphs, which is liable to cause high error rates in dynamic knowledge graph construction. In this article, we propose a knowledge graph conflict resolution method, knowledge graph evolution algorithm based on deep learning (Kgedl), which can resolve facts confliction with high precision by combing time attributes, semantic embedding representations, and graph structure features. Kgedl first trains the semantic embedding vector through the relationships between entities. Then, the path embedding vector is trained from the graph structures of knowledge graphs, and the time attributes of entities are combined with the semantic and path embedding vectors. Finally, Kgedl uses a recurrent neural network to make the inconsistent facts appear in the dynamic evolution of the knowledge graph consistent. A large number of experiments on real datasets show that Kgedl outperforms the state-of-the-art methods. Especially, Kgedl achieves 23% higher performance than the classical method numerical Probabilistic Soft Logic (nPSL).in the metric HITS@10. Also, extensive experiments verified that our proposal possess better robustness by adding noise data.

The task of discovering equivalent entities in knowledge graphs (KGs), so‐called KG entity alignment, has drawn much attention to overcome the incompleteness problem of KGs. The majority of existing techniques learns the pointwise representations of entities in the Euclidean space with translation assumption and graph neural network approaches. However, real vectors inherently neglect the complex relation structures and lack the expressiveness of embeddings; hence, they may guide the embeddings to be falsely generated which results in alignment performance degradation. To overcome these problems, we propose a novel KG alignment framework, ComplexGCN, which learns the embeddings of both entities and relations in complex spaces while capturing both semantic and neighborhood information simultaneously. The proposed model ensures richer expressiveness and more accurate embeddings by successfully capturing various relation structures in complex spaces with high‐level computation. The model further incorporates relation label and direction information with a low degree of freedom. To compare our proposal against the state‐of‐the‐art baseline techniques, we conducted extensive experiments on real‐world datasets. The empirical results show the efficiency and effectiveness of the proposed method.

Modelling Multi-relations for Convolutional-based Knowledge Graph Embedding

The accurate identification of disease-associated genes is crucial for understanding the molecular mechanisms underlying various diseases. Most current methods focus on constructing biological networks and utilizing machine learning, particularly deep learning, to identify disease genes. However, these methods overlook complex relations among entities in biological knowledge graphs. Such information has been successfully applied in other areas of life science research, demonstrating their effectiveness. Knowledge graph embedding methods can learn the semantic information of different relations within the knowledge graphs. Nonetheless, the performance of existing representation learning techniques, when applied to domain-specific biological data, remains suboptimal. To solve these problems, we construct a biological knowledge graph centered on diseases and genes, and develop an end-to-end knowledge graph completion framework for disease gene prediction using interactional tensor decomposition named KDGene. KDGene incorporates an interaction module that bridges entity and relation embeddings within tensor decomposition, aiming to improve the representation of semantically similar concepts in specific domains and enhance the ability to accurately predict disease genes. Experimental results show that KDGene significantly outperforms state-of-the-art algorithms, whether existing disease gene prediction methods or knowledge graph embedding methods for general domains. Moreover, the comprehensive biological analysis of the predicted results further validates KDGene's capability to accurately identify new candidate genes. This work proposes a scalable knowledge graph completion framework to identify disease candidate genes, from which the results are promising to provide valuable references for further wet experiments. Data and source codes are available at https://github.com/2020MEAI/KDGene.

Knowledge graph embedding methods learn continuous vector representations for entities in knowledge graphs and have been used successfully in a large number of applications. We present a novel and scalable paradigm for the computation of knowledge graph embeddings, which we dub PYKE . Our approach combines a physical model based on Hooke's law and its inverse with ideas from simulated annealing to compute embeddings for knowledge graphs efficiently. We prove that PYKE achieves a linear space complexity. While the time complexity for the initialization of our approach is quadratic, the time complexity of each of its iterations is linear in the size of the input knowledge graph. Hence, PYKE's overall runtime is close to linear. Consequently, our approach easily scales up to knowledge graphs containing millions of triples. We evaluate our approach against six state-of-the-art embedding approaches on the DrugBank and DBpedia datasets in two series of experiments. The first series shows that the cluster purity achieved by PYKE is up to 26% (absolute) better than that of the state of art. In addition, PYKE is more than 22 times faster than existing embedding solutions in the best case. The results of our second series of experiments show that PYKE is up to 23% (absolute) better than the state of art on the task of type prediction while maintaining its superior scalability. Our implementation and results are open-source and are available at http://github.com/dice-group/PYKE.

Knowledge representation learning can realize the representation of entity and relation semantic information by projecting the triple entity or relation in knowledge graph into vector space, and can effectively calculate entity, relation and their complex semantic association. However, most of the existing models only use the structure information of triples, ignoring the semantic rich entity description. In addition, knowledge representation learning methods usually use negative sampling method. Through random sampling, the negative samples generated by this method are easy to be recognized by the model, and with the iteration of training, the contribution to the improvement of the model performance becomes smaller and smaller. Therefore, we propose a Knowledge Representation learning method based on Generative adversarial network and Pre-training language model (KRGP). Specifically, it uses the idea of generative adversarial network, semantic matching model and translation distance model to generate high-quality negative samples. We combine descriptions of head and tail entities and relations corresponding to negative sample triples into text sequences, and then use next sentence prediciton task of BERT to transform text sequences into a binary classification problem. Experimental results show that this method can improve the performance of link prediction and triple classification on multiple data sets.KeywordsKnowledge representation learningGenerative adversarial networkEntity descriptionPre-training model

The representing learning makes specialty of knowledge graph and it indicates the difference between different entities. The knowledge graph representing with the low-dimensional space. In fact, most of the method usually concern of description of entity which is hard for existing strategies to take benefit of. Here, we recommend a new representing learning method with knowledge graphs that uses entity description. We evaluate our method on two tasks like knowledge graph and entity classification. Experimental effects on actual-world datasets show that our version plays higher than different baseline fashions, especially under the zero-short setting, which indicate that our technique for novel the entity description.

LHD 2.0: A text mining approach to typing entities in knowledge graphs

Web‐scale knowledge graphs such as the global Linked Data cloud consist of billions of individual statements about millions of entities. In recent years, this has fueled the interest in knowledge graph summarization techniques that compute representative subgraphs for a given collection of nodes. In addition, many of the most densely connected entities in knowledge graphs are places and regions, often characterized by thousands of incoming and outgoing relationships to other places, actors, events, and objects. In this article, we propose a novel summarization method that incorporates spatially explicit components into a reinforcement learning framework in order to help summarize geographic knowledge graphs, a topic that has not been considered in previous work. Our model considers the intrinsic graph structure as well as the extrinsic information to gain a more comprehensive and holistic view of the summarization task. By collecting a standard data set and evaluating our proposed models, we demonstrate that the spatially explicit model yields better results than non‐spatial models, thereby demonstrating that spatial is indeed special as far as summarization is concerned.

The task of finding associative entities in knowledge graph (KG) is to provide a ranking list of entities according to their association degrees. However, many entities are not only linked in KG but also associated in terms of user behaviors, which facilitates finding associative entities accurately. This manuscript incorporates KG with user-generated data to propose the Association Entity Graph Model (AEGM) to evaluate the association degrees. They first propose the joint weighting function to evaluate the entity associations and prove its submodularity theoretically as well as the greedy algorithm to select the candidates efficiently. They define the entity association information to score the entity association and give the hill climbing search based algorithm for AEGM construction. Following, they embed AEGM to calculate the association degrees and obtain the associative entities efficiently. Extensive experiments on three datasets show that the proposed method can achieve a better performance than some state-of-the-art competitors in accurately finding associative entities.

Numerous large knowledge graphs, such as DBpedia, Wikidata, Yago and Freebase, have been developed in the last decade, which contain millions of facts about various entities in the world. These knowledge graphs have proven to be incredibly useful for intelligent Web search, question understanding, in-context advertising, social media mining, and biomedicine. As some researchers have pointed out, a knowledge graph is not just a graph database, but it should have a layer of conceptual knowledge, which is usually represented as a set of first-order rules. However, it is challenging to automatically extract first-order rules from large knowledge graphs. In particular, traditional models are usually unable to handle rule learning in large knowledge graphs. This chapter aims to present state-of-the-art techniques and models on learning first-order rules using representation learning. By first recalling some basics of rule learning in knowledge graphs, we will introduce useful techniques of embedding-based rule learning through major models such as RLvLR and TyRuLe, which embed paths in knowledge graphs into latent spaces for candidate rule search. Then we evaluate the rule learning efficiency and the quality of automatically learned rules by applying them in link prediction. Before the chapter is concluded, we will also discuss some future research problems in the area.

A comprehensive survey of entity alignment for knowledge graphs