Chapter 14. Embedding-Based First-Order Rule Learning in Large Knowledge Graphs

Knowledge Graphs (KGs) represent heterogeneous domain knowledge on the Web and within organizations. There exist shapes constraint languages to define validating shapes to ensure the quality of the data in KGs. Existing techniques to extract validating shapes often fail to extract complete shapes, are not scalable, and are prone to produce spurious shapes. To address these shortcomings, we propose the Quality Shapes Extraction (QSE) approach to extract validating shapes in very large graphs, for which we devise both an exact and an approximate solution. QSE provides information about the reliability of shape constraints by computing their confidence and support within a KG and in doing so allows to identify shapes that are most informative and less likely to be affected by incomplete or incorrect data. To the best of our knowledge, QSE is the first approach to extract a complete set of validating shapes from WikiData. Moreover, QSE provides a 12x reduction in extraction time compared to existing approaches, while managing to filter out up to 93% of the invalid and spurious shapes, resulting in a reduction of up to 2 orders of magnitude in the number of constraints presented to the user, e.g., from 11,916 to 809 on DBpedia.

Knowledge graphs (KGs) are essential resources for many applications including Web search and question answering. As KGs are often automatically constructed, they may contain incorrect facts. Detecting them is a crucial, yet extremely expensive task. Prominent solutions detect and explain inconsistency in KGs with respect to accompanying ontologies that describe the KG domain of interest. Compared to machine learning methods they are more reliable and human-interpretable but scale poorly on large KGs. In this paper, we present a novel approach to dramatically speed up the process of detecting and explaining inconsistency in large KGs by exploiting KG abstractions that capture prominent data patterns. Though much smaller, KG abstractions preserve inconsistency and their explanations. Our experiments with large KGs (e.g., DBpedia and Yago) demonstrate the feasibility of our approach and show that it significantly outperforms the popular baseline.

Learning structured embeddings of knowledge graphs with generative adversarial framework

Incompleteness of large knowledge graphs (KG) has motivated many researchers to propose methods to automatically find missing edges in KGs. A promising approach for KG completion (link prediction) is embedding a KG into a continuous vector space. There are different methods in the literature that learn a continuous representation of KG (latent features of KG). The benchmark dataset FB15k has been widely employed to evaluate these methods. However, It has been noted that FB15k contains many pairs of edges in which a pair represents the same relationship in reverse directions. Therefore, the inverse of numerous test triples occurs in the training set. To address this problem, FB15k-237, a subset of FB15k, was created by removing those inverse-duplicate relations to form a more challenging, realistic dataset. There is not any study that investigates how the aforementioned bias in this widely used benchmark dataset affects the results of embedding-based knowledge graph completion methods and whether their promising results are largely due to the bias. Motivated by this question, we conducted extensive experiments and report the link prediction results on FB15K and FB15k-237 using several embedding-based methods. We compare the results of different methods to see how their performances change in absence of inverse relations. Our experiment results demonstrate that the performance of embedding models in link prediction task diminishes tremendously when the inverse relationships do not exist anymore.

Machine learning algorithms for link prediction can be valuable tools for hypothesis generation. However, many current algorithms are black boxes or lack good user interfaces that could facilitate insight into why predictions are made. We present LinkExplorer, a software suite for predicting, explaining and exploring links in large biomedical knowledge graphs. LinkExplorer integrates our novel, rule-based link prediction engine SAFRAN, which was recently shown to outcompete other explainable algorithms and established black-box algorithms. Here, we demonstrate highly competitive evaluation results of our algorithm on multiple large biomedical knowledge graphs, and release a web interface that allows for interactive and intuitive exploration of predicted links and their explanations. A publicly hosted instance, source code and further documentation can be found at https://github.com/OpenBioLink/Explorer. Supplementary data are available at Bioinformatics online.

Scheduling satellite tasks requires intricate coordination of numerous interconnected activities, posing challenges for ground station operators who struggle to decipher autonomous decision-making systems without explanatory responses. This paper advocates for explainable artificial intelligence to address this gap, leveraging large language models and knowledge graphs to enhance the transparency of scheduling logic. The study explores how operators can retrieve relevant information through natural language queries by integrating language processing with knowledge graphs as core data structures. This approach allows for manual access to knowledge graphs, supplemented with textual explanations, and enables operators to explore different scheduling scenarios, thus improving system robustness and adaptability. Additionally, this paper analyzes how current knowledge graphs and natural language processing techniques are utilized and how they can enhance explainability in satellite scheduling. It investigates the modeling of satellite schedules and environmental data within the knowledge graph. The study introduces a novel approach for generating categorized queries, executable code for knowledge graph data extraction via language modeling, and providing explanatory answers. These are evaluated for correctness, validity, and linguistic quality, demonstrating the effectiveness of this approach.

A variety of knowledge graph embedding approaches have been developed. Most of them obtain embeddings by learning the structure of the knowledge graph within a link prediction setting. As a result, the embeddings reflect only the structure of a single knowledge graph, and embeddings for different knowledge graphs are not aligned, e.g., they cannot be used to find similar entities across knowledge graphs via nearest neighbor search. However, knowledge graph embedding applications such as entity disambiguation require a more global representation, i.e., a representation that is valid across multiple sources. We propose to learn universal knowledge graph embeddings from large-scale interlinked knowledge sources. To this end, we fuse large knowledge graphs based on the owl:sameAs relation such that every entity is represented by a unique identity. We instantiate our idea by computing universal embeddings based on DBpedia and Wikidata yielding embeddings for about 180 million entities, 15 thousand relations, and 1.2 billion triples. We believe our computed embeddings will support the emerging field of graph foundation models. Moreover, we develop a convenient API to provide embeddings as a service. Experiments on link prediction suggest that universal knowledge graph embeddings encode better semantics compared to embeddings computed on a single knowledge graph. For reproducibility purposes, we provide our source code and datasets open access.

Smart city systems are increasingly built with data analytics and machine learning techniques, basing on massive data sets. They have a heavy impact on human behavior and quality of life, and thus need to deliver a controllable and sufficiently transparent experience for the users.The aim of my work is to make smart city systems more interoperable and explainable, involving data visualization and communication techniques, sensor data processing, as well as the associated intelligent data value chain production and consumption. Here, the data and the information are shared employing Knowledge Graphs, that are becoming a key enabler for large-scale processing of massive collections of interrelated facts. Examples include the Google Knowledge Graph with dozens of billion facts, dataCommons, DBPedia, YAGO, and Knowledge Vault, a very large scale probabilistic knowledge graph created with information extraction methods for unstructured or semi-structured information. Specifically, Knowledge Graphs provide the means of development of the newest methods for data management, data fusion, data merging, and graph and network optimization and modeling, serving as a source of high quality data and a base for information integration.In particular, Knowledge Graphs help to infer new relationships out of existing facts, giving context and meaning to the content, and can be used in applications. For example, the data generated by a computer vision system could be semantically represented and shared across numerous systems, taking into account the needs and requirements of these systems, as well as the context, provenance, licensing and consent aspects of the generated data. I demonstrate Knowledge Graphs-based methods in advanced smart city applications from the domains such as automation and construction of buildings, energy efficiency, tourism, transport.

While there has been a trend in the last decades for publishing large-scale and highly-interconnected Knowledge Graphs (KGs), their users often get overwhelmed by the task of understanding their content as a result of their size and complexity. Data profiling approaches have been proposed to summarize large KGs into concise and meaningful representations, so that they can be better explored, processed, and managed. Profiles based on schema patterns represent each triple in a KG with its schema-level counterpart, thus covering the entire KG with profiles of considerable size. In this paper, we provide empirical evidence that profiles based on schema patterns, if explored with suitable mechanisms, can be useful to help users understand the content of big and complex KGs. ABSTAT provides concise pattern-based profiles and comes with faceted interfaces for profile exploration. Using this tool we present a user study based on query completion tasks. We demonstrate that users who look at ABSTAT profiles formulate their queries better and faster than users browsing the ontology of the KGs. The latter is a pretty strong baseline considering that many KGs do not even come with a specific ontology to be explored by the users. To the best of our knowledge, this is the first attempt to investigate the impact of profiling techniques on tasks related to knowledge graph understanding with a user study.

Relational graph neural networks have garnered particular attention to encode graph context in knowledge graphs (KGs). Although they achieved competitive performance on small KGs, how to efficiently and effectively utilize graph context for large KGs remains an open problem. To this end, we propose the Relation-based Embedding Propagation (REP) method. It is a post-processing technique to adapt pre-trained KG embeddings with graph context. As relations in KGs are directional, we model the incoming head context and the outgoing tail context separately. Accordingly, we design relational context functions with no external parameters. Besides, we use averaging to aggregate context information, making REP more computation-efficient. We theoretically prove that such designs can avoid information distortion during propagation. Extensive experiments also demonstrate that REP has significant scalability while improving or maintaining prediction quality. Particularly, it averagely brings about 10% relative improvement to triplet-based embedding methods on OGBL-WikiKG2 and takes 5%-83% time to achieve comparable results as the state-of-the-art GC-OTE.

We consider the task of exploratory search through graph queries on knowledge graphs. We propose to assist the user by expanding the query with intuitive suggestions to provide a more informative (full) query that can retrieve more detailed and relevant answers. To achieve this result, we propose a model that can bridge graph search paradigms with well-established techniques for information-retrieval. Our approach does not require any additional knowledge from the user and builds on principled language modelling approaches. We empirically show the effectiveness and efficiency of our approach on a large knowledge graph and how our suggestions are able to help build more complete and informative queries.

Currently knowledge graph has been widely applied to various fields. Although the data scale is large, there is still a lot of useful but implicit information in it. Thus, a powerful reasoning system is required to derive these data. However, current reasoning systems cannot accomplish this task very well. On the one hand, stand-alone reasoning systems cannot meet the demand of large data. On the other hand, the reasoning ability of existing distributed reasoning systems is limited because of the lack of expressive inference rules. In this paper, we propose and implement a distributed reasoning system KGRL for knowledge graph based on OWL2 RL. It has a more powerful reasoning ability due to more expressive rules. It also supports optimization for redundant data. Besides, a rule-based algorithm is designed to find the inconsistent data. Experimental results show that KGRL can derive more implicit information efficiently compared to other reasoning systems. Moreover, KGRL is capable of eliminating redundant data, which can reduce the storage of knowledge graph by an average of 42%. Finally, KGRL also performs well for the detection of inconsistencies in knowledge graph.

The prosperous development of cognition and human-level intelligence have made the creation of large-scale domain knowledge graphs (KGs) a hot research field. Entity alignment (EA), which discovers entity pairs that represent the same real object in different KGs, is the key technology for building large scale knowledge graph. Existing EA methods rely mainly on the vector representation of the entity and ignoring the semantic features of an entity in a KG, result in poor alignment quality. To address this limitation, we propose a new EA framework based on knowledge embeddings (KEs) and type matching constraints. By embedding two KGs into a unified vector space, we combine the similarity of the entity vector and the degree of the entity type matching to perform more accurate cross-KG EA. The experimental results show that our proposed method has very good performance on both cross-KG and cross-language data sets. On the DBP-YG dataset, in terms of Hit@1, Hits@10, and MRR, our methods have an average increase of 27% over the baseline method MTransE. In most cases, our method significantly improves the accuracy of EA compared with state-of-the-art methods.

Inferring missing links in knowledge graphs (KG) has attracted a lot of attention from the research community. In this paper, we tackle a practical query answering task involving predicting the relation of a given entity pair. We frame this prediction problem as an inference problem in a probabilistic graphical model and aim at resolving it from a variational inference perspective. In order to model the relation between the query entity pair, we assume that there exists an underlying latent variable (paths connecting two nodes) in the KG, which carries the equivalent semantics of their relations. However, due to the intractability of connections in large KGs, we propose to use variation inference to maximize the evidence lower bound. More specifically, our framework (Diva) is composed of three modules, i.e. a posterior approximator, a prior (path finder), and a likelihood (path reasoner). By using variational inference, we are able to incorporate them closely into a unified architecture and jointly optimize them to perform KG reasoning. With active interactions among these sub-modules, Diva is better at handling noise and coping with more complex reasoning scenarios. In order to evaluate our method, we conduct the experiment of the link prediction task on multiple datasets and achieve state-of-the-art performances on both datasets.

Triple Pattern Fragments (TPFs) are a novel interface for accessing data in knowledge graphs on the web. So far, work on performance evaluation and optimization has focused mainly on SPARQL query execution over TPF servers. However, in order to devise querying techniques that efficiently access large knowledge graphs via TPFs, we need to identify and understand the variables that influence the performance of TPF servers on a fine-grained level. In this work, we assess the performance of TPFs by measuring the response time for different requests and analyze how the requests’ properties, as well as the TPF server configuration, may impact the performance. For this purpose, we developed the Triple Pattern Fragment Profiler to determine the performance of TPF server. The resource is openly available at https://doi.org/10.5281/zenodo.1211621. To this end, we conduct an empirical study over four large knowledge graphs in different server environments and configurations. As part of our analysis, we provide an extensive evaluation of the results and focus on the impact of the variables: triple pattern type, answer cardinality, page size, backend and the environment type on the response time. The results suggest that all variables impact on the measured response time and allow for deriving suggestions for TPF server configurations and query optimization.