A lightweight CNN-based knowledge graph embedding model with channel attention for link prediction.

Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors. These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities. For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding. Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning.

Knowledge Graph (KG) embedding has attracted more attention in recent years. Most KG embedding models learn from time-unaware triples. However, the inclusion of temporal information besides triples would further improve the performance of a KGE model. In this regard, we propose ATiSE, a temporal KG embedding model which incorporates time information into entity/relation representations by using Additive Time Series decomposition. Moreover, considering the temporal uncertainty during the evolution of entity/relation representations over time, we map the representations of temporal KGs into the space of multi-dimensional Gaussian distributions. The mean of each entity/relation embedding at a time step shows the current expected position, whereas its covariance (which is temporally stationary) represents its temporal uncertainty. Experimental results show that ATiSE significantly outperforms the state-of-the-art KGE models and the existing temporal KGE models on link prediction over four temporal KGs.

Knowledge Graph (KG) embedding approaches have been proved effective to infer new facts for a KG based on the existing ones–a problem known as KG completion. However, most of them have focused on static KGs, in fact, relational facts in KGs often show temporal dynamics, e.g., the fact (US, has president, Barack Obama, [2009–2017]) is only valid from 2009 to 2017. Therefore, utilizing available time information to develop temporal KG embedding models is an increasingly important problem. In this paper, we propose a new hyperplane-based time-aware KG embedding model for temporal KG completion. By employing the method of time-specific hyperplanes, our model could explicitly incorporate time information in the entity-relation space to predict missing elements in the KG more effectively, especially temporal scopes for facts with missing time information. Moreover, in order to model and infer four important relation patterns including symmetry, antisymmetry, inversion and composition, we map facts happened at the same time into a polar coordinate system. During training procedure, a time-enhanced negative sampling strategy is proposed to get more effective negative samples. Experimental results on datasets extracted from real-world temporal KGs show that our model significantly outperforms existing state-of-the-art approaches for the KG completion task.

Computing knowledge graph (KG) embeddings is a technique to learn distributional representations for components of a knowledge graph while preserving structural information. The learned embeddings can be used in multiple downstream tasks such as question answering, information extraction, query expansion, semantic similarity, and information retrieval. Over the past years, multiple embedding techniques have been proposed based on different underlying assumptions. The most actively researched models are translation-based which treat relations as translation operations in a shared (or relation-specific) space. Interestingly, almost all KG embedding models treat each triple equally, regardless of the fact that the contribution of each triple to the global information content differs substantially. Many triples can be inferred from others, while some triples are the foundational (basis) statements that constitute a knowledge graph, thereby supporting other triples. Hence, in order to learn a suitable embedding model, each triple should be treated differently with respect to its information content. Here, we propose a data-driven approach to measure the information content of each triple with respect to the whole knowledge graph by using rule mining and PageRank. We show how to compute triple-specific weights to improve the performance of three KG embedding models (TransE, TransR and HolE). Link prediction tasks on two standard datasets, FB15K and WN18, show the effectiveness of our weighted KG embedding model over other more complex models. In fact, for FB15K our TransE-RW embeddings model outperforms models such as TransE, TransM, TransH, and TransR by at least 12.98% for measuring the Mean Rank and at least 1.45% for HIT@10. Our HolE-RW model also outperforms HolE and ComplEx by at least 14.3% for MRR and about 30.4% for HIT@1 on FB15K. Finally, TransR-RW show an improvement over TransR by 3.90% for Mean Rank and 0.87% for HIT@10.

Knowledge graph completion (KGC, also referred to as link prediction) aims at predicting missing entities and relations in knowledge graphs (KGs). Knowledge graph embedding (KGE) techniques have been proven to be effective for link prediction. Currently, a series of convolutional neural networks (CNNs) based models (e.g., ConvE and its extended models) have attained excellent results for link prediction. However, several aspects that are important for link prediction using CNNs have not been considered and enhanced simultaneously, which significantly limit the performance of these models. In this paper we explore an effective KGE model based on CNNs. We investigate and discover four extremely important aspects that have a strong influence on ConvE: entity and relation embeddings, entity-to-relation interaction approaches, CNN structure, and loss function. Based on the optimization of the above four aspects, we propose a novel KGE method called ConvEICF. Through extensive experiments, we find that ConvEICF outperforms the previous state-of-the-art link prediction baselines on FB15k-237 and WN18RR datasets. In particular, ConvEICF achieves a Hits@10 score that is 11.2% and 6.5% better than ConvE on FB15k-237 and WN18RR datasets respectively. Additionally, through in-depth experiments we observe an interesting phenomenon and important finding that the very common 1-N scoring technique in KGE can be considerably improved by just adding a dropout operation. Our code is available at https://github.com/NEU-IDKE/ConvEICF.

Knowledge graph (KG) embedding aims to encode entities and relations into low-dimensional vector spaces, in turn, can support various machine learning models on KG related tasks with good performance. However, existing methods for knowledge graph embedding fail to consider the influence of the embedding space, which makes them still unsatisfactory in practical applications. In this study, we try to improve the expressiveness of the embedding space from the perspective of the metric. Specifically, we first point out the implications of Minkowski metric used in KG embedding and then make a quantitative analysis. To solve the limitations, we introduce a new metric, named Cycle metric, based on the oscillation property of the periodic function. Furthermore, we find that the function period has a significant influence on the expressiveness of the embedding space. Given a fully trained model, the smaller the period, the better the expressive ability. Finally, to validate the findings, we propose a new model, named CyclE by combining Cycle Metric and the popular KG embeddings models. Comprehensive experimental results show that Cycle is more appropriate than Minkowski for KG embedding.

Knowledge Graph Embedding (KGE) is a technique used to capture structural information from Knowledge Graphs (KGs), enabling various downstream applications such as recommender system. KGE models trained on integrated KGs from multiple organizations tend to outperform those trained on a single KG, owing to the greater richness and diversity of information. Therefore, Federated Knowledge Graph Embedding (FKGE) emerges as a promising approach for privacy-preserving training of KGE models on KGs across organizations (clients). Existing FKGE framework learns a uniform global KGE model that achieves global optima by minimizing aggregated loss across clients. However, heterogeneity among KGs often leads to divergent local optima. This presents a fundamental trade-off: ensuring global optima can compromise local performance, while focusing on local optima can decrease global model utility. To overcome this, we propose Federated Local Adaptive Knowledge Graph Embedding (FedAdap) by drawing inspiration from partial federated learning. FedAdap employs a multilayer convolutional neural network, wherein its lower layers are shared across clients to learn shared information, it maps a seed KGE model into an alignment vector space representation. Its upper layers remain private, transforming the alignment vector space representation to an adaptive KGE model tailored to the local KG. Through this, FedAdap allows clients to leverage shared information while maintaining local adaptability and mitigating the impact of KGs heterogeneity. Experiments on data sets FB15k-237 and NELL-995 show that FedAdap outperforms its counterparts in link prediction tasks.

CTEA: Context and Topic Enhanced Entity Alignment for knowledge graphs

A framework for differentially-private knowledge graph embeddings

Neuro-Symbolic Artificial Intelligence (AI) focuses on integrating symbolic and sub-symbolic systems to enhance the performance and explainability of predictive models. Symbolic and sub-symbolic approaches differ fundamentally in how they represent data and make use of data features to reach conclusions. Neuro-symbolic systems have recently received significant attention in the scientific community. However, despite efforts in neural-symbolic integration, symbolic processing can still be better exploited, mainly when these hybrid approaches are defined on top of knowledge graphs. This work is built on the statement that knowledge graphs can naturally represent the convergence between data and their contextual meaning (i.e., knowledge). We propose a hybrid system that resorts to symbolic reasoning, expressed as a deductive database, to augment the contextual meaning of entities in a knowledge graph, thus, improving the performance of link prediction implemented using knowledge graph embedding (KGE) models. An entity context is defined as the ego network of the entity in a knowledge graph. Given a link prediction task, the proposed approach deduces new RDF triples in the ego networks of the entities corresponding to the heads and tails of the prediction task on the knowledge graph (KG). Since knowledge graphs may be incomplete and sparse, the facts deduced by the symbolic system not only reduce sparsity but also make explicit meaningful relations among the entities that compose an entity ego network. As a proof of concept, our approach is applied over a KG for lung cancer to predict treatment effectiveness. The empirical results put the deduction power of deductive databases into perspective. They indicate that making explicit deduced relationships in the ego networks empowers all the studied KGE models to generate more accurate links.

Recently, Hyper-relational Knowledge Graphs (HKGs) have been proposed as an extension of traditional Knowledge Graphs (KGs) to better represent real-world facts with additional qualifiers. As a result, researchers have attempted to adapt classical Knowledge Graph Embedding (KGE) models for HKGs by designing extra qualifier processing modules. However, it remains unclear whether the superior performance of Hyper-relational KGE (HKGE) models arises from their base KGE model or the specially designed extension module. In this paper, we data-wise convert HKGs to KG format using decomposition methods and then evaluate several classical KGE models' performance on HKGs. Our results show that some KGE models achieve comparable performance to HKGE models. Upon further analysis, we find that the decomposition methods alter the original HKG topology and fail to fully preserve HKG information. Moreover, we observe that current HKGE models are either insufficient in capturing the graph's long-range dependency or struggle to integrate main-triple and qualifier information due to the information compression issue. To further justify our findings and provide a direction for HKGE research, we propose FormerGNN, which employs a qualifier integrator to preserve the original HKG topology, a GNN-based graph encoder to capture the graph's long-range dependencies, and an improved approach for integrating main-triple and qualifier information to mitigate compression issues. Our experimental results demonstrate that FormerGNN outperforms existing HKGE models.

We study the problem of generating data poisoning attacks against Knowledge Graph Embedding (KGE) models for the task of link prediction in knowledge graphs. To poison KGE models, we propose to exploit their inductive abilities which are captured through the relationship patterns like symmetry, inversion and composition in the knowledge graph. Specifically, to degrade the model's prediction confidence on target facts, we propose to improve the model's prediction confidence on a set of decoy facts. Thus, we craft adversarial additions that can improve the model's prediction confidence on decoy facts through different inference patterns. Our experiments demonstrate that the proposed poisoning attacks outperform state-of-art baselines on four KGE models for two publicly available datasets. We also find that the symmetry pattern based attacks generalize across all model-dataset combinations which indicates the sensitivity of KGE models to this pattern.

Knowledge graphs exhibit a typical hierarchical structure and find extensive applications in various artificial intelligence domains. However, large-scale knowledge graphs need to be completed, which limits the performance of knowledge graphs in downstream tasks. Knowledge graph embedding methods have emerged as a primary solution to enhance knowledge graph completeness. These methods aim to represent entities and relations as low-dimensional vectors, focusing on handling relation patterns and multi-relation types. Researchers need to pay more attention to the crucial feature of hierarchical relationships in real-world knowledge graphs. We propose a novel knowledge graph embedding model called Hierarchy-Aware Paired Relation Vectors Knowledge Graph Embedding (HPRE) to bridge this gap. By leveraging the power of 2D coordinates, HPRE adeptly model relation patterns, multi-relation types, and hierarchical features in the knowledge graph. Specifically, HPRE employs paired relation vectors to capture the distinct characteristics of head and tail entities, facilitating a better fit for relational patterns and multi-relation scenarios. Additionally, HPRE employs angular coordinates to differentiate entities at various levels of the hierarchy, effectively representing the hierarchical nature of the knowledge graph. The experimental results show that the HPRE model can effectively learn the hierarchical features of the knowledge graph and achieve state-of-the-art experimental results on multiple real-world datasets for the link prediction task.

Biomedical knowledge graphs (KGs), which can help with the understanding of complex biological systems and pathologies, have begun to play a critical role in medical practice and research. However, challenges remain in their embedding and use due to their complex nature and the specific demands of their construction. Existing studies often suffer from problems such as sparse and noisy datasets, insufficient modeling methods and non-uniform evaluation metrics. In this work, we established a comprehensive KG system for the biomedical field in an attempt to bridge the gap. Here, we introduced PharmKG, a multi-relational, attributed biomedical KG, composed of more than 500000 individual interconnections between genes, drugs and diseases, with 29 relation types over a vocabulary of ~8000 disambiguated entities. Each entity in PharmKG is attached with heterogeneous, domain-specific information obtained from multi-omics data, i.e. gene expression, chemical structure and disease word embedding, while preserving the semantic and biomedical features. For baselines, we offered nine state-of-the-art KG embedding (KGE) approaches and a new biological, intuitive, graph neural network-based KGE method that uses a combination of both global network structure and heterogeneous domain features. Based on the proposed benchmark, we conducted extensive experiments to assess these KGE models using multiple evaluation metrics. Finally, we discussed our observations across various downstream biological tasks and provide insights and guidelines for how to use a KG in biomedicine. We hope that the unprecedented quality and diversity of PharmKG will lead to advances in biomedical KG construction, embedding and application.

Unveiling the power of knowledge graph embedding in knowledge aware deep recommender systems for e-commerce: A comparative study