Out-of-Sample Representation Learning for Knowledge Graphs

Representation Learning (RL) of knowledge graphs aims to project both entities and relations into a continuous low dimensional space. There exits two kinds of representation methods for entities in Knowledge Graphs (KGs), including structure-based representation and description-based representation. Most methods represent entities with fact triples of KGs through translating embedding models, which can't integrate the rich information in entities descriptions with triple structure information. In this paper, we propose a novel RL method named as Representation Learning with Complete semantic Description of Knowledge Graphs (RLCD), which can exploit all semantic information of entity descriptions and fact triples of KGs, to enrich the semantic representations of KGs. More specifically, we explore Doc2Vec encoder model to encode all semantic information of entity descriptions without losing the relevance in the context of entities descriptions, and further learn knowledge representations from triples with entity descriptions. The experiment results show that RLCD gets better performance that state-of-the-art method DKRL, in terms of mean rank value and HITS. Moreover, RLCD is much faster than DKRL.

Numerous large knowledge graphs, such as DBpedia, Wikidata, Yago and Freebase, have been developed in the last decade, which contain millions of facts about various entities in the world. These knowledge graphs have proven to be incredibly useful for intelligent Web search, question understanding, in-context advertising, social media mining, and biomedicine. As some researchers have pointed out, a knowledge graph is not just a graph database, but it should have a layer of conceptual knowledge, which is usually represented as a set of first-order rules. However, it is challenging to automatically extract first-order rules from large knowledge graphs. In particular, traditional models are usually unable to handle rule learning in large knowledge graphs. This chapter aims to present state-of-the-art techniques and models on learning first-order rules using representation learning. By first recalling some basics of rule learning in knowledge graphs, we will introduce useful techniques of embedding-based rule learning through major models such as RLvLR and TyRuLe, which embed paths in knowledge graphs into latent spaces for candidate rule search. Then we evaluate the rule learning efficiency and the quality of automatically learned rules by applying them in link prediction. Before the chapter is concluded, we will also discuss some future research problems in the area.

The representation learning of knowledge graph refers to embedding entities and relations in knowledge graph into a low-dimensional dense vector space. Existing knowledge graph embedding models mostly chose Euclidean Space as their vector space and consider each fact triple in knowledge graph independently. However, Euclidean Space is unable to represent knowledge effectively due to its strict constraints and mathematical expression. Besides, entities in knowledge graph are not isolated, while these models ignore the association between entities. To solve the problems above, we proposed a text-enhanced knowledge graph representation learning model in Hyperbolic Space. We utilize rich semantic information of entity description by using Transformer Encoder to enhance the ability of knowledge representation. Besides, we embed entities and relations into Hyperbolic Space, which can better capture hierarchical information of the knowledge graph. Experiments on benchmark dataset show that our method achieves better performance compared with other state-of-art methods.KeywordsKnowledge graphRepresentation learningEmbedding modelHyperbolic space

This paper presents compact yet comprehensive feature representations for the electroencephalogram (EEG) signal to achieve efficient epileptic seizure prediction performance. The initial EEG feature vectors are formed by acquiring the dominant amplitude and frequency components on an epoch-by-epoch basis from the EEG signals. These extracted parameters can reveal the intrinsic EEG signal changes as well as the underlying stage transitions. To improve the efficacy of feature extraction, an elimination-based feature selection method has been applied on the initial feature vectors. This diminishes redundant and noisy points, providing each patient with a lower dimensional and independent final feature form. In this context, our study is distinguished from that of others currently prevailing. Usually, these latter approaches adopted feature extraction processes, which employed time-consuming high-dimensional parameter sets. Machine learning approaches that are considered as state of the art have been employed to build patient-specific binary classifiers that can divide the extracted feature parameters into preictal and interictal groups. Through out-of-sample evaluation on the intracranial EEG recordings provided by the publicly available Freiburg dataset, promising prediction performance has been attained. Specifically, we have achieved 98.8% sensitivity results on the 19 patients included in our experiment, where only one of 83 seizures across all patients was not predicted. To make this investigation more comprehensive, we have conducted extensive comparative studies with other recently published competing approaches, in which the advantages of our method are highlighted.

The knowledge graph with relational abundant information has been widely used as the basic data support for the retrieval platforms. Image and text descriptions added to the knowledge graph enrich the node information, which accounts for the advantage of the multi-modal knowledge graph. In the field of cross-modal retrieval platforms, multi-modal knowledge graphs can help to improve retrieval accuracy and efficiency because of the abundant relational information provided by knowledge graphs. The representation learning method is significant to the application of multi-modal knowledge graphs. This paper proposes a distributed collaborative vector retrieval platform (DCRL-KG) using the multimodal knowledge graph VisualSem as the foundation to achieve efficient and high-precision multimodal data retrieval. Firstly, use distributed technology to classify and store the data in the knowledge graph to improve retrieval efficiency. Secondly, this paper uses BabelNet to expand the knowledge graph through multiple filtering processes and increase the diversification of information. Finally, this paper builds a variety of retrieval models to achieve the fusion of retrieval results through linear combination methods to achieve high-precision language retrieval and image retrieval. The paper uses sentence retrieval and image retrieval experiments to prove that the platform can optimize the storage structure of the multi-modal knowledge graph and have good performance in multi-modal space.

Entity representation learning is both fundamental and crucial for link prediction in knowledge graph. Existing entity representation learning approaches mainly focus on learning from knowledge triples with single modal text data which could only describe the things or concepts in words. However, multimodal data commonly exists in the real worlds and may describe things or concepts from different aspects. Therefore, multimodal entities have emerged in knowledge graph in recent years, which greatly enrich the knowledge graph by different information. This paper focuses on how to learn more robust and comprehensive representations of multimodal entities for better link prediction. A new representation learning method for multimodal entities is proposed, called Graph Attention with Bi-pooling Representation Learning (GBRL), where multimodal fusion and knowledge embedding are combined for learning representations of an entity with consideration of its multimodal neighbors. Firstly, feature vectors are obtained by pre-trained models for each modality. Then, the bilinear pooling is utilized for combining feature vectors of multimodal entities into multimodal feature vectors. A two-layer graph attention module is followed to generate multimodal representations of an entity from its multimodal neighbors. Finally, the inception decoder uses multimodal knowledge embedding to calculate scores for link prediction. The experimental results have shown that the proposed method performs well on link prediction by consideration of multimodal entities comparing with traditional methods that using only single modal text data.

Multi-modal knowledge graphs representation learning via multi-headed self-attention

In recent years, the related research of entity alignment has mainly focused on entity alignment via knowledge embeddings and graph neural networks; however, these proposed models usually suffer from structural heterogeneity and the large-scale problem of knowledge graph. A novel entity alignment model based on graph isomorphic network and compressed sensing is proposed. First, for the problem of structural heterogeneity, graph isomorphic network encoder is applied in knowledge graph to capture structural similarity of entity relation. Second, for the problem of large scale, key node and community are integrated for priority entity alignment to improve execution speed. However, the exiting node importance ranking algorithm cannot accurately identify key node in knowledge graph. So the compressed sensing is adopted in node importance ranking to improve the accuracy of identifying key node. The authors have carried out several experiments to test the effect and efficiency of the proposed entity alignment model.

In this chapter, we introduce the basic concept of a knowledge graph (KG). While a knowledge graph seems to be a very simple way of representing information, it turns out to be quite powerful and is almost a lingua franca of sorts between humans and machines. On the one hand, knowledge graphs, if constructed properly, contain much of the useful information that can be found in ‘natural language’ documents like newswire or even social media. On the other hand, machines can process knowledge graphs in a variety of ways, leading to applications and breakthroughs in areas like semantic search, question answering, entity resolution, and representation learning.

To conquer incompleteness of knowledge graphs (KGs), we focus on a relation prediction task that completes KGs by ranking all relationships between entities and selecting the top one. Most existing methods for relation prediction are representation-based models, which learn the structural information of KGs to obtain the embeddings of entities and relationships but fail to effectively use relation paths to predict potential relationships between two entities. The traditional path-based logical models consider the path characteristics but ignore the global semantics of entities in KGs, such as surrounding entities and relationships. In this paper, we propose path embedded hybrid reasoning model (PE-HRM), a novel reasoning mode based on logical reasoning, representation learning and neural network. PE-HRM considers structural characteristics of entities and path features between them and effectively fuses these features. Finally, we evaluate PE-HRM on datasets FB15K237 and NELL995. Experimental results show that PE-HRM significantly outperforms baseline models.

RuMER-RL: A hybrid framework for sparse knowledge graph explainable reasoning

Purpose Path information in knowledge graphs can provide explicit explanations for recommendation decisions, thus becoming a focus in explainable recommendation research. However, limited studies about explainable recommendation problems result in low model transparency and poor persuasiveness, affecting user experience. Therefore, the goal is to provide accurate and interpretable recommendations for recommendations. Design/methodology/approach This study proposes a recommendation reasoning method based on knowledge graph and reinforcement learning. To alleviate the noise problem in the state space, a multi-head attention mechanism is used to learn state expressions. A dual critic network is used to optimize long-term and short-term rewards simultaneously, achieving path reasoning in the knowledge graph to provide short-term and long-term value explanations for recommendation decisions. Findings We conduct extensive experiments on the real-world benchmark dataset and the domain dataset to validate the effectiveness of our method on the recommendation and explanation tasks, which proves the method's ability to generate high-quality and interpretable course recommendations. Originality/value Developing explainable recommendation methods based on the combination of knowledge graph and reinforcement learning is crucial to overcome the current limitations. A recommendation system that integrates knowledge reasoning, autonomous learning and interpretability may meet the needs of the modern education field for explainable recommendation systems.

Performing complex First-Order Logic (FOL) queries on knowledge graphs is crucial for advancing knowledge reasoning. Knowledge graphs encapsulate rich semantic interactions among entities, encompassing both explicit structural knowledge represented by triples <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(e_{1}, r, e_{2})$</tex-math></inline-formula> and implicit relational knowledge through multi-hop paths <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(e_{1} \stackrel{r_{1}}{\rightarrow } \cdots e_{3} \cdots \stackrel{r_{2}}{\rightarrow } e_{2})$</tex-math></inline-formula>. Traditional models often focus solely on either triple-level or path-level knowledge, overlooking the benefits of integrating both to enhance logic query answering. This oversight leads to suboptimal representation learning and inefficient query reasoning. To overcome these challenges, we introduce a new <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</b>emantic-<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b>ware representation learning model for <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</b>uery-answering <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</b>mbeddings (<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SAQE</b>). Specifically, SAQE employs a joint learning approach that integrates triple-level and path-level knowledge semantics and captures both explicit and implicit contextual nuances within the knowledge graph, yielding more accurate and contextually relevant representations. To efficiently handle the large combinatorial search spaces in FOL reasoning, we propose a novel hierarchical reasoning optimization strategy by a multi-hop tree thus optimizing subqueries rooted at variable nodes in a divide-and-conquer manner. Theoretical analysis confirms that SAQE effectively supports various types of FOL reasoning and enhances generalizations for query answering. Extensive experiments demonstrate that our model achieves state-of-the-art performance across several established datasets.

Several real-world complex systems have graph-structured data, including social networks, biological networks, and knowledge graphs. A continuous increase in the quantity and quality of these graphs demands learning models to unlock the potential of this data and execute tasks, including node classification, graph classification, and link prediction. This tutorial presents machine learning on graphs, focusing on how representation learning - from traditional approaches (e.g., matrix factorization and random walks) to deep neural architectures - fosters carrying out those tasks. We also introduce representation learning over dynamic and knowledge graphs. Lastly, we discuss open problems, such as scalability and distributed network embedding systems.

Representation learning for temporal knowledge graphs has attracted increasing attention in recent years. In this paper, we study the problem of learning dynamic embeddings for temporal knowledge graphs. We address this problem by proposing a Dynamic Bayesian Knowledge Graphs Embedding model (DBKGE), which is able to dynamically track the semantic representations of entities over time in a joint metric space and make predictions for the future. Unlike other temporal knowledge graph embedding methods, DBKGE is a novel probabilistic representation learning method that aims at inferring dynamic embeddings of entities in a streaming scenario. To obtain high-quality embeddings and model their uncertainty, our DBKGE embeds entities with means and variances of Gaussian distributions. Based on amortized inference, an online inference algorithm is proposed to jointly learn the latent representations of entities and smooth their changes across time. Experiments on Yago and Wiki datasets demonstrate that our proposed algorithm outperforms the state-of-the-art static and temporal knowledge graph embedding models.