Learning of Knowledge Graphs with Entity Descriptions

Representation Learning (RL) of knowledge graphs aims to project both entities and relations into a continuous low dimensional space. There exits two kinds of representation methods for entities in Knowledge Graphs (KGs), including structure-based representation and description-based representation. Most methods represent entities with fact triples of KGs through translating embedding models, which can't integrate the rich information in entities descriptions with triple structure information. In this paper, we propose a novel RL method named as Representation Learning with Complete semantic Description of Knowledge Graphs (RLCD), which can exploit all semantic information of entity descriptions and fact triples of KGs, to enrich the semantic representations of KGs. More specifically, we explore Doc2Vec encoder model to encode all semantic information of entity descriptions without losing the relevance in the context of entities descriptions, and further learn knowledge representations from triples with entity descriptions. The experiment results show that RLCD gets better performance that state-of-the-art method DKRL, in terms of mean rank value and HITS. Moreover, RLCD is much faster than DKRL.

Knowledge Graphs (KGs) comprise of interlinked information in the form of entities and relations between them in a particular domain and provide the backbone for many applications. However, the KGs are often incomplete as the links between the entities are missing. Link Prediction is the task of predicting these missing links in a KG based on the existing links. Recent years have witnessed many studies on link prediction using KG embeddings which is one of the mainstream tasks in KG completion. To do so, most of the existing methods learn the latent representation of the entities and relations whereas only a few of them consider contextual information as well as the textual descriptions of the entities. This paper introduces an attentive encoder-decoder based link prediction approach considering both structural information of the KG and the textual entity descriptions. Random walk based path selection method is used to encapsulate the contextual information of an entity in a KG. The model explores a bidirectional Gated Recurrent Unit (GRU) based encoder-decoder to learn the representation of the paths whereas SBERT is used to generate the representation of the entity descriptions. The proposed approach outperforms most of the state-of-the-art models and achieves comparable results with the rest when evaluated with FB15K, FB15K-237, WN18, WN18RR, and YAGO3-10 datasets.

Multilingual knowledge graph (KG) embeddings provide latent semantic representations of entities and structured knowledge with cross-lingual inferences, which benefit various knowledge-driven cross-lingual NLP tasks. However, precisely learning such cross-lingual inferences is usually hindered by the low coverage of entity alignment in many KGs. Since many multilingual KGs also provide literal descriptions of entities, in this paper, we introduce an embedding-based approach which leverages a weakly aligned multilingual KG for semi-supervised cross-lingual learning using entity descriptions. Our approach performs co-training of two embedding models, i.e. a multilingual KG embedding model and a multilingual literal description embedding model. The models are trained on a large Wikipedia-based trilingual dataset where most entity alignment is unknown to training. Experimental results show that the performance of the proposed approach on the entity alignment task improves at each iteration of co-training, and eventually reaches a stage at which it significantly surpasses previous approaches. We also show that our approach has promising abilities for zero-shot entity alignment, and cross-lingual KG completion.

The representation learning of knowledge graph refers to embedding entities and relations in knowledge graph into a low-dimensional dense vector space. Existing knowledge graph embedding models mostly chose Euclidean Space as their vector space and consider each fact triple in knowledge graph independently. However, Euclidean Space is unable to represent knowledge effectively due to its strict constraints and mathematical expression. Besides, entities in knowledge graph are not isolated, while these models ignore the association between entities. To solve the problems above, we proposed a text-enhanced knowledge graph representation learning model in Hyperbolic Space. We utilize rich semantic information of entity description by using Transformer Encoder to enhance the ability of knowledge representation. Besides, we embed entities and relations into Hyperbolic Space, which can better capture hierarchical information of the knowledge graph. Experiments on benchmark dataset show that our method achieves better performance compared with other state-of-art methods.KeywordsKnowledge graphRepresentation learningEmbedding modelHyperbolic space

Knowledge Graph Completion (KGC) aims at complementing missing relationships between entities in a Knowledge Graph (KG). While closed-world KGC approaches utilizing the knowledge within KG could only complement very limited number of missing relations, more and more approaches tend to get knowledge from open-world resources such as online encyclopedias and newswire corpus. For instance, a recent proposed open-world KGC model called ConMask learns embeddings of the entity’s name and parts of its text-description to connect unseen entities to the KG. However, this model does not make full use of the rich feature information in the text descriptions, besides, the proposed relationship-dependent content masking method may easily miss to find the target-words. In this paper, we propose to use a Multiple Interaction Attention (MIA) mechanism to model the interactions between the head entity description, head entity name, the relationship name, and the candidate tail entity descriptions, to form the enriched representations. Our empirical study conducted on two real-world data collections shows that our approach achieves significant improvements comparing to state-of-the-art KGC methods.

In this paper, we present a model which incorporates biomedical knowledge graph, graph embedding and deep learning methods for literature-based discovery. Firstly, the relations between entities are extracted from biomedical abstracts and then a knowledge graph is constructed by using these obtained relations. Secondly, the graph embedding technologies are applied to convert the entities and relations in the knowledge graph into a low-dimensional vector space. Thirdly, a bidirectional Long Short-Term Memory network is trained based on the entity associations represented by the pre-trained graph embeddings. Finally, the learned model is used for open and closed literature-based discovery tasks. The experimental results show that our method could not only effectively discover hidden associations between entities, but also reveal the corresponding mechanism of interactions. It suggests that incorporating knowledge graph and deep learning methods is an effective way for capturing the underlying complex associations between entities hidden in the literature.

Knowledge graphs display various entities and their relationships in the real world based on knowledge representation and can analyze and predict the intrinsic relationship between knowledge embeddings through data mining and information processing, which are widely used in search engines, web analytics and smart recommendation areas. As existing knowledge graph information is continuously created and grown, it is a curial task to determine whether the information in knowledge graph is correct and to complete the missing information. In response to this challenge, the researchers proposed a number of graph convolutional network (GCN)-based models to characterize knowledge graphs. The state-of-the-art model is R-GCN, which can effectively extract features. This paper deeply studies the algorithm ideas and results of the R-GCN model, explores whether the model can be further optimized in entity classification and link prediction, and finally, improves the original model and proposes a relational aggregation graph convolutional network. Specifically, this paper finds that a subset of the set of entities may be directly connected to a central entity. All the entities in this subset possess partially identical attributes. At the same time, the relationships between these entities and the central entity may be similar. These similar attributes and relationships can be abstractly aggregated into virtual entities and virtual relationships, respectively, to better extract the topological relationship features. This paper uses the FB15k dataset to evaluate the performance of the proposed model on knowledge graph completion tasks. The experimental results show that the proposed RA-GCN model achieves a certain level improvement compared with the original model and that it can extract knowledge graph topological relationship characteristics more effectively.

Web pages about finance and economics on the Internet contain vast amounts of data with descriptions of various entities and their relationships. These HTML pages contain semi-structured as well as structured data, which can construct graphable knowledge acquisition. This paper proposes a semi-automatic construction method of financial knowledge graph based on Web. Populated template is used to construct the bait template for the data source. Knowledge graph is constructed by extracting triples data from multiple data sources by enticing templates. In addition, this paper designs an application system to build the knowledge graph for the financial field, which can significantly reduce the human participation and realize the construction of the knowledge graph quickly.

Numerous large knowledge graphs, such as DBpedia, Wikidata, Yago and Freebase, have been developed in the last decade, which contain millions of facts about various entities in the world. These knowledge graphs have proven to be incredibly useful for intelligent Web search, question understanding, in-context advertising, social media mining, and biomedicine. As some researchers have pointed out, a knowledge graph is not just a graph database, but it should have a layer of conceptual knowledge, which is usually represented as a set of first-order rules. However, it is challenging to automatically extract first-order rules from large knowledge graphs. In particular, traditional models are usually unable to handle rule learning in large knowledge graphs. This chapter aims to present state-of-the-art techniques and models on learning first-order rules using representation learning. By first recalling some basics of rule learning in knowledge graphs, we will introduce useful techniques of embedding-based rule learning through major models such as RLvLR and TyRuLe, which embed paths in knowledge graphs into latent spaces for candidate rule search. Then we evaluate the rule learning efficiency and the quality of automatically learned rules by applying them in link prediction. Before the chapter is concluded, we will also discuss some future research problems in the area.

Purpose Path information in knowledge graphs can provide explicit explanations for recommendation decisions, thus becoming a focus in explainable recommendation research. However, limited studies about explainable recommendation problems result in low model transparency and poor persuasiveness, affecting user experience. Therefore, the goal is to provide accurate and interpretable recommendations for recommendations. Design/methodology/approach This study proposes a recommendation reasoning method based on knowledge graph and reinforcement learning. To alleviate the noise problem in the state space, a multi-head attention mechanism is used to learn state expressions. A dual critic network is used to optimize long-term and short-term rewards simultaneously, achieving path reasoning in the knowledge graph to provide short-term and long-term value explanations for recommendation decisions. Findings We conduct extensive experiments on the real-world benchmark dataset and the domain dataset to validate the effectiveness of our method on the recommendation and explanation tasks, which proves the method's ability to generate high-quality and interpretable course recommendations. Originality/value Developing explainable recommendation methods based on the combination of knowledge graph and reinforcement learning is crucial to overcome the current limitations. A recommendation system that integrates knowledge reasoning, autonomous learning and interpretability may meet the needs of the modern education field for explainable recommendation systems.

CTEA: Context and Topic Enhanced Entity Alignment for knowledge graphs

Cloud manufacturing has emerged as a service-oriented paradigm, in which knowledge graphs (KGs) play a crucial role in enabling modularization and on-demand servitization by converting unstructured resources into a structured graph representation and factual knowledge for manufacturing tasks. KG embedding converts entities and relations into a low-dimensional space while expressing rich semantics of high-dimensional KGs. Existing works just focus on translating relations surrounding the nodes of graphs, while ignoring the applicability for modeling and inferring multiple relation patterns in the manufacturing context. Furthermore, the relative importance of complex manufacturing relations among different dimensions in embedded procedures has been ignored, leading to unclear representation learning of complex relations and entities in KGs. To overcome this issue, in this article, a novel relation and entropy weight-aware embedding model is proposed, named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TransCE</i> . <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TransCE</i> performs coordinated transformations in vector space by normalizing the distance to integrate coordination. An entropy-based weighting method is also proposed to represent complex relations and entities surrounding the edges of graph embedding and assign the weighted value of relations to support the score function. Extended experiments are performed on several datasets, and a manufacturing indicates that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TransCE</i> shows remarkable improvement relative to baseline models.

Graph, as an important data representation, is ubiquitous in many real world applications ranging from social network analysis to biology. How to correctly and effectively learn and extract information from graph is essential for a large number of machine learning tasks. Graph embedding is a way to transform and encode the data structure in high dimensional and non-euclidean feature space to a low dimensional and structural space, which is easily exploited by other machine learning algorithms. We have witnessed a huge surge of such embedding methods, from statistical approaches to recent deep learning methods such as the graph convolutional networks (GCN). Deep learning approaches usually outperform the traditional methods in most graph learning benchmarks by building an end-to-end learning framework to optimize the loss function directly. However, most of the existing GCN methods can only perform convolution operations with node features, while ignoring the handy information in edge features, such as relations in knowledge graphs. To address this problem, we present CensNet, Convolution with Edge-Node Switching graph neural network, for learning tasks in graph-structured data with both node and edge features. CensNet is a general graph embedding framework, which embeds both nodes and edges to a latent feature space. By using line graph of the original undirected graph, the role of nodes and edges are switched, and two novel graph convolution operations are proposed for feature propagation. Experimental results on real-world academic citation networks and quantum chemistry graphs show that our approach achieves or matches the state-of-the-art performance in four graph learning tasks, including semi-supervised node classification, multi-task graph classification, graph regression, and link prediction.

Knowledge graph (KG) embedding aims to encode entities and relations into low-dimensional vector spaces, in turn, can support various machine learning models on KG related tasks with good performance. However, existing methods for knowledge graph embedding fail to consider the influence of the embedding space, which makes them still unsatisfactory in practical applications. In this study, we try to improve the expressiveness of the embedding space from the perspective of the metric. Specifically, we first point out the implications of Minkowski metric used in KG embedding and then make a quantitative analysis. To solve the limitations, we introduce a new metric, named Cycle metric, based on the oscillation property of the periodic function. Furthermore, we find that the function period has a significant influence on the expressiveness of the embedding space. Given a fully trained model, the smaller the period, the better the expressive ability. Finally, to validate the findings, we propose a new model, named CyclE by combining Cycle Metric and the popular KG embeddings models. Comprehensive experimental results show that Cycle is more appropriate than Minkowski for KG embedding.

Entity linking is a pivotal factor for building robust Question Answering systems over Knowledge Graphs (KGQA), and representation of entities occupies an important position for tackling entity linking for questions. To alleviate the deficiency of entity descriptions that contextual knowledge is insufficient for entity representations, we introduce entity experiences as a new text-style contextual knowledge source to enrich entity representations, and propose an Experience Enhanced Entity Linking framework called as \(E^3L\). For the modeling of entity experiences, we derive embeddings using the entity mentions occurring in history questions, and design an attention-based retriever to capture key information relative to user questions. Then the distilled entity experiences are integrated with entity descriptions to enhance entity representations, and question representations are refined with a multi-level attention mechanism. Finally, entity linking is improved with the entity representations and the refined representations of questions. Experimental results on end-to-end benchmark datasets demonstrate that our approach achieves state-of-the-art F1-score, and provides an effective way to improve test performance for universal models using entity experiences without fine-tuning.KeywordsQuestion Answering over Knowledge GraphsEntity linkingMention detectionEntity disambiguationEntity experience