Intentional knowledge map representation learning method based on fusion of action roadmap

Multi-modal knowledge graphs representation learning via multi-headed self-attention

Recently, a significant number of studies have focused on knowledge graph completion using rule-enhanced learning techniques, supported by the mined soft rules in addition to the hard logic rules. However, due to the difficulty in determining the confidences of the soft rules without the global semantics of knowledge graph such as the semantic relatedness between relations, the knowledge representation may not be optimal, leading to degraded effectiveness in its application to knowledge graph completion tasks. To address this challenge, this paper proposes a retrofit framework that iteratively enhances the knowledge representation and confidences of soft rules. Specifically, the soft rules guide the learning of knowledge representation, and the representation, in turn, provides global semantic of the knowledge graph to optimize the confidences of soft rules. Extensive evaluation shows that our method achieves new state-of-the-art results on link prediction and triple classification tasks, brought by the fine-tuned confidences of soft rules.

The representation learning in knowledge graph aims to embed the semantic information of the research object into dense low dimensional distributed real valued vectors. The classical knowledge representation model, TransR, uses the relation specific matrix to represent the entities and relations in different semantic spaces, so as to solve the separation problem of semantic space. However, the TransR model only considers the different types of relations, ignores the entity types, and has defects in the processing of inverse relations. To solve the model problems, a novel knowledge representation method for multiple pattern embeddings based on entity-relation mapping matrix (M-TransER) is presented. The M-TransER model introduces the entity-relation mapping matrix, which makes the model consider the entity type while considering the relation type. On the basis that entities and relations are modeled as positive translation geometric distance, the inverse translation geometric distance model and the symmetrical relation model are fused. Eliminate the defects of the model in dealing with inverse relations, and improve the efficiency of symmetric relation processing. Experimental results on open datasets show that the M-TransER model has made great progress and demonstrated its superiority over the existing models and methods in both link prediction and triple classification tasks.

Derived from knowledge bases, knowledge graphs represent knowledge expressions in graphs, which utilize nodes and edges to denote entities and relations conceptually. Knowledge graph can be described in textual triple form, consisting of head entities, tail entities and relations between entities. In order to represent elements in knowledge graphs, knowledge graph embedding techniques are proposed to map entities and relations into continuous vector spaces as numeric vectors for computational efficiency. Convolution-based knowledge graph embedding models have promising performance for knowledge graph representation learning. However, the input of those neural network-based models is frequently in handmade forms and may suffer from low efficiency in feature extraction procedure of the models. In this paper, a convolutional autoencoder is proposed for knowledge graph representation learning with entity pairs as input, aiming to obtain corresponding hidden relation representation. In addition, a bi-directional relation encoding network is utilized to represent semantic of entities in different directional relation patterns, as an encoder to output representation for initialization of the convolutional autoencoder. Experiments are conducted on standard datasets including, WN18RR, Kinship, NELL-995 and FB15k-237 as a link prediction task. Besides, input embedding matrix composed of different ingredients is designed to evaluate performances of the convolutional autoencoder. The results demonstrate that our model is effective in learning representation from entity feature interactions.

Neuro-Symbolic Artificial Intelligence (AI) focuses on integrating symbolic and sub-symbolic systems to enhance the performance and explainability of predictive models. Symbolic and sub-symbolic approaches differ fundamentally in how they represent data and make use of data features to reach conclusions. Neuro-symbolic systems have recently received significant attention in the scientific community. However, despite efforts in neural-symbolic integration, symbolic processing can still be better exploited, mainly when these hybrid approaches are defined on top of knowledge graphs. This work is built on the statement that knowledge graphs can naturally represent the convergence between data and their contextual meaning (i.e., knowledge). We propose a hybrid system that resorts to symbolic reasoning, expressed as a deductive database, to augment the contextual meaning of entities in a knowledge graph, thus, improving the performance of link prediction implemented using knowledge graph embedding (KGE) models. An entity context is defined as the ego network of the entity in a knowledge graph. Given a link prediction task, the proposed approach deduces new RDF triples in the ego networks of the entities corresponding to the heads and tails of the prediction task on the knowledge graph (KG). Since knowledge graphs may be incomplete and sparse, the facts deduced by the symbolic system not only reduce sparsity but also make explicit meaningful relations among the entities that compose an entity ego network. As a proof of concept, our approach is applied over a KG for lung cancer to predict treatment effectiveness. The empirical results put the deduction power of deductive databases into perspective. They indicate that making explicit deduced relationships in the ego networks empowers all the studied KGE models to generate more accurate links.

Knowledge Graph Embedding (KGE) is a technique used to capture structural information from Knowledge Graphs (KGs), enabling various downstream applications such as recommender system. KGE models trained on integrated KGs from multiple organizations tend to outperform those trained on a single KG, owing to the greater richness and diversity of information. Therefore, Federated Knowledge Graph Embedding (FKGE) emerges as a promising approach for privacy-preserving training of KGE models on KGs across organizations (clients). Existing FKGE framework learns a uniform global KGE model that achieves global optima by minimizing aggregated loss across clients. However, heterogeneity among KGs often leads to divergent local optima. This presents a fundamental trade-off: ensuring global optima can compromise local performance, while focusing on local optima can decrease global model utility. To overcome this, we propose Federated Local Adaptive Knowledge Graph Embedding (FedAdap) by drawing inspiration from partial federated learning. FedAdap employs a multilayer convolutional neural network, wherein its lower layers are shared across clients to learn shared information, it maps a seed KGE model into an alignment vector space representation. Its upper layers remain private, transforming the alignment vector space representation to an adaptive KGE model tailored to the local KG. Through this, FedAdap allows clients to leverage shared information while maintaining local adaptability and mitigating the impact of KGs heterogeneity. Experiments on data sets FB15k-237 and NELL-995 show that FedAdap outperforms its counterparts in link prediction tasks.

A multi-hierarchical aggregation-based graph convolutional network for industrial knowledge graph embedding towards cognitive intelligent manufacturing

Representation Learning (RL) of knowledge graphs aims to project both entities and relations into a continuous low dimensional space. There exits two kinds of representation methods for entities in Knowledge Graphs (KGs), including structure-based representation and description-based representation. Most methods represent entities with fact triples of KGs through translating embedding models, which can't integrate the rich information in entities descriptions with triple structure information. In this paper, we propose a novel RL method named as Representation Learning with Complete semantic Description of Knowledge Graphs (RLCD), which can exploit all semantic information of entity descriptions and fact triples of KGs, to enrich the semantic representations of KGs. More specifically, we explore Doc2Vec encoder model to encode all semantic information of entity descriptions without losing the relevance in the context of entities descriptions, and further learn knowledge representations from triples with entity descriptions. The experiment results show that RLCD gets better performance that state-of-the-art method DKRL, in terms of mean rank value and HITS. Moreover, RLCD is much faster than DKRL.

Knowledge graph embedding models are used to learn low‐dimensional representations of entities and relations in knowledge graphs. In this paper, we propose Multi‐RAttE, an attention‐based learning method for multiple relational knowledge graph embedding representation, which divides the information transfer in the knowledge graph into cross‐relational information transfer and relation‐specific information transfer, and divides the embedding of knowledge graph entities into structural embedding and multi‐relational embedding for joint learning. To objectively analyse the performance of the Multi‐RAttE model, we select two typical datasets and different representative baseline models for experimental evaluation on several tasks such as link prediction, multi‐relation prediction and node classification. The experimental results show that the Multi‐RAttE model improves 8% over the state‐of‐the‐art model Composition‐based Multi‐Relational Graph Convolutional Networks (CompGCN) in terms of Hits@1 metric on the link prediction task on FB15k‐237 dataset; on the multi‐relation prediction task, the accuracy improves by 1.8% and 3.7% in the auc metric and F1 metric, respectively. The experimental results have proved that the Multi‐RAttE model can effectively perform the representation of multiple relations.

Clinical decision support system for hypertension medication based on knowledge graph

Computing knowledge graph (KG) embeddings is a technique to learn distributional representations for components of a knowledge graph while preserving structural information. The learned embeddings can be used in multiple downstream tasks such as question answering, information extraction, query expansion, semantic similarity, and information retrieval. Over the past years, multiple embedding techniques have been proposed based on different underlying assumptions. The most actively researched models are translation-based which treat relations as translation operations in a shared (or relation-specific) space. Interestingly, almost all KG embedding models treat each triple equally, regardless of the fact that the contribution of each triple to the global information content differs substantially. Many triples can be inferred from others, while some triples are the foundational (basis) statements that constitute a knowledge graph, thereby supporting other triples. Hence, in order to learn a suitable embedding model, each triple should be treated differently with respect to its information content. Here, we propose a data-driven approach to measure the information content of each triple with respect to the whole knowledge graph by using rule mining and PageRank. We show how to compute triple-specific weights to improve the performance of three KG embedding models (TransE, TransR and HolE). Link prediction tasks on two standard datasets, FB15K and WN18, show the effectiveness of our weighted KG embedding model over other more complex models. In fact, for FB15K our TransE-RW embeddings model outperforms models such as TransE, TransM, TransH, and TransR by at least 12.98% for measuring the Mean Rank and at least 1.45% for HIT@10. Our HolE-RW model also outperforms HolE and ComplEx by at least 14.3% for MRR and about 30.4% for HIT@1 on FB15K. Finally, TransR-RW show an improvement over TransR by 3.90% for Mean Rank and 0.87% for HIT@10.

BackgroundIn Literature-based Discovery (LBD), Swanson’s original ABC model brought together isolated public knowledge statements and assembled them to infer putative hypotheses via logical connections. Modern LBD studies that scale up this approach through automation typically rely on a simple entity-based knowledge graph with co-occurrences and/or semantic triples as basic building blocks. However, our analysis of a knowledge graph constructed for a recent LBD system reveals limitations arising from such pairwise representations, which further negatively impact knowledge inference. Using LBD as the context and motivation in this work, we explore limitations of using pairwise relationships only as knowledge representation in knowledge graphs, and we identify impacts of these limitations on knowledge inference. We argue that enhanced knowledge representation is beneficial for biological knowledge representation in general, as well as for both the quality and the specificity of hypotheses proposed with LBD.ResultsBased on a systematic analysis of one co-occurrence-based LBD system focusing on Alzheimer’s Disease, we identify 7 types of limitations arising from the exclusive use of pairwise relationships in a standard knowledge graph—including the need to capture more than two entities interacting together in a single event—and 3 types of negative impacts on knowledge inferred with the graph—Experimentally infeasible hypotheses, Literature-inconsistent hypotheses, and Oversimplified hypotheses explanations. We also present an indicative distribution of different types of relationships. Pairwise relationships are an essential component in representation frameworks for knowledge discovery. However, only 20% of discoveries are perfectly represented with pairwise relationships alone. 73% require a combination of pairwise relationships and nested relationships. The remaining 7% are represented with pairwise relationships, nested relationships, and hypergraphs.ConclusionWe argue that the standard entity pair-based knowledge graph, while essential for representing basic binary relations, results in important limitations for comprehensive biological knowledge representation and impacts downstream tasks such as proposing meaningful discoveries in LBD. These limitations can be mitigated by integrating more semantically complex knowledge representation strategies, including capturing collective interactions and allowing for nested entities. The use of more sophisticated knowledge representation will benefit biological fields with more expressive knowledge graphs. Downstream tasks, such as LBD, can benefit from richer representations as well, allowing for generation of implicit knowledge discoveries and explanations for disease diagnosis, treatment, and mechanism that are more biologically meaningful.

Knowledge graphs (KGs) are widely used in many real-world applications, such as information retrieval, question answering system, and personal recommendation. However, most KGs are suffering from the incompleteness problem. To deal with the task of link prediction, previous knowledge graph embedding methods require numerous reference instances for each relation. It is worth noting that most relations in KGs have only a few reference instances available. Existing works for few-shot link prediction evaluate the authenticity of triplets from a single relation perspective. In this paper, we propose Hybrid Matching Network (HMNet) for few-shot link prediction, evaluating triplets from entity and relation two perspectives. At the entity-aware matching network, HMNet uses attentive inductive embedding layer to aggregate entity features and relation-aware topology, and then provides entity-aware score to implement first perspective evaluation. At the relation-aware matching network, HMNet integrates feature attention mechanism to implement relation perspective evaluation. Experiments on two public datasets indicate that HMNet achieves promising performance in few-shot link prediction.

The knowledge graph with relational abundant information has been widely used as the basic data support for the retrieval platforms. Image and text descriptions added to the knowledge graph enrich the node information, which accounts for the advantage of the multi-modal knowledge graph. In the field of cross-modal retrieval platforms, multi-modal knowledge graphs can help to improve retrieval accuracy and efficiency because of the abundant relational information provided by knowledge graphs. The representation learning method is significant to the application of multi-modal knowledge graphs. This paper proposes a distributed collaborative vector retrieval platform (DCRL-KG) using the multimodal knowledge graph VisualSem as the foundation to achieve efficient and high-precision multimodal data retrieval. Firstly, use distributed technology to classify and store the data in the knowledge graph to improve retrieval efficiency. Secondly, this paper uses BabelNet to expand the knowledge graph through multiple filtering processes and increase the diversification of information. Finally, this paper builds a variety of retrieval models to achieve the fusion of retrieval results through linear combination methods to achieve high-precision language retrieval and image retrieval. The paper uses sentence retrieval and image retrieval experiments to prove that the platform can optimize the storage structure of the multi-modal knowledge graph and have good performance in multi-modal space.

The representation learning of knowledge graph refers to embedding entities and relations in knowledge graph into a low-dimensional dense vector space. Existing knowledge graph embedding models mostly chose Euclidean Space as their vector space and consider each fact triple in knowledge graph independently. However, Euclidean Space is unable to represent knowledge effectively due to its strict constraints and mathematical expression. Besides, entities in knowledge graph are not isolated, while these models ignore the association between entities. To solve the problems above, we proposed a text-enhanced knowledge graph representation learning model in Hyperbolic Space. We utilize rich semantic information of entity description by using Transformer Encoder to enhance the ability of knowledge representation. Besides, we embed entities and relations into Hyperbolic Space, which can better capture hierarchical information of the knowledge graph. Experiments on benchmark dataset show that our method achieves better performance compared with other state-of-art methods.KeywordsKnowledge graphRepresentation learningEmbedding modelHyperbolic space