Knowledge Interaction and Diffusion Augmentation for Knowledge Graph Recommendation

Knowledge Graph (KG), which commonly consists of fruitful connected facts about items, presents an unprecedented opportunity to alleviate the sparsity problem in recommender system. However, existing KG based recommendation methods mainly rely on handcrafted meta-path features or simple triple-level entity embedding, which cannot automatically capture entities' long-term relational dependencies for the recommendation. In this paper, a two-channel neural interaction method named Knowledge Graph enhanced Neural Collaborative Filtering with Residual Recurrent Network (KGNCF-RRN) is proposed, which leverages both long-term relational dependencies KG context and user-item interaction for recommendation. (1) For the KG context interaction channel, we propose a Residual Recurrent Network (RRN) to construct context-based path embedding, which incorporates residual learning into traditional recurrent neural networks (RNNs) to efficiently encode the long-term relational dependencies of KG. The self-attention network is then applied to the path embedding to capture the polysemy of various user interaction behaviours. (2) For the user-item interaction channel, the user and item embeddings are fed into a newly designed two-dimensional interaction map. (3) Finally, above the two-channel neural interaction matrix, we employ a convolutional neural network to learn complex correlations between user and item. Extensive experimental results on three benchmark datasets show that our proposed approach outperforms existing state-of-the-art approaches for knowledge graph based recommendation.

Unveiling the power of knowledge graph embedding in knowledge aware deep recommender systems for e-commerce: A comparative study

Recent studies have shown that the problems of cold processing and data sparsity in recommender systems can be well alleviated with the help of knowledge graph information. Meanwhile, since the information in recommender systems can be represented by graph data structure, graph neural network has great advantages in recommendatory system. Therefore, graph neural network can better mine the deep relationship between objects in the recommender systems which combines with information from knowledge graph. This paper reviews the application of graph neural network in knowledge graph aided recommendation systems. Specifically, this paper firstly sorts out the related knowledge of knowledge graph, recommendation system and graph neural network, including the cold processing and data sparsity, the structure of knowledge graph, and the basic modules of graph neural network. This paper also summarizes the methods such as KGAT and CKAN used in knowledge graph aided recommendation systems in recent years, and gives some discussion on the development of this field.

Collaborative filtering (CF) usually suffers from data sparsity and cold starts. Knowledge graphs (KGs) are widely used to improve recommendation performance. To verify that knowledge graphs can further alleviate the above problems, this paper proposes an end-to-end framework that uses attentive knowledge graph perceptual propagation for recommendations (AKGP). This framework uses a knowledge graph as a source of auxiliary information to extract user–item interaction information and build a sub-knowledge base. The fusion of structural and contextual information is used to construct fine-grained knowledge graphs via knowledge graph embedding methods and to generate initial embedding representations. Through multi-layer propagation, the structured information and historical preference information are embedded into a unified vector space, and the potential user–item vector representation is expanded. This article used a knowledge perception attention module to achieve feature representation, and finally, the model was optimized using the stratified sampling joint learning method. Compared with the baseline model using MovieLens-1M, Last-FM, Book-Crossing and other data sets, the experimental results demonstrate that the model outperforms state-of-the-art KG-based recommendation methods, and the shortcomings of the existing model are improved. The model was applied to product design data and historical maintenance records provided by an automotive parts manufacturing company. The predictions of the recommended system are matched to the product requirements and possible failure records. This helped reduce costs and increase productivity, helping the company to quickly determine the cause of failures and reduce unplanned downtime.

Although there are a small number of work to conduct patent research by building knowledge graph, but without constructing patent knowledge graph using patent documents and combining latest natural language processing methods to mine hidden rich semantic relationships in existing patents and predict new possible patents. In this paper, we propose a new patent vacancy prediction approach named PatentMiner to mine rich semantic knowledge and predict new potential patents based on knowledge graph (KG) and graph attention mechanism. Firstly, patent knowledge graph over time (e.g. year) is constructed by carrying out named entity recognition and relation extrac-tion from patent documents. Secondly, Common Neighbor Method (CNM), Graph Attention Networks (GAT) and Context-enhanced Graph Attention Networks (CGAT) are proposed to perform link prediction in the constructed knowledge graph to dig out the potential triples. Finally, patents are defined on the knowledge graph by means of co-occurrence relationship, that is, each patent is represented as a fully connected subgraph containing all its entities and co-occurrence relationships of the patent in the knowledge graph; Furthermore, we propose a new patent prediction task which predicts a fully connected subgraph with newly added prediction links as a new pa-tent. The experimental results demonstrate that our proposed patent predic-tion approach can correctly predict new patents and Context-enhanced Graph Attention Networks is much better than the baseline. Meanwhile, our proposed patent vacancy prediction task still has significant room to im-prove.

Inductive Knowledge Graph Completion (KGC) aims to infer missing facts between newly emerged entities within knowledge graphs (KGs), posing a significant challenge. While recent studies have shown promising results in inferring such entities through knowledge subgraph reasoning, they suffer from (i) the semantic inconsistencies of similar relations, and (ii) noisy interactions inherent in KGs due to the presence of unconvincing knowledge for emerging entities. To address these challenges, we propose a Semantic Structure-aware Denoising Network (S2DN) for inductive KGC. Our goal is to learn adaptable general semantics and reliable structures to distill consistent semantic knowledge while preserving reliable interactions within KGs. Specifically, we introduce a semantic smoothing module over the enclosing subgraphs to retain the universal semantic knowledge of relations. We incorporate a structure refining module to filter out unreliable interactions and offer additional knowledge, retaining robust structure surrounding target links. Extensive experiments conducted on three benchmark KGs demonstrate that S2DN surpasses the performance of state-of-the-art models. These results demonstrate the effectiveness of S2DN in preserving semantic consistency and enhancing the robustness of filtering out unreliable interactions in contaminated KGs.

Recommender systems (RSs) have become important tools in the modern lifestyle; they have been integrated into all domains, spanning from entertainment (music, films, etc.) to more sensitive fields such as security and health care. Their success does not mean that they are ideal or flawless; quite the opposite, RSs suffer from plenty of drawbacks and challenges that need to be resolved. Data sparsity is a common problem in recommender systems; it has been of top interest among researchers. Numerous approaches from different perspectives have been proposed to mitigate it, including knowledge graphs (KGs), which quickly gained popularity due to the rich semantics residing in their components. In this paper, we will conduct a systematic literature review to explore and analyze in depth the existing contributions. Our work focuses on investigating the effectiveness of KGs to mitigate the data sparsity in RSs by discovering the techniques used, understanding how KGs are exploited, and what type of knowledge is extracted from them, besides studying their evaluation measures and discussing future directions that can strengthen the application of KGs to mitigate data sparsity in recommender systems.

Recent progress in personalized recommendation has shown great potential in exploiting structure information provided by a knowledge graph (KG). As a heterogeneous information network, KG contains rich semantic relatedness among entities, which contributes to addressing notorious issues such as data sparsity and cold start. State-of-the-art KG-based recommendation approaches try to propagate information along KG links to encode long-range connectivities into hidden representations. However, most of them only model the user or item representation independently, lacking a focus on user-item interaction. To this end, we propose the Interactive Knowledge Graph Attention Network (IKGAT), which directly models user-item interaction and high-order structure information within KG. For the user representation, following an interactive attention mechanism, we use the item to attend over the user's neighbors and then propagate their information to update the representation. Such a process is extended to multi-hops away to obtain richer neighborhood information. Similarly, the item representation is updated under the supervision of the user. With that design, IKGAT can capture collaborative signals and user preferences effectively. Experiment results on three public datasets show that IKGAT consistently outperforms the state-of-the-art approaches, especially when the dataset is sparse.

Arabic Knowledge Graph Construction: A close look in the present and into the future

Federated recommender systems have distinct advantages in terms of privacy protection over traditional recommender systems that are centralized at a data center. However, previous work on federated recommender systems does not fully consider the limitations of storage, RAM, energy and communication bandwidth in a mobile environment. The scales of the models proposed are too large to be easily run on mobile devices. And existing federated recommender systems need to fine-tune recommendation models on each device, making it hard to effectively exploit collaborative filtering information among users/devices. Our goal in this paper is to design a novel federated learning framework for rating prediction (RP) for mobile environments. We introduce a federated matrix factorization (MF) framework, named meta matrix factorization (MetaMF). Given a user, we first obtain a collaborative vector by collecting useful information with a collaborative memory module. Then, we employ a meta recommender module to generate private item embeddings and a RP model based on the collaborative vector in the server. To address the challenge of generating a large number of high-dimensional item embeddings, we devise a rise-dimensional generation strategy that first generates a low-dimensional item embedding matrix and a rise-dimensional matrix, and then multiply them to obtain high-dimensional embeddings. We use the generated model to produce private RPs for the given user on her device. MetaMF shows a high capacity even with a small RP model, which can adapt to the limitations of a mobile environment. We conduct extensive experiments on four benchmark datasets to compare MetaMF with existing MF methods and find that MetaMF can achieve competitive performance. Moreover, we find MetaMF achieves higher RP performance over existing federated methods by better exploiting collaborative filtering among users/devices.

Knowledge graphs encode semantic knowledge that can be exploited to enhance different data-driven tasks, e.g., query answering, data mining, ranking or recommendation. However, knowledge graphs may be incomplete, and relevant relations may be not included in the graph, affecting accuracy of these data-driven tasks. We tackle the problem of relation discovery in a knowledge graph, and devise $$\mathcal {KOI}$$KOI, a semantic based approach able to discover relations in portions of knowledge graphs that comprise similar entities. $$\mathcal {KOI}$$KOI exploits both datatype and object properties to compute the similarity among entities, i.e., two entities are similar if their datatype and object properties have similar values. $$\mathcal {KOI}$$KOI implements graph partitioning techniques that exploit similarity values to discover relations from knowledge graph partitions. We conduct an experimental study on a knowledge graph of TED talks with state-of-the-art similarity measures and graph partitioning techniques. Our observed results suggest that $$\mathcal {KOI}$$KOI is able to discover missing edges between related TED talks that cannot be discovered by state-of-the-art approaches. These results reveal that combining semantics encoded both in the similarity measures and in the knowledge graph structure, has a positive impact on the relation discovery problem.

Integrating the Knowledge Graphs (KGs) into recommendation systems enhances personalization and accuracy. However, the long-tail distribution of knowledge graphs often leads to data sparsity, which limits the effectiveness in practical applications. To address this challenge, this study proposes a knowledge-aware recommendation algorithm framework that incorporates multi-level contrastive learning. This framework enhances the Collaborative Knowledge Graph (CKG) through a random edge dropout method, which constructs feature representations at three levels: user-user interactions, item-item interactions and user-item interactions. A dynamic attention mechanism is employed in the Graph Attention Networks (GAT) for modeling the KG. Combined with the nonlinear transformation and Momentum Contrast (Moco) strategy for contrastive learning, it can effectively extract high-quality feature information. Additionally, multi-level contrastive learning, as an auxiliary self-supervised task, is jointly trained with the primary supervised task, which further enhances recommendation performance. Experimental results on the MovieLens and Amazon-books datasets demonstrate that this framework effectively improves the performance of knowledge graph-based recommendations, addresses the issue of data sparsity, and outperforms other baseline models across multiple evaluation metrics.

Nowadays, side-information is widely used to rein-force the user-item interaction and helps to handle the sparsity issue and cold start problem of conventional recommendation algorithms. Due to the overlook of the relationship between items and entities and the higher-order connectivity information, most existing approaches are hard to get users' deep preferences. In this paper, we propose KGANCL, a Knowledge-aware Graph Attention Network with Distributed & Cross Learning. It focuses on using different forms of the knowledge graph to strengthen both users' and items' embedding representations, respectively. Firstly, Graph Attention Network is adopted for user embedding learning, which can give different importance scores to different neighbors, and a user-item KG graph is used to integrate adjacent information to enhance the representation. Secondly, a cross module is used for item embedding learning, which shares the high-order interaction between the recommender system and the knowledge graph. We also use the idea of distributed processing for embeddings in different entities to improve the learning efficiency. Experimental results demonstrate that KGANCL can provide better recommendations compared with the state-of-the-art baseline models. Our model can also maintain superior prediction accuracy even in little-known interaction scenarios.

A review of graph neural networks and pretrained language models for knowledge graph reasoning

BackgroundIntensive Care Unit (ICU) readmissions represent both a health risk for patients,with increased mortality rates and overall health deterioration, and a financial burden for healthcare facilities. As healthcare became more data-driven with the introduction of Electronic Health Records (EHR), machine learning methods have been applied to predict ICU readmission risk. However, these methods disregard the meaning and relationships of data objects and work blindly over clinical data without taking into account scientific knowledge and context. Ontologies and Knowledge Graphs can help bridge this gap between data and scientific context, as they are computational artefacts that represent the entities of a domain and their relationships to each other in a formalized way.Methods and resultsWe have developed an approach that enriches EHR data with semantic annotations to ontologies to build a Knowledge Graph. A patient’s ICU stay is represented by Knowledge Graph embeddings in a contextualized manner, which are used by machine learning models to predict 30-days ICU readmissions. This approach is based on several contributions: (1) an enrichment of the MIMIC-III dataset with patient-oriented annotations to various biomedical ontologies; (2) a Knowledge Graph that defines patient data with biomedical ontologies; (3) a predictive model of ICU readmission risk that uses Knowledge Graph embeddings; (4) a variant of the predictive model that targets different time points during an ICU stay. Our predictive approaches outperformed both a baseline and state-of-the-art works achieving a mean Area Under the Receiver Operating Characteristic Curve of 0.827 and an Area Under the Precision-Recall Curve of 0.691. The application of this novel approach to help clinicians decide whether a patient can be discharged has the potential to prevent the readmission of 40% of Intensive Care Unit patients, without unnecessarily prolonging the stay of those who would not require it.ConclusionThe coupling of semantic annotation and Knowledge Graph embeddings affords two clear advantages: they consider scientific context and they are able to build representations of EHR information of different types in a common format. This work demonstrates the potential for impact that integrating ontologies and Knowledge Graphs into clinical machine learning applications can have.