Joint Knowledge Pruning and Recurrent Graph Convolution for News Recommendation

IoT devices generate substantial amounts of measurements and use many different formats to store this data. In order to make IoT devices interoperable, a number of ontologies have been developed, such as the Smart Applications REFerence (SAREF) ontology, to create knowledge graphs that can represent all kinds of measurements from IoT devices. We call the resulting knowledge graphs: IoT measurement knowledge graphs. In this thesis, we set out to investigate what differentiates IoT measurement knowledge graphs from regular knowledge graphs. In the first half, we describe in detail how to create IoT measurement knowledge graphs from real-world measurement data. In the second half, we investigate what happens when we use the graphs in different scenarios, with different methods and applications. In order to generate an IoT measurement knowledge graph, we create a mapping to transform the measurement data coming from IoT devices. We explore multiple ways to create this mapping and evaluate the resulting IoT measurement knowledge graph with competency questions. Using our mapping, we create OfficeGraph, by transforming IoT measurement data recorded over a year from 444 IoT devices located in an office building. This IoT measurement knowledge graph is validated by answering competency questions created with the support of the building owners, showing that we can answer questions we were not able to answer without it. Besides the entities, literals, and relations in knowledge graphs, the combination of these, the context of entities, provides additional knowledge. Therefore, if we want to use entities in knowledge graphs to train machine learning models, it would be a waste to take only the entities from the graph, because this would leave out information. Representation models, such as RDF2Vec and Graph Convolutional Networks (GCNs), can be used to learn (embedding) representations for entities that take context into account, creating a representation based on which relations, entities, and literals occur near the entity in the graph. We learn over IoT measurement knowledge graphs using multiple representation models and experiment with the effect of making more knowledge available to the representation models through semantic enrichment. This is done by making implicit information, such as (e.g.) consecutive measurements, explicitly available, by adding a relation between two measurements. Results show that the semantic enrichment has a positive effect on the learnability, based on the evaluation score, of the entity representations. Due to the dynamic nature of IoT measurement knowledge graphs, it is to be expected that new measurements are made after the initial IoT measurement knowledge graphs are created. Therefore, we investigate the possibility of estimating representations for new entities that are based on the existing entity representations. We introduce the embedding estimation method, which used the numerical attributes of entities, the measurement values, to find entities similar to the new entity. Then by averaging the embedding representations of those entities, it creates a new, estimated, embedding for the new entity. We perform experiments to test the embedding estimation method, and show that there is a trade-off between the amount of time saved by not re-training the entire pipeline, but also a decrease in accuracy, with the estimated embeddings. This thesis offers a focused research into IoT measurement knowledge graph. We provide insight into how IoT measurement knowledge graph can be created and how they differ from regular knowledge graphs, specifically in being numerical, dynamic, and shallow knowledge graphs. We suggest and validate remedies to the shallowness of IoT measurement knowledge graphs, through the semantic enrichments. Furthermore we show how the numerical aspect can be a strength, through the embedding estimation method, which can help with the dynamic nature of IoT measurement knowledge graph.

News recommendation is very important to help users find interested news and alleviate information overload. Different users usually have different interests and the same user may have various interests. Thus, different users may click the same news article with attention on different aspects. In this paper, we propose a neural news recommendation model with personalized attention (NPA). The core of our approach is a news representation model and a user representation model. In the news representation model we use a CNN network to learn hidden representations of news articles based on their titles. In the user representation model we learn the representations of users based on the representations of their clicked news articles. Since different words and different news articles may have different informativeness for representing news and users, we propose to apply both word- and news-level attention mechanism to help our model attend to important words and news articles. In addition, the same news article and the same word may have different informativeness for different users. Thus, we propose a personalized attention network which exploits the embedding of user ID to generate the query vector for the word- and news-level attentions. Extensive experiments are conducted on a real-world news recommendation dataset collected from MSN news, and the results validate the effectiveness of our approach on news recommendation.

To address the sparsity and cold-start problem of collaborative filtering, researchers usually make use of side information, such as social networks or item attributes, to improve the performance of recommendation. In this article, we consider the knowledge graph (KG) as the source of side information. To address the limitations of existing embedding-based and path-based methods for KG-aware recommendation, we propose RippleNet , an end-to-end framework that naturally incorporates the KG into recommender systems. RippleNet has two versions: (1) The outward propagation version, which is analogous to the actual ripples on water, stimulates the propagation of user preferences over the set of knowledge entities by automatically and iteratively extending a user’s potential interests along links in the KG. The multiple “ripples” activated by a user’s historically clicked items are thus superposed to form the preference distribution of the user with respect to a candidate item. (2) The inward aggregation version aggregates and incorporates the neighborhood information biasedly when computing the representation of a given entity. The neighborhood can be extended to multiple hops away to model high-order proximity and capture users’ long-distance interests. In addition, we intuitively demonstrate how a KG assists with recommender systems in RippleNet, and we also find that RippleNet provides a new perspective of explainability for the recommended results in terms of the KG. Through extensive experiments on real-world datasets, we demonstrate that both versions of RippleNet achieve substantial gains in a variety of scenarios, including movie, book, and news recommendations, over several state-of-the-art baselines.

Finding similar entities among knowledge graphs is an essential research problem for knowledge integration and knowledge graph connection. This paper aims at finding semantically similar entities between two knowledge graphs. This can help end users and search agents more effectively and easily access pertinent information across knowledge graphs. Given a query entity in one knowledge graph, the proposed approach tries to find the most similar entity in another knowledge graph. The main idea is to leverage graph embedding, clustering, regression and sentence embedding. In this approach, RDF2Vec has been employed to generate vector representations of all entities of the second knowledge graph and then the vectors have been clustered based on cosine similarity using K medoids algorithm. Then, an artificial neural network with multilayer perception topology has been used as a regression model to predict the corresponding vector in the second knowledge graph for a given vector from the first knowledge graph. After determining the cluster of the predicated vector, the entities of the detected cluster are ranked through sentence-BERT method and finally the entity with the highest rank is chosen as the most similar one. To evaluate the proposed approach, experiments have been conducted on real-world knowledge graphs. The experimental results demonstrate the effectiveness of the proposed approach.

Interaction-knowledge semantic alignment for recommendation

KCRec: Knowledge-aware representation Graph Convolutional Network for Recommendation

Graph convolutional neural networks have difficulties in model training takes a lot of time and lack of universality in meta-path design when processing knowledge graphs. To solve these issues, this paper proposes an efficient recommendation algorithm integrating knowledge graphs with graph convolutional networks(EKG-GCN). The algorithm is mainly based on the graph convolution algorithm. Firstly, the relationship between users and entities is standardized and scored, then a new neighborhood aggregation method is proposed. Some nodes in the knowledge graph are set as central nodes. A distance-influence function is set up according to the distance from other nodes to the central node, and the feature extraction of information in the knowledge graph is completed according to the distance-influence function. Finally, the knowledge graph features are used as an auxiliary to complete the recommendation task. In the experiment, three data sets were selected to compare with other advanced models, and it was found that the model is in an advantageous position in terms of training time and recommendation effect, and has a certain degree of universality.

Knowledge graph (KG) enhanced recommendation has demonstrated improved performance in the recommendation system (RecSys) and attracted considerable research interest. Recently the literature has adopted neural graph networks (GNNs) on the collaborative knowledge graph and built an end-to-end KG-enhanced RecSys. However, the majority of these approaches have three limitations: (1) treat the collaborative knowledge graph as a homogeneous graph and overlook the highly heterogeneous relationships among items, (2) lack of design to explicitly leverage the rich side information, and (3) overlook the rich knowledge in user preference.To fill this gap, in this paper, we explore the rich, heterogeneous relationship among items and propose a new KG-enhanced recommendation model called Collaborative Meta-Knowledge Enhanced Recommender System (MetaKRec). In particular, we focus on modeling the rich, heterogeneous semantic relationships among items and construct several collaborative Meta-KGs to explicitly depict the relatedness of the items under the guidance of meta-knowledge. In addition to the knowledge obtained from KG, we leverage user knowledge that extracts from user preference to construct the Meta-KGs. The constructed Meta-KGs can capture the knowledge from both the knowledge graph and user preference. Furthermore. we utilize a light convolution encoder to recursively integrate the item relationship in each collaborative Meta-KGs. This scheme allows us to explicitly gather the heterogeneous semantic relationships among items and encode them into the representations of items. In addition, we propose channel attention to fuse the item and user representations from different Meta-KGs. Extensive experiments are conducted on four real-world benchmark datasets, demonstrating significant gains over the state-of-the-art baselines on both regular and cold-start recommendation settings.

Retrieval-augmented generation (RAG) enhances the reasoning capabilities of large language models (LLMs) by incorporating external knowledge. Among available sources, knowledge graphs (KGs) offer a structured and reliable foundation for factual information, making them increasingly popular in efforts to improve reasoning faithfulness in RAG. Most existing KG-based RAG methods rely on LLMs to extract knowledge from KGs. However, these approaches often require costly fine-tuning and struggle to navigate deep graph structures, limiting their effectiveness in multi-hop reasoning tasks. To address these challenges, we propose Stepwise Contrastive Reasoning (SCR), a lightweight framework that integrates graph structure and textual context for efficient and interpretable RAG over KGs. SCR combines relational message passing layers to encode KG entities with a Transformer encoder for processing question text. It decomposes reasoning into a series of alignment steps. At each step, SCR compares the current topic entity and its neighbors with the question representation, selecting the most relevant entity as the next topic entity. The question is then updated with this entity's textual description. This process continues until the selected entity no longer changes, indicating that the answer entity has been reached. Through stepwise alignment, SCR enables compact models to perform faithful and interpretable reasoning over large-scale KGs. Extensive experiments on several widely used KGQA benchmarks demonstrate that SCR not only achieves state-of-the-art performance but also effectively boosts the capabilities of smaller language models to match those of LLMs.

Recent progress in personalized recommendation has shown great potential in exploiting structure information provided by a knowledge graph (KG). As a heterogeneous information network, KG contains rich semantic relatedness among entities, which contributes to addressing notorious issues such as data sparsity and cold start. State-of-the-art KG-based recommendation approaches try to propagate information along KG links to encode long-range connectivities into hidden representations. However, most of them only model the user or item representation independently, lacking a focus on user-item interaction. To this end, we propose the Interactive Knowledge Graph Attention Network (IKGAT), which directly models user-item interaction and high-order structure information within KG. For the user representation, following an interactive attention mechanism, we use the item to attend over the user's neighbors and then propagate their information to update the representation. Such a process is extended to multi-hops away to obtain richer neighborhood information. Similarly, the item representation is updated under the supervision of the user. With that design, IKGAT can capture collaborative signals and user preferences effectively. Experiment results on three public datasets show that IKGAT consistently outperforms the state-of-the-art approaches, especially when the dataset is sparse.

Most of the online queries target entities and the type of an entity is a key piece of information. Entity type helps us to understand what an entity is and how it relates to other entities mentioned in a document. Search engine result pages (SERPs) often surface facts and entity type information from a background Knowledge Graph (KG) in response to queries that carry a semantic information need. In a KG, an entity usually holds multiple type properties. For example, popular types attached to the entity `Donald Trump' via rdfs:type statements might be Person, Businessman, and Leader. However, other types of this entity, e.g., Solicitor, Restaurateur, and Writer might also be interesting to some users. Unpopular entity types can be useful for tail queries like, for example, `Is Donald Trump an American Television Producer' or `Is Donald Trump an American Television Actor'. It is then important to, given an entity in a KG, rank entity types attached to the entity by relevance to a certain user and information need as not always the most popular type is the most informative within a textual context. In this paper we address the entity type ranking problem by means of KG embedding models. In our work, we show that entity type ranking can be seen as a special case of the KG completion problem. Embeddings can be learned from the structural, probabilistic and contextual description information of the entities. We propose and evaluate our methods to find the most relevant entity type based on collection statistics and on the graph structure interconnecting entities and types. Experimental results show that our proposed approaches outperform the state-of-the-art type ranking models while, at the same time, being more efficient and scalable. Our approach focuses on the task of ranking a set of types associated to an entity in a background knowledge graph to select the most relevant types.

Neuro-Symbolic Artificial Intelligence (AI) focuses on integrating symbolic and sub-symbolic systems to enhance the performance and explainability of predictive models. Symbolic and sub-symbolic approaches differ fundamentally in how they represent data and make use of data features to reach conclusions. Neuro-symbolic systems have recently received significant attention in the scientific community. However, despite efforts in neural-symbolic integration, symbolic processing can still be better exploited, mainly when these hybrid approaches are defined on top of knowledge graphs. This work is built on the statement that knowledge graphs can naturally represent the convergence between data and their contextual meaning (i.e., knowledge). We propose a hybrid system that resorts to symbolic reasoning, expressed as a deductive database, to augment the contextual meaning of entities in a knowledge graph, thus, improving the performance of link prediction implemented using knowledge graph embedding (KGE) models. An entity context is defined as the ego network of the entity in a knowledge graph. Given a link prediction task, the proposed approach deduces new RDF triples in the ego networks of the entities corresponding to the heads and tails of the prediction task on the knowledge graph (KG). Since knowledge graphs may be incomplete and sparse, the facts deduced by the symbolic system not only reduce sparsity but also make explicit meaningful relations among the entities that compose an entity ego network. As a proof of concept, our approach is applied over a KG for lung cancer to predict treatment effectiveness. The empirical results put the deduction power of deductive databases into perspective. They indicate that making explicit deduced relationships in the ego networks empowers all the studied KGE models to generate more accurate links.

Knowledge Graph (KG) contains rich real-world auxiliary information, which can be leveraged to improve the performance of recommender systems. Nevertheless, existing recommender systems usually sample and aggregate neighbor entities and relations that link to target items to enrich the representations of items or users, whereas ignoring combinatorial features among different neighbor entities and relations. To resolve the problem mentioned above, we propose an end-to-end Knowledge Graph based Wide & Deep (KGWD) framework to leverage combinatorial features effectively. At the wide level, KGWD introduces a novel Triplet Compressed Interaction Network (TriCIN) to generate high-order combinatorial features among different triplets associated with the target item automatically. At the deep level, KGWD discovers users’ potential long-distance preferences by mining multi-hop neighbor information over the KG. We conduct experiments on three real-world datasets, i.e., Yelp2018, Last-FM, and Amazon-book, to evaluate the performance of KGWD. Experimental results demonstrate that KGWD outperforms state-of-the-art schemes significantly. Specifically, in all three datasets, KGWD improves the F1-score by more than 5% over the state-of-the-art.

Knowledge graph conflict resolution is a method to solve the knowledge conflict problem in constructing knowledge graphs. The existing methods ignore the time attributes of facts and the dynamic changes of the relationships between entities in knowledge graphs, which is liable to cause high error rates in dynamic knowledge graph construction. In this article, we propose a knowledge graph conflict resolution method, knowledge graph evolution algorithm based on deep learning (Kgedl), which can resolve facts confliction with high precision by combing time attributes, semantic embedding representations, and graph structure features. Kgedl first trains the semantic embedding vector through the relationships between entities. Then, the path embedding vector is trained from the graph structures of knowledge graphs, and the time attributes of entities are combined with the semantic and path embedding vectors. Finally, Kgedl uses a recurrent neural network to make the inconsistent facts appear in the dynamic evolution of the knowledge graph consistent. A large number of experiments on real datasets show that Kgedl outperforms the state-of-the-art methods. Especially, Kgedl achieves 23% higher performance than the classical method numerical Probabilistic Soft Logic (nPSL).in the metric HITS@10. Also, extensive experiments verified that our proposal possess better robustness by adding noise data.

In knowledge graph-based recommendation systems, modeling based on item features is a crucial direction. To further enhance the effectiveness of recommendation, we can study common similarities among items based on their shared attributes. The attribute information of items can assist recommendation models in understanding the structure and semantics of items. Existing models face the challenge of not considering user intent when interpreting item attribute information. In this paper, we propose the KGCIN model, which explores user preferences on the knowledge graph. By integrating the structure and semantic information of the knowledge graph and increasing the receptive field factor of entities, the KGCIN model ultimately obtains item embeddings with neighborhood information aggregation results that incorporate user preference understanding. On the other hand, user representations are obtained through information aggregation in the user intent graph. Finally, personalized recommendations for users are made by aggregating representations of user preferences and item information. Experiments in this paper are conducted on sparse dataset Last.FM and dense dataset MOOPer, respectively, validating that the proposed model outperforms other methods based on knowledge graph information fusion or user preference understanding on different datasets. According to experimental results, the model shows improvements in recommendation metrics such as precision and recall.