Neural Collaborative Recommendation with Knowledge Graph

Unveiling the power of knowledge graph embedding in knowledge aware deep recommender systems for e-commerce: A comparative study

Recommender systems (RSs) have become important tools in the modern lifestyle; they have been integrated into all domains, spanning from entertainment (music, films, etc.) to more sensitive fields such as security and health care. Their success does not mean that they are ideal or flawless; quite the opposite, RSs suffer from plenty of drawbacks and challenges that need to be resolved. Data sparsity is a common problem in recommender systems; it has been of top interest among researchers. Numerous approaches from different perspectives have been proposed to mitigate it, including knowledge graphs (KGs), which quickly gained popularity due to the rich semantics residing in their components. In this paper, we will conduct a systematic literature review to explore and analyze in depth the existing contributions. Our work focuses on investigating the effectiveness of KGs to mitigate the data sparsity in RSs by discovering the techniques used, understanding how KGs are exploited, and what type of knowledge is extracted from them, besides studying their evaluation measures and discussing future directions that can strengthen the application of KGs to mitigate data sparsity in recommender systems.

Offshore sites show greater potential for wind energy utilization than most onshore sites. When planning an offshore wind power farm, the speed of offshore wind is used to estimate various operation parameters, such as the power output, extreme wind load, and fatigue load. Accurate speed prediction is crucial to the running of wind power farms and the security of smart grids. Unlike onshore wind, offshore wind has the characteristics of random, intermittent, and chaotic, which will cause the time series of wind speeds to have strong nonlinearity. It will bring greater difficulties to offshore wind speed predictions, which traditional recurrent neural networks cannot deal with for lacking in long-term dependency. An offshore wind speed prediction method is proposed by using a clockwork recurrent network (CWRNN). In a CWRNN model, the hidden layer is subdivided into several parts and each part is allocated a different clock speed. Under the mechanism, the long-term dependency of the recurrent neural network can be easily addressed, which can furthermore effectively solve the problem of strong nonlinearity in offshore speed winds. The experiments are performed by using the actual data of two different offshore sites located in the Caribbean Sea and one onshore site located in the interior of the United States, to verify the performance of the model. The results show that the prediction model achieves significant accuracy improvement.

The development of the Internet-of-Things (IoT) security is comparatively slower than the pace of the IoT innovations. The seamless IoT network operates in an untrusted environment and is exposed to many malicious active attacks. As the process of identifying the modulation format of signals is corrupted by noise and fading, automatic modulation classification (AMC) can be viewed as an effective approach to counter physical-layer threats for IoT as it can detect and identify the pilot jamming, deceptive jamming, and Sybil attacks. Nowadays, data-driven deep learning (DL) techniques, which are capable of extracting discriminative features and perform better robustness to channel and noise conditions, have drawn widespread attention. The deep residual network (ResNet) has a strong representative ability, which can learn latent information repeatedly from the received signals and improve the classification accuracy. Meanwhile, the gated recurrent unit (GRU), which is capable of exploiting temporal information of the received signal can expand the dimension of the signal features for satisfactory classification performance. Considering the advantages of the above networks, this article proposes a novel gated recurrent residual neural network (GrrNet) for feature-based AMC, where the amplitude and phase of the received signal are utilized as the inputs of GrrNet. In GrrNet, a ResNet extractor module is first designed to extract the highly representative features and then temporal information is obtained by the subsequent GRU module which is capable of processing the representative features with the arbitrary length for modulation classification. Moreover, extensive simulations are conducted to verify the classification performance and robustness of the proposed GrrNet and it is shown that GrrNet outperforms other recent DL-based AMC methods. Moreover, the influence of the network parameters, symbol length, and frequency offset on performance is also explored.

The local climate zone (LCZ) scheme was originally proposed to provide an interdisciplinary taxonomy for urban heat island (UHI) studies. In recent years, the scheme has also become a starting point for the development of higher-level products, as the LCZ classes can help provide a generalized understanding of urban structures and land uses. LCZ mapping can therefore theoretically aid in fostering a better understanding of spatio-temporal dynamics of cities on a global scale. However, reliable LCZ maps are not yet available globally. As a first step toward automatic LCZ mapping, this work focuses on LCZ-derived land cover classification, using multi-seasonal Sentinel-2 images. We propose a recurrent residual network (Re-ResNet) architecture that is capable of learning a joint spectral-spatial-temporal feature representation within a unitized framework. To this end, a residual convolutional neural network (ResNet) and a recurrent neural network (RNN) are combined into one end-to-end architecture. The ResNet is able to learn rich spectral-spatial feature representations from single-seasonal imagery, while the RNN can effectively analyze temporal dependencies of multi-seasonal imagery. Cross validations were carried out on a diverse dataset covering seven distinct European cities, and a quantitative analysis of the experimental results revealed that the combined use of the multi-temporal information and Re-ResNet results in an improvement of approximately 7 percent points in overall accuracy. The proposed framework has the potential to produce consistent-quality urban land cover and LCZ maps on a large scale, to support scientific progress in fields such as urban geography and urban climatology.

Multitype view of knowledge contrastive learning for recommendation

Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mechanism concept can help RNNs remember and forget previous information. Representing the hidden layers of an RNN with more expressive operations (i.e., tensor products) helps it learn a more complex relationship between the current input and the previous hidden layer information. These ideas can generally improve RNN performances. In this paper, we proposed a novel RNN architecture that combine the concepts of gating mechanism and the tensor product into a single model. By combining these two concepts into a single RNN, our proposed models learn long-term dependencies by modeling with gating units and obtain more expressive and direct interaction between input and hidden layers using a tensor product on 3-dimensional array (tensor) weight parameters. We use Long Short Term Memory (LSTM) RNN and Gated Recurrent Unit (GRU) RNN and combine them with a tensor product inside their formulations. Our proposed RNNs, which are called a Long-Short Term Memory Recurrent Neural Tensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor Network (GRURNTN), are made by combining the LSTM and GRU RNN models with the tensor product. We conducted experiments with our proposed models on word-level and character-level language modeling tasks and revealed that our proposed models significantly improved their performance compared to our baseline models.

Recommender systems are known to suffer from the popularity bias problem: popular items are recommended frequently, and nonpopular ones rarely, if at all. Prior studies focused on tackling this issue by increasing the number of recommended nonpopular (long-tail) items. However, these methods ignore the users’ personal popularity preferences and increase the exposure rate of the nonpopular items indiscriminately, which may hurt the user experience because different users have diverse interests in popularity. In this work, we propose a novel debias framework with knowledge graph (AWING), which adaptively alleviates popularity bias from the users’ perspective. Concretely, we explore fine-grained preferences (including popularity preference) behind a user-item interaction by using the heterogeneous graph transformer over the knowledge graph embedded with popularity nodes and endow the preferences with explicit semantics. Based on this idea, we can manipulate how much popularity preference affects recommendation results and improves the exposure rate of nonpopular items while considering the popularity preferences of different users. Experiments on public datasets show that the proposed method AWING can effectively alleviate popularity bias and ensure the user experience at the same time. The case study further demonstrates the feasibility of AWING on the explainable recommendation task.

Nowadays, side-information is widely used to rein-force the user-item interaction and helps to handle the sparsity issue and cold start problem of conventional recommendation algorithms. Due to the overlook of the relationship between items and entities and the higher-order connectivity information, most existing approaches are hard to get users' deep preferences. In this paper, we propose KGANCL, a Knowledge-aware Graph Attention Network with Distributed & Cross Learning. It focuses on using different forms of the knowledge graph to strengthen both users' and items' embedding representations, respectively. Firstly, Graph Attention Network is adopted for user embedding learning, which can give different importance scores to different neighbors, and a user-item KG graph is used to integrate adjacent information to enhance the representation. Secondly, a cross module is used for item embedding learning, which shares the high-order interaction between the recommender system and the knowledge graph. We also use the idea of distributed processing for embeddings in different entities to improve the learning efficiency. Experimental results demonstrate that KGANCL can provide better recommendations compared with the state-of-the-art baseline models. Our model can also maintain superior prediction accuracy even in little-known interaction scenarios.

Knowledge graphs have emerged as a transformative tool in enhancing personalized recommendation systems. By integrating diverse datasets into a structured semantic network, knowledge graphs offer a holistic view of relationships and entities that can significantly improve the relevance and accuracy of recommendations. Unlike traditional recommendation algorithms that rely primarily on user behaviour and item similarity, knowledge graphs leverage contextual information and complex interconnections among entities to deliver more nuanced and context-aware suggestions. This abstract explores the pivotal role of knowledge graphs in advancing personalized recommendation systems, focusing on their ability to capture intricate relationships between users, items, and attributes. By mapping out these relationships, knowledge graphs facilitate a deeper understanding of user preferences and item characteristics, enabling the generation of more tailored and precise recommendations. Additionally, the incorporation of external knowledge sources into the graph can further enrich the recommendation process, leading to enhanced user satisfaction and engagement. The paper reviews various methodologies for integrating knowledge graphs into recommendation systems, including graph-based algorithms and machine learning techniques. It also examines real-world applications and case studies where knowledge graphs have demonstrated substantial improvements in recommendation quality. Ultimately, the utilization of knowledge graphs represents a significant leap forward in personalizing user experiences, offering a promising avenue for future research and development in the field of recommendation systems.

A primary concern of the current recommendation system is how to provide personalized recommendation to users and improve the accuracy and user satisfaction. The Knowledge Graph(KG) provides a new way to improve the recommendation system. This paper proposes a movie recommendation model based on Recurrent Neural Network(RNN) and KG─RKAN, which uses the auxiliary information in the KG to look for the potential interests of users for personalized recommendations. In addition, in order to solve the problem of user's individual interests, an attention module was designed in RKAN,using different weights to converge user's interest; multiple sets of negative samples were used for comparison to balance model training; data collected from the real movie data set Movielens and IMDB was mapped into a new data set for testing. Experiments show that the model has significantly improved the recommendation accuracy, and can better explain the reasons behind recommendations.

Knowledge graphs (KGs) provide rich semantic and structured knowledge to address data sparsity in traditional recommendation systems, yet efficiently leveraging KGs for better performance remains challenging. This paper proposes KIDRec, a knowledge graph recommendation framework integrating knowledge interaction and diffusion-augmented learning. This framework captures deep KG semantics via high-order entity interactions and uses a diffusion model to generate robust knowledge representations through “noise injection and denoising” on collaborative user embeddings and item embeddings. It then incorporates an adaptive contrastive learning module with hard negative sample enhancement and a dynamic temperature coefficient. Experiments on Book-Crossing, Last.FM, and MovieLens20m show KIDRec outperforms baselines in AUC (0.782, 0.877, 0.979) and F1 (0.682, 0.796, 0.932), with relative AUC improvements of 0.031, 0.032, and 0.002, respectively, while alleviating data sparsity.

Computational intelligence (CI) based prediction models increase the efficient and effective utilization of resources for wind prediction. However, the traditional recurrent neural networks (RNN) are difficult to train on data having long-term temporal dependencies, thus susceptible to an inherent problem of vanishing gradient. This work proposed a method based on an advanced version of RNN known as long short-term memory (LSTM) architecture, which updates recurrent weights to overcome the vanishing gradient problem. This, in turn, improves training performance. The RNN model is developed based on stack LSTM and bidirectional LSTM. The parameters like mean absolute error (MAE), standard deviation error (SDE), and root mean squared error (RMSE) are utilized as performance measures for comparison with recent state-of-the-art techniques. Results showed that the proposed technique outperformed the existing techniques in terms of RMSE and MAE against all the used wind farm datasets. Whereas, a reduction in SDE is observed for larger wind farm datasets. The proposed RNN approach performed better than the existing models despite fewer parameters. In addition, the approach requires minimum processing power to achieve compatible results.

Modeling user preferences from users' historical sequences is one of the core problems of sequential recommendation. Previous studies only considered transition patterns between user-items, ignoring transition patterns between item features and item-item interactions. Recently, there has been interest in integrating knowledge graphs as auxiliary information into sequential recommendation. Most of the existing methods deal with the heterogeneous information in the knowledge graph in a coarse-grained manner. We believe that fine-grained processing of information in knowledge graphs can help recommendation systems understand changes in user preferences, i.e. dividing hetero-geneous information into item-to-item relationships and item-to-attribute relationships. In this paper, we propose a dual-level self-attention network for sequential recommendation. Specifically, we divide the knowledge graph heterogeneous information about items into relation level and attribute level, representing item-item relationship and item-attribute relationship, respectively. Afterwards, the self-attention network is used to learn user preferences at dual-level, respectively. Then, the outputs of the above dual levels are integrated for next item recommendation. Based on extensive experiments on three real-world data sets, our model achieves significant improvements compared to state-of-the-art baseline methods.

Recent studies have shown that the problems of cold processing and data sparsity in recommender systems can be well alleviated with the help of knowledge graph information. Meanwhile, since the information in recommender systems can be represented by graph data structure, graph neural network has great advantages in recommendatory system. Therefore, graph neural network can better mine the deep relationship between objects in the recommender systems which combines with information from knowledge graph. This paper reviews the application of graph neural network in knowledge graph aided recommendation systems. Specifically, this paper firstly sorts out the related knowledge of knowledge graph, recommendation system and graph neural network, including the cold processing and data sparsity, the structure of knowledge graph, and the basic modules of graph neural network. This paper also summarizes the methods such as KGAT and CKAN used in knowledge graph aided recommendation systems in recent years, and gives some discussion on the development of this field.