Aspect-Level Deep Collaborative Filtering via Heterogeneous Information Networks

Joint latent factors and attributes to discover interpretable preferences in recommendation

Latent factor (LF) models are highly effective in extracting useful knowledge from High-Dimensional and Sparse (HiDS) matrices which are commonly seen in various industrial applications. An LF model usually adopts iterative optimizers, which may consume many iterations to achieve a local optima, resulting in considerable time cost. Hence, how to accelerate the training process of an LF model becomes a highly significant issue. To address it, this work innovatively proposes a randomized latent factor (RLF) model. It incorporates the principle of randomized learning techniques for neural networks into the LF analysis on HiDS matrices to alleviate the computational burden greatly. It also extends the standard learning process for randomized neural networks in context of LF analysis to make the resulting model represent an HiDS matrix correctly. Experimental results on three HiDS matrices from industrial applications demonstrate that compared with state-of-the-art LF models, RLF is able to achieve significantly higher computational efficiency and comparable prediction accuracy for missing data. More importantly, it provides a novel, effective, and efficient approach to LF analysis on HiDS matrices.

How to extract useful knowledge from a high-dimensional and sparse (HiDS) matrix efficiently is critical for many big data-related applications. A latent factor (LF) model has been widely adopted to address this problem. It commonly relies on an iterative learning algorithm like stochastic gradient descent. However, an algorithm of this kind commonly consumes many iterations to converge, resulting in considerable time cost on large-scale datasets. How to accelerate an LF model's training process without accuracy loss becomes a vital issue. To address it, this study innovatively proposes a <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</u> ultilayered-and-randomized <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</u> atent <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</u> actor (MLF) model. Its main idea is two-fold: a) adopting randomized-learning to train LFs for implementing a ‘one-iteration’ training process for saving time; and 2) adopting the principle of a generally multilayered structure as in a deep forest or multilayered extreme learning machine to structure its LFs, thereby enhancing its representative learning ability. Empirical studies on six HiDS matrices from real applications demonstrate that compared with state-of-the-art LF models, an MLF model achieves significantly higher computational efficiency with satisfactory prediction accuracy. It has the potential to handle LF analysis on a large scale HiDS matrix with real-time requirements.

In this paper, a revised recommendation system is constructed that ensembles deep neural network and matrix factorization under its framework and uses the explicit feedback for collaborative filtering. Recent works used deep neural network in recommendation for processing auxiliary attributes, but their interaction function is just an inner product on latent features of users and items. For modelling the recommendation system in this research, multi-layer perceptron was used to learn the interaction function. Experiments show significant decrease in MAE and RMSE to be 0.69 and 0.94 respectively, which is comparatively better than general collaborative filtering methods.

Rating prediction using review texts with underlying sentiments

With the advent of the Internet era, the recommendation system has developed rapidly. Heterogeneous information networks representation learning is widely used in recommendation systems due to its advantages in complex information modeling. Although the performance of the recommendation system has been improved, there are still two shortcomings: 1. There is a lot of noise data in the instances of the meta-path generated by random walks of meta-paths, which will reduce the performance of the recommendation system. 2. Traditional recommendation algorithms fail to make full use of the relevant meta-path information in heterogeneous information networks, which makes the recommendation results lack of interpretability. To solve these problems, we propose a recommendation system based on heterogeneous network representation learning and neural network model. Firstly, use the matrix factorization and the similarity calculation to select the meta-path instances with good quality as the pre-training vectors of the recommendation system. Then, we combine LSTM with Attention mechanism to learn the user, item and meta-path embeddings, and use MLP to make prediction after fusion to jointly improve the recommendation effect. We conducted experiments on Movielens datasets to evaluate the performance of our proposed recommendation.

Recommender systems are designed to identify the items that a user will like or find useful based on the user's prior preferences and activities. These systems have become ubiquitous and are an essential tool for information filtering and (e-)commerce. Over the years, collaborative filtering, which derive these recommendations by leveraging past activities of groups of users, has emerged as the most prominent approach for solving this problem. This talk will present some of our recent work towards improving the performance of collaborative filtering-based recommender systems and understanding some of their fundamental limitations and characteristics. It will start by analyzing how the ratings that users provide to a set of items relate to their ratings of the set's individual items and, using these insights, will present rating prediction approaches that utilize distant supervision. It will then discuss extensions to approaches based on sparse linear and latent factor models that postulate that users' preferences are a combination of global and local preferences, which are shown to lead to better user modeling and as such improved prediction performance. Finally, the talk will conclude by discussing what can be accurately predicted by latent factor approaches and by analyzing the estimation error of sparse linear and latent factor models and how its characteristics impacts the performance of top N recommendation algorithms.

Mixture factor analysis is examined as a means of flexibly estimating nonnormally distributed continuous latent factors in the presence of both continuous and dichotomous observed variables. A simulation study compares mixture factor analysis with normal maximum likelihood (ML) latent factor modeling. Different results emerge for continuous versus dichotomous outcomes. For dichotomous outcomes, normal ML path estimates have bias that worsens as latent factor skew/kurtosis increases and does not diminish as sample size increases, whereas the mixture factor analysis model produces nearly unbiased estimators as sample sizes increase (500 and greater) and offers near nominal coverage probability. For continuous outcome variables, both methods produce factor loading estimates with minimal bias regardless of latent factor skew, but the mixture factor analysis is more efficient. The method is demonstrated using data motivated by a study on youth with cystic fibrosis examining predictors of treatment adherence. In summary, mixture factor analysis provides improvements over normal ML estimation in the presence of skewed/kurtotic latent factors, but due to variability in the estimator relating the latent factor to dichotomous outcomes and computational issues, the improvements were only fully realized, in this study, at larger sample sizes (500 and greater).

Traditional recommender systems (RS) only consider homogeneous data and cannot fully model heterogeneous information of complex objects and relations. Recent advances in the study of Heterogeneous Information Network (HIN) have shed some light on how to leverage heterogeneous information in RS. However, existing HIN-based recommendation models assume HIN is invariable and merely use HIN as a data source for assisting recommendation, which limits their performance. In this paper, we propose a multi-task learning framework, called MTRec, for recommendation over HIN. MTRec relies on self-attention mechanism to learn the semantics of meta-paths in HIN and jointly optimizes the tasks of both recommendation and link prediction. Using a Bayesian task weight learner, MTRec is able to achieve the balance of two tasks during optimization automatically. Moreover, MTRec provides good interpretabilities of recommendation through a "translation" mechanism which is used to model the three-way interactions among users, items and the meta-paths connecting them. Experimental results demonstrate the superiority of MTRec over state-of-the-art HIN-based recommendation models, and the case studies we provide illustrate that MTRec enhances the explainability of RS.

Despite today’s extensive utilisation of MOOCs, video recommendation in online learning platforms still confronts numerous obstacles. The fundamental reason is that the traditional recommendation methods cannot achieve precision recommendation of learning resources that target at students’ needs. The heterogeneous information network is widely used in learning resource recommendation nowadays thanks to its capacity of intuitively representing different entities and complex relationships between entities. However, it’s tricky to use rich semantic information in it to do entity modelling. To solve this problem, this paper analyses and mines students’ behavioural data on MOOCs to obtain their personalized features, then uses graph embedding to incorporate weight and construct a weighted heterogeneous information network to recommend online video courses. The experiments utilise the data of the MOOCCube to investigate the effectiveness of various meta-path combinations. The optimal number of layers for the graph convolutional network, latent factors, and entity embedding dimension are determined through comparative studies. We also compare our model’s performance to that of other relevant models and utilise ablation experiments to verify the effectiveness of our model.

Information network derived from various domains are studied recently. Searching for Similarity is a major task into such types Information Network. Lot of research on computing similar objects is done in Homogeneous Information Network. But real world scenario can be described easily by Heterogeneous Information Network (HIN) which consists of different types of entities and relationship among them. Due to multiple type of entities and links between them in HIN, it is necessary to find the similarities between the nodes of HIN. In Homogeneous Information Network, there is only single type of node and links in between them. There are many existing methods by which similarity among the nodes of Homogeneous Information Network can be calculated. But those methods cannot be applied for the HIN because semantic meaning behind each path cannot be considered. If we want to apply techniques of Homogeneous Information Network on HIN then we need to project HIN into Homogeneous Information Network which causes loss of Information. So there is a need to apply different techniques or similarity measures on HIN to calculate the similarities between nodes in HIN. There are many similarity measures implemented by researchers for HIN. Similarity search basically concentrates on discovering the most similarity objects for a given query entity. In a comparative analysis section, we have discussed some of the measures used for similarity.

This work introduces a novel estimation method, called LOVE, of the entries and structure of a loading matrix $A$ in a latent factor model $X=AZ+E$, for an observable random vector $X\\in \\mathbb{R}^{p}$, with correlated unobservable factors $Z\\in \\mathbb{R}^{K}$, with $K$ unknown, and uncorrelated noise $E$. Each row of $A$ is scaled, and allowed to be sparse. In order to identify the loading matrix $A$, we require the existence of pure variables, which are components of $X$ that are associated, via $A$, with one and only one latent factor. Despite the fact that the number of factors $K$, the number of the pure variables and their location are all unknown, we only require a mild condition on the covariance matrix of $Z$, and a minimum of only two pure variables per latent factor to show that $A$ is uniquely defined, up to signed permutations. Our proofs for model identifiability are constructive, and lead to our novel estimation method of the number of factors and of the set of pure variables, from a sample of size $n$ of observations on $X$. This is the first step of our LOVE algorithm, which is optimization-free, and has low computational complexity of order $p^{2}$. The second step of LOVE is an easily implementable linear program that estimates $A$. We prove that the resulting estimator is near minimax rate optimal for $A$, with respect to the $\\|\\ \\|_{\\infty ,q}$ loss, for $q\\geq 1$, up to logarithmic factors in $p$, and that it can be minimax-rate optimal in many cases of interest.\n¶ The model structure is motivated by the problem of overlapping variable clustering, ubiquitous in data science. We define the population level clusters as groups of those components of $X$ that are associated, via the matrix $A$, with the same unobservable latent factor, and multifactor association is allowed. Clusters are respectively anchored by the pure variables, and form overlapping subgroups of the $p$-dimensional random vector $X$. The Latent model approach to OVErlapping clustering is reflected in the name of our algorithm, LOVE.\n¶ The third step of LOVE estimates the clusters from the support of the columns of the estimated $A$. We guarantee cluster recovery with zero false positive proportion, and with false negative proportion control. The practical relevance of LOVE is illustrated through the analysis of a RNA-seq data set, devoted to determining the functional annotation of genes with unknown function.

Heterogeneous network user authentication keys are vulnerable to plain text attacks, resulting in poor encryption performance. Therefore, a heterogeneous bilinear pairing signcryption scheme is proposed. The symbol frequency detection method is used to detect symbols of user authentication data, establish an ORK user protocol in a heterogeneous network, and perform a pairwise signcryption scheme to linearly encode user cognitive information in a heterogeneous network. Use Ryption technology to authenticate and re-sign users in heterogeneous networks, and use a bilinear pairing signature encryption scheme to control cipher text in user processes to encrypt identity authentication information in heterogeneous networks. Establish a statistical analysis model to encrypt user authentication information in a heterogeneous network to achieve intelligence. Simulation results show that the method has good scrambling ability and strong anti-attack ability. It is used for bilinear pairing signature encryption of user authentication keys in heterogeneous networks.

Extracting a core structure from heterogeneous information network using h-subnet and meta-path strength

One important problem of recommender system is rating prediction. In this paper, we use the movie rating data from MovieLens as an example to show how to use users’ attributes to improve the accuracy of rating prediction. Through data analysis, we observe that users having similar attributes tend to share more similar preferences and users with a special attribute have their own preferred items. Based on the two observations, we assume that a user’s rating to an item is determined by both the user intrinsic characteristics and the user common characteristics. Using the widely adopted latent factor model for rating prediction, in our proposed solution, we use two kinds of latent factors to model a user: one for the user intrinsic characteristics and the other for the user common characteristics. The latter encodes the influence of users’ attributes which include user age, gender and occupation. On the other hand, we jointly use user attributes or item category information and rating data for calculating similarity of users or items. The similarity calculating results are used in our proposed latent factor model as a regularization term to regularize users or items latent factors gap. Experimental results on MovieLens show that by incorporating users’ attributes influences, much lower prediction error is achieved than the state-of-the-art models. The prediction error is further reduced by incorporating influences from item category popularity and item popularity.