Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation

GPS: Factorized group preference-based similarity models for sparse sequential recommendation

Long- and short-term self-attention network for sequential recommendation

Modeling user preferences from users' historical sequences is one of the core problems of sequential recommendation. Previous studies only considered transition patterns between user-items, ignoring transition patterns between item features and item-item interactions. Recently, there has been interest in integrating knowledge graphs as auxiliary information into sequential recommendation. Most of the existing methods deal with the heterogeneous information in the knowledge graph in a coarse-grained manner. We believe that fine-grained processing of information in knowledge graphs can help recommendation systems understand changes in user preferences, i.e. dividing hetero-geneous information into item-to-item relationships and item-to-attribute relationships. In this paper, we propose a dual-level self-attention network for sequential recommendation. Specifically, we divide the knowledge graph heterogeneous information about items into relation level and attribute level, representing item-item relationship and item-attribute relationship, respectively. Afterwards, the self-attention network is used to learn user preferences at dual-level, respectively. Then, the outputs of the above dual levels are integrated for next item recommendation. Based on extensive experiments on three real-world data sets, our model achieves significant improvements compared to state-of-the-art baseline methods.

User preferences are usually dynamic in real-world recommender systems, and a user»s historical behavior records may not be equally important when predicting his/her future interests. Existing recommendation algorithms -- including both shallow and deep approaches -- usually embed a user»s historical records into a single latent vector/representation, which may have lost the per item- or feature-level correlations between a user»s historical records and future interests. In this paper, we aim to express, store, and manipulate users» historical records in a more explicit, dynamic, and effective manner. To do so, we introduce the memory mechanism to recommender systems. Specifically, we design a memory-augmented neural network (MANN) integrated with the insights of collaborative filtering for recommendation. By leveraging the external memory matrix in MANN, we store and update users» historical records explicitly, which enhances the expressiveness of the model. We further adapt our framework to both item- and feature-level versions, and design the corresponding memory reading/writing operations according to the nature of personalized recommendation scenarios. Compared with state-of-the-art methods that consider users» sequential behavior for recommendation, e.g., sequential recommenders with recurrent neural networks (RNN) or Markov chains, our method achieves significantly and consistently better performance on four real-world datasets. Moreover, experimental analyses show that our method is able to extract the intuitive patterns of how users» future actions are affected by previous behaviors.

Markov chains have been suggested since the early sixties for modeling manpower flow, mostly in governmental and public agencies. Numerous books and articles have proclaimed Markov chains as a superior forecasting technique. The flow of data for a consecutive number of years has been the main subject of interest for Markov chains method. Recently, smoothing techniques have been introduced in Markov chain method. This paper aims to review those smoothing technique and analyze their potential in enhancing the conventional Markov chain method.

The status of things always changes with the process of time. Markov chains approach considers that as long as the current status is known, the future state of things can be forecasted without understanding the past state. Considering the spatial autocorrelation of spatial things, Markov chains is combined with spatial autocorrelation to develop the spatial Markov chains to study the influence of regional background on regional transition. The industrial structure of regions alters in different periods of time, and its development process and tendency can be approached approximately by using Markov chains methods. Based on the coefficient dataset of industrial structure at the county level in Sichuan province from 2000 to 2007, this paper attempts to apply Markov chains and Spatial Markov chain to investigate the spatial and temporal characteristics of industrial structure level in Sichuan. Firstly, all the coefficient data of industrial structure in Sichuan are classified into 4 different classes (low, middle-low, middle-high and high) and Markov transition probability matrix is estimated to explore whether the convergence of industrial structure level exists in Sichuan. Secondly, conditioning on each region's spatial lag at the beginning of each year, spatial Markov matrices are constructed to investigate the relationship between transition probability of different regions and their neighbors, and maps are accordingly made in order to visualize spatial patterns of class transitions. Thirdly, the evolutionary trends of industrial structure level in the next twenty years, fifty years and one hundred years are forecasted respectively by computing the twentieth, fiftieth and hundredth power of Markov transition probability matrix. Finally, the measures to improve the industrial structure level in Sichuan province are given.

Sequential recommendation (SR) aims to predict the subsequent behaviors of users by understanding their successive historical behaviors. Recently, some methods for SR are devoted to alleviating the data sparsity problem (i.e., limited supervised signals for training), which take account of contrastive learning to incorporate self-supervised signals into SR. Despite their achievements, it is far from enough to learn informative user/item embeddings due to the inadequacy modeling of complex collaborative information and co-action information, such as user-item relation, user-user relation, and item-item relation. In this paper, we study the problem of SR and propose a novel multi-level contrastive learning framework for sequential recommendation, named MCLSR. Different from the previous contrastive learning-based methods for SR, MCLSR learns the representations of users and items through a cross-view contrastive learning paradigm from four specific views at two different levels (i.e., interest- and feature-level). Specifically, the interest-level contrastive mechanism jointly learns the collaborative information with the sequential transition patterns, and the feature-level contrastive mechanism re-observes the relation between users and items via capturing the co-action information (i.e., co-occurrence). Extensive experiments on four real-world datasets show that the proposed MCLSR outperforms the state-of-the-art methods consistently.

Group-aware graph neural networks for sequential recommendation

We study the user profile completion and enrichment problem, where the goal is to estimate the unknown values of user profiles. We investigate how the type of the features (categorical or continuous) suggests the use of a specific approach for this task. In particular, in this context, we validate the hypothesis that a classification method like K-nearest neighbor search fits better for categorical features and matrix factorization methods such as Non-negative Matrix Factorization perform superior on continuous features. We study different variants of K-nearest neighbor search (with different metrics) and demonstrate how they perform in different settings. Moreover, we investigate the impact of shifting the variables on the quality of (non-negative) factorization and the prediction error. We validate our methods via extensive experiments on real-world datasets and, finally, based on the results and observations, we discuss a hybrid approach to accomplish this task.

Soil erosion prediction using Markov and CA-Markov chains methods and remote sensing drought indicators

Since the first introduced Collaborative Filtering Recommenders (CFR) there have been many attempts to improve their performance by enhancing the prediction accuracy. Even though rating prediction is the prevailing paradigm in CFR, there are other issues which have gained significant attention with respect to the content and its variety. Coverage, which constitutes the degree to which recommendations cover the set of available items, is an important factor along with diversity of the items proposed to an individual, often measured by an average dissimilarity between all pairs of recommended items. In this paper, we argue that coverage and diversity cannot be effectively addressed by conventional CFR with pure similarity-based neighborhood creation processes, especially in sparse datasets. Motivated by the need for including wider content characteristics, we propose a novel neighbor selection technique which emphasizes on variety in preferences (to cover polyphony in selection). Our approach consists of a new metric, named Exploriometer, which acts as a personality trait for users based on their rating behavior. We favor users who are explorers in order to increase polyphony, and subsequently coverage and diversity; but we still select similar users when we create neighborhoods as a solid basis in order to keep accuracy levels high. The proposed approach has been experimented by two real-world datasets (MovieLens and Yahoo! Music ) with coverage, diversity and accuracy aware recommendations extracted by both traditional CFR and CFR enhanced with our neighborhood creation process. We also introduce a new metric, inspired by the Pearson Correlation Coefficient, to estimate the diversity of recommended items. The derived results demonstrate that our neighbor selection technique can enhance coverage and diversity of the recommendations, especially on sparse datasets.

Location-based social networks (LBSNs), such as Gowalla, Facebook, Foursquare, Brightkite, and so on, have attracted millions of users to share their social friendship and their locations via check-ins in the past few years. Plenty of valuable information is accumulated based on the check-in behaviors, which makes it possible to learn users’ moving patterns as well as their preferences. In LBSNs, point-of-interest (POI) recommendation is one of the most significant tasks because it can help targeted users explore their surroundings as well as help third-party developers provide personalized services. Matrix factorization is a promising method for this task because it can capture users’ preferences to locations and is widely adopted in traditional recommender systems such as movie recommendation. However, the sparsity of the check-in data makes it difficult to capture users’ preferences accurately. Geographical influence can help alleviate this problem and have a large impact on the final recommendation result. By studying users’ moving patterns, we find that users tend to check in around several centers and different users have different numbers of centers. Based on this, we propose a Multi-center Gaussian Model (MGM) to capture this pattern via modeling the probability of a user’s check-in on a location. Moreover, users are usually more interested in the top 20 or even top 10 recommended POIs, which makes personalized ranking important in this task. From previous work, directly optimizing for pairwise ranking like Bayesian Personalized Ranking (BPR) achieves better performance in the top- k recommendation than directly using matrix matrix factorization that aims to minimize the point-wise rating error. To consider users’ preferences, geographical influence and personalized ranking, we propose a unified POI recommendation framework, which unifies all of them together. Specifically, we first fuse MGM with matrix factorization methods and further with BPR using two different approaches. We conduct experiments on Gowalla and Foursquare datasets, which are two large-scale real-world LBSN datasets publicly available online. The results on both datasets show that our unified POI recommendation framework can produce better performance.

In order to alleviate the problem of rating sparsity in recommendation system, this paper proposes a model called Constrained Probabilistic Matrix Factorization with Neural Network (CPMF-NN). In user modeling, it takes the influence of users’ interaction items into consideration. In item modeling, it utilizes convolutional neural network to extract the item latent features from the corresponding documents. In the process of fusion of latent feature vectors, multi-layer perceptron is used to grasp the nonlinear structural characteristics of user-item interactions. Through extensive experiments on three real-world datasets, the results show that CPMF-NN achieves good performance on different sparse data sets.

Users' historical activities are usually contained in real-life sequential recommendation systems to predict their future behaviors. In this situation, traditional Factorization Machines (FMs) approaches may be not suitable. Recently, a new surge of interest aims to use recurrent neural networks(RNN) to encode users' dynamic features with temporal characteristics. However, most of these works fail to reproduce computational simplicity of FMs. In this paper, we propose an architecture of extended-FM for sequential recommendation, which presents temporal feature interactions in an explicit way as traditional FM's formula. Our approach also allows us to accomplish computation of the model in linear time. Furthermore, we merge extended-FM into higher-order interaction framework without significant changes to the deeper models themselves. We conduct comprehensive experiments on two real-world datasets. The results demonstrate that extended-FM outperforms traditional FMs as well as deep learning feature combination models on sequential recommendation tasks.

An increased interest in sequential recommendation has been observed in recent years. Many models have been proposed to leverage the sequential user-item interaction data, which includes those based on Markov Chain or recurrent neural networks. Most of these models are designed for the scenario where each historical record composed of single item. However, the records could be a subset of items (or session) such as music playlists and baskets in e-commerce applications. How to leverage the session structure to improve the effectiveness of the recommendation system is a challenge. To this end, we propose a MEmory-augmented Attention Network for Sequential recommendation (MEANS), to effectively recommend next items given the sequential session data. The most recent sessions are stored into external memory after a max-pooling operation. The long-term user preference are learned through an attention network which is stacked on the memory layer. Finally, the mixture of long-term and short-term preference is feeded into the prediction layer to make recommendations. Extensive experiments on four real datasets show that MEANS outperforms various state-of-the-art sequential recommendation models.