Metadata-dependent Infinite Poisson Factorization for Efficiently Modelling Sparse and Large Matrices in Recommendation

Modelling sparse and large data sets is highly in demand yet challenging in recommender systems. With the computation only on the non-zero ratings, Poisson Factorization (PF) enabled by variational inference has shown its high efficiency in scalable recommendation, e.g., modeling millions of ratings. However, as PF learns the ratings by individual users on items with the Gamma distribution, it cannot capture the coupling relations between users (items) and the rating popularity (i.e., favorable rating scores that are given to one item) and rating sparsity (i.e., those users (items) with many zero ratings) for one item (user). This work proposes Coupled Poisson Factorization (CPF) to learn the couplings between users (items), and the user/item attributes (i.e., metadata) are integrated into CPF to form the Metadata-integrated CPF (mCPF) to not only handle sparse but also popular ratings in very large-scale data. Our empirical results show that the proposed models significantly outperform PF and address the key limitations in PF for scalable recommendation.

INTRODUCTION: Collaborative filtering (CF) algorithms based on deep learning and matrix factorization aim to learn deeper latent features of users and items from user ratings. However, such methods often face challenges due to the sparsity of rating data, which limits their recommendation performance. OBJECTIVES: This study aims to design a hybrid recommendation algorithm named auto-encoder deep learning and matrix factorization (AED-MF), which integrates deep learning and matrix factorization to address the sparsity issue in rating data and improve the extraction of deeper latent features of users and items. METHODS: The AED-MF algorithm combines an auto-encoder-based deep learning approach with matrix factorization to model both deeper hidden representations and nonlinear relationships between users and items. The methodology includes data downloading, mounting, cleaning, model training, and experimental evaluation, with matrix factorization applied to predict missing ratings in the rating matrix. RESULTS: The proposed AED-MF algorithm demonstrated strong performance in key recommendation metrics, effectively learning deeper user and item representations and applying them to capture complex nonlinear relationships. It also reduced the impact of rating data sparsity while maintaining high recommendation accuracy. CONCLUSION: The AED-MF recommendation algorithm successfully alleviates the problem of sparse rating data and enhances recommendation accuracy by leveraging the strengths of both deep learning and matrix factorization, offering an effective solution for improving collaborative filtering-based recommender systems.

Recommender systems are essential engines to deliver product recommendations for e-commerce businesses. Successful adoption of recommender systems could significantly influence the growth of marketing targets. Collaborative filtering is a type of recommender system model that uses customers' activities in the past, such as ratings. Unfortunately, the number of ratings collected from customers is sparse, amounting to less than 4%. The latent factor model is a kind of collaborative filtering that involves matrix factorization to generate rating predictions. However, using only matrix factorization would result in an inaccurate recommendation. Several models include product review documents to increase the effectiveness of their rating prediction. Most of them use methods such as TF-IDF and LDA to interpret product review documents. However, traditional models such as LDA and TF-IDF face some shortcomings, in that they show a less contextual understanding of the document. This research integrated matrix factorization and novel models to interpret and understand product review documents using LSTM and word embedding. According to the experiment report, this model significantly outperformed the traditional latent factor model by more than 16% on an average and achieved 1% on an average based on RMSE evaluation metrics, compared to the previous best performance. Contextual insight of the product review document is an important aspect to improve performance in a sparse rating matrix. In the future work, generating contextual insight using bidirectional word sequential is required to increase the performance of e-commerce recommender systems with sparse data issues.

Introduction: Recommendation systems (RS) are pivotal in enhancing personalized user experiences, particularly in the educational domain, where tailored book recommendation systems (BRS) support individualized learning. Despite their importance, selecting an optimal recommendation algorithm remains challenging, especially in scenarios involving large and sparse datasets. Objectives: This study aims to evaluate the performance of four recommendation algorithms—Random Baseline, Popular, User-Based Collaborative Filtering (UBCF), and Singular Value Decomposition (SVD)—to identify their strengths and tradeoffs regarding accuracy and computational efficiency. The goal is to provide actionable insights for selecting appropriate algorithms for educational platforms and similar applications. Methods: The evaluation employed a dataset containing over 10,000 books, 53,424 users, and 981,756 ratings. The algorithms were assessed using Root Mean Square Error (RMSE) to measure predictive accuracy and training time to indicate computational efficiency. The analysis addressed challenges such as data sparsity and rating biases. Results: The results demonstrated that SVD achieved the highest accuracy (RMSE: 0.950), effectively uncovering latent relationships in sparse datasets. UBCF, with a slightly lower accuracy (RMSE: 1.020), offered a balance between accuracy and computational efficiency, making it suitable for real-time applications. Conversely, simpler algorithms like Random Baseline and Popular exhibited faster training times but significantly lower predictive accuracy, highlighting the limitations of non-personalized methods. Conclusions: This study underscores the tradeoffs between accuracy and efficiency among recommendation algorithms. While SVD is ideal for accuracy-driven applications, UBCF provides a practical alternative for scenarios requiring computational efficiency. The findings have substantial implications for educational platforms and e-commerce, where personalized recommendations enhance user satisfaction and engagement. Future research should focus on integrating deep learning models and expanding evaluation criteria, including user satisfaction and diversity, to further improve the performance of recommendation systems.

Collaborative filtering algorithms, such as matrix factorization techniques, are recently gaining momentum due to their promising performance on recommender systems. However, most collaborative filtering algorithms suffer from data sparsity. Active learning algorithms are effective in reducing the sparsity problem for recommender systems by requesting users to give ratings to some items when they enter the systems. In this paper, a new matrix factorization model, called Enhanced SVD (ESVD) is proposed, which incorporates the classic matrix factorization algorithms with ratings completion inspired by active learning. In addition, the connection between the prediction accuracy and the density of matrix is built to further explore its potentials. We also propose the Multi-layer ESVD, which learns the model iteratively to further improve the prediction accuracy. To handle the imbalanced data sets that contain far more users than items or more items than users, the Item-wise ESVD and User-wise ESVD are presented, respectively. The proposed methods are evaluated on the famous Netflix and Movielens data sets. Experimental results validate their effectiveness in terms of both accuracy and efficiency when compared with traditional matrix factorization methods and active learning methods.

Evaluation Of Recommendation System On A Movie Dataset

The rating matrix of a personalized recommendation system contains a high percentage of unknown rating scores which lowers the quality of the prediction. Besides, during data streaming into memory, some rating scores are misplaced from its appropriate cell in the rating matrix which also decrease the quality of the prediction. The singular value decomposition algorithm predicts the unknown rating scores based on the relation between the implicit feedback of both users and items, but exploiting neither the user similarity nor item similarity which leads to low accuracy predictions. There are several factorization methods used in improving the prediction performance of the collaborative filtering technique such as baseline, matrix factorization, neighbour-base. However, the prediction performance of the collaborative filtering using factorization methods is still low while baseline and neighbours-base have limitations in terms of over fitting. Therefore, this paper proposes Ensemble Divide and Conquer (EDC) approach for solving 2 main problems which are the data sparsity and the rating scores’ deviation (misplace). The EDC approach is founded by the Singular Value Decomposition (SVD) algorithm which extracts the relationship between the latent feedback of users and the latent feedback of the items. Furthermore, this paper addresses the scale of rating scores as a sub problem which effect on the rank approximation among the users’ features. The latent feedback of the users and items are also SVD factors. The results using the EDC approach are more accurate than collaborative filtering and existing methods of matrix factorization namely SVD, baseline, matrix factorization and neighbours-base. This indicates the significance of the latent feedback of both users and items against the different factorization features in improving the prediction accuracy of the collaborative filtering technique.

In the Big Data Era, recommender systems perform a fundamental role in data management and information filtering. In this context, Collaborative Filtering (CF) persists as one of the most prominent strategies to effectively deal with large datasets and is capable of offering users interesting content in a recommendation fashion. Nevertheless, it is well‐known CF recommenders suffer from data sparsity, mainly in cold‐start scenarios, substantially reducing the quality of recommendations. In the vast literature about the aforementioned topic, there are numerous solutions, in which the state‐of‐the‐art contributions are, in some sense, conditioned or associated with traditional CF methods such as Matrix Factorization (MF), that is, they rely on linear optimization procedures to model users and items into low‐dimensional embeddings. To overcome the aforementioned challenges, there has been an increasing number of studies exploring deep learning techniques in the CF context for latent factor modelling. In this research, authors conduct a systematic review focusing on state‐of‐the‐art literature on deep learning techniques applied in collaborative filtering recommendation, and also featuring primary studies related to mitigating the cold start problem. Additionally, authors considered the diverse non‐linear modelling strategies to deal with rating data and side information, the combination of deep learning techniques with traditional CF‐based linear methods, and an overview of the most used public datasets and evaluation metrics concerning CF scenarios.

Purpose – Project ATHENA aims to develop an application to address information overload, primarily focused on Recommendation Systems (RSs) with the personalization and user experience design of a modern system. Method – Two machine learning (ML) algorithms were used: (1) TF-IDF for Content-based filtering (CBF); (2) Classification with Matrix Factorization- Singular Value Decomposition (SVD) applied with Collaborative filtering (CF) and mean (normalization) for prediction accuracy of the CF. Data sampling in academic Research and Development (R&D) of Philippine Council for Agriculture, Aquatic, and Natural Resources Research and Development (PCAARRD) e-Library and Project SARAI publications plus simulated data used as training sets to generate a recommendation of items that uses the three RS filtering (CF, CBF, and personalized version of item recommendations). Series of Testing and TAM performed and discussed. Results – Findings allow users to engage in online information and quickly evaluate retrieved items produced by the application. Compatibility-testing (CoT) shows the application is compatible with all major browsers and mobile-friendly. Performance-testing (PT) recommended v-parameter specs and TAM evaluations results indicate strongly associated with overall positive feedback, thoroughly enough to address the information-overload problem as the core of the paper. Conclusion – A modular architecture presented addressing the information overload, primarily focused on RSs with the personalization and design of modern systems. Developers utilized Two ML algorithms and prototyped a simplified version of the architecture. Series of testing (CoT and PT) and evaluations with TAM were performed and discussed. Project ATHENA added a UX feature design of a modern system. Recommendations – High-end hardware specs of v-Parameter are recommended, at least 8-cores of vCPU with 16GiB of memory and sufficient driver-type size to run the model and End-to-end jobs execution for continuously maintaining the application. Research Implications – Future Developers must use/integrate a large dataset. Other ML approaches can expand Hybrid-RS better. Implicit data and additional filtering methods may enhance the application in the future.

Recommendation system can predict the ratings of users to items by leveraging machine learning algorithms. The use of recommendation systems is common in e-commerce websites now-a-days. Since enormous amounts of data including users’ click streams, purchase history, demographics, social networking comments and user-item ratings are stored in e-commerce systems databases, the volume of the data is getting bigger at high speed, and the data is sparse. However, the recommendations and predictions must be made in real time, enabling to bring enormous benefits to human beings. Apache spark is well suited for applications which require high speed query of data, transformation and analytics results. Therefore, the recommendation system developed in this research is implemented on Apache Spark. Also, the matrix factorization using Alternating Least Squares (ALS) algorithm which is a type of collaborative filtering is used to solve overfitting issues in sparse data and increases prediction accuracy. The overfitting problem arises in the data as the user-item rating matrix is sparse. In this research a recommendation system for e-commerce using alternating least squares (ALS) matrix factorization method on Apache Spark MLlib is developed. The research shows that the RMSE value is significantly reduced using ALS matrix factorization method and the RMSE is 0.870. Consequently, it is shown that the ALS algorithm is suitable for training explicit feedback data set where users provide ratings for items.

Matrix Factorization (MF) is one of the most successful Collaborative Filtering (CF) techniques used in recommender systems due to its effectiveness and ability to deal with very large user-item rating matrix. However, when the rating matrix sparseness increases its performance deteriorates. Expanding MF to include side-information of users and items has been shown by many researchers both to improve general recommendation performance and to help alleviate the data-sparsity and cold-start issues in CF. In regard to item feature side-information, most schemes incorporate this information through a two stage process: intermediate results (e.g., on item similarity) are first computed based on item attributes; these are then combined with MF. In this paper, focussing on item side-information, we propose a model that directly incorporates item features into the MF framework in a single step process. The model, which we name FeatureMF, does this by projecting every available attribute datum in each of the item features into the same latent factor space with users and items, thereby in effect enriching item representation in MF. Results are presented of comparative performance experiments of the model against three state-of-the-art item information enriched models, as well as against four reference benchmark models, using two public real-world datasets, Douban and Yelp, with four training:test ratio scenarios each. It is shown to yield the best recommendation performance over all these models across all contexts including data-sparsity situations, in particular, achieving over 0.9% to over 6.5% MAE recommendation performance improvement over the next best model, HERec. FeatureMF is also found to alleviate cold start and to scale well, almost linearly, in regard to computational time, as a function of dataset size.

BackgroundMatrix factorization is a well established pattern discovery tool that has seen numerous applications in biomedical data analytics, such as gene expression co-clustering, patient stratification, and gene-disease association mining. Matrix factorization learns a latent data model that takes a data matrix and transforms it into a latent feature space enabling generalization, noise removal and feature discovery. However, factorization algorithms are numerically intensive, and hence there is a pressing challenge to scale current algorithms to work with large datasets. Our focus in this paper is matrix tri-factorization, a popular method that is not limited by the assumption of standard matrix factorization about data residing in one latent space. Matrix tri-factorization solves this by inferring a separate latent space for each dimension in a data matrix, and a latent mapping of interactions between the inferred spaces, making the approach particularly suitable for biomedical data mining.ResultsWe developed a block-wise approach for latent factor learning in matrix tri-factorization. The approach partitions a data matrix into disjoint submatrices that are treated independently and fed into a parallel factorization system. An appealing property of the proposed approach is its mathematical equivalence with serial matrix tri-factorization. In a study on large biomedical datasets we show that our approach scales well on multi-processor and multi-GPU architectures. On a four-GPU system we demonstrate that our approach can be more than 100-times faster than its single-processor counterpart.ConclusionsA general approach for scaling non-negative matrix tri-factorization is proposed. The approach is especially useful parallel matrix factorization implemented in a multi-GPU environment. We expect the new approach will be useful in emerging procedures for latent factor analysis, notably for data integration, where many large data matrices need to be collectively factorized.

Recommender systems are useful tools to give personalized recommendations to users. One of the most popular techniques used in these systems is collaborative filtering. Recommender system algorithms get into trouble with data sparsity and scalability. These challenges cause lack of convergence in our algorithms. In this research, we propose a new method based on matrix factorization which alleviates data sparsity. We suggest a new method which can be performed as a preprocessing method for initial latent factor matrices of users and items. Initialized latent factors in matrix factorization lead to two advantages: (1) sparsity and scalability would be covered and (2) convergence of algorithms would be faster. We have shown that our method has improved the accuracy of optimization-based matrix factorization technique. Also it has increased the speed of matrix factorization convergence.

A multi-attention deep neural network model base on embedding and matrix factorization for recommendation

In recent years, Recommender System (RS) research work has covered a wide variety of Artificial Intelligence techniques, ranging from traditional Matrix Factorization (MF) to complex Deep Neural Networks (DNN). Traditional Collaborative Filtering (CF) recommendation methods such as MF, have limited learning capabilities as it only considers the linear combination between user and item vectors. For learning non-linear relationships, methods like Neural Collaborative Filtering (NCF) incorporate DNN into CF methods. Though, CF methods still suffer from cold start and data sparsity. This paper proposes an improved hybrid-based RS, namely Neural Matrix Factorization++ (NeuMF++), for effectively learning user and item features to improve recommendation accuracy and alleviate cold start and data sparsity. NeuMF++ is proposed by incorporating effective latent representation into NeuMF via Stacked Denoising Autoencoders (SDAE). NeuMF++ can also be seen as the fusion of GMF++ and MLP++. NeuMF is an NCF framework which associates with GMF (Generalized Matrix Factorization) and MLP (Multilayer Perceptrons). NeuMF achieves state-of-the-art results due to the integration of GMF linearity and MLP non-linearity. Concurrently, incorporating latent representations has shown tremendous improvement in GMF and MLP, which result in GMF++ and MLP++. Latent representation obtained through the SDAEs' latent space allows NeuMF++ to effectively learn user and item features, significantly enhancing its learning capability. However, sharing feature extractions among GMF++ and MLP++ in NeuMF++ might hinder its performance. Hence, allowing GMF++ and MLP++ to learn separate features provides more flexibility and greatly improves its performance. Experiments performed on a real-world dataset have demonstrated that NeuMF++ achieves an outstanding result of a test root-mean-square error of 0.8681. In future work, we can extend NeuMF++ by introducing other auxiliary information like text or images. Different neural network building blocks can also be integrated into NeuMF++ to form a more robust recommendation model.