Recommender Systems with Condensed Local Differential Privacy

Recommender systems are collecting and analyzing user data to provide better user experience. However, several privacy concerns have been raised when a recommender knows user's set of items or their ratings. A number of solutions have been suggested to improve privacy of legacy recommender systems, but the existing solutions in the literature can protect either items or ratings only. In this paper, we propose a recommender system that protects both user's items and ratings. For this, we develop novel matrix factorization algorithms under local differential privacy (LDP). In a recommender system with LDP, individual users randomize their data themselves to satisfy differential privacy and send the perturbed data to the recommender. Then, the recommender computes aggregates of the perturbed data. This framework ensures that both user's items and ratings remain private from the recommender. However, applying LDP to matrix factorization typically raises utility issues with i) high dimensionality due to a large number of items and ii) iterative estimation algorithms. To tackle these technical challenges, we adopt dimensionality reduction technique and a novel binary mechanism based on sampling. We additionally introduce a factor that stabilizes the perturbed gradients. With MovieLens and LibimSeTi datasets, we evaluate recommendation accuracy of our recommender system and demonstrate that our algorithm performs better than the existing differentially private gradient descent algorithm for matrix factorization under stronger privacy requirements.

Recommender systems have been an efficient strategy to deal with information overload by producing personalized predictions. Recommendation systems based on deep learning have accomplished magnificent results, but most of these systems are traditional recommender systems that use a single rating. In this work, we introduce a multi-criteria collaborative filtering recommender by combining deep neural network and matrix factorization. Our model consists of two parts: the first part uses a fused model of deep neural network and matrix factorization to predict the criteria ratings and the second one employs a deep neural network to predict the overall rating. The experimental results on two datasets, including a real-world dataset, show that the proposed model outperformed several state-of-the-art methods across different datasets and performance evaluation metrics.

In this paper, a revised recommendation system is constructed that ensembles deep neural network and matrix factorization under its framework and uses the explicit feedback for collaborative filtering. Recent works used deep neural network in recommendation for processing auxiliary attributes, but their interaction function is just an inner product on latent features of users and items. For modelling the recommendation system in this research, multi-layer perceptron was used to learn the interaction function. Experiments show significant decrease in MAE and RMSE to be 0.69 and 0.94 respectively, which is comparatively better than general collaborative filtering methods.

SummaryIn big data mining, the K‐prototypes has become a popular clustering method for mixed data owing to its simplicity and efficiency. However, the data clustering process of the K‐prototypes method will cause the risk of user privacy leakage because user data usually contain sensitive information. To address this issue, general solutions introduce a trusted third‐party model for privacy protection in clustering analysis, but it is difficult to find a fully trusted entity in reality. In this article, we propose a local differential privacy K‐prototypes (LDPK) mechanism, which does not require any trusted third party to perform privacy preprocessing on user data. Our mechanism first uses local differential privacy to disturb user data, then completes the clustering through the interaction between server and user. Furthermore, we propose a privacy protection enhancement mechanism (ELDPK) by extending the LDPK mechanism, which disturbs the user's clustering information in each iteration to protect user privacy further. Theoretical analysis proves the privacy and feasibility of our proposed scheme, and the experimental results prove that our proposed scheme can guarantee the quality of the clustering results under the premise of satisfying the local differential privacy.

A number of studies have recently been made on discrete distribution estimation in the local model, in which users obfuscate their personal data (e.g., location, response in a survey) by themselves and a data collector estimates a distribution of the original personal data from the obfuscated data. Unlike the centralized model, in which a trusted database administrator can access all users’ personal data, the local model does not suffer from the risk of data leakage. A representative privacy metric in this model is LDP (Local Differential Privacy), which controls the amount of information leakage by a parameter ∈ called privacy budget. When ∈ is small, a large amount of noise is added to the personal data, and therefore users’ privacy is strongly protected. However, when the number of users ℕ is small (e.g., a small-scale enterprise may not be able to collect large samples) or when most users adopt a small value of ∈, the estimation of the distribution becomes a very challenging task. The goal of this paper is to accurately estimate the distribution in the cases explained above. To achieve this goal, we focus on the EM (Expectation-Maximization) reconstruction method, which is a state-of-the-art statistical inference method, and propose a method to correct its estimation error (i.e., difference between the estimate and the true value) using the theory of Rilstone et al. We prove that the proposed method reduces the MSE (Mean Square Error) under some assumptions.We also evaluate the proposed method using three largescale datasets, two of which contain location data while the other contains census data. The results show that the proposed method significantly outperforms the EM reconstruction method in all of the datasets when ℕ or ∈ is small.

In recent years, Recommender System (RS) research work has covered a wide variety of Artificial Intelligence techniques, ranging from traditional Matrix Factorization (MF) to complex Deep Neural Networks (DNN). Traditional Collaborative Filtering (CF) recommendation methods such as MF, have limited learning capabilities as it only considers the linear combination between user and item vectors. For learning non-linear relationships, methods like Neural Collaborative Filtering (NCF) incorporate DNN into CF methods. Though, CF methods still suffer from cold start and data sparsity. This paper proposes an improved hybrid-based RS, namely Neural Matrix Factorization++ (NeuMF++), for effectively learning user and item features to improve recommendation accuracy and alleviate cold start and data sparsity. NeuMF++ is proposed by incorporating effective latent representation into NeuMF via Stacked Denoising Autoencoders (SDAE). NeuMF++ can also be seen as the fusion of GMF++ and MLP++. NeuMF is an NCF framework which associates with GMF (Generalized Matrix Factorization) and MLP (Multilayer Perceptrons). NeuMF achieves state-of-the-art results due to the integration of GMF linearity and MLP non-linearity. Concurrently, incorporating latent representations has shown tremendous improvement in GMF and MLP, which result in GMF++ and MLP++. Latent representation obtained through the SDAEs' latent space allows NeuMF++ to effectively learn user and item features, significantly enhancing its learning capability. However, sharing feature extractions among GMF++ and MLP++ in NeuMF++ might hinder its performance. Hence, allowing GMF++ and MLP++ to learn separate features provides more flexibility and greatly improves its performance. Experiments performed on a real-world dataset have demonstrated that NeuMF++ achieves an outstanding result of a test root-mean-square error of 0.8681. In future work, we can extend NeuMF++ by introducing other auxiliary information like text or images. Different neural network building blocks can also be integrated into NeuMF++ to form a more robust recommendation model.

With the continuous upgrading of smart devices, people are using smartphones more and more frequently. People not only browse the information they need on the Internet, but also more and more people get daily necessities through online shopping. Faced with a variety of recommendation systems, it becomes more and more difficult for people to keep their privacy from being collected while using them. Therefore, ensuring the privacy security of users when they use the recommendation system is increasingly becoming the focus of people. This paper summarizes the related technologies. A recommendation algorithm based on collaborative filtering, matrix factorization as well as the randomized response is proposed, which satisfies local differential privacy (LDP). Besides, this paper also discusses the key technologies used in privacy protection in the recommendation system. Besides, This paper includes the algorithm flow of the recommendation system. Finally, the experiment proves that our algorithm has higher accuracy while guaranteeing user privacy.

Collaborative filtering and deep learning based recommendation system for cold start items

The Recommender System (RS) has obtained a pivotal role in e-commerce. To improve the performance of RS, review text information has been extensively utilized. However, it is still a challenge for RS to extract the most informative feature from a tremendous amount of reviews. Another significant issue is the modeling of user–item interaction, which is rarely considered to capture high- and low-order interactions simultaneously. In this paper, we design a multi-level attention mechanism to learn the usefulness of reviews and the significance of words by Deep Neural Networks (DNN). In addition, we develop a hybrid prediction structure that integrates Factorization Machine (FM) and DNN to model low-order user–item interactions as in FM and capture the high-order interactions as in DNN. Based on these two designs, we build a Multi-level Attentional and Hybrid-prediction-based Recommender (MAHR) model for recommendation. Extensive experiments on Amazon and Yelp datasets showed that our approach provides more accurate recommendations than the state-of-the-art recommendation approaches. Furthermore, the verification experiments and explainability study, including the visualization of attention modules and the review-usefulness prediction test, also validated the reasonability of our multi-level attention mechanism and hybrid prediction.

Nowadays, people pay more and more attention to personal privacy issues, and how to collect data while protecting personal privacy and data security has become a major challenge. Local differential privacy, as a new privacy protection model, can collect and protect user data without a trusted third party. Among the many local differential privacy algorithms, RAPPOR(Randomized Aggregatable Privacy-Preserving Ordinal Response) is suitable for situations where the privacy budget is small. However, RAPPOR uses Bloom filters to map the data, which can lead to the same encoding result for different data, and the existing solution is to group the data. The paper, we propose the digital RAPPOR algorithm D-RAPPOR (Digitization-RAPPOR) does not require grouping of user data, D-RAPPOR is based on user data on the ordinal number of candidate values The D-RAPPOR algorithm is suitable for data sets with known candidate values. In this paper, the D-RAPPOR algorithm is used for experimental analysis of electricity data, which can be collected safely. The method can obtain the individual frequencies of the customer's electricity data while ensuring the security of the customer's electricity data. The control center can analyze the results of the algorithm to achieve the purpose of regulating electricity. Finally, the algorithm proposed in this paper solves the problem that the original algorithm produces the same encoding, and the results of the D-RAPPOR algorithm are slightly better than the RAPPOR algorithm under the same privacy budget.

Frequent itemset mining is a basic data mining task and has many applications in other data mining tasks. However, it is likely that the user's personal privacy information may be leaked in the mining process. In recent years, applying the local differential privacy protection model to mine all frequent itemsets is a relatively reliable and secure protection method. In local differential privacy, users first perturb the original data then send it to the aggregator, which prevents the aggregator leaking user's private information throughout the process. There are two major problems with data mining using local differential privacy, one is that the accuracy of the results is relatively low after mining, and the other is that the user transmits too much data to the server, which results in higher communication costs. In this paper, we use Hadamard response algorithm to improve the accuracy of the results while reduce the communication cost. Finally, we use FP-tree for frequent itemset mining to compare the Hadamard response with previous algorithms.

In an era of information explosion, recommender systems are vital tools to deliver personalized recommendations for users. The key of recommender systems is to forecast users' future behaviors based on previous user-item interactions. Due to their strong expressive power of capturing high-order connectivities in user-item interaction data, recent years have witnessed a rising interest in leveraging Graph Neural Networks (GNNs) to boost the prediction performance of recommender systems. Nonetheless, classic Matrix Factorization (MF) and Deep Neural Network (DNN) approaches still play an important role in real-world large-scale recommender systems due to their scalability advantages. Despite the existence of GNN-acceleration solutions, it remains an open question whether GNN-based recommender systems can scale as efficiently as classic MF and DNN methods. In this paper, we propose a Linear-Time Graph Neural Network (LTGNN) to scale up GNN-based recommender systems to achieve comparable scalability as classic MF approaches while maintaining GNNs' powerful expressiveness for superior prediction accuracy. Extensive experiments and ablation studies are presented to validate the effectiveness and scalability of the proposed algorithm. Our implementation based on PyTorch is available.

Recent advances in Large Language Models (LLMs) have been changing the paradigm of Recommender Systems (RS). However, when items in the recommendation scenarios contain rich textual information, such as product descriptions in online shopping or news headlines on social media, LLMs require longer texts to comprehensively depict the historical user behavior sequence. This poses significant challenges to LLM-based recommenders, such as over-length limitations, extensive time and space overheads, and suboptimal model performance. To this end, in this paper, we design a novel framework for harnessing Large Language Models for Text-Rich Sequential Recommendation (LLM-TRSR). Specifically, we first propose to segment the user historical behaviors and subsequently employ an LLM-based summarizer for summarizing these user behavior blocks. Particularly, drawing inspiration from the successful application of Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) models in user modeling, we introduce two unique summarization techniques in this paper, respectively hierarchical summarization and recurrent summarization. Then, we construct a prompt text encompassing the user preference summary, recent user interactions, and candidate item information into an LLM-based recommender, which is subsequently fine-tuned using Supervised Fine-Tuning (SFT) techniques to yield our final recommendation model. We also use Low-Rank Adaptation (LoRA) for Parameter-Efficient Fine-Tuning (PEFT). We conduct experiments on two public datasets, and the results clearly demonstrate the effectiveness of our approach.

With the development of social networks, the research of integrated social information recommendation models has received extensive attention. However, most existing social recommendation models are based on the matrix factorization technique which ignore the impact of the relationships between items on users’ interests, resulting in a decline of recommendation accuracy. To solve this problem, this paper proposes a double regularization matrix factorization recommendation algorithm. The algorithm first uses attribute information and manifold learning to calculate similarity. Then, the matrix factorization model is constrained through the regularization of item association relations and user social relations. Experimental results on real datasets show that the proposed method can effectively alleviate problems such as cold start and data sparsity in the recommender system and improve the recommendation accuracy compared with those of existing methods.

Recommender systems play an essential role in guiding users through overwhelming content by predicting items of interest based on prior interactions. This thesis advances shallow collaborative filtering methods—particularly those characterized by simplicity, interpretability, and efficiency—by extending their capabilities in three key directions: interactivity, probabilistic modeling, and temporal awareness. First, we propose TEASER, a transparent and explainable recommendation model that constructs user profiles in the space of item metadata. By embedding users directly in an interpretable feature space, TEASER enables real-time interaction with recommendation explanations and adjustments, facilitating enhanced user control and understanding. Offline experiments and an interactive demo application demonstrate that allowing user feedback significantly boosts performance over static models. Second, we examine the limitations of classical implicit matrix factorization models such as Weighted Matrix Factorization (WMF), particularly their treatment of unknown interactions. We introduce a probabilistic reinterpretation of WMF that preserves its scalability and effectiveness while modeling unknowns as truly missing data rather than implicit negatives. This perspective leads to improved robustness and interpretability without sacrificing performance. Finally, we extend the modeling capacity of shallow recommendation models to incorporate context and time. We explore tensor decomposition techniques, introducing novel variants such as Weighted Tensor Factorization (WTF) and time-aware adaptations that support continuous-time modeling. These approaches avoid coarse time binning and instead embed time through smooth functions, allowing recommender systems to capture subtle temporal dynamics in user behavior. Through a combination of theoretical insights, methodological innovations, and empirical evaluations across multiple datasets, this work demonstrates that shallow collaborative filtering models can be substantially enriched while maintaining their inherent benefits. The proposed models bridge the gap between performance and transparency, offering practical tools for building trustworthy and interpretable recommender systems.