A Survey of Collaborative Filtering Recommender Algorithms and Their Evaluation Metrics

Collaborative filtering (CF), a technique used by recommendation systems, predicts and recommends items (information, products or services) that the user might like. Amazon.com's recommender system is one of the most famous examples of CF. Recommendation systems are popular in both commercial and research sectors, and they are applied in a variety of applications such as movies, music, books, social connections and venues. In particular, movie recommendation systems produce personal recommendations for movies. Existing CF algorithms employed in movie recommendation systems predict the unknown rating of a given user for a movie using only the ratings (i.e., preferences) of other like-minded users who have seen the movie. In such approaches, there exist certain limits in improving the accuracy of recommendation systems. This paper proposes an algorithm for movie recommendation that exploits the genre of the movie to enhance the accuracy of rating predictions. The proposed algorithm 1) numerically measures the correlation between movie genres using movie rating information; 2) classifies movies using the genre correlations and generates a list of recommended movies for the target user with the classified movies; and finally 3) predicts the ratings of the movies in the list using traditional CF algorithms. The experimental results show that the proposed algorithm yields higher accuracy in movie rating predictions than existing movie recommendation algorithms.

Evaluation Of Recommendation System On A Movie Dataset

Introduction: Recommendation systems (RS) are pivotal in enhancing personalized user experiences, particularly in the educational domain, where tailored book recommendation systems (BRS) support individualized learning. Despite their importance, selecting an optimal recommendation algorithm remains challenging, especially in scenarios involving large and sparse datasets. Objectives: This study aims to evaluate the performance of four recommendation algorithms—Random Baseline, Popular, User-Based Collaborative Filtering (UBCF), and Singular Value Decomposition (SVD)—to identify their strengths and tradeoffs regarding accuracy and computational efficiency. The goal is to provide actionable insights for selecting appropriate algorithms for educational platforms and similar applications. Methods: The evaluation employed a dataset containing over 10,000 books, 53,424 users, and 981,756 ratings. The algorithms were assessed using Root Mean Square Error (RMSE) to measure predictive accuracy and training time to indicate computational efficiency. The analysis addressed challenges such as data sparsity and rating biases. Results: The results demonstrated that SVD achieved the highest accuracy (RMSE: 0.950), effectively uncovering latent relationships in sparse datasets. UBCF, with a slightly lower accuracy (RMSE: 1.020), offered a balance between accuracy and computational efficiency, making it suitable for real-time applications. Conversely, simpler algorithms like Random Baseline and Popular exhibited faster training times but significantly lower predictive accuracy, highlighting the limitations of non-personalized methods. Conclusions: This study underscores the tradeoffs between accuracy and efficiency among recommendation algorithms. While SVD is ideal for accuracy-driven applications, UBCF provides a practical alternative for scenarios requiring computational efficiency. The findings have substantial implications for educational platforms and e-commerce, where personalized recommendations enhance user satisfaction and engagement. Future research should focus on integrating deep learning models and expanding evaluation criteria, including user satisfaction and diversity, to further improve the performance of recommendation systems.

The necessity for sophisticated recommender systems in the movie recommendation sphere has become particularly pronounced, generating a more personalized movie recommendation due to people nowadays who like to watch movies online. Efficient recommender systems make use of advanced machine learning (ML) techniques in the pursuit of accurate and meaningful recommendations. This paper endeavours to give a comprehensive overview of technologies known as recommender systems, concentrating on ML methods found at the base. Different strategies have been applied in this work, which include collaborative filtering (CF), content-based filtering (CB), hybrid approaches, Generative AI and so on. The merits and demerits of each technique are listed and explained briefly. In addition, the actual application’s results are also presented in this paper. To evaluate the performance of the techniques, some of the important datasets that are used in evaluating recommender systems are also discussed along with measurement metrics to determine the effectiveness of technique. Example metrics used are Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and so on. This paper synthesizes existing research to evaluate the advantages and limitations of diverse recommendation techniques, which aims to give suggestion on how to improve the design of movie recommendation systems so that the performance of the technique can be improved.

Collaborative Filtering (CF) Recommendation System is a prominent technology which is widely online. Large variety of available CF algorithms and the multitude of their possible parameters have a huge impact on quality of the outcome on ECommerce. Unfortunately, the literature on CF recommender system evaluation presents different evaluation metrics at different situation and could not provide any suggestion about the best. At the same time, it is a fact to accept that the accuracy measure should be different for each CF algorithm and depends upon the classification accuracy of that particular algorithm. As an initiative to address this problem in the present research paper, predictive accuracy metrics, classification accuracy metrics and rank accuracy metrics are considered as the classification accuracy metrics to know the overall competence for significant features of the chosen CF algorithms. At this juncture, normalization, a distinctive evaluation methodology, has been adopted to accomplish unique evaluation results of recommender systems. In this research paper, different accuracy metrics assessment would be brought into a common scale by taking into consideration of normalization process to evaluate metrics of the CF algorithms. A comprehensive comparative analysis is carried out and tabulated.

Because of its simplicity and effectiveness, collaborative filtering (CF) became one of the most successful recommendation algorithms. User-based CF is one classic method of CF algorithms. In order to solve the problem that common rating items are often too few to be used to effectively calculate the similarity of two users in user-based CF, we proposed an improved collaborative filtering model with item similarity called ISCF in this paper. In ISCF model, the similarity of items was considered in user-based collaborative filtering, which contributes to alleviate the problem of data sparsity and therefore calculate the similarity of user. Experimental results illustrate that our approach ISCF outperforms the average method and user-based CF. Compared with user-based CF, the average improvement in the percentage of ISCF at MAE and RMSE are 21.9% and 17.7%, respectively. In addition, our approach ISCF can predict more items than user-based CF, and the average improvement in the percentage of ISCF at prediction diversity is 33.86%.

Recommender systems are powerful tools that filter and recommend content/information relevant to a given user. Collaborative filtering is the most popular technique used in building recommender systems and it has been successfully incorporated in many applications. These conventional recommendation systems require a minimum number of users, items, and ratings in order to provide effective recommendations. This results in the infamous cold-start problem where the system is not able to produce effective recommendations for new users. Recently, there has been an escalation in the popularity and usage of social networks, which persuades people to share their experiences in the form of reviews and ratings on social media. The components of social media such as the influence of friends, interests, and enjoyment create the opportunities to develop solutions for sparsity and cold start problems of recommendation systems. This paper aims to observe these patterns and analyze three of the existing social recommendation systems, SocialMF, SocialFD, and GraphRec. SocialMF and SocialFD algorithms are based on matrix factorization and distance metric learning respectively whereas GraphRec is an attention based deep learning model. Through extensive experimentation with the datasets that these algorithms were tested on and one new dataset, we compared the results based on evaluation metrics including Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). To investigate how trust impacts the performance of these models, we evaluated them by modifying the trust and social component. Experimental results show that there is no conclusive evidence that trust propagation plays a major part in these models. Moreover, these models show a slightly improved performance in the absence of trust statements.

Collaborative filtering recommendation algorithm is the most important recommended application methods at present. The selection of users (items) similarity measurement methods in this method may affect the final recommendation effect. Probability matrix factorization algorithm is one of the more and more methods applied in collaborative filtering algorithms. This paper proposes to convert the users (items) consumption network into a bipartite graph and use belief propagation algorithm to obtain fuzzy nearest neighbor set, improving users (items) similarity measure and incorporating it into probability matrix factorization. Experimental tests on the data set showed that the proposed method performance was good.

The popularity of movies has increased in recent years. There are thousands of films produced each year. These films make it challenging for movie lovers to pick the ideal film to see. We propose a recommendation system that strives to offer guidance in selecting films. Depending on the method employed, recommendation systems can be categorized into three groups: collaborative filtering, content-based filtering, and hybrid filtering. In this work, collaborative filtering, one of the methods frequently used in recommendation systems was used. There are two ways to the Collaborative Filtering approach: User-Based Collaborative Filtering (UBCF) and Item-Based Collaborative Filtering (IBCF). There are two methods for finding similar items or users: Cosine and Pearson similarities. The Cosine similarity approach is one way to determine how similar two items are. Additionally, the Pearson Correlation Coefficient approach, which determines similarities between objects by calculating linear correlations between two sets, is the most widely employed. This study aims to determine which system produces the highest item similarity in IBCF and predicted ratings to actual ratings using 90% training and 10% testing data. The data set taken from MovieLens.org consists of 943 users from 1664 movies with 99392 ratings. The MovieLens data collection will be analyzed with the RStudio and the R package recommenderlab. The results reveal that the IBCF with Cosine similarities shows the number of items recommended n top-rated movies to each user for 10 movies. The IBCF can identify the most recommended films and creates a frequency distribution of items.

In this paper, we focus on how to overcome several limitations in the traditional research of collaborative filtering(CF). We present a novel CF recommendation algorithm, named DSNP(Dynamic Similar Neighbor Probability). This algorithm improves the neighbors’ similarities computations of both users and items to choose the neighbors dynamically as the recommendation sets. How to select the confident subsets which are the most effective neighbors to the target object, it is the first stage. A major innovation is by defining a dynamic neighbor probability over the trustworthy subsets. Moreover, we define a prediction algorithm that combines the advantages of dynamic neighbor coefficient with the user-based CF and the item-based CF algorithms. Experimental results show that the algorithm can achieve consistently better prediction accuracy than traditional CF algorithms, and effectively leverage the result between user-based CF and item-based CF. Furthermore, the algorithm can alleviate the dataset sparsity problem.

Recommender system is the system which, by using automated information filtering technology, recommends products or services to the customers who are likely to be interested in. Those systems are widely used in many different Web retailers such as Amazon.com, Netfix.com, and CDNow.com. Various recommender systems have been developed. Among them, Collaborative Filtering (CF) has been known as the most successful and commonly used approach. CF identifies customers whose tastes are similar to those of a given customer, and recommends items those customers have liked in the past. Numerous CF algorithms have been developed to increase the performance of recommender systems. However, the relative performances of CF algorithms are known to be domain and data dependent. It is very time-consuming and expensive to implement and launce a CF recommender system, and also the system unsuited for the given domain provides customers with poor quality recommendations that make them easily annoyed. Therefore, predicting in advance whether the performance of CF recommender system is acceptable or not is practically important and needed. In this study, we propose a decision making guideline which helps decide whether CF is adoptable for a given application with certain transaction data characteristics. Several previous studies reported that sparsity, gray sheep, cold-start, coverage, and serendipity could affect the performance of CF, but the theoretical and empirical justification of such factors is lacking. Recently there are many studies paying attention to Social Network Analysis (SNA) as a method to analyze social relationships among people. SNA is a method to measure and visualize the linkage structure and status focusing on interaction among objects within communication group. CF analyzes the similarity among previous ratings or purchases of each customer, finds the relationships among the customers who have similarities, and then uses the relationships for recommendations. Thus CF can be modeled as a social network in which customers are nodes and purchase relationships between customers are links. Under the assumption that SNA could facilitate an exploration of the topological properties of the network structure that are implicit in transaction data for CF recommendations, we focus on density, clustering coefficient, and centralization which are ones of the most commonly used measures to capture topological properties of the social network structure. While network density, expressed as a proportion of the maximum possible number of links, captures the density of the whole network, the clustering coefficient captures the degree to which the overall network contains localized pockets of dense connectivity. Centralization reflects the extent to which connections are concentrated in a small number of nodes rather than distributed equally among all nodes. We explore how these SNA measures affect the performance of CF performance and how they interact to each other. Our experiments used sales transaction data from H department store, one of the well?known department stores in Korea. Total 396 data set were sampled to construct various types of social networks. The dependant variable measuring process consists of three steps; analysis of customer similarities, construction of a social network, and analysis of social network patterns. We used UCINET 6.0 for SNA. The experiments conducted the 3-way ANOVA which employs three SNA measures as dependant variables, and the recommendation accuracy measured by F1-measure as an independent variable. The experiments report that 1) each of three SNA measures affects the recommendation accuracy, 2) the density's effect to the performance overrides those of clustering coefficient and centralization (i.e., CF adoption is not a good decision if the density is low), and 3) however though the density is low, the performance of CF is comparatively good when the clustering coefficient is low. We expect that these experiment results help firms decide whether CF recommender system is adoptable for their business domain with certain transaction data characteristics.

Recommendations Systems (RS) are widely used by E-commerce and other web sites to recommend items to its users. Collaborating Filtering (CF) is very successful technique used in many web sites for recommendation. Many hybrid techniques with collaborative filtering are proposed in recent years for increasing the performance of recommendation system. In this paper a new user similarity based algorithm is proposed for recommendation system. The algorithm is based upon similarity between the users. The proposed algorithm is applied on a sample data set and its performance is calculated. The value of Mean Absolute Error (MAE) for the proposed algorithm is very reasonable.

Collaborative and content-based filtering are the majorly used approaches towards implementing recommender systems which can successfully predict the item(s) to be recommended to users based upon their preferences. Each of the methods has its own advantages and disadvantages that are specific to the situation they are used in. The hybrid approach aims to combine both techniques in multiple ways to optimize results and overcome shortcomings. In this paper, we propose a hybrid approach based recommendation engine to improve accuracy. While content based recommendations are made on the similarity of items’ attributes, collaborative recommendations are based on similarity of users. Collaborative filtering is achieved by matrix factorization technique. One of the most effective algorithms for matrix factorization: Singular Value Decomposition (SVD) has been used to perform collaborative filtering and combined with content based model to predict the item ratings per user. The paper aims to achieve the final result with minimum possible errors as per user preferences and recommended items.

The goal of a recommender system is to suggest items of interest to a user based on historical behavior of a community of users. Given detailed enough history, item-based collaborative filtering (CF) often performs as well or better than almost any other recommendation method. However, in cold-start situations - where a user, an item, or the entire system is new - simple non-personalized recommendations often fare better. We improve the scalability and performance of a previous approach to handling cold-start situations that uses filterbots, or surrogate users that rate items based only on user or item attributes. We show that introducing a very small number of simple filterbots helps make CF algorithms more robust. In particular, adding just seven global filterbots improves both user-based and item-based CF in cold-start user, cold-start item, and cold-start system settings. Performance is better when data is scarce, performance is no worse when data is plentiful, and algorithm efficiency is negligibly affected. We systematically compare a non-personalized baseline, user-based CF, item-based CF, and our bot-augmented user- and item-based CF algorithms using three data sets (Yahoo! Movies, MovieLens, and EachMovie) with the normalized MAE metric in three types of cold-start situations. The advantage of our naive filterbot approach is most pronounced for the Yahoo! data, the sparsest of the three data sets.

Nowadays, a recommendation system is an important technique in the development of electronic-commerce systems and the most popular approaches that use in many recommendation systems are a collaborative filtering algorithm. However, it still has problems such as scalability and sparse data. There are several previous methods used to deal with the weakness of collaborative filtering techniques such as a hybrid user model, but the results show their disadvantages in practical use. In this paper, we proposed a hybrid recommender system with review helpfulness features, which we have used to construct the hybrid model. In the experiment, the results of three recommendation techniques are compared: collaborative filtering based on hybrid user model, user-based collaborative filtering and our proposed method. The experimental results show that our proposed method is more efficient than other methods with the same dataset, in terms of accuracy. In addition, our proposed algorithm can decrease time consuming by constructing the hybrid model in the off-line phase and calculates the recommendation results in on-line phase.