A Big Data Analysis Method Based on Modified Collaborative Filtering Recommendation Algorithms

Recommender system is the system which, by using automated information filtering technology, recommends products or services to the customers who are likely to be interested in. Those systems are widely used in many different Web retailers such as Amazon.com, Netfix.com, and CDNow.com. Various recommender systems have been developed. Among them, Collaborative Filtering (CF) has been known as the most successful and commonly used approach. CF identifies customers whose tastes are similar to those of a given customer, and recommends items those customers have liked in the past. Numerous CF algorithms have been developed to increase the performance of recommender systems. However, the relative performances of CF algorithms are known to be domain and data dependent. It is very time-consuming and expensive to implement and launce a CF recommender system, and also the system unsuited for the given domain provides customers with poor quality recommendations that make them easily annoyed. Therefore, predicting in advance whether the performance of CF recommender system is acceptable or not is practically important and needed. In this study, we propose a decision making guideline which helps decide whether CF is adoptable for a given application with certain transaction data characteristics. Several previous studies reported that sparsity, gray sheep, cold-start, coverage, and serendipity could affect the performance of CF, but the theoretical and empirical justification of such factors is lacking. Recently there are many studies paying attention to Social Network Analysis (SNA) as a method to analyze social relationships among people. SNA is a method to measure and visualize the linkage structure and status focusing on interaction among objects within communication group. CF analyzes the similarity among previous ratings or purchases of each customer, finds the relationships among the customers who have similarities, and then uses the relationships for recommendations. Thus CF can be modeled as a social network in which customers are nodes and purchase relationships between customers are links. Under the assumption that SNA could facilitate an exploration of the topological properties of the network structure that are implicit in transaction data for CF recommendations, we focus on density, clustering coefficient, and centralization which are ones of the most commonly used measures to capture topological properties of the social network structure. While network density, expressed as a proportion of the maximum possible number of links, captures the density of the whole network, the clustering coefficient captures the degree to which the overall network contains localized pockets of dense connectivity. Centralization reflects the extent to which connections are concentrated in a small number of nodes rather than distributed equally among all nodes. We explore how these SNA measures affect the performance of CF performance and how they interact to each other. Our experiments used sales transaction data from H department store, one of the well?known department stores in Korea. Total 396 data set were sampled to construct various types of social networks. The dependant variable measuring process consists of three steps; analysis of customer similarities, construction of a social network, and analysis of social network patterns. We used UCINET 6.0 for SNA. The experiments conducted the 3-way ANOVA which employs three SNA measures as dependant variables, and the recommendation accuracy measured by F1-measure as an independent variable. The experiments report that 1) each of three SNA measures affects the recommendation accuracy, 2) the density's effect to the performance overrides those of clustering coefficient and centralization (i.e., CF adoption is not a good decision if the density is low), and 3) however though the density is low, the performance of CF is comparatively good when the clustering coefficient is low. We expect that these experiment results help firms decide whether CF recommender system is adoptable for their business domain with certain transaction data characteristics.

Collaborative filtering (CF) recommender systems have been extensively developed and widely deployed in various social websites, promoting products or services to the users of interest. Meanwhile, work has been attempted at poisoning attacks to CF recommender systems for distorting the recommend results to reap commercial or personal gains stealthily. While existing poisoning attacks have demonstrated their effectiveness with the offline social datasets, they are impractical when applied to the real setting on online social websites. This paper develops a novel and practical poisoning attack solution toward the CF recommender systems without knowing involved specific algorithms nor historical social data information a priori. Instead of directly attacking the unknown recommender systems, our solution performs certain operations on the social websites to collect a set of sampling data for use in constructing a surrogate model for deeply learning the inherent recommendation patterns. This surrogate model can estimate the item proximities, learned by the recommender systems. By attacking the surrogate model, the corresponding solutions (for availability and target attacks) can be directly migrated to attack the original recommender systems. Extensive experiments validate the generated surrogate model's reproductive capability and demonstrate the effectiveness of our attack upon various CF recommender algorithms.

Nowadays, collaborative filtering recommender systems have been widely deployed in many commercial companies to make profit. Neighborhood‐based collaborative filtering (CF) is common and effective. To date, despite its effectiveness, there has been little effort to explore their robustness and the impact of data poisoning attacks on their performance. Can the neighborhood‐based recommender systems be easily fooled? To this end, we shed light on the robustness of neighborhood‐based recommender systems and propose a novel data poisoning attack framework, encoding the purpose of attack and constraint against them. We first illustrate how to calculate the optimal data poisoning attack, namely, UNAttack. We inject a few well‐designed fake users into the recommender systems such that target items will be recommended to as many normal users as possible. Extensive experiments are conducted on three real‐world datasets to validate the effectiveness and the transferability of our proposed method. In addition, some interesting phenomena can be found. For example, (i) neighborhood‐based recommender systems with Euclidean distance‐based similarity have strong robustness and (ii) the fake users can be transferred to attack the state‐of‐the‐art CF recommender systems such as neural CF and Bayesian personalized ranking matrix factorization.

“Shilling” attacks or “profile injection” attacks have always major challenges in collaborative filtering recommender systems (CFRSs). Many efforts have been devoted to improve collaborative filtering techniques which can eliminate the “shilling” attacks. However, most of them focused on detecting push attack or nuke attack which is rated with the highest score or lowest score on the target items. Few pay attention to grey attack when a target item is rated with a lower or higher scores than the average score, which shows a more hidden rating behavior than push or nuke attack. In this paper, we present a novel detection method to make recommender systems resistant to such attacks. To characterize grey ratings, we exploit rating deviation of item to discriminate between grey attack profiles and genuine profiles. In addition, we also employ novelty and popularity of item to construct rating series. Since it is difficult to discriminate between the rating series of attacker and genuine users, we incorporate into discrete wavelet transform (DWT) to amplify these differences based on the rating series of rating deviation, novelty and popularity, respectively. Finally, we respectively extract features from rating series of rating deviation-based, novelty-based and popularity-based by using amplitude domain analysis method and combine all clustered results as our detection results. We conduct a list of experiments on the Book-Crossing dataset in diverse attack models. Experimental results were included to validate the effectiveness of our approach in comparison with benchmarked methods.

The incorporation of collaborative filtering recommendation algorithms into blended teaching approaches for college English marks a big step forward in language education. This study investigates the theoretical underpinnings, practical ramifications, and pedagogical issues for adding collaborative filtering algorithms into blended learning environments. Collaborative filtering algorithms produce individualized learning resource recommendations based on user interactions and preferences, increasing student engagement and motivation. However, issues like as data privacy, algorithmic biases, and user approval must be addressed to ensure that collaborative filtering is implemented effectively in the classroom. This study explains the potential of collaborative filtering recommendation algorithms to improve language learning results in college English courses by doing a thorough literature review. It also provides ideas for educators and policymakers on how to handle the complexity of incorporating collaborative filtering recommendation algorithms into blended learning models. Finally, this paper adds to the promotion of novel pedagogical approaches and the improvement of language learning experiences among college students. As a result, the enhanced collaborative filtering recommendation system had a slightly greater recall rate and precision than the pre-optimized method.

Collaborative filtering recommender systems are essentially information systems which are capable of combining the judgment of a large group of people to make personalized recommendations and thereby alleviate the so-called information overload problem. However,collaborative filtering recommender systems are generally vulnerable to shilling attacks. Attackers can inject carefully chosen profiles into recommender systems in order to bias the recommendation results to their benefits. This may lead to a significant negative impact on the robustness of the systems. The main contribution of this paper is to build a probabilistic model for attack detection in the framework of probabilistic generative model. Experimental results show that this model can effectively detect shilling attacks of typical types.

Recommender system is one of the most important technologies in electronic commerce. And the collaborative filtering is almost the popular approach used in the recommender systems. With the development of electronic commerce systems, the magnitudes of users and items grow rapidly, resulted in the extreme sparsity of user rating data set. Traditional similarity measure methods work poor in this situation, make the quality of recommendation system decreased dramatically. Sparsity of users’ ratings is the major reason causing the poor quality. To address this issue, a collaborative filtering recommendation algorithm based on singular value decomposition (SVD) smoothing is presented. This approach predicts item ratings that users have not rated by the employ of SVD technology, and then uses Pearson correlation similarity measurement to find the target users’ neighbors, lastly produces the recommendations. The collaborative filtering recommendation algorithm based on SVD smoothing can alleviate the sparsity problems of the user item rating dataset, and can provide better recommendation than traditional collaborative filtering algorithms.

Collaborative filtering is one of the most widely used methods for user rating prediction in recommender systems. To evaluate a collaborative filtering system, rating datasets are typically used, which comprise thousands to millions of records consisting of user–item–rating tuples. Initially, a similarity metric is used to quantify the closeness between each user and every other user in the dataset, typically based on the ratings that each pair of users has given to the same items. Subsequently, the K users having the largest similarity to the target user are used to produce rating predictions, which lead to recommendations. A particularly challenging case arises when the rating dataset is very sparse. In this scenario, it is difficult not only to find users with commonly rated items but also to determine the optimal similarity metric and suitable values for variable K. Setting a small value for K results in extremely low prediction coverage, leading to unsuccessful recommendations, while setting a very large K value increases memory requirements and prediction/recommendation generation time. Through a multiparameter experiment, this work aims to determine the optimal settings for rating predictions when very sparse datasets are used in collaborative filtering recommender systems.

Collaborative filtering (CF) technique in recommender systems (RS) is a well-known and popular technique that exploits relationships between users or items to make product recommendations to an active user. The effectiveness of existing memory based algorithms depend on the similarity measure that is used to identify nearest neighbours. However, similarity measures utilize only the ratings of co-rated items while computing the similarity between a pair of users or items. In most of the e-commerce applications, the rating matrix is too sparse since even active users of an online system tend to rate only a few items of the entire set of items. Therefore, co-rated items among users are even sparser. Moreover, the ratings a user gives an individual item tells us nothing about his comprehensive interest without which the generated recommendations may not be satisfactory to a user. In order to be able to address these issues, a comprehensive study is made of the various existing measures of similarity in a collaborative filtering recommender system (CFRS) and a hierarchical categorization of products has been proposed to understand the interest of a user in a wider scope so as to provide better recommendations as well as to alleviate data sparsity.

Collaborative filtering (CF) is considered a powerful technique for generating personalized recommendation. However, significant vulnerabilities have recently been identified in collaborative filtering recommender systems. Malicious users can inject a large number of biased profiles into such a system in order to make recommendations that favor or disfavor given items. The average attack model is a somewhat more sophisticated attack than the random attack model. In this paper, we examine the robustness of our topic-level trust-based recommendation algorithm that incorporate topic-level trust model into classic collaborative filtering algorithm under the average attack. The results of our experiments show that topic-level trust based collaborative filtering algorithm offers significant improvements in stability over the standard k-nearest neighbor approach under average attack.

Recent literature predicts that including context in a recommender system may improve its performance. The context-based recommendation approaches are classified as pre-filtering, post-filtering and contextual modeling. Little research has been done on studying whether including context in a recommender system improves the recommendation performance and no research has compared yet the different approaches to contextual RS. The research contribution of this work lies in studying the effect of the context on the recommendation performance and comparing a pre-filtering approach to a post-filtering using a collaborative filtering recommender system.

Predicting products a customer would like on the basis of other customers ratings for these products has become a well-known approach adopted by many personalized recommendation systems on the Internet. With the development of electronic commerce, the number of customers and products grows rapidly, resulted in the sparsely of the rating dataset. Poor quality is one major challenge in collaborative filtering recommender systems. To solve this problem, slope one algorithm has a good performance. But there is also some places that not make sense. Slope one algorithm assumes that all users are at the same level, and it has neglected the individual differences. So, we propose an improved slope one algorithm, which will consider the weights of the user. Finally, we experimentally evaluate our approach and compare it to the original Slope One. The experiment shows that our method provides better recommendation results than it.

Everyday there are masses of information generated and the existence of a large amount of information makes it hardly to mining the wanted information. Personalized recommendation is the process to alleviative the problem. Collaborative filtering is one of the most popular technologies in the personal recommendation system. As the user rating matrix becoming extremely sparsity, traditional collaborative filtering recommendation algorithm calculates similarity between items using the rating data, and it does not consider the semantic relationship between different items, thus recommendation quality is very poor. To solve this problem, the paper combines the item semantic similarity and the item rating similarity, which takes into account the influence of item semantic and user rating to enhance the item-based collaborative filtering. The personalized collaborative filtering recommendation algorithm combining the item semantic similarity and item rating similarity can mitigate the sparsity problem in the electronic commerce recommender systems.

Collaborative filtering, which makes personalized predictions by learning the historical behaviors of users, is widely used in recommender systems. It makes the prediction and recommend by similarity of users, and it can handle the various work. But the traditional collaborative filtering ignores the connection of users and items. Affect the recommendation's results. To find similar users by measuring the customer relationship between the neighbors can improve the accuracy of prediction user interests'. Then it can improve the accuracy of the recommendation. So collaborative filtering recommendation algorithm based on MDP model is proposed. It can find the connection of users purchase and next purchase. So it can predict users next purchase. Then can recommend items to users. The test results shows the algorithm of this paper have more accuracy.

Recommender systems provide users with product information and suggestions, which has gradually become an important research tool in e-commerce IT technology, which has attracted a lot of attention of researchers. Collaborative filtering recommendation technology has been the most successful recommendation technology so far, but there are two major problems—recommendation quality and scalability. At present, research at home and abroad mainly focuses on recommendation quality, and there is less discussion on scalability. The scalability problem is that as the size of the system increases, the response time of the system increases to a point where users cannot afford it. Existing solutions often result in a significant drop in recommendation quality while reducing recommendation response time. In this paper, the clustering analysis subsystem based on the genetic algorithm is innovatively introduced into the traditional collaborative filtering recommendation system, and its design and implementation are given. In addition, when obtaining the nearest neighbors, only the clustered users of the target user are searched, making it a collaborative filtering recommender system based on genetic clustering. The experimental results show that the response time of the traditional collaborative filtering recommender system increases linearly with the increase in the number of users while the response time of the collaborative filtering recommender system based on genetic clustering remains unchanged with the increase in the number of users. On the other hand, the recommendation quality of the collaborative filtering recommender system based on genetic clustering is basically not degraded compared with that of the traditional collaborative filtering recommender system. Therefore, the collaborative filtering recommender system based on genetic clustering can effectively solve the scalability problem of the collaborative filtering recommender system.