Personalized Music Recommendation Algorithm Based on Spark Platform

With the arrival of the era of big data, recommendation algorithm has become a hot research field in internet life and social business. However, the traditional recommendation algorithms have problems of cold start, scoring matrix data sparse, affecting the accuracy of recommendation. This paper proposes a hybrid recommendation algorithm based on the SVD (singular value decomposition) and Slope One algorithm, experiments results show that the algorithm proposed by this paper has superior performance compared with the traditional collaborative filtering recommendation algorithm. Keywords-SVD; Slope-One; Recommendation Algorithm; Collaborative Filtering

This paper proposes a model of agricultural product purchase intention based on live broadcast AI and recommendation algorithm (LiveRec). By combining AI behavior analysis in live broadcast with traditional recommendation algorithm, this model can adjust the recommendation strategy in real time according to the user's viewing behavior, interaction, comments and other information, thereby improving the user's purchase intention. This paper makes use of the live broadcast data from the e-commerce platform of agricultural products, including user behavior data and sales records, to evaluate the effectiveness of the model in the forecast of buying intention and the exchange rate. Experiments show that the precision of the prediction is increased by 18 %, and the conversion ratio is 15 % higher than that of conventional logistic regression and SVM. In particular, the model has good adaptability and real time performance in the user's personalized recommendation. The study indicates that the model based on live AI and recommended algorithm can increase the sales efficiency of agriculture products, and it will have a bright future. Future research will further optimize the model's real time performance and processing efficiency, and extend its application to other electronic commerce.

Random group recommendation model based on fuzzy clustering

Aiming at the problem that the traditional Collaborative Filtering algorithm has low recommendation accuracy, in the paper, we propose a collaborative filtering recommendation algorithm based on the global trust degree integrating the direct trust information in the social networks.We first transform the local trust relationships to the global trust relationship by the rules in the trust network, and get the trust rank of all users in the trust networks; Then we use the global trust value to instead of the similarity information value as the weights of a predicted formula in the traditional collaborative recommendation algorithm, and integrate the weights to the matrix factorization-based recommendation model.

Collaborative filtering, which plays an important role in making personalized recommendation, is one of the most traditional and effective recommendation algorithms. However, this algorithm suffers the sparse user rating record matrix problem which would result in poor recommendation precision. A usual approach to alleviate this problem is filling empty values with user average rating value. This approach solve the sparse matrix problem to some degree, but the inaccuracy of the filling values seriously impact the veracity of recommendation. To further enhance the recommending precision, this paper propose a new method of user-based collaborative filtering based on predictive value padding. This algorithm would predict the empty values in user-item matrix by integrating content-based recommendation algorithm and user activity level before calculating user similarity. It considers both the role of user and the item attributes in order to make a more accurate prediction. Experimental results on movie-lens dataset has shown that our new algorithm improves recommendation accuracy significantly compared with traditional user-based collaborative filtering algorithm and has an obvious advantage over the recommendation result after padding with average rating value as well.

The traditional collaborative filtering recommendation algorithm can be divided into the user-based and the item-based two methods, which only uses the information in the rating matrix. Because of the limitation of the information capacity they used, it is difficult to further improve the accuracy of the recommendation, and cold start problem also affects the normal operation of the recommendation system. This paper presented a collaborative filtering recommendation algorithm (UISA) fusing user-based, item-based and social networks data. The algorithm uses the data of the neighbor relations in social networks, calculating the users' friends not reflected in the rating matrix. At the same time, we can calculate the similarity between items by using the data of item text in social networks, mining similar items not reflected in the rating matrix. In this way, it can fundamentally expand available information capacity of the traditional filtering collaboration recommendation algorithms, improve the recommendation accuracy, alleviate cold start problem. Experimental results based on KDD CUP 2012 real data show that compared with the traditional collaborative filtering system, this system has obvious advantages in the recommendation accuracy and ease of cold start.

In the era of the Internet, all people's behaviors on the Internet will be stored in the server in the form of data, which leads to the image level growth of data today. In the context of big data, how to effectively analyze various existing data to obtain the necessary information is an urgent challenge that various industries need to overcome. Recommendation algorithms are one of them, mainly using existing data to recommend information of interest to users. In traditional recommendation algorithms, collaborative filtering recommendation algorithms encounter difficulties such as cold start and data sparsity. In order to better explore data features, deep learning algorithms have begun to be applied. Recurrent neural networks can not only learn input data but also perform self-learning, which can better extract features between data and improve the accuracy of recommendations. However, the information that users are interested in is greatly influenced by time, in order to improve the accuracy of recommendations. This study investigates the recursive neural network recommendation algorithm with added time series, and experiments have shown that this recommendation algorithm can indeed improve the accuracy of recommendations more accurately.

In view of the problem of how to improve the accuracy of education resources recommendation in Internet era, a recommendation algorithm based on implicit feedback and pairwise education resources are proposed. Firstly, the feature extraction of learner-education resource is carried out, and the implicit features of learner-education resource are extracted by implicit semantic model, and the prediction results of factor decomposition are used as input to sorting learning algorithm. Converting sorting problems to classification problems uses the theory of ordering learning. Based on the experiment comparison with the traditional recommendation algorithm, it is compared with the comparison algorithm, which has better effect on the recommendation of educational resources. The algorithm increases by nearly 30% on metrics MAP, NDGG.

A key challenge of the collaborative filtering (CF) information filtering is how to obtain the reliable and accurate results with the help of peers' recommendation. Since the similarities from small-degree users to large-degree users would be larger than the ones in opposite direction, the large-degree users' selections are recommended extensively by the traditional second-order CF algorithms. By considering the users' similarity direction and the second-order correlations to depress the influence of mainstream preferences, we present the directed second-order CF (HDCF) algorithm specifically to address the challenge of accuracy and diversity of the CF algorithm. The numerical results for two benchmark data sets, MovieLens and Netflix, show that the accuracy of the new algorithm outperforms the state-of-the-art CF algorithms. Comparing with the CF algorithm based on random walks proposed by Liu et al. (Int. J. Mod. Phys. C, 20 (2009) 285) the average ranking score could reach 0.0767 and 0.0402, which is enhanced by 27.3% and 19.1% for MovieLens and Netflix, respectively. In addition, the diversity, precision and recall are also enhanced greatly. Without relying on any context-specific information, tuning the similarity direction of CF algorithms could obtain accurate and diverse recommendations. This work suggests that the user similarity direction is an important factor to improve the personalized recommendation performance.

With the development of economy, tourism is favored by more and more people, and location-based point-of-interest recommendation (POI) begins to emerge. This paper proposes the lstm-based recommendation algorithm for cross-attention mechanism (L-Attention). Embedding method was used to accurately learn the location sequence and location information of users, and feature information of location data was extracted through LSTM network. At the same time, the cross-attention mechanism is introduced to conduct dynamic modeling of personalized time check-in sequence, which can integrate the user's behavioral preference, location sequence information and the influence of sequential location on next-poi, so as to improve the accuracy of recommendation. This article uses Foursquare as an experimental data set. The results show that compared with the traditional recommendation algorithm which only considers the popularity of POI and the deep learning recommendation algorithm which only considers the timing, the accuracy of the proposed algorithm is greatly improved.

With the development of science and technology, many optimization problems in real life have developed into high-dimensional optimization problems. The meta-heuristic optimization algorithm is regarded as an effective method to solve high-dimensional optimization problems. However, considering that traditional meta-heuristic optimization algorithms generally have problems such as low solution accuracy and slow convergence speed when solving high-dimensional optimization problems, an adaptive dual-population collaborative chicken swarm optimization (ADPCCSO) algorithm is proposed in this paper, which provides a new idea for solving high-dimensional optimization problems. First, in order to balance the algorithm’s search abilities in terms of breadth and depth, the value of parameter G is given by an adaptive dynamic adjustment method. Second, in this paper, a foraging-behavior-improvement strategy is utilized to improve the algorithm’s solution accuracy and depth-optimization ability. Third, the artificial fish swarm algorithm (AFSA) is introduced to construct a dual-population collaborative optimization strategy based on chicken swarms and artificial fish swarms, so as to improve the algorithm’s ability to jump out of local extrema. The simulation experiments on the 17 benchmark functions preliminarily show that the ADPCCSO algorithm is superior to some swarm-intelligence algorithms such as the artificial fish swarm algorithm (AFSA), the artificial bee colony (ABC) algorithm, and the particle swarm optimization (PSO) algorithm in terms of solution accuracy and convergence performance. In addition, the APDCCSO algorithm is also utilized in the parameter estimation problem of the Richards model to further verify its performance.

With the diversification of applications and the diversity of users, the era of big data is always producing various new types of data, such as graph data, time series data, and spatial data. When predicting the future trend of the system, we always need to learn from the historical working state, and time series data plays a huge value in this aspect. Hardware resources are becoming cheaper, computing power has made major breakthroughs, and multi-machine and multi-core scenarios have become commonplace. At present, the focus of large-scale time series data processing is either focused on data collection and storage, or on single-machine processing, which has been unable to meet the existing needs. In response to this problem, this paper designs and implements a distributed large-scale time series processing and analysis system based on the Spark platform. The system framework is mainly divided into storage layer, operator layer and algorithm layer. At the storage layer, the system completes the organization and indexing of large-scale time series data based on HDFS and Hive. At the operator layer, the system provides users with basic operations commonly used for time series data on the Spark platform, and allows users to directly use these operators to implement custom time series related processing algorithms. At the algorithm level, the system implements some commonly used time series analysis algorithms in the Spark platform, including time series similarity query, clustering and prediction, so that users can directly use these algorithms to analyze time series. Finally, the feasibility and practicability of this system are verified by testing the performance and function of the system.

With the development of cloud computing and distributed cluster technology, the concept of big data has been expanded and extended in terms of capacity and value, and machine learning technology has also received unprecedented attention in recent years. Traditional machine learning algorithms cannot solve the problem of effective parallelization, so a parallelization support vector machine based on Spark big data platform is proposed. Firstly, the big data platform is designed with Lambda architecture, which is divided into three layers: Batch Layer, Serving Layer, and Speed Layer. Secondly, in order to improve the training efficiency of support vector machines on large-scale data, when merging two support vector machines, the “special points” other than support vectors are considered, that is, the points where the nonsupport vectors in one subset violate the training results of the other subset, and a cross-validation merging algorithm is proposed. Then, a parallelized support vector machine based on cross-validation is proposed, and the parallelization process of the support vector machine is realized on the Spark platform. Finally, experiments on different datasets verify the effectiveness and stability of the proposed method. Experimental results show that the proposed parallelized support vector machine has outstanding performance in speed-up ratio, training time, and prediction accuracy.

In view of the sparsity of user-resource interaction data, information asymmetry caused by cold start, and weak personalized resource allocation in the current integration of enterprises with vocational schools and universities, this article adopts the Collaborative Filtering (CF) algorithm to mine the user’s behavior patterns and preferences, establish a more intelligent resource recommendation model, and enhance the relevance and accuracy of resource recommendation. The intelligent resource recommendation system plays a key role in the construction of smart cities, fully optimizing the interaction between educational institutions and enterprises, and providing strong support for promoting the development of smart cities. First, an embedded feature expression for users and resources is established, and the deep embedding learning method is used to extract the association between high-dimensional and nonlinear features. Secondly, based on the traditional CF algorithm, the neural network architecture is combined to achieve accurate matching between users and resources. Finally, through experiments in user cold start and resource cold start scenarios, the performance improvement of the recommendation system is verified and compared with the widely used Matrix Factorization (MF) and Neural Collaborative Filtering (NCF) for analysis. According to the experimental results, the average precision of the proposed method is 0.804, which is 7.34% and 5.51% higher than MF and NCF respectively. The conclusion shows that the CF algorithm helps to enhance the resource recommendation effect and provides a new perspective for enhancing service personalization.

CF (Collaborative filtering) algorithm has the widest and most successful applications in personalized recommendations. However, due to its over-reliance on the users' historical data, it is difficult to avoid data sparseness and cold start issues. The data sparseness and cold start may cause poor recommendation accuracy of the collaborative filtering algorithm. A hybrid optimal collaborative filtering algorithm based on users' characteristics and trust is proposed in this paper. In the process of users' similarity calculation, the age and gender of users' characteristics are introduced to make the determination of nearest neighbor more accurate. Besides, in order to improve the recommendation accuracy of the traditional CF recommendation algorithm, the trust relationship is introduced into the prediction score by measuring the users' trust, and this improvement will be used in the recommendation of new items in order to improve the recommendation accuracy of the traditional CF recommendation algorithm. The experimental results of Movie lens data set show that the improved recommendation accuracy of the recommendation system can be achieved by the proposed algorithm. Also, the problems of cold start and sparse data can be solved effectively.