When Sparsity Meets Noise in Collaborative Filtering

SummaryAccurate prediction of the solubility of chemical substances in solvents remains a challenge. The sparsity of high-quality solubility data is recognized as the biggest hurdle in the development of robust data-driven methods for practical use. Nonetheless, the effects of the quality and quantity of data on aqueous solubility predictions have not yet been scrutinized. In this study, the roles of the size and the quality of data sets on the performances of the solubility prediction models are unraveled, and the concepts of actual and observed performances are introduced. In an effort to curtail the gap between actual and observed performances, a quality-oriented data selection method, which evaluates the quality of data and extracts the most accurate part of it through statistical validation, is designed. Applying this method on the largest publicly available solubility database and using a consensus machine learning approach, a top-performing solubility prediction model is achieved.

In crowdsourced preference aggregation, it is often assumed that all the annotators are subject to a common preference or social utility function which generates their comparison behaviors in experiments. However, in reality, annotators are subject to variations due to multi-criteria, abnormal, or a mixture of such behaviors. In this paper, we propose a parsimonious mixed-effects model, which takes into account both the fixed effect that the majority of annotators follows a common linear utility model, and the random effect that some annotators might deviate from the common significantly and exhibit strongly personalized preferences. The key algorithm in this paper establishes a dynamic path from the social utility to individual variations, with different levels of sparsity on personalization. The algorithm is based on the Linearized Bregman Iterations, which leads to easy parallel implementations to meet the need of large-scale data analysis. In this unified framework, three kinds of random utility models are presented, including the basic linear model with L2 loss, Bradley-Terry model, and Thurstone-Mosteller model. The validity of these multi-level models are supported by experiments with both simulated and real-world datasets, which shows that the parsimonious multi-level models exhibit improvements in both interpretability and predictive precision compared with traditional HodgeRank.

A geometric mixed norm approach to shallow water acoustic channel estimation and tracking

As many diseases are known to be related to microbes, interests in statistical methods for Microbiome-Wide Association Studies (MWAS) are also increasing. In this respect, we systematically investigate the properties of statistical methods for MWAS and compare their performances using simulation data generated from Human Microbiome Project data. We first assessed the type I error rates of eight commonly used methods over different levels of sparsity. Among these, ANCOM and metagenomeSeq methods yielded the well preserved type I error rates regardless of sparsity. In the power comparison study, metagenomeSeq showed higher power than ANCOM. In conclusion, we recommend using metagenomeSeq for the analysis of metagenome data.

The Compressive Sensing (CS) allows the acquisition of signals already compressed and the posterior reconstruction with much less number of measures than the minimum required by the Nyquist theorem. A subarea of CS which improves the performance at the reconstruction stage is named Model-Based CS. Some works have been developed within this subarea. However, most of them consider only the noise generated by sparse approximation, disregarding the noise generated by the quantization stage and its influence on efficiency and robustness of CS. The objective of this study is to investigate the influence of the noise generated by the quantization stage in Model-Based CS efficiency for images with different levels of sparsity and different distributions of coefficients in the frequency domain. In this work, the image acquisition stage is implemented using the partial Fourier matrix which results in a vector of measures. Then, different steps of uniform scalar quantization are added to this vector and the image reconstruction stage is performed using the Compressive Sampling Matching Pursuit (CoSaMP) on a quadtree model. PSNR and bits rate (BR) are then used to evaluate the efficiency of CoSaMP with quantization noise. The performance of this proposed Model-Based CS using different quantization steps were slightly better than other studies using the same model in terms of PSNR, but with the advantage of obtaining smaller values of bit rate (BR maior que 2 bpp).

This paper deals with sparse signal reconstruction by designing a discrete-time projection neural network. Sparse signal reconstruction can be converted into an L1 -minimization problem, which can also be changed into the unconstrained basis pursuit denoising problem. To solve the L1 -minimization problem, an iterative algorithm is proposed based on the discrete-time projection neural network, and the global convergence of the algorithm is analyzed by using Lyapunov method. Experiments on sparse signal reconstruction and several popular face data sets are organized to illustrate the effectiveness and performance of the proposed algorithm. The experimental results show that the proposed algorithm is not only robust to different levels of sparsity and amplitude of signals and the noise pixels but also insensitive to the diverse values of scalar weight. Moreover, the value of the step size of the proposed algorithm is close to 1/2, thus a fast convergence rate is potentially possible. Furthermore, the proposed algorithm achieves better classification performance compared with some other algorithms for face recognition.

Unsupervised learning approaches, such as Sparse Partial Least Squares (SPLS), may provide useful insights into the brain mechanisms by finding relationships between two sets of variables (i.e. Views) from the same subjects. The algorithm outputs two sets of paired weight vectors, where each pair expresses an between both views. However, each effect can be described by a different number of variables. In this paper, we propose a novel approach to find multivariate associations between combinations of clinical/behavioural variables and brain voxels/regions which provides an unique solution with different levels of sparsity per weight vector pair. The effects described by the weight vector pairs are ranked by how much data covariance they explain. The proposed method was able to find statistically significant effects or relationships in a dementia dataset between clinical/demographic information and brain scans. Its adaptive nature allowed not only to determine an optimal sparse solution, but also provided the flexibility to select the adequate number of clinical/demographic variables and voxels to describe each effect, which enabled it to distinguish the effects associated with age from the ones associated with dementia.

A recommender system uses specific algorithms and techniques in order to suggest specific services, goods or other type of recommendations that users could be interested in. User’s preferences or ratings are used as inputs and top-N recommendations are produced by the system. The evaluation of the recommendations is usually based on accuracy metrics such as the Mean Absolute Error (MAE) and the Root Mean Squared Error (RMSE), while on the other hand Precision and Recall is used to measure the quality of the top-N recommendations. Recommender systems development has been mainly focused in the development of new recommendation algorithms. However, one of the major problems in modern offline recommendation system is the sparsity of the datasets and the selection of the suitable users Y that could produce the best recommendations for users X. In this paper, we propose an algorithm that uses Fuzzy sets and Fuzzy norms in order to evaluate the correlation between users in the data set so the system can select and use only the most relevant users. At the same time, we are extending our previous work about Reproduction of experiments in recommender systems by developing new explanations and variables for the proposed new algorithm. Our proposed approach has been experimentally evaluated using a real dataset and the results show that it is really efficient and it can increase both accuracy and quality of recommendations.

To accurately and actively provide users with their potentially interested information or services is the main task of a recommender system. Collaborative filtering is one of the most widely adopted recommender methods, whereas it is suffering the issue of sparse rating data that will severely degenerate the quality of recommendations. To address this issue, the article proposes a novel method, named the FTRA (Fusing Trust and Ratings), trying to improve the performance of collaborative filtering recommendation by means of elaborately integrating twofold sparse information, i.e., the conventional rating data given by users and the social trust network among the same users. The performance of FTRA is rigorously validated by comparing it with six representative methods on a real-world dataset. The experimental results show that the FTRA outperforms all other competitors in terms of both precision and recall. More importantly, our work suggests that the strategy of augmenting sparse rating data by fusing trust networks does significantly improve the quality of conventional collaborative filtering recommendation, and its quality could be further improved by means of designing more effective integrating schemes.

The exponential growth of online information has LED to significant challenges in navigating data overload, particularly in the tourism industry. Travelers are overwhelmed with choices regarding destinations, accommodations, dining, and attractions, making it difficult to select options that best meet their needs. Recommender systems have emerged as a promising solution to this problem, aiding users in decision-making by providing personalized suggestions based on their preferences. Traditional collaborative filtering (CF) methods, however, face limitations, such as data sparsity and reliance on single rating scores, which do not fully capture the complexity of user preferences. This study proposes a hybrid multi-criteria trust-enhanced CF (HMCTeCF) algorithm to improve the accuracy and robustness of tourism recommendations. HMCTeCF improves the quality of recommendations by integrating multi-criteria user preferences with trust relationships among users and between items. Experimental results using real-world datasets, including Restaurants-TripAdvisor and Hotels-TripAdvisor, demonstrate that HMCTeCF outperforms benchmark CF-based recommendation methods. It achieves higher prediction accuracy and coverage rate, effectively addressing the data sparsity problem. This innovative algorithm facilitates a more personalized and enriching travel experience, particularly in scenarios with limited user data. The findings highlight the importance of considering multiple criteria and trust relationships in developing robust recommendation systems for the tourism industry.

The effect of data sources and quality on the predictive capacity of CLIMEX models: An assessment of Teleonemia scrupulosa and Octotoma scabripennis for the biocontrol of Lantana camara in Australia

Currently collaborative filtering is widely used in e-commerce, digital libraries and other areas of personalized recommendation service system. Nearest-neighbor algorithm is the earliest proposed and the main collaborative filtering recommendation algorithm, but the data sparsity and cold-start problems seriously affect the recommendation quality. To solve these problems, A collaborative filtering recommendation algorithm based on users' social relationships is proposed. 0n the basis of traditional filtering recommendation technology, it combines with the interested objects of user's social relationship and takes the advantage of the tags to projects marked by users and their interested objects to improve the methods of recommendation. The experimental results of MAE ((Mean Absolute Error)) verify that this method can get better quality of recommendation.

To solve the information explosion problem and enhance user experience in various online applications, recommender systems have been developed to model users’ preferences. Although numerous efforts have been made toward more personalized recommendations, recommender systems still suffer from several challenges, such as data sparsity and cold-start problems. In recent years, generating recommendations with the knowledge graph as side information has attracted considerable interest. Such an approach can not only alleviate the above mentioned issues for a more accurate recommendation, but also provide explanations for recommended items. In this paper, we conduct a systematical survey of knowledge graph-based recommender systems. We collect recently published papers in this field, and group them into three categories, i.e., embedding-based methods, connection-based methods, and propagation-based methods. Also, we further subdivide each category according to the characteristics of these approaches. Moreover, we investigate the proposed algorithms by focusing on how the papers utilize the knowledge graph for accurate and explainable recommendation. Finally, we propose several potential research directions in this field.

(Purpose/Significance). This paper aims at the problems of inaccurate recommendation effect caused by data sparseness and cold start in the traditional collaborative filtering-based book personalized recommendation algorithm. So this paper proposes a collaborative filtering recommendation algorithm which improves the similarity solution method and the filling method of missing data. (Method/Process). By considering the influence of the user's common rating book collection on the similarity calculation, the average rating value of all books is used as the threshold, and the user's common rating weight is introduced into the user's similarity calculation. As for data filling, according to the user's average rating, the basic attributes such as the age and gender of users are coded, and then Euclidean distance is initially calculated, making hierarchical clustering on users. What's more, Shope-one algorithm is used to calculate the filling value of the former m similar users，and add the weight value of the degree simultaneously to get the final filling value, so as to improve the data filling method. (Result/Conclusion). Experiments were carried out with the data set of Book-Crossing Data set through Python. The experimental results show that the improved collaborative filtering algorithm has a significantly improvement in the accuracy and quality of book recommendation.

Online advertising, one typical application of recommendation system, calls for effective and accurate recommendations of keywords. Extreme sparse and large scale data makes online advertising a challenging problem. To achieve better performance and accuracy of the recommendation, a better model with a short turnaround time is needed. In this paper, we address the problem of personalized online advertising for extreme sparse and large scale data. We develop a novel machine learning model (Similarity Powered Pairwise Amplifier Network, SPPAN for short). The complexity of this model (a.k.a. the number of parameters) grows with the amount of observed data, which makes it suitable to extremely sparse data. The training algorithm based on gradient descent makes it easy to parallelize. The similarity model combines the user neighborhood and item neighborhood ideas in collaborative filtering smartly, obtaining a cost-effective way to handle large scale data. The proposed framework is evaluated on a large set of real-world data set from a large internet company (expressed by Company A). The experiment results demonstrate that the proposed SPPAN model can greatly improve the prediction and recommendation accuracy on that extreme sparse data set compared with existing approaches.