An Iterative Distance-Based Model for Unsupervised Weighted Rank Aggregation

Learning the true rank ordering among objects by aggregating a set of expert opinion rank order lists is an important and ubiquitous problem in many applications ranging from social choice theory to recommendation systems and search aggregation. We study the problem of unsupervised rank aggregation where no ground truth ordering information in available, neither about the true preference ordering among any set of objects nor about the quality of individual rank lists. Aggregating the often inconsistent and poor quality rank lists in such an unsupervised manner is a highly challenging problem, and standard consensus-based methods fall short in terms of both quality and scalability. In this manuscript we propose a novel framework to bypass these issues by using object attributes to augment the standard rank aggregation framework. We design algorithms that learn joint models on both rank lists and object features to obtain an aggregated rank ordering that is more accurate and robust, and also helps weed out rank lists of dubious validity. We validate our techniques on synthetic datasets where our algorithm is able to estimate the true rank ordering even when the rank lists are corrupted. Experiments on three real datasets, MQ2008, MQ2007 and OHSUMED, show that using object features can result in significant improvement in performance over existing rank aggregation methods that do not use object information. Furthermore, when at least some of the rank lists are of high quality, our methods are able to effectively exploit such information to output an aggregated rank ordering of high accuracy.

The problem of interpreting or aggregating multiple rankings is common to many real-world applications. Perhaps the simplest and most common approach is a weighted rank aggregation, wherein a (convex) weight is applied to each input ranking and then ordered. This paper describes a new tool for visualizing and displaying ranking information for the weighted rank aggregation method. Traditionally, the aim of rank aggregation is to summarize the information from the input rankings and provide one final ranking that hopefully represents a more accurate or truthful result than any one input ranking. While such an aggregated ranking is, and clearly has been, useful to many applications, it also obscures information. In this paper, we show the wealth of information that is available for the weighted rank aggregation problem due to its structure. We apply weight set decomposition to the set of convex multipliers, study the properties useful for understanding this decomposition, and visualize the indifference regions. This methodology reveals information–that is otherwise collapsed by the aggregated ranking–into a useful, interpretable, and intuitive decision support tool. Included are multiple illustrative examples, along with heuristic and exact algorithms for computing the weight set decomposition.

Rank aggregation (RA), the process of combining multiple ranked lists into a single ranking, has played an important role in integrating information from individual genomic studies that address the same biological question. In previous research, attention has been focused on aggregating full lists. However, partial and/or top ranked lists are prevalent because of the great heterogeneity of genomic studies and limited resources for follow-up investigation. To be able to handle such lists, some ad hoc adjustments have been suggested in the past, but how RA methods perform on them (after the adjustments) has never been fully evaluated. In this article, a systematic framework is proposed to define different situations that may occur based on the nature of individually ranked lists. A comprehensive simulation study is conducted to examine the performance characteristics of a collection of existing RA methods that are suitable for genomic applications under various settings simulated to mimic practical situations. A non-small cell lung cancer data example is provided for further comparison. Based on our numerical results, general guidelines about which methods perform the best/worst, and under what conditions, are provided. Also, we discuss key factors that substantially affect the performance of the different methods.

In credit risk evaluation the accuracy of a classifier is very significant for classifying the high-risk loan applicants correctly. Feature selection is one way of improving the accuracy of a classifier. It provides the classifier with important and relevant features for model development. This study uses the ensemble of multiple feature ranking techniques for feature selection of credit data. It uses five individual rank based feature selection methods. It proposes a novel rank aggregation algorithm for combining the ranks of the individual feature selection methods of the ensemble. This algorithm uses the rank order along with the rank score of the features in the ranked list of each feature selection method for rank aggregation. The ensemble of multiple feature selection techniques uses the novel rank aggregation algorithm and selects the relevant features using the 80%, 60%, 40% and 20% thresholds from the top of the aggregated ranked list for building the C4.5, MLP, C4.5 based Bagging and MLP based Bagging models. It was observed that the performance of models using the ensemble of multiple feature selection techniques is better than the performance of 5 individual rank based feature selection methods. The average performance of all the models was observed as best for the ensemble of feature selection techniques at 60% threshold. Also, the bagging based models outperformed the individual models most significantly for the 60% threshold. This increase in performance is more significant from the fact that the number of features were reduced by 40% for building the highest performing models. This reduces the data dimensions and hence the overall data size phenomenally for model building. The use of the ensemble of feature selection techniques using the novel aggregation algorithm provided more accurate models which are simpler, faster and easy to interpret.

Normally, statistical methods are used to generate rankings for genes in terms of their ability to distinguish between normal and malignant tumors from a gene expression dataset. However, different statistical methods yield different ranks for same gene and there is no universally accepted method for ranking. Therefore rank aggregation is required to find the overall ranking of the set of genes. There are various rank aggregation methods in the existing literature to integrate the rankings produced by various statistical tests. Moreover, the problem of integration of some partial rankings, containing unequal numbers of genes, is more challenging. In this article, a multiobjective genetic algorithm based rank aggregation method is proposed to integrate some partial rankings in an unbiased way. The first objective is to minimize the total distance from the reference ranking to the input rankings. For distance calculation, the Scaled Footrule Distance is used. The second objective is to minimize the standard deviation among those distances in order to avoid bias toward a particular input ranking. The proposed method is applied on some real-life microarray gene expression datasets, and the performance of it is compared with that of several existing rank aggregation techniques with respect to accuracy and the AUC (Area under ROC curve) value. Again, for real-life datasets, accuracy is plotted for visual comparison.

Aggregating multiple rankings in a database is an important task well studied by the database community. High-stakes application domains include hiring, lending, and education where multiple decision makers rank candidates and their input is then combined into an overall consensus ranking. While state-of-art aggregation methods succeed in giving equal say to each decision maker, to date no methods ensure fair representation for groups of candidates being ranked, who risk being impacted by discriminatory bias. We present the first solution to this open problem of guaranteeing fairness for disadvantaged groups of candidates, while still producing a good consensus of the base rankings. We design a family of exact fair algorithms achieving optimality for fair rank aggregation. We also develop approximate methods achieving fairness with guaranteed minimal approximation error scaling to millions of candidates in the rankings. A comparative study evaluates our proposed methods, revealing trade-offs between aggregation accuracy and different degrees of unfair bias in a rich variety of rank aggregation scenarios. Our real-world case study demonstrates that our solutions mitigate unfair bias using real-world data.

Unsupervised deep learning and semi-automatic data labeling in weed discrimination

For a given query and a set of images ranked lists retrieved from multiple search engines, the metasearch technique aims at combining these lists to build an unified ranking with improved relevance. Rank aggregation is an approach that has been widely used to support this task. This paper investigates the use of rank aggregation methods in the metasearch scenario for diverse image retrieval. Although metasearch systems are usually driven by the relevance of the final result, the impact on diversification has also been analyzed. The experimental findings suggest metasearch based on rank aggregation allows significant improvements, both in terms of relevance and diversity.

Existing methods dealing with the problem of rank aggregation in the context of meta-search in information retrieval are considered as a passive learner machine and suffer in the presence of unreliable ranking lists. This paper proposes a novel approach which selects the most informative features and instances to be labeled from which the model will learn. To train an efficient ranking aggregation model from few labeled instances, we use Multiple Hyperplane Ranker algorithm in an active learning environment. Experimental results on the OHSUMED dataset show that our method outperforms the existing methods.

Jamming attack detection in a pair of RF communicating vehicles using unsupervised machine learning

Rank aggregation has recently become a common approach for combining individual rankings into a consensus and for quantifying and improving performance in various applications, such as elections, web page rankings, and sports. During the past few years, rankings from many sources have become increasingly high-dimensional and partial. In this study, we develop a rank aggregation method by constructing a directed weighted competition graph. We introduce the concept of “ratio of out- and in-degrees (ROID)” to transform high-dimensional partial rankings into a single consensus. Moreover, we provide a novel effectiveness measure for the aggregate ranking according to its deviations from the ground truth ranking. The proposed method is compared with four typical methods with synthetic rankings. The results indicate that our method outperforms the other four by a significant margin and can be particularly efficient in aggregating high-dimensional rankings. The empirical results validate the effectiveness and feasibility of our method.

Data-driven turbulence modeling is currently a popular approach in constructing new RANS models with improved performance. Even though data-driven models have generally failed to extrapolate outside training cases well, they harbor potential in design-space exploration. Currently the most popular data-driven models typically employ a corrective approach, where a Reynolds stress correction improves upon an existing precursor RANS simulation without coupling with the RANS solver. Corrective models are trained on data from a RANS simulation and thus are limited in predictive capability and accuracy. On the other hand, the rarely used iterative models utilize DNS data for training and completely couple with the solver, and thus they have the potential to provide more accuracy. In this article, we propose the use of model derived turbulence variables to train iterative data-driven models. The model derived variables are arrived at by freezing the flow variables on the grid and solving turbulent transport equations to convergence. A model form is arrived that is Galilean invariant, frame invariant and unit invariant for a particular design space in consideration. A dense feed-forward neural network is used as a surrogate model for the functional mapping in the model. Numerical results for the popular periodic hills benchmark problem are presented to demonstrate the effectiveness of iterative models for design space exploration.

Person re-identification aims to match different persons observed in non-overlapping camera views. Researchers have proposed many person descriptors based on global or local descriptions, while both of them have achieved satisfying matching results, however, their ranking lists usually vary a lot for the same query person. These motivate us to investigate an approach to aggregate them to optimize the original matching results. In this paper, we proposed a coupled-view based ranking optimization method through cross KNN rank aggregation and graph-based re-ranking to revise the original ranking lists. Its core assumption is that the images of the same person should share the similar visual appearance in both global and local views. Extensive experiments on two datasets show the superiority of our proposed method with an average improvement of 20-30% over the state-of-the-art methods at CMC@1.

We present an iterative and unsupervised learning approach for the speaker verification task. In conventional speaker verification, Probabilistic Linear Discriminant Analysis (PLDA) has been widely used as a supervised backend. However, PLDA requires fully labeled training data, which is often difficult to obtain in reality. To automatically retrieve the speaker labels of unlabeled training data, we propose to use the Affinity Propagation (AP) - a clustering method that takes pairwise data similarity as input - to generate the labels for the PLDA modeling. We further propose an iterative refinement strategy that incrementally updates the similarity input of the AP clustering with the previous iteration's PLDA scoring outputs. Moreover, we evaluate the performance of different PLDA scoring methods for the multiple enrollment task and show that the generalized hypothesis testing achieves the best results. Experiments were conducted on the NIST SRE 2010 and the 2014 i-vector challenge database. The results show that our proposed iterative and unsupervised PLDA model learning approach outperformed the cosine similarity baseline by 35% relatively.

Stroke survivors often suffer from movement impairments that significantly affect their daily activities. The advancements in sensor technology and IoT have provided opportunities to automate the assessment and rehabilitation process for stroke survivors. This paper aims to provide a smart post-stroke severity assessment using AI-driven models. With the absence of labelled data and expert assessment, there is a research gap in providing virtual assessment, especially for unlabeled data. Inspired by the advances in consensus learning, in this paper, we propose a consensus clustering algorithm, PSA-NMF, that combines various clusterings into one united clustering, i.e., cluster consensus, to produce more stable and robust results compared to individual clustering. This paper is the first to investigate severity level using unsupervised learning and trunk displacement features in the frequency domain for post-stroke smart assessment. Two different methods of data collection from the U-limb datasets—the camera-based method (Vicon) and wearable sensor-based technology (Xsens)—were used. The trunk displacement method labelled each cluster based on the compensatory movements that stroke survivors employed for their daily activities. The proposed method uses the position and acceleration data in the frequency domain. Experimental results have demonstrated that the proposed clustering method that uses the post-stroke assessment approach increased the evaluation metrics such as accuracy and F-score. These findings can lead to a more effective and automated stroke rehabilitation process that is suitable for clinical settings, thus improving the quality of life for stroke survivors.