Graph-based rank aggregation method for high-dimensional and partial rankings

Normally, statistical methods are used to generate rankings for genes in terms of their ability to distinguish between normal and malignant tumors from a gene expression dataset. However, different statistical methods yield different ranks for same gene and there is no universally accepted method for ranking. Therefore rank aggregation is required to find the overall ranking of the set of genes. There are various rank aggregation methods in the existing literature to integrate the rankings produced by various statistical tests. Moreover, the problem of integration of some partial rankings, containing unequal numbers of genes, is more challenging. In this article, a multiobjective genetic algorithm based rank aggregation method is proposed to integrate some partial rankings in an unbiased way. The first objective is to minimize the total distance from the reference ranking to the input rankings. For distance calculation, the Scaled Footrule Distance is used. The second objective is to minimize the standard deviation among those distances in order to avoid bias toward a particular input ranking. The proposed method is applied on some real-life microarray gene expression datasets, and the performance of it is compared with that of several existing rank aggregation techniques with respect to accuracy and the AUC (Area under ROC curve) value. Again, for real-life datasets, accuracy is plotted for visual comparison.

This paper presents novel approaches for combining re-ranking and rank aggregation methods aiming at improving the effectiveness of Content-Based Image Retrieval (CBIR) systems. Given a query image as input, CBIR systems retrieve the most similar images in a collection by taking into account image visual properties. In this scenario, accurately ranking collection images is of great relevance. Aiming at improving the effectiveness of CBIR systems, re-ranking and rank aggregation algorithms have been proposed. However, different re-ranking and rank aggregation approaches, applied to different image descriptors, may produce different and complementary image rankings. In this paper, we present four novel approaches for combining these rankings aiming at obtaining more effective results. Several experiments were conducted involving shape, color, and texture descriptors. The proposed approaches are also evaluated on multimodal retrieval tasks, considering visual and textual descriptors. Experimental results demonstrate that our approaches can improve significantly the effectiveness of image retrieval systems.

World Wide Web (WWW) is a platform that explores a wide range of information used for the development of web applications. Some examples of these applications include social network analysis, personalized item recommendations, and web page classification and ranking. Among these applications, search engines and web page ranking are particularly important as they consistently index and store billions of web pages on the internet. The main objective of this paper is to create an innovative framework for the classification and re-ranking of web pages using intelligent techniques. The framework is structured into two key phases: classification and re-ranking-based retrieval. In the initial classification phase, a series of pre-processing steps are implemented, including the elimination of HTML tags, punctuation, stop words, and the application of stemming. After pre-processing, a word-to-vector conversion is performed, followed by feature extraction utilizing Principal Component Analysis (PCA). This sequence of actions leads to optimal feature selection, which is vital for the precise classification of web pages. Given the multitude of features present in web pages that can compromise classification accuracy, this study employs a novel meta-heuristic algorithm, the Opposition Based-Tunicate Swarm Algorithm (O-TSA), to facilitate optimal feature selection. The refined features are subsequently processed through the Enhanced Convolutional-Recurrent Neural Network (E-CRNN), enhanced by O-TSA, resulting in the effective classification of diverse web page categories. In the second phase, the re-ranking process is executed using O-TSA, which establishes the objective function based on a similarity function (correlation) for URL matching, leading to optimal re-ranking of web.

Rank aggregation is the process of aggregating multiple base rankers into a single but more comprehensive ranker, which plays an important role in many domains such as recommender system, meta-search, database, genomics, etc. Works related to the comparison of rank aggregation methods all don't have a suitable and general data generation mechanism to produce data with various characteristics and lack a more reasonable and effective algorithm evaluation performance index.Therefore, this paper presents a general data generation mechanism based on Mallows model to produce synthetic controllable datasets, uses generalized Kendall rank correlation coefficient and rank-biased overlap to evaluate and compare the performance of two kinds of methods under different settings.Besides, we also consider the comparison between indices and the impact of data characteristics on the algorithms.This paper may be helpful to researchers and decision-makers from multiple domains.

Metasearch engine is a system that provides unified access to multiple existing search engines. After the results returned from all used component search engines are collected, the metasearch system merges the results into a single ranked list which is expected to be better than the results of the best of the participating search systems. The success of a metasearch engine depends mainly on their rank aggregation method. The system is a better one, if the aggregated list of results displayed before the user satisfies the user with his information need. In this paper, we discuss the development of a metasearch engine that performs user feedback based metasearching using modified rough set based aggregation. Metasearching using the modified rough set based aggregation is performed in two phases namely the ranking rule learning phase and the rank aggregation phase. For each query in the training set, we mine the ranking rules and select the best rules-set by performing cross-validation test. Once the system is trained, we use the best rule set to get the overall ranking for the results returned from different search systems in response to other queries. We also present few snapshots of our system.

We study the problem of combining active learning suggestions to identify informative training examples by empirically comparing methods on benchmark datasets. Many active learning heuristics for classification problems have been proposed to help us pick which instance to annotate next. But what is the optimal heuristic for a particular source of data? Motivated by the success of methods that combine predictors, we combine active learners with bandit algorithms and rank aggregation methods. We demonstrate that a combination of active learners outperforms passive learning in large benchmark datasets and removes the need to pick a particular active learner a priori. We discuss challenges to finding good rewards for bandit approaches and show that rank aggregation performs well.

In the past few years, segmentation and classification techniques have become a cornerstone of many successful remote sensing algorithms aiming at delineating geographic target objects. One common strategy relies on using multiple complex features to guide the delineation process with the objective of gathering complementary information for improving classification results. However, a persistent problem in this approach is how to combine different and noncorrelated feature descriptors automatically. In this regard, one solution is to combine them through multiple classifier systems (MCSs) in which the diversity of simple/noncomplex classifiers is an essential issue in the definition of appropriate strategies for classifier fusion. In this paper, we propose a novel strategy for selecting classifiers (whereby a classifier is taken as a pair of learning method plus image descriptor) to be combined in MCS. In the proposed solution, diversity measures are used to assess the degree of agreement/disagreement between pairs of classifiers and ranked lists are created to sort them according to their diversity score. Thereafter, the classifiers are also sorted according to their performance through different evaluation measures (e.g., kappa and tau indices). In the end, a rank aggregation method is proposed to select the most suitable classifiers based on both the diversity and the effectiveness performance of classifiers. The proposed fusion framework has targeted at coffee crop classification and urban recognition but it is general enough to be used in a variety of other pattern recognition problems. Experimental results demonstrate that the novel strategy yields good results when compared to several baselines while using fewer classifiers and being much more efficient.

The analytic hierarchy process (AHP) is a method to help people make a complex decision by analyzing and synthesizing multiple criteria for the decision objective in a hierarchy. We adopt the AHP as a new strategy for the semantic concept detection (SCD) in video so that multiple factors involved in SCD, including multiple modalities and relating concepts, can be hierarchically analyzed and synthesized. In this paper, we first explain, by an example, why and how the SCD problem can be analyzed by the AHP. Then, following the idea of the AHP, we develop a new rank aggregation (RA) method, called AHP-RA. Experimental results of RA for SCD in video show the effectiveness of this method.

We propose a novel method for group decision making. It is applicable in the context of multiple criteria ranking problems, where alternatives need to be ordered from the best to the worst by multiple Decision Makers (DMs). In the first stage, incomplete preference information of each DM is analyzed within the framework of stochastic analysis. Specifically, the Monte Carlo simulation is applied for exploiting the space of preference model parameters compatible with each DM’s preferences. In this way, we estimate the values of stochastic acceptability indices that quantify the support given to the preference, indifference, and incomparability relations for each pair of alternatives. In the second stage, such stochastic rankings are aggregated into a group compromise recommendation that minimizes either an average or a maximal distance from each DM’s input. Apart from accounting for the utilitarian and egalitarian perspectives, the dedicated mathematical programming models deal with the processing and constructing of complete or partial rankings. The proposed method is coupled with the robust variants of PROMETHEE I and II methods, however, it can be combined with any method from the broad family of Stochastic Multicriteria Acceptability Analysis (SMAA) techniques. Its applicability for supporting real-world group decision making is demonstrated in an illustrative case study concerning the ranking of project proposals by a research funding agency.KeywordsMultiple criteria rankingRank aggregationStochastic acceptability indexRobustness analysisBinary linear programmingPROMETHEE

The problem of interpreting or aggregating multiple rankings is common to many real-world applications. Perhaps the simplest and most common approach is a weighted rank aggregation, wherein a (convex) weight is applied to each input ranking and then ordered. This paper describes a new tool for visualizing and displaying ranking information for the weighted rank aggregation method. Traditionally, the aim of rank aggregation is to summarize the information from the input rankings and provide one final ranking that hopefully represents a more accurate or truthful result than any one input ranking. While such an aggregated ranking is, and clearly has been, useful to many applications, it also obscures information. In this paper, we show the wealth of information that is available for the weighted rank aggregation problem due to its structure. We apply weight set decomposition to the set of convex multipliers, study the properties useful for understanding this decomposition, and visualize the indifference regions. This methodology reveals information–that is otherwise collapsed by the aggregated ranking–into a useful, interpretable, and intuitive decision support tool. Included are multiple illustrative examples, along with heuristic and exact algorithms for computing the weight set decomposition.

A new hierarchical ranking aggregation method

The need to meaningfully combine sets of rankings often comes up when one deals with ranked data. Although a number of heuristic and supervised learning approaches to rank aggregation exist, they require domain knowledge or supervised ranked data, both of which are expensive to acquire. In order to address these limitations, we propose a mathematical and algorithmic framework for learning to aggregate (partial) rankings without supervision. We instantiate the framework for the cases of combining permutations and combining top-k lists, and propose a novel metric for the latter. Experiments in both scenarios demonstrate the effectiveness of the proposed formalism.

Rank aggregation is a popular problem that combines different ranked lists from various sources (frequently called voters or judges), and generates a single aggregated list with improved ranking of its items. In this context, a portion of the existing methods attempt to address the problem by treating all voters equally. Nevertheless, several related works proved that the careful and effective assignment of different weights to each voter leads to enhanced performance. In this article, we introduce an unsupervised algorithm for learning the weights of the voters for a specific topic or query. The proposed method is based on the fact that if a voter has submitted numerous elements which have been placed in high positions in the aggregated list, then this voter should be treated as an expert, compared to the voters whose suggestions appear in lower places or do not appear at all. The algorithm iteratively computes the distance of each input list with the aggregated list and modifies the weights of the voters until all weights converge. The effectiveness of the proposed method is experimentally demonstrated by aggregating input lists from six TREC conferences.CCS CONCEPTS• Information systems → Rank aggregation; •Theory of computation → Unsupervised learning and clustering.

Aggregating multiple rankings in a database is an important task well studied by the database community. High-stakes application domains include hiring, lending, and education where multiple decision makers rank candidates and their input is then combined into an overall consensus ranking. While state-of-art aggregation methods succeed in giving equal say to each decision maker, to date no methods ensure fair representation for groups of candidates being ranked, who risk being impacted by discriminatory bias. We present the first solution to this open problem of guaranteeing fairness for disadvantaged groups of candidates, while still producing a good consensus of the base rankings. We design a family of exact fair algorithms achieving optimality for fair rank aggregation. We also develop approximate methods achieving fairness with guaranteed minimal approximation error scaling to millions of candidates in the rankings. A comparative study evaluates our proposed methods, revealing trade-offs between aggregation accuracy and different degrees of unfair bias in a rich variety of rank aggregation scenarios. Our real-world case study demonstrates that our solutions mitigate unfair bias using real-world data.

Learning the true rank ordering among objects by aggregating a set of expert opinion rank order lists is an important and ubiquitous problem in many applications ranging from social choice theory to recommendation systems and search aggregation. We study the problem of unsupervised rank aggregation where no ground truth ordering information in available, neither about the true preference ordering among any set of objects nor about the quality of individual rank lists. Aggregating the often inconsistent and poor quality rank lists in such an unsupervised manner is a highly challenging problem, and standard consensus-based methods fall short in terms of both quality and scalability. In this manuscript we propose a novel framework to bypass these issues by using object attributes to augment the standard rank aggregation framework. We design algorithms that learn joint models on both rank lists and object features to obtain an aggregated rank ordering that is more accurate and robust, and also helps weed out rank lists of dubious validity. We validate our techniques on synthetic datasets where our algorithm is able to estimate the true rank ordering even when the rank lists are corrupted. Experiments on three real datasets, MQ2008, MQ2007 and OHSUMED, show that using object features can result in significant improvement in performance over existing rank aggregation methods that do not use object information. Furthermore, when at least some of the rank lists are of high quality, our methods are able to effectively exploit such information to output an aggregated rank ordering of high accuracy.