Ranking with query-dependent loss for web search

The queries people issue to a search engine and the results clicked following a query change over time. For example, after the earthquake in Japan in March 2011, the query japan spiked in popularity and people issuing the query were more likely to click government-related results than they would prior to the earthquake. We explore the modeling and prediction of such temporal patterns in Web search behavior. We develop a temporal modeling framework adapted from physics and signal processing and harness it to predict temporal patterns in search behavior using smoothing, trends, periodicities, and surprises. Using current and past behavioral data, we develop a learning procedure that can be used to construct models of users' Web search activities. We also develop a novel methodology that learns to select the best prediction model from a family of predictive models for a given query or a class of queries. Experimental results indicate that the predictive models significantly outperform baseline models that weight historical evidence the same for all queries. We present two applications where new methods introduced for the temporal modeling of user behavior significantly improve upon the state of the art. Finally, we discuss opportunities for using models of temporal dynamics to enhance other areas of Web search and information retrieval.

One of the most important parts of search engines is the ranking unit. Many different classical ranking algorithms based on content (such as TF-IDF and BM25) and connectivity (such as HITS and PageRank) have been used in web search engines to find pages in response to a user query. Although these algorithms have been developed to improve retrieval results, none of them can take advantage of power of contents as well as useful link structures. Thus, it remains a challenging research question how to effectively combine these available information to maximize search accuracy. In this study, we investigate the application of different ensemble models in ranking algorithms. Some of them are simple such as Sum, Product and Borda rule, and the others are more complicated methods. We present three complex ensemble approaches. The first one is OWA operator to merge the results of various ranking algorithms. In the second approach, a state-of-the-art method, simulated click-through data, is used to learn how to combine many content and connectivity features of web pages. Moreover, we present a modified version of SVM classifier customized for ranking problems as the third complex fusion approach. The proposed methods are evaluated using the LETOR and dotIR benchmark data sets. The experimental results show that in most of the cases ensemble methods give better results and the improvements are very encouraging. These results also show that the OWA and SVM fusion methods are promising respect to other ensemble models.

E-commerce customers and general Web users should not believe that the products suggested by Recommender systems or results displayed by Web search engines are either complete or relevant to their search aspirations. The economic priority of Web related businesses requires a higher rank on Web snippets or product suggestions in order to receive additional customers; furthermore, Web search engines and recommender systems revenue is obtained from advertisements and pay-per-click. This survey paper presents a review of Web Search Engines, Ranking Algorithms, Citation Analysis and Recommender Systems. In addition, Neural networks and Deep Learning are also analyzed including their use in learning relevance and ranking. Finally, this survey paper also introduces the Random Neural Network with its practical applications.

Ranking information resources is a task that usually happens within more complex workflows and that typically occurs in any form of information retrieval, being commonly implemented by Web search engines. By filtering and rating data, ranking strategies guide the navigation of users when exploring large volumes of information items. There exist a considerable number of ranking algorithms that follow different approaches focusing on different aspects of the complex nature of the problem, and reflecting the variety of strategies that are possible to apply. With the growth of the web of linked data, a new problem space for ranking algorithms has emerged, as the nature of the information items to be ranked is very different from the case of Web pages. As a consequence, existing ranking algorithms have been adapted to the case of Linked Data and some specific strategies have started to be proposed and implemented. Researchers and organizations deploying Linked Data solutions thus require an understanding of the applicability, characteristics and state of evaluation of ranking strategies and algorithms as applied to Linked Data. We present a classification that formalizes and contextualizes under a common terminology the problem of ranking Linked Data. In addition, an analysis and contrast of the similarities, differences and applicability of the different approaches is provided. We aim this work to be useful when comparing different approaches to ranking Linked Data and when implementing new algorithms.

Recently, ranking data with respect to the intrinsic geometric structure (manifold ranking) has received considerable attentions, with encouraging performance in many applications in pattern recognition, information retrieval and recommendation systems. Most of the existing manifold ranking methods focus on learning a ranking function that varies smoothly along the data manifold. However, beyond smoothness, a desirable ranking function should vary monotonically along the geodesics of the data manifold, such that the ranking order along the geodesics is preserved. In this article, we aim to learn a ranking function that varies linearly and therefore monotonically along the geodesics of the data manifold. Recent theoretical work shows that the gradient field of a linear function on the manifold has to be a parallel vector field. Therefore, we propose a novel ranking algorithm on the data manifolds, called Parallel Field Ranking. Specifically, we try to learn a ranking function and a vector field simultaneously. We require the vector field to be close to the gradient field of the ranking function, and the vector field to be as parallel as possible. Moreover, we require the value of the ranking function at the query point to be the highest, and then decrease linearly along the manifold. Experimental results on both synthetic data and real data demonstrate the effectiveness of our proposed algorithm.

A key task in query understanding is interpreting user intentions from the limited words that the user submitted to the search engines. Query classification (QC) has been widely studied for this purpose, which classifies queries into a set of target categories as user search intents. Query classification is an important as well as difficult problem in the field of information retrieval, since the queries are usually short in length, ambiguous and noisy. In this case, traditional “bag-of-words” based classification methods fail to achieve high accuracy in the task of QC. In this paper, we propose to mine explicit “Concept” information to help resolve this problem. Specifically, we first leverage existing knowledge bases to enrich the short query from the concept level. Then we discuss the usage of the mined concept information and propose a novel language model based query classification method which takes both words and concepts into consideration. Experimental results show that the mined concepts are very informative and effective to improve query classification.

Semi-supervised ranking is a relatively new and important learning problem. Some semi-supervised learning to rank algorithms have been proposed by adapting existing pair-wise ranking algorithms into the semi-supervised setting. In contrast, there is comparatively little work in developing semi-supervised ranking algorithm based on the list-wise approach. In this paper, we proposed a semi-supervised extension of one of the state-of-the art list-wise ranking algorithm--AdaRank, for the task of learning to rank with partially labeled data. The proposed algorithm is a two-staged approach that combines the process of label propagation and a regularized version of AdaRank. The label propagation process is used to propagate the relevance labels from labeled instances to other unlabeled ones, so that more training data are available to learn the ranking function. The regularized version of AdaRank optimizes a novel performance measure, which incorporates both the usual Information Retrieval IR metrics and the smoothness of the ranking function. Experimental results on benchmark datasets show the advantage of our methods over existing pair-wise algorithms for the Information Retrieval measures.

Image search is becoming prevalent in web search as the number of digital photos grows exponentially on the internet. For a successful image search system, removing outliers in the top ranked results is a challenging task. Typical content based image search engines take an input image from one class as a query and compute relevance between the query and images in a database. The results often contain a large number of outliers, since these outliers may be similar to the query image in some way. In this paper we present a novel search scheme using query images from multiple classes. Instead of conducting query search for one image class at a time, we conduct multi-class query search jointly. By using several query classes that are similar to each other for multi-class query, we can utilize information across similar classes to fine tune the similarity measure to remove outliers. This strategy can be used for any information search application. In this work, we use content based image search to illustrate the concept.

The search for information is a task most commonly performed by web search engines. A common problem in web search is the personalization of search results. Personalization is essentially a customization of the results ranking formula. This paper focuses on the automation of the ranking formula according to the user's preferences. A genetic algorithm is used to find the proper combination of the ranking factors. The ranking function is a sum of chosen ranking factors weighted appropriately. The factors are formally described and range from simple (word count) to complex (semantic classification) in their complexity. The paper presents an approach that uses a genetic algorithm to determine the proper combination of the factors of the ranking functions. The end result gives a proper personalization of the search result ranking.

We study ranked enumeration for Conjunctive Queries (CQs) where the answers are ordered by a given ranking function (e.g., an ORDER BY clause in SQL). We develop “ any-k ” algorithms, which, without knowing the number k of desired answers, push down the ranking into joins by carefully ordering the computation of intermediate tuples and avoiding materialization of join answers until they are needed. For this to be possible, the ranking function needs to obey a certain type of monotonicity. Supported ranking functions include the common sum-of-weights, where answers are compared by the sum of input-tuple weights, as well as any commutative selective dioid. Our results extend a well-known unranked-enumeration dichotomy, which states that only free-connex CQs are tractable (under certain hardness hypotheses and for CQs without self-joins). For this class of queries and appropriately monotone ranking functions, the data complexity of our ranked enumeration approach for the time to the k th CQ answer is \({\mathcal {O}}(n + k \log k)\) with n denoting the size of the input, which is only a logarithmic factor slower than the \({\mathcal {O}}(n + k)\) unranked-enumeration guarantee. A core insight of our work is that ranked enumeration for CQs is closely related to Dynamic Programming and the fundamental task of path enumeration in a weighted DAG . We uncover a previously unknown tradeoff, both for this problem and for CQs, under the lens of combined complexity where the query size is not considered a constant: one any- k algorithm has lower complexity when the number of returned answers is small, the other when their number is large. This tradeoff is eliminated under a stricter monotonicity property that we define and exploit for a novel algorithm that asymptotically dominates all previously known alternatives , including Eppstein’s algorithm for sum-of-weights path enumeration. We empirically demonstrate the findings of our theoretical analysis in an experimental study that highlights the superiority of our approach over the join-then-rank approach that existing database systems follow. The project web page at https://northeastern-datalab.github.io/anyk/ contains further information including code, slides, and videos.

This paper addresses the problem of specifying Web searches and retrieving, filtering, and rating Web pages so as to improve the relevance and quality of hits, based on the user’s search intent and preferences. We present a methodology and architecture for an agent-based system, called WebSifter II, that captures the semantics of a user’s decision-oriented search intent, transforms the semantic query into target queries for existing search engines, and then ranks the resulting page hits according to a user-specified weighted-rating scheme. Users create personalized search taxonomies via our Weighted Semantic-Taxonomy Tree. Consulting a Web taxonomy agent such as WordNet helps refine the terms in the tree. The concepts represented in the tree are then transformed into a collection of queries processed by existing search engines. Each returned page is rated according to user-specified preferences such as semantic relevance, syntactic relevance, categorical match, page popularity and authority/hub rating.

A new approach for an intelligent local Web search in telematics is proposed. It is based on voice Web search, driver's physical location, search distance, search directory and driver's search intentions. In order to handle fuzzy query, perception index (PI) is also proposed. Kinking these issues is technologically demanding and potentially useful in telematics.

Understanding Web users' search intent expressed by their queries is essential for a search engine to provide the appropriate answers. Web query classification (QC) algorithms have been widely studied to improve the accuracy and meet users' demands. Some QC algorithms convert queries into vectors and use SVM or CRF model as the classifier. However, with the volume of data increasing, the time consumed significantly increases. In this paper, we propose a method in which we split the queries into words and convert queries into a graph, after that, we adopt a liner equation as the classifier. Experimental results exhibit that our method has similar accuracy but higher efficiency compared with the existing methods. Our method can decrease the training time by 10% compared with the SVM algorithm, and also outperform the CRF model.

Web search ranking models learn from human interactions to improve retrieval performance, but they are presently limited by their use of behavioral factors, such as click-through data or dwell time, that do not account for differences in their users’ cognition. However, it is well understood that users’ behavior varies according to their abilities in processing information, making inferences, and interacting with computing systems. As a result, researchers may miss opportunities to design ranking models that are optimized for their users’ cognitive abilities. To address this, we report an approach for search result re-ranking that incorporates cognitive ability information in the ranking model. We report extensive empirical in-the-wild experiments with data from simulated tasks and real-world tasks of twenty participants to measure, predict, and use these data to train search result re-ranking models. Our results demonstrate that cognitive ability data significantly improves the effectiveness of re-ranking models in simulated-task and real-world conditions, and that cognitive abilities can be predicted from regular user interactions without requiring separate cognitive testing for each user. In particular, the models show improved performance in predicting the position of the documents the users select during search sessions. Our findings show that search engines have significant potential to improve their ranking performance by accounting for users’ cognitive ability.

Learning to rank has been intensively studied and has shown great value in many fields, such as web search, question answering and recommender systems. This paper focuses on listwise document ranking, where all documents associated with the same query in the training data are used as the input. We propose a novel ranking method, referred to as WassRank, under which the problem of listwise document ranking boils down to the task of learning the optimal ranking function that achieves the minimum Wasserstein distance. Specifically, given the query level predictions and the ground truth labels, we first map them into two probability vectors. Analogous to the optimal transport problem, we view each probability vector as a pile of relevance mass with peaks indicating higher relevance. The listwise ranking loss is formulated as the minimum cost (the Wasserstein distance) of transporting (or reshaping) the pile of predicted relevance mass so that it matches the pile of ground-truth relevance mass. The smaller the Wasserstein distance is, the closer the prediction gets to the ground-truth. To better capture the inherent relevance-based order information among documents with different relevance labels and lower the variance of predictions for documents with the same relevance label, ranking-specific cost matrix is imposed. To validate the effectiveness of WassRank, we conduct a series of experiments on two benchmark collections. The experimental results demonstrate that: compared with four non-trivial listwise ranking methods (i.e., LambdaRank, ListNet, ListMLE and ApxNDCG), WassRank can achieve substantially improved performance in terms of nDCG and ERR across different rank positions. Specifically, the maximum improvements of WassRank over LambdaRank, ListNet, ListMLE and ApxNDCG in terms of nDCG@1 are 15%, 5%, 7%, 5%, respectively.