Learning to Reinforce Search Effectiveness

Analysis of multiple query reformulations on the web: The interactive information retrieval context

Session search is a complex search task that involves multiple search iterations triggered by query reformulations. We observe a Markov chain in session search: user's judgment of retrieved documents in the previous search iteration affects user's actions in the next iteration. We thus propose to model session search as a dual-agent stochastic game: the user agent and the search engine agent work together to jointly maximize their long term rewards. The framework, which we term win-win is based on Partially Observable Markov Decision Process. We mathematically model dynamics in session search, including decision states, query changes, clicks, and rewards, as a cooperative game between the user and the search engine. The experiments on TREC 2012 and 2013 Session datasets show a statistically significant improvement over the state-of-the-art interactive search and session search algorithms.

When we search information on the Web, we usually enter a query that best represents our information need, and then modify it as the search results are shown. Many thus have studied search queries to understand how users express and find their information need on the Web. For example, studies have shown that query length affects search performance (Belkin et al, 2003). Then, researchers have begun to examine image search queries on the Web. They analyzed search logs from general or commercial search engines or studied specific image database systems based on log analysis (Goodrum & Sprink, 2001; Jorgensen & Jorgensen, 2005; Goodrum, Bejune, & Siochi, 2003; Pu, 2005; Cunningham, Bainbridge, & Masoodian, 2004). While these studies provide basic understanding of image query formulation, further characteristics of user's queries in the interactive searching process have not been explored. This study thus builds on previous studies on image search queries by examining the patterns of query formulation and modification in visual information search. Specifically, first, it investigates the effects of tasks and other contextual factors on query length and query changes; and second, it compares the query length and changes with the user's satisfaction of search results. Participants Twenty-nine college students were recruited for data collection during the spring and summer semesters of 2008, mostly from the Department of Media Studies at the Catholic University of America (CUA). They included 22 from Media Studies and 7 from other fields taking courses in the department; 22 were females and 7 males; and an average age of 21 years. Background information collected were years of computer experience, hours per day using the Web, self-rated online searching expertise, and self-rated digital image searching experience. The seven-point Likert scale was used for self-rating variables. Each participant took part in three different search sessions with his or her own search topic at CUA with the researcher's laptop that had an Internet connection. The university's homepage was set as the default home for the Web browser. Each participant filled out a pre-search questionnaire to collect a search task, topic, self-rated topic familiarity using 7-Likert scale. The participants chose their own preferred browser between Internet Explorer and Firefox. The search session was recorded by Camtasia v5.0 as screen capture. After the session, a post-search questionnaire was given to the participants to assess the level of satisfaction and confidence of their searching using 7-Likert scale. There was no time limit to complete a search session. The participants were free to end a search session at any time. Search Tasks The participants brought their own task and topics to find images. There were three types of search tasks identified: academic task, work-related task, and personal interest (Table 1). Search queries were acquired from screen capture log files. A total of 979 queries were obtained as the entire string of terms submitted by the participants. Table 2 shows a distribution of search queries on a general search engine, a search engine's image search, and other specific sites. Findings showed that Google Images Search was the most frequent source for forming a search query. An average word per query was 3.07 during the first session, 2.95 during the second session, and 2.89 for the last session. This finding of short queries is similar to that of previous studies on Web image queries (Goodrum & Spink, 2001; Jorgensen & Jorgensen, 2005). Analysis of data is in progress. A poster will summarize detail findings and provide answers to the following questions: What are the patterns of query modification? Are there any differences of query length and number of query change based on the task and topic familiarities? Is there any relationship between query length and number of query change and users' satisfaction and confidence with search results? Do different stages of a user's particular task affect query length and number of query change? The author would like to thank OCLC/ALISE for their generous funding for this project.

There are many existing studies of user behavior in simple tasks (e.g., navigational and informational search) within a short duration of 1--2 queries. However, we know relatively little about user behavior, especially browsing and clicking behavior, for longer search session solving complex search tasks. In this paper, we characterize and compare user behavior in relatively long search sessions (10 minutes; about 5 queries) for search tasks of four different types. The tasks differ in two dimensions: (1) the user is locating facts or is pursuing intellectual understanding of a topic; (2) the user has a specific task goal or has an ill-defined and undeveloped goal. We analyze how search behavior as well as browsing and clicking patterns change during a search session in these different tasks. Our results indicate that user behavior in the four types of tasks differ in various aspects, including search activeness, browsing style, clicking strategy, and query reformulation. As a search session progresses, we note that users shift their interests to focus less on the top results but more on results ranked at lower positions in browsing. We also found that results eventually become less and less attractive for the users. The reasons vary and include downgraded search performance of query, decreased novelty of search results, and decaying persistence of users in browsing. Our study highlights the lack of long session support in existing search engines and suggests different strategies of supporting longer sessions according to different task types.

In this paper, a two-level deep learning framework is presented to model human information foraging behavior with search engines. A recurrent neural network architecture is designed using LSTM as the base unit to explicitly consider the temporal and spatial dependencies of information scents, the key concept in Information Foraging Theory. The target is to predict several major search behaviors, such as query abandonment, query reformulation, number of clicks, and information gain. The memory capability and the sequence structure of LSTM allow to naturally mimic not only what users are perceiving and performing at the moment but also what they have seen and learned from the past during the search dynamics. The promising results indicate that our information scent models with different input variations were better, compared to the state-of-the art neural click models, at predicting some search behaviors. When incorporating the knowledge from a previous query in the same search session, the prediction of current query abandonment, pagination, and information gain has been improved. Compared to the well known neural click models that model search behaviors under a single search query thread, this study takes a broader view to consider an entire search session which may contain multiple queries. More importantly, our model takes the search result relevance pattern on the Search Engine Results Pages (SERP) as a whole as the information scent input to the deep learning model, instead of considering one search result at each step. The results have insights on the impact of information scents on how people forage for information, which has implications for designing or refining a set of design guidelines for search engines.

This article discusses a multiple search session model of end-users' interaction with information retrieval systems based on results from an exploratory study investigating end-users' search sessions over time with online public access catalogs (OPAC) or CD-ROM databases at different stages of their information seeking related to a current research project. Interviews were conducted with 200 academic end-users to investigate the occurrence of multiple search sessions. Results show that at the time of the interview, 57% of end-users had conducted multiple search sessions during their research project and 86% of end-users conducted their first search session at the beginning stage of their information-seeking process. Forty-nine percent of end-users had conducted between 1 and 6 search sessions and 8% more than 6 search sessions. Seventy percent of multiple search session end-users' had modified their search terms since their first search session. The implications of the findings for end-user training, information retrieval systems design, and further research are discussed. © 1996 John Wiley & Sons, Inc.

QIRM: A quantum interactive retrieval model for session search

The recent increase in the volume and variety of video content available online presents growing challenges for video search. Users face increased difficulty in formulating effective queries and search engines must deploy highly effective algorithms to provide relevant results. Although lately much effort has been invested in optimizing video search engine results, relatively little attention has been given to predicting for which queries results optimization is most useful, i.e., predicting which queries will fail. Being able to predict when a video search query would fail is likely to make the video search result optimization more efficient and effective, improve the search experience for the user by providing support in the query formulation process and in this way boost the development of video search engines in general. While insight about a query's performance in general could be obtained using the well-known concept of query performance prediction (QPP), we propose a novel approach for predicting a failure of a video search query in the specific context of a search session. Our 'context-aware query failure' prediction approach uses a combination of 'user indicators' and 'engine indicators' to predict whether a particular query is likely to fail in the context of a particular search session. User indicators are derived from the search log and capture the patterns of query (re)formulation behavior and the click-through data of a user during a typical video search session. Engine indicators are derived from the video search results list and capture the visual variance of search results that would be offered to the user for the given query. We validate our approach experimentally on a test set containing 1+ million video search queries and show its effectiveness compared to a set of conventional QPP baselines. Our approach achieves a 13% relative improvement over the baseline.

The dominant existing routing strategies employed in peer-to-peer(P2P) based information retrieval(IR) systems are similarity-based approaches. In these approaches, agents depend on the content similarity between incoming queries and their direct neighboring agents to direct the distributed search sessions. However, such a heuristic is myopic in that the neighboring agents may not be connected to more relevant agents. In this paper, an online reinforcement-learning based approach is developed to take advantage of the dynamic run-time characteristics of P2P IR systems as represented by information about past search sessions. Specifically, agents maintain estimates on the downstream agents' abilities to provide relevant documents for incoming queries. These estimates are updated gradually by learning from the feedback information returned from previous search sessions. Based on this information, the agents derive corresponding routing policies. Thereafter, these agents route the queries based on the learned policies and update the estimates based on the new routing policies. Experimental results demonstrate that the learning algorithm improves considerably the routing performance on two test collection sets that have been used in a variety of distributed IR studies.

Search engine users issue queries to formulate their information need and gain useful insights. However, it is challenging for search engines to understand different users’ search type intents and return appropriate results. Simulating user search behaviour allows information retrieval systems (IR) to parameterise the a-priori distribution of search types using different back-end configurations and user interface variants to improve the retrieval functionality. In this paper, we propose a formal Markov approach in which we utilise the context discovery process to model user-type specific behaviour by capturing the user’s query change in a search session. Contextual Markov models have been used in the past to improve the prediction of user intentions, we investigate here their efficiency in simulating user-type specific interactions. Additionally, we provide an empirical and classification-based evaluation that can be used in simulation assessment. Overall, we report that the proposed approach reliably simulates user-type specific behaviour on a real-world academic search engine log dataset.

In recent years, the opportunity to use search engines has increased due to the greater variety and number of Internet-capable devices. Search engines have become indispensable for many users, who provide vast amounts of information as input. However, there has been recent recognition of the risk entailed by search engine providers storing and analyzing information related to user privacy. Existing research evaluates protection of user privacy from search engines that try to extract related information from users' query strings (Jones et al. I know what you did last summer--query logs and user privacy. In: Proceedings of the sixteenth ACM conference on conference on information and knowledge management, pp. 909---914, 2007). The searchable encryption technique is effective when searching encrypted queries and is useful when users search their own data stored in external cloud storage. However, search engine providers make a profit based on the query strings of their many users, so they are not expected to adopt this approach. Private information retrieval (PIR) is an established technique which ensures that no information is leaked to the search engine. However, PIR is based on a model with strict limitations on retrieval and is impractical. In this paper, we define a measure to quantify the amount of information that is leaked during a search. The measure is defined based on the entropy of query strings. We propose a practical search scheme that reduces the amount of leaked information. The proposed scheme is simple and can be implemented using a typical personal computer. We evaluate the system by experiment and confirm that the proposed scheme works as intended and is of acceptable usability.

Modeling multidimensional relevance in information retrieval (IR) has attracted much attention in recent years. However, most existing studies are conducted through relatively small‐scale user studies, which may not reflect a real‐world and natural search scenario. In this article, we propose to study the multidimensional user relevance model (MURM) on large scale query logs, which record users' various search behaviors (e.g., query reformulations, clicks and dwelling time, etc.) in natural search settings. We advance an existing MURM model (including five dimensions: topicality, novelty, reliability, understandability, and scope) by providing two additional dimensions, that is, interest and habit. The two new dimensions represent personalized relevance judgment on retrieved documents. Further, for each dimension in the enriched MURM model, a set of computable features are formulated. By conducting extensive document ranking experiments on Bing's query logs and TREC session Track data, we systematically investigated the impact of each dimension on retrieval performance and gained a series of insightful findings which may bring benefits for the design of future IR systems.

Learning a high-dimensional dense representation for vocabulary terms, also known as a word embedding, has recently attracted much attention in natural language processing and information retrieval tasks. The embedding vectors are typically learned based on term proximity in a large corpus. This means that the objective in well-known word embedding algorithms, e.g., word2vec, is to accurately predict adjacent word(s) for a given word or context. However, this objective is not necessarily equivalent to the goal of many information retrieval (IR) tasks. The primary objective in various IR tasks is to capture relevance instead of term proximity, syntactic, or even semantic similarity. This is the motivation for developing unsupervised relevance-based word embedding models that learn word representations based on query-document relevance information. In this paper, we propose two learning models with different objective functions; one learns a relevance distribution over the vocabulary set for each query, and the other classifies each term as belonging to the relevant or non-relevant class for each query. To train our models, we used over six million unique queries and the top ranked documents retrieved in response to each query, which are assumed to be relevant to the query. We extrinsically evaluate our learned word representation models using two IR tasks: query expansion and query classification. Both query expansion experiments on four TREC collections and query classification experiments on the KDD Cup 2005 dataset suggest that the relevance-based word embedding models significantly outperform state-of-the-art proximity-based embedding models, such as word2vec and GloVe.

Satisfaction prediction is one of the prime concerns in search performance evaluation. It is a non-trivial task for two major reasons: (1) The definition of satisfaction is rather subjective and different users may have different opinions in satisfaction judgement. (2) Most existing studies on satisfaction prediction mainly rely on users' click-through or query reformulation behaviors but there are many sessions without such kind of interactions. To shed light on these research questions, we construct an experimental search engine that could collect users' satisfaction feedback as well as mouse click-through/movement data. Different from existing studies, we compare for the first time search users' and external assessors' opinions on satisfaction. We find that search users pay more attention to the utility of results while external assessors emphasize on the efforts spent in search sessions. Inspired by recent studies in predicting result relevance based on mouse movement patterns (namely motifs), we propose to estimate the utilities of search results and the efforts in search sessions with motifs extracted from mouse movement data on search result pages (SERPs). Besides the existing frequency-based motif selection method, two novel selection strategies (distance-based and distribution-based) are also adopted to extract high quality motifs for satisfaction prediction. Experimental results on over 1,000 user sessions show that the proposed strategies outperform existing methods and also have promising generalization capability for different users and queries.

Collaboration has been identified as an important aspect in information seeking. People meet to discuss and share ideas and through this interaction an information need is quite often identified. However the process of resolving this information need, through interacting with a search engine and performing a search task, is still an individual activity. We propose an environment which allows users to collaborate to satisfy a shared information need. We discuss ways to divide the search task amongst collaborators and propose the use of relevance feedback, a common information retrieval process, to enable the transfer of knowledge across collaborators during a search session. We describe the process by which co-searchers can collaborate effectively with little redundancy and how we can combine relevance judgements from multiple searchers into a coherent model for synchronous collaborative information retrieval