Personalization of search results using interaction behaviors in search sessions

When we search information on the Web, we usually enter a query that best represents our information need, and then modify it as the search results are shown. Many thus have studied search queries to understand how users express and find their information need on the Web. For example, studies have shown that query length affects search performance (Belkin et al, 2003). Then, researchers have begun to examine image search queries on the Web. They analyzed search logs from general or commercial search engines or studied specific image database systems based on log analysis (Goodrum & Sprink, 2001; Jorgensen & Jorgensen, 2005; Goodrum, Bejune, & Siochi, 2003; Pu, 2005; Cunningham, Bainbridge, & Masoodian, 2004). While these studies provide basic understanding of image query formulation, further characteristics of user's queries in the interactive searching process have not been explored. This study thus builds on previous studies on image search queries by examining the patterns of query formulation and modification in visual information search. Specifically, first, it investigates the effects of tasks and other contextual factors on query length and query changes; and second, it compares the query length and changes with the user's satisfaction of search results. Participants Twenty-nine college students were recruited for data collection during the spring and summer semesters of 2008, mostly from the Department of Media Studies at the Catholic University of America (CUA). They included 22 from Media Studies and 7 from other fields taking courses in the department; 22 were females and 7 males; and an average age of 21 years. Background information collected were years of computer experience, hours per day using the Web, self-rated online searching expertise, and self-rated digital image searching experience. The seven-point Likert scale was used for self-rating variables. Each participant took part in three different search sessions with his or her own search topic at CUA with the researcher's laptop that had an Internet connection. The university's homepage was set as the default home for the Web browser. Each participant filled out a pre-search questionnaire to collect a search task, topic, self-rated topic familiarity using 7-Likert scale. The participants chose their own preferred browser between Internet Explorer and Firefox. The search session was recorded by Camtasia v5.0 as screen capture. After the session, a post-search questionnaire was given to the participants to assess the level of satisfaction and confidence of their searching using 7-Likert scale. There was no time limit to complete a search session. The participants were free to end a search session at any time. Search Tasks The participants brought their own task and topics to find images. There were three types of search tasks identified: academic task, work-related task, and personal interest (Table 1). Search queries were acquired from screen capture log files. A total of 979 queries were obtained as the entire string of terms submitted by the participants. Table 2 shows a distribution of search queries on a general search engine, a search engine's image search, and other specific sites. Findings showed that Google Images Search was the most frequent source for forming a search query. An average word per query was 3.07 during the first session, 2.95 during the second session, and 2.89 for the last session. This finding of short queries is similar to that of previous studies on Web image queries (Goodrum & Spink, 2001; Jorgensen & Jorgensen, 2005). Analysis of data is in progress. A poster will summarize detail findings and provide answers to the following questions: What are the patterns of query modification? Are there any differences of query length and number of query change based on the task and topic familiarities? Is there any relationship between query length and number of query change and users' satisfaction and confidence with search results? Do different stages of a user's particular task affect query length and number of query change? The author would like to thank OCLC/ALISE for their generous funding for this project.

Given the distinct preferences of different users while using search engines, search personalization has become an important problem in information retrieval. Most approaches to search personalization are based on identifying topics a user may be interested in and personalizing search results based on this information.While topical interests information of users can be highly valuable in personalizing search results and improving user experience, it ignores the fact that two different users that have similar topical interests may still be interested in achieving very different tasks with respect to this topic (e.g. the type of tasks a broker is likely to perform related to finance is likely to be very different than that of a regular investor). Hence, considering user's topical interests jointly with the type of tasks they are likely to be interested in could result in better personalisedWe present an approach that uses search task information embedded in search logs to represent users by their actions over a task-space as well as over their topical-interest space. In particular, we describe a tensor based approach that represents each user in terms of (i) user's topical interests and (ii) user's search task behaviours in a coupled fashion and use these representations for personalization. Additionally, we also integrate user's historic search behavior in a coupled matrix-tensor factorization framework to learn user representations. Through extensive evaluation via query recommendations and user cohort analysis, we demonstrate the value of considering topic specific task information while developing user models.

Satisfaction prediction is one of the prime concerns in search performance evaluation. It is a non-trivial task for two major reasons: (1) The definition of satisfaction is rather subjective and different users may have different opinions in satisfaction judgement. (2) Most existing studies on satisfaction prediction mainly rely on users' click-through or query reformulation behaviors but there are many sessions without such kind of interactions. To shed light on these research questions, we construct an experimental search engine that could collect users' satisfaction feedback as well as mouse click-through/movement data. Different from existing studies, we compare for the first time search users' and external assessors' opinions on satisfaction. We find that search users pay more attention to the utility of results while external assessors emphasize on the efforts spent in search sessions. Inspired by recent studies in predicting result relevance based on mouse movement patterns (namely motifs), we propose to estimate the utilities of search results and the efforts in search sessions with motifs extracted from mouse movement data on search result pages (SERPs). Besides the existing frequency-based motif selection method, two novel selection strategies (distance-based and distribution-based) are also adopted to extract high quality motifs for satisfaction prediction. Experimental results on over 1,000 user sessions show that the proposed strategies outperform existing methods and also have promising generalization capability for different users and queries.

There are many existing studies of user behavior in simple tasks (e.g., navigational and informational search) within a short duration of 1--2 queries. However, we know relatively little about user behavior, especially browsing and clicking behavior, for longer search session solving complex search tasks. In this paper, we characterize and compare user behavior in relatively long search sessions (10 minutes; about 5 queries) for search tasks of four different types. The tasks differ in two dimensions: (1) the user is locating facts or is pursuing intellectual understanding of a topic; (2) the user has a specific task goal or has an ill-defined and undeveloped goal. We analyze how search behavior as well as browsing and clicking patterns change during a search session in these different tasks. Our results indicate that user behavior in the four types of tasks differ in various aspects, including search activeness, browsing style, clicking strategy, and query reformulation. As a search session progresses, we note that users shift their interests to focus less on the top results but more on results ranked at lower positions in browsing. We also found that results eventually become less and less attractive for the users. The reasons vary and include downgraded search performance of query, decreased novelty of search results, and decaying persistence of users in browsing. Our study highlights the lack of long session support in existing search engines and suggests different strategies of supporting longer sessions according to different task types.

PurposeThis paper aims to describe a study of the queries generated from a user experiment for cross‐language information retrieval (CLIR) from a historic image archive.Design/methodology/approachA controlled lab‐based user study was carried out using a prototype Italian‐English image retrieval system. Participants were asked to carry out searches for 16 images provided to them, a known‐item search task. Italian speaking users generated 618 queries for a set of known‐item search tasks. User's interactions with the system were recorded and queries were analysed manually quantitatively and qualitatively. The queries generated by user's interaction with the system were analysed and the results used to suggest recommendations for the future development of cross‐language retrieval systems for digital image libraries.FindingsResults highlight the diversity in requests for similar visual content and the weaknesses of machine translation for query translation. Through the manual translation of queries the authors show the benefits of using high‐quality translation resources. The results show the individual characteristics of users while performing known‐item searches and the overlap obtained between query terms and structured image captions, highlighting the use of user's search terms for objects within the foreground of an image.Research limitations/implicationsThis research looks in depth into one case of interaction and one image repository. Despite this limitation, the discussed results are likely to be valid across other languages and image repositories.Practical implicationsTo develop effective systems requires studying user's search behaviours, particularly in digital image libraries.Originality/valueThe growing quantity of digital visual material in digital libraries offers the potential to apply techniques from CLIR to provide cross‐language information access services. The value of this paper is in the provision of empirical evidence to support recommendations for effective cross‐language image retrieval system design.

Estimation of session duration for an e-commerce search engine is important for various downstream applications, including user satisfaction prediction, personalization, and diversification of search results. It has been shown in previous studies that search session length has a strong correlation with user's explore vs specific purchase intent. Based on previous work [14], we hypothesize that early prediction of session length distribution can be used to control the degree of explore vs exploit (loosely related to diversification v/s personalization) for Search Engine Result Pages (SERPs) in the user's session to follow. In this work, we try to early predict the user's session length, which will enable the control on explore v/s exploit of the search results. Towards this end, based on previous work and strong empirical evidence, we hypothesize session lengths are Weibull distributed and propose its parameters being modeled by a Recurrent Neural Network over actions in user's search sessions. Through experimentation, we demonstrate that our method performs better as compared to strong baselines for the same.

Session search aims to improve ranking effectiveness by incorporating user interaction information, including short-term interactions within one session and global interactions from other sessions or other users. While various session search models have been developed and a large number of interaction features have been used, there is a lack of a systematic investigation on how different features would influence the session search. In this paper, we propose to classify typical interaction features into four categories current query, current session, query change, and collective intelligence. Their impact on the session search performance is investigated through a systematic empirical study, under the widely used Learning-to-Rank framework. One of our key findings, different from what have been reported in the literature, is: features based on current query and collective intelligence have a more positive influence than features based on query change and current session. This would provide insights for development of future session search techniques.

With the advancement of the Internet, search has expanded to every aspect of our daily lives including online shopping. Previous research has discovered that the focused, one‐time search task model does not apply in many real‐life settings and people engage in multitasking sessions. In this poster, we examined the relationships among tasks in multitasking product search through query log analysis. We analyzed 2,910 users' 47,387 queries in 18,102 product search sessions from taobao.com , the biggest Chinese C2C e‐commerce site. Results show that: (1) 35.7% of all search sessions were multi‐tasking sessions; (2) users' issued more queries and spent more time in multitasking sessions, but the number of queries per task remained similar for mono‐tasking and multitasking sessions; (3) about 80% of the tasks were unrelated in multitasking sessions, while the other 20% were hierarchical and sibling tasks. The results provide exploratory understanding of the relationships among multiple product search tasks, and could be useful for query recommendation and product recommendation.

Understanding user intent from her sequential search behaviors, i.e. predicting the intent of each user query in a search session, is crucial for modern Web search engines. However, due to the huge number of user behavior variables and coarse level intent labels defined by human editors, it is very difficult to directly model user behavioral dynamics or user intent dynamics in user search sessions. In this paper, we propose a novel Sparse Hidden-Dynamic Conditional Random Fields (SHDCRF) model for user intent learning from their search sessions. Through incorporating the proposed hidden state variables, SHDCRF aims to learn a substructure, i.e. a set of related hidden variables, for each intent label and they are used to model the intermediate dynamics between user intent labels and user behavioral variables. In addition, SHDCRF learns a sparse relation between the hidden variables and intent labels to make the hidden state variables explainable. Extensive experiment results, on real user search sessions from a popular commercial search engine show that the proposed SHDCRF model significantly outperforms in terms of intent prediction results that those classical solutions such as Support Vector Machine (SVM), Conditional Random Field (CRF) and Latnet-Dynamic Conditional Random Field (LDCRF).

Session search is a complex search task that involves multiple search iterations triggered by query reformulations. We observe a Markov chain in session search: user's judgment of retrieved documents in the previous search iteration affects user's actions in the next iteration. We thus propose to model session search as a dual-agent stochastic game: the user agent and the search engine agent work together to jointly maximize their long term rewards. The framework, which we term win-win is based on Partially Observable Markov Decision Process. We mathematically model dynamics in session search, including decision states, query changes, clicks, and rewards, as a cooperative game between the user and the search engine. The experiments on TREC 2012 and 2013 Session datasets show a statistically significant improvement over the state-of-the-art interactive search and session search algorithms.

Search engine users issue queries to formulate their information need and gain useful insights. However, it is challenging for search engines to understand different users’ search type intents and return appropriate results. Simulating user search behaviour allows information retrieval systems (IR) to parameterise the a-priori distribution of search types using different back-end configurations and user interface variants to improve the retrieval functionality. In this paper, we propose a formal Markov approach in which we utilise the context discovery process to model user-type specific behaviour by capturing the user’s query change in a search session. Contextual Markov models have been used in the past to improve the prediction of user intentions, we investigate here their efficiency in simulating user-type specific interactions. Additionally, we provide an empirical and classification-based evaluation that can be used in simulation assessment. Overall, we report that the proposed approach reliably simulates user-type specific behaviour on a real-world academic search engine log dataset.

It is now widely recognized that user interactions with search results can provide substantial relevance information on the documents. In this paper, we focus on extracting relevance information from one source of user interactions, user click-through data which record the sequence of documents being clicked in the result sets during a user search session. We emphasize the importance of the temporal nature of user click patterns and use conditional random fields to model its relations with the degree of relevance of the individual documents in the result sets. A key advantage of our models and algorithms is their ability to express the long-distance inter-actions conditioned on the click patterns. We test our algorithms using click-through data from a commercial search engine and evaluate the extracted relevance grades against those assigned by human judges.

Session search is an Information Retrieval (IR) task which handles a series of queries issued for a search task. In this paper, we propose a novel reinforcement learning style information retrieval framework and develop a new feedback learning algorithm to model user feedback, including clicks and query reformulations, as reinforcement signals and to generate rewards in the RL framework. From a new perspective, we view session search as a cooperative game played between two agents, the user and the search engine. We study the communications between the two agents; they always exchange opinions on whether the current stage of search is relevant and whether we should explore now. The algorithm infers user feedback models by an EM algorithm from the query logs. We compare to several state-of-the-art session search algorithms and evaluate our algorithm on the most recent TREC 2012 to 2014 Session Tracks. The experimental results demonstrates that our approach is highly effective for improving session search accuracy.

In contrast to traditional web search, personalized search provides search results that take into account the user’s preferences. However, the existing personalized search methods have limitations in providing appropriate search results for the individual’s preferences, because they do not consider the user’s recent preferences or the preferences of other users. In this paper, we propose a new search method considering the user’s recent preferences and similar users’ preferences on social media analysis. Since the user expresses personal opinions on social media, it is possible to grasp the user preferences when analyzing the records of social media activities. The proposed method collects user social activity records and determines keywords of interest using TF-IDF. Since user preferences change continuously over time, we assign time weights to keywords of interest, giving many high values to state-of-the-art user preferences. We identify users with similar preferences to extend the search results to be provided to users because considering only user preferences in personalized searches can provide narrow search results. The proposed method provides personalized search results considering social characteristics by applying a ranking algorithm that considers similar user preferences as well as user preferences. It is shown through various performance evaluations that the proposed personalized search method outperforms the existing methods.

Conversational information seeking is a major emerging research area because of the increasing popularity of conversational AI systems users utilize to perform their search tasks. Search systems and multiple other user supporting applications benefit from modeling the search tasks users carry out to satisfy their information needs. Most existing search task modeling methods are monolingual, and few methods leverage user clicks even though clicked URLs are crucial for modeling user intent. We propose a language-agnostic, user intent aware approach to model search tasks from user interactions with search systems. The proposed approach leverages user intent modeling from clicked query-document pairs, latent representations of queries in a language-agnostic space, and graph-based clusteringto model search tasks in an unsupervised approach. Experimental results demonstrate the proposed approach outperforms recent work in search task modeling, supporting user queries in multiple languages. It can also produce search task modeling results in the order of milliseconds, an essential aspect for conversational systems and user support applications requiring realtime results.KeywordsConversational searchUser intent modelingLanguage-agnostic query representation