User queries in image searching

There are many existing studies of user behavior in simple tasks (e.g., navigational and informational search) within a short duration of 1--2 queries. However, we know relatively little about user behavior, especially browsing and clicking behavior, for longer search session solving complex search tasks. In this paper, we characterize and compare user behavior in relatively long search sessions (10 minutes; about 5 queries) for search tasks of four different types. The tasks differ in two dimensions: (1) the user is locating facts or is pursuing intellectual understanding of a topic; (2) the user has a specific task goal or has an ill-defined and undeveloped goal. We analyze how search behavior as well as browsing and clicking patterns change during a search session in these different tasks. Our results indicate that user behavior in the four types of tasks differ in various aspects, including search activeness, browsing style, clicking strategy, and query reformulation. As a search session progresses, we note that users shift their interests to focus less on the top results but more on results ranked at lower positions in browsing. We also found that results eventually become less and less attractive for the users. The reasons vary and include downgraded search performance of query, decreased novelty of search results, and decaying persistence of users in browsing. Our study highlights the lack of long session support in existing search engines and suggests different strategies of supporting longer sessions according to different task types.

We consider a search task as a set of queries that serve the same user information need. Analyzing search tasks from user query streams plays an important role in building a set of modern tools to improve search engine performance. In this paper, we propose a probabilistic method for identifying and labeling search tasks based on the following intuitive observations: queries that are issued temporally close by users in many sequences of queries are likely to belong to the same search task, meanwhile, different users having the same information needs tend to submit topically coherent search queries. To capture the above intuitions, we directly model query temporal patterns using a special class of point processes called Hawkes processes, and combine topic models with Hawkes processes for simultaneously identifying and labeling search tasks. Essentially, Hawkes processes utilize their self-exciting properties to identify search tasks if influence exists among a sequence of queries for individual users, while the topic model exploits query co-occurrence across different users to discover the latent information needed for labeling search tasks. More importantly, there is mutual reinforcement between Hawkes processes and the topic model in the unified model that enhances the performance of both. We evaluate our method based on both synthetic data and real-world query log data. In addition, we also apply our model to query clustering and search task identification. By comparing with state-of-the-art methods, the results demonstrate that the improvement in our proposed approach is consistent and promising.

What are the intents or goals behind human interactions with image search engines? Knowing why people search for images is of major concern to Web image search engines because user satisfaction may vary as intent varies. Previous analyses of image search behavior have mostly been query-based, focusing on what images people search for, rather than intent-based, that is, why people search for images. To date, there is no thorough investigation of how different image search intents affect users' search behavior. In this paper, we address the following questions: (1) Why do people search for images in text-based Web image search systems? (2) How does image search behavior change with user intent? (3) Can we predict user intent effectively from interactions during the early stages of a search session? To this end, we conduct both a lab-based user study and a commercial search log analysis. We show that user intents in image search can be grouped into three classes: Explore/Learn, Entertain, and Locate/Acquire. Our lab-based user study reveals different user behavior patterns under these three intents, such as first click time, query reformulation, dwell time and mouse movement on the result page. Based on user interaction features during the early stages of an image search session, that is, before mouse scroll, we develop an intent classifier that is able to achieve promising results for classifying intents into our three intent classes. Given that all features can be obtained online and unobtrusively, the predicted intents can provide guidance for choosing ranking methods immediately after scrolling.

Comparing to general Web search engines, image search engines present search results differently, with two-dimensional visual image panel for users to scroll and browse quickly. These differences in result presentation can significantly impact the way that users interact with search engines, and therefore affect existing methods of search evaluation. Although different evaluation metrics have been thoroughly studied in the general Web search environment, how those offline and online metrics reflect user satisfaction in the context of image search is an open question. To shed light on this, we conduct a laboratory user study that collects both explicit user satisfaction feedbacks as well as user behavior signals such as clicks. Based on the combination of both externally assessed topical relevance and image quality judgments, offline image search metrics can be better correlated with user satisfaction than merely using topical relevance. We also demonstrate that existing offline Web search metrics can be adapted to evaluate on a two-dimensional presentation for image search. With respect to online metrics, we find that those based on image click information significantly outperform offline metrics. To our knowledge, our work is the first to thoroughly establish the relationship between different measures and user satisfaction in image search.

Collaboration has been identified as an important aspect in information seeking. People meet to discuss and share ideas and through this interaction an information need is quite often identified. However the process of resolving this information need, through interacting with a search engine and performing a search task, is still an individual activity. We propose an environment which allows users to collaborate to satisfy a shared information need. We discuss ways to divide the search task amongst collaborators and propose the use of relevance feedback, a common information retrieval process, to enable the transfer of knowledge across collaborators during a search session. We describe the process by which co-searchers can collaborate effectively with little redundancy and how we can combine relevance judgements from multiple searchers into a coherent model for synchronous collaborative information retrieval

ABSTRACTSearching for information could be difficult or problematic in situations where searchers possess inadequate knowledge and/or vague information needs, often leading to unsuccessful outcomes. Existing studies on search outcomes (successes and failures) have lacked a detailed analysis at query (expression of information need) level from a searcher's perspective and the connection between queries and their corresponding search tasks. In the work reported here, we adopt a mixed‐method approach to both explore how and why searchers succeeded or failed to fulfill their information need with each query and to link these query level outcomes to searchers' overall task goal realization. Twenty‐six participants finished two search tasks in three days in naturalistic settings and reported their experiences in online diaries. The log data containing their actions in nearly 200 query segments, their diary responses and post‐task semi‐structured interviews inspire both in‐depth discussion on search experiences and interesting comparison between successful and unsuccessful searchers. While quantitative analysis depicts the overall relationships between behavioral variables and search outcomes at both query and task levels, qualitative analysis reveals why those relationships took place. In general, searchers who viewed more pages and spent more time in each query segment were more likely to succeed. We also report reasons behind these phenomena.

A picture lifelog is a type of lifelog that consists of pictures, mainly taken by the user. Recently, users have been able to easily create picture lifelogs because many portable devices such as smart phones have a camera. When a user sees a picture in their picture lifelog, it is sometimes difficult to recall the events related to the picture. Therefore, we proposed to combine search queries on a picture lifelog in order to support memory retrieval. Search queries are input into a web search engine to satisfy a user's need for information. Recently, because of the prevalence of smart phones, the opportunity to input search queries has increased to anytime and anywhere. Search queries are stored in a cloud user database such as Google search history. In addition, those search queries imply what the user was thinking at the time. We investigated whether search queries enable a user to recall their thoughts regarding picture lifelogs. Thus, we conducted an experiment to ascertain whether search queries reminded a user of past events. As a result, we reveal that displaying a picture with search queries performed around the time it was taken tends to improve users' memories better than its time, location, or emails sent during that time.

Different types of search engines are used to search image and text contents. Two types of image search methods are available in the Internet. They are query keyword based model and content based image retrieval models. Text query strings are used in the textual image retrieval model. Content based image retrieval (CBIR) model uses the visual information of the images. Image search methods use the text annotation and image visual features. Google image search and Bing image search engines are used to fetch images from the web. Image query string is used to search image on Internet. One click query image selection method is used to submit user intention for image retrieval. Content based image re-ranking is performed with visual and textual similarity metrics. Adaptive Weight Schema is used for the similarity analysis. Feature weight learning algorithm is applied to estimate feature weights for the images and its category. Query is expanded with keyword and visual information. Rank boost framework algorithm is enhanced to rank images with photographic quality. Content similarity and visual quality factors are used for the re-ranking process. In this paper, we propose an image indexing and retrieval using speech annotations based on a predefined structured syntax. To improve the retrieval effectively, N-best lists for index generation is used .so, a query expansion technique is explored to enhance the query terms. All this process is automatic, without extra effort from the user. This is critically important in web-based image search engine for any commercial, where the user interface has to be extremely simple.

Vector-based image search, which represents images and search queries as vectors in a high-dimensional space, performs image search by identifying images whose vector representations closely match the vector representation of the query. Search engine optimization has been traditionally achieved by using relevant keywords to increase organic search engine discoverability. For images to be surfaced as a result of a search query, these relevant keywords have been inserted into the page that hosts the image or embedded directly into an image as metadata that can be decoded and indexed by a text-based search engine. However, when a search engine uses vectors to operate, the static nature of an image's vector representation prevents adaptation of content to increase discoverability. In this paper, we present a novel approach to alter an image in a way that enhances its discoverability with a known vector-based search engine while minimizing the visual impact of these alterations for the average user. The alteration is guided by the maximization of intended search queries and the minimization of unwanted search queries.

Framework for tasks suggestion on web search based on unsupervised learning techniques

Multimedia searching is an important activity on the Web, especially among the younger generation. The purpose of this study aims to examine college students’ multimedia information needs and searching on the Internet. While there is a clear pattern among students with respect to their multimedia uses, searching sources, relevance criteria and searching barriers, some differences exist especially according to searching of different multimedia types such as image, audio and video. For multimedia uses, information/data-focused uses are frequently found in image and video, while the use of audio is mainly for object-focused searches. As multimedia searching sources, audio and video files present a similar pattern of being high in media specific searching sources and low in generic search engines. Browsing through related blogs and homepages is an important part of searching for media files accounting for approximately 20% of total search for each media. The relevance criteria used by study participants when search for image files was primarily concerned with topicality while the contextual and media quality in the audio and video types are also considered important. Searching barriers for audio and video files are categorized into three broad aspects, including access and search quality, preview limitations and collection limitations, while obstacles for image files searching include access difficulties and low qualities of various collection.

Personalization of search results offers the potential for significant improvement in information retrieval performance. User interactions with the system and documents during information-seeking sessions provide a wealth of information about user preferences and their task goals. In this paper, we propose methods for analyzing and modeling user search behavior in search sessions to predict document usefulness and then using information to personalize search results. We generate prediction models of document usefulness from behavior data collected in a controlled lab experiment with 32 participants, each completing uncontrolled searching for 4 tasks in the Web. The generated models are then tested with another data set of user search sessions in radically different search tasks and constrains. The documents predicted useful and not useful by the models are used to modify the queries in each search session using a standard relevance feedback technique. The results show that application of the models led to consistently improved performance over a baseline that did not take account of user interaction information. These findings have implications for designing systems for personalized search and improving user search experience.

Current Google image search adopt a hybrid search approach in which a text-based query (e.g., “Paris landmarks”) is used to retrieve a set of relevant images, which are then refined by the user (e.g., by re-ranking the retrieved images based on similarity to a selected example). We conjecture that given such hybrid image search engines, learning per-query distance functions over image features can improve the estimation of image similarity. We propose scalable solutions to learning query-specific distance functions by 1) adopting a simple large-margin learning framework, 2) using the query-logs of text-based image search engine to train distance functions used in content-based systems. We evaluate the feasibility and efficacy of our proposed system through comprehensive human evaluation, and compare the results with the state-of-the-art image distance function used by Google image search.

Studies have shown that the people depicted in image search results tend to be of majority groups with respect to socially salient attributes such as gender or race. This skew goes beyond that which already exists in the world - i.e., the search results for images of people are more imbalanced than the ground truth would suggest. For example, Kay et al. showed that although 28% of CEOs in the U.S. are women, only 10% of the top 100 results for "CEO" in Google Image Search are women. Similar observations abound across search terms and across socially salient attributes. Most existing approaches to correct for this kind of bias assume that the images of people include labels denoting the relevant socially salient attributes. These labels are explicitly used to either change the dataset, adjust the training of the algorithm, and/or in the execution of the method. However, such labels are often unknown. Further, using machine learning techniques to infer these labels may often not be possible within acceptable accuracy ranges and may not be desirable due to the additional biases this process could incur. As observed in prior work, alternate approaches consider the diversity of image features, which often do not translate to images of visibly diverse people. We develop a novel approach that takes as input a visibly diverse control set of images of people and uses this set as part of a procedure to select a set of images of people in response to a query. The goal is to have a resulting set that is more visibly diverse in a manner that emulates the diversity depicted in the control set. It accomplishes this by evaluating the similarity of the images selected by a black-box algorithm with the images in the diversity control set, and incorporating this "diversity score" into the final selection process. Importantly, this approach does not require images to be labelled at any point; effectively, it gives a way to implicitly diversify the set of images selected. We provide two variants of our approach: the first is a modification of the well known MMR algorithm to incorporate the diversity scores, and the second is a more efficient variant that does not consider within-list redundancy. We evaluate these approaches empirically on two image datasets: 1) a new dataset we collect which contains the top 100 Google Image results for 96 occupations, for which we evaluate gender and skin-tone diversity with respect to occupations and 2) the well-known CelebA dataset containing images of celebrities for which we can evaluate gender diversity with respect to facial features such as "smiling" or "glasses". Both of our approaches produce image sets that significantly improve the visible diversity of the results (i.e., include a larger fraction of anti-stereotypical images) with respect to current Google Image Search results and other state-of-the-art algorithms for diverse image summarization. Further, they seem to come at a minimal cost to accuracy. These empirical results demonstrate the effectiveness of simple label-independent interventions to diversify image search.

Everyone knows that thousand of words are represented by a single image. As a result, image search has become a very popular mechanism for the Web-searchers. Image search means, the search results are produced by the search engine should be a set of images along with their Web-page Unified Resource Locator (URL). Now Web-searcher can perform two types of image search, they are “Text to Image” and “Image to Image” search. In “Text to Image” search, search query should be a text. Based on the input text data, system will generate a set of images along with their Web-page URL as an output. On the other hand, in “Image to Image” search, search query should be an image and based on this image, system will generate a set of images along with their Web-page URL as an output. According to the current scenarios, “Text to Image” search mechanism always not returns perfect result. It matches the text data and then displays the corresponding images as an output, which is not always perfect. To resolve this problem, Web researchers have introduced the “Image to Image” search mechanism. In this paper, we have also proposed an alternate approach of “Image to Image” search mechanism using Histogram.