Annotate! a tool for collaborative information retrieval

The dominant existing routing strategies employed in peer-to-peer(P2P) based information retrieval(IR) systems are similarity-based approaches. In these approaches, agents depend on the content similarity between incoming queries and their direct neighboring agents to direct the distributed search sessions. However, such a heuristic is myopic in that the neighboring agents may not be connected to more relevant agents. In this paper, an online reinforcement-learning based approach is developed to take advantage of the dynamic run-time characteristics of P2P IR systems as represented by information about past search sessions. Specifically, agents maintain estimates on the downstream agents' abilities to provide relevant documents for incoming queries. These estimates are updated gradually by learning from the feedback information returned from previous search sessions. Based on this information, the agents derive corresponding routing policies. Thereafter, these agents route the queries based on the learned policies and update the estimates based on the new routing policies. Experimental results demonstrate that the learning algorithm improves considerably the routing performance on two test collection sets that have been used in a variety of distributed IR studies.

Text arrays, created by the online community, contain specific cognitive capabilities. Analyses cover the following: mass media materials, social networks, forums, blogs, political materials, biographies and diaries, scientific publications, belles-letters and other. As practice shows, standard search systems are not always able to find the required data out of this huge volume of the data file. A difficult task for the automated computer text processing is the semantic analysis, which is interpretation of meaning of the text, its content and semantics. Performance of this task requires knowledge of meaning of words and sentences; the way to describe these values formally, and to carry out operations with them, even their storage in computer memory, cause difficulties. That is why in automated text processing computer is not able to search texts of a certain subject, without explicitly specified keywords or phrases, as well as to find texts with the similar meaning, which is quite difficult for the search procedure. Modern information retrieval systems (IRS) are mainly based on key words. The major features for this approach are frequency of word occurrence in document collection, its uniqueness, morphological and syntactic properties. The problem is that the full-text information retrieval systems initially do not imply any semantic connection between the documents and the information they contain, and do not take into account the context and many other issues of importance for semantic interpretation, which makes full-text information retrieval systems unsuitable solution for contextual search. Semantic information retrieval systems should settle the issue of full-text IRS and assist the computer in formal description of semantic meaning of the documents and the data about it. This paper examines possible organization of semantic information-retrieval system based on associations, and uses associative vector spaces as the basic semantic structures.

Exploiting user session data is a promising approach for testing a Web application. However, the effectiveness of user session testing techniques depends on the set of collected user session data: the wider this set, the greater the capability of the approach to detect failures, but the wider the user session data set, the greater the cost of collecting, analysing and storing data. In this paper, a technique for reducing a set of user sessions into an equivalent smaller one will be proposed. This technique allows equivalent user behaviours included in user sessions to be identified and classified, and produces a reduced set of user sessions that can be used to design test suites with a reduced effort. Some preliminary case studies were carried out to validate the proposed technique and to evaluate its effectiveness. Results of a case study will be presented in the paper

This article discusses a multiple search session model of end-users' interaction with information retrieval systems based on results from an exploratory study investigating end-users' search sessions over time with online public access catalogs (OPAC) or CD-ROM databases at different stages of their information seeking related to a current research project. Interviews were conducted with 200 academic end-users to investigate the occurrence of multiple search sessions. Results show that at the time of the interview, 57% of end-users had conducted multiple search sessions during their research project and 86% of end-users conducted their first search session at the beginning stage of their information-seeking process. Forty-nine percent of end-users had conducted between 1 and 6 search sessions and 8% more than 6 search sessions. Seventy percent of multiple search session end-users' had modified their search terms since their first search session. The implications of the findings for end-user training, information retrieval systems design, and further research are discussed. © 1996 John Wiley & Sons, Inc.

In the BlueView project, digital library services are developed and partially implemented based on the architecture of virtual document servers. Using standard tools like full-text databases or information retrieval systems, object-relational database management systems, and replication and caching services, different heterogeneous local document servers have been integrated into one virtual server. Access to heterogeneous local systems can now be performed via a single query interface, integrating features of database query systems and information retrieval or search services. BlueView distributed query processing allows for combining structured queries against meta data with content-based (text retrieval) queries against full-text collections. Legacy systems such as existing preprint servers, search engines, or meta data (catalog) systems can also be integrated. Meta data and full-text documents can be replicated within this distributed architecture. In particular, in the view of members of a certain university, metadata and documents can be replicated to a local document server and integrated under different local user views. Aspects such as terms and conditions, licenses, diverse versions of documents, different formats, and data structures are considered as well. As application scenarios, preprint servers in Mathematics and the ACM SIGMOD Anthology as local document servers have been integrated into a virtual document server.

Humans have used electronic information retrieval (IR) systems for more than 50 years as they evolved from experimental systems to full-scale Web search engines and digital libraries. The fields of library and information science (LIS), cognitive science, human factors and computer science have historically been the leading disciplines in conducting research that seeks to model human interaction with IR systems for all kinds of information related behaviors. As technology problems have been mastered, the theoretical and applied framework for studying human interaction with IR systems has evolved from systems-centered to more user-centered, or cognitive-centered approaches. However, cognitive information retrieval (CIR) research that focuses on user interaction with IR systems is still largely under-funded and is often not included at computing and systems design oriented conferences. But CIR-focused research continues, and there are signs that some IR systems designers in academia and the Web search business are realizing that user behavior research can provide valuable insights into systems design and evaluation. The goal of our book is to provide an overview of new CIR research directions. This book does not provide a history of the research field of CIR. Instead, the book confronts new ways of looking at the human information condition with regard to our increasing need to interact with IR systems. The need has grown due to a number of factors, including the increased importance of information to more people in this information age. Also, IR was once considered document-oriented, but has now evolved to include multimedia, text, and other information objects. As a result, IR systems and their complexity have proliferated as users and user purposes for using them have also proliferated. Human interaction with IR systems can often be frustrating as people often lack an understanding of IR system functionality. New more holistic directions in CIR are emerging that conceptualize human-IR system interaction at the human-computer interaction level (taking into account interface issues), the information behavior level (taking into account the role of IR system interaction in the total range of people’s information behaviors), and the

Session search is an Information Retrieval (IR) task which handles a series of queries issued for a search task. In this paper, we propose a novel reinforcement learning style information retrieval framework and develop a new feedback learning algorithm to model user feedback, including clicks and query reformulations, as reinforcement signals and to generate rewards in the RL framework. From a new perspective, we view session search as a cooperative game played between two agents, the user and the search engine. We study the communications between the two agents; they always exchange opinions on whether the current stage of search is relevant and whether we should explore now. The algorithm infers user feedback models by an EM algorithm from the query logs. We compare to several state-of-the-art session search algorithms and evaluate our algorithm on the most recent TREC 2012 to 2014 Session Tracks. The experimental results demonstrates that our approach is highly effective for improving session search accuracy.

Information Retrieval (IR) systems aim to retrieve data that satisfies certain requirements and constitute an important service in many types of networks, including Delay/Disruption Tolerant Networks (DTNs). In current DTN based IR systems, the data that satisfies a query is assumed to be stored on a single node. Therefore, once a node receives a query in which it has the corresponding data, the query can be resolved completely. However, in scenarios where a query requires data from multiple nodes, these IR systems may fail. Henceforth, in this paper, we propose Distributed Data-Centric Information Retrieval (DDC-IR), a data centric IR system that supports all query types; e.g., continuous and complex. More importantly, it is designed specifically to operate in DTNs. It also incorporates a new packet, aka Query Reply Packet, that includes both a query and one or more replies. We show how this packet facilitates efficient query resolution and enables data centric routing. In addition, it uses caching so that nodes store popular queries that has the effect of speeding up query resolution. We have conducted an extensive simulation study to compare DDC-IR to state of the art IR systems using the popular Random Waypoint model and a trace-file containing student movements on a campus. The results show that DDC-IR is able to resolve 50 % more queries and has an 80 % lower buffer occupancy level than existing IR systems. We also tested DDC-IR in networks with varying sizes. For networks with 100 nodes, DDC-IR is able to resolve queries while current IR systems fail to resolve any queries. In particular, when the number of nodes increases, current IR systems fail to resolve any queries, whilst DDC-IR is able to resolve complex and continuous queries. The influence of the number of sub-queries on query resolution time is also studied. Specifically, when the number of sub-queries in a complex query increases from five to nine, DDC-IR uses 50 % more time to resolve a query. In comparison, prior IR systems fail to resolve any queries.

This paper investigates the impact of three approaches to XML retrieval: using Zettair, a full-text information retrieval system; using eXist, a native XML database; and using a hybrid system that takes full article answers from Zettair and uses eXist to extract elements from those articles. For the content-only topics, we undertake a preliminary analysis of the INEX 2003 relevance assessments in order to identify the types of highly relevant document components. Further analysis identifies two complementary sub-cases of relevance assessments (General and Specific) and two categories of topics (Broad and Narrow). We develop a novel retrieval module that for a content-only topic utilises the information from the resulting answer list of a native XML database and dynamically determines the preferable units of retrieval, which we call Coherent Retrieval Elements. The results of our experiments show that -- when each of the three systems is evaluated against different retrieval scenarios (such as different cases of relevance assessments, different topic categories and different choices of evaluation metrics) -- the XML retrieval systems exhibit varying behaviour and the best performance can be reached for different values of the retrieval parameters. In the case of INEX 2003 relevance assessments for the content-only topics, our newly developed hybrid XML retrieval system is substantially more effective than either Zettair or eXist, and yields a robust and a very effective XML retrieval.

The continuous use of the web for daily operations by businesses, consumers, and the government has created a great demand for reliable web applications. One promising approach to testing the functionality of web applications leverages user-session data collected by web servers. User-session-based testing automatically generates test cases based on real user profiles. The key contribution of this paper is the application of concept analysis for clustering user sessions and a set of heuristics for test case selection. Existing incremental concept analysis algorithms are exploited to avoid collecting and maintaining large user-session data sets and thus to provide scalability. We have completely automated the process from user session collection and test suite reduction through test case replay. Our incremental test suite update algorithm coupled with our experimental study indicate that concept analysis provides a promising means for incrementally updating reduced test suites in response to newly captured user sessions with little loss in fault detection capability and program coverage.

Evaluation is crucial in assessing the effectiveness of new information retrieval and human computer interaction techniques and systems. Relevance judgements are often performed by humans, which makes obtaining them expensive and time consuming. Consequently, relevance judgements are usually performed only on a subset of a given collection of data or experimental results with a focus on the top ranked documents. However, when assessing the performance of exploratory search systems, the diversity or subjective relevance of documents that the user was presented with over a search session are often of more importance than the relative ranking of top documents. In order to perform these types of assessment, all the documents in a given collection need to be judged for relevance. In this paper, we propose an approach based on topic modeling that can greatly accelerate document relevance judgment of an entire document collection with an expert assessor needing to mark only a small subset of documents from a given collection. Experimental results show a substantial overlap between relevance judgments compared to a human assessor.

The recent increase in the volume and variety of video content available online presents growing challenges for video search. Users face increased difficulty in formulating effective queries and search engines must deploy highly effective algorithms to provide relevant results. Although lately much effort has been invested in optimizing video search engine results, relatively little attention has been given to predicting for which queries results optimization is most useful, i.e., predicting which queries will fail. Being able to predict when a video search query would fail is likely to make the video search result optimization more efficient and effective, improve the search experience for the user by providing support in the query formulation process and in this way boost the development of video search engines in general. While insight about a query's performance in general could be obtained using the well-known concept of query performance prediction (QPP), we propose a novel approach for predicting a failure of a video search query in the specific context of a search session. Our 'context-aware query failure' prediction approach uses a combination of 'user indicators' and 'engine indicators' to predict whether a particular query is likely to fail in the context of a particular search session. User indicators are derived from the search log and capture the patterns of query (re)formulation behavior and the click-through data of a user during a typical video search session. Engine indicators are derived from the video search results list and capture the visual variance of search results that would be offered to the user for the given query. We validate our approach experimentally on a test set containing 1+ million video search queries and show its effectiveness compared to a set of conventional QPP baselines. Our approach achieves a 13% relative improvement over the baseline.

Gathering information and making decisions based on retrieved information are the important tasks that every intelligence analyst is doing. User modeling techniques have been exploited to help analysts to search for information effectively. To justify the effects of any user modeling technique on helping analysts retrieve quality documents relevant to their tasks, we need to have a comprehensive evaluation method, which assesses the improvement of retrieval performance and user performance. In this paper, we describe our evaluation of a cognitive user model for information retrieval with regards to retrieval performance. Our user model captures user intent dynamically by analyzing behavioral information from retrieved relevant documents for improving the retrieval performance and user's performance. In this evaluation, we assess the user model's short-term effects on a single query, and the user model's long-term effects on the whole search session. We compare our approach with the best traditional approach for relevance feedback in information retrieval, the Ide dec-hi, which is the approach of modifying queries using term frequency from relevant/non-relevant documents. We use the oldest collection of information retrieval on aerodynamics called CRANFIELD. The results of this evaluation show that by exploring user intent, we achieve competitive performance in the feedback run compared to Ide dec-hi. At the same time, our user model approach offers the advantages of retrieving more quality documents at the initial run compared to the term frequency inverted document frequency (TFIDF) approach. Our results have shown that our user modeling approach can be used to improve efficiency, learnability and interactivity of an information retrieval system.

Estimation of session duration for an e-commerce search engine is important for various downstream applications, including user satisfaction prediction, personalization, and diversification of search results. It has been shown in previous studies that search session length has a strong correlation with user's explore vs specific purchase intent. Based on previous work [14], we hypothesize that early prediction of session length distribution can be used to control the degree of explore vs exploit (loosely related to diversification v/s personalization) for Search Engine Result Pages (SERPs) in the user's session to follow. In this work, we try to early predict the user's session length, which will enable the control on explore v/s exploit of the search results. Towards this end, based on previous work and strong empirical evidence, we hypothesize session lengths are Weibull distributed and propose its parameters being modeled by a Recurrent Neural Network over actions in user's search sessions. Through experimentation, we demonstrate that our method performs better as compared to strong baselines for the same.

Purpose – People often engage in different information-seeking strategies (ISSs) within a single information-seeking episode. A critical concern for the design of information retrieval (IR) systems is how to provide support for these different behaviors in a manner which searchers can easily understand, navigate and use, as they move from one ISS to another. The purpose of this paper is to describe a dialogue structure that was implemented in an experimental IR system, in order to address this concern. Design/methodology/approach – The authors conducted a user-centered experiment to evaluate the IR systems. Participants were asked to search for information on two different task types, with four different topics per task, in both the experimental system and a baseline system emulating state-of-the-art IR systems. The authors report here the results related explicitly to the use of the experimental system's dialogue structure. Findings – For one of the task types, most participants followed the search steps as predicted in the dialogue structures, and those who did so completed the task in fewer moves. For the other task type, predicted order of moves was often not followed, but participants again used fewer moves when following the predicted order. Results demonstrate that the dialogue structures the authors designed indeed support effective human information behavior patterns in a variety of ways, and that searchers can effectively use a system which changes to support different ISSs. Originality/value – This study shows that it is both possible and beneficial, to design an IR system which can support multiple ISSs, and that such a system can be understood and used successfully.