Special issue on interactive question answering: Introduction

Question-answering systems are recognized as popular and frequently effective means of information seeking on the web. In such systems, information seekers can receive a concise response to their queries by presenting their questions in natural language. Interactive question answering is a recently proposed and increasingly popular solution that resides at the intersection of question answering and dialogue systems . On the one hand, the user can ask questions in normal language and locate the actual response to her inquiry; on the other hand, the system can prolong the question-answering session into a dialogue if there are multiple probable replies, very few, or ambiguities in the initial request. By permitting the user to ask more questions, interactive question answering enables users to interact with the system and receive more precise results dynamically. This survey offers a detailed overview of the interactive question-answering methods that are prevalent in current literature. It begins by explaining the foundational principles of question-answering systems, hence defining new notations and taxonomies to combine all identified works inside a unified framework. The reviewed published work on interactive question-answering systems is then presented and examined in terms of its proposed methodology, evaluation approaches, and dataset/application domain. We also describe trends surrounding specific tasks and issues raised by the community, so shedding light on the future interests of scholars. Our work is further supported by a GitHub page synthesizing all the major topics covered in this literature study. https://sisinflab.github.io/interactive-question-answering-systems-survey/

Purpose Prior Information Retrieval (IR) research synthesizes progress from individual studies, yet academia-industry collaboration dynamics remain unexplored. This study investigates: (1) productivity patterns and venues, (2) citations-downloads relationships, (3) topic evolution, and (4) collaboration trends. Design/methodology/approach We perform an analysis of 53,471 ACM IR papers (2000–2018) using bibliometrics and DistilBERT topic modeling. Findings We find that industry-involved papers preferred WWW/CIKM venues; collaborations dominated RecSys/CSCW. We see that academia-industry collaborations achieved the highest download-to-citation conversion rates. Academia focused on algorithms; industry on applications; collaborations bridged both with rising human-centered themes. Research implications This is a pioneering large-scale bibliometrics revealing collaboration’s impact on IR knowledge evolution and provides a methodological framework for cross-sector analysis. Practical implications The paper identifies optimal venues (RecSys/CSCW) for partnerships and guides joint initiatives (shared datasets, grants) to bridge academia-industry divides and enhance research translation. Originality/value This is the first large-scale bibliometric analysis of IR academia-industry collaboration. The paper finds many novel insights, including the fact that collaboration boosts citation efficiency, enables complementary specialization, and drives topic convergence.

Humans have used electronic information retrieval (IR) systems for more than 50 years as they evolved from experimental systems to full-scale Web search engines and digital libraries. The fields of library and information science (LIS), cognitive science, human factors and computer science have historically been the leading disciplines in conducting research that seeks to model human interaction with IR systems for all kinds of information related behaviors. As technology problems have been mastered, the theoretical and applied framework for studying human interaction with IR systems has evolved from systems-centered to more user-centered, or cognitive-centered approaches. However, cognitive information retrieval (CIR) research that focuses on user interaction with IR systems is still largely under-funded and is often not included at computing and systems design oriented conferences. But CIR-focused research continues, and there are signs that some IR systems designers in academia and the Web search business are realizing that user behavior research can provide valuable insights into systems design and evaluation. The goal of our book is to provide an overview of new CIR research directions. This book does not provide a history of the research field of CIR. Instead, the book confronts new ways of looking at the human information condition with regard to our increasing need to interact with IR systems. The need has grown due to a number of factors, including the increased importance of information to more people in this information age. Also, IR was once considered document-oriented, but has now evolved to include multimedia, text, and other information objects. As a result, IR systems and their complexity have proliferated as users and user purposes for using them have also proliferated. Human interaction with IR systems can often be frustrating as people often lack an understanding of IR system functionality. New more holistic directions in CIR are emerging that conceptualize human-IR system interaction at the human-computer interaction level (taking into account interface issues), the information behavior level (taking into account the role of IR system interaction in the total range of people’s information behaviors), and the

Interactive Question Answering (IQA) requires an intelligent agent to interact with a dynamic environment in order to gather information necessary to answer a question. IQA tasks have been proposed as means of training systems to develop language or visual comprehension abilities. To this end, the Question Answering with Interactive Text (QAit) task was created to produce and benchmark interactive agents capable of seeking information and answering questions in unseen environments. While prior work has exclusively focused on IQA as a reinforcement learning problem, such methods suffer from low sample efficiency and poor accuracy in zero-shot evaluation. In this paper, we propose the use of the recently proposed Decision Transformer architecture to provide improvements upon prior baselines. By utilising a causally masked GPT-2 Transformer for command generation and a BERT model for question answer prediction, we show that the Decision Transformer achieves performance greater than or equal to current state-of-the-art RL baselines on the QAit task in a sample efficient manner. In addition, these results are achievable by training on sub-optimal random trajectories, therefore not requiring the use of online agents to gather data.

Welcome to the 40th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2017). SIGIR is the premier scientific conference in the broad area of information retrieval. We received a high-quality set of full-paper submissions to consider for inclusion in the conference program. We thank the 66 Senior Program Committee (SPC) members, 216 Program Committee (PC) members, and at least 70 additional reviewers for their contributions to paper selection. This pool of committed SIGIR volunteers was based in 33 countries and over 170 institutions, spanning academia, industry, and beyond. We recognized 8 SPC members and 29 PC members / additional reviewers whose contributions to the review process were outstanding. We reviewed 362 papers for the full paper track, and accepted 78, for an acceptance rate of 22%. In total, there were 1056 authors from 42 countries: 299 from China, 232 from the United States, and 525 from the rest of the world. The countries (with at least 10 submissions) with the highest acceptance rates were the United Kingdom (42%), Italy (42%), Singapore (38%), Australia (27%), and the United States (24%). The most popular call-for-papers topics for accepted papers were ranking algorithms (19%), text representation (17%), behavioral analysis (15%), recommender systems (14%), and user studies (14%). We made several modifications to the paper submission and review process for SIGIR 2017. We transitioned to EasyChair, one of three conference submission applications supported by ACM. We significantly updated the call for papers to broaden the range of topics considered in scope for the conference, and more accurately reflect current research trends. To boost diversity, we increased the number of women SPC members (24% this year, up from 15% last year) and increased the number of SPC members based in Asia (23% this year, up from 8% last year). To help improve review quality, we also required that PC members have at least one full paper at SIGIR and be active at the SIGIR conference in the past five years (some limited exceptions were permitted). The largest change was a modification to the role of the secondary SPC, with the goal of better utilizing their expertise and making better decisions for borderline papers. Each paper had a primary SPC member, who was responsible for writing the metareview and making a recommendation regarding the paper's disposition. If they believed it was necessary, the primary SPC could request an additional review from a secondary SPC member, who did not see the other reviews or the discussion to date. Once they submitted the additional review, the secondary SPC member could see the other reviews and participate in the discussion. The additional SPC review informed discussion and helped the primary SPC make a recommendation. Overall, secondary SPC reviews were provided for 20% of full papers. We ran an in-person SPC meeting at Microsoft in Redmond, WA. To allow more SPC members to take part, we enabled remote participation via video conference. Approximately half of the SPC participated, either in person or remotely. SPC members were asked to submit their full-paper recommendations prior to the meeting. During the meeting, we discussed papers flagged by SPCs as requiring discussion, as well as papers with "accept" and "reject" recommendations. Some of the papers marked as "reject" by SPC members before the meeting had strong support from at least one PC member. A subset of those full papers ended up being included in the conference program after a detailed discussion. The short paper track received a record number of 398 submissions and accepted 121 (30%) of them. The track was chaired by Hui Fang, Mounia Lalmas and Yiqun Liu. The demo paper track received 36 submissions and accepted 17 (47%) of them. The track was chaired by Gareth Jones and Takehiro Yamamoto. The SIGIR Symposium on IR in Practice (SIRIP 2017) had seven invited talks and five accepted regular papers. The track was chaired by Sumio Fujita and Vanessa Murdock. The doctoral consortium track received 14 submissions of which 9 (64%) were accepted. The track was chaired by Makoto P. Kato and Kira Radinsky. The tutorial track received 14 proposals of which 8 (57%) were accepted. The track was chaired by Fernando Diaz and Adam Jatowt. The workshop track received 12 proposals of which 8 (67%) were accepted. The track was chaired by Jaap Kamps and Grace Hui Yang.

INTRODUCTION:Increasing numbers of research papers about information retrieval for Health Technology Assessments (HTA), systematic reviews and other evidence syntheses are being published. It is time-consuming for information specialists to keep up-to-date with the latest developments in the field. To help searchers with this challenge, the Interest Group on Information Retrieval (IRG) of Health Technology Assessment International (HTAi) has compiled the best available research evidence on information retrieval aspects into an open-access web resource: Summarized Research in Information Retrieval for HTA (SuRe Info). The resource can be accessed at http://www.sure-info.orgMETHODS:The Sure Info team run topic-specific search strategies in selected relevant databases to identify information retrieval methods publications that fulfil the SuRe Info inclusion criteria. Eligible publications receive a structured abstract containing a brief critical appraisal. Key messages for search practice based on the appraisals and accepted best practice are summarized into topic-specific chapters.RESULTS:SuRe Info currently offers fourteen chapters, with more in development. SuRe Info chapters fall into two categories: (i) chapters about general search methods that are used across all types of research, such as how to develop search strategies and the availability and use of search filters, and (ii) chapters summarizing the methods to use when searching for specific aspects of HTA (as defined in the European Network for HTA (EUnetHTA) HTA Core Model®), including searching for evidence on clinical effectiveness and safety, and identifying economic evaluations. References at the end of each chapter are linked to appraisals of publications that have been used to develop each chapter. Links to the full-text of the publications are provided when freely available. The SuRe Info chapters are reviewed every six months and updated if new evidence is identified or if resources change.CONCLUSIONS:SuRe Info is a unique resource, identifying and summarizing current best research evidence on information retrieval aspects for HTA. It supports the timely uptake of potential efficiencies arising from new evidence that may be incorporated into the evidence identification processes of HTA organizations.

For many years, vector space models have been used in information retrieval and computational linguistics to represent terms, queries, and documents, using vector addition as a simple operator to model semantic composition. Though surprisingly successful, many aspects of meaning including word order, typed relationships, and nested structures are not captured by this modelling process.

Policy learning is an active topic in dialogue systems research, but it has not been explored in relation to interactive question answering (IQA). We take a first step in learning adaptive interaction policies for question answering : we address the question of how to acquire enough reliable query constraints, how many database results to present to the user and when to present them, given the competing trade-offs between the length of the answer list, the length of the interaction, the type of database and the noise in the communication channel. The operating conditions are reflected in an objective function which we use to derive a hand-coded threshold-based policy and rewards to train a reinforcement learning policy. The same objective function is used for evaluation. We show that we can learn strategies for this complex trade-off problem which perform significantly better than a variety of hand-coded policies, for a wide range of noise conditions, user types, types of DB and turn-penalties. Our policy learning framework thus covers a wide spectrum of operating conditions. The learned policies produce an averagerelativeincrease in reward of 86.78% over the hand-coded policies. In 93% of the cases the learned policies perform significantly better than the hand-coded ones (p< .001). Furthermore we show that the type of database has a significant effect on learning and we give qualitative descriptions of the learned IQA policies.

Evaluating interactive question answering (QA) systems with real users can be challenging because traditional evaluation measures based on the relevance of items returned are difficult to employ since relevance judgments can be unstable in multi-user evaluations. The work reported in this paper evaluates, in distinguishing among a set of interactive QA systems, the effectiveness of three questionnaires: a Cognitive Workload Questionnaire (NASA TLX), and Task and System Questionnaires customized to a specific interactive QA application. These Questionnaires were evaluated with four systems, seven analysts, and eight scenarios during a 2-week workshop. Overall, results demonstrate that all three Questionnaires are effective at distinguishing among systems, with the Task Questionnaire being the most sensitive. Results also provide initial support for the validity and reliability of the Questionnaires.

Information retrieval (IR) systems aim to select relevant material from large collections of information in response to user queries. The approaches used to accomplish this have been the focus of much research and development over the past 50 years, and have led to the algorithms underlying many commercial and Web-based search engines today. This entry describes the common components that go into the design of IR systems (from text processing to inverted file indexes). The major classes (or models) of retrieval algorithms (Boolean, vector, and probabilistic) are described along with formal definitions of the basic form of these algorithms and some of the variations in common use in IR research. In addition, the entry examines query expansion techniques, and in particular relevance feedback, and how they are used in IR systems

Ongoing research in information retrieval offers new approaches, demonstrating that the traditional methods may have reached their limitations. In fact, actual results to date have shown that advanced systems will demonstrate effectiveness at combining multiple text and user need representations. The paper suggests supplementing the traditional conceptual abstracts with a syntactical description designed to improve retrieval performance. This choice is carefully motivated and examples are given. Without attempting to reason about the document contents, a domain specific knowledge representation is used to form clusters of concepts. The point is to represent the joint structure of the component concepts involved in a composite concept. This approach extends the reach of key concept assignment to: interactive query elicitation; multiword phrase search; finer grained thesaurus relationships. The paper emphasizes that intellectual assignment of features is part of an artificial intelligence approach to information retrieval. The principles of implementation are also described.

Academic research in information retrieval did not make its way into commercial retrieval products until the last 15 years. Early web search engines also made little use of information retrieval research, in part because of significant differences in the retrieval environment on the Web, such as higher transaction volume and much shorter queries. Recently, however, academic research has taken root in search engines. This paper describes recent developments with a probabilistic retrieval model originating prior to the Web, but with features which could lead to effective retrieval on the Web. Just as graph structure algorithms make use of the graph structure of hyper-linking on the Web, which can be considered a form of relevance judgments, the model of this paper suggests how relevance judgments of web searchers, not just web authors, can be taken into account in ranking. This paper also shows how the combination of expert opinion probabilistic information retrieval model can be made computationally efficient through a new derivation of the mean and standard deviation of the model’s main probability distribution.

BackgroundTest collections for information retrieval are scarce. Domain specific test collections even more so, and medical test collections in the Swedish language non-existent prior to the making of the MedEval test collection. Most research in information retrieval has been performed in the English language, thus most test collections contain English documents. However, English is morphologically poor compared to many other European languages and a number of interesting and important aspects have not been investigated. Building a medical test collection in Swedish opens new research opportunities.MethodsThis article describes the making of and potential uses of MedEval, a Swedish medical test collection with assessments, not only for topical relevance, but also for target reader group: Doctors or Patients. A user of the test collection may choose if she wishes to search in the Doctors or the Patients scenario where the topical relevance assessments have been adjusted with consideration to user group, or to search in a scenario which regards only topical relevance.In addition to having three user groups, MedEval, in its present form, has two indexes, one where the terms are lemmatized and one where the terms are lemmatized and the compounds split and the constituents indexed together with the whole compound.ResultsDifferences discovered between the documents written for medical professionals and documents written for laypersons are presented. These differences may be utilized in further studies of retrieval of documents aimed at certain groups of readers. Differences between the groups of documents are, for example, that professional documents have a higher ratio of compounds, have a greater average word length and contain more multi-word expressions.An experiment is described where the user scenarios have been utilized, searching with expert terms and lay terms, separately and in combination in the different scenarios. The tendency discovered is that the medical expert gets best results using expert terms and the lay person best results using lay terms, but also quite good results using expert terms or lay and expert terms in combination.ConclusionsThe many features of MedEval gives a variety of research possibilities, such as comparing the effectiveness of search terms when it comes to retrieving documents aimed at the different user groups or to study the effect of compound decomposition in retrieval of documents. As Swedish, the language of MedEval, is a morphologically more complex language than English, it is possible to study additional aspects of the effect of natural language processing in information retrieval, for example utilizing different inflectional word forms in the retrieval of expert vs lay documents. MedEval is the first Swedish test collection of the medical domain.AvailabilityThe Department of Swedish at the University of Gothenburg is in the process of making the MedEval test collection available to academic researchers.

Research in Information Retrieval has progressed against a background of rapidly increasing corpus size and heterogeneity, with every advance in technology quickly followed by a desire to organize and search more unstructured, more heterogeneous, and even bigger corpora. But as retrieval problems get larger and more complicated, evaluating the ranking performance of a retrieval engine gets harder: evaluation requires human judgments of the relevance of documents to queries, and for very large corpora the cost of acquiring these judgments may be insurmountable. This cost limits the types of problems researchers can study as well as the data they can be studied on. We present methods for understanding performance differences between retrieval engines in the presence of missing and noisy relevance judgments. The work introduces a model of the cost of experimentation that incorporates the cost of human judgments as well as the cost of drawing incorrect conclusions about differences between engines in both the training and testing phases of engine development. Through adopting a view of evaluation that is more concerned with distributions over performance differences rather than estimates of absolute performance, the expected cost can be minimized so as to reliably differentiate between engines with less than 1% of the human effort that has been used in past experiments.

Purpose – People often engage in different information-seeking strategies (ISSs) within a single information-seeking episode. A critical concern for the design of information retrieval (IR) systems is how to provide support for these different behaviors in a manner which searchers can easily understand, navigate and use, as they move from one ISS to another. The purpose of this paper is to describe a dialogue structure that was implemented in an experimental IR system, in order to address this concern. Design/methodology/approach – The authors conducted a user-centered experiment to evaluate the IR systems. Participants were asked to search for information on two different task types, with four different topics per task, in both the experimental system and a baseline system emulating state-of-the-art IR systems. The authors report here the results related explicitly to the use of the experimental system's dialogue structure. Findings – For one of the task types, most participants followed the search steps as predicted in the dialogue structures, and those who did so completed the task in fewer moves. For the other task type, predicted order of moves was often not followed, but participants again used fewer moves when following the predicted order. Results demonstrate that the dialogue structures the authors designed indeed support effective human information behavior patterns in a variety of ways, and that searchers can effectively use a system which changes to support different ISSs. Originality/value – This study shows that it is both possible and beneficial, to design an IR system which can support multiple ISSs, and that such a system can be understood and used successfully.