Creating an Academic Conversational Agent for Dynamic Information Retrieval

Purpose Human–agent interaction (HAI) is increasingly influencing our personal and work lives through the proliferation of conversational agents (CAs) in various domains. As such, these agents combine intuitive natural language interactions by also delivering personalization through artificial intelligence capabilities. However, research on CAs as well as practical failures indicates that CA interaction oftentimes fails miserably. To reduce these failures, this paper introduces the concept of building common ground for more successful HAIs. Design/methodology/approach Based on a systematic literature analysis, we identified 38 articles meeting the eligibility criteria. We critically reviewed this body of knowledge within a formal narrative synthesis structured around the use of common ground in the interaction with CAs. Findings Based on the systematic review, our analysis reveals five mechanisms for achieving common ground: embodiment, social features, joint action, knowledge base and mental model of conversational agent. We point out the relationships between these mechanisms as they are related to each other in directional and bidirectional ways. Research limitations/implications Our findings contribute to theory with several implications for CA research. First, we provide implications about the organization of common ground mechanisms for CAs. Second, we provide insights into the mechanisms and nomological network for achieving common ground when interacting with CAs. Third, we provide a broad research agenda for future CA research that centers around the important topic of common ground for HAI. Originality/value We offer novel insights into grounding mechanisms and highlight the potentials when considering common ground in different HAI processes. Consequently, we secure further understanding and deeper insights of possible mechanisms of common ground to shape future HAI processes.

Production information retrieval is an important issue for manufacturing industries. Effective data transmission and retrieval mechanism with data protection is an urgent research topic. Therefore, a socket information retrieval mechanism and agent services for information retrieval on the basis of injection molding machines is proposed in this research and is addressed as Production Information Retrieval Agent Architecture (PIRAA). The proposed agent adapted a customized packet format for client-server connection communication and sent process parameters back to a production process database. Using the experimental results, a cloud-based platform was implemented based on injection molding machines. The results show the PIRAA framework can be used for developing cloud-based platforms in a stabilized and remarkable way.

Conversational agent technology is an emerging paradigm for creating a social environment in online groups that is conducive to effective teamwork. Prior work has demonstrated advantages in terms of learning gains and satisfaction scores when groups learning together online have been supported by conversational agents that employ Balesian social strategies. This prior work raises two important questions that are addressed in this article. The first question is one of generality. Specifically, are the positive effects of the designed support specific to learning contexts? Or are they in evidence in other collaborative task domains as well? We present a study conducted within a collaborative decision-making task where we see that the positive effects of the Balesian social strategies extend to this new context. The second question is whether it is possible to increase the effectiveness of the Balesian social strategies by increasing the context sensitivity with which the social strategies are triggered. To this end, we present technical work that increases the sensitivity of the triggering. Next, we present a user study that demonstrates an improvement in performance of the support agent with the new, more sensitive triggering policy over the baseline approach from prior work. The technical contribution of this article is that we extend prior work where such support agents were modeled using a composition of conversational behaviors integrated within an event-driven framework. Within the present approach, conversation is orchestrated through context-sensitive triggering of the composed behaviors. The core effort involved in applying this approach involves building a set of triggering policies that achieve this orchestration in a time-sensitive and coherent manner. In line with recent developments in data-driven approaches for building dialog systems, we present a novel technique for learning behavior-specific triggering policies, deploying it as part of our efforts to improve a socially capable conversational tutor agent that supports collaborative learning.

Conversational agents have been widely adopted in dialogue systems for various business purposes. Many existing conversational agents are rule-based and require significant human intervention to adapt the knowledge and conversational flow. In this paper, we propose a graph-based adaptive conversational agent model which is capable of learning knowledge from human beings and adapting the knowledge-base according to human-agent interactions. Studies to evaluate the proposed model are conducted and presented, which compare the responses from the proposed adaptive agent model and a conventional agent.

As information retrieval (IR) systems, such as search engines and conversational agents, become ubiquitous in various domains, the need for transparent and explainable systems grows to ensure accountability, fairness, and unbiased results. Despite recent advances in explainable AI and IR techniques, there is no consensus on the definition of explainability. Existing approaches often treat it as a singular notion, disregarding the multidimensional definition postulated in the literature. In this paper, we use psychometrics and crowdsourcing to identify human-centered factors of explainability in Web search systems and introduce SSE (Search System Explainability), an evaluation metric for explainable IR (XIR) search systems. In a crowdsourced user study, we demonstrate SSE's ability to distinguish between explainable and non-explainable systems, showing that systems with higher scores indeed indicate greater interpretability. We hope that aside from these concrete contributions to XIR, this line of work will serve as a blueprint for similar explainability evaluation efforts in other domains of machine learning and natural language processing.

The Conversational Agents of today cannot indulge in a continuous conversation with a human due to the Conversational Agents inability to infer and apply context from previous interactions. The Masters Thesis is intended to address this issue by building context modeling for the Conversational Agents for the scenario of Searching for Flights and answering the research question of what context is needed to provide a natural(-ish) conversation with an agent that will result in the user successfully searching for flights.

Conversational agents are getting more popular and applied in a wide range of practical application areas. The main task of these agents is not only to generate context-appropriate responses to a given query but also to make the conversation human-like. Thanks to the ability of deep learning based models in natural language modeling, recent studies have made progress in designing conversational agents that can provide more semantically accurate responses. However, the naturalness in such conversation setting has not been given adequate attention in these studies. This paper aims to incorporate both important criteria of accuracy and naturalness of conversation in developing a new model for conversational agents. To this end, inspired by the idea of Turing test and the idea of adversarial learning strategy, we propose to design a model based on generative deep neural networks that interestingly allow to generate accurate responses optimized by the mechanics of imitating human-generated conversations. Experimental results demonstrate that the proposed models produce more natural and accurate responses, yielding significant gains in BLEU scores.

Pregnant women constantly need some information to support nutritional decisions during pregnancy, and many do not receive such assistance at all. This study aims to present a conversational agent to provide reliable information to pregnant women, focusing on nutritional education and evaluating the perception of pregnant women and health professionals about the agent. As a scientific contribution, this article developed and implemented a conversational agent in a real environment capable of generating reliable responses on the basis of a set of health documents. We proposed an intervention study with 25 women and 10 healthcare providers through a survey to measure the perceptions of these groups towards conversational agents. The results show that the intended design could ensure positive support for pregnant women, clarify certain issues for the public, and remove some knowledge barriers. The results showed no significant difference between the groups (p-value = 0.713). Depending on the perception of the pregnant group, the conversational agent model can teach new knowledge during the prenatal period (Mean = 4.56). The model presented for health professionals could already be indicated as a support tool for pregnant women (Mean = 4.7).

Generally, to start the operation of a chatbot based on information retrieval, real data generated by a domain expert is needed so that the chatbot can start its interaction and continue improving from it. It takes a lot of time and cost to create knowledge bases with all the linguistic variations. This work proposes and evaluates a data augmentation methodology for a Portuguese language corpus, based on paraphrase and reverse translation, focused on generating data to categorize intentions in conversational agents in order to reduce the time to obtain and prepare quality data. The experiments carried out in the data augmentation task and evaluated in an NLU model showed that the method is effective, presenting an average gain in accuracy of 6%.

It is our great pleasure to welcome you to New Brunswick, New Jersey and the third ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2018). CHIIR, as you may know, has roots in two previously successful communities: The Information Interaction in Context (IIiX) conference and the Human Computer Information Retrieval (HCIR) symposium, which ran from 2006 and 2007 respectively. These meetings were focused on issues that involved Human Information Behavior (HIB), Interactive Information Retrieval (IIR) and Human-Computer Interaction (HCI). Those roots, as well as decades of research and development on information seeking and other forms of information interaction, human-centered design, and information retrieval are the backbone of CHIIR. At the same time, this vibrant new community has picked up more scholars and themes as it continues to attract new students and professionals, as well as emerging research topics that are evident in this year's program. In fact, CHIIR is one of the few forums that explicitly supports presentation of bold new ideas through its Perspective Papers category. In addition, we have picked up Demos as a new category for presentations this year. All of these wonderful things could not be possible without a large number of people making significant contributions. We are quite fortunate to have Katriina Byström, Falk Scholer, and Jeff Huang as our Program Chairs this year. These three brave souls took upon a tremendous effort of working with dozens of submissions, reviewers, and meta-reviewers and constructing a high-quality program while coordinating from three different continents. Soo Young Rieh and Preben Hansen, on the other hand dealt with higher than expected load for short papers and still managed to get them reviewed and assessed on time. Even our Doctoral Consortium (DC) this year ended up being quite challenging as we received twice as many submissions as before, but our fine DC chairs Jaceck Gwizdka and Vivek Singh found world-class mentors for these doctoral students. Thanks to our Workshops and Tutorials Chair Xiaojun Yuan, we have a fantastic lineup of tutorials on the first day and workshops on the last day. When the whole program was ready for the conference, it was our Proceedings Chair Roberto Gonzalez-Ibanez who managed to put together these proceedings that you are holding in your hands, or more likely viewing in the ACM Digital Library! We are thankful for having an energetic and resourceful local arrangement committee with Michael Cole as the Treasurer, Fernando Diaz as the Sponsorship Chair, Kaitlin Costello as the Local Organizing Chair, and Sunyoung Kim and Matt Mitsui as the Student Volunteers Chairs. And it was our Publicity Chair Anita Crescenzi, who helped us with not only the outreach efforts, but also with the registration system. Finally, none of these would have happened without the authors and reviewers from more than 30 countries whose contributions are reflected in every aspect of this conference. So, thank you all!

Collaborative Information Retrieval in an information-intensive domain

Providing educational services to students with disabilities requires a unique combination of professional knowledge, and adherence to complex compliance and auditing goals. This study investigates the information access gaps within special education health services. Through think-aloud activities with therapy supervisors, practitioners, and administrative staff, we elicited explicit and implicit information needs, and identified significant challenges in current information seeking and documentation practices in this highly regulated environment. Thematic analysis of 329 coded segments (including 120 task descriptions, 123 pain points, and 86 desired features) reveals systematic mismatches between information capture and retrieval practices, which practitioners navigate daily, highlighting a necessity for information seeking beyond traditional search and demonstrating use cases for AI-augmented information retrieval. We present design implications, developed in collaboration with these stakeholders, focused on enhancing task documentation and designing interaction to support these needs, offering a roadmap for future information interaction systems in this complex, multi-stakeholder domain.

The importance of biomedical concept relationships and document concept interrelationships are discussed and some of the ways in which concept relationships have been used in information search and retrieval are reviewed. We look at examples of innovative approaches utilizing biomedical concept identification and relationships for improved document and information retrieval and analysis that support knowledge creation and management.

People increasingly use large language model (LLM)-based conversational agents to obtain information. However, the information these models provide is not always factually accurate. Thus, it is critical to understand what helps users adequately assess the credibility of the provided information. Here, we report the results of two preregistered experiments in which participants rated the credibility of accurate versus partially inaccurate information ostensibly provided by a dynamic text-based LLM-powered agent, a voice-based agent, or a static text-based online encyclopedia. We found that people were better at detecting inaccuracies when identical information was provided as static text compared to both types of conversational agents, regardless of whether information search applications were branded (ChatGPT, Alexa, and Wikipedia) or unbranded. Mediation analysis overall corroborated the interpretation that a conversational nature poses a threat to adequate credibility judgments. Our research highlights the importance of presentation mode when dealing with misinformation.

Task complexity is a critical task characteristic that influences users' information seeking and search behavior. In general, researchers differentiate task complexity from objective and subjective task complexity. Both significantly affect task performance. However, few studies have been done to examine how task complexity could be measured from users' perspective in information science. The present study identifies a set of objective and subjective measures and conducted a survey. The survey asked users to judge the complexity of a task, and then give the reasons why they made that judgment based on the measures. Six simulated task situations were developed for the survey and 168 valid questionnaires (84% return rate) were analyzed. The results indicate that the number of words hard to understand, the number of languages required for search results, and the number of domain areas involved in a task could significantly predict task complexity. The study helps further understand the attributes of task complexity and has implications in research on interactive information retrieval (IIR), task‐based information seeking and search, and personalization of information retrieval (IR).