Misalignment in Semantic User Model Elicitation via Conversational Agents: A Case Study in Navigation Support for Visually Impaired People

This paper presents an adaptive e-learning system which supports collaborative learning based on a location-based social network and semantic modeling. In the system, a social network among e-learning learners is dynamically constructed on the basis of the location information of learners using GPS sensors for collaborative learning. In addition, user modeling for semantic social network and adaptive e-learning is supported. For these, a mechanism that supports the construction of a dynamic social network using the location information of smart phones is provided. The user modeling for the construction of a semantic social network and the support of adaptive e-learning considers the information such as the users' achievement, preference, and learning goal. Through this system, e-learners can find mentors who are able to facilitate their learning and create communities for collaborative learning offline and online. The support of a location-based social network service and semantic user model in our system would increase interactions among e-learners and improve satisfaction regarding their mobile learning environment.

Conversational User Interfaces (CUI) on mobile devices are the most accessible and widespread examples of voice-based interaction in the wild. This paper presents a survey of mobile conversation user interface research since the commercial deployment of Apple's Siri, the first readily available consumer CUI. We present and discuss Text Entry & Typing, Application Control, Speech Analysis, Conversational Agents, Spoken Output, & Probes as the prevalent themes of research in this area. We also discuss this body of work in relation to the domains of Health & Well-being, Education, Games, and Transportation. We conclude this paper with a discussion on Multi-modal CUIs, Conversational Repair, and the implications for CUIs of greater access to the context of use.

The use of conversational agents to support team-based training in virtual environments is a nascent area of research within the Computer Supported Collaborative Work (CSCW) and Conversational User Interface (CUI) communities. To support this research, we developed a Wizard of Oz (WoZ) system enabling experimenters to play the role of the conversational agent who must work with a team of three human participants to herd together a group of autonomous robots (target agents) within a virtual task environment. The experimenter can select from one of several tailored responses. The aim of this environment is to determine whether the identified responses facilitate effective team performance and to collect training data to build a conversational agent capable of engaging in team based virtual learning environments.

This white paper examines the necessity and viability of integrating Emotional Intelligence (EI) into Conversational AI Agents (CAs) to address the limitations of traditional, sentiment-based customer support systems. Drawing upon affective computing and organizational psychology, the study establishes a theoretical framework for computational empathy, detailing the multimodal architectural requirements for robust emotion recognition across linguistic and acoustic channels.1 Synthesis of recent case studies indicates that advanced engagement strategies, notably Reinforcement Learning (RL), deliver significant performance gains, including a reported +15.2% increase in customer satisfaction (CSAT). Crucially, the deployment of these systems must navigate profound ethical challenges, including inherent algorithmic bias stemming from cultural variability and the critical risks of privacy invasion and emotional manipulation. The paper concludes by proposing a design framework centered on transparency, accountability, and cultural sensitivity, ensuring that EICAs transition the customer service domain from operational efficiency to one of genuine emotional resonance and relational value. Keywords Emotional AI, Affective Computing, Customer Service, Empathy Modeling, Conversational Agents, Human–AI Interaction, Algorithmic Bias, Multimodal Systems.

Maximizing educational impacts with learning technologies is one of the areas that researchers and practitioners are concerned about in the field of Human-Computer Interaction (HCI) and human-centered artificial intelligence (HCAI). In this case study, we report user experiences and lessons learned of the Enactive Virtual Environment for teaching practice (EVETeach) with AI-powered virtual student agents called Evelyn. We conducted a user study with a case study research design. We collected multiple sources of data from 24 student teachers, including participatory observations, field notes, semi-structured interviews, computer-based conversation logs, audio-, video-, and screen-recordings, and a cognitive walkthrough. We identified the following salient emerging findings as lessons learned: 1) Student teachers value and relate to the teaching practices in virtual reality simulation with AI-powered conversational agents, 2) AI-powered conversational agents inject humor to facilitate situational and social teaching practice, and 3) AI-powered conversational student agents maintain authentic discourse to promote student teachers’ pedagogical reasoning.

Human-computer interaction with people whose visual perception is limited is possible only with tactile and voice interfaces, the latter are being used more and more recently. The aim of the work is to create an intelligent assistant for an indoor navigation system designed for blind and visually impaired people. The development of an intelligent assistant is based on a semantic user model and a four-level ontological model of the subject domain. To build a dialogue between an intelligent assistant and a user, we use the theory of speech acts, argumentation theory and case-based reasoning. The developed software system is aimed at identifying the desires and user needs and proposing possible user actions aimed at achieving them. The system allows for the decomposition of user tasks and the formation of a sequence of their execution based on semantic models of the user and the subject domain.

Conversational agents are becoming increasingly popular. These systems present an extremely rich and challenging research space for addressing many aspects of user awareness and adaptation, such as user profiles, contexts, personalities, emotions, social dynamics, conversational styles, etc. Adaptive interfaces are of long-standing interest for the HCI community. Meanwhile, new machine learning approaches are introduced in the current generation of conversational agents, such as deep learning, reinforcement learning, and active learning. It is imperative to consider how various aspects of user-awareness should be handled by these new techniques. The goal of this workshop is to bring together researchers in HCI, user modeling, and the AI and NLP communities from both industry and academia, who are interested in advancing the state-of-the-art on the topic of user-aware conversational agents. Through a focused and open exchange of ideas and discussions, we will work to identify central research topics in user-aware conversational agents and develop a strong interdisciplinary foundation to address them.

New developments in healthcare require an increased disease self-management of patients. Intelligent digital assistants equipped with a conversational user interface are intended to support patients in this challenging task by providing reminders, answering questions, or supporting in self-monitoring tasks. In this paper, we study the potentials of intelligent conversational agents in healthcare. We realized three systems for three different use cases (patient education, disease management, self anamnesis). Based on these implementations and experiences with usability tests, we performed an analysis of strengths, weaknesses, opportunities and threats (SWOT) using a questionnaire. The results show that conversational agents used in healthcare applications can be helpful. However, they have to be integrated into the healthcare process, supporting also the interaction between the healthcare team and a patient. In order to be attractive for a long-term usage, the scope of operation should autonomously adapt to the current health situation of a patient to provide relevant functionality as needed.

This study evaluates conversational large language models (LLMs) as pedagogical brainstorming tools for historical source code analysis through structured prompt-based approaches adapted from Critical Code Studies (CCS). The research tests whether conversational interfaces like ChatGPT-4o can support initial exploration of complex historical codebases by adapting CCS perspectives into conversational prompt formats. The dual-prompt evaluation separates technical parsing from interpretive reasoning, assessing how effectively conversational interfaces extract structural information while generating preliminary interpretive hypotheses. Using the Apple Lisa source code as a case study, this analysis documents both pedagogical utility and systematic limitations. The findings demonstrate that while conversational LLMs can preserve developer annotations, parse visual artifacts such as ASCII diagrams, and generate educationally valuable insights for approaching unfamiliar programming languages, the conversational user interface significantly constrains systematic analytical capabilities. Through three case studies examining ASCII typography, architectural diagrams, and interface implementations, the analysis illustrates how conversational interfaces function as structured brainstorming partners while necessitating rigorous validation against primary sources. This evaluation contributes to understanding AI’s pedagogical role in digital humanities by positioning conversational LLMs as question-framing tools rather than interpretive authorities, while identifying systematic implementation requirements necessary for rigorous computational approaches to historical software analysis.

The use of Conversational agents (CAs) in healthcare is an emerging field. These CAs seem to be effective in accomplishing administrative tasks, e.g. providing locations of care facilities and scheduling appointments. Modern CAs use machine learning (ML) to recognize, understand and generate a response. Given the criticality of many healthcare settings, ML and other component errors may result in CA failures and may cause adverse effects on patients. Therefore, in-depth assurance is required before the deployment of ML in critical clinical applications, e.g. management of medication dose or medical diagnosis. CA safety issues could arise due to diverse causes, e.g. related to user interactions, environmental factors and ML errors. In this paper, we classify failures of perception (recognition and understanding) of CAs and their sources. We also present a case study of a CA used for calculating insulin dose for gestational diabetes mellitus (GDM) patients. We then correlate identified perception failures of CAs to potential scenarios that might compromise patient safety.

The purpose of this scoping review is to examine the professional representation of conversational agents that are used for health care. Professional characteristics associated with these agents will be identified, and the prevalence of these characteristics will be determined. Conversational agents that are used for health care lack the qualifications and capabilities of real health professionals, but this fact may not be clear to some patients and health seekers. This problem may be exacerbated when conversational agents are described as health professionals or are given professional titles or appearances. To date, the professional representation of conversational agents that are used for health care has received little attention in the literature. This review will include scholarly publications on conversational agents that are used for health care, particularly descriptive/developmental case studies and intervention/evaluation studies. This review will consider conversational agents designed for patients and health seekers, but not health professionals or trainees. Agents addressing physical and/or mental health will be considered. This review will be conducted in accordance with JBI methodology for scoping reviews. The databases to be searched will include MEDLINE (PubMed), Embase (Elsevier), CINAHL with Full Text (EBSCO), Scopus (Elsevier), Web of Science (Clarivate), ACM Guide to Computing Literature (ACM Digital Library), and IEEE Xplore (IEEE). The extracted data will include study characteristics, basic characteristics of the conversational agent, and characteristics relating to the professional representation of the conversational agent. The extracted data will be presented in tabular format and summarized using frequency analysis. These results will be accompanied by a narrative summary.

Developing domain-specific conversational agents (CAs) has been challenged by the need for extensive domain-focused data. Recent advancements in Large Language Models (LLMs) make them a viable option as a knowledge backbone. LLMs behaviour can be enhanced through prompting, instructing them to perform downstream tasks in a zero-shot fashion (i.e. without training). To this end, we incorporated structural knowledge into prompts and used prompted LLMs to prototyping domain-specific CAs. We demonstrate a case study in a specific domain-textile circularity – TextileBot, we present the design, development, and evaluation of the TextileBot. Specially, we conducted an in-person user study (N = 30) with Free Chat and Information-Gathering tasks with TextileBots to gather insights from the interaction. We analyse the human–agent interactions, combining quantitative and qualitative methods. Our results suggest that participants engaged in multi-turn conversations, and their perceptions of the three variation agents and respective interactions varied demonstrating the effectiveness of our prompt-based LLM approach. We discuss the dynamics of these interactions and their implications for designing future voice-based CAs.

Nowadays Personalized Service Collaboration (PSC) becomes more feasible due to the increasing availability of mobile application and web service. The semantic user model comprised of Resource Description Framework (RDF) plays a significant role in PSC. In practice, preferences within (personal data/model) often change with different time, context and dimension, which are dynamic information and contribute to PSC. Therefore, personal data model which represents time, context and dimension is increasingly required to provide a more complete user model view and more value-added user information to all relevant applications and services for PSC. In this paper, a novel personal data model based on RDF is proposed, which takes into account how to represent implicated and varied preference that corresponding to different time, context and dimension. The presented model aims to solve the issues that store, organize and represent unified personal data formally in service collaboration environment which harbors different kinds of personal date from various mobile applications and web services. Finally, the comparison and analysis between proposed personal data model and conventional user model are presented. The experiment result on real data set demonstrate the feasibility and effectiveness of proposed personal data model.

The incompleteness and uncertainty about the state of the world and about the consequences of actions are unavoidable. If we want to predict the performance of multiuser computing systems, we have the uncertainty of what the users are going to do, and how that affects system performance. Intelligent interface agent development is one way to mitigate the uncertainty about user behaviors by predicting what users will do based on learned users' behaviors, preferences, and intentions. This work focuses on developing user models that can analyze and predict user behavior in multi-agent systems. We have developed a formal theory of user behavior prediction based on hidden Markov models. This work learns the user model through a time-series action analysis and abstraction by taking users' preferences and intentions into account in order to formally define user modeling.

An underlying problem of current interface agent research is the failure to adequately address effective and efficient knowledge representations and associated methodologies suitable for modeling the users' interactions with the system. These user models lack the representational complexity to manage the uncertainty and dynamics involved in predicting user intent and modeling user behavior. A utility theory-based approach is presented for effective user intent prediction by incorporating the ability to explicitly model users' goals, the uncertainty in the users' intent in pursuing these goals, and the dynamics of users' behavior. We present an interface agent architecture, CIaA, that incorporates our approach and discuss the integration of CIaA with three disparate domains — a probabilistic expert system shell, a natural language input database query system, and a virtual space plane —that are being used as test beds for our interface agent research.