Artificial Intelligence for HCI: A Modern Approach

This work open up a conversation about the opportunities that Science Fiction offers to Human-Computer Interaction (HCI) and Design research. In doing so, first, it briefly challenges the term Design Fiction, an emerging concept with increasing popularity in HCI and Design research. Design Fictions are either, manifest design propositions, or intangible, imagined future interfaces, speculative interactions and would-be environments. Design Fictions as useful as they are, however, are not necessarily something new nor innovative in themselves. Speculative design and evaluation methods all have been staple methods of Computer Science research since the establishment of HCI as a discrete field sometime in the 1980s. Second, the paper proceeds to propose that the domain of Science Fiction could not only be useful, but rightly considered, as a legitimate field of creative inquire in HCI and Design education. By reframing the notion of Science Fiction from an ‘anecdotal gimmick’ toward an ‘area of study’ a novel, wide-ranging space for design inspiration, ingenuity and creativity opens up for conceptual exploration. We believe that such an elevation of Science Fiction as a serious research topic has strong potential to inform the Computer Science, HCI and Design research application and education endeavours of the future.KeywordsComputer science research and educationDesign FictionSpeculative future studiesPopular culture in scienceScience Fiction

In recent years, gesture recognition and speech recognition, as important input methods in Human–Computer Interaction (HCI), have been widely used in the field of virtual reality. In particular, with the rapid development of deep learning, artificial intelligence, and other computer technologies, gesture recognition and speech recognition have achieved breakthrough research progress. The search platform used in this work is mainly the Google Academic and literature database Web of Science. According to the keywords related to HCI and deep learning, such as “intelligent HCI”, “speech recognition”, “gesture recognition”, and “natural language processing”, nearly 1000 studies were selected. Then, nearly 500 studies of research methods were selected and 100 studies were finally selected as the research content of this work after five years (2019–2022) of year screening. First, the current situation of the HCI intelligent system is analyzed, the realization of gesture interaction and voice interaction in HCI is summarized, and the advantages brought by deep learning are selected for research. Then, the core concepts of gesture interaction are introduced and the progress of gesture recognition and speech recognition interaction is analyzed. Furthermore, the representative applications of gesture recognition and speech recognition interaction are described. Finally, the current HCI in the direction of natural language processing is investigated. The results show that the combination of intelligent HCI and deep learning is deeply applied in gesture recognition, speech recognition, emotion recognition, and intelligent robot direction. A wide variety of recognition methods were proposed in related research fields and verified by experiments. Compared with interactive methods without deep learning, high recognition accuracy was achieved. In Human–Machine Interfaces (HMIs) with voice support, context plays an important role in improving user interfaces. Whether it is voice search, mobile communication, or children’s speech recognition, HCI combined with deep learning can maintain better robustness. The combination of convolutional neural networks and long short-term memory networks can greatly improve the accuracy and precision of action recognition. Therefore, in the future, the application field of HCI will involve more industries and greater prospects are expected.

Human-Computer Interaction (HCI) research has been focusing on properties such as usability, accessibility, inclusive design, user experience. As new security risks are emerging with the continuous increase of Internet interconnections and the development of the Internet of Things, more recently security, trust and resilience have also become important for the development of interactive systems. Since users have been identified as one of the major security weaknesses in today’s technologies, HCI becomes a fundamental pillar for designing more secure (but still usable) systems. However, interactive system properties might overlap and sometimes create conflicts in user-centered development processes. For example, security could reduce system usability by placing a burden on users when they have to deal with passwords. The HCI research has provided several tools and techniques that can support designers in making decisions, but there are no “cookbooks”. This workshop promotes sharing of experiences in managing and resolving conflicts of multiple interactive system properties within the context of a user-centered design process. We are concerned by theories, methods and approaches for dealing with interactive system properties, managing potential conflicts and trade-offs. This workshop is organized by the IFIP WG 13.2 on Methodology for User-Centered System Design and the IFIP WG 13.5 on Human Error, Resilience, Reliability and Safety in System Development.

Human-computer interaction for virtual-real fusion

We explore the use of cinematic “pre-visualization” (previs) techniques as a rapid ideation and design futuring method for human computer interaction (HCI) research. Previs approaches, which are widely used in animation and film production, use digital design tools to create medium-fidelity videos that capture richer interaction, motion, and context than sketches or static illustrations. When used as a design futuring method, previs can facilitate rapid, iterative discussions that reveal tensions, challenges, and opportunities for new research. We performed eight one-week design futuring sprints, in which individual HCI researchers collaborated with a lead designer to produce concept sketches, storyboards, and videos that examined future applications of their research. From these experiences, we identify recurring themes and challenges and present a One Week Futuring Workbook that other researchers can use to guide their own futuring sprints. We also highlight how variations of our approach could support other speculative design practices.

Background: Unprecedented growth in artificial intelligence (AI) is changing the way people interact with computers- the concept known as human-computer interaction (HCI), several AI approaches like machine learning or natural language processing have boosted the user experience (UX). Nevertheless, there are some ethical concerns and differences in users’ awareness of AI systems that make it difficult to consider the general advantages of AI in HCI. Objective: Therefore, this research seeks to explore the impact of enhanced AI techniques on the level of UX in HCI with a focus on the facet of user satisfaction, trust, and usefulness of AI-based personalization techniques. It also solves the ethical problem associated with the AI systems. Methods: A quantitative research method was used with the use of structured questionnaires with 250 participants. It also captured the user engagement with AI systems, as well as their satisfaction and trust levels as well as their concerns relating to AI ethics. Descriptive analysis was used to examine the collected data through statistical tests such as; the Shapiro-Wilk test for normality and Cronbach’s Alpha for reliability. Results: It is seen that most of the users are happy with the AI technologies integrated into their lives, with special emphasis on the aspect of personalization. However, the data was nonparametric (Nomography = 0 05) and so was the internal consistency (Cronbach alpha -0. 033 on the Likert scale items were needed for survey tools of higher specific focus. The results of the survey showed that ethical considerations running across the respondents as the most significant factor of discrepancy between satisfaction and trust in areas of tension. Conclusion: In the context of AI methodologies, most are perceived to have positive effects in enhancing the UX regarding individualized engagements. However, ethical issues should be met and addressed while determining the reliability of the tools used in ascertaining the impact of AI. There is a need for future studies to address these issues and improve the techniques used for evaluating AI’s diverse position in HCI.

Human Computer Interaction Research in Web Design and Evaluation presents research from academics and industry experts, covering various aspects of human computer interaction (HCI) Web design from theories to evaluation. This book highlights the use of methods from the HCI area in Web design, and how these methods can be used in a practical sense. Human Computer Interaction Research in Web Design and Evaluation is a comprehensive book on HCI and Web design that focuses on various areas of research including theories, analysis, design, and evaluation. This book not only features the human aspect of Web design, but also highlights the social and cultural issues in designing for a wider audience.

The interest about enhancing the interface usability of applications and entertainment platforms has increased in last years. The research in human-computer interaction on conversational agents, named also chatbots, and natural language dialogue systems equipped with audio-video interfaces has grown as well. One of the most pursued goals is to enhance the realness of interaction of such systems. For this reason they are provided with catchy interfaces using humanlike avatars capable to adapt their behavior according to the conversation content. This kind of agents can vocally interact with users by using Automatic Speech Recognition (ASR) and Text To Speech (TTS) systems; besides they can change their “emotions” according to the sentences entered by the user. In this framework, the visual aspect of interaction plays also a key role in human-computer interaction, leading to systems capable to perform speech synchronization with an animated face model. These kind of systems are called Talking Heads. Several implementations of talking heads are reported in literature. Facial movements are simulated by rational free form deformation in the 3D talking head developed in Kalra et al. (2006). A Cyberware scanner is used to acquire surface of a human face in Lee et al. (1995). Next the surface is converted to a triangle mesh thanks to image analysis techniques oriented to find reflectance local minima and maxima. In Waters et al. (1994) the DECface system is presented. In this work, the animation of a wireframe face model is synchronized with an audio stream provided by a TTS system. An input ASCII text is converted into a phonetic transcription and a speech synthesizer generates an audio stream. The audio server receives a query to determine the phoneme currently running and the shape of the mouth is computed by the trajectory of the main vertexes. In this way, the audio samples are synchronized with the graphics. A nonlinear function controls the translation of the polygonal vertices in such a way to simulate the mouth movements. Synchronization is achieved by calculating the deformation length of the mouth, based on the duration of an audio samples group. BEAT (Behavior Expression Animation Toolkit) an intelligent agent with human characteristics controlled by an input text is presented in Cassell et al. (2001). A talking head for the Web with a client-server architecture is described in Ostermann et al. (2000). The client application comprises the browser, the TTS engine, and the animation renderer. A 15

Voice assistants (VAs) have become an integral part of many individuals’ everyday lives. However, we have little understanding of the factors that influence perceptions about them. We synthesize literature from several domains such as computers, technology-human interaction, communications, and marketing and identify various theories used to understand VA adoption - Technology Acceptance Model (TAM), Unified Theory of Acceptance, and Use of Technology (UTAUT) and Uses and Gratification theory (U&G). Prior research in VA has majorly focused on the uses and benefits VA offer and subsequently the different perceptions consumers build towards the VA. The current research contributes to these streams by deepening the understanding of the disruptions that occur in VA-human interactions. Using advanced technology such as artificial intelligence, these VAs perform several tasks based on user commands. Despite that, they are prone to errors and breakdowns, and most users have encountered such disruptions at least once in their usage. To date, there is no research in marketing that studies the disruptions that hinder consumer goals occurring in human-VA interactions. To bridge this gap, we identify three types of disruption that lead to failures in human-VA interaction - VA-caused, dyad-caused, and consumer-caused based on an extensive literature review. We anticipate that these disruptions in VA, lead to a failed interactions or outcomes that are deviated from user expectations. Furthermore, based on past literature research in human computer interaction and marketing, we developed a typology of failures subjective to consumers and categorize these VA failures into three distinct categories: responsiveness error, expertise error, and social skill violation. The current research further expands understanding on how one specific type of failure-social skill violation, affects consumer behavioral intentions. Our empirical research based on the SEEK theory of anthropomorphism, tests the effect of social skill violation on consumer perceptions, such as uncanniness and anthropomorphism. Through a series of five studies, the current research demonstrates that failures have a negative influence on usage intentions. This effect is mediated by uncanniness perceptions. The current research also demonstrates that activating mind perception through agency and the ability to experience information about the VA, mitigates the dominant negative effect of failures on usage intentions. Furthermore, contrary to intuitive beliefs, we identify an indirect positive effect of social skill violation on usage intention mediated by anthropomorphism perceptions. The study tests the Effectance motivation of SEEK theory of anthropomorphism and suggests that failures activate effectance motivation, which in turn increases

The advent of artificial intelligence (AI) has revolutionized human-computer interaction, making it more natural and intuitive. This research paper presents the development and implementation of an AI Desktop Voice Assistant designed to enhance productivity and accessibility for users. The voice assistant leverages advanced speech recognition, natural language processing (NLP), and machine learning techniques to understand and execute user commands. Key functionalities include voice-activated application control, web searches, personalized reminders, and real-time information retrieval. Our system integrates with widely-used APIs and services, providing a seamless user experience across various tasks. The paper discusses the architecture of the voice assistant, detailing the integration of components such as the speech recognition engine, NLP models, and the dialogue management system. We explore the challenges encountered during development, including accurate speech recognition in noisy environments and handling ambiguous user commands. Solutions implemented to address these challenges are also presented. Furthermore, we conduct a usability study to evaluate the effectiveness and user satisfaction of the voice assistant. The results indicate a high level of user engagement and satisfaction, demonstrating the practical benefits of incorporating AI-driven voice interfaces into desktop environments. Our findings contribute to the ongoing research in human-computer interaction, suggesting pathways for future enhancements in AI voice assistants..

Human-computer interaction (HCI) research is concerned with the design of interfaces that allow easy and efficient use of computer systems. This report is the result of a workshop held to define the state of the art and to identify HCI research directions. The workshop was held on March 4 and 5, 1991 at George Washington University. The participants considered four areas of HCI research: theory and models; input/output devices; tools and architectures; and computer-supported cooperative work (CSCW). This report contains information about each of these areas.

The article is devoted to the current and promising direction of the implementation of the Industry 4.0 program based on the introduction of artificial intelligence (AI) approaches in the industrial production of the oil and gas industry. A review of the application of modern bioinspired AI approaches for solving problems of control and diagnostics of industrial equipment is carried out. The application of the approach of artificial immune systems (AIS) and various modified AIS algorithms is substantiated in the creation of innovative and competitive applications for solving various specific engineering problems and creating a modern highly efficient technology for automating production processes at oil refineries. The theoretical foundations for the creation of an intelligent innovative technology based on a unified artificial immune system (UAIS) for distributed systems for complex objects control using modern computer technology have been developed. Definitions of UAIS and immunological homeostasis are given. Unification is carried out with the aim of reducing the variety of modified AIS algorithms and their optimal use for solving the assigned tasks for the technological processes control. The classification and assessment of the effectiveness of the application of modified AIS algorithms is carried out. The block diagram of an intelligent control system based on the developed UAIS is presented.

Open source software (OSS) projects are often seen as participatory and egalitarian settings where people collaboratively develop software to serve their needs as well as the needs of others. In this paper, however, we argue that power and politics also characterize OSS development, and that this has serious implications for OSS usability. The existing Human-Computer Interaction (HCI) research on OSS usability has already shown that power and politics play a role; this study offers a theoretical treatment of the matter. A theoretical framework on power and empowerment is utilized in analyzing empirical data on OSS usability as well as the existing body of knowledge on the topic. With the help of this framework, HCI research can address the aspects of power and empowerment in OSS usability in a more systematic and comprehensive manner.

Background: Enumeration of CTCs in blood samples with the CELLSEARCH® system is a prognostic biomarker in metastatic breast, prostate and colorectal cancer (full intended use documents.cellsearchctc.com). Currently, the final CTC identification is performed by human reviewers through visual assessment, which is a time-consuming procedure that could be affected by subjective interpretations. Modern Artificial Intelligence (AI) methods can provide a solution by automating the process of CTC identification, thus eliminating subjectivity and producing faster and more reproducible results. We have recently presented an automated algorithm (for research use only) based on deep learning models for cell segmentation and classification which allows the automated identification of CTCs in CELLSEARCH® fluorescent images (Ansaloni et al., EACR 2022). The purpose of this study was to assess inter-rater variability among human reviewers and to compare the AI performance against them. Methods: Blood samples derived from metastatic cancer patients were processed with CELLTRACKS® AUTOPREP®. The AI algorithm was trained on fluorescent images of 7255 CTCs and 32876 non-CTC events (training set) acquired by CELLTRACKS ANALYZER II® (CTAII) from 90 breast, 122 prostate and 54 colorectal cancer samples. 10 human reviewers, qualified for CTC image analysis, performed blind labeling of a separate dataset independently (test set: 55 samples taken from 27 breast and 28 prostate cancer). We studied inter-rater agreement and variability using Fleiss Kappa and Coefficient of Variation (CV). The labels provided by 5 experienced reviewers (average experience ≅ 15 years) were used to generate the ground truth (GT) by majority voting. The CTC detection performance of the AI was ranked against the other 5 reviewers (average experience ≅ 5 years). Results: In the test dataset, 2990 out of 9554 events were identified as CTCs by the GT review on CTAII images. In the task of classifying each event as CTC or non-CTC, the inter-rater agreement among the 10 human reviewers measured by Fleiss Kappa was 0.85. These classification results provided the number of CTCs per patient counted by each reviewer. The inter-rater variability on the CTC enumeration among the 10 reviewers was evaluated for each sample (median CV = 17.9% and interquartile range = 17.6% , excluding 27 samples with mode = 0 CTC). Measured against the GT classification, the AI performance (accuracy = 94.7%; F1 = 91.6%) was at the top of the human reviewers’ ranking (accuracy = 92.4 - 93.6 - 93.8 - 94.0 - 95.8%; F1 = 86.8 - 89.2 - 89.2 - 89.5 - 93.1%). Conclusion: The AI algorithm reached top-ranking performance in CTC identification, surpassing 4 out of 5 expert reviewers. These results show the applicability of AI to cell classification to remove human subjectivity from the review process and to maximize standardization among different research centers. Citation Format: Luca Biasiolli, Pietro Ansaloni, Nicolò Gentili, Ramona Miserendino, Chiara Rossi, Giulio Signorini, Gianni Medoro. Deep learning for circulating tumor cell (CTC) identification with the CELLSEARCH system: achieving top human-level performance [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 3373.

Artificial intelligence refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions. In the present review we chose ten influential papers from the last five years and through Kintsugi, shed the light on recent evolution of artificial intelligence in anesthesiology. A comprehensive search in in Medline, Embase, Web of Science and Scopus databases was conducted. Each author searched the databases independently and created a list of six articles that influenced their clinical practice during this period, with a focus on their area of competence. During a subsequent step, each researcher presented his own list and most cited papers were selected to create the final collection of ten articles. In recent years purely methodological works with a cryptic technology (black-box) represented by the intact and static vessel, translated to a "modern artificial intelligence" in clinical practice and comprehensibility (glass-box). The purposes of this review are to explore the ten most cited papers about artificial intelligence in anesthesiology and to understand how and when it should be integrated in clinical practice.