Identifying interaction types and functionality for automated vehicle virtual assistants: An exploratory study using speech acts cluster analysis

BackgroundVirtual assistants can be used to deliver innovative health programs that provide appealing, personalized, and convenient health advice and support at scale and low cost. Design characteristics that influence the look and feel of the virtual assistant, such as visual appearance or language features, may significantly influence users’ experience and engagement with the assistant.ObjectiveThis scoping review aims to provide an overview of the experimental research examining how design characteristics of virtual health assistants affect user experience, summarize research findings of experimental research examining how design characteristics of virtual health assistants affect user experience, and provide recommendations for the design of virtual health assistants if sufficient evidence exists.MethodsWe searched 5 electronic databases (Web of Science, MEDLINE, Embase, PsycINFO, and ACM Digital Library) to identify the studies that used an experimental design to compare the effects of design characteristics between 2 or more versions of an interactive virtual health assistant on user experience among adults. Data were synthesized descriptively. Health domains, design characteristics, and outcomes were categorized, and descriptive statistics were used to summarize the body of research. Results for each study were categorized as positive, negative, or no effect, and a matrix of the design characteristics and outcome categories was constructed to summarize the findings.ResultsThe database searches identified 6879 articles after the removal of duplicates. We included 48 articles representing 45 unique studies in the review. The most common health domains were mental health and physical activity. Studies most commonly examined design characteristics in the categories of visual design or conversational style and relational behavior and assessed outcomes in the categories of personality, satisfaction, relationship, or use intention. Over half of the design characteristics were examined by only 1 study. Results suggest that empathy and relational behavior and self-disclosure are related to more positive user experience. Results also suggest that if a human-like avatar is used, realistic rendering and medical attire may potentially be related to more positive user experience; however, more research is needed to confirm this.ConclusionsThere is a growing body of scientific evidence examining the impact of virtual health assistants’ design characteristics on user experience. Taken together, data suggest that the look and feel of a virtual health assistant does affect user experience. Virtual health assistants that show empathy, display nonverbal relational behaviors, and disclose personal information about themselves achieve better user experience. At present, the evidence base is broad, and the studies are typically small in scale and highly heterogeneous. Further research, particularly using longitudinal research designs with repeated user interactions, is needed to inform the optimal design of virtual health assistants.

Investigating take-over, driving, non-driving related task (NDRT) performance, and trust of conditionally automated vehicles (AVs) in critical transitions on a test track. Most experimental results addressing driver take-over were obtained in simulators. The presented experiment aimed at validating relevant findings while uncovering potential effects of motion cues and real risk. Twenty-two participants responded to four critical transitions on a test track. Non-driving related task modality (reading on a handheld device vs. auditory) and take-over timing (cognitive load) were varied on two levels. We evaluated take-over and NDRT performance as well as gaze behavior. Further, trust and workload were assessed with scales and interviews. Reaction times were significantly faster than in simulator studies. Further, reaction times were only barely affected by varying visual, physical, or cognitive load. Post-take-over control was significantly degraded with the handheld device. Experiencing the system reduced participants' distrust, and distrusting participants monitored the system longer and more frequently. NDRTs on a handheld device resulted in more safety-critical situations. The results confirm that take-over performance is mainly influenced by visual-cognitive load, while physical load did not significantly affect responses. Future take-over request (TOR) studies may investigate situation awareness and post-take-over control rather than reaction times only. Trust and distrust can be considered as different dimensions in AV research. Conditionally AVs should offer dedicated interfaces for NDRTs to provide an alternative to using nomadic devices. These interfaces should be designed in a way to maintain drivers' situation awareness. This paper presents a test track experiment addressing conditionally automated driving systems. Twenty-two participants responded to critical TORs, where we varied NDRT modality and take-over timing. In addition, we assessed trust and workload with standardized scales and interviews.

Virtual personal assistants (VPAs) like Siri and Alexa have become common objects in households. Users frequently rely on these systems to search the internet or help retrieve information. As such, it is important to know how using these products affect cognitive processes like memory. Previous research suggests that visual speech perception influences auditory perception in human-human interactions. However, many of these VPAs are designed as a box or sphere that does not interact with the user visually. This lack of visual speech perception when interacting with a VPA could affect the human interaction with a system and their retention of information such as determining how many ounces are in a cup or how to greet someone in another language. This poses the question of whether the design of these VPAs is preventing the ability of users to retain the information they get from these systems. To test this, we designed an experiment that will explore interactions between user memory and either a traditional audio presentation (as is found with Siri or Alexa, for example) or one that allows for visual speech perception. Participants were asked to listen to an audio clip of a nonsensical story. In one condition, participants were asked to listen while looking at a blank screen (analogous to the lack of visual feedback inherent when working with current VPA designs). After a block of 25 audio clips, the participants took a test on the information heard. This process was repeated with an animated face with synchronized mouth movements instead of a black screen. Other participants will experience the same two presentations, but in reverse order as to counterbalance condition presentation. Data collection is currently underway. We predicted that VPA paired with synchronized lip movement would promote visual speech perception and thus help participants retain information. While we are still collecting data, the trend currently does not show a significant difference between audio and lip movement conditions. This could be an indication of differing abilities in lipreading.

This study investigates the impact of silent and alerted failures on driver performance across two levels of scenario criticality during automated vehicle transitions of control. Recent analyses of automated vehicle crashes show that many crashes occur after a transition of control or a silent automation failure. A substantial amount of research has been dedicated to investigating the impact of various factors on drivers' responses, but silent failures and their interactions with scenario criticality are understudied. A driving simulator study was conducted comparing scenario criticality, alert presence, and two driving scenarios. Bayesian regression models and Fisher's exact tests were used to investigate the impact of alert and scenario criticality on takeover performance. The results show that silent failures increase takeover times and the intensity of posttakeover maximum accelerations and decrease the posttakeover minimum time-to-collision. While the predicted average impact of silent failures on takeover time was practically low, the effects on minimum time-to-collision and maximum accelerations were safety-significant. The analysis of posttakeover control interaction effects shows that the effect of alert presence differs by the scenario criticality. Although the impact of the absence of an alert on takeover performance was less than that of scenario criticality, silent failures seem to play a substantial role-by leading to an unsafe maneuver-in critical automated vehicle takeovers. Understanding the implications of silent failure on driver's takeover performance can benefit the assessment of automated vehicles' safety and provide guidance for fail-safe system designs.

As user demand scales for intelligent personal assistants (IPAs) such as Apple’s Siri, Google’s Google Now, and Microsoft’s Cortana, we are approaching the computational limits of current datacenter (DC) architectures. It is an open question how future server architectures should evolve to enable this emerging class of applications, and the lack of an open-source IPA workload is an obstacle in addressing this question. In this article, we present the design of Sirius, an open end-to-end IPA Web-service application that accepts queries in the form of voice and images, and responds with natural language. We then use this workload to investigate the implications of four points in the design space of future accelerator-based server architectures spanning traditional CPUs, GPUs, manycore throughput co-processors, and FPGAs. To investigate future server designs for Sirius, we decompose Sirius into a suite of eight benchmarks (Sirius Suite) comprising the computationally intensive bottlenecks of Sirius. We port Sirius Suite to a spectrum of accelerator platforms and use the performance and power trade-offs across these platforms to perform a total cost of ownership (TCO) analysis of various server design points. In our study, we find that accelerators are critical for the future scalability of IPA services. Our results show that GPU- and FPGA-accelerated servers improve the query latency on average by 8.5× and 15×, respectively. For a given throughput, GPU- and FPGA-accelerated servers can reduce the TCO of DCs by 2.3× and 1.3×, respectively.

Trade-off between safety, mobility and stability in automated vehicle following control: An analytical method

Automated driving requires a large number of sensors. They are used to detect environmental influences and regulate vehicle guidance. Many of these sensors have to fulfil requirements regarding position and functional installation space. This creates a conflict of objectives between aesthetically appealing integration into the design and functional integration into trim parts or an aerodynamic concept. In addition, the perception of the sensors by vehicle users and other road users could favour the recognition of an automated vehicle (AV). Numerous studies showed the fundamental importance of a vehicle's exterior, especially with regard to the various semantic aspects of the resulting product effect. This is particularly apparent with regard to the communication between the driver or AV and other road users. Therefore, in this paper we consider the preconditioning of the user in relation to the degree of integration of sensor technology for automated driving. For this purpose, sensor configurations differing in shape and colour are applied to a concept car and examined in an eye-tracking study with regard to their influence on the perception of passengers and other road users. This will provide designers and engineers with insights into the design of sensor clusters in the exterior design of future vehicles.METHODSIn a first step, existing studies were systematically examined to determine the influence of the exterior design on the perception of the vehicle. In addition to the basic idea of appeal, the preconditioning by the design was examined in particular. It has been shown that the exterior design has an influence on the user's driving style, the spatial perception and the brand affiliation recognition. In more recent studies, the influence of so-called external HMIs (eHMI) on the perception of an automated vehicle has been increasingly investigated. Studies on the influence of more or less exposed sensors for automated driving on the perception of users or pedestrians are not known.In a second step, an eye-tracking study was therefore designed. In the study, the test persons were sequentially shown renderings in ¾ front and ¾ rear perspective of a concept car with different sensor configurations on the screen. The relevant sensor configurations are based on a previous study in which we conducted a position analysis and derived integration strategies for sensors. The stimulus patterns differ specifically in the sub-forms of structure (position), shape and colour. The test persons evaluated each of the stimulus patterns by means of a questionnaire with regard to their subjective impression in the use cases "crossing the street as a pedestrian" and "getting into the vehicle and being driven home". System trust, the perception of safety, recognisability and the judgement of appeal were assessed. Absolute dwell time, relative dwell time and fixation were recorded via eye tracking.RESULTSThe statistical evaluation of the results has shown that the sensors already have an influence on system trust and the perception of safety of the users and other road users in both use cases considered. In particular, for the configuration without visible sensors and additively integrated sensors, significant differences emerged as expected. As expected, there was a high degree of dispersion in the opinion towards liking the design of the users. Areas of interest were derived from the eye-tracking data via heat maps. The more accurate understanding of the perception of the degree of integration of sensors into the exterior design can support the work of designers by giving them the freedom to realise significantly more innovative designs in their design proposals compared to previous vehicles.OUTLOOKExtending the investigations to much larger vehicles, especially trucks or robotic vehicles according to SAE Level 5 without passengers could show whether the perception of sensors can already be used as a salient feature in exterior design. Furthermore, it should be investigated whether there are interactions with eHMI on AVs. In addition, system trust and safety perception could be investigated in an extended virtual reality test design with a variety of other relevant use cases such as: "oncoming AV at right-before-left intersection" to show influences of immersion. Also, the influence of time and the associated habituation factor certainly plays a role. For this reason, the effects of road users getting used to the sight with sensors should also be considered.ACKNOWLEDGMENTSThis research was supported by Federal Ministry for Economic Affairs and Climate Action in the national research project RUMBA.

Pedestrians' perceptions, fixations, and decisions towards automated vehicles with varied appearances.

The determinants of Portuguese preference for vehicle automation: A descriptive and explanatory study

Voice activated virtual assistants are growing rapidly in number, variety and visibility, driven by media coverage, corporate communications, and inclusion in a growing variety of devices. This trend can also be observed by how difficult it is becoming to find, among internet users, people who have not used or even heard of this new technology. Having said this, there is a visible shortage of academic research on this topic. Therefore, in the interest of creating a knowledge base around voice activated virtual assistants based on artificial intelligence, this multi-country exploratory research study was carried out. It concludes by providing information about the usage of four main voice activated virtual assistants (Siri, Google Assistant, Cortana and Alexa), comparisons between virtual assistant most frequently used and countries, and finally it establishes the base for further research activities.

While traffic signals, signs, and road markings provide explicit guidelines for those operating in and around the roadways, some decisions, such as determinations of “who will go first,” are made by implicit negotiations between road users. In such situations, pedestrians are today often dependent on cues in drivers’ behavior such as eye contact, postures, and gestures. With the introduction of more automated functions and the transfer of control from the driver to the vehicle, pedestrians cannot rely on such non-verbal cues anymore. To study how the interaction between pedestrians and automated vehicles (AVs) might look like in the future, and how this might be affected if AVs were to communicate their intent to pedestrians, we designed an external vehicle interface called automated vehicle interaction principle (AVIP) that communicates vehicles’ mode and intent to pedestrians. The interaction was explored in two experiments using a Wizard of Oz approach to simulate automated driving. The first experiment was carried out at a zebra crossing and involved nine pedestrians. While it focused mainly on assessing the usability of the interface, it also revealed initial indications related to pedestrians’ emotions and perceived safety when encountering an AV with/without the interface. The second experiment was carried out in a parking lot and involved 24 pedestrians, which enabled a more detailed assessment of pedestrians’ perceived safety when encountering an AV, both with and without the interface. For comparison purposes, these pedestrians also encountered a conventional vehicle. After a short training course, the interface was deemed easy for the pedestrians to interpret. The pedestrians stated that they felt significantly less safe when they encountered the AV without the interface, compared to the conventional vehicle and the AV with the interface. This suggests that the interface could contribute to a positive experience and improved perceived safety in pedestrian encounters with AVs – something that might be important for general acceptance of AVs. As such, this topic should be further investigated in future studies involving a larger sample and more dynamic conditions.

As AI becomes more advanced in natural language processing, the research on AI's communication in teaming environments is getting more attention in CSCW/HCI. Even though AI's communication serves as an essential part of facilitating team coordination and shaping team outcomes, the impact of AI's communication modality on human-AI teamwork is still understudied. Using a mixed-design experiment and follow-up group interviews with 100 participants, we investigate the impact of AI's communication modality, one of AI's most essential communication characteristics, on team coordination and team outcomes in two different human-AI team compositions, human-human-AI teams, and human-AI-AI teams. Our findings highlight the trade-offs between AI's verbal communication and visual communication, which inspire two design recommendations on how to apply AI's verbal and visual communication to support human-AI coordination effectively. Our study generates an initial understanding of the role that AI's communication will have in ensuring team effectiveness in human-AI teams in the CSCW/HCI field.

Assessment of the cooperation between driver and vehicle automation: A framework

Introduction of human-centric AI assistant to aid radiologists for multimodal breast image classification

Many individuals with disabilities use virtual home assistants (VHAs) to communicate and execute daily tasks (Pradhan, Mehta & Findlater 2018). Because VHAs are controlled by verbal communication, some groups of users, including people with speech impairments, face barriers to effective interaction with VHAs. My research addresses the problems that individuals with speech impairments, specifically those with dysarthria, experience in interacting with these devices. I propose an alternate technique for interaction using nonverbal voice cues, such as humming, hissing and vowel sounds. The technique requires from the user only the ability to vocalise.