Virtual Personal Assistant Design Effects on Memory Encoding

Introduction: To the best of our knowledge, there is a lack of reliable, validated, and standardized (Dutch) measuring instruments to document visual speech perception in a structured way. This study aimed to: (1) evaluate the effects of age, gender, and the used word list on visual speech perception examined by a first version of the Dutch Test for (Audio-)Visual Speech Perception on word level (TAUVIS-words) and (2) assess the internal reliability of the TAUVIS-words. Methods: Thirty-nine normal-hearing adults divided into the following 3 age categories were included: (1) younger adults, age 18–39 years; (2) middle-aged adults, age 40–59 years; and (3) older adults, age >60 years. The TAUVIS-words consist of 4 word lists, i.e., 2 monosyllabic word lists (MS 1 and MS 2) and 2 polysyllabic word lists (PS 1 and PS 2). A first exploration of the effects of age, gender, and test stimuli (i.e., the used word list) on visual speech perception was conducted using the TAUVIS-words. A mixed-design analysis of variance (ANOVA) was conducted to analyze the results statistically. Lastly, the internal reliability of the TAUVIS-words was assessed by calculating the Chronbach α. Results: The results revealed a significant effect of the used list. More specifically, the score for MS 1 was significantly better compared to that for PS 2, and the score for PS 1 was significantly better compared to that for PS 2. Furthermore, a significant main effect of gender was found. Women scored significantly better compared to men. The effect of age was not significant. The TAUVIS-word lists were found to have good internal reliability. Conclusion: This study was a first exploration of the effects of age, gender, and test stimuli on visual speech perception using the TAUVIS-words. Further research is necessary to optimize and validate the TAUVIS-words, making use of a larger study sample.

Background: Pathological and physiological disabilities of human auditory receptors reduce speech intelligibility under hearing loss (HL) at high frequencies. Objectives: The objective of the study is to investigate the speech-intelligibility-enhancement (SIE) in elderly with high-frequency HL (HFHL) through visual-speech-perception (VSP). The hypothesis is that the VSP of spondee words (two-syllable words with equal stress on both syllables) is properly recognized and reproduced with HFHL. Also; to decide the statistical relevance of the precise significant difference in speech discrimination (SD) with VSP on the subjected ear. Methods: Observational and descriptive studies of SIE with VSP on 12 elderly listeners (24 clear ears; without wax impaction) with progressive and bilateral HFHL were examined. The entire experimental records were checked for distribution with normal (Gaussian) using the Shapiro–Wilk's and paired Student's t-test (parametric test) had the 5% (0.05) significance level (α). Results: We analyzed, improved, and better performance in SIE with HFHL through VSP. The statistical P (probability) measure was calculated nothing, i.e., 0 (P < 0.05). Therefore, investigators strongly and carefully discarded the null hypothesis consideration. There was some significant value of statistical variation also exists with VSP. Moreover, with the help of the confidence-interval analysis, the same conclusion was achieved. Conclusions: In this study, it is concluded that the elderly-individuals of this experiment, whose mother tongue and official language is Hindi and English, respectively, they demonstrate the SIE with HFHL through VSP. This investigation also helps to improve the hearing sensitivity to some extent with VSP.

A representation of abstract linguistic categories in the visual system underlies successful lipreading

Cognitive interaction with virtual assistants: From philosophical foundations to illustrative examples in aeronautics

The Internet of Things promises to make relationships with technology more personal than ever. Voice-controlled virtual or artificial intelligence assistants such as Amazon’s Alexa or Google Assistant execute the commands of their users, providing information, entertainment, utility and convenience while enabling consumers to bypass the advertising they would typically see on a screen. This ‘screen-less’ communication presents significant challenges for brands used to ‘pushing’ messages to audiences in exchange for the content they seek in hopes of creating preference. It also raises ethical questions about data collection, usage and privacy. Little is known about the role marketing will play in the increasingly connected, voice-controlled home. This case study will explore critical cases to describe the implications, applications and opportunities for voice-controlled personal assistants in marketing and advertising in the USA.

An interactive virtual assistant can be very useful for older people because it is cost-effective and can easily assist them. It provides ways to give them social support and manage their loneliness by giving them activities, like playing games, solving puzzles, and listening to music. The importance of progress in virtual assistants goes beyond enhancing the features of virtual assistants. Such analysis can be important in finding patterns of trust and attitudes in human interactions, leading to improved adoption of virtual assistants. One of the specific areas where this can be useful is increasing the social acceptance of these assistants among the elderly population. In this study, we surveyed to gauge the popularity of virtual assistants in the elder population, which consisted of a total of 20 questions for which 104 responses were received. Seven hypotheses were formulated for the various Technology Acceptance Model (TAM) parameters and were supported by statistical analysis. The findings highlight that the virtual assistants not only show promising potential for improving the quality of life of older adults but also explain the role of trust, perceived reliability, and emotional comfort in shaping their long-term acceptance.

Onboard virtual assistants with the ability to converse with users are gaining favour in supporting effective human-machine interaction to meet safe standards of operation in automated vehicles (AVs). Previous studies have highlighted the need to communicate situation information to effectively support the transfer of control and responsibility of the driving task. This study explores ‘interaction types’ used for this complex human-machine transaction, by analysing how situation information is conveyed and reciprocated during a transfer of control scenario. Two human drivers alternated control in a bespoke, dual controlled driving simulator with the transfer of control being entirely reliant on verbal communication. Handover dialogues were coded based on speech-act classifications, and a cluster analysis was conducted. Four interaction types were identified for both virtual assistants (i.e., agent handing over control) - Supervisor, Information Desk, Interrogator and Converser, and drivers (i.e., agent taking control) - Coordinator, Perceiver, Inquirer and Silent Receiver. Each interaction type provides a framework of characteristics that can be used to define driver requirements and implemented in the design of future virtual assistants to support the driver in maintaining and rebuilding timely situation awareness, whilst ensuring a positive user experience. This study also provides additional insight into the role of dialogue turns and takeover time and provides recommendations for future virtual assistant designs in AVs.

It has been proposed that the common metric between audio and visual speech perception could be just kinematics (say dynamics), a somewhat rather old gibsonian proposal illustrated until recently by moving point-lights displays. Results reported here address the same question in the frame of another paradigm than moving dots. Full images of the face were presented in moving and static conditions along a naturally silent vowel-to-vowel rounding gesture (our “silent pause lipreading paradigm”). In order to compare the benefit of motion vs. shape, the change in angle paradigm for face perception was used for feature/gesture identification. The question is then: Does viewer-centred optimisation of the projection of the rounding gesture override motion benefit? Such a saturation effect found for profiles supports a shape-from-shading processing for front views, enhanced by the shape-from-motion system, such systems being recruited in case of undersampling (point-lights) or for out of shape projections. These computational approaches avoid calling for a direct perception account, i.e. motion-from-motion in all cases. Moreover they leave open the alternative for representational formats: motion or shape from a non optimal, or even non recognisable, shape put in motion. So shape remains a possible format for static as well as for dynamic stimuli, at least for a set of category features of speech sounds, in our case the main visible vowel gesture, rounding.KeywordsAnticipationface angle viewrounding feature/gestureshape-frommotionshape-from-shading

Common-coding theory posits that (1) perceiving an action activates the same representations of motor plans that are activated by actually performing that action, and (2) because of individual differences in the ways that actions are performed, observing recordings of one's own previous behavior activates motor plans to an even greater degree than does observing someone else's behavior. We hypothesized that if observing oneself activates motor plans to a greater degree than does observing others, and if these activated plans contribute to perception, then people should be able to lipread silent video clips of their own previous utterances more accurately than they can lipread video clips of other talkers. As predicted, two groups of participants were able to lipread video clips of themselves, recorded more than two weeks earlier, significantly more accurately than video clips of others. These results suggest that visual input activates speech motor activity that links to word representations in the mental lexicon.

A single pool of untrained subjects was tested for interactions across two bimodal perception conditions: audio-tactile, in which subjects heard and felt speech, and visual-tactile, in which subjects saw and felt speech. Identifications of English obstruent consonants were compared in bimodal and no-tactile baseline conditions. Results indicate that tactile information enhances speech perception by about 10 percent, regardless of which other mode (auditory or visual) is active. However, within-subject analysis indicates that individual subjects who benefit more from tactile information in one cross-modal condition tend to benefit less from tactile information in the other.

Cognitive Processing in Bilinguals

High-density electrocorticography reveals sensorimotor cortex engagement in two distinct sites with different roles during audiovisual, audio, and visual speech perception.

Although many audio-visual speech experiments have focused on situations where the presence of an incongruent visual speech signal influences the perceived utterance heard by an observer, there are also documented examples of a related effect in which the presence of an incongruent audio speech signal influences the perceived utterance seen by an observer. This study examined the effects that different distracting audio signals had on performance in a color and number keyword speechreading task. When the distracting sound was noise, time-reversed speech, or continuous speech, it had no effect on speechreading. However, when the distracting audio signal consisted of speech that started at the same time as the visual stimulus, speechreading performance was substantially degraded. This degradation did not depend on the semantic similarity between the target and masker speech, but it was substantially reduced when the onset of the audio speech was shifted relative to that of the visual stimulus. Overall, these results suggest that visual speech perception is impaired by the presence of a simultaneous mismatched audio speech signal, but that other types of audio distracters have little effect on speechreading performance.

Research has shown that visual speech perception can assist accuracy in identification of spoken words. However, little is known about the dynamics of the processing mechanisms involved in audiovisual integration. In particular, architecture and capacity, measured using response time methodologies, have not been investigated. An issue related to architecture concerns whether the auditory and visual sources of the speech signal are integrated “early” or “late.” We propose that “early” integration most naturally corresponds to coactive processing whereas “late” integration corresponds to separate decisions parallel processing. We implemented the double factorial paradigm in two studies. First, we carried out a pilot study using a two-alternative forced-choice discrimination task to assess architecture, decision rule, and provide a preliminary assessment of capacity (integration efficiency). Next, Experiment 1 was designed to specifically assess audiovisual integration efficiency in an ecologically valid way by including lower auditory S/N ratios and a larger response set size. Results from the pilot study support a separate decisions parallel, late integration model. Results from both studies showed that capacity was severely limited for high auditory signal-to-noise ratios. However, Experiment 1 demonstrated that capacity improved as the auditory signal became more degraded. This evidence strongly suggests that integration efficiency is vitally affected by the S/N ratio.

AI assistant is my new best friend! Role of emotional disclosure, performance expectations and intention to reuse