Learn2Smile: Learning non-verbal interaction through observation

The lack of visibility in text-based chat can hinder communication, especially when nonverbal cues are instrumental to the production and understanding of messages. However, communicating rich nonverbal cues such as facial expressions may be technologically more costly (e.g., demand of bandwidth for video streaming) and socially less desirable (e.g., disclosing other personal and context information through video). We consider how to balance the tension by supporting people to convey facial expressions without compromising the benefits of invisibility in text communication. We present KinChat, an enhanced text chat tool that integrates motion sensing and 2D graphical visualization as a technique to convey information of key facial features during text conversations. We conducted two studies to examine how KinChat influences the de-identification and awareness of facial cues in comparison to other techniques using raw and blurring-processed videos, as well as its impact on real-time text chat. We show that feature-based visualization of facial expression can preserve both awareness of facial cues and non-identifiability at the same time, leading to better understanding and reduced anxiety.

Coronavirus Disease 2019 (COVID-19), Videoconferencing, and Gender

Modern neuroimaging provides a common platform for neuroscience and related disciplines to explore the human brain, mind, and behavior. We base our review on the social shaping of the human mind and discuss various aspects of brain function related to social interaction. Despite private mental contents, people can share their understanding of the world using, beyond verbal communication, nonverbal cues such as gestures, facial expressions, and postures. The understanding of nonverbal messages is supported by the brain's mirroring systems that are shaped by individual experience. Within the organism-environment system, tight links exist between action and perception, both within an individual and between several individuals. Therefore, any comprehensive brain imaging study of the neuronal basis of social cognition requires appreciation of the situated and embodied nature of human cognition, motivating simultaneous monitoring of brain and bodily functions within a socially relevant environment. Because single-person studies alone cannot unravel the dynamic aspects of interpersonal interactions, it seems both necessary and beneficial to move towards "two-person neuroscience"; technological shortcomings and a limited conceptual framework have so far hampered such a leap. We conclude by discussing some major disorders of social interaction.

We present a demonstration of a Robot Receptionist: a situated interactive robot that can coordinate turn-taking and handle multi-party engagement and dialogue in dynamic environments, where users might enter or leave the scene at any time. We use a Furhat robot, which is highly expressive and can use verbal communication as well as non-verbal cues, such as facial expressions. The system demonstrated and described here is composed of several modules, including scene analysis, engagement policies, and a dialogue manager.

For an efficient and smooth human-robot interaction, communication protocols such as verbal and non-verbal communication, emotions, and personality plays an important role. Human-Robot-Interaction is an emerging field and robots are now a part of daily life where it can grasp both verbal and non-verbal cues. Personality prediction is an important research area in Human-Robot Interaction (HRI). Several important question in personality prediction includes: which personality traits will be important and which specific phycology model and robot do researchers use? Secondly, how emotions, facial expression, paralanguage, and bodily movements are related to personality traits? And finally, how will we acquire data to train a robot and what kind of questionnaires can be used? With the support of prior research studies and experiments, this paper will contribute towards developing the ground basis for personality prediction using a robot.

Affective support can play a central role in adaptive learning environments. Although virtual human tutors hold significant promise for providing affective support, a key open question is how a tutor's facial expressions can influence learners' performance. In this paper, we report on a study to examine the influence of a human tutor agent's facial expressions on learners' performance and emotions during learning. Results from the study suggest that learners' performance is significantly better when a human tutor agent facially expresses emotions that are congruent with the content relevancy. Results also suggest that learners facially express significantly more confusion when the human tutor agent provides incongruent facial expressions. These results can inform the design of virtual humans as pedagogical agents can inform the design of virtual humans as pedagogical agents and designing intelligent learner-agent interactions.

This paper presents an approach for generating photorealistic video sequences of dynamically varying facial expressions in human-agent interactions. To this end, we study human-human interactions to model the relationship and influence of one individual's facial expressions in the reaction of the other. We introduce a two level optimization of generative adversarial models, wherein the first stage generates a dynamically varying sequence of the agent's face sketch conditioned on facial expression features derived from the interacting human partner. This serves as an intermediate representation, which is used to condition a second stage generative model to synthesize high-quality video of the agent face. Our approach uses a novel L1 regularization term computed from layer features of the discriminator, which are integrated with the generator objective in the GAN model. Session constraints are also imposed on video frame generation to ensure appearance consistency between consecutive frames. We demonstrated that our model is effective at generating visually compelling facial expressions. Moreover, we quantitatively showed that agent facial expressions in the generated video clips reflect valid emotional reactions to behavior of the human partner.

Past and present researches in deception detection make use of questioning to ascertain the truth and deceit in the response of the interviewee. In such questioning method, either the verbal or nonverbal cues are closely monitored and analysed to arrive at a decision. Since no single verbal or nonverbal cue is able to reliably detect deception the research proposes to use both the verbal and nonverbal cues to detect deception. Therefore, this research aims to develop a Support Vector Machine and Decision Tree Model to classify the extracted verbal, nonverbal and VerbNon features as deceptive or truthful. The verbal cues capture the speech of the suspect while the nonverbal cues capture the facial expressions of the suspect. The Praat (a tool for speech analysis) was used in extracting all the verbal cues while the nonverbal features were extracted using the Active Shape Model (ASM). The work was implemented in 2015a MatLab. The analysis of the result shows that Decision Tree performs better than SVM in the classification with a percentage score of 93.5% for Nonverbal cues as against that of SVM having percentage score of 91.9%. For verbal and VerbNon cues, Decision Tree recorded 89.9% and 97.6% accuracy while SVM recorded 89.2% and 97.1% accuracy.

A large majority of Malaysia's 34.9 million online users are active online communication participants, making offline communication progressively obsolete. This high penetration figure allows online communication apps to be unlocked countless times a day, as thousands of text messages are sent regularly by online users. Nevertheless, unlike offline communication, online communication, like instant messaging (IM), does not normally transmit non-verbal cues such as facial expressions, touches, or voices as they are primarily text-based. As a result, it takes more time and effort for users to understand one another in order to make up for the lack of non-verbal cues. Our study focused on analysing material and non-material want requests used by adult Malay speakers of English in IM contexts related to work. We categorised these requests into directness, conventional indirectness, and non-conventional indirectness, employing a modified version of Hassall's (1999) analytical framework for request classifications. The results disclosed that participants preferred directness more than indirectness when making requests. The direct strategy within the imperative sub-strategy was favoured the most when non-material want requests (such as asking other participants to join online meetings) were performed. Nevertheless, when material want requests (like wanting to buy food containers) were carried out, the direct strategy under the want statement sub-strategy emerged as the most preferred method. Our research revealed that in IM, where non-verbal cues were insufficient, directness was predominantly adopted to convey certainty, clarity, and efficacy, implying directness as non-imposed. Hence, it is suggested to make work-related IM requests directly, as it allows speakers to present requests clearly, which subsequently creates quality relationships and maintains productivity. Keywords: Adult Malay speakers of English; instant messaging (IM); request Head Act; request strategies; work- related group communication DOI: http://doi.org/10.17576/3L-2026-3201-20

People can identify emotion from facial expressions easily, but it is more difficult to do it with a computer. It is now feasible to identify feelings from images because of recent advances in computational intelligence. Emotional responses are those mental states of thoughts that develop without conscious effort and are naturally associated with facial muscles, resulting in different facial expressions such as happy, sad, angry, contempt, fear, surprise etc. Emotions play an important role in nonverbal cues that represent a person’s interior thoughts. Intimate robots are expanding in every domain, whether it is completing requirements of elderly people, addressing psychiatric patients, child rehabilitation, or even childcare, as the human–robot interface is grabbing on every day with the increased demand for automation in every industry. We evaluate and test machine learning algorithms on the FER 2013 data set to recognize human emotion from facial expressions, with some of them achieving the highest accuracy and the others failing to detect emotions. Many researchers have used various machine learning methods to identify human emotions during the last few years. In this research article, we analyze eight frequently used machine learning techniques on the FER 2013 dataset to determine which method performs best at categorizing human facial expression. After analyzing the results, it is found that the accuracy of some of the algorithms is quite satisfactory, with 37% for Logistic Regression, 33% for K-neighbors classifier, 100% for Decision Tree Classifier, 78% for Random Forests, 57% for Ada-Boost, 100% for Gaussian NB, 33% for LDA (Linear Discriminant Analysis), and 99% for QDA (Quadratic Discriminant Analysis). Furthermore, the experimental results show that the Decision Tree and Gaussian NB Classifier can correctly identify all of the emotions in the FER 2013 dataset with 100% classification accuracy, while Quadratic Discriminant Analysis can do so with 99% accuracy.KeywordsFacial expressionEmotionMachine learningKNNQDADecision treeGaussian NBRandom Forest

Background: Effective communication is essential in nursing care, especially for patients with cognitive impairments. While verbal communication is commonly used, non-verbal communication strategies such as gestures, facial expressions, and body language can also play a crucial role in ensuring patient understanding and comfort. However, the effectiveness of these two communication approaches remains underexplored in nursing research. Objective: This study aims to compare the effectiveness of verbal and non-verbal communication in nursing care for patients with cognitive impairments, evaluating their impact on patient understanding, comfort, and compliance. Methods: A randomized controlled trial was conducted at a hospital in Ambon, Indonesia, from April to December 2024. A total of 300 patients with cognitive impairments were recruited, with 150 patients in each group (verbal communication group and non-verbal communication group). The effectiveness of each communication method was assessed based on patient comprehension, comfort level, and compliance with nursing instructions, using pre- and post-intervention surveys and observational assessments. Results: The results demonstrated that both communication methods had a significant positive effect on patient understanding and comfort. However, the non-verbal communication group showed a slightly higher improvement in patient comfort, while the verbal communication group had a more substantial impact on comprehension. Conclusion: Both verbal and non-verbal communication techniques are essential in nursing care for patients with cognitive impairments. Understanding the strengths of each method and incorporating them into daily practice can enhance the quality of care and improve patient outcomes

A method for teaching about verbal and nonverbal communication is described. The Interpersonal Perception Task (IPT) consists of a videotape of 30 brief scenes. Viewers answer interpretive questions by decoding the verbal and nonverbal cues in the scenes. Information is presented in all communication channels; several categories of interaction are represented; and for each scene, there is an objective criterion of accurate judgment. Instructional techniques using the IPT highlight the subtlety and complexity of communication cues, teach about specific cues to accuracy, demonstrate the relative importance of communication channels, and help students understand the process of interpretation.

Political oratory is a diverse genre which effectively juxtaposes speech and non-verbal cues. Politicians from diverse backgrounds and cultures often incorporate non-verbal elements, such as hand gestures, into their speeches. These hand gestures possess high utility and universality; thus, researchers believe that hand gestures and speech work together as a unified system for production and comprehension. This research explores the complex relationship between non-verbal and verbal communication within the dynamic realm of political public speech, drawing on McNeill’s (354) Gesture theory. McNeill’s theory classifies gestures into four categories: iconic, metaphoric, beat, and deictic. Employing a qualitative approach, the study provides an in-depth analysis of Imran Khan's gestures and relationship with spoken words. Iconic and metaphoric gestures were found to reinforce spoken content, while deictic and beat gestures directed attention to specific subjects and enhanced overall communication of meaning. The study found that Imran Khan effectively employed all four types of hand gestures and demonstrated a clear correlation between these gestures and his spoken words, thereby revealing his underlying thought processes. This study confirms the intricate correlation between gestures and language, emphasising their importance in influential public speaking. This resource provides unique perspectives on effective communication tactics, emphasising the significance of non-verbal clues in global conversations.

People communicate using verbal and non-verbal cues, including gaze cues. Gaze allocation can be influenced by social factors; however, most research on gaze cueing has not considered these factors. The presence of social roles was manipulated in a natural, everyday collaborative task while eye movements were measured. In pairs, participants worked together to make a cake. Half of the pairs were given roles ("Chef" or "Gatherer") and the other half were not. Across all participants we found, contrary to the results of static-image experiments, that participants spent very little time looking at each other, challenging the generalisability of the conclusions from lab-based paradigms. However, participants were more likely than not to look at their partner when receiving an instruction, highlighting the typical coordination of gaze cues and verbal communication in natural interactions. The mean duration of instances in which the partners looked at each other (partner gaze) was longer in the roles condition, and these participants were quicker to align their gaze with their partners (shared gaze). In addition, we found some indication that when hearing spoken instructions, listeners in the roles condition looked at the speaker more than listeners in the no roles condition. We conclude that social context can affect our gaze behaviour during a social interaction.

Autonomous systems’ potential to instruct the public during real-life emergencies to foster instantaneous trust and compliance and their impact on rescue operations remain largely unexplored. To determine the requirements for designing technologies capable of delivering instructions in high-risk situations, we needed to understand the key communication elements for establishing immediate trust dynamics, ultimately fostering compliance and contributing to effective life-saving efforts. This paper adopts a participatory approach to curate perspectives from emergency rescue professionals in the UK, gathered through a survey, whose responses were analysed to identify the themes in the dataset and ultimately to elicit verbal and nonverbal elements and message delivery techniques to address the challenges to compliance in interpersonal communication during emergencies. Participants indicated that the adoption of autonomous systems for communication could positively impact rescue operations. They highlighted that verbal communications need to be concise and informative, while nonverbal cues must effectively reinforce verbal messages under distressful conditions. However, challenges such as accountability, adaptability, reliability, and affordability are still prevalent. We formalise a novel communication model designed to engender instantaneous trust between the rescuer and the rescued. We find that verbal elements in the model must increase the situational awareness of the rescued and sufficiently inform them of the context. In contrast, the nonverbal elements should foster credibility, consistency, reliability and positivity between the communicating parties. Based on the professionals’ responses, we further advance recommendations for the use of autonomous systems in emergency rescue scenarios in terms of increasing accountability and accessibility.