Social Conformity in Immersive Virtual Environments: The Impact of Agents' Gaze Behavior.

Eye contact is presumed to be one of the most important non-verbal cues in human communication. It supports mutual understanding and builds the foundation for social interaction. In recent years, a variety of systems that support eye contact have been developed. However, research hardly focuses on investigating the impact of eye contact on social presence. In a study with 32 participants, we investigated the role of eye contact and gaze behavior with respect to social presence. Our results indicate that not only a system‘s capability to enable eye contact but also a user‘s consciously perceived eye contact are important to experience that the communication partner is ‘there’, i.e., social presence. Considering social presence as a source for social capital, i.e., valuable relationships that are characterized by trust and reciprocity, we discuss in what way social presence can serve as a contributing factor in video-mediated communication.

Eye contact and eye gaze behaviors within communication contexts are considered vital to clear linguistic transmission. Although much research has been done on eye contact and gaze behaviors, surprisingly little literature exists in this area concerning deaf populations. Due to linguistic necessity, certain factors influencing gaze behaviors during communication will differ between hearing and deaf groups. The present article explains a sequential model of eye gaze and eye contact behaviors (Patterson, 1982) developed and researched among hearing populations and explores possible similarities and differences in these behaviors in deaf people. It is found that characterizations of eye contact and eye gaze behaviors applied to hearing populations are not completely applicable to deaf groups. Empirical research is needed to validate the ideas presented here, and future research directions are indicated.

For efficient collaboration between participants, eye gaze is seen as being critical for interaction. Teleconferencing systems such as the AcessGrid allow users to meet across geographically disparate rooms but as of now there seems no substitute for face to face meetings. This paper gives an overview of some preliminary work that looks towards integrating eye gaze into an immersive Collaborative Virtual Environment and assessing the impact that this would have on interaction between the users of such a system. An experiment was conducted to assess the difference between users abilities to judge what objects an avatar is looking at with only head gaze being viewed and also with eye and head gaze data being displayed. The results from the experiment show that eye gaze is of vital importance to the subjects correctly identifying what a person is looking at in an immersive virtual environment. This is followed by a description of how the eye tracking system has been integrated into an immersive collaborative virtual environment and some preliminary results from the use of such a system.

Event Abstract Back to Event Can state-of-the-art saliency systems model infant gazing behavior in tutoring situations? Vikram Tadmeri Narayan1*, Katrin S. Lohan1, Marko Tscherepanow1, Katharina Rohlfing1 and Britta Wrede1 1 Bielefeld University, Applied Informatics Group, CoR-Lab, Germany I. I NTRODUCTION The behavior for a humanoid robot is often modeled in accordance with human behavior. Current research suggests that analyzing infant behavior as a basis for designing the robot behavior can guide us to a natural robot interface. Based on this idea many researchers support saliency systems as a bottom-up inspired way to simulate infant-like gazing behavior. In the field of saliency systems many different approaches have proposed and quantified in terms of speed, quality and other technical issues. But so far, no one compared and quantified them in terms of natural infant tutor interaction. The question we would like to address in this paper is: Can state-of-the-art saliency systems model infant gazing behavior in tutoring situations? By addressing these issues we want to take a step towards an autonomous robot system, which could be used more natural interaction experiments in future. II. M ETHODS FOR A NALYZING We compared 7 different saliency systems with the gazing behavior of infants between the age of 8-11 months. The infant’s gazing behavior and the interaction structure of the tutoring situation was manually annotated. Based on this annotation we selected images as input for the saliency systems. The interaction data used in the course of this research is described in the following subsections. The employed saliency systems are briefly described in subsection II-B. A. Parent-child interaction Out of the well-known Bielefeld-motionese corpus [8], we focus on the youngest participants comprising 12 families of 8 to 11 months old children to analyze their gazing behavior, as the main feedback and controlling capabilities of these infants is based on the gazing behavior [13]. We selected the stacking cups task, because it was often analyzed [7],[12] and the interaction is very detailed analysed. We selected several pictures based on an action segmentation. The pictures represent the beginning and ending point of the stacking cups task. We have chosen this points because at these points the scene is changing. Setting: Parents were instructed to demonstrate a stacking cups task to an interaction partner. The first interaction partner was their infant, and another adult constituted the second. Fig. 1 illustrates the top-view of the experimental setup, and shows sample image frames of cameras which were set behind the parent and the interaction partner and focused on each of them. The stacking cups task was to sequentially pick up the green (a1), the yellow (a2), and the red (a3) cup and put them into the blue one on the white tray. B. Saliency Systems Frequency tuned saliency model [1] is perhaps the simplest of all the existing saliency systems. The absolute difference of each pixel to the image mean is accounted as its saliency value. They recommend to decompose a given input image into L*, a*, b* color space and perform the aforementioned operation on each of the color plane and fuse the results to compute the final saliency map. Graph-based visual saliency models [4] envisages the input image as a complete graph with each pixel as its nodes. A Weber’s [3] law based dissimilarity measure is employed to compute the dissimilarity between any two given pixels. A normalizing function is further employed to compute the final saliency map. Experiments carried out in [4], [10] have shown that the method has high performance in correlating with human eye gaze and also for object segmentation. The Multi resolution saliency map [5] is perhaps the most cited saliency system to date. The authors propose a computational model for the theoretical framework presented by Koch and Ulman [6] to model visual attention. An input image is analyzed with opponent color maps, pixels gradient and orientation at different scale spaces. A winner take all (WTA) network fuses these multiple maps into a single saliency map. Local steering kernel-based saliency [9] is a based on the center-surround paradigm, which is employed to compute the saliency of a selected region. They propose a novel local steering kernel which is employed to compute the similarity between a center region and the surrounding patches. The methodology is inherently robust to image brightness and contrast changes, has very few parameters that requires to be fined-tuned. Symmetry based Saliency [2] is compute rectangular regions of interest centered on a pixel and compute first order moments of features to calculate the saliency of the given pixel. The methodology is among the most best in terms of programming simplicity. The authors also claim the biological plausibility of the model. Random center-surround pattern based Saliency [11] is a methodology that employs a biologically plausible dissimilarity metric which is employed to compute the contrast between any two random pixels on the input image. The contrasts are updated and normalized to generate the final saliency map. The said methodology is shown to have state-of the-art performance in the task of salient region detection. For the Random rectangular regions of interest based saliency [10], the same authors as in [11] propose to compute frequent- tuned salient region detection [1], on random rectangular regions of an image for a large number of times and then sum them to compute the final saliency map. Such a formulation is shown to have excellent correlation with human eye-gaze and has good performance for the task of salient regions as shown in their experiments. The methodology also has only two parameters which requires fine tuning, which reduces the implementation complexity. III. DATA The beginning and concluding snapshots of 12 motionese videos were cropped, and were parsed into the seven aforementioned saliency systems. A 15 x 15 region centered on the maximally salient point was chosen as the focus of attention. see Fig. 1. Results: Results are summarized in Fig. 2. The percentages describes the matching accuracy of a given saliency system to the child’s eye gazing behavior. It seems that the methodology of Seo and Milanfar [9] has the best matching with the gazing behavior of a 8-11 month old child. Figure 1 Figure 2 Acknowledgements Tadmeri Narayan Vikram gratefully acknowledges the financial support from the EU FP7 Marie Curie ITN RobotDoc. Contract No. 235065. Katrin S. Lohan, Katharina Rohlfing, and Britta Wrede gratefully acknowledge the financial support from the FP7 European Project ITALK.

Gaze plays an important role in human-human communication. Adequate gaze control of a virtual agent is also essential for successful and believable human-agent interaction. Researchers on HAI have developed gaze control models by taking account of gaze duration, frequency, and timing of gaze aversion. However, cultural differences in gaze behaviors have not been focused enough. We aimed to investigate cultural differences in gaze behaviors and their perception by developing virtual agents with Japanese gaze behaviors, American gaze behaviors, and fixed gaze behaviors. We then compared their effects on the impressions of the agents and interactions. Our first experimental results with Japanese participants suggested that the impression of the agent is affected by participants' shyness and familiarity of the gaze patterns performed by the agent. This paper reports the results of our second experiment that investigated the effects of the gender of agents and the gender of participants on perception of culture-specific gaze behaviors. The preliminary results suggests there are differences between the gender of participants and agents on the perception of favorable gaze behaviors. The female participants had a better impression on the agent of the same gender that performed the American gaze behavior, while the male participants preferred the agents with the Japanese gaze behavior. These results indicate favorable gaze behaviors vary according to users' gender and agents' gender.

In this study, non-verbal behavior in diversely-skilled groups was observed while participating in a collaborative educational game with a humanoid robot. Research has indicated that a mediating robot gaze can equalize the verbal contributions from each differently skilled participant, promoting inclusion and learning. The experiment results were further analyzed, extending to non-verbal effects. The initial results from two experiments under different robot gaze behavior indicate that modifications in the robot gaze can lead to different gaze behavior in participants. It was observed that a gaze mediating behavior in the robot led to increased gaze change frequency among participants as well as more time spent mirroring the robot's gaze. These initial results show promise in how a robot can balance attention in a collaborative learning environment.

Background: Previous research has explored gender differences in visuo-spatial intelligence and gaze behaviour; however, no comprehensive review has synthesised findings across parameters or examined their underlying causes. Aim: This review critically evaluates literature on gender differences in gaze behaviour and visuo-spatial intelligence. It integrates findings from visual software (cognitive and perceptual processes) and visual hardware (biological and physiological capacities), identifies methodological inconsistencies, and examines sociocultural moderators influencing performance. A dual-framework approach is proposed to explain the interaction between these domains. Method: Electronic searches were carried out across multiple databases, including Current Contents, ScienceDirect, EBM Reviews, CISTI Source (1990–February 2025), Google Scholar, SportDiscus (1990–February 2025), PubMed (1990–February 2025), the Cochrane Database of Systematic Reviews, and various international e-catalogues. Results: Seventy-seven full-text articles were reviewed. Consistent gender differences were observed in both biological capacities and cognitive strategies, although methodological variations contributed to inconsistencies. Sociocultural factors such as play experience and training exposure were identified as influential moderators. Conclusion: Gender differences in visio-spatial skills reflect biological, cognitive and social influences. An interactionist framework is recommended for future research. Contribution: This review advances understanding by introducing the distinction between ‘visual hardware’ and ‘visual software’, critically evaluating eye-tracking research and proposing a more nuanced interactionist framework for future investigations.

Investigating factors affecting social presence and user satisfaction with Mobile Instant Messaging

PurposeThe purpose of this paper is to examine how the use of immersive virtual reality (IVR) impacts on the surprise aspects of designing.Design/methodology/approachThe empirical case is a new hospital in the UK wherein a CAVE (Cave Automatic Virtual Environment) type of an IVR environment was used performing six design review sessions during the bid preparation stage. Drawing from a former video-based study, the authors conducted follow-up discussions with the participants to access their perspectives on design surprises emerging from their engagement with the IVR. The study developed a reflective methodology, interviewing participants about their experiences of doing design in the immersive environment. Retrospective discussions were conducted in a data review format, through playing back video clips of the IVR design sessions and asking the participants to reflect on their IVR design experience and on design surprises emerging from their engagement with the IVR.FindingsThe findings indicate that IVRs, such as the CAVE, are not only enhancing existing understandings of design but also challenging the participants' understanding of the design as they experience the immersive version of it, provoking ruptures in current procedures and driving unanticipated changes to the design.Originality/valueThis qualitative study of surprise in design work using IVRs (for a real-life design project) brings new insights into emerging practices of designing using immersive technology, such as the CAVE.

This paper presents follow-up longitudinal data to research that previously suggested the possibility of abnormal gaze behavior marked by decreased eye contact in a subgroup of 6-month-old infants at risk for autism (Merin, Young, Ozonoff & Rogers, 2007). Using eye-tracking data and behavioral data recorded during a live mother-infant interaction involving the still-face procedure, the predictive utility of gaze behavior and affective behaviors at 6 months was examined using diagnostic outcome data obtained longitudinally over the following 18 months. Results revealed that none of the infants previously identified as showing lower rates of eye contact had any signs of autism at outcome. In contrast, three infants who were diagnosed with autism demonstrated consistent gaze to the eye region and typical affective responses at 6 months. Individual differences in face scanning and affective responsivity during the live interaction were not related to any continuous measures of symptom frequency or symptom severity. In contrast, results of growth curve models for language development revealed significant relationships between face scanning and expressive language. Greater amounts of fixation to the mother's mouth during live interaction predicted higher levels of expressive language at outcome and greater rates of growth. These findings suggest that although gaze behavior at 6 months may not provide early markers for autism as initially conceived, gaze to the mouth in particular may be useful in predicting individual differences in language development.

Including others in a playful activity fundamentally changes the concept of play into social play. This thesis focuses on social play in digital gaming. Previous studies have revealed the crucial part of social interaction in play, as the intrinsic need to belong seems to be the core motivation for engaging in such activities. Besides interacting with the game and focusing on the game content, gamers are confronted with emotions, behaviours, opinions and performances of others that can easily be perceived when playing side by side (co-located co-play). However, the widespread penetration of the Internet also allows for social play without the restriction of co-players having to be in the same room (mediated co-play). In online co-play settings a smaller amount of social information can be exchanged compared to co-located co-play. These settings therefore differ in the way gamers can interact with each other, which has an influence on how digital gaming is experienced. Focus groups, contextual inquiries, and four experimental studies were employed to uncover which aspects of social play in digital gaming make mediated and co-located co-play feel different. The first study was conducted to explore the motivations of gamers to physically meet others to play digital games, compared to meeting them online. Based on interpersonal communications, focus groups and contextual inquiries, findings revealed that the appeal of co-located above online co-play results from the sociable, warm, sensitive, personal and intimate interactions that are possible when playing side-byside. Furthermore, co-located co-play offers the possibility to experience social fun before and after game play, which often is not afforded in online co-play. The first lab experiment was conducted to empirically demonstrate to what extent player experience is influenced by the way co-players are present; e.g. as a virtual (human controlled), mediated or co-located co-player. Results on self-reports indicated that playing sidebyside significantly adds to the enjoyment and involvement in games compared to playing against a virtual (i.e. computer controlled) or distant co-player. These results could be explained by introducing the concept of social presence, which is defined as the feeling of being together with another individual. Results demonstrated that social presence mediated the enjoyment in social play. In the second and third lab experiment the possibilities for verbal and non-verbal interaction were manipulated between coplayers in co-located co-play. One experiment investigated this for competitive play; a the other for collaborative play. Self-reports showed that interactions through auditory cues positively influenced player experience. Interestingly, the presence of visual cues of one’s co-player had no significant influence on play. Observation data of players confirmed that audio cues (e.g., talking, laughing) were far more often used than visual cues (e.g., eye contact, making gestures); visual cues were only used before and after play. Furthermore, we demonstrated that social presence – due to interaction by audio cues – mediates the enjoyment in social play. A final lab experiment was conducted to investigate the importance of the connection between players through the game for our results in the previous chapters. In this experiment players were colocated, but not always played together and/or were not always able to see each other’s scores. Results showed that a subjective shared experience increases the strength of the social connection between players, and positively affects feelings of social presence, enjoyment and involvement. Similar to the previous findings, social presence mediated the increase in feelings of enjoyment and involvement. In sum, our set of studies offered empirical support for when, why and how social interaction influences players’ experience in co-play settings. Furthermore, the results indicate the importance of social presence as a mediating factor of enjoyment and involvement in social play. This provides new theoretical insights for communication experiences in other media, and social presence in general. Furthermore, findings may be useful to game designers who may want to enhance players’ experiences in during digital play.

Recent advances in 3D reconstruction and tracking technologies have made it possible to volumetrically capture human body and performance at real time. In the field of human-computer interaction, however, no works have been reported on the user study made with such volumetric capture avatars. This paper investigates how the volumetric capture avatar affects users’ sense of social presence in immersive virtual environments. In our experiments, the volumetric capture avatar of an actor is compared with the actor captured in 2D video and another 3D avatar obtained by pre-scanning the actor. The experiment results show that users have the highest sense of social presence with the volumetric capture avatar when performing dynamic tasks whereas they have higher sense of social presence with the volumetric capture avatar and 2D video than with the pre-scanned avatar when performing static tasks. These imply that the emerging volumetric capture techniques can be an attractive tool for mixed reality, telepresence, and many other 3D applications.

Do female inpatients with fibromyalgia (FM) differ from healthy women in their nonverbal affective behavior? The data culled from 15 interviews with female FM inpatients and 15 interviews with healthy women were analyzed and the analyses of facial expression were subsequently correlated with gaze behavior. FM patients exhibited neither a reduction of total activity of facial expression nor of absolute frequency of primary affects in comparison to healthy subjects who did (also in eye contact) however exhibit a significantly higher proportion of "genuine joy" and lower proportion of "contempt." The absence of reduced total activity of facial expression is in contrast to the elaborate descriptions of symptoms provided by the patients, but detailed analyses show that elements which stabilize relationships are lacking and that dissociative elements are implanted in the interactions.

We present a service providing real-time feedback to participants of small group meetings on the social dynamics of the meeting. The service measures and visualizes properties of participants' behaviour that are relevant to the social dynamics of the meeting: speaking time and gaze behaviour. The dynamic visualization is offered to meeting participants during the meeting through a peripheral display. Whereas an initial version was evaluated using wizards to obtain the required information about gazing behaviour and speaking activity instead of perceptual systems, in the current paper we employ a system including automated perceptual components. We describe the system properties and the perceptual components. The service was evaluated in a within-subjects experiment, where groups of participants discussed topics of general interest, with a total of 82 participants. It was found that the presence of the feedback about speaking time influenced the behaviour of the participants in such a way that it made over-participators to behave less dominant and under-participators to become more active. Feedback on eye gaze behaviour did not affect participants' gazing behaviour (both for listeners and for speakers) during the meeting.

Purpose:Using eye-tracking, we assessed the receptive verb vocabularies of late talkers and typically developing children (Experiment 1) and autistic preschoolers (Experiment 2). We evaluated how many verbs participants knew and how quickly they processed the linguistic prompt.Method:Participants previewed two dynamic scenes side-by-side (e.g., “stretching” and “clapping”) and were then prompted to find the target verb. Children’s eye gaze behaviors were operationalized using established approaches in the field with modifications in consideration for the type of stimuli (dynamic scenes versus static images) and the populations included. Accuracy was calculated as a proportion of time spent looking to the target, and linguistic processing was operationalized as latency of children’s first look to the target.Results:In Experiment 1, there were no group differences in the proportion of verbs known, but late talkers required longer to demonstrate their knowledge than typically developing children. Latency was predicted by age but not language abilities. In Experiment 2, autistic children’s accuracy and latency were both predicted by receptive language abilities.Conclusion:Eye gaze can be used to assess receptive verb vocabulary in a variety of populations, but in operationalizing gaze behavior, we must account for between- and within-group differences. Bootstrapped cluster-permutation analysis is one way to create individualized measures of children’s gaze behavior, but more research is warranted using an individual differences approach with this type of analysis. Finally, latency may not be a valid measure for dynamic scene stimuli for children under three years old.