Following the recent initiatives for the democratization of AI, deep fake generators have become increasingly popular and accessible, causing dystopian scenarios towards social erosion of trust. A particular domain, such as biological signals, attracted attention towards detection methods that are capable of exploiting authenticity signatures in real videos that are not yet faked by generative approaches. In this paper, we first propose several prominent eye and gaze features that deep fakes exhibit differently. Second, we compile those features into signatures and analyze and compare those of real and fake videos, formulating geometric, visual, metric, temporal, and spectral variations. Third, we generalize this formulation to the deep fake detection problem by a deep neural network, to classify any video in the wild as fake or real. We evaluate our approach on several deep fake datasets, achieving 92.48% accuracy on FaceForensics++, 80.0% on Deep Fakes (in the wild), 88.35% on CelebDF, and 99.27% on DeeperForensics datasets. Our approach outperforms most deep and biological fake detectors with complex network architectures without the proposed gaze signatures. We conduct ablation studies involving different features, architectures, sequence durations, and post-processing artifacts.

Gaze-based assistive technologies (ATs) that feature speech have the potential to improve the life of people with communication disorders. However, due to a limited understanding of how different speech types affect the cognitive load of users, an evaluation of ATs remains a challenge. Expanding on previous work, we combined temporal changes in pupil size and ocular movements (saccades and fixation differentials) to evaluate cognitive workload of two types of speech (natural and synthetic) mixed with noise, through a listening test. While observed pupil sizes were significantly larger at lower signal-to-noise levels, as participants listened and memorised speech stimuli; saccadic eye-movements were significantly more frequent for synthetic speech. In the synthetic condition, there was a strong negative correlation between pupil dilation and fixation differentials, indicating a higher strain on participants’ cognitive resources. These results suggest that combining oculo-motor indices can aid our understanding of the cognitive implications of different speech types.

Measuring where people look in real-world tasks has never been easier but analyzing the resulting data remains laborious. One solution integrates head-mounted eye tracking with motion capture but no best practice exists regarding what calibration data to collect. Here, we compared four ~1 min calibration routines used to train linear regression gaze vector models and examined how the coordinate system, eye data used and location of fixation changed gaze vector accuracy on three trial types: calibration, validation (static fixation to task relevant locations), and task (naturally occurring fixations during object interaction). Impressively, predicted gaze vectors show ~1 cm of error when looking straight ahead toward objects during natural arms-length interaction. This result was achieved predicting fixations in a Spherical coordinate frame, from the best monocular data, and, surprisingly, depends little on the calibration routine.

Graphical user authentication (GUA) is a common alternative to text-based user authentication, where people are required to draw graphical passwords on background images. Recent research provides evidence that gamification of the graphical password creation process influences people to make less predictable choices. Aiming to understand the underlying reasons from a visual behavior perspective, in this paper, we report a small-scale eye-tracking study that compares the visual behavior developed by people who follow a gamified approach and people who follow a non-gamified approach to make their graphical password choices. The results show that people who follow a gamified approach have higher gaze-based entropy, as they fixate on more image areas and for longer periods, and thus, they have an increased effective password space, which could lead to better and less predictable password choices.

People tend to develop different cognitive styles, which influence how we process information when interacting with computer systems. Field Dependence-Independence is one of the most well-known cognitive styles that influences how we process information in visual search tasks. Considering that such tasks are common in video games, this paper investigates whether information processing differences, derived from Field Dependence-Independence cognitive style, reflect on different eye trajectories when playing a visual search game. We performed a small-scale eye-tracking study to investigate it. The results of the scanpath analysis indicated that such differences exist. The study results provide a first step towards understanding how people who share different cognitive styles differ in the scanpaths they develop when playing a visual search game.

Mobile, simultaneous tracking of both the head and eyes is typically achieved through integration of separate head and eye tracking systems because off-the-shelf solutions do not yet exist. Similarly, joint visualization and analysis of head and eye movement data is not possible with standard software packages because these were designed to support either head or eye tracking in isolation. Thus, there is a need for software that supports joint analysis of head and eye data to characterize and investigate topics including head-eye coordination and reconstruction of how the eye is moving in space. To address this need, we have begun developing VEDBViz which supports simultaneous graphing and animation of head and eye movement data recorded with the Intel RealSense T265 and Pupil Core, respectively. We describe current functionality as well as features and applications that are still in development.

Developments in wearable eye tracking devices make them an attractive solution for studies of eye movements during naturalistic head/body motion. However, before these systems’ potential can be fully realized, a thorough assessment of potential sources of error is needed. In this study, we examine three possible sources for the Pupil Core eye tracking goggles: camera motion during head/body motion, choice of calibration marker configuration, and eye movement estimation. In our data, we find that up to 36% of reported eye motion may be attributable to camera movement; choice of appropriate calibration routine is essential for minimizing error; and the use of a secondary calibration for eye position remapping can improve eye position errors estimated from the eye tracker.

Recent advances in appearance-based models have shown improved eye tracking performance in difficult scenarios like occlusion due to eyelashes, eyelids or camera placement, and environmental reflections on the cornea and glasses. The key reason for the improvement is the accurate and robust identification of eye parts (pupil, iris, and sclera regions). The improved accuracy often comes at the cost of labeling an enormous dataset, which is complex and time-consuming. This work presents two semi-supervised learning frameworks to identify eye-parts by taking advantage of unlabeled images where labeled datasets are scarce. With these frameworks, leveraging the domain-specific augmentation and novel spatially varying transformations for image segmentation, we show improved performance on various test cases with limited labeled samples. For instance, for a model trained on just 4 and 48 labeled images, these frameworks improved by at least 4.7% and 0.4% respectively, in segmentation performance over the baseline model, which is trained only with the labeled dataset.

Detecting the point-of-gaze in the real world is a challenging problem in eye-tracking applications. The point-of-gaze is estimated using geometry constraints, and user-calibration is required. In addition, the distances of the focused targets are variable and large in the real world. Therefore, a calibration-free approach without geometry constraints is needed to estimate the point-of-gaze. Recent studies have investigated smooth pursuit eye movements (smooth pursuits) for human-computer interaction applications, and we consider that these smooth pursuits can also be employed in eye tracking. Therefore, we developed a method for estimating the point-of-gaze using smooth pursuits without any requirement for implicit and explicit user-calibration. In this method, interest points are extracted from the scene image, and the point-of-gaze is detected using these points, which are strongly correlated with eye movements. We performed a comparative experiment in a real environment and demonstrated the feasibility of the proposed method.

Gaze text entry (GTE) with use of visual keyboards displayed on computer screen is an important topic for both the scientists dealing with the gaze interaction and potential users i.e. people with physical disabilities and their families. The most commonly used technique for GTE is based on dwell-time regions, at which the user needs to look longer to activate the associated action, in our case - entering the letter. In the article, we present the results of tests of three GTE systems (gaze keyboards) on a sample of 29 participants. We compare the objective measures of usability, namely the text entry rate and the number of errors, as well as subjective ones, obtained using SUS questionnaire. Additionally, two similar keyboards based on the ‘Qwerty’ buttons layout were compared in terms of time to the first fixation and its duration in the areas of interest (AOI) corresponding to the visual buttons. One of these gaze keyboards, the so called ’Molecular’ one, contains dynamic elements that have been designed and implemented in our laboratory, and which aim is to support the search for buttons by increasing the size of buttons with suggested letters, without significant change of their positions.