Detecting colour vision deficiencies via Webcam-based Eye-tracking: A case study

Validating and improving the correction of ocular artifacts in electro-encephalography

Introduction: Assessment of point-of-care ultrasound (POCUS) competency has been reliant on practical, visual and written examinations performed 1-on-1 with an examiner. These tools attempt to assess competency through subjective ratings, checklists and multiple-choice questions that are labour intensive using surrogate measures. Eye-tracking has been used on a limited basis in various fields of medicine for training and assessment. This technology explores visual processing and holds great promise as a tool to monitor training progress towards the development of expertise. We hypothesize that eye-tracking may differentiate novices and experts as they progress to become competent in interpretation of POCUS images and provide an objective measure in assessment of competency. Methods: Medical students, residents and attending physicians working in an academic emergency department were recruited. Participants viewed a series of 16 ultrasound video clips in a POCUS protocol for Focused Assessment using Sonography in Trauma (FAST). The gaze pattern of the participants was recorded using a commercially available eye-tracking device. The primary outcome was the gaze parameters including total gaze time in the area of interest (AOI), average time to fixation on the AOI, number of fixations in the AOI and average duration of first fixation on the AOI. Secondary outcome was the accuracy on the interpretation of the FAST scan. Results: Four novices and eight experts completed this study. The total gaze time in the AOI (mean +/- SD) was 76.72 +/- 18.84s among experts vs 53.64 +/- 10.33s among novices (p = 0.048), average time to fixation on the AOI was 0.561 +/- 0.319s vs 1.048 +/- 0.280s (p = 0.027), number of fixations in the AOI was 158.9 +/- 29.0 vs 121.8 +/- 17.5 (p = 0.042) and average duration of first fixation was 0.444 +/- 0.119s vs 0.390 +/- 0.024s (p = 0.402). The accuracy of the answers was 79.7 +/- 14.1% vs 45.3 +/- 21.9% (p = 0.007). Conclusion: In this pilot study, eye tracking shows potential to differentiate between POCUS experts and novices by their gaze patterns. Gaze patterns captured by eye tracking may not necessarily translate to cognitive processing. However, it allows educators to visualise the thought processes of the learner by their gaze patterns and provide insight on how to guide them towards competency. Future studies are needed to further validate the metrics for competency in POCUS applications.

Eye and hand movement tracking has been proven to be a successful tool and is widely used to figure out characteristics of human cognition in language or visual processing (Just & Carpenter, 1976 Cognitive Psychology8441–480). Eye movement has proven to be a successful measure to figure out characteristics of human language and visual processing (Rayner, 1998 Psychological Bulletin124(3) 372–422). Recently, mouse tracking was used for social-cognition-like categorization of sex-atypical faces and studying spoken-language processes (Magnuson, 2005 PNAS102(28) 9995–9996; Spivey et al., 2005 PNAS102 10393–10398). Here, we present a framework that uses both eye gaze and hand movement simultaneously for analyzing the relation of them during memory retrieval. We tracked eye and mouse movements when the subject was watching a drama and playing a multimodal memory game (MMG), a cognitive task designed to investigate the recall memory mechanisms in watching video dramas (Zhang, 2009 AAAI 2009 Spring Symposium: Agents that Learn from Human Teachers 144–149). Experimental results show that eye tracking and mouse tracking provide complementary information about underlying cognitive processes. Also, we found some interesting patterns in eye-hand movement during multimodal memory recall.

Human behaviour is not only influenced by the physical presence of others, but also implied social presence. This study examines the impact of awareness of being eye-tracked on eye movement behaviour in a laboratory setting. During a classic yes/no face recognition task, participants were made to believe that their eye movements were recorded (or not recorded) by eye trackers. Their looking patterns with and without the awareness of being eye-tracked were compared while perceiving social (faces, faces-and-bodies) and non-social (inanimate objects) video stimuli. Area-of-interest (AOI) analysis revealed that misinformed participants (who were not aware that their eye movements were being recorded) looked more at the body (chest and waist) compared to informed participants (who believed they were being eye-tracked), whereas informed participants fixated longer on the mouth and shorter on the eyes of female models than misinformed participants did. These findings highlight the potential impact of an awareness of being eye tracked on one’s eye movement pattern when perceiving a social stimulus. We conclude that even within laboratory settings an eye tracker may function as an implied social presence that leads individuals to modify their eye movement behaviour according to socially-derived inhibitory norms.

How Does Social Influence Really Affect Consumer Decisions? Insights from an Eye Tracking Study

Eye tracking analysis can be used to evaluate human performance. However, it is a challenging task to map eye tracking data to multi-element moving objects if the objects take different complex shapes and dynamically move within the display. We propose algorithms that approximate multi-element moving objects into simpler areas of interest (AOIs). Based on minimum AOIs required to represent objects, three AOI types (rectangle, circle, and triangle) were created that maintains the same sizes regardless of object shapes. The algorithms were applied to a task of observing a simulated air traffic control display. Two types of eye fixation data were used for evaluation: Simulated data using uniform distribution and human participant data. The number of eye fixations differed significantly among AOI types when simulated data were used ( p < .001) but were insignificant when human data were used ( p = .94). We discuss how eye tracking data may be affected by AOI types, and we offer suggestions to improve simulated eye fixations and AOI designs for multi-element moving objects. This pilot study provides ground work for better AOI design and possibly for automating the development of accurate AOIs for multi-element moving objects.

IntroductionThe use of eye tracking (ET) in mathematics education research has increased in recent years. Eye tracking is a promising research tool in the domain of functions, especially in graph interpretation. It promises to gain insights into learners’ approaches and ways of thinking. However, for the domain of functions and graph interpretation, it has not yet been investigated how eye-tracking data can be interpreted. In particular, it is not clear how eye movements may reflect students’ cognitive processes. Thus, in this study, we investigate in how far the eye-mind hypothesis (EMH), which states broadly that what the eye fixates is currently being processed, can be applied to this subdomain. This is particularly true for contextual graphs, whose data originate from real-world situations, and which are of central importance for the development of mathematical literacy. The aim of our research is to investigate how eye movements can be interpreted in the domain of functions, particularly in students’ interpretations of contextual graphs.MethodsWe conducted an exploratory case study with two university students: The students’ eye movements were recorded while they worked on graph interpretation tasks in three situational contexts at different question levels. Additionally, we conducted subsequent stimulated recall interviews (SRIs), in which the students recalled and reported their original thoughts while interpreting the graphs.ResultsWe found that the students’ eye movements were often related to students’ cognitive processes, even if indirectly at times, and there was only limited ambiguity in the interpretation of eye movements. However, we also found domain-specific as well as domain-general challenges in interpreting eye movements.DiscussionOur results suggest that ET has a high potential to gain insights into students’ graph interpretation processes. Furthermore, they point out what aspects, such as ambiguity and peripheral vision, need to be taken into consideration when investigating eye movements in the domain of functions.

Eye tracking is a process that identifies a specific point in both space and time that is being looked at by the observer. This information can also be used in real-time to control applications using the eyes. Recent innovations in the video game industry include alternative input modalities to provide an enhanced, more immersive user experience. In particular, eye gaze control has recently been explored as an input modality in video games. This book is an introduction for those interested in using eye tracking to control or analyze video games and virtual environments. Key concepts are illustrated through three case studies in which gaze control and voice recognition have been used in combination to control virtual characters and applications. The lessons learned in the case studies are presented and issues relating to incorporating eye tracking in interactive applications are discussed. The reader will be given an introduction to human visual attention, eye movements and eye tracking technologies. Previous work in the field of studying fixation behavior in games and using eye tracking for video game interaction will also be presented. The final chapter discusses ideas for how this field can be developed further to create richer interaction for characters and crowds in virtual environments. Alternative means of interaction in video games are especially important for disabled users for whom traditional techniques, such as mouse and keyboard, may be far from ideal. This book is also relevant for those wishing to use gaze control in applications other than games.

The present study utilized a new experimental set-up synchronizing eye movements and head motion for investigating referential change occurring in a reading task. We examined the effects of a change in narrative perspective during reading on eye movements and head motion. Forty-four participants read texts on a digital tablet, and both participants' eye movements and head movements were recorded using eye tracker and motion capture. The results showed longer eye fixation duration, longer reading time and decreasing head motions when perspective changed. Recent studies have supported the dynamic engagement hypothesis suggesting that there is a fluctuation in cognitive engagement reflected by postural movements. Our findings on head movements seem to validate this hypothesis of a bodily engagement in reading. The results provided by our study showed that the novel methodology of eye and head movement recordings we used was proven to be informative in studying the reader's embodiment during reading.

The COVID-19 pandemic has accelerated our transition to an online and self-directed learning environment. In an effort to design better e-learning materials, we investigated the effectiveness of collecting psychophysiological eye-tracking data from participants in response to visual stimuli. In particular, we focused on collecting fixation data since this is closely related to human attention. Current wearable devices allow the measurement of visual data unobtrusively and in real-time, leading to new applications in wearable technology. Despite their accuracy, head-mounted eye trackers are too expensive for deployment on large-scale deployment. Therefore, we developed a low-cost, webcam-based eye tracking solution and compared its performance with a commercial head-mounted eye tracker. Four-minute lecture slides on the 3^rd year electronic engineering course were presented as stimuli to eight learners for data collection. Their eye movement was collected within the pre-defined area of interest (AOI). Our results demonstrate that a low-cost webcam-based eye-tracking solution, combined with machine learning algorithms, can achieve similar accuracy to the head-worn tracker. Based on these results, learners can use the eye tracker for attention guidance. Our work also demonstrates that these webcam-based eye trackers can be scaled up and used in large classrooms to provide real-time information to instructors regarding student attention and behaviour.

An adequate behavioral response depends on attentional and mnesic processes. When these basic cognitive functions are impaired, the use of non-immersive Virtual Reality Applications (VRAs) can be a reliable technique for assessing the level of impairment. However, most non-immersive VRAs use indirect measures to make inferences about visual attention and mnesic processes (e.g., time to task completion, error rate). To examine whether the eye movement analysis through eye tracking (ET) can be a reliable method to probe more effectively where and how attention is deployed and how it is linked with visual working memory during comparative visual search tasks (CVSTs) in non-immersive VRAs. The eye movements of 50 healthy participants were continuously recorded while CVSTs, selected from a set of cognitive tasks in the Systemic Lisbon Battery (SLB). Then a VRA designed to assess of cognitive impairments were randomly presented. The total fixation duration, the number of visits in the areas of interest and in the interstimulus space, along with the total execution time was significantly different as a function of the Mini Mental State Examination (MMSE) scores. The present study demonstrates that CVSTs in SLB, when combined with ET, can be a reliable and unobtrusive method for assessing cognitive abilities in healthy individuals, opening it to potential use in clinical samples.

Attention and working memory are essential aspects of human cognitive abilities. When these capacities are impaired, virtual reality applications (VRAs) can be a tool to assess and rehabilitate. However, most VRAs use indirect manual reaction time measures to make inferences about visual attention processes. Since behavioral measures are quite indirect, eye tracking (ET) can offer a better alternative to probe more directly where and how attention is deployed. We propose two tasks with different levels of perceptual complexity for assessing visual attention and performance. Two comparative visual search tasks, as part of a set of cognitive tasks in the Systemic Lisbon Battery (SLB), a VRA designed to assess of cognitive impairments, were randomly presented to 39 healthy participants while the eye movements were continuously recorded. The total fixation duration, the number of visits in the areas of interest, along with the total execution time changed as a function of the overall score of cognitive ability. The present study demonstrates that the spot the differences tasks in SLB, combined with ET, are a reliable, sensitive and unobtrusive method to assess cognitive abilities in healthy individuals with a relevant potential use in clinical samples.

41. Application of eye tracking in neuroscience

Personal calibration is a procedure which offsets the impact of personal differences on gaze tracking performance. During the calibration, the user has to look precisely at five to twenty markers that are shown the computer screen in succession. While the calibration marker is displayed on the screen, the user should gaze at the marker without eye movement until it disappears. This is a troublesome and wearying task. If she doesn’t look at the exact position of the marker, or if there is any involuntary eye movement or eye blink, the result may contain gaze tracking error. The difficulty of calibration explodes when the number of calibration markers increases. Estimating human eye gaze without personal calibration is one of the most important goals in the field of gaze tracking technology. For researchers who want to capture the user’s eye movement by a gaze tracking system, it is well known that the high cost of traditional personal calibration procedures makes eye tracking studies difficult. For example, it is extremely difficult to observe a child’s eye movement because of the personal calibration barrier. Since infants cannot understand the observer’s instructions, personal calibration often fails. A lot of effort is needed to attract the children’s attention to the calibration markers to accomplish personal calibration. The problem of personal calibration is not limited to children. For example, it is difficult to measure the eye movements of a large number of subjects because calibration is so time consuming.

A system for stabilising and monitoring eye movements for linac-based stereotactic radiotherapy associated with the mobile eye, the Eye Tracker, was developed. Whilst the Eye Tracker design is based on a previously reported system, the purpose of this study was to confirm that the modified version can be used with clinically acceptable treatment margins. We report the estimates of the margin required to account for inter- and intra-fraction eye motion based on data from 12 consecutive patients treated with the Eye Tracker system in place. Patients were immobilised in a head and neck mask and were required to fixate on a light source. A camera system monitored eye movements relative to CT simulation baseline measurements. The Exactrac system (Brainlab, Feldkirchen, Germany) combined with the Varian TrueBeamSTx (Varian Medical Systems, Palo Alto, CA) confirmed pre- and intra-treatment setup of the head position. Displacement/rotation of the image of the pupil/iris was determined in the lateral and superior-inferior directions using a video display. A standard margin equation was applied to estimate the margin required to account for inter- and intra-fraction eye movement. The average displacement in both directions was 0.1-0.2mm (0.36mm SD). All patients maintained a position within 1mm of the intended position during treatment. Based on a Bayesian estimation of the systematic and treatment errors, accounting for displacements in two-planes and a standard deviation of the penumbral width of 1.3mm, the estimated margins to achieve coverage of the GTV with the 95% isodose in 90% of patients was found to be less than 1mm. Small random and systematic uncertainties due to inter- and intra-fraction movement of the eye were achieved with the Eye Tracker. Whilst the estimated margins are small (<1mm) they need to be considered in addition to contouring and treatment delivery uncertainties.