Eye-Controlled Mouse for Physically Disabled Individual Using OpenCV

Real-time human eye recognition and tracking systems with human-computer interaction mechanism are being adopted to advance user experience in smart devices and consumer electronic systems. Eye tracking systems measure eye gaze and pupil response non-intrusively. This paper presents an analysis of pupil response to video structure and content. A consumer device that can assess user response to a video can provide better experiences with approaches such as real-time content adaptation. The first set of experiments involved presenting different video content to subjects and measuring eye response with an eye tracker. Results show that pupil constrictions and magnitude of constrictions vary with content. Significant changes in video and scene cuts led to sharp constrictions. User response to videos can provide insights that can improve subjective quality assessment metrics. This paper also presents an analysis of the pupillary response to quality changes in videos with the second set of experiments. The results of the tests show pupil constrictions for noticeable changes in perceived quality. Using real-time eye tracking systems for video analysis and quality evaluation can open a new class of applications for consumer electronic systems.

This paper describes a flexible method for synchronizing electroencephalography (EEG) and eye tracking (ET) recordings to the presentation of visual stimuli. The method consists of embedding a synchronization signal in the visual stimuli, and recording this signal with both the EEG and ET equipment. The signal is recorded by the EEG device as an additional data channel, and with the camera used in the ET equipment by modulating the global illumination of the scene in time with the synchronizing signal. The prototype system where this method was implemented resulted in a single sample of jitter in both the EEG and ET system, while the ET system achieved a spatial resolution of 1.26 degrees. The system will be used in future work with augmented memory applications.

Eye tracking has become an essential human–machine interaction modality in virtual reality (VR) and augmented reality (AR) applications requiring high throughput (e.g., more than 240 frames per second), small form factor, and enhanced visual privacy. Existing eye tracking systems have adopted bulky, lens-based cameras, and thus suffer from both a large form factor and high communication cost between the camera and back-end processor. This work presents a camera, algorithm, and accelerator co-designed lensless eye tracking system dubbed EyeCoD, which, to the best of our knowledge, is the first to provide a general, front-end eye tracking solution for AR/VR while satisfying the requirements for both high throughput and smaller form factor. Specifically, EyeCoD integrates system-, algorithm-, and accelerator-level techniques to boost system efficiency without sacrificing eye tracking accuracy. We believe that our EyeCoD system will pave the way for next-generation eye tracking solutions in VR/AR and shed light on future innovations for intelligent imaging systems.

Eye tracking provides highly efficient, intuitive, and seamless human-machine interaction by measuring eye movements and revealing user attention and intention. Currently, video-based eye tracking systems, while achieving sub-degree accuracy, suffer from high power consumption due to data acquisition and processing of redundant pixels. Here, we present an energy-efficient eye-tracking system employing a position-sensitive detector (PSD) with a graphene photoconductive mesh, which operates with only four readout signals, reducing data volume by over three orders of magnitude compared to video-based systems, while achieving superior accuracy. With the synergistic effects of lateral photoelectric and interfacial gating, the mesh-structured PSD operates under eye-safe ultra-weak light (as low as 10pW) without a detection dead zone. Leveraging the real-time, precise tracking capability of the PSD, the eye-tracking system determines gaze orientation by tracking the reflected trajectories of light beams from the cornea, achieving a 1kHz tracking frequency and < 0.1° angular accuracy. Human-machine interaction applications such as eye-controlled typing and a Gluttonous Snake game are demonstrated using the PSD-enabled eye tracker. The PSD-based eye tracking holds significant promise for the application of human-machine interaction in miniaturized systems.

The advancement of eye tracking technology, including improved portability, usability, and affordability, has facilitated the transition of eye trackers from controlled laboratory settings to diverse real-world environments. One significant challenge in these real-world settings is the wide variation in illumination levels, potentially affecting eye tracking accuracy. Despite anecdotal evidence suggesting the sensitivity of head-mounted eye trackers to environmental light conditions, systematic research on this topic has been limited. This study introduces a standardized protocol for investigating the influence of lighting conditions on the accuracy of head-mounted eye trackers. We examined the performance of SMI Eye Tracking Glasses 2 Wireless under three distinct illumination levels: Low (Dark), Moderate (Normal), and High (Bright). Our experiment involved 9 participants performing accuracy tests under each lighting condition. We measured gaze accuracy while varying the illumination level during both the calibration and data recording phases. Results indicate that eye tracking accuracy is affected by the illumination level used during data collection. Best accuracy was achieved when data recording was performed under the same illumination level as the calibration. Notably, calibration under moderate illumination led to the most consistent performance across all lighting conditions. These findings provide valuable insights for researchers and practitioners using head-mounted eye trackers in varying lighting environments. The standardized protocol presented can be applied to evaluate other eye tracking systems, contributing to developing more robust mobile eye tracking techniques. Future research should investigate the generalizability of these results across different eye tracking technologies and explore potential software solutions to mitigate illumination-related accuracy issues.

A system for stabilising and monitoring eye movements for linac-based stereotactic radiotherapy associated with the mobile eye, the Eye Tracker, was developed. Whilst the Eye Tracker design is based on a previously reported system, the purpose of this study was to confirm that the modified version can be used with clinically acceptable treatment margins. We report the estimates of the margin required to account for inter- and intra-fraction eye motion based on data from 12 consecutive patients treated with the Eye Tracker system in place. Patients were immobilised in a head and neck mask and were required to fixate on a light source. A camera system monitored eye movements relative to CT simulation baseline measurements. The Exactrac system (Brainlab, Feldkirchen, Germany) combined with the Varian TrueBeamSTx (Varian Medical Systems, Palo Alto, CA) confirmed pre- and intra-treatment setup of the head position. Displacement/rotation of the image of the pupil/iris was determined in the lateral and superior-inferior directions using a video display. A standard margin equation was applied to estimate the margin required to account for inter- and intra-fraction eye movement. The average displacement in both directions was 0.1-0.2mm (0.36mm SD). All patients maintained a position within 1mm of the intended position during treatment. Based on a Bayesian estimation of the systematic and treatment errors, accounting for displacements in two-planes and a standard deviation of the penumbral width of 1.3mm, the estimated margins to achieve coverage of the GTV with the 95% isodose in 90% of patients was found to be less than 1mm. Small random and systematic uncertainties due to inter- and intra-fraction movement of the eye were achieved with the Eye Tracker. Whilst the estimated margins are small (<1mm) they need to be considered in addition to contouring and treatment delivery uncertainties.

The process of eye tracking has been subject of intensive research during the last years. The main aim consists in the determination of the point of gaze or the eye movement relative to the head. In this work, we use a static video camera to realize the tracking of the eyes movement mainly due to its important features such as low cost, fully automated and easy application. The proposed approach encompasses a solution to the problems of eye detection and eye tracking. Our eye detection method employs a set of cascaded Haar classifiers that enable the fast recognition of eye regions in a real-time sequence of images. On the other hand, the eye tracking method uses particle filtering that is robust in clutter environments and allows the fusion of sensor data as well as the tracking of nonlinear systems. The results demonstrate that the eye tracking system is capable of robustly track the eyes under variable light conditions and under several different face orientations.

BackgroundOcular following responses (OFRs)—small-amplitude, short-latency reflexive eye movements—have been used to study visual motion processing, with potential diagnostic applications. However, they are difficult to record with commercial, video-based eye trackers, especially in children.ObjectiveWe aimed to design and develop a noninvasive eye tracker specialized for measuring OFRs, trading off lower temporal resolution and a smaller range for higher spatial resolution.MethodsWe developed a high-resolution eye-tracking system based on a high-resolution camera operating in the near-infrared spectral range, coupled with infrared illuminators and a dedicated postprocessing pipeline, optimized to measure OFRs in children. To assess its performance, we: (1) evaluated our algorithm for compensating small head movements in both artificial and real-world settings, (2) compared OFRs measured simultaneously by our system and a reference scleral search coil eye-tracking system, and (3) tested the system’s ability to measure OFRs in a clinical setting with children.ResultsThe simultaneous measurement by our system and a reference system showed that our system achieved an in vivo resolution of approximately 0.06°, which is sufficient for recording OFRs. Head motion compensation was successfully tested, showing a displacement error of less than 5 μm. Finally, robust OFRs were detected in 16 children during recording sessions lasting less than 5 minutes.ConclusionsOur high-resolution, noninvasive eye-tracking system successfully detected OFRs with minimal need for subject cooperation. The system effectively addresses the limits of other OFR measurement methods and offers a versatile solution suitable for clinical applications, particularly in children, where eye tracking is more challenging. The system could potentially be suitable for diagnostic applications, particularly in pediatric populations where early detection of visual disorders like stereodeficiencies is critical.

Eye tracking allows psychologists to make broad distinctions between groups of participants. Therefore, it may be entirely possible to exploit these distinctions in order to create screening tools for potentially diagnosing various conditions. These diagnostic techniques may have a number of advantages over exiting techniques. However, in order to develop such screening tools it would be beneficial if eye tracking systems were easy to access and use. There are a number of ways to improve accessibility of eye trackers: affordability, transportation, ease of use. This positional paper explores how HCI (Human Computer Interaction) eye trackers can be used for diagnostic purposes of psychological conditions.

A challenging goal today is the use of computer networking and advanced monitoring technologies to extend human intellectual capabilities in medical decision making. Modern commercial eye trackers are used in many of research fields, but the improvement of eye tracking technology, in terms of precision on the eye movements capture, has led to consider the eye tracker as a tool for vision analysis, so that its application in medical research, e.g. in ophthalmology, cognitive psychology and in neuroscience has grown considerably. The improvements of the human eye tracker interface become more and more important to allow medical doctors to increase their diagnosis capacity, especially if the interface allows them to remotely administer the clinical tests more appropriate for the problem at hand. In this paper, we propose a client/server eye tracking system that provides an interactive system for monitoring patients eye movements depending on the clinical test administered by the medical doctors. The system supports the retrieval of the gaze information and provides statistics to both medical research and disease diagnosis.

Eye tracking systems that estimate the point-of-gaze are essential in extended reality (XR) systems as they enable new interaction paradigms and technological improvements. It is important for these systems to maintain accuracy when the headset moves relative to the head (known as device slippage) due to head movements or user adjustment. One of the most accurate eye tracking techniques, which is also insensitive to shifts of the system relative to the head, uses two or more infrared (IR) light emitting diodes to illuminate the eye and an IR camera to capture images of the eye. An essential step in estimating the point-of-gaze in these systems is the precise determination of the location of two or more corneal reflections (virtual images of the IR-LEDs that illuminate the eye) in images of the eye. Eye trackers tend to have multiple light sources to ensure at least one pair of reflections for each gaze position. The use of multiple light sources introduces a difficult problem: the need to match the corneal reflections with the corresponding light source over the range of expected eye movements. Corneal reflection detection and matching often fail in XR systems due to the proximity of camera and steep illumination angles of light sources with respect to the eye. The failures are caused by corneal reflections having varying shape and intensity levels or disappearance due to rotation of the eye, or the presence of spurious reflections. We have developed a fully convolutional neural network, based on the UNET architecture, that solves the detection and matching problem in the presence of spurious and missing reflections. Eye images of 25 people were collected in a virtual reality headset using a binocular eye tracking module consisting of five infrared light sources per eye. A set of 4,000 eye images were manually labelled for each of the corneal reflections, and data augmentation was used to generate a dataset of 40,000 images. The network is able to correctly identify and match 91% of corneal reflections present in the test set. This is comparable to a state-of-the-art deep learning system, but our approach requires 33 times less memory and executes 10 times faster. The proposed algorithm, when used in an eye tracker in a VR system, achieved an average mean absolute gaze error of 1°. This is a significant improvement over the state-of-the-art learning-based XR eye tracking systems that have reported gaze errors of 2-3°.

Eye tracking systems that estimate the point-of-gaze are essential in extended reality (XR) systems as they enable new interaction paradigms and technological improvements. It is important for these systems to maintain accuracy when the headset moves relative to the head (known as device slippage) due to head movements or user adjustment. One of the most accurate eye tracking techniques, which is also insensitive to shifts of the system relative to the head, uses two or more infrared (IR) light emitting diodes to illuminate the eye and an IR camera to capture images of the eye. An essential step in estimating the point-of-gaze in these systems is the precise determination of the location of two or more corneal reflections (virtual images of the IR-LEDs that illuminate the eye) in images of the eye. Eye trackers tend to have multiple light sources to ensure at least one pair of reflections for each gaze position. The use of multiple light sources introduces a difficult problem: the need to match the corneal reflections with the corresponding light source over the range of expected eye movements. Corneal reflection detection and matching often fail in XR systems due to the proximity of camera and steep illumination angles of light sources with respect to the eye. The failures are caused by corneal reflections having varying shape and intensity levels or disappearance due to rotation of the eye, or the presence of spurious reflections. We have developed a fully convolutional neural network, based on the UNET architecture, that solves the detection and matching problem in the presence of spurious and missing reflections. Eye images of 25 people were collected in a virtual reality headset using a binocular eye tracking module consisting of five infrared light sources per eye. A set of 4,000 eye images were manually labelled for each of the corneal reflections, and data augmentation was used to generate a dataset of 40,000 images. The network is able to correctly identify and match 91% of corneal reflections present in the test set. This is comparable to a state-of-the-art deep learning system, but our approach requires 33 times less memory and executes 10 times faster. The proposed algorithm, when used in an eye tracker in a VR system, achieved an average mean absolute gaze error of 1°. This is a significant improvement over the state-of-the-art learning-based XR eye tracking systems that have reported gaze errors of 2-3°.

A new motion-based gated proton therapy for the treatment of orbital tumors using real-time eye-tracking system was designed and evaluated. We developed our system by image-pattern matching, using a normalized cross-correlation technique with LabVIEW 8.6 and Vision Assistant 8.6 (National Instruments, Austin, TX). To measure the pixel spacing of an image consistently, four different calibration modes such as the point-detection, the edge-detection, the line-measurement, and the manual measurement mode were suggested and used. After these methods were applied to proton therapy, gating was performed, and radiation dose distributions were evaluated. Moving phantom verification measurements resulted in errors of less than 0.1 mm for given ranges of translation. Dosimetric evaluation of the beam-gating system versus nongated treatment delivery with a moving phantom shows that while there was only 0.83 mm growth in lateral penumbra for gated radiotherapy, there was 4.95 mm growth in lateral penumbra in case of nongated exposure. The analysis from clinical results suggests that the average of eye movements depends distinctively on each patient by showing 0.44 mm, 0.45 mm, and 0.86 mm for three patients, respectively. The developed automatic eye-tracking based beam-gating system enabled us to perform high-precision proton radiotherapy of orbital tumors.

Preliminary Usability Testing with Eye Tracking and FCAT Analysis on Occupational Safety and Health Websites

Feasibility of Saccadic Vector Optokinetic Perimetry: A Method of Automated Static Perimetry for Children Using Eye Tracking