Design and Implementation of Human Intention Estimation System Based on Eye Movement Tracking

Cognitive dysfunction is the most common and important nonmotor symptom in Parkinson’s disease (PD) and can occur at any stage. However, there is still a lack of effective biomarkers to evaluate the decline in cognitive function and predict the progression of the disease, especially in the early stage. At present, the cognitive scale is widely used to evaluate the cognitive function of patients with PD, but its sensitivity and accuracy are relatively limited, especially in the early identification of mild cognitive impairment. Eye movement tracking is an advanced neurophysiological measurement method that serves as a powerful means to study the relationship between behavior and neural mechanisms. In recent years, eye movement tracking has been found to provide a nonverbal and less cognitive method to measure the disease progress of patients with cognitive impairment. Moreover, there is a good correlation between eye movement tracking and the traditional cognitive assessment scale, indicating that eye movement tracking can be used to evaluate and monitor the cognitive status, disease severity, and disease progression of patients with PD. Compared to the traditional cognitive scale, the eye movement detected by the instrument has better objectivity and repeatability. Existing studies have found that executive dysfunction is one of the most important manifestations of cognitive dysfunction in patients with PD and is related to an increase in the error rate of the saccade, an increase in the disinhibition of the delayed saccade task, and a prolongation of the saccade reaction time. This suggests that eye movement measurement plays an important role in the early diagnosis, progression, and differential diagnosis of PD and may even help to predict the disease progression of patients with PD and cognitive impairment. In this article, we review the correlation between cognitive impairment and eye movement disorder in patients with PD.

People with disabilities suffer from inability to communicate with their surroundings, so Human-Computer Interaction (HCI) technologies are used to have a means of communication for people with disabilities with their surroundings. HCI is an emerging technology in the disciplines of Artificial Intelligence and Biomedical Engineering. To power an external device, HCI technology uses several basic signals such as ECG, EMG, and EEG. Electrooculography (EOG) is a technique for measuring the potential difference between the cornea and the retina located between the front and back of the human eye, and the main application of EOG is to determine the directions of different eye movements. This study aims to assess eye movement for communication by persons with disabilities using electrocardiogram (EOG) data. In this study, the Supporting Vector Machine (SVM) and Long- Short term memory (LSTM) classification techniques was used and two types of features (statistical and time domain features) were used. Classification accuracy was 90.7% and 93.9% when using SVM with statistical domain and time domain features, respectively ,whereas Classification accuracy was 90.1% when using LSTM .

This workshop aims at identifying, examining, structuring and sharing educational resources and approaches to support the process of teaching/learning Human-Computer Interaction (HCI) Engineering. The broadening of the range of available interaction technologies and their applications, many times in safety and mission critical areas, to novel and less understood application domains, brings the question of how to address this ever-changing nature in university curricula usually static. Beyond, as these technologies are taught in diverse curricula (ranging from Human Factors and psychology to hardcore computer science), we are interested in what the best approaches and best practices are to integrate HCI Engineering topics in the curricula of programs in software engineering, computer science, human-computer interaction, psychology, design, etc. The workshop is proposed on behalf of the IFIP Working Groups 2.7/13.4 on User Interface Engineering and 13.1 on Education in HCI and HCI Curricula.

Estimating heading during real and simulated eye movements

Color fusion and rivalry are two key information integration mechanisms in binocular vision, representing the visual system's processing patterns for consistent and conflicting inputs, respectively. This study hypothesizes that there are quantifiable differences in eye movement indicators under states of binocular color fusion and rivalry, which can be verified through multi-paradigm eye movement experiments. The experiment recruited eighteen subjects with normal vision (nine males and nine females), employing the Gaze Stability paradigm, Straight Curve Eye Hopping paradigm, and Smoothed Eye Movement Tracking paradigm for eye movement tracking. Each paradigm included a binocular color rivalry experimental group (R-G) and two binocular color fusion control groups (R-R, G-G). Data analysis indicates significant differences in indicators such as Average Saccade Amplitude, Median Saccade Amplitude, and SD of Saccade Amplitude between binocular color fusion and rivalry states. For instance, through Z-Score normalization and cross-paradigm merged analysis, specific ranges of these indicators were identified to distinguish between the two states. When the Average Saccade Amplitude falls within the range of -0.905--0.693, it indicates a state of binocular color rivalry; when the range is 0.608-1.294, it reflects a state of binocular color fusion. Subsequently, ROC curve analysis confirmed the effectiveness of the experimental paradigms in analyzing the mechanisms of binocular color fusion and rivalry, with AUC values of 0.990, 0.741, and 0.967, respectively. These results reveal the potential of eye movement behaviors as biomarkers for the dynamic processing of visual conflicts. This finding provides empirical support for understanding the neural computational models of binocular vision and lays a methodological foundation for developing visual impairment assessment tools based on eye movement features.

The tracking of eye movements is a powerful tool used to understand the process of visual search. Recording and analyzing eye movements allow researchers to better understand how individuals make use of visual information. While eye movement research has been conducted for over 70 years, recent technological breakthroughs have made this equipment accessible to a wider range of researchers. Eye movements can be tracked across paper-based materials, computer screens, or any plane defined in 3-D space. The focus of this presentation is how eye tracking methods can be used in the design of instructional materials, particularly materials that make use of graphics for instruction. Past research on engineering graphics instructional materials presented at ASEE and similar conferences has focused on outcomes such as test scores and satisfaction ratings. Eye tracking data can build on these data sources by providing a deeper understanding of “why” a student may have or have not performed well using a particular set of instructional materials. Background will be given on the basic technologies currently employed and typical measures used to evaluate perceptual and cognitive processes involved with visualization. Next, it will be shown how this information can be used to evaluate instructional materials and provide guidance for redesign. Particular focus will be on the evaluation of graphic elements used in instruction. Finally, examples will be given of how these methodologies have been used to evaluate instructional materials in engineering and preengineering instruction.

The patterns of optic flow seen during self-motion can be used to determine the direction of one's own heading. Tracking eye movements which typically occur during everyday life alter this task since they add further retinal image motion and (predictably) distort the retinal flow pattern. Humans employ both visual and nonvisual (extraretinal) information to solve a heading task in such case. Likewise, it has been shown that neurons in the monkey medial superior temporal area (area MST) use both signals during the processing of self-motion information. In this article we report that neurons in the macaque ventral intraparietal area (area VIP) use visual information derived from the distorted flow patterns to encode heading during (simulated) eye movements. We recorded responses of VIP neurons to simple radial flow fields and to distorted flow fields that simulated self-motion plus eye movements. In 59% of the cases, cell responses compensated for the distortion and kept the same heading selectivity irrespective of different simulated eye movements. In addition, response modulations during real compared with simulated eye movements were smaller, being consistent with reafferent signaling involved in the processing of the visual consequences of eye movements in area VIP. We conclude that the motion selectivities found in area VIP, like those in area MST, provide a way to successfully analyze and use flow fields during self-motion and simultaneous tracking movements.

Eye and hand movement tracking has been proven to be a successful tool and is widely used to figure out characteristics of human cognition in language or visual processing (Just & Carpenter, 1976 Cognitive Psychology8441–480). Eye movement has proven to be a successful measure to figure out characteristics of human language and visual processing (Rayner, 1998 Psychological Bulletin124(3) 372–422). Recently, mouse tracking was used for social-cognition-like categorization of sex-atypical faces and studying spoken-language processes (Magnuson, 2005 PNAS102(28) 9995–9996; Spivey et al., 2005 PNAS102 10393–10398). Here, we present a framework that uses both eye gaze and hand movement simultaneously for analyzing the relation of them during memory retrieval. We tracked eye and mouse movements when the subject was watching a drama and playing a multimodal memory game (MMG), a cognitive task designed to investigate the recall memory mechanisms in watching video dramas (Zhang, 2009 AAAI 2009 Spring Symposium: Agents that Learn from Human Teachers 144–149). Experimental results show that eye tracking and mouse tracking provide complementary information about underlying cognitive processes. Also, we found some interesting patterns in eye-hand movement during multimodal memory recall.

Predictive eye movements do not discriminate between dyslexic and control children

With the development of social life and improvement of computer science and technology, human-computer interaction(HCI) technology has been paid more and more attentions and become an important aspect of research field. Gestures, as a natural ways of communication between people, contain rich information. So gesture recognition has become a branch of human-computer interaction, which has gradually become a hot research topic. This paper presents a method for gesture recognition based on data glove. Sparse decomposition is firstly conducted on the raw data, then a support vector machine(SVM) classifier is applied on the dictionary part to achieve recognition results. The experimental results reveal that our method can achieve very high recognition accuracy on the data glove gestures.

<title>Abstract</title> Meta Quest Pro-based eye and facial tracking technology plays an essential role in studies related to visual attention, cognitive processes, user interactions, and human behavior analysis. However, differences in eye anatomy, such as outer canthal distance (OCD), interpupillary distance (IPD), and intercanthal distance (ICD), may impact the accuracy &amp; reliability of eye movement data. This study aims to evaluate the impact of inter-eye distance variations on eye-movement-based feature extraction in VR environments using the head-mounted device Meta Quest Pro. The data collection process involved eye measurements (OCD, IPD, and ICD), an eye calibration test before the experiment, and three experimental tasks: fixed gaze task, regular eye movement task, and irregular eye movement task. The Meta Quest Pro recorded participants’ eye movements while they performed VR-based tasks, followed by the extraction of eye-related features from the device. The dataset underwent Shapiro–Wilk, and D’Agostino–Pearson normality tests, followed by statistical analysis (Spearman Correlation, Kruskal–Wallis, and Mann–Whitney U test) to assess differences in eye-movement correlations across wide, narrow, and average eye distance groups. The results of statistical tests did not indicate a significant influence of inter-eye distance on eye movement tracking accuracy. This study is important for research, including learning disabilities research, child and adult behavior analysis, and human–computer interaction, ensuring that eye data remains accurate and reliable across diverse users.

Which text presentation format is best to use on a mobile phone? We have evaluated Scrolling, Paging, Leading, and Rapid Serial Visual Presentation (RSVP) in a balanced repeated-measurement study employing 16 subjects on a mobile phone. Besides monitoring reading speed, comprehension, and task load, we also tracked eye movements while reading. The results show that text presented in the Page format is read significantly faster than both the Scrolling and the RSVP format without any significant differences in comprehension. The Page format was moreover significantly less demanding to use compared to Leading and RSVP for most factors according to NASA-TLX. RSVP was found to significantly decrease eye movements whereas Leading was found to increase them. Leading yielded very irregular eye movements, but this did not affect reading speed. These results show that Paging offers best readability on a mobile phone, an interesting finding given that Scrolling is the format predominantly used today.

With the rise of virtual reality technology and eye movement tracking, the cognitive classification of eye movement emotion combined with virtual reality technology was a research hotspot recently. In this research, 96 pictures were selected from the Chinese Affective Picture System (CAPS), and 35 subjects were stimulated with pictures in virtual scene. The corresponding eye movement data were tracked synchronously. Eye movement data were progressed with blink data reconstruction and baseline correction, after which time and frequency domain features were extracted. Finally, for three emotion model dimensions including valence, arousal and dominance expressed as high, medium and low, models are trained with Support Vector Machine (SVM), with the highest classification accuracy of single person reaches 78.1%. The results of this paper are instructive for the processing of eye movement data, and the eye tracking data combined with virtual reality technology can be used for emotion recognition research through the proposed preprocessing methods and feature extraction.

Three-dimensional geovisualization in landscape design can be used to evaluate the efforts of mitigating CO2 emissions. This study evaluated subjects’ emotional preferences for 3D landscape design through an eye movement tracking experiment. In the case that the color of the building materials was positively correlated with low carbon emissions, green, blue, and gray were typical representatives of low carbon emissions. Through the eye movement tracking experiment, subjects’ emotional preferences for different building colors were obtained. The results show that the fixation trajectory is consistent with the preset green and energy saving parameters, and the design effect of the architectural landscape can be evaluated by detecting virtual eye movement tracking. There is a coupling relationship between virtual eye movement tracking, expert interviews, and evaluation results, so that it presents a logical relationship between virtual eye movement, the color of low-carbon materials, and carbon emissions. In addition, the affective preference analysis and entropy weight method confirmed their effectiveness in the evaluation of the 3D landscape design effect, which had a positive impact on the CO2 emission reduction of the construction industry. These results will contribute to the development of 3D landscape design in the architecture industry and provide new ideas and methods for the carbon peak project.

Eye-gaze detection and tracking have been an active research field in the past years as it adds convenience to a variety of applications. It is considered a significant untraditional method of human computer interaction. Head movement detection has also received researchers' attention and interest as it has been found to be a simple and effective interaction method. Both technologies are considered the easiest alternative interface methods. They serve a wide range of severely disabled people who are left with minimal motor abilities. For both eye tracking and head movement detection, several different approaches have been proposed and used to implement different algorithms for these technologies. Despite the amount of research done on both technologies, researchers are still trying to find robust methods to use effectively in various applications. This paper presents a state-of-art survey for eye tracking and head movement detection methods proposed in the literature. Examples of different fields of applications for both technologies, such as human-computer interaction, driving assistance systems, and assistive technologies are also investigated.