Multivariate Analysis of Gaze Behavior and Task Performance Within Interface Design Evaluation

Cognitive tunneling occurs when the pilot's attention becomes locked on non-conformal, superimposed head-up display (HUD) symbology, while neglecting to scan the out-the-window scene, as a result of locating the HUD symbology near (in visual angle) the outside scene information (Foyle, McCann, Sanford & Schwirzke, 1993). Previous studies have shown that cognitive tunneling could be eliminated by placing the HUD symbology at least 8 deg from the out-the-window path being tracked. Limitations to previous research have included experimental designs that tested participants in multiple HUD information locations without fostering an efficient eyescan strategy for any one HUD location. Experiment 1 dedicates a participant to a single HUD location with blocked presentation. The results indicate that cognitive tunneling is not only eliminated by placing HUD symbology greater than 8 deg, but path tracking performance improves with symbology placed in an upper location on the HUD. Experiment 2 shows that the resulting performance decrement when information is overlaying the path (0 deg) may be associated with symbology compellingness, regardless of symbology relevance to the task.

Since autonomous vehicles (AV) are in the testing process, it is an open question of how pedestrians will communicate with self-driving cars. Nowadays, explicit communication pattern is the main way of pedestrian-driver interaction, however, AV may use implicit communication when making crossing decisions. This study aims to analyze pedestrians' gaze behavior when crossing the road using an eye camera and find the most applicable location for the LED interface on AVs. 10 pedestrian crossings in Gyor, Hungary were analyzed using the synchronized eye-tracking (ET) technology and regular video cameras for combined data processing. The data were analyzed using digital image processing techniques and statistical methods to identify where pedestrians looked and whether a pedestrian-driver interaction was captured during the crossing maneuver.

: This technical report presents the results of the arc segment attitude reference (ASAR) head-up display (HUD) symbology as a primary flight reference test and evaluation. Testing was requested by the Air Force Research Laboratory, Human Effectiveness Directorate, Wright-Patterson AFB, Ohio. The responsible test organization was the 412th Test Wing, Edwards AFB, California. The test execution organization was the ASAR HUD test team from the USAF Test Pilot School located at Edwards AFB. Testing was performed on the USAF NF-16D Variable-Stability In-Flight Simulator Test Aircraft serial number 86-0048, from 7 to 23 July 2004 and was comprised of 5 ground simulation familiarization and test sessions totaling 12.0 ground test hours, 2 calibration flights and 15 test sorties totaling 23.8 flight test hours. The overall test objective was to determine if the use of the ASAR in the MIL-STD- 1787C HUD symbology as the principal attitude reference, in place of the climb-dive ladder, improved pilot performance during unusual attitude recoveries, vertical banked S-turn (S-B) and vertical dive S-turn (S-D) maneuvers, and precision ILS approaches. The vertical S-B maneuver was a continuous series of rate climbs and descents flown on an initial constant heading. The vertical S-D maneuver was similar except that the direction of turn was reversed simultaneously with each change of vertical direction. The overall test objective was met.

Head-up displays (HUDs) enable a pilot to manage aircraft activities by facilitating simultaneous access to the flight instrument data and to the outside scene. However, HUDs can also distract a pilot. This study shows that HUD luminance non-uniformity may force inappropriate distribution of attention between the events shown on HUD symbology and the outside scene because of the resultant differential contrast in the display area. Results of statistical analysis demonstrate considerable effects of HUD image luminance and ambient luminance, as well as their interaction, on the detection of events displayed on an HUD and the outside scene.

Eye-tracking technology can assist researchers in understanding motivational decision-making and choice processes by analysing consumers’ gaze behaviour. Previous studies showed that attention is related to decision, as the preferred stimulus is generally the most observed and the last visited before a decision is made. In this work, the relationship between gaze behaviour and decision-making was explored using eye-tracking technology. Images of six wardrobes incorporating different sustainable design strategies were presented to 57 subjects, who were tasked with selecting the wardrobe they intended to keep the longest. The amount of time spent looking was higher when it was the chosen version. Detailed analyses of gaze plots and heat maps derived from eye-tracking records were employed to identify different patterns of gaze behaviour during the selection process. These patterns included alternating attention between a few versions or comparing them against a reference, allowing the identification of stimuli that initially piqued interest but were ultimately not chosen, as well as potential doubts in the decision-making process. These findings suggest that doubts that arise before making a selection warrant further investigation. By identifying stimuli that attract attention but are not chosen, this study provides valuable insights into consumer behaviour and decision-making processes.

Head-up displays (HUDs) have become an integral part of the fighter aircraft and are now becoming increasingly popular in various nonmilitary sectors also. It enhances the pilot's performance by providing him with all of the flight, aircraft, target, and weapon data in collimated fashion superimposed on his forward view. The collimation of HUD symbology and infrared raster image removes requirement of refocusing eyes on outside events. However, there are attention capture and cognitive tunneling issues related with usage of HUD in aircraft. In this paper, effect of obstruction due to beam combiner frame in form of misaccommodation, misconvergence, and limited horizontal field of view (FOV) on attention capture of aircraft pilot has been presented. While the beam combiner frame is necessary to hold wavelength selective glasses, they provide obscuration in forward view of pilot in both, total field of view (TFOV) as well as instantaneous field of view (IFOV). Angle of combiner frame structure and its width present different degrees of obscuration to the pilot within the head motion box (HMB). These limitations cause inappropriate distribution of pilot's attention on outside events and aircraft events, as he has to adjust his head position to view the obscured part of outside world. This forces him to focus his attention more on outside events which may make him miss some event on HUD symbology. The study and experimental results have been presented in detail to corroborate this fact.

In a preliminary exploratory study, our goal was to train deep neural network models to mimic children's and/or adults' gaze behavior in certain social situations to reach this objective. Additionally, we aim to identify potential differences in gaze behavior between these two age groups based on our participants' gaze data. Furthermore, we aimed to assess the practical effectiveness of our adult and children models by deploying them on a Nao robot in real-life settings. To achieve this, we first created two video clips, one animation and one live-action, to depict some social situations. Using an eye-tracking device, we collected eye-tracking data from 24 participants, including 12 children and 12 adults. Then, we utilized deep neural networks, specifically LSTM and Transformer Networks, to analyze and model the gaze patterns of each group of participants. Our results indicate that when the models attempted to predict people's locations (in the next frame), they had an accuracy in the range of 62%-70% with one attempt, which increased by ~20% when attempted twice (i.e. the two highest-ranked predicted labels as outputs). As expected, the result underscores that gaze behavior is not a wholly unique phenomenon. We obtained feedback from 57 new participants to evaluate the robot's functionality. These participants were asked to watch two videos of the robot's performance in each mode and then complete a comprehensive questionnaire. The questionnaire results indicate that the participants expressed satisfaction with the robot's interaction, including its attention, intelligence, and responsiveness to human actions. However, they did not perceive the robot as a social companion comparable to a human. This exploratory study tries to address/show potentials of the social acceptance of robots based on human nonverbal behavioral cues for future research.

Combining virtual reality and mobile eye tracking to provide a naturalistic experimental environment for shopper research

Flying an aircraft in low visibility is still a challenging task for the pilot. It requires precise and accurate situational awareness (SA) in real-time. A Head-up Display (HUD) is used to project collimated internal and external flight information on a transparent screen in the pilot’s forward field of view, which eliminates the change of eye position between Head-Down-Display (HDD) instruments and outer view through the windshield. Implementation of HUD increases the SA and reduces the workload for the pilot. But to provide a better flying capability for the pilot, projecting extensive information on HUD causes human factor issues that reduce pilot performance and lead to accidents in low visibility conditions. The literature shows that human error is the leading cause of more than 70% of aviation accidents. In this study, the ability of the pilot able to read background and symbology information of HUD at a different level of background seen complexity, such as symbology brightness, transition time, amount of Symbology, size etc., in low visibility conditions is discussed. The result shows that increased complexity on the HUD causes more detection errors.

Eye tracking technology has become increasingly important for psychological analysis, medical diagnosis, driver assistance systems, and many other applications. Various gaze-tracking models have been established by previous researchers. However, there is currently no near-eye display system with accurate gaze-tracking performance and a convenient user experience. In this paper, we constructed a complete prototype of the mobile gaze-tracking system ‘Etracker’ with a near-eye viewing device for human gaze tracking. We proposed a combined gaze-tracking algorithm. In this algorithm, the convolutional neural network is used to remove blinking images and predict coarse gaze position, and then a geometric model is defined for accurate human gaze tracking. Moreover, we proposed using the mean value of gazes to resolve pupil center changes caused by nystagmus in calibration algorithms, so that an individual user only needs to calibrate it the first time, which makes our system more convenient. The experiments on gaze data from 26 participants show that the eye center detection accuracy is 98% and Etracker can provide an average gaze accuracy of 0.53° at a rate of 30–60 Hz.

The take-off and landing of an aircraft is a complex and challenging task for the pilot. Most accidents occur in this phase of flight while flying duration is only 4% of total flying time. Implementation of Head-up displays (HUDs) in the cockpit enhances the pilot’s SA and reduces workload. But presenting large amounts of data on HUD with different display attributes degrades pilot performance, leading to an accident in high workload conditions. This study examines the effect of the sudden change in Symbology and presents large data at a time on pilot performance. It also discussed a model to reduce human errors during take-off and landing modes of operation in low visibility conditions. The work discussed in this manuscript is novel as no experiment has been conducted to observe the human factor’s relation to operational modes and display attributes on see-through displays under low visibility conditions.

In the realm of virtual reality (VR) research, the synergy of methodological advancements, technical innovation, and novel applications is paramount. Our work encapsulates these facets in the context of spatial ability assessments conducted within a VR environment. This paper presents a comprehensive and integrated framework of VR, eye-tracking, and electroencephalography (EEG), which seamlessly combines measuring participants’ behavioral performance and simultaneously collecting time-stamped eye tracking and EEG data to enable understanding how spatial ability is impacted in certain conditions and if such conditions demand increased attention and mental allocation. This framework encompasses the measurement of participants’ gaze pattern (e.g., fixation and saccades), EEG data (e.g., Alpha, Beta, Gamma, and Theta wave patterns), and psychometric and behavioral test performance. On the technical front, we utilized the Unity 3D game engine as the core for running our spatial ability tasks by simulating altered conditions of space exploration. We simulated two types of space exploration conditions: (1) microgravity condition in which participants’ idiotropic (body) axis is in statically and dynamically misaligned with their visual axis; and (2) conditions of Martian terrain that offers a visual frame of reference (FOR) but with limited and unfamiliar landmarks objects. We specifically targeted assessing human spatial ability and spatial perception. To assess spatial ability, we digitalized behavioral tests of Purdue Spatial Visualization Test: Rotations (PSVT: R), the Mental Cutting Test (MCT), and the Perspective Taking Ability (PTA) test and integrated them into the VR settings to evaluate participants’ spatial visualization, spatial relations, and spatial orientation ability, respectively. For spatial perception, we applied digitalized versions of size and distance perception tests to measure participants’ subjective perception of size and distance. A suite of C# scripts orchestrated the VR experience, enabling real-time data collection and synchronization. This technical innovation includes the integration of data streams from diverse sources, such as VIVE controllers, eye-tracking devices, and EEG hardware, to ensure a cohesive and comprehensive dataset. A pivotal challenge in our research was synchronizing data from EEG, eye tracking, and VR tasks to facilitate comprehensive analysis. To address this challenge, we employed the Unity interface of the OpenSync library, a tool designed to unify disparate data sources in the fields of psychology and neuroscience. This approach ensures that all collected measures share a common time reference, enabling meaningful analysis of participant performance, gaze behavior, and EEG activity. The Unity-based system seamlessly incorporates task parameters, participant data, and VIVE controller inputs, providing a versatile platform for conducting assessments in diverse domains. Finally, we were able to collect synchronized measurements of participants’ scores on the behavioral tests of spatial ability and spatial perception, their gaze data and EEG data. In this paper, we present the whole process of combining the eye-tracking and EEG workflows into the VR settings and collecting relevant measurements. We believe that our work not only advances the state-of-the-art in spatial ability assessments but also underscores the potential of virtual reality as a versatile tool in cognitive research, therapy, and rehabilitation.

IntroductionChildren with Autism Spectrum Disorder (ASD) often find it difficult to maintain eye contact, which is vital for social communication. Eye tracking (ET) technology helps determine how long children with ASD focus on someone, how frequently they do so, and in which direction their gaze moves. ET provides insights into social attention by enabling precise, real-time tracking of gaze patterns as individuals process social information visually. It is a dependable method for identifying and developing social attentional biomarkers, particularly in challenging conditions like ASD.ObjectiveThis study aims to implement deep learning (DL) algorithms using eye-tracking data from social attention tasks involving children with ASD.MethodsThe approach was tested using standard datasets collected from individuals with and without ASD through eye-tracking technology. Convolutional neural networks (CNNs) and long short-term memory (LSTM) models were used to analyze data from children with ASD. Data preprocessing techniques addressed missing data and converted categorical features into numerical values. Mutual information-based feature selection was employed to reduce the feature set by identifying the most relevant features, thereby improving system performance. These features were then analyzed using LSTM and CNN-LSTM models to evaluate their potential for diagnosing ASD.ResultsThe experimental results showed that the highest accuracy achieved was 99.78% with the CNN-LSTM model. Furthermore, the findings indicated that the proposed method outperformed previous studies.ConclusionThe system successfully diagnosed ASD using the ET dataset. This approach shows promise for clinical application, assisting healthcare professionals in diagnosing ASD more accurately through advanced artificial intelligence technology.

A comparison of selected simple supervised learning algorithms to predict driver intent based on gaze data

Eye-tracking application in social sciences including entrepreneurship education has increased significantly in the recent years. This technology has been used to investigate the learning process and how to foster it through instructions delivered, material used and the learning environment created. Traditional research with eye-tracking application mainly concentrates on visual aspects in the learning process including but not limited to text comprehension. A growing area of eye-tracking technologies is focused on entrepreneurship education including teacher education because schools are considered as an important stage for developing entrepreneurial competences.In general, the area of the application of eye tracking has become extremely wide in different sciences which also positively contributes to research in education. Transdisciplinary and multidisciplinary approaches are helpful to ensure multiple perspective as well as to ensure the validity of research data and results.This chapter is an attempt to critically reflect on how eye-tracking methodology is applied for research on entrepreneurship education and what are growing methodological challenges in it. At the end some implications for further studies in the field of entrepreneurship education are discussed as well as limitations of eye-tracking-based studies are highlighted.