Visual selectivity for heading in the macaque ventral intraparietal area.

Estimating heading during real and simulated eye movements

Modeling and simulation of human eye movement have a wide range of applications in many domains. Various attempts have been made to model and simulate eye movements in a physically accurate manner. All the existing models show limitations and problems in simulating secondary and tertiary eye movements. Recent investigation of pulley models (passive and active hypotheses) in representing human eye motion has recognized mathematical complexity in modeling eye behavior. Sophisticated techniques of modeling are required to investigate eye movements. This paper presents a procedure for eye movement simulation through geometrical modeling (an OpenSim script with its recent MATLAB binding) for binocular vision. First order neural dynamics with Millard's muscle model are used to actuate six Extra Ocular Muscles (EOMs). Pulse-step inputs are used to generate the muscle forces around the eye globe. The implemented model is successful in simulating horizontal and vertical movements of the human eye with respect to the prescribed activation. The developed technique is evaluated using responses from lumped parameter models and EOG recordings.

Terrestrial navigation naturally involves translations within the horizontal plane and eye rotations about a vertical (yaw) axis to track and fixate targets of interest. Neurons in the macaque ventral intraparietal (VIP) area are known to represent heading (the direction of self-translation) from optic flow in a manner that is tolerant to rotational visual cues generated during pursuit eye movements. Previous studies have also reported that eye rotations modulate the response gain of heading tuning curves in VIP neurons. We tested the hypothesis that VIP neurons simultaneously represent both heading and horizontal (yaw) eye rotation velocity by measuring heading tuning curves for a range of rotational velocities of either real or simulated eye movements. Three findings support the hypothesis of a joint representation. First, we show that rotation velocity selectivity based on gain modulations of visual heading tuning is similar to that measured during pure rotations. Second, gain modulations of heading tuning are similar for self-generated eye rotations and visually simulated rotations, indicating that the representation of rotation velocity in VIP is multimodal, driven by both visual and extraretinal signals. Third, we show that roughly one-half of VIP neurons jointly represent heading and rotation velocity in a multiplicatively separable manner. These results provide the first evidence, to our knowledge, for a joint representation of translation direction and rotation velocity in parietal cortex and show that rotation velocity can be represented based on visual cues, even in the absence of efference copy signals.

The question investigated was whether or not eye movements accompanied by abnormal retinal image movements, movements that are either or both at a different rate or in a different direction than the eye movement, predictably lead to perceived movement. Os reported whether or not they saw a visual target move when the movement of the target was either dependent on and simultaneous with their eye movements or when the target movement was independent of their eye movements. In the main experiment, observations were made when the ratio between eye and target movement fem/tm) was 2/5, 1/5, 1/10, 1/20, and 0. All these ratios were tested when the direction of the target movement was in the same (H+), opposite (H−), and at right angles to (V+, V−) the movement of the eyes. Eye movements, target movements, and reports of target movement were recorded. Results indicate that a discrepancy between eye and target movement greater than 20% predictably leads to perceived target movement, whereas a discrepancy of 5% or less rarely leads to perceived movement. The results are interpreted as support for the operation of a compensatory mechanism during eye movements.

When organisms move forward or backward through their environment, they produce particular patterns of optic flow on their retina, i.e., the images of objects move in characteristic ways away from or towards a focus of expansion or contraction (Gibson, 1950, 1954). For straight motion without eye or head movements, the flow pattern is radial with a speed gradient and complete symmetry (Gibson, 1950). It is then straightforward to compute the heading direction, calculate the time to collision with objects or, indeed, take action to avoid collisions (van den Berg, 2000; Warren, 1998; Warren et al., 1988, 1991). The situation becomes more difficult when eye movements (e.g., pursuit movements) are made simultaneously, which is the rule and not the exception. Fixating an object that moves from left to right would create a lateral flow on the retina, giving a rightward motion to the images of all other objects. We call this a parallel flow to express the fact that all motion vectors would be parallel. This motion would be added to the appropriate radial motion for each image point, resulting in a complex flow pattern. This complex flow still contains the original radial flow that is needed for the extraction of heading direction. But the complex pattern would have to be decomposed to yield this information.

1. Purkinje cell activity was recorded from lobules VI and VII of the cerebellar vermis during the performance of visuooculomotor tasks designed to dissociate the signals related to head, smooth-pursuit eye, and retinal image movements. Task-related modulations in the simple spike discharge rates of 157 cells were observed in three alert monkeys. 2. Of 65 Purkinje cells that were completely tested for all three signals, all exhibited smooth-pursuit eye movement-related activity. An additional vestibular or visual response was observed in 17 and 11% of the cells, respectively. Eye, head, and retinal image velocity signals were all recorded in the same unit in 52% of the Purkinje cells. The responses of 5% of the fully tested cells were associated with changes in the direction of eye, head, and retinal image movement. 3. The observed sensorioculomotor responses were direction selective in 98% of the Purkinje cells. For the Purkinje cells that were fully tested, 60% of the cells exhibited peak discharge rates for ipsilateral and 40% for contralateral eye velocity. Of these Purkinje cells, 45% exhibited eye, head, and retinal image velocity signals with equivalent direction preferences. 4. Of 42 Purkinje cells tested, 88% demonstrated some kinds of interactive responses during combined eye and sensory stimulation. The interaction of eye and head velocity signals has been discussed in a companion paper (38). The modulation in discharge rate observed during tracking in the presence of a random dot background pattern could be predicted from the dissociated responses to smooth pursuit in the dark and to movements of the background pattern during suppression of eye movements. 5. The sensitivity to smooth-pursuit eye velocity averaged 1.4 times the sensitivity to head velocity. In 80% of the Purkinje cells, however, the sensitivity to eye velocity exceeded the sensitivity to head velocity by an average of only 10%. The sensitivity to smooth-pursuit eye velocity averaged 1.6 times the sensitivity to retinal image velocity. 6. An increase in Purkinje cell discharge rate was observed during the open-loop period of the initiation of smooth-pursuit eye movements. This open-loop response was consistent with the presence of a visual signal during ocular pursuit, since these cells were also shown to be responsive to a dissociated retinal image velocity signal. Furthermore, the magnitude of the open-loop response indicated an enhancement of the sensitivity to retinal image velocity when visual information became behaviorally significant.(ABSTRACT TRUNCATED AT 400 WORDS)

To solve the problem of user visual attention distribution and spatial misalignment of interface elements in current user interface layout design due to reliance on subjective experience, this paper proposes an optimization method for fashion brand e-commerce user interface design that uses eye tracking simulation and integrates brand characteristics. This paper builds a fashion-specific visual behavior database through multimodal eye movement data collection, and uses the spatiotemporal attention mechanism of the Transformer-XL (Transformer with Extra Long Context) model to predict the gaze hotspots and scanning paths of users in dynamic tasks; then, this paper designs a multi-agent reinforcement learning optimizer, encodes aesthetic rules such as brand logo size and main color ratio as hard constraints, and generates an interface space distribution plan through element competition-collaboration game; finally, this paper develops a real-time interactive prototype based on the Unity engine to achieve dynamic layout closed-loop optimization driven by the collaborative efforts of brand constraints and eye movement simulation heat maps. Experimental results show that this method reduces the task completion time by 32% in the optimization of the light luxury clothing homepage, the overlap between the simulated and real eye movement hot spots reaches 88%, and the brand consistency score increases by 46% (from 3.2 to 4.7), significantly reducing the user's cognitive load and improving the user experience. The conclusion confirms that by integrating eye movement behavior quantification with brand characteristics, it is possible to break through the traditional design's reliance on static aesthetics, provide a “cognitive adaptation-brand expression” dual-goal collaborative intelligent design paradigm for fashion interfaces, and promote the simultaneous improvement of user conversion rate and brand value.

Many studies have documented the involvement of medial superior temporal extrastriate area (MST) in the perception of heading based on optic flow information. Furthermore, both heading perception and the responses of MST neurons are relatively stable in the presence of eye movements that distort the retinal flow information on which perception is based. Area VIP in the posterior parietal cortex also contains a robust representation of optic flow cues for heading. However, the studies in the two areas were frequently conducted using different stimuli, making quantitative comparison difficult. To remedy this, we studied MST using a family of random dot heading stimuli that we have previously used in the study of VIP. These stimuli simulate observer translation through a three-dimensional cloud of points, and a range of forward headings was presented both with and without horizontal smooth pursuit eye movements. We found that MST neurons, like VIP neurons, respond robustly to these stimuli and partially compensate for the presence of pursuit. Quantitative comparison of the responses revealed no substantial difference between the heading responses of MST and VIP neurons or in their degree of pursuit tolerance.

Cognitive dysfunction is the most common and important nonmotor symptom in Parkinson’s disease (PD) and can occur at any stage. However, there is still a lack of effective biomarkers to evaluate the decline in cognitive function and predict the progression of the disease, especially in the early stage. At present, the cognitive scale is widely used to evaluate the cognitive function of patients with PD, but its sensitivity and accuracy are relatively limited, especially in the early identification of mild cognitive impairment. Eye movement tracking is an advanced neurophysiological measurement method that serves as a powerful means to study the relationship between behavior and neural mechanisms. In recent years, eye movement tracking has been found to provide a nonverbal and less cognitive method to measure the disease progress of patients with cognitive impairment. Moreover, there is a good correlation between eye movement tracking and the traditional cognitive assessment scale, indicating that eye movement tracking can be used to evaluate and monitor the cognitive status, disease severity, and disease progression of patients with PD. Compared to the traditional cognitive scale, the eye movement detected by the instrument has better objectivity and repeatability. Existing studies have found that executive dysfunction is one of the most important manifestations of cognitive dysfunction in patients with PD and is related to an increase in the error rate of the saccade, an increase in the disinhibition of the delayed saccade task, and a prolongation of the saccade reaction time. This suggests that eye movement measurement plays an important role in the early diagnosis, progression, and differential diagnosis of PD and may even help to predict the disease progression of patients with PD and cognitive impairment. In this article, we review the correlation between cognitive impairment and eye movement disorder in patients with PD.

As we move forward in the environment, we experience a radial expansion of the retinal image, wherein the center corresponds to the instantaneous direction of self-motion. Humans can precisely perceive their heading direction even when the retinal motion is distorted by gaze shifts due to eye/body rotations. Previous studies have suggested that both retinal and extra-retinal strategies can compensate for the retinal image distortion. However, the relative contributions of each strategy remain unclear. To address this issue, we devised a two-alternative-headings discrimination task, in which participants had either real or simulated pursuit eye movements. The two conditions had the same retinal input but either with or without extra-retinal eye movement signals. Thus, the behavioral difference between conditions served as a metric of extra-retinal contribution. We systematically and independently manipulated pursuit speed, heading speed, and the reliability of retinal signals. We found that the levels of extra-retinal contributions increased with increasing pursuit speed (stronger extra-retinal signal), and with decreasing heading speed (weaker retinal signal). In addition, extra-retinal contributions also increased as we corrupted retinal signals with noise. Our results revealed that the relative magnitude of retinal and extra-retinal contributions was not fixed but rather flexibly adjusted to each specific task condition. This task-dependent, flexible integration appears to take the form of a reliability-based weighting scheme that maximizes heading performance.

Validating and improving the correction of ocular artifacts in electro-encephalography

Background: Scanning laser ophthalmoscopes with adaptive optics correction of ocular aberrations provide retinal images of unprecedented resolution, allowing for real-time imaging of photoreceptors. Eye movements made by the subject/patient during recording produce distortions that must be corrected before multiple frames can be added together to achieve noise reduction or to build a mosaic image from different retinal areas. These distortions also provide a high spatial and temporal resolution record of the miniature eye movements made during fixation. Here we report simultaneous measurements of fixation eye movements with an Adaptive Optics Scanning Laser Ophthalmoscope (AOSLO) and a dual-Purkinje image (dPi) eye tracker in order to cross-validate these two methods of recording miniature eye movement. Method: Foveas of three subjects were imaged with a one degree square scan using the Houston AOSLO, at a resolution of 8 pixels per arc minute. A Generation V dPi tracker from SRI was placed in front of the AOSLO, and eye movements were recorded at the same time from the same eye being imaged by the AOSLO. AOSLO movies were analyzed off line to extract retinal image motion. The resulting traces were then overlaid on the dPi recordings for comparison. Results: The two methods produced records that agreed to within about one arc minute, with more significant disagreements occurring after eye blinks. Microsaccades in the dPi record were accompanied by overshoots that have previously been associated with lens wobble. AOSLO traces also showed saccade-related overshoots, but of much smaller amplitude. Conclusions: Eye movement recordings measured with dual Purkinje image trackers predict retinal image motion to a precision of about 1 arc minute, except for 10–20 milliseconds following each saccade and 500 – 1000 milliseconds following each eye blink. Retinal image motion measured directly from AOSLO recordings can be recovered to a precision of just a few arc seconds.

Eyes are the window of the mind, and eye movement contains a lot of effective information. With the breakthrough of artificial intelligence and human-computer interaction, eye movement tracking technology has gradually moved to the foreground in recent years. This paper takes medical health as a typical scenario, carries out innovative research around the potential application of eye movement tracking technology in patient communication and human-computer interaction, proposes and implements an intention estimation system based on eye movement tracking. Based on the physiology, medicine and ergonomics of eye movement, the system makes original innovations, extracts a set of eye movement parameters with high recognition accuracy and high expression efficiency, summarizes eye movement instructions that can meet the application of silent environment, and finally forms a series of embryonic applications related to interactive instructions.

Ocular responses to radial optic flow and single accelerated targets in humans

The tracking of eye movements is a powerful tool used to understand the process of visual search. Recording and analyzing eye movements allow researchers to better understand how individuals make use of visual information. While eye movement research has been conducted for over 70 years, recent technological breakthroughs have made this equipment accessible to a wider range of researchers. Eye movements can be tracked across paper-based materials, computer screens, or any plane defined in 3-D space. The focus of this presentation is how eye tracking methods can be used in the design of instructional materials, particularly materials that make use of graphics for instruction. Past research on engineering graphics instructional materials presented at ASEE and similar conferences has focused on outcomes such as test scores and satisfaction ratings. Eye tracking data can build on these data sources by providing a deeper understanding of “why” a student may have or have not performed well using a particular set of instructional materials. Background will be given on the basic technologies currently employed and typical measures used to evaluate perceptual and cognitive processes involved with visualization. Next, it will be shown how this information can be used to evaluate instructional materials and provide guidance for redesign. Particular focus will be on the evaluation of graphic elements used in instruction. Finally, examples will be given of how these methodologies have been used to evaluate instructional materials in engineering and preengineering instruction.