An Analysis Scheme to Interpret Students' Cognitive Process in Error Finding Test

Error finding is a widely-used kind of test to assess the students' ability in program comprehension and debugging. Recently some eye-tracking based approaches have been proposed to analyze students' cognitive process in programming, but few of them focused on the performance assessment in programming courses. In our previous work, an eye-tracking assisted framework was proposed to assess the overall performance of one student in a web-based error-find test. Aiming to provide individual diagnostics, we extend that work in this paper, and study 15 students' performance in different phases of the online test. For each student, his performance is analyzed in two phases, i.e. the code overview phase and the subsequent phase. In the first phase, the behavior of first-time code browsing and first-time error finding is studied and compared. We find that the performance of first error-finding action has obvious correlation with the final test score. In the subsequent phase, the students' performances are examined in activity sequence separated by the click events. We propose a new metric, namely matched ratio of program execution, to describe the proportion of gaze path matching with the program execution. Data analysis results show that, it is more helpful in interpreting students' behavior and describing the students' working efficiency in the test.

IntroductionThe use of eye tracking (ET) in mathematics education research has increased in recent years. Eye tracking is a promising research tool in the domain of functions, especially in graph interpretation. It promises to gain insights into learners’ approaches and ways of thinking. However, for the domain of functions and graph interpretation, it has not yet been investigated how eye-tracking data can be interpreted. In particular, it is not clear how eye movements may reflect students’ cognitive processes. Thus, in this study, we investigate in how far the eye-mind hypothesis (EMH), which states broadly that what the eye fixates is currently being processed, can be applied to this subdomain. This is particularly true for contextual graphs, whose data originate from real-world situations, and which are of central importance for the development of mathematical literacy. The aim of our research is to investigate how eye movements can be interpreted in the domain of functions, particularly in students’ interpretations of contextual graphs.MethodsWe conducted an exploratory case study with two university students: The students’ eye movements were recorded while they worked on graph interpretation tasks in three situational contexts at different question levels. Additionally, we conducted subsequent stimulated recall interviews (SRIs), in which the students recalled and reported their original thoughts while interpreting the graphs.ResultsWe found that the students’ eye movements were often related to students’ cognitive processes, even if indirectly at times, and there was only limited ambiguity in the interpretation of eye movements. However, we also found domain-specific as well as domain-general challenges in interpreting eye movements.DiscussionOur results suggest that ET has a high potential to gain insights into students’ graph interpretation processes. Furthermore, they point out what aspects, such as ambiguity and peripheral vision, need to be taken into consideration when investigating eye movements in the domain of functions.

Dysphagia frequently co-occurs with cognitive impairment, suggesting that swallowing may involve cognitive processes. However, the relationship between cognitive load and swallowing performance remains unclear. This study aimed to examine the effects of different levels of cognitive load on swallowing performance under dual-task conditions in healthy adults. Forty-three graduate students (mean age: 24.49 ± 1.63 years, male/female: 22/21) were recruited and performed five tasks: swallowing single task (SST), low-load cognitive single task (STlow), high-load cognitive single task (SThigh), low-load cognitive-swallowing dual task (DTlow), and high-load cognitive-swallowing dual task (DThigh). Here, a dual-task refers to the concurrent execution of a swallowing task and a cognitive task, designed to evaluate the competition for attentional resources and performance trade-offs. The low-load cognitive task consisted of a digit forward-span task, while the high-load cognitive task involved a digit backward-span task, which demands greater cognitive processing complexity and working memory load. Swallowing measures (e.g., swallowing volume) and cognitive task accuracy were recorded, and dual-task cost (DTC) was calculated to evaluate performance changes under dual-task conditions. Swallowing volume differed significantly among SST, DTlow, and DThigh (p < 0.05), with lower volume in DThigh compared to SST (p < 0.05). Cognitive accuracy was significantly lower in DThigh than in DTlow (p < 0.05), indicating greater interference under high cognitive load. The findings confirm that swallowing engages executive function and attentional resources, and suggest that participants prioritize swallowing performance during dual-task conditions by adjusting prefrontal activation patterns rather than increasing overall cortical activation to manage interference. This study provides preliminary evidence for the cognitive involvement in swallowing and highlights the need to consider cognitive load in dysphagia assessment and rehabilitation.

Peripheral target identification performance modulates eye movements

Reading is a complex cognitive process. Through eye tracking techniques the cognitive features can be observed in depth, and therefore a better grasp of the instant cognitive processing of language can be achieved. Compared to traditional methods, the greatest advantage of eye tracking is that is that the reader proceeds with reading coherently and fluently without interrupt and interference, which secures the results in higher ecological validity. This paper is mainly about studies on eye movement features in ESL reading, and particularly illustrates eye movement patterns, the relationship between eye movement and perceptual processing, and eye movement control, aiming at providing some basic evidences for explaining the relationship between eye movement behavior and cognitive processing thus helping to solve some theoretical problems in reading. DOI: http://dx.doi.org/10.11591/telkomnika.v13i2.6936

Saccadic eye movements allow humans to explore the visual environment, quickly moving the fovea and attention to points of interest for detailed visual processing. Despite these rapid changes in visual input, the brain maintains a stable visual representation through saccadic suppression of blurred input and predictive remapping of receptive fields. In the laboratory, visually-guided saccades are driven primarily by the onset of new stimuli, while volitional saccades are driven by instructions or internal motivation. The distinction between these two classes of saccades is not definitive, but used to illustrate different cognitive loads that fall upon a spectrum based on the balance of visual input and cognitive control processes required to perform simple behavioral tasks. Visually-guided prosaccades depend upon a simple stimulus-response mapping whereby the eye movement is directed toward the peripheral stimulus quickly and accurately. Decision-making processes nonetheless slow the latencies of these basic visual-motor responses to ensure selection of an appropriate saccade target. To dissect the time course of saccade programming, researchers can include a secondary stimulus step or distractor stimulus in the saccade paradigm that alters response characteristics based on additional cognitive processes that occur. Attentional processes, for example, have a strong influence on the way in which participants respond: A pre-trial location cue attracts attention and reduces saccade latencies at that location and increases latencies at uncued locations. If attention is re-directed to central fixation, however, latencies to the originally cued location will be slower than other parts of the visual field, an effect known as inhibition of return. Furthermore, attentional preparation may contribute to the production of fast latency “express” saccades that occur in gap paradigms. The offset of fixation warns the participant of the upcoming stimulus appearance and disengages fixation neurons, allowing saccade-related activity to increase in expectation of the peripheral stimulus. On the other end of the spectrum, “volitional” forms of saccades are elicited via instructional manipulations that require participants to suppress an immediate saccade to a visual stimulus or otherwise rely upon endogenous goals to produce a saccade at a certain time. Antisaccades, ocular motor delayed response saccades, and predictive saccades all fall into this category. Antisaccade tasks require a saccade to the mirror image location of a visual stimulus, depending upon suppression of the visually-driven saccade tendency and a sensorimotor transformation of spatial coordinates to generate the volitional saccade successfully. In this task, the internal saccade programs representing the antisaccade and visually-guided saccade responses are modelled as competing activations racing toward a threshold for motor generation. An ocular motor delayed response task includes a delay period between the presentation of the peripheral stimulus and the time when participants are instructed to respond. Spatial memory processes thus are needed to maintain the target location after the visual stimulus itself is extinguished. Typically, both tasks result in slower latencies, more errors, and poorer spatial accuracy than their visually-guided counterparts. Finally, a predictive saccade task consists of a visual stimulus alternating between two locations at regular intervals. After a few trials, participants learn the spatiotemporal pattern and begin to anticipate the movement, generating saccades to the target location based on internal timing mechanisms before the visual stimulus appears. The aforementioned saccade tasks are supported by a well characterized and widespread neural saccade circuitry. This circuitry includes occipital cortex, posterior parietal cortex, frontal and supplementary eye fields, thalamus, basal ganglia, cerebellum, and the superior colliculus. Visual input initiates neural activation in occipital cortex which spreads to parietal and frontal regions for attentional processing, visuospatial calculations, and saccade motor preparation. Volitional saccade tasks show greater strength and/or extent of activation in this circuitry, and recruitment of new regions, including prefrontal cortex and anterior cingulate cortex, to support additional cognitive control requirements, such as inhibition, working memory, and motor learning. Along with input from the cerebellum, thalamus, and basal ganglia, cortical signals ultimately are integrated within the retinotopic maps of the superior colliculus where a single target location is selected and a motor program triggered via the brainstem ocular motor nuclei. Numerous neuroimaging studies in healthy humans using fMRI, EEG, MEG, and PET report activations in these brain areas during saccades and support findings from lesion studies and intracranial recordings in non-human primates. This consistency makes the saccade network a valuable model for studying cognitive control and understanding how the visual system integrates sensory input with internal goals to create a stable visual representation that guides efficient ocular motor behavior.

This is an accepted manuscript of an article published by ACM in W4A '20: Proceedings of the 17th International Web for All Conference on 20/04/2020, available online: https://doi.org/10.1145/3371300.3383340 &#13;\nThe accepted version of the publication may differ from the final published version.

A comprehensive decoding of cognitive load

Functional imaging studies have shown that certain brain regions, including posterior cingulate cortex (PCC) and ventral anterior cingulate cortex (vACC), consistently show greater activity during resting states than during cognitive tasks. This finding led to the hypothesis that these regions constitute a network supporting a default mode of brain function. In this study, we investigate three questions pertaining to this hypothesis: Does such a resting-state network exist in the human brain? Is it modulated during simple sensory processing? How is it modulated during cognitive processing? To address these questions, we defined PCC and vACC regions that showed decreased activity during a cognitive (working memory) task, then examined their functional connectivity during rest. PCC was strongly coupled with vACC and several other brain regions implicated in the default mode network. Next, we examined the functional connectivity of PCC and vACC during a visual processing task and show that the resultant connectivity maps are virtually identical to those obtained during rest. Last, we defined three lateral prefrontal regions showing increased activity during the cognitive task and examined their resting-state connectivity. We report significant inverse correlations among all three lateral prefrontal regions and PCC, suggesting a mechanism for attenuation of default mode network activity during cognitive processing. This study constitutes, to our knowledge, the first resting-state connectivity analysis of the default mode and provides the most compelling evidence to date for the existence of a cohesive default mode network. Our findings also provide insight into how this network is modulated by task demands and what functions it might subserve.

The study compares performance and eye-movement indicators in students from Russia, Japan, and China when searching for English words among randomly organized letters. The assumption was that social and linguistic experience affects visual scanning strategies and cognitive processing of verbal material. The study involved 64 respondents, demonstrating similar levels of English proficiency. Eye movements were recorded with SMI Gaze & Eye-tracking System (SMIRED 250 Hz). Russian respondents were the most successful at performing the task. Presumably, their success is determined by the experience in analyzing letter strings during sequential cognitive processing. Significant differences between the groups were established for both fixation duration and saccadic amplitude that describe the characteristics of cognitive processing (Velichkovsky et al., 2005). The Russian sample was characterized by high fixation durations (m = 239.5 ms) and low saccade amplitudes (m = 4.1°), demonstrating the focal pattern of attention distribution. The opposite trend toward ambient and gliding type of processing was recorded in the Japanese sample. In addition, the parameters of intersaccadic angles and saccade directions were analyzed. Return saccades were most characteristic of the Japanese group with direction changes in the range of 90–135°. Russian students demonstrated saccades of a sequential pattern in the range of 0–45°. Chinese students considered letter sets only once showing the smallest number of return saccades (the range of 135–180°). Based on the results of this study, the basic characteristics of visual semantic search can be associated with the first language experience that defines the patterns and the architecture of cognitive processing.

Attention plays a central role in cognitive processing; ineffective attention may induce accidents in flight operations. The objective of the current research was to examine military pilots' attention distributions between chasing a moving target and a stationary target. In the current research, 37 mission-ready F-16 pilots participated. Subjects' eye movements were collected by a portable head-mounted eye-tracker during tactical training in a flight simulator. The scenarios of chasing a moving target (air-to-air) and a stationary target (air-to-surface) consist of three operational phases: searching, aiming, and lock-on to the targets. The findings demonstrated significant differences in pilots' percentage of fixation during the searching phase between air-to-air (M = 37.57, SD = 5.72) and air-to-surface (M = 33.54, SD = 4.68). Fixation duration can indicate pilots' sustained attention to the trajectory of a dynamic target during air combat maneuvers. Aiming at the stationary target resulted in larger pupil size (M = 27,105, SD = 6565), reflecting higher cognitive loading than aiming at the dynamic target (M = 23,864, SD = 8762). Pilots' visual behavior is not only closely related to attention distribution, but also significantly associated with task characteristics. Military pilots demonstrated various visual scan patterns for searching and aiming at different types of targets based on the research settings of a flight simulator. The findings will facilitate system designers' understanding of military pilots' cognitive processes during tactical operations. They will assist human-centered interface design to improve pilots' situational awareness. The application of an eye-tracking device integrated with a flight simulator is a feasible and cost-effective intervention to improve the efficiency and safety of tactical training.Li W-C, Yu C-S, Braithwaite G, Greaves M. Pilots' attention distributions between chasing a moving target and a stationary target. Aerosp Med Hum Perform. 2016; 87(12):989-995.

Trace visualization within the Software City metaphor: Controlled experiments on program comprehension

A web-based Java programming learning assistant system (JPLAS) has been developed for novice students to study Java programming by themselves while enhancing code reading and code writing skills. One type of the implemented exercise problem is code writing problem (CWP), which asks students to create a source code that can pass the given test code. The correctness of this answer code is validated by running them on JUnit. In previous works, a Python-based answer code validation program was implemented to assist teachers. It automatically verifies the source codes from all the students for one test code, and reports the number of passed test cases by each code in the CSV file. While this program plays a crucial role in checking the correctness of code behaviors, it cannot detect code plagiarism that can often happen in programming courses. In this paper, we implement a code plagiarism checking function in the answer code validation program, and present its application results to a Java programming course at Okayama University, Japan. This function first removes the whitespace characters and the comments using the regular expressions. Next, it calculates the Levenshtein distance and similarity score for each pair of source codes from different students in the class. If the score is larger than a given threshold, they are regarded as plagiarism. Finally, it outputs the scores as a CSV file with the student IDs. For evaluations, we applied the proposed function to a total of 877 source codes for 45 CWP assignments submitted from 9 to 39 students and analyzed the results. It was found that (1) CWP assignments asking for shorter source codes generate higher scores than those for longer codes due to the use of test codes, (2) proper thresholds are different by assignments, and (3) some students often copied source codes from certain students.

In this study, we proposed an objective estimation of decline of arousal level by analyzing microsaccade rate and pupil fluctuation while subjects were continuously gazing a fixation target. Previous studies show that the slow eye movements (SEMs) could be a candidate for an indicator of decline of arousal. However, it is not sufficient to evaluate transition of arousal states since SEMs appear just prior to sleep onset. To establish more objective assessment of arousal, we examined the effects of the transition of arousal on microsaccade rate and pupil fluctuation. The subjects were instructed to indicate by mouse clicks when they were aware of having slept. We have analyzed the eye movement and pupil fluctuation data in advance of the occurrence of SEMs which were detected just before the mouse clicks. In the results, longitudinal pupil diameter shrinking and gradual rise of microsaccade rate were observed prior to SEMs. These results suggest that the arousal level could be evaluated by monitoring eye movements and pupil fluctuations.

<title>Abstract</title> Meta Quest Pro-based eye and facial tracking technology plays an essential role in studies related to visual attention, cognitive processes, user interactions, and human behavior analysis. However, differences in eye anatomy, such as outer canthal distance (OCD), interpupillary distance (IPD), and intercanthal distance (ICD), may impact the accuracy &amp; reliability of eye movement data. This study aims to evaluate the impact of inter-eye distance variations on eye-movement-based feature extraction in VR environments using the head-mounted device Meta Quest Pro. The data collection process involved eye measurements (OCD, IPD, and ICD), an eye calibration test before the experiment, and three experimental tasks: fixed gaze task, regular eye movement task, and irregular eye movement task. The Meta Quest Pro recorded participants’ eye movements while they performed VR-based tasks, followed by the extraction of eye-related features from the device. The dataset underwent Shapiro–Wilk, and D’Agostino–Pearson normality tests, followed by statistical analysis (Spearman Correlation, Kruskal–Wallis, and Mann–Whitney U test) to assess differences in eye-movement correlations across wide, narrow, and average eye distance groups. The results of statistical tests did not indicate a significant influence of inter-eye distance on eye movement tracking accuracy. This study is important for research, including learning disabilities research, child and adult behavior analysis, and human–computer interaction, ensuring that eye data remains accurate and reliable across diverse users.