Measuring visual clutter

We investigated the relationship between visual clutter and visual search in real-world scenes. Specifically, we investigated whether visual clutter, indexed by feature congestion, sub-band entropy, and edge density, correlates with search performance as assessed both by traditional behavioral measures (response time and error rate) and by eye movements. Our results demonstrate that clutter is related to search performance. These results hold for both traditional search measures and for eye movements. The results suggest that clutter may serve as an image-based proxy for search set size in real-world scenes.

This study focuses on understanding and quantifying the effects of clutter in multi-plane Augmented Reality User Interfaces (AR UI’s). It determines if image analysis measures of visual clutter can be expanded from single plane AR UI’s to multi-plane AR UI’s as well as examining the effects of multiple UI planes, UI clutter, and target clutter have on user performance. Image analysis measures of clutter were specifically chosen as they can be applied to complex and naturalistic images that may be found in AR UI applications and have shown use in predicting user performance in single plane AR UI’s. In this experiment, twelve participants performed a visual search task of locating a target object in an array of objects where objects were located in planes of different distances. Participants completed this task under two different clutter levels (low and high) against seven different AR UI plane arrays (2m, 4m, 6m, 2m4m, 2m6m, 4m6m, 2m4m6m) where the target was found in every available AR UI plane for each combination, with repetition. Task performance was measured through response time. Results show significant differences in response time between clutter levels, and number of planes. The end goal of this research is to build upon previous studies to develop a clutter score algorithm for AR UI’s that can predict user performance based on image analysis measures.

One of the puzzles of visual search is that discriminability of a single target from a single distractor poorly predicts search performance. Last year (Rosenholtz, Chan, & Balas, VSS 2009) we suggested that in crowded visual search displays, the key determinant of search performance is instead peripheral discriminability between a patch containing both target and distractors and a patch containing multiple distractors. Using a model of peripheral vision in which the visual system represents the visual input by summary statistics over each local pooling region (Balas, Nakano, & Rosenholtz, 2009), we predicted peripheral discriminability (d′) of crowded target-present and distractor-only patches, and showed that this in turn predicted the relative difficulty of a number of standard search tasks. Here, our goal is to make quantitative predictions of visual search performance using this framework. Specifically, we model both reaction time vs. set size (RT/setsize) slopes and number of fixations to find the target. To this end, we have derived the ideal saccadic targeter for the case in which the input consists of independent noisy “targetness” measurements from multiple, overlapping pooling regions. The radius of each pooling region is roughly half its eccentricity, in accordance with Bouma's Law. For crowded pooling regions, our predicted d′ allows us to compute the likelihood of observing a given amount of “targetness,” conditioned on whether or not the given pooling region contains a target. For uncrowded pooling regions, e.g. near fixation, discriminability is maximal. An additional parameter controls the amount of memory from previous fixations. The model performs well at predicting RT/setsize slopes, and reasonably well at predicting mean number of fixations to find the target. Best predictions come when the model has minimal memory. This suggests that search performance is indeed constrained by the extend to which peripheral vision can discriminate between target-present and distractor-only patches.

Visual search and visual lobe size: can training on one affect the other?

Measuring attention to traffic is common in pedestrian safety studies. However, little is known of how visual search patterns affect safe crossing decisions or to what pedestrians are actually attending....

Standard models of visual search have focused upon asking participants to search for a single target in displays where the objects do not overlap one another, and where the objects are presented on a single depth plane. This stands in contrast to many everyday visual searches wherein variations in overlap and depth are the norm, rather than the exception. Here, we addressed whether presenting overlapping objects on different depths planes to one another can improve search performance. Across 4 different experiments using different stimulus types (opaque polygons, transparent polygons, opaque real-world objects, and transparent X-ray images), we found that depth was primarily beneficial when the displays were transparent, and this benefit arose in terms of an increase in response accuracy. Although the benefit to search performance only appeared in some cases, across all stimulus types, we found evidence of marked shifts in eye-movement behavior. Our results have important implications for current models and theories of visual search, which have not yet provided detailed accounts of the effects that overlap and depth have on guidance and object identification processes. Moreover, our results show that the presence of depth information could aid real-world searches of complex, overlapping displays. (PsycINFO Database Record

The effect of motion on visual search has been extensively investigated, but that of uniform linear motion of display on search performance for tasks with different target–distractor shape representations has been rarely explored. The present study conducted three visual search experiments. In Experiments 1 and 2, participants finished two search tasks that differed in target–distractor shape representations under static and dynamic conditions. Two tasks with clear and blurred stimuli were performed in Experiment 3. The experiments revealed that target–distractor shape representation modulated the effect of motion on visual search performance. For tasks with low target–distractor shape similarity, motion negatively affected search performance, which was consistent with previous studies. However, for tasks with high target–distractor shape similarity, if the target differed from distractors in that a gap with a linear contour was added to the target, and the corresponding part of distractors had a curved contour, motion positively influenced search performance. Motion blur contributed to the performance enhancement under dynamic conditions. The findings are useful for understanding the influence of target–distractor shape representation on dynamic visual search performance when display had uniform linear motion.

Advance knowledge of the time required by an observer to detect a target visually is of interest, e.g., in preparing flight scenarios, in modeling mission performance, in evaluating camouflage effectiveness, and in visual-scene generator calibration. A wide range of computational models has therefore been developed to predict human visual search and detection performance. This study is performed to test the quality of the predictions of three of these models: ORACLE, Visdet, and a formula by Travnikova. The three different models are used to predict the results of an experiment in which observers searched for military vehicles in complex rural scenes. The models predict either the mean time required to find the target, or the probability of finding the target after a given amount of time, from a few physical parameters describing the scene (the mean scene luminance, the angular dimensions of the field of view and the target, the intrinsic target contrast, etc.). None of the models reliably predicts observer performance for most of the scenes used in this study. ORACLE and Visdet both overestimate the detection probability for most situations. The formula by Travnikova does not apply to the scenes used here.

Visual search is an important component of many real world tasks such as industrial inspection and driving. Several studies have shown that age has an impact on visual search performance. In general older people demonstrate poorer performance on such tasks as compared to younger people. However, there is controversy regarding the source of the age-performance effect. The objective of this study was to examine the relationship between component abilities and visual search performance, in order to identify the locus of age-related performance differences. Six abilities including reaction time, working memory, selective attention and spatial localization were identified as important components of visual search performance. Thirty-two subjects ranging in age from 18 - 84 years, categorized in three different age groups (young, middle, and older) participated in the study. Their component abilities were measured and they performed a visual search task. The visual search task varied in complexity in terms of type of targets detected. Significant relationships were found between some of the component skills and search performance. Significant age effects were also observed. A model was developed using hierarchical multiple linear regression to explain the variance in search performance. Results indicated that reaction time, selective attention, and age were important predictors of search performance with reaction time and selective attention accounting for most of the variance.

To determine whether image enhancement improves visual search performance and whether enhanced images were also preferred by subjects with vision impairment. Subjects (n = 24) with vision impairment (vision: 20/52 to 20/240) completed visual search and preference tasks for 150 static images that were enhanced to increase object contours' visual saliency. Subjects were divided into two groups and were shown three enhancement levels. Original and medium enhancements were shown to both groups. High enhancement was shown to group 1, and low enhancement was shown to group 2. For search, subjects pointed to an object that matched a search target displayed at the top left of the screen. An "integrated search performance" measure (area under the curve of cumulative correct response rate over search time) quantified performance. For preference, subjects indicated the preferred side when viewing the same image with different enhancement levels on side-by-side high-definition televisions. Contour enhancement did not improve performance in the visual search task. Group 1 subjects significantly (p < 0.001) rejected the High enhancement, and showed no preference for medium enhancement over the original images. Group 2 subjects significantly preferred (p < 0.001) both the medium and the low enhancement levels over original. Contrast sensitivity was correlated with both preference and performance; subjects with worse contrast sensitivity performed worse in the search task (ρ = 0.77, p < 0.001) and preferred more enhancement (ρ = -0.47, p = 0.02). No correlation between visual search performance and enhancement preference was found. However, a small group of subjects (n = 6) in a narrow range of mid-contrast sensitivity performed better with the enhancement, and most (n = 5) also preferred the enhancement. Preferences for image enhancement can be dissociated from search performance in people with vision impairment. Further investigations are needed to study the relationships between preference and performance for a narrow range of mid-contrast sensitivity where a beneficial effect of enhancement may exist.

Any object-oriented action requires that the object be first brought into the attentional foreground, often through visual search. Outside the laboratory, this would always take place in the presence of a scene representation acquired from ongoing visual exploration. The interaction of scene memory with visual search is still not completely understood. Feature integration theory (FIT) has shaped both research on visual search, emphasizing the scaling of search times with set size when searches entail feature conjunctions, and research on visual working memory through the change detection paradigm. Despite its neural motivation, there is no consistently neural process account of FIT in both its dimensions. We propose such an account that integrates (1) visual exploration and the building of scene memory, (2) the attentional detection of visual transients and the extraction of search cues, and (3) visual search itself. The model uses dynamic field theory in which networks of neural dynamic populations supporting stable activation states are coupled to generate sequences of processing steps. The neural architecture accounts for basic findings in visual search and proposes a concrete mechanism for the integration of working memory into the search process. In a behavioral experiment, we address the long-standing question of whether both the overall speed and the efficiency of visual search can be improved by scene memory. We find both effects and provide model fits of the behavioral results. In a second experiment, we show that the increase in efficiency is fragile, and trace that fragility to the resetting of spatial working memory.

Applying models of visual search to map display design

Research in visual search can be vital to improving performance in careers such as radiology and airport security screening. In these applied, or "field," searches, accuracy is critical, and misses are potentially fatal; however, despite the importance of performing optimally, radiological and airport security searches are nevertheless flawed. Extensive basic research in visual search has revealed cognitive mechanisms responsible for successful visual search as well as a variety of factors that tend to inhibit or improve performance. Ideally, the knowledge gained from such laboratory-based research could be directly applied to field searches, but several obstacles stand in the way of straightforward translation; the tightly controlled visual searches performed in the lab can be drastically different from field searches. For example, they can differ in terms of the nature of the stimuli, the environment in which the search is taking place, and the experience and characteristics of the searchers themselves. The goal of this chapter is to discuss these differences and how they can present hurdles to translating lab-based research to field-based searches. Specifically, most search tasks in the lab entail searching for only one target per trial, and the targets occur relatively frequently, but field searches may contain an unknown and unlimited number of targets, and the occurrence of targets can be rare. Additionally, participants in lab-based search experiments often perform under neutral conditions and have no formal training or experience in search tasks; conversely, career searchers may be influenced by the motivation to perform well or anxiety about missing a target, and they have undergone formal training and accumulated significant experience searching. This chapter discusses recent work that has investigated the impacts of these differences to determine how each factor can influence search performance. Knowledge gained from the scientific exploration of search can be applied to field searches but only when considering and controlling for the differences between lab and field.

BackgroundSuperior visual search is a replicated finding in the literature on Autism Spectrum Disorder (ASD). Conversely, results from the literature on Attention-Deficit/Hyperactivity Disorder (ADHD) are more mixed, with some studies showing typical performance and others pointing out less efficient serial search in ADHD. Finally, most studies on visual search in Schizophrenia highlighted deficits in focal attentional processing. However, similarities between attentional impairments in the three clinical groups have also been reported. The primary goal of our study is to determine the diagnostic specificity of search deficits. To our knowledge, the literature systematically comparing the visual search performance in patients with ADHD and Schizophrenia is limited to one study, while no study has so far included an ASD group in the comparison.MethodsFour groups of young adults, namely 29 typically developing (TD; 19.8±1.6, 41% males), 26 with ASD (19.7±1.9, 96% males), 28 with ADHD (19.9±1.4, 54% males) and 21 with Early-Onset Schizophrenia (SCZ; 19.7±1.7, 71% males) were presented a visual search task in which they had to quickly detect a target item among several similar-to-target distractor items. Eye movements were recorded binocularly with the Eye Link 1000+ system. Participants were matched on age and full-scale IQ. The ocular-motor behaviour was analysed using Data Viewer 3.2 and SPSS 2.3.ResultsInitiation of search – latency of the first saccade on the search grid – was typical in all participants except those with SCZ, who had significant higher intra-subject variability (ISV) than both TD and ADHD, but no delay in initiating search. Within search, ASD manifested significantly reduced mean and ISV of total search duration – between the first saccade on the grid and the last fixation on target – and of the first part of search – between the first saccade on the grid and the first fixation on target – in comparison with all other groups, including TD. Conversely, SCZ and ASD were significantly more variable than TD and ADHD regarding the duration of first fixation on target, while also being, to a lesser extent, slower than ADHD, but not than TD. Additionally, SCZ needed a higher frequency of fixations on target than ASD, but not compared to ADHD or TD, before making a decision. In the post-search phase – between the onset of the last fixation on target and the button press on the keyboard – SCZ were the slowest and most variable group, followed by ASD and ADHD who differed?non-significantly? from controls. The overall search performance – between trial onset and button press – resulted in typical manual mean RT in ASD and ADHD while being atypically longer in SCZ, compared to all other groups. Compared to TD, ISV was at par in ADHD, lower in ASD, and higher in SCZ.DiscussionResults suggest that the ability to extract individual targets is intact in ASD and ADHD. However, ASD only show a bias toward local information, as indicated by more variable first fixation duration, despite intact global processing. By contrast, lower search efficiency in SCZ might be explained by both (a) abnormal global processing due to impairment in the guidance mechanisms that affect the time until the first fixation on target, and (b) a deficit in central discrimination, with resulting difficulties in extracting critical features of the target.

Many previous studies on visual search have reported inter-trial effects, that is, observers respond faster when some target property, such as a defining feature or dimension, or the response associated with the target repeats versus changes across consecutive trial episodes. However, what processes drive these inter-trial effects is still controversial. Here, we investigated this question using a combination of Bayesian modeling of belief updating and evidence accumulation modeling in perceptual decision-making. In three visual singleton (‘pop-out’) search experiments, we explored how the probability of the response-critical states of the search display (e.g., target presence/absence) and the repetition/switch of the target-defining dimension (color/ orientation) affect reaction time distributions. The results replicated the mean reaction time (RT) inter-trial and dimension repetition/switch effects that have been reported in previous studies. Going beyond this, to uncover the underlying mechanisms, we used the Drift-Diffusion Model (DDM) and the Linear Approach to Threshold with Ergodic Rate (LATER) model to explain the RT distributions in terms of decision bias (starting point) and information processing speed (evidence accumulation rate). We further investigated how these different aspects of the decision-making process are affected by different properties of stimulus history, giving rise to dissociable inter-trial effects. We approached this question by (i) combining each perceptual decision making model (DDM or LATER) with different updating models, each specifying a plausible rule for updating of either the starting point or the rate, based on stimulus history, and (ii) comparing every possible combination of trial-wise updating mechanism and perceptual decision model in a factorial model comparison. Consistently across experiments, we found that the (recent) history of the response-critical property influences the initial decision bias, while repetition/switch of the target-defining dimension affects the accumulation rate, likely reflecting an implicit ‘top-down’ modulation process. This provides strong evidence of a disassociation between response- and dimension-based inter-trial effects.