Generating Novel Information Salient Maps for Foreground Object Detection in Video

Visual saliency map has been proposed as a computational model for estimating the visual attention of a human. The saliency map is used to estimate the bottom-up visual attention for still and moving images. However, under the condition including the top-down visual attention, e.g. advertisement, the accuracy of visual attention estimated from the saliency map decreased. The character feature is considered as one of the factors that the deterioration of the saliency map accuracy is induced. In this study, we hypothesized that the features of characters have the saliency to induce the visual attention. Two types of experiments were performed to test this hypothesis. The still images inserted with both of characters (HIRAGANAs in Japanese or Thai strings) and simple symbols were used as visual stimuli in the experiments. The visual stimuli were presented to the subjects for a short period (2s) to exclude the effect of top-down attention. In result, the fixation probability of the character (both HIRAGANAs and Thais) region in the image was higher than that of the symbol region. The paired t-test provided the significant difference of the fixation ratio between HIRAGANAs and symbols (p < 0.001). The same is true for the paired t-test between Thais and symbols (p < 0.001). Thus, the present results indicate the visual saliency of characters.

Characteristics of the human visual field are well known to be different in central (fovea) and peripheral areas. Existing computational models of visual saliency, however, do not take into account this biological evidence. The existing models compute visual saliency uniformly over the retina and, thus, have difficulty in accurately predicting the next gaze (fixation) point. This paper proposes to incorporate human visual field characteristics into visual saliency, and presents a computational model for producing such a saliency map. Our model integrates image features obtained by bottom-up computation in such a way that weights for the integration depend on the distance from the current gaze point where the weights are optimally learned using actual saccade data. The experimental results using a large number of fixation/saccade data with wide viewing angles demonstrate the advantage of our saliency map, showing that it can accurately predict the point where one looks next.

Data storage via multimedia technology is more preferred as the information in multimedia contain rich meanings and are concise when compared to the traditional textual information. However, efficient information retrieval is a crucial factor in such storage. This paper presents a cognitive classification based visual saliency guided model for the efficient retrieval of information from multimedia data storage. The Itti visual saliency model is described here for generation of an overall saliency map with the integration of color saliency, intensity and direction maps. Multi-feature fusion paradigms are used for providing clear description of the image pattern. The definition is based on two stages namely complexity based on cognitive load and classification of complexity at a cognitive level. The image retrieval system is finalized by integrating a group sparse logistic regression model. In complex scenarios, the baselines are overcome by the proposed system when tested on multiple databased as compared to other state-of-the-art models.

We propose a method of predicting human egocentric visual attention using bottom-up visual saliency and egomotion information. Computational models of visual saliency are often employed to predict human attention; however, its mechanism and effectiveness have not been fully explored in egocentric vision. The purpose of our framework is to compute attention maps from an egocentric video that can be used to infer a person’s visual attention. In addition to a standard visual saliency model, two kinds of attention maps are computed based on a camera’s rotation velocity and direction of movement. These rotation-based and translation-based attention maps are aggregated with a bottom-up saliency map to enhance the accuracy with which the person’s gaze positions can be predicted. The efficiency of the proposed framework was examined in real environments by using a head-mounted gaze tracker, and we found that the egomotion-based attention maps contributed to accurately predicting human visual attention.

This article proposes a bottom-up visual saliency model that uses the wavelet transform to conduct multiscale analysis and computation in the frequency domain. First, we compute the multiscale magnitude spectra by performing a wavelet transform to decompose the magnitude spectrum of the discrete cosine coefficients of an input image. Next, we obtain multiple saliency maps of different spatial scales through an inverse transformation from the frequency domain to the spatial domain, which utilizes the discrete cosine magnitude spectra after multiscale wavelet decomposition. Then, we employ an evaluation function to automatically select the two best multiscale saliency maps. A final saliency map is generated via an adaptive integration of the two selected multiscale saliency maps. The proposed model is fast, efficient, and can simultaneously detect salient regions or objects of different sizes. It outperforms state-of-the-art bottom-up saliency approaches in the experiments of psychophysical consistency, eye fixation prediction, and saliency detection for natural images. In addition, the proposed model is applied to automatic ship detection in optical satellite images. Ship detection tests on satellite data of visual optical spectrum not only demonstrate our saliency model's effectiveness in detecting small and large salient targets but also verify its robustness against various sea background disturbances.

ABSTRACTLandmark visual salience (characterized by features that contrast with their surroundings and visual peculiarities) and semantic salience (characterized by features with unusual or important meaning and content in the environment) are two important factors that affect an individual’s visual attention during wayfinding. However, empirical evidence regarding which factor dominates visual guidance during indoor wayfinding is rare, especially in real-world environments. In this study, we assumed that semantic salience dominates the guidance of visual attention, which means that semantic salience will correlate with participants’ fixations more significantly than visual salience. Notably, in previous studies, semantic salience was shown to guide visual attention in static images or familiar scenes in a laboratory environment. To validate this assumption, first, we collected the eye movement data of 22 participants as they found their way through a building. We then computed the landmark visual and semantic salience using computer vision models and questionnaires, respectively. Finally, we conducted correlation tests to verify our assumption. The results failed to validate our assumption and show that the role of salience in visual guidance in a real-world wayfinding process is different from the role of salience in perceiving static images or scenes in a laboratory. Visual salience dominates visual attention during indoor wayfinding, but the roles of salience in visual guidance are mixed across different landmark classes and tasks. The results provide new evidence for understanding how pedestrians visually interpret landmark information during real-world indoor wayfinding.

Computational modeling of visual attention is an active area of research. These models have been successfully employed in applications such as robotics. However, most computational models of visual attention are developed in the context of natural scenes, and their role with medical images is not well investigated. As radiologists interpret a large number of clinical images in a limited time, an efficient strategy to deploy their visual attention is necessary. Visual saliency maps, highlighting image regions that differ dramatically from their surroundings, are expected to be predictive of where radiologists fixate their gaze. We compared 16 state-of-art saliency models over three medical imaging modalities. The estimated saliency maps were evaluated against radiologists' eye movements. The results show that the models achieved competitive accuracy using three metrics, but the rank order of the models varied significantly across the three modalities. Moreover, the model ranks on the medical images were all considerably different from the model ranks on the benchmark MIT300 dataset of natural images. Thus, modality-specific tuning of saliency models is necessary to make them valuable for applications in fields such as medical image compression and radiology education.

Visual saliency and spike timing in the ventral visual pathway

When looking at a scene, we frequently move our eyes to place consecutive interesting regions on the fovea, the retina centre. At each fixation, only this specific foveal region is analysed in detail by the visual system. The visual attention mechanisms control eye movements and depend on two types of factor: bottom-up and top-down factors. Bottom-up factors include different visual features such as colour, luminance, edges, and orientations. In this paper, we evaluate quantitatively the relative contribution of basic low-level features as candidate guiding factors to visual attention and hence to eye movements. We also study how these visual features can be combined in a bottom-up saliency model. Our work consists of three interactive parts: a functional saliency model, a statistical model and eye movement data recorded during free viewing of natural scenes. The functional saliency model, inspired by the primate visual system, decomposes a visual scene into different feature maps. The statistical model indicates which features best explain the recorded eye movements. We show an essential role of high frequency luminance and an important contribution of central fixation bias. The relative contribution of features, calculated by the statistical model, is then used to combine the different feature maps into a saliency map. Finally, the comparison between the saliency model and experimental data confirmed the influence of these contributions.

Several computational visual saliency models have been proposed in the context of viewing natural scenes. We aim to investigate the relevance of computational saliency models in medical images in the context of abnormality detection. We report on two studies aimed at understanding the role of visual saliency in medical images. Diffuse lesions in Chest X-Ray images, which are characteristic of Pneumoconiosis and high contrast lesions such as 'Hard Exudates' in retinal images were chosen for the study. These approximately correspond to conjunctive and disjunctive targets in a visual search task. Saliency maps were computed using three popular models namely Itti-Koch [7], GBVS [3] and SR [4]. The obtained maps were evaluated against gaze maps and ground truth from medical experts.Our results show that GBVS is seen to perform the best (Mdn. ROC area = 0.77) for chest X-Ray images while SR performs the best (ROC area = 0.73) for retinal images, thus asserting that searching for conjunctive targets calls for a more local examination of an image while disjunctive targets call for a global examination. Based on the results of the above study, we propose extensions for the two best performing models. The first extension makes use of top down knowledge such as lung segmentation. This is shown to improve the performance of GBVS to some extent. In the second case the extension is by way of including multi-scale information. This is shown to significantly (by 28.76%) improve abnormality detection. The key insight from these studies is that bottom saliency continues to play a predominant role in examining medical images.

Modeling visual saliency map of an image provides important information for image semantic understanding in many applications. Most existing computational visual saliency models follow a bottom-up framework that generates independent saliency map in each selected visual feature space and combines them in a proper way. Two big challenges to be addressed explicitly in these methods are (1) which features should be extracted for all pixels of the input image and (2) how to dynamically determine importance of the saliency map generated in each feature space. In order to address these problems, we present a novel saliency map computational model based on tensor decomposition and reconstruction. Tensor representation and analysis not only explicitly represent image's color values but also imply two important relationships inherent to color image. One is reflecting spatial correlations between pixels and the other one is representing interplay between color channels. Therefore, saliency map generator based on the proposed model can adaptively find the most suitable features and their combinational coefficients for each pixel. Experiments on a synthetic image set and a real image set show that our method is superior or comparable to other prevailing saliency map models.

Visual saliency distribution maps for explaining time-series AI models used in continuous production of textile fibers

Image saliency estimation via random walk guided by informativeness and latent signal correlations

Radiologists' eye movements during medical image interpretation reflect their perceptual-cognitive processes of diagnostic decisions. The eye movement data can be modeled to represent clinically relevant regions in a medical image and potentially integrated into an artificial intelligence (AI) system for automatic diagnosis in medical imaging. In this article, we first conduct a large-scale eye-tracking study involving 13 radiologists interpreting 191 chest X-ray (CXR) images, establishing a best-of-its-kind CXR visual saliency benchmark. We then perform analysis to quantify the reliability and clinical relevance of saliency maps (SMs) generated for CXR images. We develop CXR image saliency prediction method (CXRSalNet), a novel saliency prediction model that leverages radiologists' gaze information to optimize the use of unlabeled CXR images, enhancing training and mitigating data scarcity. We also demonstrate the application of our CXR saliency model in enhancing the performance of AI-powered diagnostic imaging systems.

In response to the problem of high false alarm rates in infrared dim-small target detection with complex ground backgrounds, this paper proposes an infrared dim-small detection algorithm based on saliency map, combined with target motion features. This algorithm firstly transforms the small target detection problem of IR images into a problem of recovering low rank matrices and sparse matrices from original matrices, and then removes the false targets with an improved morphological filtering algorithm. By observing the target motion characteristics between multiple frames, a method combining with Kalman filter is proposed to correlate the target in order to further reduce the false alarm rate. The simulation results show that the proposed algorithm can effectively reduce the false alarm rate of infrared vehicle target detection in complex background, ensuring a high detection rate and real-time performance, while also solving the problem of short-term target obscuration and disappearance.