Visual Saliency Map from Tensor Analysis

Characteristics of the human visual field are well known to be different in central (fovea) and peripheral areas. Existing computational models of visual saliency, however, do not take into account this biological evidence. The existing models compute visual saliency uniformly over the retina and, thus, have difficulty in accurately predicting the next gaze (fixation) point. This paper proposes to incorporate human visual field characteristics into visual saliency, and presents a computational model for producing such a saliency map. Our model integrates image features obtained by bottom-up computation in such a way that weights for the integration depend on the distance from the current gaze point where the weights are optimally learned using actual saccade data. The experimental results using a large number of fixation/saccade data with wide viewing angles demonstrate the advantage of our saliency map, showing that it can accurately predict the point where one looks next.

Visual saliency distribution maps for explaining time-series AI models used in continuous production of textile fibers

Visual saliency map has been proposed as a computational model for estimating the visual attention of a human. The saliency map is used to estimate the bottom-up visual attention for still and moving images. However, under the condition including the top-down visual attention, e.g. advertisement, the accuracy of visual attention estimated from the saliency map decreased. The character feature is considered as one of the factors that the deterioration of the saliency map accuracy is induced. In this study, we hypothesized that the features of characters have the saliency to induce the visual attention. Two types of experiments were performed to test this hypothesis. The still images inserted with both of characters (HIRAGANAs in Japanese or Thai strings) and simple symbols were used as visual stimuli in the experiments. The visual stimuli were presented to the subjects for a short period (2s) to exclude the effect of top-down attention. In result, the fixation probability of the character (both HIRAGANAs and Thais) region in the image was higher than that of the symbol region. The paired t-test provided the significant difference of the fixation ratio between HIRAGANAs and symbols (p < 0.001). The same is true for the paired t-test between Thais and symbols (p < 0.001). Thus, the present results indicate the visual saliency of characters.

Data storage via multimedia technology is more preferred as the information in multimedia contain rich meanings and are concise when compared to the traditional textual information. However, efficient information retrieval is a crucial factor in such storage. This paper presents a cognitive classification based visual saliency guided model for the efficient retrieval of information from multimedia data storage. The Itti visual saliency model is described here for generation of an overall saliency map with the integration of color saliency, intensity and direction maps. Multi-feature fusion paradigms are used for providing clear description of the image pattern. The definition is based on two stages namely complexity based on cognitive load and classification of complexity at a cognitive level. The image retrieval system is finalized by integrating a group sparse logistic regression model. In complex scenarios, the baselines are overcome by the proposed system when tested on multiple databased as compared to other state-of-the-art models.

Visual saliency estimation through label propagation is proposed. Since salient regions are generally distinctive from the background, visual saliency is estimated in an image by the similarity to salient and non‐salient seeds. First, salient and non‐salient seeds from the initial saliency map estimated by superpixel segmentation and three common priors are obtained. Then, labels of seeds are propagated to unlabelled superpixels based on label propagation to obtain the visual saliency map. Experimental results demonstrate that the proposed method is very effective in estimating salient regions in images compared with existing state‐of‐the‐art methods.

Shortcut learning in deep learning models occurs when unintended features are prioritized, resulting in degenerated feature representations and reduced generalizability and interpretability. However, shortcut learning in the widely used vision transformer (ViT) framework is largely unknown. Meanwhile, introducing domain-specific knowledge is a major approach to rectifying the shortcuts that are predominated by background-related factors. For example, eye-gaze data from radiologists are effective human visual prior knowledge that has the great potential to guide the deep learning models to focus on meaningful foreground regions. However, obtaining eye-gaze data can still sometimes be time-consuming, labor-intensive, and even impractical. In this work, we propose a novel and effective saliency-guided ViT (SGT) model to rectify shortcut learning in ViT with the absence of eye-gaze data. Specifically, a computational visual saliency model (either pretrained or fine-tuned) is adopted to predict saliency maps for input image samples. Then, the saliency maps are used to filter the most informative image patches. Considering that this filter operation may lead to global information loss, we further introduce a residual connection that calculates the self-attention across all the image patches. The experiment results on natural and medical image datasets show that our SGT framework can effectively learn and leverage human prior knowledge without eye-gaze data and achieves much better performance than baselines. Meanwhile, it successfully rectifies the harmful shortcut learning and significantly improves the interpretability of the ViT model, demonstrating the promise of transferring human prior knowledge derived visual saliency in rectifying shortcut learning.

Saliency detection is a technique to analyze image surroundings to extract relevant regions from the background. In this paper, we propose a simple and effective saliency detection method based on image sparse representation and color features combination. First, the input image is segmented into non-overlapping super-pixels, so as to perform the saliency detection at the region level to reduce computational complexity. Then, a background optimization selection scheme is used to construct an appropriate background template. Based on this, a primary saliency map is obtained by using image sparse representation. Next, through the linear combination of color coefficients we generate an improved saliency map with more prominent salient regions. Finally, the two saliency maps are integrated within Bayesian framework to obtain the final saliency map. Experimental results show that the proposed method has desirable detection performance in terms of detection accuracy and running time.

Several computational visual saliency models have been proposed in the context of viewing natural scenes. We aim to investigate the relevance of computational saliency models in medical images in the context of abnormality detection. We report on two studies aimed at understanding the role of visual saliency in medical images. Diffuse lesions in Chest X-Ray images, which are characteristic of Pneumoconiosis and high contrast lesions such as 'Hard Exudates' in retinal images were chosen for the study. These approximately correspond to conjunctive and disjunctive targets in a visual search task. Saliency maps were computed using three popular models namely Itti-Koch [7], GBVS [3] and SR [4]. The obtained maps were evaluated against gaze maps and ground truth from medical experts.Our results show that GBVS is seen to perform the best (Mdn. ROC area = 0.77) for chest X-Ray images while SR performs the best (ROC area = 0.73) for retinal images, thus asserting that searching for conjunctive targets calls for a more local examination of an image while disjunctive targets call for a global examination. Based on the results of the above study, we propose extensions for the two best performing models. The first extension makes use of top down knowledge such as lung segmentation. This is shown to improve the performance of GBVS to some extent. In the second case the extension is by way of including multi-scale information. This is shown to significantly (by 28.76%) improve abnormality detection. The key insight from these studies is that bottom saliency continues to play a predominant role in examining medical images.

The saliency map is useful for many applications such as image compression, display, and visualization. However, the bottom-up model used in most saliency map construction methods is computationally expensive. The purpose of this paper is to improve the efficiency of the model for automatic construction of the saliency map of an image while preserving its accuracy. In particular, we remove the contrast sensitivity function and the visual masking component of the bottom-up visual attention model and retain the components related to perceptual decomposition and center-surround interaction that are critical properties of human visual system. The simplified model is verified by performance comparison with the ground truth. In addition, a salient region enhancement technique is adopted to enhance the connectivity of the saliency map, and the saliency maps of three color channels are fused to enhance the prediction accuracy. Experimental results show that the average correlation between our algorithm and the ground truth is close to that between the original model and the ground truth, while the computational complexity is reduced by 98%.

This paper presents a new technique for extracting visual saliency from experimental eye tracking data. An eye-tracking system is employed to determine which features that a group of human observers considered to be salient when viewing a set of video images. With this information, a biologically inspired saliency map is derived by transforming each observed video image into a feature space representation. By using a feature normalisation process based on the relative abundance of visual features within the background image and those dwelled on eye tracking scan paths, features related to visual attention are determined. These features are then back projected to the image domain to determine spatial areas of interest for unseen video images. The strengths and weaknesses of the method are demonstrated with feature correspondence for 2D to 3D image registration of endoscopy videos with computed tomography data. The biologically derived saliency map is employed to provide an image similarity measure that forms the heart of the 2D/3D registration method. It is shown that by only processing selective regions of interest as determined by the saliency map, rendering overhead can be greatly reduced. Significant improvements in pose estimation efficiency can be achieved without apparent reduction in registration accuracy when compared to that of using a non-saliency based similarity measure.

In this article, we propose a simple and efficient method for computing an image saliency map, which performs well on both salient region detection and as well as eye gaze prediction tasks. A large number of distinct sub-windows with random co-ordinates and scales are generated over an image. The saliency descriptor of a pixel within a random sub-window is given by the absolute difference of its intensity value to the mean intensity of the sub-window. The final saliency value of a given pixel is obtained as the sum of all saliency descriptors corresponding to this pixel. Any given pixel can be included by one or more random sub-windows. The recall-precision performance of the proposed saliency map is comparable to other existing saliency maps for the task of salient region detection. It also achieves state-of-the-art performance for the task of eye gaze prediction in terms of receiver operating characteristics.KeywordsBottom-up Visual AttentionSaliency MapSalient Region DetectionEye Fixation

We propose a calibration-free gaze sensing method using visual saliency maps. Our goal is to construct a gaze estimator only using eye images captured from a person watching a video clip. The key is treating saliency maps of the video frames as probability distributions of gaze points. To efficiently identify gaze points from saliency maps, we aggregate saliency maps based on the similarity of eye appearances. We establish mapping between eye images to gaze points by Gaussian process regression. The experimental result shows that the proposed method works well with different people and video clips and achieves 6 degrees of accuracy, which is useful for estimating a person's attention on monitors.

Region of interest (ROI) detection plays an important role in object detection. It needs to be accurate and fast in some applications like real time disaster victim detection systems. ROI can reduce time and search space in detecting objects. In this paper visual saliency map is used for ROI detection. In most literature, most of ROI detection models only concentrate on reducing false positive (detecting wrong objects as intended ones) rate rather than false negative (missing intended object). In disaster victim detection, missing disaster victims is more important than detecting other objects like victim. So, the proposed method also focuses on reducing false negative error rate in object detection. In the proposed system, local entropy feature is added in Graph Based Visual Saliency (GBVS) map in addition to colour, orientation and shape feature maps.

The conceptual model of visual saliency in human vision system has been employed in extracting salient features from images and multimedia data in the last decade. This paper proposes to employ the visual saliency for moving object detection. The crucial factor is to compute a saliency map such that visual attention can be performed. This paper proposes a new method for saliency map construction based on information theory and spatio-temporal model, called information saliency map (ISM). The ISM provides rich information content of the video. Moving object detection are then performed based on the ISM. Two popular and publicly available visual surveillance databases from CAVIAR and PETS are selected for evaluation. Experimental results show that the proposed method is robust for moving object detection in complex background and illumination changes. The average detection rate is 90.35% while the false alarm rate is 2.46% in CAVIAR (INRIA entrance hall) dataset with ground truth data, and it has shown merits comparing with the current state of the art.

Hot-spot effect is well-known in the practical application of solar cells. The faults of hot spots seriously affect the performance and service life of solar cells. In order to highlight the infrared and visible complementary information of hot spots and discover more details, this paper presents a novel method, which fuses an infrared image saliency map based on random rectangular regions of interest with visible image by NonSubsampled Contourlet Transform (NSCT). Firstly, the infrared hot spots image was detected by saliency map which is randomly sampled. Secondly, the infrared and visible image of the hot spots are transformed into high-frequency and low-frequency sub-bands by NSCT. Finally, absolute value maximization and saliency map guidance are used to fuse low-frequency and high-frequency sub-bands respectively. The experimental results demonstrate that, compared with a set of related fusion methods, the presented method can better highlight the hot spots and other abundant information in visibility. It achieves accurate defects locating ability and has better performance on subjective and objective quality evaluation.