A biologically inspired computational model for image saliency detection

Saliency mechanism has been considered crucial in the human visual system and helpful to object detection and recognition. This paper addresses an information theoretic model for visual saliency detection. It consists of two steps: first, using the Non-negative Matrix Factorization with sparseness constraints (NMFsc) algorithm to learn the basis functions from a set of randomly sampled natural image patches; and then, applying information theoretic principle to generate the saliency map by the Salient Information (SI) which is calculated from the coefficients represented by basis functions. We compare our model with the previous methods on natural images. Experimental results show that our model performs better than existing approaches.

Many image analysis, computer vision and multimedia applications involving consumer images rely on or benefit from saliency maps which represent where important areas of images are located. To date, virtually all image saliency detection techniques tend to produce highly blurry saliency maps. However, there are situations where the awareness of objects and their boundaries in images can greatly facilitate solutions to the problems. In this paper, we propose a novel, effective and efficient image saliency detection algorithm. Notably, our algorithm is capable of detecting salient regions of an image with clear boundaries, corresponding to objects in the scene. Yet, the algorithm is robust to background clutter commonly found in typical consumer images. A comparison of our algorithm with several state-of-the-art image saliency detection algorithms reveals the favorable performance of our algorithm in terms of the quality of saliency maps and the computational time.

Saliency and co-saliency detection aim to distinguish conspicuous foreground objects from single and multiple images, thus are essential in many multimedia and vision applications. To achieve balanced efficiency and accuracy, most recent successful saliency detectors are based on superpixels. However, saliency detection with single-scale superpixel segmentation may fail in capturing intrinsic salient objects of complex natural scenes with small-scale high-contrast backgrounds. To tackle this problem, we present a simple strategy using multiscale superpixels to jointly detect salient object via low-rank optimisation. Specifically, we first build a multiscale superpixel pyramid and derive the corresponding saliency map by multimodal saliency features and priors at each single scale. Then, we use joint low-rank analysis of multiscale saliency maps to obtain a more reliable and adaptively-fused saliency map, which properly takes all scales saliency into account. We further propose a GMM generative co-saliency prior to enable the above approach to detect co-salient objects from multiple images. Extensive experiments on benchmark datasets validate the effectiveness and superiority of the proposed saliency and co-saliency detector over state-of-the-arts.

Image saliency detection is very useful in many computer vision tasks while it still remains a challenging problem. In this paper, we propose a new computational saliency detection model which is implemented with a coarse to fine strategy under the Bayesian framework. First, saliency points are applied to get a coarse location of the saliency region. And then, based on the rough region, we compute a prior map for the Bayesian model to achieve the final saliency map. Experimental results on a public available dataset show the effectiveness of the proposed prior map and the strength of our saliency map compared with several previous method.

The object contour detection in natural scenes is an important research problem in computer vision. The difficulty is that the integrity of contour extracted is interfered with a large number of texture edges in the background seriously. In recent years, some researchers have introduced image contour detection method with biological visual features and achieved preferable results. Such as, based on the fact of visual outer region suppression, they can suppress a certain amount of texture edge while extracting the contour of the object, and obtain the proper contour edge. However, when this method is used for contour detection in some image which the texture is similar to the contour, the texture edge still cannot be removed well, and the result is unsatisfactory. In consideration of the merit and demerit of the feature suppressed method, we propose an improved algorithm for object contour detection in natural scene based on image saliency. Firstly, the initial contour is obtained by using the suppression effect of non-classical receptive field, and the saliency map is obtained by using the hyper-complex Fourier transform image saliency detection method, and then the initial contour and the saliency map are integrated into the final contour. Compared with the traditional contour detection method, the contour extraction algorithm based on image saliency can remove more background information and improve the accuracy and integrity of object contour extracted in natural scenes.

Saliency prediction typically relies on hand-crafted (multiscale) features that are combined in different ways to form a "master" saliency map, which encodes local image conspicuity. Recent improvements to the state of the art on standard benchmarks such as MIT1003 have been achieved mostly by incrementally adding more and more hand-tuned features (such as car or face detectors) to existing models. In contrast, we here follow an entirely automatic data-driven approach that performs a large-scale search for optimal features. We identify those instances of a richly-parameterized bio-inspired model family (hierarchical neuromorphic networks) that successfully predict image saliency. Because of the high dimensionality of this parameter space, we use automated hyperparameter optimization to efficiently guide the search. The optimal blend of such multilayer features combined with a simple linear classifier achieves excellent performance on several image saliency benchmarks. Our models outperform the state of the art on MIT1003, on which features and classifiers are learned. Without additional training, these models generalize well to two other image saliency data sets, Toronto and NUSEF, despite their different image content. Finally, our algorithm scores best of all the 23 models evaluated to date on the MIT300 saliency challenge, which uses a hidden test set to facilitate an unbiased comparison.

Though there are many computational models proposed for saliency detection, few of them take object boundary information into account. This paper presents a hierarchical saliency detection model incorporating probabilistic object boundaries, which is based on the observation that salient objects are generally surrounded by explicit boundaries and show contrast with their surroundings. We perform adaptive thresholding operation on ultrametric contour map, which leads to hierarchical image segmentations, and compute the saliency map for each layer based on the proposed robust center bias, border bias, color dissimilarity and spatial coherence measures. After a linear weighted combination of multi-layer saliency maps, and Bayesian enhancement procedure, the final saliency map is obtained. Extensive experimental results on three challenging benchmark datasets demonstrate that the proposed model outperforms eight state-of-the-art saliency detection models.

In existing superpixel-wise saliency detection algorithms, superpixel generation often is an isolated preprocessing step. The performance of saliency maps is determined by the accuracy of superpixels to a certain extent. However, it is still a challenge to develop a stable superpixel generation method. In this article, we attempt to incorporate the superpixel generation and saliency calculation steps into an end-to-end trainable deep network. First, we employ a recently proposed differentiable superpixel generation method to over-segment the synthetic aperture radar (SAR) images, which outputs the possibility that the pixels assigned to neighbor superpixels (soft superpixel). In saliency calculation part, as one of our main contributions, we propose a differentiable and computationally simple saliency model, i.e., lacunarity cue. It is inspired by the fact that generally the backscattering intensity of regions of interest (ROIs) in SAR images irregularly fluctuates, while the areas with consistent pixels are often ignored as the clusters. We improve the pixelwise box differential dimension algorithm to measure the irregularity of scattering points in a superpixel. The superpixel generation and saliency calculation can be implemented under a unified deep network. Hence, the shapes of the superpixels can be iteratively adjusted according to the saliency maps until the ROIs are correctly detected. Experiments on real SAR images with different sizes and scenes show that the saliency maps can effectively highlight the target areas, thus outperforming the state-of-the-art saliency detection models.

The effectiveness of depth information in saliency detection has been fully proved. However, it is still worth exploring how to utilize the depth information more efficiently. Erroneous depth information may cause detection failure, while non-salient objects may be closer to the camera which also leads to erroneously emphasis on non-salient regions. Moreover, most of the existing RGB-D saliency detection models have poor robustness when the salient object touches the image boundaries. To mitigate these problems, we propose a multi-stage saliency detection model with the bilateral absorbing Markov chain guided by depth information. The proposed model progressively extracts the saliency cues with three level (low-, mid-, and high-level) stages. First, we generate low-level saliency cues by explicitly combining color and depth information. Then, we design a bilateral absorbing Markov chain to calculate mid-level saliency maps. In mid-level, to suppress boundary touch problem, we present the background seed screening mechanism (BSSM) for improving the construction of the two-layer sparse graph and better selecting background-based absorbing nodes. Furthermore, the cross-modal multi-graph learning model (CMLM) is designed to fully explore the intrinsic complementary relationship between color and depth information. Finally, to obtain a more highlighted and homogeneous saliency map in high-level, we structure a depth-guided optimization module which combines cellular automata and suppression-enhancement function pair. This optimization module refines the saliency map in color space and depth space, respectively. Comprehensive experiments on three challenging benchmark datasets demonstrate the effectiveness of our proposed method both qualitatively and quantitatively.

Radiologists' eye movements during medical image interpretation reflect their perceptual-cognitive processes of diagnostic decisions. The eye movement data can be modeled to represent clinically relevant regions in a medical image and potentially integrated into an artificial intelligence (AI) system for automatic diagnosis in medical imaging. In this article, we first conduct a large-scale eye-tracking study involving 13 radiologists interpreting 191 chest X-ray (CXR) images, establishing a best-of-its-kind CXR visual saliency benchmark. We then perform analysis to quantify the reliability and clinical relevance of saliency maps (SMs) generated for CXR images. We develop CXR image saliency prediction method (CXRSalNet), a novel saliency prediction model that leverages radiologists' gaze information to optimize the use of unlabeled CXR images, enhancing training and mitigating data scarcity. We also demonstrate the application of our CXR saliency model in enhancing the performance of AI-powered diagnostic imaging systems.

This article proposes a bottom-up visual saliency model that uses the wavelet transform to conduct multiscale analysis and computation in the frequency domain. First, we compute the multiscale magnitude spectra by performing a wavelet transform to decompose the magnitude spectrum of the discrete cosine coefficients of an input image. Next, we obtain multiple saliency maps of different spatial scales through an inverse transformation from the frequency domain to the spatial domain, which utilizes the discrete cosine magnitude spectra after multiscale wavelet decomposition. Then, we employ an evaluation function to automatically select the two best multiscale saliency maps. A final saliency map is generated via an adaptive integration of the two selected multiscale saliency maps. The proposed model is fast, efficient, and can simultaneously detect salient regions or objects of different sizes. It outperforms state-of-the-art bottom-up saliency approaches in the experiments of psychophysical consistency, eye fixation prediction, and saliency detection for natural images. In addition, the proposed model is applied to automatic ship detection in optical satellite images. Ship detection tests on satellite data of visual optical spectrum not only demonstrate our saliency model's effectiveness in detecting small and large salient targets but also verify its robustness against various sea background disturbances.

Data-driven saliency detection has attracted strong interest as a result of applying convolutional neural networks to the detection of eye fixations. Although a number of image-based salient object and fixation detection models have been proposed, video fixation detection still requires more exploration. Different from image analysis, motion and temporal information is a crucial factor affecting human attention when viewing video sequences. Although existing models based on local contrast and low-level features have been extensively researched, they failed to simultaneously consider interframe motion and temporal information across neighboring video frames, leading to unsatisfactory performance when handling complex scenes. To this end, we propose a novel and efficient video eye fixation detection model to improve the saliency detection performance. By simulating the memory mechanism and visual attention mechanism of human beings when watching a video, we propose a step-gained fully convolutional network by combining the memory information on the time axis with the motion information on the space axis while storing the saliency information of the current frame. The model is obtained through hierarchical training, which ensures the accuracy of the detection. Extensive experiments in comparison with 11 state-of-the-art methods are carried out, and the results show that our proposed model outperforms all 11 methods across a number of publicly available datasets.

Due to complex lighting circumstances and other factors, visual saliency recognition of underwater images is not as good as visual saliency detection models that function well on land when applied to underwater environments. The dearth of large-scale annotated underwater picture datasets has further hindered the development of visual saliency detection in underwater images. we propose an attention-based approach for underwater image saliency detection to address this issue. The backbone network is pre-trained using both land and underwater image datasets, which effectively solves the problem of fewer datasets. After that, a residual refinement module incorporating a channel-space attention mechanism is merged, the model as a whole is trained end-to-end using underwater image datasets, and the model accuracy is experimentally tested. The experimental results in three underwater image datasets show that our method is better than several other methods in the mainstream evaluation indicators of the saliency detection.

Image saliency detection is a very helpful step in many computer vision-based smart systems to reduce the computational complexity by only focusing on the salient parts of the image. Currently, the image saliency is detected through representation-based generative schemes, as these schemes are helpful for extracting the concise representations of the stimuli and to capture the high-level semantics in visual information with a small number of active coefficients. In this paper, we propose a novel framework for salient region detection that uses appearance-based and regression-based schemes. The framework segments the image and forms reconstructive dictionaries from four sides of the image. These side-specific dictionaries are further utilized to obtain the saliency maps of the sides. A unified version of these maps is subsequently employed by a representation-based model to obtain a contrast-based salient region map. The map is used to obtain two regression-based maps with LAB and RGB color features that are unified through the optimization-based method to achieve the final saliency map. Furthermore, the side-specific reconstructive dictionaries are extracted from the boundary and the background pixels, which are enriched with geometrical and visual information. The approach has been thoroughly evaluated on five datasets and compared with the seven most recent approaches. The simulation results reveal that our model performs favorably in comparison with the current saliency detection schemes.

Biologically inspired visual object tracking has been very popular in the last decades. Several methods have exploited some visual cognitive mechanisms to outperform trackers purely based on machine vision in accuracy and speed. Although they have shown outstanding performance on recent benchmarks, they are still far from achieving the performance of a human in tracking. Visual attention is an important mechanism, which can be applied by saliency detection models. Saliency detection models can be determined in the spatial domain or the frequency domain. Spatial domain based models mostly have complex computations and are time-consuming. Thus, we have improved and recruited Frequency Divisive Normalization (FDN) model as a saliency detection method to propose a bio-inspired object tracking algorithm. This algorithm has been developed based on Fast Fourier Transform (FFT) and uses spectral features in order to achieve a high-speed tracker. It also can be launched on normal hardware. FDN model is based on Feature Integration Theory (FIT) and operates all the processes of saliency detection in the frequency domain, including decomposing the image to the features, lateral inhibition interaction, combining all features, and computing the saliency map. In addition to, extracting color, intensity and orientation it uses spatial frequency features, which is rarely used by other trackers. Extensive evaluations on large-scale benchmark datasets show that the proposed method has good performance with respect to its runtime.