Biologically Inspired Tracking with Frequency Divisive Normalization Model

Saliency mechanism has been considered crucial in the human visual system and helpful to object detection and recognition. This paper addresses an information theoretic model for visual saliency detection. It consists of two steps: first, using the Non-negative Matrix Factorization with sparseness constraints (NMFsc) algorithm to learn the basis functions from a set of randomly sampled natural image patches; and then, applying information theoretic principle to generate the saliency map by the Salient Information (SI) which is calculated from the coefficients represented by basis functions. We compare our model with the previous methods on natural images. Experimental results show that our model performs better than existing approaches.

The past decade has witnessed the popularity of video conferencing, such as FaceTime and Skype. In video conferencing, almost every frame has a human face. Hence, it is necessary to predict attention on face videos by saliency detection, as saliency can be used as a guidance of regionof- interest (ROI) for the content-based applications. To this end, this paper proposes a novel approach for saliency detection in single-face videos. From the data-driven perspective, we first establish an eye tracking database which contains fixations of 70 single-face videos viewed by 40 subjects. Through analysis on our database, we investigate that most attention is attracted by face in videos, and that attention distribution within a face varies with regard to face size and mouth movement. Inspired by the previous work which applies Gaussian mixture model (GMM) for face saliency detection in still images, we propose to model visual attention on face region for videos by dynamic GMM (DGMM), the variation of which relies on face size, mouth movement and facial landmarks. Then, we develop a long shortterm memory (LSTM) neural network in estimating DGMM for saliency detection of single-face videos, so called LSTM-DGMM. Finally, the experimental results show that our approach outperforms other state-of-the-art approaches in saliency detection of single-face videos.

Many image processing algorithms are implemented in a combination of spatial and frequency domains. The fast Fourier transform (FFT) is the workhorse of such algorithms. One limitation of the FFT is artefacts that result from the implicit periodicity within the spatial domain. A new periodic plus smooth decomposition has recently been proposed for removing such artefacts, although this comes at the cost of an additional 2D FFT. In this paper, we restructure the decomposition to enable it to be calculated with a single 1D FFT, which can significantly accelerate artefact free Fourier transformation. The cost of this acceleration is a small amount of additional storage to hold the representation of the smooth image component.

In existing superpixel-wise saliency detection algorithms, superpixel generation often is an isolated preprocessing step. The performance of saliency maps is determined by the accuracy of superpixels to a certain extent. However, it is still a challenge to develop a stable superpixel generation method. In this article, we attempt to incorporate the superpixel generation and saliency calculation steps into an end-to-end trainable deep network. First, we employ a recently proposed differentiable superpixel generation method to over-segment the synthetic aperture radar (SAR) images, which outputs the possibility that the pixels assigned to neighbor superpixels (soft superpixel). In saliency calculation part, as one of our main contributions, we propose a differentiable and computationally simple saliency model, i.e., lacunarity cue. It is inspired by the fact that generally the backscattering intensity of regions of interest (ROIs) in SAR images irregularly fluctuates, while the areas with consistent pixels are often ignored as the clusters. We improve the pixelwise box differential dimension algorithm to measure the irregularity of scattering points in a superpixel. The superpixel generation and saliency calculation can be implemented under a unified deep network. Hence, the shapes of the superpixels can be iteratively adjusted according to the saliency maps until the ROIs are correctly detected. Experiments on real SAR images with different sizes and scenes show that the saliency maps can effectively highlight the target areas, thus outperforming the state-of-the-art saliency detection models.

The effectiveness of depth information in saliency detection has been fully proved. However, it is still worth exploring how to utilize the depth information more efficiently. Erroneous depth information may cause detection failure, while non-salient objects may be closer to the camera which also leads to erroneously emphasis on non-salient regions. Moreover, most of the existing RGB-D saliency detection models have poor robustness when the salient object touches the image boundaries. To mitigate these problems, we propose a multi-stage saliency detection model with the bilateral absorbing Markov chain guided by depth information. The proposed model progressively extracts the saliency cues with three level (low-, mid-, and high-level) stages. First, we generate low-level saliency cues by explicitly combining color and depth information. Then, we design a bilateral absorbing Markov chain to calculate mid-level saliency maps. In mid-level, to suppress boundary touch problem, we present the background seed screening mechanism (BSSM) for improving the construction of the two-layer sparse graph and better selecting background-based absorbing nodes. Furthermore, the cross-modal multi-graph learning model (CMLM) is designed to fully explore the intrinsic complementary relationship between color and depth information. Finally, to obtain a more highlighted and homogeneous saliency map in high-level, we structure a depth-guided optimization module which combines cellular automata and suppression-enhancement function pair. This optimization module refines the saliency map in color space and depth space, respectively. Comprehensive experiments on three challenging benchmark datasets demonstrate the effectiveness of our proposed method both qualitatively and quantitatively.

Data-driven saliency detection has attracted strong interest as a result of applying convolutional neural networks to the detection of eye fixations. Although a number of image-based salient object and fixation detection models have been proposed, video fixation detection still requires more exploration. Different from image analysis, motion and temporal information is a crucial factor affecting human attention when viewing video sequences. Although existing models based on local contrast and low-level features have been extensively researched, they failed to simultaneously consider interframe motion and temporal information across neighboring video frames, leading to unsatisfactory performance when handling complex scenes. To this end, we propose a novel and efficient video eye fixation detection model to improve the saliency detection performance. By simulating the memory mechanism and visual attention mechanism of human beings when watching a video, we propose a step-gained fully convolutional network by combining the memory information on the time axis with the motion information on the space axis while storing the saliency information of the current frame. The model is obtained through hierarchical training, which ensures the accuracy of the detection. Extensive experiments in comparison with 11 state-of-the-art methods are carried out, and the results show that our proposed model outperforms all 11 methods across a number of publicly available datasets.

In this paper, a novel and efficient framework by exploiting Quaternionic Distance Based Weber Local Descriptor (QDWLD) and object cues is proposed for image saliency detection. In contrast to the existing saliency detection models, the advantage of the proposed approach is that it can combine quaternion number system and object cues simultaneously, which is independent of image contents and scenes. Firstly, QDWLD, which was initially designed for detecting outliers in color images, is used to represent the directional cues in an image. Meanwhile, two low-level priors, namely the Convex-Hull-Based center and color contrast cue of the image, are utilized and fused as an object-level cue. Finally, by combining QDWLD with object cues, a reliable saliency map of the image can be computed and estimated. Experimental results, based on two widely used and openly available database, show that the proposed method is able to produce reliable and promising salient maps/estimations, compared to other state-of-the-art saliency-detection models.

Frequency-tuned saliency detection analyzes image saliency from the perspective of frequency domain and fully combines image segmentation method, which outputs well- defined boundaries of salient objects. However, the method ignores spatial relationships across image parts. This paper proposes an improved saliency detection method on the basis of the frequency-tuned method. In this method, we first segment the input image into regions and then analyze the image from the frequency domain. After that, we preprocess it using Gaussian filter to eliminate noise and coding artifacts. For each region, we can get saliency map in region-level based on region contrast. Finally, salient regions are selected by winner-take-all (WTA) neural network and Inhibition of Return(IOR) mechanism. The proposed salient region detection algorithm combines the virtues of frequency-tuning and region contrast. The experimental results show the feasibility and validity of this algorithm. Keywords-Image retrieval; Semantic gap; Saliency detection; Region-level saliency map; Visual attention

<p>For a diverse range of applications in machine vision from social media searches to robotic home care providers, it is important to replicate the mechanism by which the human brain selects the most important visual information, while suppressing the remaining non-usable information. Many computational methods attempt to model this process by following the traditional model of visual attention. The traditional model of attention involves feature extraction, conditioning and combination to capture this behaviour of human visual attention. Consequently, the model has inherent design choices at its various stages. These choices include selection of parameters related to the feature computation process, setting a conditioning approach, feature importance and setting a combination approach. Despite rapid research and substantial improvements in benchmark performance, the performance of many models depends upon tuning these design choices in an ad hoc fashion. Additionally, these design choices are heuristic in nature, thus resulting in good performance only in certain settings. Consequentially, many such models exhibit low robustness to difficult stimuli and the complexities of real-world imagery. Machine learning and optimisation technique have long been used to increase the generalisability of a system to unseen data. Surprisingly, artificial learning techniques have not been investigated to their full potential to improve generalisation of visual attention methods. The proposed thesis is that artificial learning can increase the generalisability of the traditional model of visual attention by effective selection and optimal combination of features. The following new techniques have been introduced at various stages of the traditional model of visual attention to improve its generalisation performance, specifically on challenging cases of saliency detection: 1. Joint optimisation of feature related parameters and feature importance weights is introduced for the first time to improve the generalisation of the traditional model of visual attention. To evaluate the joint learning hypothesis, a new method namely GAOVSM is introduced for the tasks of eye fixation prediction. By finding the relationships between feature related parameters and feature importance, the developed method improves the generalisation performance of baseline method (that employ human encoded parameters). 2. Spectral matting based figure-ground segregation is introduced to overcome the artifacts encountered by region-based salient object detection approaches. By suppressing the unwanted background information and assigning saliency to object parts in a uniform manner, the developed FGS approach overcomes the limitations of region based approaches. 3. Joint optimisation of feature computation parameters and feature importance weights is introduced for optimal combination of FGS with complementary features for the first time for salient object detection. By learning feature related parameters and their respective importance at multiple segmentation thresholds and by considering the performance gaps amongst features, the developed FGSopt method improves the object detection performance of the FGS technique also improving upon several state-of-the-art salient object detection models. 4. The introduction of multiple combination schemes/rules further extends the generalisability of the traditional attention model beyond that of joint optimisation based single rules. The introduction of feature composition based grouping of images, enables the developed IGA method to autonomously identify an appropriate combination strategy for an unseen image. The results of a pair-wise ranksum test confirm that the IGA method is significantly better than the deterministic and classification based benchmark methods on the 99% confidence interval level. Extending this line of research, a novel relative encoding approach enables the adapted XCSCA method to group images having similar saliency prediction ability. By keeping track of previous inputs, the introduced action part of the XCSCA approach enables learning of generalised feature importance rules. By more accurate grouping of images as compared with IGA, generalised learnt rules and appropriate application of feature importance rules, the XCSCA approach improves upon the generalisation performance of the IGA method. 5. The introduced uniform saliency assignment and segmentation quality cues enable label free evaluation of a feature/saliency map. By accurate ranking and effective clustering, the developed DFS method successfully solves the complex problem of finding appropriate features for combination (on an-image-by-image basis) for the first time in saliency detection. The DFS method enables ground truth free evaluation of saliency methods and advances the state-of-the-art in data driven saliency aggregation by detection and deselection of redundant information. The final contribution is that the developed methods are formed into a complete system where analysis shows the effects of their interactions on the system. Based on the saliency prediction accuracy versus computational time trade-off, specialised variants of the proposed methods are presented along with the recommendations for further use by other saliency detection systems. This research work has shown that artificial learning can increase the generalisation of the traditional model of attention by effective selection and optimal combination of features. Overall, this thesis has shown that it is the ability to autonomously segregate images based on their types and subsequent learning of appropriate combinations that aid generalisation on difficult unseen stimuli.</p>

Though there are many computational models proposed for saliency detection, few of them take object boundary information into account. This paper presents a hierarchical saliency detection model incorporating probabilistic object boundaries, which is based on the observation that salient objects are generally surrounded by explicit boundaries and show contrast with their surroundings. We perform adaptive thresholding operation on ultrametric contour map, which leads to hierarchical image segmentations, and compute the saliency map for each layer based on the proposed robust center bias, border bias, color dissimilarity and spatial coherence measures. After a linear weighted combination of multi-layer saliency maps, and Bayesian enhancement procedure, the final saliency map is obtained. Extensive experimental results on three challenging benchmark datasets demonstrate that the proposed model outperforms eight state-of-the-art saliency detection models.

Salient regions usually carry important information in images. Existing feature encryption algorithms aim at extracting edge features as significant information rather than salient regions for encryption purpose. Moreover, most of them protect significant information by transforming the input image into texture-like or noise-like encrypted image which is obviously a visual sign of encrypted image, and thus can be easily attacked. In this paper, we propose a salient regions encryption scheme to generate visually meaningful ciphertext. First, salient regions are efficiently extracted by a saliency detection model in the compressed domain. Then we pre-encrypt these salient regions by a chaos-based encryption algorithm. With optical encryption theory, the pre-encrypted salient regions are finally transformed into a visually meaningful ciphertext. To the best of our knowledge, it is the first time to use salient regions as important visual information for encryption to obtain cipertext in images. The experimental results demonstrate that the salient regions can be largely hidden with the proposed method.

Saliency detection is an effective tool to extract regions of interest (ROIs) from remote sensing images. However, some existing saliency detection models focus on extracting ROI from a single image, which cannot accurately detect ROI against complex background interference. In this paper, a novel visual saliency analysis and ROI extraction model is proposed to effectively extract common ROIs from remote sensing images and exclude images without ROIs. Firstly, the single saliency maps are generated by frequency-tuned (FT) method. Secondly, the cluster method based on synthesized features is proposed to group regions with similar feature into a cluster for multiple images. Thirdly, computing the mean of saliency value as the cluster saliency suppresses the saliency value of non-common ROIs. Finally, a ROI extraction method based on the maximum saliency value is proposed to extract ROIs while eliminating the image without ROIs. Experimental results indicate our model outperforms other state-of-the-art saliency detection models, achieving highest ROC and maximal PRF values.

Visual saliency detection based on homology similarity and an experimental evaluation

Saliency detection is an active research field in computer vision in recent years. As RGB-D sensors are more and more widely used in a robot system, the demand of corresponding saliency detection datasets and algorithms are growing rapidly. In this paper, we built a RGB-D saliency detection dataset NJUSTDS1000 contains 1000 real RGB-D scenes. The labeling of saliency ground truth of this dataset is based on color and depth fixation map. Then we propose a spectral and spatial analysis based RGB-D saliency detection model. It uses quaternion to present multi-channel features and dose fast saliency detection based on spectral analysis. Then a two-steps scale adaptive saliency fusion process is carried out in spatial domain, which are scale adaptive superpixel based saliency smoothing and multi-layer cellular automata based saliency maps fusion. We validate the proposed model on NJUSTDS 1000 and MSRA10K.

Multi-scale phase spectrum based salient target detection approach applied under sea surface environment is proposed in this paper. Visual attention models based on frequency domain are summarized and analyzed, on the basis of which a framework of visual attention approach using multi-scale phase spectrum is presented. Under the sea surface environment, multi-scale color and intensity channels are chose for salient target detection. The saliency map of sea surface image is obtained by fusion of color and intensity saliency maps from multi-scale phase spectrum. The proposed method is easier to implement than spectral residual based approaches and it can detect different size targets simultaneously. Results of experiments show satisfied performance of the new method.