Saliency detection for seismic applications using multi-dimensional spectral projections and directional comparisons

Nowadays, ship detection in sea-sky background is not only useful in maritime visual surveillance, but also helpful in maritime search and rescue. Since ships are salient objects in infrared images with sea-sky background, we present a novel and effective algorithm based on saliency for ship detection in this situation. Our algorithm adopts global saliency, local saliency and background prior to generate saliency maps. Ships are finally segmented in saliency maps. Our algorithm is compared with four classic salient object detection algorithms. And experimental results show our algorithm outperforms the other four algorithms in qualitative and quantitative terms. Keyword list: ship detection ; salient object detection ; infrared image ; sea - sky background . 1. INTRODUCTION Since ships are important vehicles for sea transportation, ship detection in sea-sky background is extensively used in maritime visual surveillance and reconnaissance, avoidance of ship collision accidents, maritime search and rescue, etc. Infrared imaging systems can almost work in any environment and all-weather conditions with its visual capacity in darkness and frog, which makes it a very suitable tool in this application. In recent years, ship detection in infrared images with sea-sky background attracts increasing attention. Many methods are proposed in this field, including image segmentation methods [1-2] and morphological methods [3]. However, due to complex sea-sky environment and large variation of ship sizes, these methods cannot solve the issue completely. And since ships are often salient objects in infrared images with sea-sky background, some researchers have recently tried salient object detection ideas used in color images [4-6]. And the paper is focused on using salient object detection methods to solve the issue. Salient objects are objects which stand out in the image and attract most people ¶VLPPHGLDWHDWWHQWLRQ7KHVWDQGDUGway to detect salient ob MHFWVLVILUVWO\XVLQJVDOLHQF\GHWHFWLRQPHWKRGVWRJHQHUDWHVDOLHQF\PDSVLQZKLFKDSL[HO¶VJUH\value is proportional to its degree of saliency and then segmenting salient objects in saliency maps. Many salient detection methods can be divided into three categories: local contrast methods, global rarity methods and other methods. In local contrast methods, regions which have high contrast with local neighbouring regions are regarded as salient regions. Itti et al. [7] decompose an image into several feature maps . The saliency in each feature map is measured by local center-surround difference in several scales and saliency of different feature maps are integrated to a final saliency

Saliency detection has been widely used to predict human fixation. In this paper, a Visual Saliency Detection Algorithm in Compressed HEVC Domain is proposed which consists of three parts: static saliency detection, dynamic saliency detection and competitive fusion. Firstly, the Gauss model is used to filter out the background of the static features which are extracted by down-sampling and DCT. Secondly, the motion vectors are used to represent the dynamic feature. Then the dynamic saliency is calculated by filtering out the background of dynamic feature. Finally, the competitive fusion model is used to adaptively combine the characteristic of static and dynamic saliency maps. Experimental results show that the proposed method is superior to classic state-of-the-art saliency detection methods with 0.05 AUC value increasing and 0.17 KL divergence decreasing on average. The average time of one frame detection is 2.3 seconds.

There are various saliency detection methods have been proposed recent years. These methods can often complement each other so combining them in appropriate way will be an effective solution of saliency analysis. Existing aggregation methods assigned weights to each entire saliency map, ignoring that features perform differently in certain parts of an image and their gaps between distinguishing the foreground from the backgrounds. In this work, we present a Bayesian probability based framework for multi-feature aggregation. We address saliency detection as a two-class classification problem. Saliency maps generated from each feature have been decomposed into pixels. By the statistic results of different saliency value’s reliability on foreground and background detection, we can generate an accurate, uniform and per-pixel saliency mask without any manual set parameters. This approach can significantly suppress feature’s misclassification while preserve their sensitivity on foreground or background. Experiment on public saliency benchmarks show that our method achieves equal or better results than all state-of-the-art approaches. A new dataset contains 1500 images with human labeled ground truth is also constructed.

Deep (machine) learning in recent years has significantly increased the predictive modeling strengths for applications in the areas of computer vision, speech recognition, and natural language processing. Being heavily data-driven, conventional general-purpose deep learning methods mainly rely on training data to learn task-specific data representations. In this thesis, we focus on leveraging novel learning algorithms in deep learning to tackle visual recognition challenges at the three key levels -- namely pixel, object, and image levels. Throughout the thesis, we demonstrate how each and every of our proposed methods can address a specific visual recognition problem. Firstly, we apply deep learning to improve the saliency detection performance of deep networks. A coarse saliency map of the input image's salient object is initially predicted by a convolutional-deconvolutional network (or equivalently a fully convolutional network). Following that, the coarse saliency map is sequentially refined by a separate network called Recurrent Attentional Convolutional-Deconvolutional Network (RACDNN) that shares its network weights across its sequential steps. At every sequential (temporal) step, the same RACDNN is applied to a newly selected/attended image sub-region and it performs saliency map refinement for that particular sub-region. Despite having many learnable network parameters and no access to additional training data, RADCNN can be trained well and it outperforms the "single-step" (coarse saliency detection) baseline and state-of-the-art saliency detection methods by large margins, on several saliency detection benchmark datasets. Secondly, we introduce auto-encoder weight transfer networks (AE-WTN+), a deep model for scaling up object detection. Conventional WTN transfers object class knowledge from the classification weights of a pretrained large-scale image classification network, to the classification weights of an object detection network. However, they suffer from under-fitting and are not able to preserve the rich object class information of novel classes. In this work, we propose a new WTN method -- AE-WTN+ which makes use of normalization techniques to ease network optimization. Inputs and intermediate features are normalized to improve trainability for reduced under-fitting. More importantly, a novel auto-encoding (reconstruction) loss is proposed to encourage information preservation of all classes in the weight predictions of AE-WTN+. This enables the predicted weights of AE-WTN+ to maintain the class discriminability and neighbourhood relationships contained in the pre-trained large-scale classification weights. Experiments on large-scale detection datasets validate the effectiveness of AE-WTN+ on both seen and novel classes. Thirdly, we improve the efficiency of cross-layer network connections for very deep convolutional network in the context of image classification. This gives rise to a new convolutional network architecture called Deluge Networks (DelugeNets) which pass the output (feature activations) of any preceding composite network layer as direct inputs to all succeeding composite layers. For the incoming feature activations of preceding composite layers, the succeeding composite layers first apply rather lightweight cross-layer depthwise convolutional weights on them for feature aggregation. The heavy lifting is carried out by the last convolutional weight layer in each preceding composite layer that performs cross-channel convolutions. The proposed technique enables unimpeded information flow among many layers in very deep DelugeNets with great efficiency. Experiments on small-scale and large-scale datasets show that DelugeNets perform competitively to state-of-the-art deep convolutional networks at lower computational and parameter costs. Fourthly, we propose Stochastic Downsampling (SDPoint), a method to overcome a major limitation suffered by conventional deep convolutional networks -- their computational costs during inference are fixed and cannot be changed. SDPoint downsamples the feature maps at a randomly selected point (layer index) in the network hierarchy during training. The many downsampling configurations known as SDPoint instances each entail a unique downsampling point and downsampling ratio. The same convolutional network weights are shared across the different SDPoint instances that downsample feature maps differently. During inference, the computational cost of a network trained with SDPoint can be instantaneously adjusted to fit a given computational budget, by manually selecting an appropriate SDPoint instance. Additionally, sharing weights between different feature map scales provides significant regularization benefits, making the networks trained with SDPoint less sensitive to scales and more competent. In image classification experiments, SDPoint demonstrates significant cost-accuracy and computational advantages, over independently-trained (non-weight-shared) baselines with various inference costs.

The background information is full of essential clues which could be used to distinguish the foreground from images, especially when images contain multiple targets or complex backgrounds. In this paper, we formulate the saliency detection task as a labelling problem. We propose a novel saliency detection method via fusing a set of features based on background information. We firstly extract background features referred to as uniqueness feature, dense feature, and sparse feature. Specifically, uniqueness feature is defined using the color distinction and spatial distance based on the K-means algorithm; dense feature of the background segments is calculated by the PCA algorithm; sparse feature is computed based on the sparse encode algorithm. Then we fuse these background features under the CRF frame. Finally, we evaluate our proposed method on a new constructed dataset from THUS10000, SOD and ECSSD datasets to cover different scenarios. The experimental results show that our method can be well against the previous methods in terms of precision and recall.

Mesh surface saliency detection is an important preprocessing step for many 3D applications. The salient region can be used to find the objects that are important on 3D surface, the benefits of saliency detection in the 3D domain include mesh simplification, registration, segmentation, compression, etc. This paper proposes a novel saliency detection method by diffusing the shape index field with non-local means filer, generating a random centre surround operator to yield saliency map and enhancing the saliency with the Retinex theory. The effectiveness of this method is demonstrated by simplification and registration. Experimental results demonstrate that the proposed approach has achieved competitive results.

Saliency detection has a great significance in improving image analysis and processing techniques. In this paper, we propose a center-focus global contrast saliency detection method. First, we achieve the weight of focus-center feature map from frequency domain and get the salient center by weighted coordinates; Secondly we calculate the Euclidean distance between the image of each pixel and the saliency center for generating the focus-center feature map; Then the image is segmented by SLIC, and taking segments as a unit build the color global contrast feature map; Finally three feature maps are normalized and integrated as the saliency map. The saliency detection method proposed in this paper compare with 7 saliency detection methods by the public database Achtana1000 [12] and our method have higher precision and recall than others.

Saliency map can express the position of eye trace, which is a key factor to evaluate the perception quality of video or image. Many saliency detection methods have been proposed based on the human vision system characteristics, but they are time-consuming or with low accuracy. Spectrum residual based algorithm is one typical saliency detection method. It calculates the saliency map of image effectively. The SR method can't detect the saliency in video sequences due to the intrinsic property difference between image and video. In this paper, we propose a saliency detection method for video. According to the extensive simulation results, we find amplitude difference and phase difference are very important for saliency map detection. On the basis of the spectrum features of the adjacent frames in video, we take spectrum difference between current test frame and its adjacent frame into account to calculate the saliency map. The proposed algorithm using the video sequences with one eye track point. The results indicate that the method proposed in this paper is more effective than others.

Saliency detection is used to identify the most important and informative area in a scene, and it is widely used in various vision tasks, including image quality assessment, image matching, and object recognition. Manifold ranking (MR) has been used to great effect for the saliency detection, since it not only incorporates the local spatial information but also utilizes the labeling information from background queries. However, MR completely ignores the feature information extracted from each superpixel. In this paper, we propose an MR-based matrix factorization (MRMF) method to overcome this limitation. MRMF models the ranking problem in the matrix factorization framework and embeds query sample labels in the coefficients. By incorporating spatial information and embedding labels, MRMF enforces similar saliency values on neighboring superpixels and ranks superpixels according to the learned coefficients. We prove that the MRMF has good generalizability, and develops an efficient optimization algorithm based on the Nesterov method. Experiments using popular benchmark data sets illustrate the promise of MRMF compared with the other state-of-the-art saliency detection methods.

The ability of human visual system to detect visual saliency is extraordinarily fast and reliable. However, computational modeling of this basic intelligent behavior still remains a challenge. This paper presents a simple method for the visual saliency detection. Our model is independent of features, categories, or other forms of prior knowledge of the objects. By analyzing the log-spectrum of an input image, we extract the spectral residual of an image in spectral domain, and propose a fast method to construct the corresponding saliency map in spatial domain. We test this model on both natural pictures and artificial images such as psychological patterns. The result indicate fast and robust saliency detection of our method.

Due to complex lighting circumstances and other factors, visual saliency recognition of underwater images is not as good as visual saliency detection models that function well on land when applied to underwater environments. The dearth of large-scale annotated underwater picture datasets has further hindered the development of visual saliency detection in underwater images. we propose an attention-based approach for underwater image saliency detection to address this issue. The backbone network is pre-trained using both land and underwater image datasets, which effectively solves the problem of fewer datasets. After that, a residual refinement module incorporating a channel-space attention mechanism is merged, the model as a whole is trained end-to-end using underwater image datasets, and the model accuracy is experimentally tested. The experimental results in three underwater image datasets show that our method is better than several other methods in the mainstream evaluation indicators of the saliency detection.

The past decade has witnessed the popularity of video conferencing, such as FaceTime and Skype. In video conferencing, almost every frame has a human face. Hence, it is necessary to predict attention on face videos by saliency detection, as saliency can be used as a guidance of regionof- interest (ROI) for the content-based applications. To this end, this paper proposes a novel approach for saliency detection in single-face videos. From the data-driven perspective, we first establish an eye tracking database which contains fixations of 70 single-face videos viewed by 40 subjects. Through analysis on our database, we investigate that most attention is attracted by face in videos, and that attention distribution within a face varies with regard to face size and mouth movement. Inspired by the previous work which applies Gaussian mixture model (GMM) for face saliency detection in still images, we propose to model visual attention on face region for videos by dynamic GMM (DGMM), the variation of which relies on face size, mouth movement and facial landmarks. Then, we develop a long shortterm memory (LSTM) neural network in estimating DGMM for saliency detection of single-face videos, so called LSTM-DGMM. Finally, the experimental results show that our approach outperforms other state-of-the-art approaches in saliency detection of single-face videos.

This paper presents a novel unmanned aerial vehicle tracking framework. First, hierarchical convolutional neural network features are used to track the object independently in a correlation filter tracking framework. Second, a stability criterion is proposed, which is based on the variance of tracking results of each layer. Next, tracking result is adaptively fused via the variance. Meanwhile, the criterion can be used to measure the quality of tracking results. A saliency detection method is utilized to generate candidate regions when tracking failure occurs. By virtue of this method, our tracking algorithm can robustly cope with appearance changes and prevent drifting issues. Experimental results show that our proposed tracking algorithm performs favorably against state-of-the-art methods on two benchmark datasets.

The success of current deep saliency detection methods heavily depends on the availability of large-scale supervision in the form of per-pixel labeling. Such supervision, while labor-intensive and not always possible, tends to hinder the generalization ability of the learned models. By contrast, traditional handcrafted features based unsupervised saliency detection methods, even though have been surpassed by the deep supervised methods, are generally dataset-independent and could be applied in the wild. This raises a natural question that "Is it possible to learn saliency maps without using labeled data while improving the generalization ability?". To this end, we present a novel perspective to unsupervised saliency detection through learning from multiple noisy labeling generated by "weak" and "noisy" unsupervised handcrafted saliency methods. Our end-to-end deep learning framework for unsupervised saliency detection consists of a latent saliency prediction module and a noise modeling module that work collaboratively and are optimized jointly. Explicit noise modeling enables us to deal with noisy saliency maps in a probabilistic way. Extensive experimental results on various benchmarking datasets show that our model not only outperforms all the unsupervised saliency methods with a large margin but also achieves comparable performance with the recent state-of-the-art supervised deep saliency methods.

Saliency detection technology has been greatly developed and applied in recent years. However, the performance of current methods is not satisfactory in complex scenes. One of the reasons is that the performance improvement is often carried out through utilizing complicated mathematical models and involving multiple features rather than classifying the scene complexity and respectively detecting saliency. To break this unified detection schema for generating better results, we propose a method of scene classification-oriented saliency detection via the modularized prescription in this paper. Different scenes are described by a scene complexity expression model, and they are analyzed and discriminately detected by different pipelines. This process seems like that doctors can tailor the treatment prescriptions when they meet different symptoms. Moreover, two SVM-based classifiers are trained for scene classification and sky region identification, and the proposed sky region discrimination and erase model can be used to efficiently decrease the saliency interference by the high luminance of the background sky regions. Experimental results demonstrate the effectiveness and superiority of the proposed method in both higher precision and better smoothness, especially for detecting in structure complex scenes.