Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective

Nowadays, ship detection in sea-sky background is not only useful in maritime visual surveillance, but also helpful in maritime search and rescue. Since ships are salient objects in infrared images with sea-sky background, we present a novel and effective algorithm based on saliency for ship detection in this situation. Our algorithm adopts global saliency, local saliency and background prior to generate saliency maps. Ships are finally segmented in saliency maps. Our algorithm is compared with four classic salient object detection algorithms. And experimental results show our algorithm outperforms the other four algorithms in qualitative and quantitative terms. Keyword list: ship detection ; salient object detection ; infrared image ; sea - sky background . 1. INTRODUCTION Since ships are important vehicles for sea transportation, ship detection in sea-sky background is extensively used in maritime visual surveillance and reconnaissance, avoidance of ship collision accidents, maritime search and rescue, etc. Infrared imaging systems can almost work in any environment and all-weather conditions with its visual capacity in darkness and frog, which makes it a very suitable tool in this application. In recent years, ship detection in infrared images with sea-sky background attracts increasing attention. Many methods are proposed in this field, including image segmentation methods [1-2] and morphological methods [3]. However, due to complex sea-sky environment and large variation of ship sizes, these methods cannot solve the issue completely. And since ships are often salient objects in infrared images with sea-sky background, some researchers have recently tried salient object detection ideas used in color images [4-6]. And the paper is focused on using salient object detection methods to solve the issue. Salient objects are objects which stand out in the image and attract most people ¶VLPPHGLDWHDWWHQWLRQ7KHVWDQGDUGway to detect salient ob MHFWVLVILUVWO\XVLQJVDOLHQF\GHWHFWLRQPHWKRGVWRJHQHUDWHVDOLHQF\PDSVLQZKLFKDSL[HO¶VJUH\value is proportional to its degree of saliency and then segmenting salient objects in saliency maps. Many salient detection methods can be divided into three categories: local contrast methods, global rarity methods and other methods. In local contrast methods, regions which have high contrast with local neighbouring regions are regarded as salient regions. Itti et al. [7] decompose an image into several feature maps . The saliency in each feature map is measured by local center-surround difference in several scales and saliency of different feature maps are integrated to a final saliency

The advent of new image reconstruction and image processing techniques for CT images has increased the need for robust objective image quality assessment methods. One of the most common quality assessment methods is the measurement of signal detectability for a known signal at a known location using supervised classification techniques. However, this method requires a large number of simulations or physical measurements, and its underlying assumptions may be considered clinically unrealistic. In this study we focus on objective assessment of image quality in terms of detection of a signal with unknown location, size, shape, and contrast. We explore several unsupervised saliency detection methods which assume no knowledge about the signal, along with a template matching technique which uses information about the signal's size and shape in the object domain, for simulated phantoms that have been reconstructed using filtered back projection (FBP) and iterative reconstruction algorithms (IRA). The performance of each of the image reconstruction algorithms is then measured using the area under the localization receiver operating characteristic curve (LROC) and exponential transformation of the free response operating characteristic curve (EFROC). Our results indicate that unsupervised saliency detection methods can be effectively used to determine image quality in terms of signal detectability for unknown signals given only a small number of sample images.

Deep (machine) learning in recent years has significantly increased the predictive modeling strengths for applications in the areas of computer vision, speech recognition, and natural language processing. Being heavily data-driven, conventional general-purpose deep learning methods mainly rely on training data to learn task-specific data representations. In this thesis, we focus on leveraging novel learning algorithms in deep learning to tackle visual recognition challenges at the three key levels -- namely pixel, object, and image levels. Throughout the thesis, we demonstrate how each and every of our proposed methods can address a specific visual recognition problem. Firstly, we apply deep learning to improve the saliency detection performance of deep networks. A coarse saliency map of the input image's salient object is initially predicted by a convolutional-deconvolutional network (or equivalently a fully convolutional network). Following that, the coarse saliency map is sequentially refined by a separate network called Recurrent Attentional Convolutional-Deconvolutional Network (RACDNN) that shares its network weights across its sequential steps. At every sequential (temporal) step, the same RACDNN is applied to a newly selected/attended image sub-region and it performs saliency map refinement for that particular sub-region. Despite having many learnable network parameters and no access to additional training data, RADCNN can be trained well and it outperforms the "single-step" (coarse saliency detection) baseline and state-of-the-art saliency detection methods by large margins, on several saliency detection benchmark datasets. Secondly, we introduce auto-encoder weight transfer networks (AE-WTN+), a deep model for scaling up object detection. Conventional WTN transfers object class knowledge from the classification weights of a pretrained large-scale image classification network, to the classification weights of an object detection network. However, they suffer from under-fitting and are not able to preserve the rich object class information of novel classes. In this work, we propose a new WTN method -- AE-WTN+ which makes use of normalization techniques to ease network optimization. Inputs and intermediate features are normalized to improve trainability for reduced under-fitting. More importantly, a novel auto-encoding (reconstruction) loss is proposed to encourage information preservation of all classes in the weight predictions of AE-WTN+. This enables the predicted weights of AE-WTN+ to maintain the class discriminability and neighbourhood relationships contained in the pre-trained large-scale classification weights. Experiments on large-scale detection datasets validate the effectiveness of AE-WTN+ on both seen and novel classes. Thirdly, we improve the efficiency of cross-layer network connections for very deep convolutional network in the context of image classification. This gives rise to a new convolutional network architecture called Deluge Networks (DelugeNets) which pass the output (feature activations) of any preceding composite network layer as direct inputs to all succeeding composite layers. For the incoming feature activations of preceding composite layers, the succeeding composite layers first apply rather lightweight cross-layer depthwise convolutional weights on them for feature aggregation. The heavy lifting is carried out by the last convolutional weight layer in each preceding composite layer that performs cross-channel convolutions. The proposed technique enables unimpeded information flow among many layers in very deep DelugeNets with great efficiency. Experiments on small-scale and large-scale datasets show that DelugeNets perform competitively to state-of-the-art deep convolutional networks at lower computational and parameter costs. Fourthly, we propose Stochastic Downsampling (SDPoint), a method to overcome a major limitation suffered by conventional deep convolutional networks -- their computational costs during inference are fixed and cannot be changed. SDPoint downsamples the feature maps at a randomly selected point (layer index) in the network hierarchy during training. The many downsampling configurations known as SDPoint instances each entail a unique downsampling point and downsampling ratio. The same convolutional network weights are shared across the different SDPoint instances that downsample feature maps differently. During inference, the computational cost of a network trained with SDPoint can be instantaneously adjusted to fit a given computational budget, by manually selecting an appropriate SDPoint instance. Additionally, sharing weights between different feature map scales provides significant regularization benefits, making the networks trained with SDPoint less sensitive to scales and more competent. In image classification experiments, SDPoint demonstrates significant cost-accuracy and computational advantages, over independently-trained (non-weight-shared) baselines with various inference costs.

Saliency map can express the position of eye trace, which is a key factor to evaluate the perception quality of video or image. Many saliency detection methods have been proposed based on the human vision system characteristics, but they are time-consuming or with low accuracy. Spectrum residual based algorithm is one typical saliency detection method. It calculates the saliency map of image effectively. The SR method can't detect the saliency in video sequences due to the intrinsic property difference between image and video. In this paper, we propose a saliency detection method for video. According to the extensive simulation results, we find amplitude difference and phase difference are very important for saliency map detection. On the basis of the spectrum features of the adjacent frames in video, we take spectrum difference between current test frame and its adjacent frame into account to calculate the saliency map. The proposed algorithm using the video sequences with one eye track point. The results indicate that the method proposed in this paper is more effective than others.

In the process of intelligentization in modern manufacturing, especially in industrial fields such as automobile manufacturing, semiconductor production, and electronic product assembly, product quality control is crucial. Traditional defect detection methods face problems such as supervised learning methods relying on a large amount of labeled data, weak generalization ability, and high cost. In the process of intelligent manufacturing, industrial product quality control is a key link to ensure production safety and product consistency. Especially in typical industrial scenarios such as automobile manufacturing, semiconductor production, and electronic product assembly, traditional defect detection methods are difficult to meet the needs of actual production lines for high-precision and high-efficiency detection due to their reliance on a large amount of labeled data, weak generalization ability, and high cost. To solve the problem of defect detection in small and zero sample scenarios in these industries, improve detection sensitivity and localization accuracy, and enhance the model's generalization ability to unknown defects, a unsupervised industrial image defect detection method based on autoencoder and Generative Adversarial Networks (GANs) is proposed. This study constructs an Multi-level Deep feature Adaptive fusion AutoEncoder (MDAAE) module, extracts multi-scale features through Pre-Trained Convolutional Neural Backbone Networks (PTCNBN), introduces Attention Mechanism (AM) to dynamically calculate feature weights, and achieves feature fusion and reconstruction. Meanwhile, the self-AM is fused to improve the GANs. The self-attention module efficiently captures long-range dependencies and generates an adversarial network training objective function to optimize the generator's unsaturated loss. The results showed that the Area Under the Curve (AUC) of the Recall-Precision curve of the research method reached 93.6 ± 0.5%, and its F1-Score value exceeded 0.890 ± 0.003 in defect types such as scratches and dents. In practical applications, the inference time of the research method remained stable at 3.0 ± 0.2 GB of central processor memory, and the Fréchet Inception Distance (FID) value fully converged at the 2230th iteration, with a stable FID value of 3. The false alarm rate was only 8.6 ± 0.7% under strong light conditions. The proposed UIIDD method based on autoencoder and GANs had good robustness, generalization ability, reliability, and efficiency. This study effectively solves the problems of poor robustness, weak generalization ability, and high cost of traditional defect detection methods.

Saliency detection has a great significance in improving image analysis and processing techniques. In this paper, we propose a center-focus global contrast saliency detection method. First, we achieve the weight of focus-center feature map from frequency domain and get the salient center by weighted coordinates; Secondly we calculate the Euclidean distance between the image of each pixel and the saliency center for generating the focus-center feature map; Then the image is segmented by SLIC, and taking segments as a unit build the color global contrast feature map; Finally three feature maps are normalized and integrated as the saliency map. The saliency detection method proposed in this paper compare with 7 saliency detection methods by the public database Achtana1000 [12] and our method have higher precision and recall than others.

There are various saliency detection methods have been proposed recent years. These methods can often complement each other so combining them in appropriate way will be an effective solution of saliency analysis. Existing aggregation methods assigned weights to each entire saliency map, ignoring that features perform differently in certain parts of an image and their gaps between distinguishing the foreground from the backgrounds. In this work, we present a Bayesian probability based framework for multi-feature aggregation. We address saliency detection as a two-class classification problem. Saliency maps generated from each feature have been decomposed into pixels. By the statistic results of different saliency value’s reliability on foreground and background detection, we can generate an accurate, uniform and per-pixel saliency mask without any manual set parameters. This approach can significantly suppress feature’s misclassification while preserve their sensitivity on foreground or background. Experiment on public saliency benchmarks show that our method achieves equal or better results than all state-of-the-art approaches. A new dataset contains 1500 images with human labeled ground truth is also constructed.

Construction of saliency map in multimedia data is useful for applications in multimedia like object segmentation, quality assessment, and object recognition. In this paper, we propose a novel saliency map model called Gabor & Curvelets based Saliency Map (GCSMP) relying on 2D Gabor and Curvelet transforms. Compared with the traditional model based on DOG and wavelets, our model takes advantage of Garbor transforms's spatial localization and Curvelet transform's edge and directional information. We also discuss the influence of center bias and object detectors in our model. Empirical validations on standard dataset demonstrate the effectiveness of the proposed technique.

The ability of human visual system to detect visual saliency is extraordinarily fast and reliable. However, computational modeling of this basic intelligent behavior still remains a challenge. This paper presents a simple method for the visual saliency detection. Our model is independent of features, categories, or other forms of prior knowledge of the objects. By analyzing the log-spectrum of an input image, we extract the spectral residual of an image in spectral domain, and propose a fast method to construct the corresponding saliency map in spatial domain. We test this model on both natural pictures and artificial images such as psychological patterns. The result indicate fast and robust saliency detection of our method.

Saliency and co-saliency detection aim to distinguish conspicuous foreground objects from single and multiple images, thus are essential in many multimedia and vision applications. To achieve balanced efficiency and accuracy, most recent successful saliency detectors are based on superpixels. However, saliency detection with single-scale superpixel segmentation may fail in capturing intrinsic salient objects of complex natural scenes with small-scale high-contrast backgrounds. To tackle this problem, we present a simple strategy using multiscale superpixels to jointly detect salient object via low-rank optimisation. Specifically, we first build a multiscale superpixel pyramid and derive the corresponding saliency map by multimodal saliency features and priors at each single scale. Then, we use joint low-rank analysis of multiscale saliency maps to obtain a more reliable and adaptively-fused saliency map, which properly takes all scales saliency into account. We further propose a GMM generative co-saliency prior to enable the above approach to detect co-salient objects from multiple images. Extensive experiments on benchmark datasets validate the effectiveness and superiority of the proposed saliency and co-saliency detector over state-of-the-arts.

Salient object detection is a process of extracting an object which is visually attractive from a single image or a video. As a powerful technique for automatic image or video segmentation, saliency detection has been focused and studied recently. In this paper, we propose a novel method for salient object detection without training or learning-based techniques. The proposed framework consists of two major steps, the generation of saliency map candidates and the selection of an optimal saliency map. To generate saliency map candidates, prior maps based on combinations of RGB color components are proposed. To select the optimal saliency map among the candidates, we propose a compactness measure, which evaluates the degree to which the generated saliency maps show objects. As a result, among recent works on saliency detection, our saliency detection method achieves the highest performance in terms of saliency detection.

Saliency detection is a technique for automatically extracting regions of interest from the background and has been widely used in the computer vision field. This study proposes a simple and effective saliency detection method combining color contrast and hash fingerprint. In our solution, the input image is segmented into nonoverlapping superpixels, so as to perform the saliency detection at the region level to reduce computational complexity. A background optimization selection is used to construct an accurate background template. Based on this, a saliency map that highlights the whole salient region is obtained by estimating color contrast. Besides, another saliency map that enhances the salient region while restraining the background is also generated through hash fingerprint matching. Ultimately, the final saliency map can be obtained by fusing the two saliency maps. Comparing the performance with other methods, the proposed algorithm works better even in the presence of complex background or very large salient regions.

Saliency detection is a technique to analyze image surroundings to extract relevant regions from the background. In this paper, we propose a simple and effective saliency detection method based on image sparse representation and color features combination. First, the input image is segmented into non-overlapping super-pixels, so as to perform the saliency detection at the region level to reduce computational complexity. Then, a background optimization selection scheme is used to construct an appropriate background template. Based on this, a primary saliency map is obtained by using image sparse representation. Next, through the linear combination of color coefficients we generate an improved saliency map with more prominent salient regions. Finally, the two saliency maps are integrated within Bayesian framework to obtain the final saliency map. Experimental results show that the proposed method has desirable detection performance in terms of detection accuracy and running time.

Saliency detection has been widely used to predict human fixation. In this paper, a Visual Saliency Detection Algorithm in Compressed HEVC Domain is proposed which consists of three parts: static saliency detection, dynamic saliency detection and competitive fusion. Firstly, the Gauss model is used to filter out the background of the static features which are extracted by down-sampling and DCT. Secondly, the motion vectors are used to represent the dynamic feature. Then the dynamic saliency is calculated by filtering out the background of dynamic feature. Finally, the competitive fusion model is used to adaptively combine the characteristic of static and dynamic saliency maps. Experimental results show that the proposed method is superior to classic state-of-the-art saliency detection methods with 0.05 AUC value increasing and 0.17 KL divergence decreasing on average. The average time of one frame detection is 2.3 seconds.

Stereoscopic perception is an important part of human visual system that allows the brain to perceive depth. However, depth information has not been well explored in existing saliency detection models. In this letter, a novel saliency detection method for stereoscopic images is proposed. Firstly, we propose a measure to evaluate the reliability of depth map, and use it to reduce the influence of poor depth map on saliency detection. Then, the input image is represented as a graph, and the depth information is introduced into graph construction. After that, a new definition of compactness using color and depth cues is put forward to compute the compactness saliency map. In order to compensate the detection errors of compactness saliency when the salient regions have similar appearances with background, foreground saliency map is calculated based on depth-refined foreground seeds selection mechanism and multiple cues contrast. Finally, these two saliency maps are integrated into a final saliency map through weighted-sum method according to their importance. Experiments on two publicly available stereo datasets demonstrate that the proposed method performs better than other 10 state-of-the-art approaches.