Salient Object Detection with Convex Hull Overlap

Despite Convolutional Neural Networks (CNNs) based methods have been successful in detecting salient objects, their underlying mechanism that decides the salient intensity of each image part separately cannot avoid inconsistency of parts within the same salient object. This would ultimately result in an incomplete shape of the detected salient object. To solve this problem, we dig into part-object relationships and take the unprecedented attempt to employ these relationships endowed by the Capsule Network (CapsNet) for salient object detection. The entire salient object detection system is built directly on a Two-Stream Part-Object Assignment Network (TSPOANet) consisting of three algorithmic steps. In the first step, the learned deep feature maps of the input image are transformed to a group of primary capsules. In the second step, we feed the primary capsules into two identical streams, within each of which low-level capsules (parts) will be assigned to their familiar high-level capsules (object) via a locally connected routing. In the final step, the two streams are integrated in the form of a fully connected layer, where the relevant parts can be clustered together to form a complete salient object. Experimental results demonstrate the superiority of the proposed salient object detection network over the state-of-the-art methods.
\n

We provide a comprehensive evaluation of salient object detection (SOD) models. Our analysis identifies a serious design bias of existing SOD datasets which assumes that each image contains at least one clearly outstanding salient object in low clutter. The design bias has led to a saturated high performance for state-of-the-art SOD models when evaluated on existing datasets. The models, however, still perform far from being satisfactory when applied to real-world daily scenes. Based on our analyses, we first identify 7 crucial aspects that a comprehensive and balanced dataset should fulfill. Then, we propose a new high quality dataset and update the previous saliency benchmark. Specifically, our SOC (Salient Objects in Clutter) dataset, includes images with salient and non-salient objects from daily object categories. Beyond object category annotations, each salient image is accompanied by attributes that reflect common challenges in real-world scenes. Finally, we report attribute-based performance assessment on our dataset.

Nowadays, salient object detection (SOD) has become a prominent research area in computer vision, which has various applications in real life. With the advancement of convolutional neural networks (CNN), numerous SOD methods have been proposed to imitate the human visual system to identify salient objects in images. Their marvelous performance comes with large models, high computational complexity, and memory consumption, raising many challenges to deploy these models on practical applications and low-resource devices. Under the high demand for developing efficient models for SOD tasks, in this paper we provide a comprehensive performance analysis focusing on existing CNN-based lightweight SOD models in terms of accuracy and efficiency. By discussing recent methods regarding the proposed ideas and main techniques, we analyze the current issues of lightweight SOD models in detail. Various metrics are considered for performance assessment followed by future research directions and conclusion.

Benefiting from the powerful discriminative feature learning capability of convolutional neural networks (CNNs), deep learning techniques have achieved remarkable performance improvement for the task of salient object detection (SOD) in recent years. However, most existing deep SOD models do not fully exploit informative contextual features, which often leads to suboptimal detection performance in the presence of a cluttered background. This paper presents a context-aware attention module that detects salient objects by simultaneously constructing connections between each image pixel and its local and global contextual pixels. Specifically, each pixel and its neighbors bidirectionally exchange semantic information by computing their correlation coefficients, and this process aggregates contextual attention features both locally and globally. In addition, an attention-guided hierarchical network architecture is designed to capture fine-grained spatial details by transmitting contextual information from deeper to shallower network layers in a top-down manner. Extensive experiments on six public SOD datasets show that our proposed model demonstrates superior SOD performance against most of the current state-of-the-art models under different evaluation metrics.

Salient object detection (SOD) task for optical remote sensing images (RSIs) plays an important role in many remote sensing applications. Most of the existing methods train their networks depending on a large amount of pixel-wise datasets. However, such expensive and time-consuming training setting prevents the approaches becoming flexible and scalable solutions. To this end, we explore efficient SOD for optical RSIs based on easily accessible weak supervision source. In this work, we propose a novel end-to-end progressive attention-based feature recovery framework with scribble supervision. Specifically, to better locate challenging salient objects in optical RSIs, an object position module (OPM) is proposed to capture and enhance the long-range semantic dependence of objects’ position information, which depends on the complementary attention mechanism. And to restore the entire salient objects, a context refinement module (CRM) is proposed, which extract local contextual information for better propagating high-level semantics to low-level details. Moreover, to improve the adaptability of the network to the changing scenarios of optical RSIs, we propose a salient region correcting (SRC) mechanism to help the predicted salient regions rectify their saliency values by constraining the saliency relationship between predictions from different augmentation models. In addition, due to the lack of dataset for weakly supervised SOD for optical RSIs, we relabeled an existing large-scale optical RSIs dataset with scribbles, namely EORSSD-S. Experimental results on benchmark datasets demonstrate that the proposed method can outperform other weakly supervised SOD methods. And the proposed method even outperformed some fully supervised methods. https://github.com/melonless/PAFR.

Nowadays, ship detection in sea-sky background is not only useful in maritime visual surveillance, but also helpful in maritime search and rescue. Since ships are salient objects in infrared images with sea-sky background, we present a novel and effective algorithm based on saliency for ship detection in this situation. Our algorithm adopts global saliency, local saliency and background prior to generate saliency maps. Ships are finally segmented in saliency maps. Our algorithm is compared with four classic salient object detection algorithms. And experimental results show our algorithm outperforms the other four algorithms in qualitative and quantitative terms. Keyword list: ship detection ; salient object detection ; infrared image ; sea - sky background . 1. INTRODUCTION Since ships are important vehicles for sea transportation, ship detection in sea-sky background is extensively used in maritime visual surveillance and reconnaissance, avoidance of ship collision accidents, maritime search and rescue, etc. Infrared imaging systems can almost work in any environment and all-weather conditions with its visual capacity in darkness and frog, which makes it a very suitable tool in this application. In recent years, ship detection in infrared images with sea-sky background attracts increasing attention. Many methods are proposed in this field, including image segmentation methods [1-2] and morphological methods [3]. However, due to complex sea-sky environment and large variation of ship sizes, these methods cannot solve the issue completely. And since ships are often salient objects in infrared images with sea-sky background, some researchers have recently tried salient object detection ideas used in color images [4-6]. And the paper is focused on using salient object detection methods to solve the issue. Salient objects are objects which stand out in the image and attract most people ¶VLPPHGLDWHDWWHQWLRQ7KHVWDQGDUGway to detect salient ob MHFWVLVILUVWO\XVLQJVDOLHQF\GHWHFWLRQPHWKRGVWRJHQHUDWHVDOLHQF\PDSVLQZKLFKDSL[HO¶VJUH\value is proportional to its degree of saliency and then segmenting salient objects in saliency maps. Many salient detection methods can be divided into three categories: local contrast methods, global rarity methods and other methods. In local contrast methods, regions which have high contrast with local neighbouring regions are regarded as salient regions. Itti et al. [7] decompose an image into several feature maps . The saliency in each feature map is measured by local center-surround difference in several scales and saliency of different feature maps are integrated to a final saliency

Salient object detection is evaluated using binary ground truth (GT) with the labels being salient object class and background. In this study, the authors corroborate based on three subjective experiments on a novel image dataset that objects in natural images are inherently perceived to have varying levels of importance. The authors' dataset, named SalMoN (saliency in multi-object natural images), has 588 images containing multiple objects. The subjective experiments performed record spontaneous attention and perception through eye fixation duration, point clicking and rectangle drawing. As object saliency in a multi-object image is inherently multi-level, they propose that salient object detection must be evaluated for the capability to detect all multi-level salient objects apart from the salient object class detection capability. For this purpose, they generate multi-level maps as GT corresponding to all the dataset images using the results of the subjective experiments, with the labels being multi-level salient objects and background. They then propose the use of mean absolute error, Kendall's rank correlation and average area under precision–recall curve to evaluate existing salient object detection methods on their multi-level saliency GT dataset. Approaches that represent saliency detection on images as local-global hierarchical processing of a graph perform well in their dataset.

Recently, Salient Object Detection (SOD) has been witnessed remarkable advancements owing to the introduction of Convolution Neural Networks (CNNs). However, when faced with complex situations, such as salient objects of different sizes appearing in the same image, the detection results are far away from satisfaction. In this paper, we propose a scale-aware multi-branch decoder to detect salient objects at different scales. Specifically, different branches of the decoder are assigned to extract distinctive features and predict maps for objects of different sizes. Meanwhile, we design two flows between branches for feature aggregation. Global Distinction Flow (GDF) allows each branch to focus on the semantic information of specific sizes object, while Hierarchical Integration Flow (HIF) combines different information on each branch and forms the final detection result. In addition, we generate new masks by decomposing the ground true saliency map and use them in different branches for supervision. Comparing with existed state-of-the-art methods, experiments conducted on five datasets demonstrate the effectiveness of our method.

The performance of recent RGB-D salient object detectors has significantly improved owing to their integration of convolutional neural networks (CNNs). However, most existing salient object detection (SOD) methods represent features using a VGGNet backbone, which lacks the ability to retain complete RGB and depth modals and must compensate by applying several skip connections. In this letter, we propose a gate fusion network (GFNet) with Res2Net architecture to solve this problem. GFNet consists of two interacting Res2Net block encoder streams and four gate fusion block (GFB) decoders to interconnect the streams and fuse features. Res2Net blocks have a robust feature retention mechanism to ensure that the decoders can learn complete information, while the GFB formulates the interdependences of the encoders and eliminates noise via a gate mechanism. We evaluated GFNet using two popular RGB-D salient detection benchmark datasets (NJU2000 and NLPR) and achieved state-of-the art performance.

MFENet: Multitype fusion and enhancement network for detecting salient objects in RGB-T images

Salient object detection (SOD) is a crucial and preliminary task for many computer vision applications, which have made progress with deep CNNs. Most of the existing methods mainly rely on the RGB information to distinguish the salient objects, which faces difficulties in some complex scenarios. To solve this, many recent RGBD-based networks are proposed by adopting the depth map as an independent input and fuse the features with RGB information. Taking the advantages of RGB and RGBD methods, we propose a novel depth-aware salient object detection framework, which has following superior designs: 1) It does not rely on depth data in the testing phase. 2) It comprehensively optimizes SOD features with multi-level depth-aware regularizations. 3) The depth information also serves as error-weighted map to correct the segmentation process. With these insightful designs combined, we make the first attempt in realizing an unified depth-aware framework with only RGB information as input for inference, which not only surpasses the state-of-the-art performance on five public RGB SOD benchmarks, but also surpasses the RGBD-based methods on five benchmarks by a large margin, while adopting less information and implementation light-weighted.

In human visual saliency, top-down and bottom-up information are combined as a basis of visual attention. Recently, deep Convolutional Neural Networks (CNN) have demonstrated strong performance on RGB salient object detection, providing an effective mechanism for combining top-down semantic information with low level features. Although depth information has been shown to be important for human perception of salient objects, the use of top-down information and the exploration of CNNs for RGB-D salient object detection remains limited. Here we propose a novel deep CNN architecture for RGB-D salient object detection that utilizes both top-down and bottom-up cues. In order to produce such an architecture, we present novel depth features that capture the ideas of background enclosure, depth contrast and histogram distance in a manner that is suitable for a learned approach. We show improved results compared to state-of-the-art RGB-D salient object detection methods. We also show that the low-level and mid-level depth features both contribute to improvements in results. In particular, the F-Score of our method is 0.848 on RGBD1000, which is 10.7% better than the current best.

Artificial visual attention has been an active research area for over two decades. Especially, the concept of saliency has been implemented in many different ways. Early approaches aimed at closely modeling saliency processing with concepts from biological attention to provide (at least in the long run) general-purpose attention for technical systems. More recent approaches have departed from this agenda, turning to more specific attention-guided tasks, most notably the accurate extraction of salient objects, for which large-scale ground truth datasets make it possible to quantify progress. While the first type of models is troubled by weak performance in these specific tasks, the second type, as we show with a new benchmark, has lost the ability to predict saliency in the original sense, which may be an important factor for future general-purpose attention systems. Here, we describe a new approach using growing neural gas to obtain pre-attentional structures for a scene at an early processing stage. On this basis, traditional saliency concepts can be applied while at the same time they can be linked to mechanisms that make models successful in salient object detection. The model shows high performance at predicting traditional saliency and makes substantial progress toward salient object detection, although it cannot reach the top-level performance of some specialized methods. We discuss the important implications of our findings.

Attention and boundary guided salient object detection

There are two main areas in saliency detection: fixation prediction and salient object detection. Researchers always use bottom-up low-level features and straightforward rules for salient object detection. However these rules are unreliable in common scenes. As a result, most salient object detection technologies can gain a fine performance in limited situation. While in a more complex scene (such as PASCAL-S dataset), they may miss the really wanted salient objects. Fixation prediction is a technology of predicting the regions which can attract human's attention. Most fixation prediction technologies are based on the discoveries of neuroscience and cognitive psychology. Their rules for fixation prediction more accord with human's sensory mechanism. Thus, fixation map can provide a more reliable salient priori. Based on above reasons, we propose a novel salient object detection method with fixation priori. We refine fixation map to gain a reliable priori, and conduct salient object detection based on the priori. We perform experiment on two datasets, and find that our method outperforms the comparative state-of-art salient object detection technologies.