WBNet: Weakly-supervised salient object detection via scribble and pseudo-background priors

Salient object detection is a process of extracting an object which is visually attractive from a single image or a video. As a powerful technique for automatic image or video segmentation, saliency detection has been focused and studied recently. In this paper, we propose a novel method for salient object detection without training or learning-based techniques. The proposed framework consists of two major steps, the generation of saliency map candidates and the selection of an optimal saliency map. To generate saliency map candidates, prior maps based on combinations of RGB color components are proposed. To select the optimal saliency map among the candidates, we propose a compactness measure, which evaluates the degree to which the generated saliency maps show objects. As a result, among recent works on saliency detection, our saliency detection method achieves the highest performance in terms of saliency detection.

There are two main areas in saliency detection: fixation prediction and salient object detection. Researchers always use bottom-up low-level features and straightforward rules for salient object detection. However these rules are unreliable in common scenes. As a result, most salient object detection technologies can gain a fine performance in limited situation. While in a more complex scene (such as PASCAL-S dataset), they may miss the really wanted salient objects. Fixation prediction is a technology of predicting the regions which can attract human's attention. Most fixation prediction technologies are based on the discoveries of neuroscience and cognitive psychology. Their rules for fixation prediction more accord with human's sensory mechanism. Thus, fixation map can provide a more reliable salient priori. Based on above reasons, we propose a novel salient object detection method with fixation priori. We refine fixation map to gain a reliable priori, and conduct salient object detection based on the priori. We perform experiment on two datasets, and find that our method outperforms the comparative state-of-art salient object detection technologies.

Effectively capturing the multi-scale, multi-channel, and boundary features of target images is beneficial for improving the accuracy of salient object detection. A multichannel feature enhancement module (MFEM) is constructed by concatenating two feature maps along the channel, performing convolution activation, dot product addition, and convolution operations. An edge feature enhancement module (EFEM) is constructed by performing an inverse attention mechanism and Gaussian-Laplace transform on the predicted map, channel-wise concatenation, and convolution operations with the feature submap. Multi-scale features are extracted using a SwinTransformer encoder and enhanced using the MFEM. The enhanced features are then fused using a cross-attention mechanism for multi-level features. The fused feature map, the high-frequency feature map obtained by Gaussian-Laplace transforming the predicted map, and the external edge features are input into the EFEM to obtain the salient object detection results. This results in a novel salient object detection model with multi-scale, multi-channel, and boundary feature enhancement: MfENet. Experimental results show that the Mean Absolute Error (MAE), S-measure and F-measure accuracies of our method on the HKU-IS dataset are 0.018,0.924 and 0.959 respectively. These results demonstrate the effectiveness of our method. Furthermore, this work highlights the importance of multi-scale and boundary awareness in enhancing salient object detection, and its contribution to the technical field relevant to the theme of this conference.

Nowadays, ship detection in sea-sky background is not only useful in maritime visual surveillance, but also helpful in maritime search and rescue. Since ships are salient objects in infrared images with sea-sky background, we present a novel and effective algorithm based on saliency for ship detection in this situation. Our algorithm adopts global saliency, local saliency and background prior to generate saliency maps. Ships are finally segmented in saliency maps. Our algorithm is compared with four classic salient object detection algorithms. And experimental results show our algorithm outperforms the other four algorithms in qualitative and quantitative terms. Keyword list: ship detection ; salient object detection ; infrared image ; sea - sky background . 1. INTRODUCTION Since ships are important vehicles for sea transportation, ship detection in sea-sky background is extensively used in maritime visual surveillance and reconnaissance, avoidance of ship collision accidents, maritime search and rescue, etc. Infrared imaging systems can almost work in any environment and all-weather conditions with its visual capacity in darkness and frog, which makes it a very suitable tool in this application. In recent years, ship detection in infrared images with sea-sky background attracts increasing attention. Many methods are proposed in this field, including image segmentation methods [1-2] and morphological methods [3]. However, due to complex sea-sky environment and large variation of ship sizes, these methods cannot solve the issue completely. And since ships are often salient objects in infrared images with sea-sky background, some researchers have recently tried salient object detection ideas used in color images [4-6]. And the paper is focused on using salient object detection methods to solve the issue. Salient objects are objects which stand out in the image and attract most people ¶VLPPHGLDWHDWWHQWLRQ7KHVWDQGDUGway to detect salient ob MHFWVLVILUVWO\XVLQJVDOLHQF\GHWHFWLRQPHWKRGVWRJHQHUDWHVDOLHQF\PDSVLQZKLFKDSL[HO¶VJUH\value is proportional to its degree of saliency and then segmenting salient objects in saliency maps. Many salient detection methods can be divided into three categories: local contrast methods, global rarity methods and other methods. In local contrast methods, regions which have high contrast with local neighbouring regions are regarded as salient regions. Itti et al. [7] decompose an image into several feature maps . The saliency in each feature map is measured by local center-surround difference in several scales and saliency of different feature maps are integrated to a final saliency

Salient Object Detection (SOD) has received much attention from the research community due to its increasing applications in the areas such as object detection and recognition, image editing, image and video compression, video summarization and so on. Most of the SOD methods are proposed in literature presuming that the digital images in which salient objects is to be detected, are free from any kind of artifact. SOD in the presence of noise has received much less attention from research community. In this paper, we study and analyze popular salient object detection methods in the presence of Gaussian, Salt and Pepper and Speckle Noises. Extensive experiments are performed on two publicly available SOD datasets viz. MSRA5K and DUT OMRON. The performance of the methods are evaluated in terms of Precision, Recall and F-measure. It is found that Context Aware Saliency Detection (CA) method gives maximum Precision while Graph Based Visual Saliency (GB) gives maximum Recall and F-measure on both the datasets in presence of any of the three noises.

Object detection is one of the hot research issues for the Unmanned Aerial Vehicle (UAV). The main problem of object detection is the inaccuracy of object’s location. Most of the current studies focus on classifying and locating objects, while ignore the influence of pixel-level object detection. By utilizing the Salient Object Detection (SOD) method, we may improve the accuracy of object’s location. In this work, we refer to the structure of U-net, and present a network structure called UA-net. In order to improve the result of small image processing, we use not only the commonly used bilinear interpolation method, but also try several other image interpolation methods. After trained and tested by PedCut2013_Segmentation dataset, the result shows that the UA-net can produce nice saliency map of pedestrians, and different interpolation methods have different quality evaluated by evaluation methods. After testing by UAV’s collected images, the saliency map can also be nice SOD results. These results show that UA-net is an applicable SOD structure for UAV to carry out the pedestrian recognition task.

Most existing light field saliency detection methods have achieved great success by exploiting unique light field data-focus information in focal slices. However, they process light field data in a slicewise way, leading to suboptimal results because the relative contribution of different regions in focal slices is ignored. How we can comprehensively explore and integrate focused saliency regions that would positively contribute to accurate saliency detection. Answering this question inspires us to develop a new insight. In this article, we propose a patch-aware network to explore light field data in a regionwise way. First, we excavate focused salient regions with a proposed multisource learning module (MSLM), which generates a filtering strategy for integration followed by three guidances based on saliency, boundary, and position. Second, we design a sharpness recognition module (SRM) to refine and update this strategy and perform feature integration. With our proposed MSLM and SRM, we can obtain more accurate and complete saliency maps. Comprehensive experiments on three benchmark datasets prove that our proposed method achieves competitive performance over 2-D, 3-D, and 4-D salient object detection methods. The code and results of our method are available at https://github.com/OIPLab-DUT/IEEE-TCYB-PANet.

We propose 3D salient object detection and segmented point cloud gridding methods to accelerate the real object-based full-color holographic system. The feasibility of the proposed methods is confirmed by experiments.

Salient Object Detection (SOD) methods have been widely investigated in order to mimic human visual system in selecting regions of interest from complex scenes. The majority of existing SOD methods have focused on designing and combining handcrafted features. This process relies on domain knowledge and expertise and becomes increasingly difficult as the complexity of candidate models increases. In this paper, we develop an automatic feature combination method for saliency features to relieve human intervention and domain knowledge. The proposed method contains three phases, two Genetic Programming (GP) phases to construct foreground and background features and a spatial blending phase to combine those features. The foreground and background features are constructed to complement each other, therefore one can improve other’s shortcomings. This method is compared with the state-of-the-art methods on four different benchmark datasets. The results indicate the new automatic method is comparable with the state-of-the-art methods and even improves SOD performance on some datasets.

News items

We present a novel, robust estimation method to distinguish salient objects from complicated, dynamic backgrounds in videos. In this method, we propose a novel approach to model motion energy based on motion magnitude, motion orientation, gradient flow field, and spatial gradient of the video frame. Furthermore, an effective spatiotemporal objectness map is also proposed to estimate a compact object-like region in the current video frame leveraging both the objectness proposals and the saliency map of the previous frame. Then the current video frame is oversegmented into the granularity of superpixels using the simple linear iterative clustering algorithm. Each superpixel is designated as a node of a graph. The similarity between adjacent superpixels will be assigned as the weight of an edge that connects these two nodes. The feature values of motion energy and spatiotemporal objectness within each superpixel will be averaged respectively, and used to graphically cluster similar superpixels to form the detected salient object. Extensive experiments comparing this proposed new method against twelve existing salient object detection (SOD) methods have been performed using the benchmark datasets unconstrained video saliency detection and densely annotated video segmentation. Superior performance of this proposed SOD method has been observed through three well-known performance metrics: precision-recall curves, F-measure curves, and the mean absolute error.

<p>For a diverse range of applications in machine vision from social media searches to robotic home care providers, it is important to replicate the mechanism by which the human brain selects the most important visual information, while suppressing the remaining non-usable information. Many computational methods attempt to model this process by following the traditional model of visual attention. The traditional model of attention involves feature extraction, conditioning and combination to capture this behaviour of human visual attention. Consequently, the model has inherent design choices at its various stages. These choices include selection of parameters related to the feature computation process, setting a conditioning approach, feature importance and setting a combination approach. Despite rapid research and substantial improvements in benchmark performance, the performance of many models depends upon tuning these design choices in an ad hoc fashion. Additionally, these design choices are heuristic in nature, thus resulting in good performance only in certain settings. Consequentially, many such models exhibit low robustness to difficult stimuli and the complexities of real-world imagery. Machine learning and optimisation technique have long been used to increase the generalisability of a system to unseen data. Surprisingly, artificial learning techniques have not been investigated to their full potential to improve generalisation of visual attention methods. The proposed thesis is that artificial learning can increase the generalisability of the traditional model of visual attention by effective selection and optimal combination of features. The following new techniques have been introduced at various stages of the traditional model of visual attention to improve its generalisation performance, specifically on challenging cases of saliency detection: 1. Joint optimisation of feature related parameters and feature importance weights is introduced for the first time to improve the generalisation of the traditional model of visual attention. To evaluate the joint learning hypothesis, a new method namely GAOVSM is introduced for the tasks of eye fixation prediction. By finding the relationships between feature related parameters and feature importance, the developed method improves the generalisation performance of baseline method (that employ human encoded parameters). 2. Spectral matting based figure-ground segregation is introduced to overcome the artifacts encountered by region-based salient object detection approaches. By suppressing the unwanted background information and assigning saliency to object parts in a uniform manner, the developed FGS approach overcomes the limitations of region based approaches. 3. Joint optimisation of feature computation parameters and feature importance weights is introduced for optimal combination of FGS with complementary features for the first time for salient object detection. By learning feature related parameters and their respective importance at multiple segmentation thresholds and by considering the performance gaps amongst features, the developed FGSopt method improves the object detection performance of the FGS technique also improving upon several state-of-the-art salient object detection models. 4. The introduction of multiple combination schemes/rules further extends the generalisability of the traditional attention model beyond that of joint optimisation based single rules. The introduction of feature composition based grouping of images, enables the developed IGA method to autonomously identify an appropriate combination strategy for an unseen image. The results of a pair-wise ranksum test confirm that the IGA method is significantly better than the deterministic and classification based benchmark methods on the 99% confidence interval level. Extending this line of research, a novel relative encoding approach enables the adapted XCSCA method to group images having similar saliency prediction ability. By keeping track of previous inputs, the introduced action part of the XCSCA approach enables learning of generalised feature importance rules. By more accurate grouping of images as compared with IGA, generalised learnt rules and appropriate application of feature importance rules, the XCSCA approach improves upon the generalisation performance of the IGA method. 5. The introduced uniform saliency assignment and segmentation quality cues enable label free evaluation of a feature/saliency map. By accurate ranking and effective clustering, the developed DFS method successfully solves the complex problem of finding appropriate features for combination (on an-image-by-image basis) for the first time in saliency detection. The DFS method enables ground truth free evaluation of saliency methods and advances the state-of-the-art in data driven saliency aggregation by detection and deselection of redundant information. The final contribution is that the developed methods are formed into a complete system where analysis shows the effects of their interactions on the system. Based on the saliency prediction accuracy versus computational time trade-off, specialised variants of the proposed methods are presented along with the recommendations for further use by other saliency detection systems. This research work has shown that artificial learning can increase the generalisation of the traditional model of attention by effective selection and optimal combination of features. Overall, this thesis has shown that it is the ability to autonomously segregate images based on their types and subsequent learning of appropriate combinations that aid generalisation on difficult unseen stimuli.</p>

The performance of recent RGB-D salient object detectors has significantly improved owing to their integration of convolutional neural networks (CNNs). However, most existing salient object detection (SOD) methods represent features using a VGGNet backbone, which lacks the ability to retain complete RGB and depth modals and must compensate by applying several skip connections. In this letter, we propose a gate fusion network (GFNet) with Res2Net architecture to solve this problem. GFNet consists of two interacting Res2Net block encoder streams and four gate fusion block (GFB) decoders to interconnect the streams and fuse features. Res2Net blocks have a robust feature retention mechanism to ensure that the decoders can learn complete information, while the GFB formulates the interdependences of the encoders and eliminates noise via a gate mechanism. We evaluated GFNet using two popular RGB-D salient detection benchmark datasets (NJU2000 and NLPR) and achieved state-of-the art performance.

Most of the existing RGB‐T salient object detection methods are usually based on dual‐stream encoding single‐stream decoding network architecture. These models always rely on the quality of fusion features, which often focus on modality‐shared features and overlook modality‐specific features, thus failing to fully utilise the rich information contained in multi‐modality data. To this end, a modality separate tri‐stream net (MSTNet), which consists of a tri‐stream encoding (TSE) structure and a tri‐stream decoding (TSD) structure is proposed. The TSE explicitly separates and extracts the modality‐shared and modality‐specific features to improve the utilisation of multi‐modality data. In addition, based on the hybrid‐attention and cross‐attention mechanism, we design an enhanced complementary fusion module (ECF), which fully considers the complementarity between the features to be fused and realises high‐quality feature fusion. Furthermore, in TSD, the quality of uni‐modality features is ensured under the constraint of supervision. Finally, to make full use of the rich multi‐level and multi‐scale decoding features contained in TSD, the authors design the adaptive multi‐scale decoding module and the multi‐stream feature aggregation module to improve the decoding capability. Extensive experiments on three public datasets show that the MSTNet outperforms 14 state‐of‐the‐art methods, demonstrating that this method can extract and utilise the multi‐modality information more adequately and extract more complete and rich features, thus improving the model's performance. The code will be released at https://github.com/JOOOOKII/MSTNet .

Salient object detection has been challenging computer vision though some advances have been made recently. In this study, we propose a novel salient object detection method by using feature clustering and compactness prior, in the situation of the absence of any prior information. The proposed method consists of four rigorous steps. Superpixel preprocessing is first employed to segment image into superpixels for suppressing noise and reducing computational complexity. Then, clustering algorithm is applied to get the classification of color features. Furthermore, two-dimensional entropy is used to measure the compactness of each cluster and build the background model. Finally, the salient feature is defined as the contrast between background region and other regions, and enhanced by designing a Gauss filter. To better evaluate the salient object detection accuracy, detailed experimental analysis is carried out by using 7 evaluation indexes. Our proposed method outperforms some peers in extensive experiments. It will inspire more similar techniques to be developed in this research topic.