Retrieving images combining saliency detection with IRM

Most existing object-based image retrieval systems are based on single object matching, with its main limitation being that one individual image region (object) can hardly represent the user’s retrieval target, especially when more than one object of interest is involved in the retrieval. Integrated Region Matching (IRM) has been used to improve the retrieval accuracy by evaluating the overall similarity between images and incorporating the properties of all the regions in the images. However, IRM does not take the user’s preferred regions into account and has undesirable time complexity. In this article, we present a Feedback-based Image Clustering and Retrieval Framework (FIRM) using a novel image clustering algorithm and integrating it with Integrated Region Matching (IRM) and Relevance Feedback (RF). The performance of the system is evaluated on a large image database, demonstrating the effectiveness of our framework in catching users’ retrieval interests in object-based image retrieval.

We propose visual comfort assessment for stereoscopic 3D (S3D) images based on salient discomfort regions. Color-based saliency successfully represents visual attention because that the human visual system (HVS) focuses on the most salient region in an image. Disparity-based saliency effectively expresses visual comfort in S3D images. Based on two saliencies, we extract salient discomfort regions which mostly determine the overall visual comfort degree in S3D images. Then, we predict the visual comfort score from salient discomfort regions by a disparity feature vector which combines saliency-weighted disparity and maximum disparity. Finally, we provide visual comfort index maps of S3D images based on visual comfort scores, thus showing perceptually salient discomfort regions and their degrees. Experimental results demonstrate that the proposed method achieves significantly high accuracy in visual comfort assessment as compared to existing methods.

The objective of this paper is twofold. First, we introduce an effective region-based solution for saliency detection. Then, we apply the achieved saliency map to better encode the image features for solving object recognition task. To find the perceptually and semantically meaningful salient regions, we extract superpixels based on an adaptive mean shift algorithm as the basic elements for saliency detection. The saliency of each superpixel is measured by using its spatial compactness, which is calculated according to the results of Gaussian mixture model (GMM) clustering. To propagate saliency between similar clusters, we adopt a modified PageRank algorithm to refine the saliency map. Our method not only improves saliency detection through large salient region detection and noise tolerance in messy background, but also generates saliency maps with a well-defined object shape. Experimental results demonstrate the effectiveness of our method. Since the objects usually correspond to salient regions, and these regions usually play more important roles for object recognition than background, we apply our achieved saliency map for object recognition by incorporating a saliency map into sparse coding-based spatial pyramid matching (ScSPM) image representation. To learn a more discriminative codebook and better encode the features corresponding to the patches of the objects, we propose a weighted sparse coding for feature coding. Moreover, we also propose a saliency weighted max pooling to further emphasize the importance of those salient regions in feature pooling module. Experimental results on several datasets illustrate that our weighted ScSPM framework greatly outperforms ScSPM framework, and achieves excellent performance for object recognition.

Saliency detection is a technique for automatically extracting regions of interest from the background and has been widely used in the computer vision field. This study proposes a simple and effective saliency detection method combining color contrast and hash fingerprint. In our solution, the input image is segmented into nonoverlapping superpixels, so as to perform the saliency detection at the region level to reduce computational complexity. A background optimization selection is used to construct an accurate background template. Based on this, a saliency map that highlights the whole salient region is obtained by estimating color contrast. Besides, another saliency map that enhances the salient region while restraining the background is also generated through hash fingerprint matching. Ultimately, the final saliency map can be obtained by fusing the two saliency maps. Comparing the performance with other methods, the proposed algorithm works better even in the presence of complex background or very large salient regions.

A salient region is the most distinctive part of the image that captures human's attention. Saliency detection is a fundamental characteristic of the human visual system. Finding computational models which are able to detect salient regions is a challenging task for image processing and computer vision applications. Salient regions of various sizes can be detected from different scales. Therefore, selecting the best scales is an important issue. In this paper, an efficient multi-scale method to find salient regions is proposed. In order to include more features in evaluating saliency of a pixel, feature maps are generated using components of both the RGB and YUV color spaces. These features are combined into quaternions. Detecting salient regions of different sizes is addressed by utilizing a scale space analysis. Salient regions are detected by convolving the image amplitude spectrum with a low-pass Gaussian kernel of multiple scales. To incorporate more meaningful information, more than one scale is considered based on entropy criterion. The final saliency map is generated by normalizing the weighted saliency maps of these scales. Experiments are conducted on a dataset of natural images to evaluate the performance of the proposed method. Results show that the proposed method provides larger values of area under receiver operating characteristics curve, precision, recall and F-measure, in comparison to some of the state-of-the-art methods.

Salient regions usually carry important information in images. Existing feature encryption algorithms aim at extracting edge features as significant information rather than salient regions for encryption purpose. Moreover, most of them protect significant information by transforming the input image into texture-like or noise-like encrypted image which is obviously a visual sign of encrypted image, and thus can be easily attacked. In this paper, we propose a salient regions encryption scheme to generate visually meaningful ciphertext. First, salient regions are efficiently extracted by a saliency detection model in the compressed domain. Then we pre-encrypt these salient regions by a chaos-based encryption algorithm. With optical encryption theory, the pre-encrypted salient regions are finally transformed into a visually meaningful ciphertext. To the best of our knowledge, it is the first time to use salient regions as important visual information for encryption to obtain cipertext in images. The experimental results demonstrate that the salient regions can be largely hidden with the proposed method.

In this article, we tackle the saliency detection task from an interesting perspective: we focus both on salient regions (or foreground) detection and nonsalient regions (or background) detection instead of only the foreground and propose a novel complementarity-aware attention network. It is a unified framework with two branches, namely, positive attention module (PAM) and negative attention module (NAM), for the foreground and background detection, respectively. More specifically, the PAM exploits a position self-attention mechanism to enhance the discriminant ability of feature representation, which can detect most of the salient object regions. Meanwhile, the NAM is designed to detect the background regions, aiming to pop out the missing object parts and details in the prediction map produced by the PAM. By fusing these two attention modules together, NAM can provide complementary cues to assist PAM for precise object detection. Furthermore, in order to capture more multiscale contextual information, we introduce a bidirectional structure with multisupervision to the proposed complementarity-aware attention module for performance improvement. Experiments on five benchmark datasets show that the proposed framework achieves comparable results compared with the state-of-the-art saliency detection methods.

Nowadays, ship detection in sea-sky background is not only useful in maritime visual surveillance, but also helpful in maritime search and rescue. Since ships are salient objects in infrared images with sea-sky background, we present a novel and effective algorithm based on saliency for ship detection in this situation. Our algorithm adopts global saliency, local saliency and background prior to generate saliency maps. Ships are finally segmented in saliency maps. Our algorithm is compared with four classic salient object detection algorithms. And experimental results show our algorithm outperforms the other four algorithms in qualitative and quantitative terms. Keyword list: ship detection ; salient object detection ; infrared image ; sea - sky background . 1. INTRODUCTION Since ships are important vehicles for sea transportation, ship detection in sea-sky background is extensively used in maritime visual surveillance and reconnaissance, avoidance of ship collision accidents, maritime search and rescue, etc. Infrared imaging systems can almost work in any environment and all-weather conditions with its visual capacity in darkness and frog, which makes it a very suitable tool in this application. In recent years, ship detection in infrared images with sea-sky background attracts increasing attention. Many methods are proposed in this field, including image segmentation methods [1-2] and morphological methods [3]. However, due to complex sea-sky environment and large variation of ship sizes, these methods cannot solve the issue completely. And since ships are often salient objects in infrared images with sea-sky background, some researchers have recently tried salient object detection ideas used in color images [4-6]. And the paper is focused on using salient object detection methods to solve the issue. Salient objects are objects which stand out in the image and attract most people ¶VLPPHGLDWHDWWHQWLRQ7KHVWDQGDUGway to detect salient ob MHFWVLVILUVWO\XVLQJVDOLHQF\GHWHFWLRQPHWKRGVWRJHQHUDWHVDOLHQF\PDSVLQZKLFKDSL[HO¶VJUH\value is proportional to its degree of saliency and then segmenting salient objects in saliency maps. Many salient detection methods can be divided into three categories: local contrast methods, global rarity methods and other methods. In local contrast methods, regions which have high contrast with local neighbouring regions are regarded as salient regions. Itti et al. [7] decompose an image into several feature maps . The saliency in each feature map is measured by local center-surround difference in several scales and saliency of different feature maps are integrated to a final saliency

Searching for an image in a database is important in different applications; hence, many algorithms have been proposed to identify the contents of the image. In some applications, but not all, identifying the content of the image as a whole can offer good results. Searching for an object inside the image is more important in most applications than identifying the image as a whole. Therefore, studies focused on segmenting the image into small sub-images and identified their contents. In view of the concepts of human attention, various literature defined saliency as a computer representation of it, where different algorithms were developed to extract the salient regions. These salient regions, which are the regions that attract human attention, are used to identify the most important regions that contain important objects in the image. In this paper, we introduce a new algorithm that utilises the saliency principles to identify the contents of an image and search for similar objects in the images stored in a database. We also demonstrate that the use of salient objects produces better and more accurate results in the image retrieval process. A new retrieval algorithm is therefore presented here, focused on identifying the objects extracted from the salient regions. To assess the efficiency of the proposed algorithm, a new evaluation method is also proposed which considers the order of the retrieved image in assessing the efficiency of the algorithm.

Video saliency detection aims to pop out the most salient regions in every frame of a video. Up to now, many efforts have been made from various aspects for video saliency detection. Unfortunately, the existing video saliency models are very likely to fail in challenging videos with complicated motions and complex scenes. Therefore, in this paper, we propose a novel framework to improve the saliency detection results generated by existing video saliency models. The proposed framework consists of three key steps including localized estimation, spatiotemporal refinement, and saliency update. Specifically, the initial saliency map of each frame in a video is first generated by using an existing saliency model. Then, by considering the temporal consistency and strong correlation among adjacent frames, the localized estimation models, which are generated by training the random forest regressor within a local temporal window, are employed to generate the temporary saliency map. Finally, by taking the appearance and motion information of salient objects into consideration, the spatiotemporal refinement step is deployed to further improve the temporary saliency map and generate the final saliency map. Furthermore, such an improved saliency map is then utilized to update the initial saliency map and provide reliable cues for saliency detection in the next frame. The experimental results on four challenging datasets demonstrate that the proposed framework is able to consistently and significantly improve the saliency detection performance of various video saliency models, thereby achieving the state-of-the-art performance.

Stereoscopic perception is an important part of human visual system that allows the brain to perceive depth. However, depth information has not been well explored in existing saliency detection models. In this letter, a novel saliency detection method for stereoscopic images is proposed. Firstly, we propose a measure to evaluate the reliability of depth map, and use it to reduce the influence of poor depth map on saliency detection. Then, the input image is represented as a graph, and the depth information is introduced into graph construction. After that, a new definition of compactness using color and depth cues is put forward to compute the compactness saliency map. In order to compensate the detection errors of compactness saliency when the salient regions have similar appearances with background, foreground saliency map is calculated based on depth-refined foreground seeds selection mechanism and multiple cues contrast. Finally, these two saliency maps are integrated into a final saliency map through weighted-sum method according to their importance. Experiments on two publicly available stereo datasets demonstrate that the proposed method performs better than other 10 state-of-the-art approaches.

Secret image sharing (SIS), as one of the applications of information theory in information security protection, has been widely used in many areas, such as blockchain, identity authentication and distributed cloud storage. In traditional secret image sharing schemes, noise-like shadows introduce difficulties into shadow management and increase the risk of attacks. Meaningful secret image sharing is thus proposed to solve these problems. Previous meaningful SIS schemes have employed steganography to hide shares into cover images, and their covers are always binary images. These schemes usually include pixel expansion and low visual quality shadows. To improve the shadow quality, we design a meaningful secret image sharing scheme with saliency detection. Saliency detection is used to determine the salient regions of cover images. In our proposed scheme, we improve the quality of salient regions that are sensitive to the human vision system. In this way, we obtain meaningful shadows with better visual quality. Experiment results and comparisons demonstrate the effectiveness of our proposed scheme.

Saliency detection is an important research topic in computer vision. The traditional methods compute image saliency map, then salient segmentation is based on the corresponding saliency map. Unfortunately, overall performance of this method is poor due to the reason of losing some fine details and spatial information within image. This paper presents a new framework to overcome the drawback, named FDSRDS(Framework for Directly Salient Region Detection and Segmentation based on graph methods). Under FDSRSD, firstly, we get the foreground image by segmenting the original image via our extended grabcut algorithm. Mostly, the saliency region is within the foreground part. Secondly, we segment the foreground image into regions by means of graph based segmentation and nearest neighbor graph . Thirdly, we use relative weber's luminance rules to calculate every region’s luminance. Finally, we get the maximum luminance region which is the saliency region. Under FDSRSD framework, algorithms we proposed capture fine details and spatial relationships in saliency computation. We demonstrate impressive results by evaluating our method with other five state-of-the-art methods on the publicly available data set.

Automatic detection of salient image regions is a useful tool with applications in intelligent camera control, virtual cinematography, video summarization and editing, evaluation of viewer preferences, and many others. This paper presents an effective method for detecting potentially salient foreground regions. Salient regions are identified by eigenvalue analysis of a graph Laplacian that is defined over the color similarity of image superpixels, under the assumption that the majority of pixels on the image boundary show non-salient background. In contrast to previous methods based on graph-cuts or graph partitioning, our method provides continuously-valued saliency estimates with complementary properties to recently proposed color contrast-based approaches. Moreover, exploiting discriminative properties of the Fiedler vector, we devise an SVM-based classifier that allows us to determine whether an image contains any salient objects at all, a problem that has been largely neglected in previous works. We also describe how the per-frame saliency detection can be extended to improve its spatiotemporal coherence when computed on video sequences. Extensive evaluation on several datasets demonstrates and validates the state-of-the-art performance of the proposed method.

The vast growth of Internet and digital contents of Internet creates much interest on automated image indexing and retrieval techniques. Most commercial Content Based Image Retrieval systems use color histograms as one of the features. In this work, we have prepared combined color histograms for three regions (top, bottom and middle) of the image. The combination of those histograms will indirectly include the space information within it. Besides this, straight-line signatures and outline sketch signatures from the image are extracted. The average precision is used as a metric to evaluate the performance of the system. Experiments were successfully implemented on 1000 images of SIMPLIcity database using MAT Lab. The performance of our method is compared with four other previous methods Integrated Region Matching[l], Fuzzy Integrated Region Matching [2], Color Salient Points and shape features [3] and Interactive CBIR using Ripplet Transform and Fuzzy Relevance Feedback Mechanism[4] which used the same data set and evaluation method. According to the arrived results, the performance of the proposed Three Region Color, Straight-Line and Outline Sketch Signatures based CBIR system was significantly better than other compared methods.