Incorporating Manifold Ranking with Active Learning in Relevance Feedback for Image Retrieval

In content-base image retrieval, relevance feedback (RF) schemes based on support vector machine (SVM) have been widely used to narrow the semantic gap between low-level visual features and high-level human perception. However, the performance of image retrieval with SVM active learning is known to be poor when the training data is insufficient. In this paper, the problem is solved by incorporating the unlabelled images into the learning process. We proposed a semi-supervised active learning algorithm which uses not only labeled training samples but also unlabeled ones to build better models. In relevance feedback, active learning algorithm is often used to reduce the cost of labeling by selecting only the most informative data. In addition, we introduced a semi-supervised approach which employed Nearest-Neighbor technique to label the unlabeled sample with a certain degree of uncertainty in its class information. Using these samples, Fuzzy support vector machine (FSVM) which takes into account the fuzzy nature of some training samples during its training is trained. We compared our method with standard active SVM on a database of 10,000 images, the experiment results show that the efficiency of SVM active learning can be improved by incorporating unlabelled images, and thus improve the overall retrieval performance. DOI: http://dx.doi.org/10.11591/telkomnika.v11i7.2807

We systematically compare four variants of evolutionary algorithms (EAs) for relevance feedback (RF) in image retrieval. We focus on the small sample size (SSS) performance, i.e., the requirement to learn the user’s information need based on a small – between 2 and 25 – number of positive and negative training images. Despite this being a fundamental requirement, none of the existing works dealing with EAs for RF in image retrieval addresses it. Common for the four variants is the hierarchical, region-based image similarity model, with region and feature weights as parameters. The difference between the variants is in the objective function of the EA used to adjust the model parameters. The objective functions include: (O-1) precision; (O-2) average rank; (O-3) ratio of within-class (i.e., positive images) and between-class (i.e., positive and negative images) scatter; and (O-4) combination of O-2 and O-3. We note that – unlike O-1 and O-2 – O-3 and O-4 are not used in any of the existing works dealing with EAs for RF. The four variants are evaluated on five test databases, containing 61,895 general-purpose images, in 619 semantic categories. Results of the evaluation reveal that variants with objective functions O-3 and O-4 consistently outperform those with O-1 and O-2. Furthermore, comparison with the representative of the existing RF methods shows that EAs are both effective and efficient approaches for SSS learning in region-based image retrieval.

This paper presents a content-based image retrieval (CBIR) system that incorporates real-valued multiple instance learning (MIL) into the user relevance feedback (RF) to learn the user's subjective visual concepts, especially where the user's most interested region and how to map the local feature vector of that region to the high-level concept pattern of the user. RF provides a way to obtain the subjectivity of the user's high-level visual concepts, and MIL enables the automatic learning of the user's high-level concepts. The user interacts with the CBIR system by relevance feedback in a way that the extent to which the image samples retrieved by the system are relevant to the user's intention is labeled. The system in turn applies the MIL method to find user's most interested image region from the feedback. A multilayer neural network that is trained progressively through the feedback and learning procedure is used to map the low-level image features to the high-level concepts.

Asymmetric propagation based batch mode active learning for image retrieval

Relevance feedback is a powerful technique to enhance Content-Based Image Retrieval (CBIR) performance. It solicits the user's relevance judgments on the retrieved images returned by the CBIR systems. The user's labeling is then used to learn a classifier to distinguish between relevant and irrelevant images. However, the top returnedimages may not be the most informative ones. The challenge is thus to determine which unlabeled images would be the most informative (i.e., improve the classifier the most) if they were labeled and used as training samples. In this paper, we propose a novel active learning algorithm, called Laplacian Optimal Design (LOD), for relevance feedback image retrieval. Our algorithm is based on aregression model which minimizes the least square error on the measured (or, labeled) images and simultaneously preserves the local geometrical structure of the image space. Specifically, we assume that if two images are sufficiently close to each other, then their measurements (or, labels) are close as well. By constructing a nearest neighbor graph, the geometrical structure of the image space can be described by the graph Laplacian. We discuss how results from the field of optimal experimental design may be used to guide our selection of a subset of images, which gives us the most amount of information. Experimental results on Corel database suggest that theproposed approach achieves higher precision in relevance feedback image retrieval.

Support vector machine (SVM) active learning is one popular and successful technique for relevance feedback in content-based image retrieval (CBIR). Despite the success, conventional SVM active learning has two main drawbacks. First, the performance of SVM is usually limited by the number of labeled examples. It often suffers a poor performance for the small-sized labeled examples, which is the case in relevance feedback. Second, conventional approaches do not take into account the redundancy among examples, and could select multiple examples that are similar (or even identical). In this work, we propose a novel scheme for explicitly addressing the drawbacks. It first learns a kernel function from a mixture of labeled and unlabeled data, and therefore alleviates the problem of small-sized training data. The kernel will then be used for a batch mode active learning method to identify the most informative and diverse examples via a min-max framework. Two novel algorithms are proposed to solve the related combinatorial optimization: the first approach approximates the problem into a quadratic program, and the second solves the combinatorial optimization approximately by a greedy algorithm that exploits the merits of submodular functions. Extensive experiments with image retrieval using both natural photo images and medical images show that the proposed algorithms are significantly more effective than the state-of-the-art approaches. A demo is available at http://msm.cais.ntu.edu.sg/LSCBIR/.

As the use of digital media increases, effective retrieval and management techniques become more important. In order to extract useful information from this huge amount of data, many content based image retrieval (CBIR) systems have been developed. A typical CBIR system captures image features such as color, texture, or shape of objects in the query image and tries to retrieve images from the database with similar features. The main advantage of CBIR systems with relevance feedback is that these systems take into account the gap between the high-level concepts and low-level features and subjectivity of human perception of visual content. In this paper, a CBIR system based on color histogram has been presented and a new approach for image storage and retrieval called association-based image retrieval (ABIR) has been studied. The results of both systems have been shown.

In content-based image retrieval (CBIR), a set of low-level features are extracted from an image to represent its visual content. Retrieval is performed by image example where a query image is given as input by the user and an appropriate similarity measure is used to find the best matches in the corresponding feature space. This approach suffers from the fact that there is a large discrepancy between the low-level visual features that one can extract from an image and the semantic interpretation of the image’s content that a particular user may have in a given situation. That is, users seek semantic similarity, but we can only provide similarity based on low-level visual features extracted from the raw pixel data, a situation known as the semantic gap. The selection of an appropriate similarity measure is thus an important problem. Since visual content can be represented by different attributes, the combination and importance of each set of features varies according to the user’s semantic intent. Thus, the retrieval strategy should be adaptive so that it can accommodate the preferences of different users. Relevance feedback (RF) learning has been proposed as a technique aimed at reducing the semantic gap. It works by gathering semantic information from user interaction. Based on the user’s feedback on the retrieval results, the retrieval scheme is adjusted. By providing an image similarity measure under human perception, RF learning can be seen as a form of supervised learning that finds relations between high-level semantic interpretations and low-level visual properties. That is, the feedback obtained within a single query session is used to personalize the retrieval strategy and thus enhance retrieval performance. In this chapter we present an overview of CBIR and related work on RF learning. We also present our own previous work on a RF learning-based probabilistic region relevance learning algorithm for automatically estimating the importance of each region in an image based on the user’s semantic intent.

This paper describes the unsupervised-interactive learning method, using the self-organizing tree map (SOTM) architecture, for the automation of relevance feedback (RF) in content-based image retrieval. The SOTM is shown to exhibit good behavior in relevance classification; providing a possible solution to minimizing user interactions in both fully automatic and semiautomatic domains, while achieving high retrieval accuracy in the context of adaptive retrieval. Computer simulation shows this system is very effective when applied to compressed domain retrieval systems for texture retrieval and the JPEG photograph database applications.

Content-based image retrieval (CBIR) has suffered from the lack of linkage between low-level features and high-level semantics. Although relevance feedback (RF) CBIR provides a promising solution involving human interaction, certain query images poorly represented by low-level features still have unsatisfactory retrieval results. An innovative method has been proposed to increase the percentage of relevance of target image database by using graph cuts theory with the maximum-flow/minimum-cut algorithm and relevance feedback. As a result, the database is reformed by keeping relevant images while discarding irrelevant images. The relevance is increased and thus during following RF-CBIR process, previously poorly represented relevant images have higher probability to appear for selection. Better performance and retrieval results can thus be achieved.

An approach for content-based image retrieval with relevance feedback based on a structured multi-feature space is proposed. It uses a novel kernel for merging multiple feature subspaces into a complementary space. The kernel exploits nature of the data by assigning appropriate weights for each feature set. The weights are dynamically adapted to user preferences in a relevance feedback scenario.

Supervised learning algorithms (relevance feedback (RF) algorithms) are often used in content based image retrieval (CBIR) systems to enhance interactive search and browsing of image databases. One of the issues associated with RF based CBIR systems is the lack of a large training set. Labeling of images is a time consuming activity and user's usually do not have the patience to label a large set. The challenge is to somehow leverage the much larger set of unlabeled images to improve the performance of CBIR systems. In this paper we propose a novel RF algorithm which learns from both labeled and unlabeled data. Our proposed algorithm also uses active learning so as to maximize the information gained from a given amount of user feedback.

Relevance feedback (RF) schemes have been widely designed to improve the performance of content-based image retrieval. Despite the success, it is not appropriate to require the user to label a large number of samples in RF. Collaborative image retrieval (CIR) aims to reduce the labeling efforts of the user by resorting to the auxiliary information. Support vector machine (SVM) active learning can select ambiguous samples as the most informative ones for the user to label with the help of the optimal hyperplane of SVM, and thus alleviate the labeling efforts of conventional RF. However, the optimal hyperplane of SVM is usually unstable and inaccurate with small-sized training data, and this is always the case in image retrieval since the user would not like to label a large number of feedback samples and cannot label each sample accurately all the time. In this paper, we propose a novel active learning method, i.e., geometric optimum experimental design (GOED), to select multiple representative samples in the database as the most informative ones for the user to label. Especially, GOED can alleviate the small-sized training data problem by leveraging the geometric structure of unlabeled samples in the reproducing kernel Hilbert space and thus further enhance the performance of image retrieval. Different from the conventional manifold regularization framework, the new method can effectively select the most informative samples for the user to label in image retrieval. By minimizing the expected average prediction variance on the test data, GOED has a clear geometric interpretation to select a set of the most representative samples in the database iteratively with the global optimum. Compared with the popular SVM active learning, our method is label-independent and can effectively avoid various potential problems caused by insufficient and inexactly labeled samples in RF, and is more appropriate and useful for image retrieval. Extensive experiments on both synthetic datasets and a real-world image database have been conducted to show the advantages of the proposed GOED for CIR.

In the current decade, we are witnessing a great interest in Content Based Image Retrieval (CBIR) together with a wealth of promising technologies, paved for a large number of new mechanisms and systems. In terms of mechanisms, a strong trend towards the employment of diverse Relevance Feedback (RF) approaches in CBIR systems to capture image(s) of interest has emerged. However, the need to select a particular technique in a given application domain depends on the nature of images in the collection at hand. So our paper mainly reviews and compares different approaches of CBIR using RF. Its ultimate goal is to present information about image database aspects and image features setting so as to support the selection of the appropriate CBIR with RF Techniques.

Content-based image retrieval (CBIR) has turned into an important and active potential research field with the advance of multimedia and imaging technology. It makes use of image features, such as color, texture and shape, to index images with minimal human intervention. A CBIR system can be used to locate medical images in large databases. In this paper we propose a CBIR system which describes the methodology for retrieving digital human brain magnetic resonance images (MRI) based on textural features and the Adaptive neuro-fuzzy inference system (ANFIS) learning to retrieve similar images from database in two categories: normal and tumoral. A fuzzy classifier has been used, because of the uncertainty in the results of classifier and capacity of learning. ANFIS is a good candidate for our categorization problem. Our proposed CBIR system can locate a query image in the category of normal or tumoral images in the online retrieval part. Finally, using a relevance feedback, we improve the effectiveness of our retrieval system. This research uses the knowledge of the CBIR approach to the application of medical decision support and discrimination between the normal and abnormal medical images based on features. We present and compare the results of the proposed method with the CBIR systems used in recent works. The experimental results indicate that the proposed method is reliable and has high image retrieval efficiency compared with the previous works.