Clustering-based subspace SVM ensemble for relevance feedback learning

In content-based image retrieval (CBIR), a set of low-level features are extracted from an image to represent its visual content. Retrieval is performed by image example where a query image is given as input by the user and an appropriate similarity measure is used to find the best matches in the corresponding feature space. This approach suffers from the fact that there is a large discrepancy between the low-level visual features that one can extract from an image and the semantic interpretation of the image’s content that a particular user may have in a given situation. That is, users seek semantic similarity, but we can only provide similarity based on low-level visual features extracted from the raw pixel data, a situation known as the semantic gap. The selection of an appropriate similarity measure is thus an important problem. Since visual content can be represented by different attributes, the combination and importance of each set of features varies according to the user’s semantic intent. Thus, the retrieval strategy should be adaptive so that it can accommodate the preferences of different users. Relevance feedback (RF) learning has been proposed as a technique aimed at reducing the semantic gap. It works by gathering semantic information from user interaction. Based on the user’s feedback on the retrieval results, the retrieval scheme is adjusted. By providing an image similarity measure under human perception, RF learning can be seen as a form of supervised learning that finds relations between high-level semantic interpretations and low-level visual properties. That is, the feedback obtained within a single query session is used to personalize the retrieval strategy and thus enhance retrieval performance. In this chapter we present an overview of CBIR and related work on RF learning. We also present our own previous work on a RF learning-based probabilistic region relevance learning algorithm for automatically estimating the importance of each region in an image based on the user’s semantic intent.

Asymmetric propagation based batch mode active learning for image retrieval

With the emerge of multimedia, communication and processing, a typical image database has a large volume of data. It is an essential to build an efficient retrieval system to browse through the entire database. In this paper we proposed precise Relevance Feed Back (RFB) Content Based Image Retrieval (CBIR) using multiple features based on interactive retrieval approach which will extensively reduces the semantic gap between low-level features and high-level semantics. Relevance Feed Back improves the retrieval accuracy of Content Based Image Retrieval(CBIR) by modifying the query based on the user's feedback in which the user can select the most relevant images and provide a weight of preference for each relevant image. In our approach we extracted multiple features by integrating color, texture and shape features with Relevance Feed Back. We extracted color features by cumulative histogram, texture feature are exploited by using Color co-occurrence matrix (CCM) and we describe the shape information contained in an image on the basis of its significant edges. We used Euclidean distance as the similarity measure. In our experiment we used Corel real-world image databases with 1000 images, divided into 10 categories, each category containing 100 images including landscapes, animals, plants, monuments, transport (cars, planes). Our results reflects that integrating color, texture and shape for retrieving similar images improves the retrieval performance with respect to retrieval accuracy and recall rate as compared with the classical color feature. During the experiment we also observed that incorporating Relevance Feed Back (RFB) with multiple features outperforms in comparison with multifeature vector without Relevance Feed Back method.

Combining manifold ranking with active learning (MRAL for short) is one popular and successful technique for relevance feedback in content-based image retrieval (CBIR). Despite the success, conventional MRAL has two main drawbacks. First, the performance of manifold ranking is very sensitive to the scale parameter used for calculating the Laplacian matrix. Second, conventional MRAL does not take into account the redundancy among examples and thus could select multiple examples that are similar to each other. In this work, a novel MRAL framework is presented to address the drawbacks. Concretely, we first propose a self-tuning manifold ranking algorithm that can adaptively calculate the Laplacian matrix via a local scaling mechanism, and then develop a hybrid active learning algorithm by integrating three well-known selective sampling criteria, which is able to effectively and efficiently identify the most informative and diversified examples for the user to label. Experiments on 10,000 Corel images show that the proposed method is significantly more effective than some existing approaches.

Relevance Feedback is a technique used for enhancing retrieval accuracy in multimedia database systems. In this paper two novel relevance feedback algorithms are proposed for 3D object databases, in which the relative scores of various users, which express users' subjectivity, are kept accumulatively as additional descriptors. Each object is interpreted as a charged particle, whose relative scores represent the value of the charge. Based on these charges, semantic forces are calculated between the 3D objects, which are repelled or attracted properly in the feature space. The forces are of dual nature, semantic and geometric, in the first algorithm, whereas they are purely semantic in the second one. Furthermore, a novel algorithm for annotation propagation is developed, which is based on a linear prediction scheme of the changes that must be made in the feature vector of a newly added 3D object in the database, according to alterations that has already taken place to objects in the database. The combination of low and high level features in one formula, is able to fill the semantic gap as much as possible till time being, while the proposed content-free retrieval method illustrates the fact that in the long run, a purely semantic algorithm can provide excellent retrieval results.

Relevance Feedback (RF) is a powerful technique in Content-Based Image Retrieval (CBIR) system and has become a very active research topic in the past few years. At the early stage of CBIR, research primarily focused on exploring various feature representation and ignored the subjectivity of human perception. There exists a gap between high-level concepts and low-level features. As an effective solution, the RF technique has been used on many CBIR systems to improve the retrieval precision. In this paper, a combined automatic weighting and relevance feedback method is proposed to improve the retrieval performance of CBIR. An approach using genetic algorithm for computing the initial weight of feature vector was introduced. By moving the query vector and updating the weighting factors simultaneously, the convergence speed of the relevance feedback retrieval is accelerated. Experimental results show that this method achieves high accuracy and effectiveness in CBIR.

Most image segmentation algorithms extract regions satisfying visual uniformity criteria. Unfortunately, because of the semantic gap between low-level features and high-level semantics, such regions usually do not correspond to meaningful parts. This has motivated researchers to develop methods that, by introducing high-level knowledge into the segmentation process, can break through the performance ceiling imposed by the semantic gap. The main disadvantage of those methods is their lack of flexibility due to the assumption that such knowledge is provided in advance. In content-based image retrieval (CBIR), relevance feedback (RF) learning has been successfully applied as a technique aimed at reducing the semantic gap. Inspired by this, we present a RF-based CBIR framework that uses multiple instance learning to perform a semantically-guided context adaptation of segmentation parameters. A partial instantiation of this framework that uses mean shift-based segmentation is presented. Experiments show the effectiveness and flexibility of the proposed framework on real images.

We design, implement and evaluate EpistAid, an interactive relevance feedback system to support physicians towards a more efficient citation screening process for Evidence Based Health Care (EBHC). The system combines a relevance feedback algorithm with an interactive interface inspired by Tinder-like swipe interaction. To evaluate its efficiency and effectiveness in the citation screening process we conducted a user study with real users (senior medicine students) using a large EBHC dataset (Epistemonikos), with around 400,000 documents. We compared two relevance feedback algorithms, Rocchio and BM25-based. The combination of Rocchio relevance feedback with the document visualization yielded the best recall and F-1 scores, which are the most important metrics for EBHC document screening. In terms of cognitive demand and effort, BM25 relevance feedback without visualization was perceived as needing more physical and cognitive effort. EpistAid has the potential of improving the process for answering clinical questions by reducing the time needed to classify documents, as well as promoting user interaction. Our results can inform the development of intelligent user interfaces for screening research articles in the clinical domain and beyond.

Supervised learning algorithms (relevance feedback (RF) algorithms) are often used in content based image retrieval (CBIR) systems to enhance interactive search and browsing of image databases. One of the issues associated with RF based CBIR systems is the lack of a large training set. Labeling of images is a time consuming activity and user's usually do not have the patience to label a large set. The challenge is to somehow leverage the much larger set of unlabeled images to improve the performance of CBIR systems. In this paper we propose a novel RF algorithm which learns from both labeled and unlabeled data. Our proposed algorithm also uses active learning so as to maximize the information gained from a given amount of user feedback.

PurposeThis paper seeks to examine the further integration of machine translation technologies with cross language information access in providing web users the capabilities of accessing information beyond language barriers. Machine translation and cross language information access are related technologies, and yet they have their own unique contributions in handling information in multiple languages. This paper aims to demonstrate that there are many opportunities to further integrate machine translation with cross language information access, and the combination can greatly empower web users in their information access.Design/methodology/approachUsing English and Chinese as the language pair for studying, this paper looks at machine translation in query translation‐based cross language information access at multiple important aspects, which include query translation, relevance feedback, interactive cross language information access, out‐of‐vocabulary term translation, and data fusion. The goal is to obtain more insights about the wide range usages of machine translation in cross language information access, and to help the community to identify promising future directions for both machine translation and cross language access.FindingsMachine translation can be applied effectively in many places in the whole cross language information access process. Queries translated by a machine translation system are high quality and are more robust in handling potential untranslated terms. Translation enhancement, a relevance feedback method using machine translation generated returned documents, is not only a valid technique by itself, but also helps to generate more robust cross language information access performance when combined with other relevance feedback techniques. Machine translation is also found to play a significant role in resolving untranslated terms and in data fusion.Originality/valueThis set of comparative empirical studies on integrating machine translation and cross language information access was performed on a common evaluation framework, and examined integration at multiple points of the cross language access process. The experimental results demonstrate the value of further integrating machine translation in cross language information access, and identify interesting future directions for both machine translation and cross language information access research.

The performance of a Content-Based Image Retrieval System (CBIR) depends on (a) the system's adaptability to the user's information needs, which permits different types of indexing and simultaneously reduces the subjectivity of human perception for the interpretation of the image visual content and (b) the efficient organization of the extracted descriptors, which represent the rich visual information. Both issues are addressed in this paper. Descriptor organization is performed using a fuzzy classification scheme fragmented into multidimensional classes, instead of the previous works where fuzzy histograms were created in one dimension using, for example, the feature vector norm. Multidimensionality relates the descriptors with one another and thus allows a compact and meaningful visual representation by mapping the elements of the resulted feature vectors with a physical visual interpretation. Furthermore, fuzzy classification is applied for all visual content descriptors, in contrast to the previous approaches where only color information is exploited. Two kinds of content descriptors are extracted in our case; global-based and region-based. The first refers to the global image characteristics, while the second exploits the region-based properties. Regions are obtained by applying a multiresolution implementation of the Recursive Shortest Spanning Tree (RSST) algorithm, called M-RSST in this paper. The second issue is addressed by proposing a computationally efficient relevance feedback mechanism based on an optimal weight updating strategy. The scheme relies on the cross-correlation measure, instead of the Euclidean distance which is mainly used in most relevance feedback algorithms. Cross-correlation is a normalized measure, which expresses how similar the two feature vectors are and thus it indicates a metric of their content similarity. The proposed scheme can be recursively implemented in the case of multiple feedback iterations, instead of the previous approaches. Furthermore, it provides reliable results regardless of the number of selected sample and the feature vector size improving relevance feedback performance, as compared to other approaches.

In this paper, a quantization-based probabilistic feature modeling approach is proposed for relevance feedback in content-based image retrieval. We demonstrate its performance by using the resulting models within a support vector machine (SVM) based technique. Each feature component is quantized mapped to probabilistic quantities representing the likelihood of the image being relevant (and irrelevant). These probabilistic quantities are then used to derive an information divergence-based kernel function for SVM classification which we introduced in earlier work. We show that the proposed method leads to the optimal maximum likelihood solution as the knowledge of the actual underlying probability model improves (i.e.,as the feature space is partitioned into arbitrarily small regions and accurate models are known for all regions). vWe investigate several practical quantization designs for feature modeling specifically in relevance feedback applications,where the scarcity of the data high dimensionality prevent usage of vector quantization parametric modeling approaches.Our proposed framework naturally takes into account the statistics of the data that is available during relevance feedback for the purpose of discriminating between relevant irrelevant images.Experiments with the Corel dataset show that quantizers specifically designed for this application achieve gains over simple uniform quantizers (e.g.,5% to 10% in retrieval accuracy) when combined with our information divergence kernel. This kernel achieves gains (e.g.,17% in retrieval accuracy after first relevance feedback)as compared to the standard radial basis function (RBF) kernel used for SVM-based relevance feedback.

In content‐based image retrieval, relevance feedback (RF) is a prominent method for reducing the “semantic gap” between the low‐level features describing the content and the usually higher‐level meaning of user's target. Recent RF methods are able to identify complex target classes after relatively few feedback iterations. However, because the computational complexity of such methods is linear in the size of the database, retrieval can be quite slow on very large databases. To address this scalability issue for active learning‐based RF, we put forward a method that consists in the construction of an index in the feature space associated to a kernel function and in performing approximatekNNhyperplanequeries with this feature space index. The experimental evaluation performed on two image databases show that a significant speedup can be achieved at the expense of a limited increase in the number of feedback rounds. © 2008 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 18, 150–159, 2008

Earlier researches have proved that the gray co-occurrence matrix representing the texture feature is more effective than many other features in the sternum image retrieval and the relevance feedback technology implementing man-machine interactive retrieval enhance retrieval efficiency. Based on these conclusions, in this paper, a new relevance feedback method based on minimal Bayesian error rate in sternum image retrieval is proposed. The comparison of feedback retrieval result shows the approach is effective.

Language model is a popular method of exploiting linguistic regularities for document retrieval. To improve retrieval performance, the scheme of relevance feedback is adopted by adjusting the query language model using the information feedback from the retrieved documents. This study presents a new Bayesian learning approach to instantaneous and unsupervised adaptation of language model for adaptive information retrieval. We aim to compensate the domain mismatch between query and documents by adapting the query language model to meet the domains of collected documents. The maximum a posteriori adaptation is executed solely by using the input query without additional collection of adaptation data. The retrieved top N documents are utilized as relevant documents and referred as feedback to estimate mixture of language models for Bayesian document retrieval. The experiments on using TREC datasets show that the proposed method significantly outperforms the other relevance feedback methods.