Manifold-Ranking-Based Keyword Propagation for Image Retrieval

In content-based image retrieval (CBIR), a set of low-level features are extracted from an image to represent its visual content. Retrieval is performed by image example where a query image is given as input by the user and an appropriate similarity measure is used to find the best matches in the corresponding feature space. This approach suffers from the fact that there is a large discrepancy between the low-level visual features that one can extract from an image and the semantic interpretation of the image’s content that a particular user may have in a given situation. That is, users seek semantic similarity, but we can only provide similarity based on low-level visual features extracted from the raw pixel data, a situation known as the semantic gap. The selection of an appropriate similarity measure is thus an important problem. Since visual content can be represented by different attributes, the combination and importance of each set of features varies according to the user’s semantic intent. Thus, the retrieval strategy should be adaptive so that it can accommodate the preferences of different users. Relevance feedback (RF) learning has been proposed as a technique aimed at reducing the semantic gap. It works by gathering semantic information from user interaction. Based on the user’s feedback on the retrieval results, the retrieval scheme is adjusted. By providing an image similarity measure under human perception, RF learning can be seen as a form of supervised learning that finds relations between high-level semantic interpretations and low-level visual properties. That is, the feedback obtained within a single query session is used to personalize the retrieval strategy and thus enhance retrieval performance. In this chapter we present an overview of CBIR and related work on RF learning. We also present our own previous work on a RF learning-based probabilistic region relevance learning algorithm for automatically estimating the importance of each region in an image based on the user’s semantic intent.

Relevance Feedback (RF) is an important technique to improve information retrieval and has emerged as one of the hottest topics for both the industry and academic researchers. The performance of RF depends on feedback information. As the volume of feedback information gradually increases, Explicit Relevance Feedback (ERF) is attractive. In this paper, we focus on the ERF in which Feedback information is given. With regards to the lack of information in ERF, we apply Pseudo Relevance Feedback (PRF) to enhance retrieval effectiveness. However, the instability of PRF can result in a negative impact on retrieval performance. We use feedback information to define features and propose a classification model to predict which query can benefit from PRF. Then, we form relevance feedback fusion by the prediction of PRF performance. This method is designed to exploit the strengths of PRF and ERF while avoiding some weaknesses of these approaches. Experiment results show that our approach is feasible and effective.

As information retrieval systems evolve to deal with multimedia content, we see the dimensions of content feature space increasing, and relevance feedback being employed to provide more accurate query results. In this paper, we propose using Tree-structured Vector Quantization (TSVQ) to index high-dimensional data for supporting efficient similarity searches and effective relevance feedback. To support efficient similarity searches, we first use TSVQ to cluster data and store each cluster in a sequential file. We then model a similarity search as a classification problem—similar objects are much more likely to be found in the clusters into which the query object is classified. When relevance feedback is considered, and thereby features are weighted differently, we show that our approach remains very effective. Our empirical study on both a 51K and a one-million-image dataset shows that tackling indexing as a classification problem and solving the problem with TSVQ is efficient, effective, and scalable with respect to both data dimensions and dataset size.

In this paper, a quantization-based probabilistic feature modeling approach is proposed for relevance feedback in content-based image retrieval. We demonstrate its performance by using the resulting models within a support vector machine (SVM) based technique. Each feature component is quantized mapped to probabilistic quantities representing the likelihood of the image being relevant (and irrelevant). These probabilistic quantities are then used to derive an information divergence-based kernel function for SVM classification which we introduced in earlier work. We show that the proposed method leads to the optimal maximum likelihood solution as the knowledge of the actual underlying probability model improves (i.e.,as the feature space is partitioned into arbitrarily small regions and accurate models are known for all regions). vWe investigate several practical quantization designs for feature modeling specifically in relevance feedback applications,where the scarcity of the data high dimensionality prevent usage of vector quantization parametric modeling approaches.Our proposed framework naturally takes into account the statistics of the data that is available during relevance feedback for the purpose of discriminating between relevant irrelevant images.Experiments with the Corel dataset show that quantizers specifically designed for this application achieve gains over simple uniform quantizers (e.g.,5% to 10% in retrieval accuracy) when combined with our information divergence kernel. This kernel achieves gains (e.g.,17% in retrieval accuracy after first relevance feedback)as compared to the standard radial basis function (RBF) kernel used for SVM-based relevance feedback.

Relevance feedback has emerged as a powerful tool to boost the retrieval performance in content-based image retrieval (CBIR). In the past, most research efforts in this field have focused on designing effective algorithms for traditional relevance feedback. Given that a CBIR system can collect and store users' relevance feedback information in a history log, an image retrieval system should be able to take advantage of the log data of users' feedback to enhance its retrieval performance. In this paper, we propose a unified framework for log-based relevance feedback that integrates the log of feedback data into the traditional relevance feedback schemes to learn effectively the correlation between low-level image features and high-level concepts. Given the error-prone nature of log data, we present a novel learning technique, named soft label support vector machine, to tackle the noisy data problem. Extensive experiments are designed and conducted to evaluate the proposed algorithms based on the COREL image data set. The promising experimental results validate the effectiveness of our log-based relevance feedback scheme empirically.

Relevance feedback has emerged as a powerful tool to boost the retrieval performance in content-based image retrieval (CBIR). In the past, most research efforts in this field have focused on designing effective algorithms for traditional relevance feedback. Given that a CBIR system can collect and store users' relevance feedback information in a history log, an image retrieval system should be able to take advantage of the log data of users' feedback to enhance its retrieval performance. In this paper, we propose a unified framework for log-based relevance feedback that integrates the log of feedback data into the traditional relevance feedback schemes to learn effectively the correlation between low-level image features and high-level concepts. Given the error-prone nature of log data, we present a novel learning technique, named soft label support vector machine, to tackle the noisy data problem. Extensive experiments are designed and conducted to evaluate the proposed algorithms based on the COREL image data set. The promising experimental results validate the effectiveness of our log-based relevance feedback scheme empirically.

Relevance Feedback is a technique used for enhancing retrieval accuracy in multimedia database systems. In this paper two novel relevance feedback algorithms are proposed for 3D object databases, in which the relative scores of various users, which express users' subjectivity, are kept accumulatively as additional descriptors. Each object is interpreted as a charged particle, whose relative scores represent the value of the charge. Based on these charges, semantic forces are calculated between the 3D objects, which are repelled or attracted properly in the feature space. The forces are of dual nature, semantic and geometric, in the first algorithm, whereas they are purely semantic in the second one. Furthermore, a novel algorithm for annotation propagation is developed, which is based on a linear prediction scheme of the changes that must be made in the feature vector of a newly added 3D object in the database, according to alterations that has already taken place to objects in the database. The combination of low and high level features in one formula, is able to fill the semantic gap as much as possible till time being, while the proposed content-free retrieval method illustrates the fact that in the long run, a purely semantic algorithm can provide excellent retrieval results.

Synthetic pattern generation for imbalanced learning in image retrieval

This paper presents a subspace SVM ensemble algorithm for adaptive relevance feedback (RF) learning. Our method deals with the case that user's relevance feedback examples are usually insufficient and overlapped together in feature space, which decreases the learning effectiveness of RF classifiers. To enhance classification efficiency in such case, multiple SVMs are learned by clustering-based training set partition, each of which fits its cluster-specific sample distribution and gives labeling regressions to test samples that fall within this cluster. To adapt features to sample distribution within each cluster, AdaBoost feature selection is conducted onto pyramid Haar of H&I bands in HSI space. In AdaBoost, we evaluate the feature discriminative ability by an entropy-based uncertainty criterion, based on which an Eigen feature subspace is constructed in cluster-specific SVM training. Finally, regression results of multiple SVMs are probabilistic assembled to give the final labeling prediction for test image. We compare our cluster-based cascade SVMs (CSS) RF method in COREL 5,000 database with: 1. Single SVM; 2. Active Learning SVM [5]; 3. Bootstrap Sampling SVM [7]. The superior experimental results demonstrate the efficiency of our algorithm.

Region-based image retrieval and relevance feedback are two important methods to bridge the gap between the low-level visual features and the high-level semantic concepts in content-based image retrieval. In this paper, we address the issue of introducing the relevance feedback mechanism into the region-based image retrieval with online feature selection during each feedback round. Our contribution is two-fold. (1) A novel region-based image representation is proposed. Based on a generative model, a fuzzy codebook is extracted from the original region-based features, which represents the images in a uniform real-value feature space. (2) A feature selection criterion is developed and an effective relevance feedback algorithm is implemented in a boosting manner to simultaneously select the optimal features from the fuzzy codebook, and generate a strong ensemble classifier over the selected features. Experimental results show that the proposed scheme can substantially improve the retrieval performance of the region-based image retrieval system.

In content‐based image retrieval, relevance feedback (RF) is a prominent method for reducing the “semantic gap” between the low‐level features describing the content and the usually higher‐level meaning of user's target. Recent RF methods are able to identify complex target classes after relatively few feedback iterations. However, because the computational complexity of such methods is linear in the size of the database, retrieval can be quite slow on very large databases. To address this scalability issue for active learning‐based RF, we put forward a method that consists in the construction of an index in the feature space associated to a kernel function and in performing approximatekNNhyperplanequeries with this feature space index. The experimental evaluation performed on two image databases show that a significant speedup can be achieved at the expense of a limited increase in the number of feedback rounds. © 2008 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 18, 150–159, 2008

This paper introduces the concept of relevance feedback for retrieving time series data. Data in medical field handles various kinds of time series data, such as EHR (electronic health record) and consumer health data. Analyzing and accessing such time series data gets to be crucial for doctors to improve the quality of medical treatment. When we are going to develop an interactive interface for supporting a doctor to retrieve time series data of patients, it is supposed that similarity judgment on time series data depends on the knowledge and purpose of a doctor. Therefore, we employs relevance feedback-based retrieval system, which modifies similarity calculation based on the feedback information given by a doctor. In order to obtain enough feedback information from a doctor, we are going to employing two kinds of feedback mechanism: relevance feedback and graphical annotation-based feedback. This paper describes the concept of our relevance feedback-based retrieval system and the feedback mechanisms.

Relevance feedback is an iterative search technique to bridge the semantic gap between the high level user intention and low level data representation. This technique interactively determines a user's desired output or query concept by asking the user whether certain proposed 3D models are relevant or not. For a relevance feedback algorithm to be effective, it must grasp a user's query concept accurately. In this paper, we propose a relevance feedback framework based on Hidden Markov Model Regression (HMMR) in content-based 3D model retrieval systems. Given a 3D model retrieval system, we collect and store user's feedback and use HMMR to enhance the retrieval performances. Experimental results show that this algorithm achieves higher search accuracy than traditional query refinement schemes.

Content-based image retrieval (CBIR) has suffered from the lack of linkage between low-level features and high-level semantics. Although relevance feedback (RF) CBIR provides a promising solution involving human interaction, certain query images poorly represented by low-level features still have unsatisfactory retrieval results. An innovative method has been proposed to increase the percentage of relevance of target image database by using graph cuts theory with the maximum-flow/minimum-cut algorithm and relevance feedback. As a result, the database is reformed by keeping relevant images while discarding irrelevant images. The relevance is increased and thus during following RF-CBIR process, previously poorly represented relevant images have higher probability to appear for selection. Better performance and retrieval results can thus be achieved.

In order to assist doctors to diagnose mammogram. In connection with the similar lesions retrieval problem of microcalcification cluster in mammogram, we pursue a new algorithm with multi-feature fusion and relevance feedback. Multi-feature fusion of this method adopts multi-distance measure to calculate the similarity directing at different features. Experiment is based on mammogram image database which contain 250 mammogram images and each image contains calcification cluster, we verified the retrieval performance by the precision - recall ratio (PVR) of single feature, feature fusion and relevance feedback. Experimental results show that the method has a better retrieval result than these methods which based single feature and feature fusion which using single distance measurement.