Long Term Relevance Feedback: A Probabilistic Axis Re-weighting Update Scheme

With the rapid growing volume of images in medical databases, development of efficient image retrieval systems to retrieve relevant or similar images to a query image has become an active research area. Despite many efforts to improve the performance of techniques for accurate image retrieval, its success in biomedicine thus far has been quite limited. This article presents an adaptive content-based image retrieval (CBIR) system for improving the performance of image retrieval in mammographic databases. In this work, the authors propose a new relevance feedback approach based on incremental learning with support vector machine (SVM) regression. Also, the authors present a new local perturbation method to further improve the performance of the proposed relevance feedback system. The approaches enable efficient online learning by adapting the current trained model to changes prompted by the user's relevance feedback, avoiding the burden of retraining the CBIR system. To demonstrate the proposed image retrieval system, the authors used two mammogram data sets: A set of 76 mammograms scored based on geometrical similarity and a larger set of 200 mammograms scored by expert radiologists based on pathological findings. The experimental results show that the proposed relevance feedback strategy improves the retrieval precision for both data sets while achieving high efficiency compared to offline SVM. For the data set of 200 mammograms, the authors obtained an average precision of 0.48 and an area under the precision-recall curve of 0.79. In addition, using the same database, the authors achieved a high pathology matching rate greater than 80% between the query and the top retrieved images after relevance feedback. Using mammographic databases, the results demonstrate that the proposed approach is more accurate than the model without using relevance feedback not only in image retrieval but also in pathology matching while maintaining its effectiveness for online relevance feedback applications.

With the emerge of multimedia, communication and processing, a typical image database has a large volume of data. It is an essential to build an efficient retrieval system to browse through the entire database. In this paper we proposed precise Relevance Feed Back (RFB) Content Based Image Retrieval (CBIR) using multiple features based on interactive retrieval approach which will extensively reduces the semantic gap between low-level features and high-level semantics. Relevance Feed Back improves the retrieval accuracy of Content Based Image Retrieval(CBIR) by modifying the query based on the user's feedback in which the user can select the most relevant images and provide a weight of preference for each relevant image. In our approach we extracted multiple features by integrating color, texture and shape features with Relevance Feed Back. We extracted color features by cumulative histogram, texture feature are exploited by using Color co-occurrence matrix (CCM) and we describe the shape information contained in an image on the basis of its significant edges. We used Euclidean distance as the similarity measure. In our experiment we used Corel real-world image databases with 1000 images, divided into 10 categories, each category containing 100 images including landscapes, animals, plants, monuments, transport (cars, planes). Our results reflects that integrating color, texture and shape for retrieving similar images improves the retrieval performance with respect to retrieval accuracy and recall rate as compared with the classical color feature. During the experiment we also observed that incorporating Relevance Feed Back (RFB) with multiple features outperforms in comparison with multifeature vector without Relevance Feed Back method.

How to find the image we need expediently in the tremendous database is one of the most important issues in content-based image retrieval (CBIR). In this paper, we presented an image retrieval system based on image content using fuzzy logic and proposed a new concept on partition the entire database based on content self-organized. In detail, we used modified fuzzy c-means (MFCM) clustering scheme to cluster the database before retrieval and defined a new method to determine the fuzzy weight. To improve the retrieval effectiveness more, this paper also used interactive query (IQ) and presented a new relevance feedback (RF) framework. On all accounts, we designed and implemented a content-based self-organized and interactive retrieval system. Experiments show that it contributes to improving retrieval effectiveness and will be applied in the E-government.

In this paper, a quantization-based probabilistic feature modeling approach is proposed for relevance feedback in content-based image retrieval. We demonstrate its performance by using the resulting models within a support vector machine (SVM) based technique. Each feature component is quantized mapped to probabilistic quantities representing the likelihood of the image being relevant (and irrelevant). These probabilistic quantities are then used to derive an information divergence-based kernel function for SVM classification which we introduced in earlier work. We show that the proposed method leads to the optimal maximum likelihood solution as the knowledge of the actual underlying probability model improves (i.e.,as the feature space is partitioned into arbitrarily small regions and accurate models are known for all regions). vWe investigate several practical quantization designs for feature modeling specifically in relevance feedback applications,where the scarcity of the data high dimensionality prevent usage of vector quantization parametric modeling approaches.Our proposed framework naturally takes into account the statistics of the data that is available during relevance feedback for the purpose of discriminating between relevant irrelevant images.Experiments with the Corel dataset show that quantizers specifically designed for this application achieve gains over simple uniform quantizers (e.g.,5% to 10% in retrieval accuracy) when combined with our information divergence kernel. This kernel achieves gains (e.g.,17% in retrieval accuracy after first relevance feedback)as compared to the standard radial basis function (RBF) kernel used for SVM-based relevance feedback.

Relevance feedback is an iterative search technique to bridge the semantic gap between the high level user intention and low level data representation. This technique interactively determines a user's desired output or query concept by asking the user whether certain proposed 3D models are relevant or not. For a relevance feedback algorithm to be effective, it must grasp a user's query concept accurately. In this paper, we propose a relevance feedback framework based on Hidden Markov Model Regression (HMMR) in content-based 3D model retrieval systems. Given a 3D model retrieval system, we collect and store user's feedback and use HMMR to enhance the retrieval performances. Experimental results show that this algorithm achieves higher search accuracy than traditional query refinement schemes.

In recent years, the rapid expansion of multimedia applications, partly due to the exponential growth of the Internet, has proliferated over the daily life of computer users (Yang & Hurson, 2006). The integration of wireless communication, pervasive computing, and ubiquitous data processing with multimedia database systems has enabled the connection and fusion of distributed multimedia data sources. In addition, the emerging applications, such as smart classroom, digital library, habitat/environment surveillance, traffic monitoring, and battlefield sensing, have provided increasing motivation for conducting research on multimedia content representation, data delivery and dissemination, data fusion and analysis, and contentbased retrieval. Consequently, research on multimedia technologies is of increasing importance in computer society. In contrast with traditional text-based systems, multimedia applications usually incorporate much more powerful descriptions of human thought—video, audio, and images (Karpouzis, Raouzaiou, Tzouveli, Iaonnou, & Kollias, 2003; Liu, Bao, Yu, & Xu, 2005; Yang & Hurson, 2005). Moreover, the large collections of data in multimedia systems make it possible to resolve more complex data operations such as imprecise query or content-based retrieval. For instance, the image database systems may accept an example picture and return the most similar images of the example (Cox, Miller, & Minka, 2000; Hsu, Chua, & Pung, 2000; Huang, Chang, & Huang, 2003). However, the conveniences of multimedia applications come with challenges to the existing data management schemes: • Efficiency: Multimedia applications generally require more resources; however, the storage space and processing power are limited in many practical systems, for example, mobile devices and wireless networks (Yang & Hurson, 2005). Due to the large data volume and complicated operations of multimedia applications, new methods are needed to facilitate efficient representation, retrieval, and processing of multimedia data while considering the technical constraints. • Semantic Gap: There is a gap between user perception of multimedia entities and physical representation/access mechanism of multimedia data. Users often browse and desire to access multimedia data at the object level (“entities” such as human beings, animals, or buildings). However, the existing multimedia retrieval systems tend to access multimedia data based on their lower-level features (“characteristics” such as color patterns and textures), with little regard to combining these features into data objects. This representation gap often leads to higher processing cost and unexpected retrieval results. The representation of multimedia data according to human’s perspective is one of the focuses in recent research activities; however, few existing systems provide automated identification or classification of objects from general multimedia collections. • Heterogeneity: The collections of multimedia data are often diverse and poorly indexed. In a distributed environment, because of the autonomy and heterogeneity of data sources, multimedia data objects are often represented in heterogeneous formats. The difference in data formats further leads to the difficulty of incorporating multimedia data objects under a unique indexing framework. • Semantic Unawareness: The present research on content-based multimedia retrieval is based on feature vectors—features are extracted from audio/video streams or image pixels, empirically or heuristically, and combined into vectors according to the application criteria. Because of the application-specific multimedia formats, the feature-based paradigm lacks scalability and accuracy.

In recent years we have seen the development of several novel content-based retrieval (CBR) systems that have had success by focusing on a specific domain and exploiting domain-speci c information. CBR systems allow users to query a media database using item content as opposed submitting a text-based query. In many CBR applications, the input to the search process is a complicated object that may be composed of several constituent parts. The proposed approach performs CBR queries by decomposing a complex query into several heterogeneous queries. We have developed a multi-index, multi-object CBR framework for geospatial imagery retrieval that extracts features specifically developed for high-resolution commercial satellite imagery. The results of these queries will be spatially summarized for a user based on both retrieval score and spatial distance. This allows results to be presented in a logical manner to allow for more efficient interpretation by the user. Further, we propose to develop an additional search capability that allows for multi-object searches by spatial configuration rather than simply by object-to-object correspondence. Additionally, to confront situations where a user has determined that certain search results are not relevant, we will provide online and memory-based relevance feedback algorithms for use with multi-index, multi-object CBR systems. The experimental results demonstrate the efficiency and accuracy of the proposed methods; moreover, through the fusion of multi-index and multi-object search techniques, we are able to construct new, sophisticated query mechanisms.

Similarity measure has been one of the critical issues for successful content-based retrieval. Simple Euclidean or quadratic forms of distance are often inadequate, as they do not correspond to perceived similarity, nor adapt to different applications. Relevance feedback and/or iterative refinement techniques, based on the user feedback, have been proposed to adjust the similarity metric or the feature space. However, this learning process potentially renders those indices for facilitating high dimensional indexing, such as R-tree useless, as those indexing techniques usually assume a predetermined similarity measure. In this paper, we propose a simultaneous learning and indexing technique, for efficient content-based retrieval of images, that can be described by feature vectors. This technique builds a compact high-dimensional index, while taking into account that the raw feature space needs to be adjusted for each new application. Consequently, much better efficiency can be achieved, as compared to those techniques which do not make provisions for efficient indexing.

Content-based retrieval in video databases has become an important task with the availability of large quantities of data in both public and proprietary archives. Most of video systems are based on feature classification, but problems appear because of “semantic gap” between high-level human concepts and the machine-readable low-level visual features. In this paper we adopt a relevance feedback approach (RF) to bridge the semantic gap by progressively collecting feedback from the user, which allows the machine to discover the semantic meanings of objects or events. Experimental tests conducted on more than 91 hours of video footage show an improvement of up to 90% in retrieval accuracy, compared to classic classification-based retrieval.

In this paper, the content-based retrieval with audio examples based on MP3 digital music archive for one singer is considered. In our proposed method, the Sub-Band Coefficients (SBC) in a MP3 frame is used for feature extraction. Both Quantization-Tree indexing (QT) approach and the Melody-Line Tracking (MLT) approach are proposed for indexing MP3 objects. Finally, a two-stage comparison method is used to measure the similarity between MP3 objects. We also employ the measurement of the similarity between query sample and database items for a singer. Evaluations on a content-based MP3 retrieval system are performed. Experimental results show that our proposed approach can achieve good performance and high accuracy.

Content-Based Image Retrieval (CBIR) system existed a gap between high-level concepts and low-level features. As an effective solution, the Relevance Feedback (RF) technique has been used on many CBIR systems to improve the retrieval precision. In order to further improve convergence speed and retrieval accuracy, a novel relevance feedback method was proposed. According to feedback from user, image feature was weighted and adjusted in the novel method.

As the size of multimedia database grows, it becomes impractical to manually annotate all contents and attributes of the media, and the difficulty in finding desired information increases. To copy with these challenges, content-based multimedia retrieval systems have been developed for various applications. The chapter not only provides a conceptual architecture for the design of content-based retrieval system, but also discusses essential components of retrieval system and their research issues, including feature extraction and representation, dimension reduction of feature vector, indexing, and query specifications. As content-based multimedia retrieval is a young research field and there exists many challenging research problems, this chapter also addresses several research issues for the future research.

We introduce the problem of domain adaptation for content-based retrieval and propose a domain adaptation method based on relative aggregation points (RAPs). Content-based retrieval including image retrieval and spoken document retrieval enables a user to input examples as a query, and retrieves relevant data based on the similarity to the examples. However, input examples and relevant data can be dissimilar, especially when domains from which the user selects examples and from which the system retrieves data are different. In content-based geographic object retrieval, for example, suppose that a user who lives in Beijing visits Kyoto, Japan, and wants to search for relatively inexpensive restaurants serving popular local dishes by means of a content-based retrieval system. Since such restaurants in Beijing and Kyoto are dissimilar due to the difference in the average cost and areas' popular dishes, it is difficult to find relevant restaurants in Kyoto based on examples selected in Beijing. We propose a solution for this problem by assuming that RAPs in different domains correspond, which may be dissimilar but play the same role. A RAP is defined as the expectation of instances in a domain that are classified into a certain class, e.g. the most expensive restaurant, average restaurant, and restaurant serving the most popular dishes. Our proposed method constructs a new feature space based on RAPs estimated in each domain and bridges the domain difference for improving content-based retrieval in heterogeneous domains. To verify the effectiveness of our proposed method, we evaluated various methods with a test collection developed for content-based geographic object retrieval. Experimental results show that our proposed method achieved significant improvements over baseline methods. Moreover, we observed that the search performance of content-based retrieval in heterogeneous domains was significantly lower than that in homogeneous domains. This finding suggests that relevant data for the same search intent depend on the search context, that is, the location where the user searches and the domain from which the system retrieves data.

The task of a content-based image retrieval (CBIR) system is to cater to users who expect to get relevant images with high precision and efficiency in response to query images. This paper presents a concept learning approach that integrates a mixture model of the data, relevance feedback and long-term continuous learning. The concepts are incrementally refined with increased retrieval experiences. The concept knowledge can be immediately transplanted to deal with the dynamic database situations such as insertion of new images, removal of existing images and query images, which are outside the database. Experimental results on Corel database show the efficacy of our approach.

This paper introduces a multimodal approach for reranking of image retrieval results based on relevance feedback. We consider the problem of reordering the ranked list of images returned by an image retrieval system, in such a way that relevant images to a query are moved to the first positions of the list. We propose a Markov random field (MRF) model that aims at classifying the images in the initial retrieval-result list as relevant or irrelevant; the output of the MRF is used to generate a new list of ranked images. The MRF takes into account (1) the rank information provided by the initial retrieval system, (2) similarities among images in the list, and (3) relevance feedback information. Hence, the problem of image reranking is reduced to that of minimizing an energy function that represents a trade-off between image relevance and interimage similarity. The proposed MRF is a multimodal as it can take advantage of both visual and textual information by which images are described with. We report experimental results in the IAPR TC12 collection using visual and textual features to represent images. Experimental results show that our method is able to improve the ranking provided by the base retrieval system. Also, the multimodal MRF outperforms unimodal (i.e., either text-based or image-based) MRFs that we have developed in previous work. Furthermore, the proposed MRF outperforms baseline multimodal methods that combine information from unimodal MRFs.