Improved AdaBoost-based image retrieval with relevance feedback via paired feature learning

As the use of digital media increases, effective retrieval and management techniques become more important. In order to extract useful information from this huge amount of data, many content based image retrieval (CBIR) systems have been developed. A typical CBIR system captures image features such as color, texture, or shape of objects in the query image and tries to retrieve images from the database with similar features. The main advantage of CBIR systems with relevance feedback is that these systems take into account the gap between the high-level concepts and low-level features and subjectivity of human perception of visual content. In this paper, a CBIR system based on color histogram has been presented and a new approach for image storage and retrieval called association-based image retrieval (ABIR) has been studied. The results of both systems have been shown.

Image retrieval is an active area of research, which is growing very rapidly. Indeed, stimulated by the rapid growth in storage capacity and processing speed, the number of images in electronic collections and the World Wide Web has considerably increased over the last few years. However, with this abundance of information, people are continuously looking for tools that help them find the image(s) they are looking for within a reasonable amount of time. These tools are image retrieval engines. When using an image retrieval engine, the user is continuously interacting with the machine. First, he1 uses the system’s interface to formulate a query that expresses his needs. Second, he provides feedback about the retrieved results at each search iteration. This allows the engine to provide more accurate results by using relevance feedback (RF) techniques. Third, he may be asked to assign a goodness score or weight to each image retrieved, which helps evaluating the system’s performance. In this chapter, we will review the main interactions between human and the machine in the context of image retrieval. We will address several issues, including: Query formulation: • How the user expresses his needs and what he is looking for • The different ways the query can be formulated: keywords-based, sentence-based, query by example image, query by sketch, query by feature values, composite queries, etc. • Query by region of interest (ROI) vs. global query. • Queries with positive example only vs. queries with both positive and negative examples. • Page zero problem: finding a good image to initiate a retrieval session. Relevance feedback: we will try to answer questions like: • Why do systems use relevance feedback? • How can the user express his needs during the relevance feedback process • How this information is exploited by the system to perform operations like feature selection or the identification of the sought image. 1 Note that the masculine gender has been used strictly to facilitate reading, and is to be understood to include the feminine. Advances in Human-Computer Interaction 216 • The different families of RF techniques. • Relevance feedback with retrieval memory, i.e., taking into account the value of old iteration queries when constructing the new one. • Whether it is useful for the system to create user profiles, and the challenges it has to face. • The number of RF iterations required to obtain satisfactory results. Viewing retrieval results: • Existing viewing techniques: 2D linear presentation, 3D-based presentation, etc. • Different ways the resulting images may be ordered and presented to the user: similarity-ordered, time-ordered, event-ordered, etc. Evaluation of the retrieval performance by the user: • How the user can express his satisfaction/dissatisfaction about the retrieved images • What about the ground truth in image retrieval evaluation? • System response time and its influence on user satisfaction. • The ease of use of the system’s interface. Other issues: • User’s needs: He may be looking for a specific image, for images that meet a given need (e.g. illustrate a concept) or simply browsing the collection looking for potentially “good” images.

Although relevance feedback (RF) has been extensively studied in the content-based image retrieval community, no commercial Web image search engines support RF because of scalability, efficiency, and effectiveness issues. In this paper, we propose a unified relevance feedback framework for Web image retrieval. Our framework shows advantage over traditional RF mechanisms in the following three aspects. First, during the RF process, both textual feature and visual feature are used in a sequential way. To seamlessly combine textual feature-based RF and visual feature-based RF, a query concept-dependent fusion strategy is automatically learned. Second, the textual feature-based RF mechanism employs an effective search result clustering (SRC) algorithm to obtain salient phrases, based on which we could construct an accurate and low-dimensional textual space for the resulting Web images. Thus, we could integrate RF into Web image retrieval in a practical way. Last, a new user interface (UI) is proposed to support implicit RF. On the one hand, unlike traditional RF UI which enforces users to make explicit judgment on the results, the new UI regards the users' click-through data as implicit relevance feedback in order to release burden from the users. On the other hand, unlike traditional RF UI which hardily substitutes subsequent results for previous ones, a recommendation scheme is used to help the users better understand the feedback process and to mitigate the possible waiting caused by RF. Experimental results on a database consisting of nearly three million Web images show that the proposed framework is wieldy, scalable, and effective.

Content-based image retrieval (CBIR) has suffered from the lack of linkage between low-level features and high-level semantics. Although relevance feedback (RF) CBIR provides a promising solution involving human interaction, certain query images poorly represented by low-level features still have unsatisfactory retrieval results. An innovative method has been proposed to increase the percentage of relevance of target image database by using graph cuts theory with the maximum-flow/minimum-cut algorithm and relevance feedback. As a result, the database is reformed by keeping relevant images while discarding irrelevant images. The relevance is increased and thus during following RF-CBIR process, previously poorly represented relevant images have higher probability to appear for selection. Better performance and retrieval results can thus be achieved.

With the rapid growing volume of images in medical databases, development of efficient image retrieval systems to retrieve relevant or similar images to a query image has become an active research area. Despite many efforts to improve the performance of techniques for accurate image retrieval, its success in biomedicine thus far has been quite limited. This article presents an adaptive content-based image retrieval (CBIR) system for improving the performance of image retrieval in mammographic databases. In this work, the authors propose a new relevance feedback approach based on incremental learning with support vector machine (SVM) regression. Also, the authors present a new local perturbation method to further improve the performance of the proposed relevance feedback system. The approaches enable efficient online learning by adapting the current trained model to changes prompted by the user's relevance feedback, avoiding the burden of retraining the CBIR system. To demonstrate the proposed image retrieval system, the authors used two mammogram data sets: A set of 76 mammograms scored based on geometrical similarity and a larger set of 200 mammograms scored by expert radiologists based on pathological findings. The experimental results show that the proposed relevance feedback strategy improves the retrieval precision for both data sets while achieving high efficiency compared to offline SVM. For the data set of 200 mammograms, the authors obtained an average precision of 0.48 and an area under the precision-recall curve of 0.79. In addition, using the same database, the authors achieved a high pathology matching rate greater than 80% between the query and the top retrieved images after relevance feedback. Using mammographic databases, the results demonstrate that the proposed approach is more accurate than the model without using relevance feedback not only in image retrieval but also in pathology matching while maintaining its effectiveness for online relevance feedback applications.

This paper presents a content-based image retrieval (CBIR) system that incorporates real-valued multiple instance learning (MIL) into the user relevance feedback (RF) to learn the user's subjective visual concepts, especially where the user's most interested region and how to map the local feature vector of that region to the high-level concept pattern of the user. RF provides a way to obtain the subjectivity of the user's high-level visual concepts, and MIL enables the automatic learning of the user's high-level concepts. The user interacts with the CBIR system by relevance feedback in a way that the extent to which the image samples retrieved by the system are relevant to the user's intention is labeled. The system in turn applies the MIL method to find user's most interested image region from the feedback. A multilayer neural network that is trained progressively through the feedback and learning procedure is used to map the low-level image features to the high-level concepts.

The use of Computer-Aided Diagnosis in dermatology raises the necessity of integrating Content-Based Image Retrieval (CBIR) technologies. The latter could be helpful to untrained users as a decision support system for skin lesion diagnosis. However, classical CBIR systems perform poorly due to semantic gap. To alleviate this problem, we propose in this paper an intelligent Content-Based Dermoscopic Image Retrieval (CBDIR) system with Relevance Feedback (RF) for melanoma diagnosis that exhibits: efficient and accurate image retrieval as well as visual features extraction that is independent of any specific diagnostic method. After submitting a query image, the proposed system uses linear kernel-based active SVM, combined with histogram intersection-based similarity measure to retrieve the K most similar skin lesion images. The dominant (melanoma, benign) class in this set will be identified as the image query diagnosis. Extensive experiments conducted on our system using a 1097 image database show that the proposed scheme is more effective than CBDIR without the assistance of RF.

Relevance feedback has emerged as a powerful tool to boost the retrieval performance in content-based image retrieval (CBIR). In the past, most research efforts in this field have focused on designing effective algorithms for traditional relevance feedback. Given that a CBIR system can collect and store users' relevance feedback information in a history log, an image retrieval system should be able to take advantage of the log data of users' feedback to enhance its retrieval performance. In this paper, we propose a unified framework for log-based relevance feedback that integrates the log of feedback data into the traditional relevance feedback schemes to learn effectively the correlation between low-level image features and high-level concepts. Given the error-prone nature of log data, we present a novel learning technique, named soft label support vector machine, to tackle the noisy data problem. Extensive experiments are designed and conducted to evaluate the proposed algorithms based on the COREL image data set. The promising experimental results validate the effectiveness of our log-based relevance feedback scheme empirically.

Relevance feedback has emerged as a powerful tool to boost the retrieval performance in content-based image retrieval (CBIR). In the past, most research efforts in this field have focused on designing effective algorithms for traditional relevance feedback. Given that a CBIR system can collect and store users' relevance feedback information in a history log, an image retrieval system should be able to take advantage of the log data of users' feedback to enhance its retrieval performance. In this paper, we propose a unified framework for log-based relevance feedback that integrates the log of feedback data into the traditional relevance feedback schemes to learn effectively the correlation between low-level image features and high-level concepts. Given the error-prone nature of log data, we present a novel learning technique, named soft label support vector machine, to tackle the noisy data problem. Extensive experiments are designed and conducted to evaluate the proposed algorithms based on the COREL image data set. The promising experimental results validate the effectiveness of our log-based relevance feedback scheme empirically.

In the field of computer vision and image processing CBIR (Content based image retrieval) system become an active research area. The CBIR technique involves feature extraction based on image features (like color, shape and texture) and defining rules for comparing image. A classifier is also used by authors for image classification. For this purpose an image database taken from James Z. Wang research group website is used by many authors. Due to enormous increase in the size of database traditional methods of image retrieval by using keywords does not provide accurate result. Thus the concept of CBIR system is emerged. In last couple of years many method of feature extraction is developed but since now researcher are not able to develop a system which has gained high accuracy of human visual perception in distinguishing pictures. Therefore earlier researchers have concentrated to increase the retrieval accuracy of CBIR system by using different combination of features. The objective of this paper is to present review on a CBIR technique which combined four kinds of features set including every kind of feature for developing feature vector. Color based features (histogram), statistical features (mean, median and standard deviation), spatial features (Gabor feature and wavelet features) and texture features (GLCM) is used for forming feature vector. A SVM classifier is used with CBIR system to increase retrieval accuracy. SVM (support vector machine) is a machine learning classifier which provides good classification accuracy. Keywords: CBIR, SVM, NN, GLCM, HSV Color space, Euclidean distance matrix.

In this chapter, we describe two new approaches to content-based image retrieval (CBIR) based on preference information provided by the user interacting with an image search system. First, we present the existing methods of image retrieval with relevance feedback, which serve then as a reference for the new approaches. The first extension of the distance function-based CBIR approach makes it possible to apply this approach to complex objects. The new algorithm is based on an approximation of user preferences by a neural network. Further, we propose another approach to image retrieval, which uses reference sets to facilitate image comparisons. The methods proposed have been implemented, and compared with each other, and with the earlier approaches. Computational experiments have proven that the new preference extraction and image retrieval procedures here proposed are numerically efficient. Finally, we provide a real-life illustration of the methods proposed: an image-based hotel selection procedure.

Content-based image retrieval (CBIR) has turned into an important and active potential research field with the advance of multimedia and imaging technology. It makes use of image features, such as color, texture and shape, to index images with minimal human intervention. A CBIR system can be used to locate medical images in large databases. In this paper we propose a CBIR system which describes the methodology for retrieving digital human brain magnetic resonance images (MRI) based on textural features and the Adaptive neuro-fuzzy inference system (ANFIS) learning to retrieve similar images from database in two categories: normal and tumoral. A fuzzy classifier has been used, because of the uncertainty in the results of classifier and capacity of learning. ANFIS is a good candidate for our categorization problem. Our proposed CBIR system can locate a query image in the category of normal or tumoral images in the online retrieval part. Finally, using a relevance feedback, we improve the effectiveness of our retrieval system. This research uses the knowledge of the CBIR approach to the application of medical decision support and discrimination between the normal and abnormal medical images based on features. We present and compare the results of the proposed method with the CBIR systems used in recent works. The experimental results indicate that the proposed method is reliable and has high image retrieval efficiency compared with the previous works.

This paper presents an image representation and matching framework for image categorization in medical image archives. Categorization enables one to determine automatically, based on the image content, the examined body region and imaging modality. It is a basic step in content-based image retrieval (CBIR) systems, the goal of which is to augment text-based search with visual information analysis. CBIR systems are currently being integrated with picture archiving and communication systems for increasing the overall search capabilities and tools available to radiologists. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (GMM) along with information-theoretic image matching via the Kullback-Leibler (KL) measure. The GMM-KL framework is used for matching and categorizing X-ray images by body regions. A multidimensional feature space is used to represent the image input, including intensity, texture, and spatial information. Unsupervised clustering via the GMM is used to extract coherent regions in feature space that are then used in the matching process. A dominant characteristic of the radiological images is their poor contrast and large intensity variations. This presents a challenge to matching among the images, and is handled via an illumination-invariant representation. The GMM-KL framework is evaluated for image categorization and image retrieval on a dataset of 1500 radiological images. A classification rate of 97.5% was achieved. The classification results compare favorably with reported global and local representation schemes. Precision versus recall curves indicate a strong retrieval result as compared with other state-of-the-art retrieval techniques. Finally, category models are learned and results are presented for comparing images to learned category models.

As we all know, the content-based image retrieval (CBIR) is very time-consuming due to the extraction and matching of high dimensional and complex features. The traditional CBIR systems could not respond to a very large number of retrieval requests at mean time, which are submitted from the Internet. In this paper, we propose a content-based parallel image retrieval system to achieve high responding ability. Our system is developed on cluster architectures. It has several retrieval servers to supply the service of content-based image retrieval. Our system adopts the Browser/Server (B/S) mode. The users could visit our system though web pages. Our system uses the Symmetrical Color-Spatial Features (SCSF)[25] to represent the content of an image. The SCSF is effective and efficient for image matching because it is independent of image distortion such as rotation and flip as well as it increases the matching accuracy. The SCSF was organized by M-tree[16], which could speedup the searching procedure. Our experiments show that the image matching is quickly and efficiently with the use of SCSF. And with the support of several retrieval servers, the system could respond to many users at mean time. (Abstract)

In the current decade, we are witnessing a great interest in Content Based Image Retrieval (CBIR) together with a wealth of promising technologies, paved for a large number of new mechanisms and systems. In terms of mechanisms, a strong trend towards the employment of diverse Relevance Feedback (RF) approaches in CBIR systems to capture image(s) of interest has emerged. However, the need to select a particular technique in a given application domain depends on the nature of images in the collection at hand. So our paper mainly reviews and compares different approaches of CBIR using RF. Its ultimate goal is to present information about image database aspects and image features setting so as to support the selection of the appropriate CBIR with RF Techniques.