An Adaptive Image Retrieval System with Relevance Feedback and Clustering

With the rapid growing volume of images in medical databases, development of efficient image retrieval systems to retrieve relevant or similar images to a query image has become an active research area. Despite many efforts to improve the performance of techniques for accurate image retrieval, its success in biomedicine thus far has been quite limited. This article presents an adaptive content-based image retrieval (CBIR) system for improving the performance of image retrieval in mammographic databases. In this work, the authors propose a new relevance feedback approach based on incremental learning with support vector machine (SVM) regression. Also, the authors present a new local perturbation method to further improve the performance of the proposed relevance feedback system. The approaches enable efficient online learning by adapting the current trained model to changes prompted by the user's relevance feedback, avoiding the burden of retraining the CBIR system. To demonstrate the proposed image retrieval system, the authors used two mammogram data sets: A set of 76 mammograms scored based on geometrical similarity and a larger set of 200 mammograms scored by expert radiologists based on pathological findings. The experimental results show that the proposed relevance feedback strategy improves the retrieval precision for both data sets while achieving high efficiency compared to offline SVM. For the data set of 200 mammograms, the authors obtained an average precision of 0.48 and an area under the precision-recall curve of 0.79. In addition, using the same database, the authors achieved a high pathology matching rate greater than 80% between the query and the top retrieved images after relevance feedback. Using mammographic databases, the results demonstrate that the proposed approach is more accurate than the model without using relevance feedback not only in image retrieval but also in pathology matching while maintaining its effectiveness for online relevance feedback applications.

This paper presents a novel method to retrieve images by learning the embedded visual concept from a set of given examples. Through a user's relevance feedback, the visual concept can be effectively learned to classify images which contain common visual entities. The learning process is started by providing a set of either positive or negative training examples and is then interactively adjusted according to the user's relevance feedback. In contrast to traditional methods, the proposed method utilizes a novel way to overcome the under-training problem which is frequently suffered in the learning process. Since no time-consuming optimization process is involved, the proposed method learns the visual concepts extremely fast. Therefore, the target concept can be learned on-line and is user-adaptable for effective retrieval of image contents. Experimental results are provided to prove the superiority of the proposed method.

This paper proposes a method for retrieving images containing a specific target person from a given image collection of group photos. This can be realized by query-by-example methods which compare the facial visual features of the target person in the given query image and of each person in the images in the image collection. However, since images are often taken under various conditions, facial appearance of the same person can vary. Since socially related people such as family and friends are often taken photos together, the people co-occurrence relations in the same images can also be a useful clue for image retrieval. Focusing on such people co-occurrence relations, we propose Bag of People (BoP) features which represent both the facial appearances of persons and their co-occurrence relations in the same images. By using the BoP features, a classifier for classifying images into two classes, images containing the target person and other images, can be trained from a small number of images labeled by user's relevance feedback. Furthermore, since the labeled images obtained by relevance feedback are much fewer than unlabeled images in the image collection, an active learning method is used to select useful images to train the classifier. When retrieving images of 24 persons in total from 550 images, after five feedback iterations, the mean average precision of 0.94 was obtained by considering the people co-occurrence relations, as against 0.69 when considering only the target person.

This paper presents a content-based image retrieval (CBIR) system that incorporates real-valued multiple instance learning (MIL) into the user relevance feedback (RF) to learn the user's subjective visual concepts, especially where the user's most interested region and how to map the local feature vector of that region to the high-level concept pattern of the user. RF provides a way to obtain the subjectivity of the user's high-level visual concepts, and MIL enables the automatic learning of the user's high-level concepts. The user interacts with the CBIR system by relevance feedback in a way that the extent to which the image samples retrieved by the system are relevant to the user's intention is labeled. The system in turn applies the MIL method to find user's most interested image region from the feedback. A multilayer neural network that is trained progressively through the feedback and learning procedure is used to map the low-level image features to the high-level concepts.

Content-based image retrieval (CBIR) has become one of the most active research areas in the past few years. Many visual feature representations have been explored and many systems built. While these research efforts establish the basis of CBIR, the usefulness of the proposed approaches is limited. Specifically, these efforts have relatively ignored two distinct characteristics of CBIR systems: (1) the gap between high level concepts and low level features; (2) subjectivity of human perception of visual content. This paper proposes a relevance feedback based interactive retrieval approach, which effectively takes into account the above two characteristics in CBIR. During the retrieval process, the user's high level query and perception subjectivity are captured by dynamically updated weights based on the user's relevance feedback. The experimental results show that the proposed approach greatly reduces the user's effort of composing a query and captures the user's information need more precisely.

This paper proposes a new texture image retrieval scheme based on contourlet transform and support vector machines (SVMs). In the scheme, the energies and the generalized Gaussian distribution (GGD) parameters are used to represent the contourlet subband features. Using the representations, a two-run SVM retrieval algorithm which employs an one-class SVM followed by a two-class SVM is proposed to carry out the perceptual similarity measurement. For the query image, the one-class SVM is used to obtain the effective initial training set with positive and negative samples. Using these initial samples, the two-class SVM is applied to refine on the image classification subject to the user’s relevance feedback. Compared with existing texture image retrieval methods, the proposed retrieval scheme is demonstrated respectively to be effective on the VisTex database of 640 texture images and the Brodatz database of 1760 texture images. Experimental results have shown that the proposed retrieval scheme can attain 99.38% and 98.07% of the average rates respectively for the two databases.KeywordsContourlet transformgeneralized Gaussian distributionsupport vector machinerelevance feedbacktexture image retrieval

Interactive image retrieval using fuzzy sets

The expressive power of logic is believed to be able to model most of the fundamental aspects of information retrieval (IR). However, it is also understood that classical logic is ineffective for handling partiality and uncertainty in IR. Applying non-classical logics such as the AGM belief revision logic and the possibilistic logic to adaptive information retrieval is appealing since they provide a powerful and rigorous framework to model partiality and uncertainty inherent in any IR processes. The maxi-adjustment method, which is an effective computational apparatus of the AGM paradigm, is applied to develop the learning components of the adaptive information agents. Essentially, maxi-adjustment allows the partial representation K of a user's information needs N to be refined gradually based on the user's relevance feedback t. Generally speaking, learning in adaptive information agents is characterised by the AGM belief revision Kt*. On the other hand, possibilistic logic supports a gradated assessment of the uncertainty arising from matching K with the imperfect characterisation d of an information object D. Information matching in adaptive information agents is underpinned by K ƕ d, where ƕ is the possibilistic inference relation. This paper illustrates how maxi-adjustment and possibilistic deduction can be applied to develop the adaptive information agents. Their impact on the agents learning autonomy and explanatory power is also discussed.

In this paper, we describe a new scheme to learn dynamic user's interests in an automated information filtering and gathering system running on the Internet. Our scheme is aimed to handle multiple domains of long-term and short-term user's interests simultaneously, which is learned through positive and negative user's relevance feedback. We developed a 3-descriptor approach to represent the user's interest categories. Using a learning algorithm derived for this representation, our scheme adapts quickly to significant changes in user interest, and is also able to learn exceptions to interest categories.

This chapter studies the user relevance feedback in image retrieval. We take this problem as a standard two-class pattern classification problem aiming at refining the retrieval precision by learning through the user relevance feedback data. However, we have investigated the problem by noting two important unique characteristics of the problem: small sample collection and asymmetric sample distributions between positive and negative samples. We have developed a novel approach to empirical Bayesian learning to solve for this problem by explicitly exploiting the two unique characteristics, which is the methodology of BAyesian Learning in ASymmetric and Small sample collections, thus called BALAS. In BALAS different learning strategies are used for positive and negative sample collections, respectively, based on the two unique characteristics. By defining the relevancy confidence as the relevant posterior probability, we have developed an integrated ranking scheme in BALAS which complementarily combines the subjective relevancy confidence and the objective similarity measure to capture the overall retrieval semantics. The experimental evaluations have confirmed the rationale of the proposed ranking scheme, and have also demonstrated that BALAS is superior to an existing relevance feedback method in the current literature in capturing the overall retrieval semantics.

Online image repositories such as Flickr contain hundreds of millions of images and are growing quickly. Along with that the needs for supporting indexing, searching and browsing is becoming more and more pressing. In this work we will employ the image content as a source of information to retrieve images. We study the representation of images by Latent Dirichlet Allocation (LDA) models for content-based image retrieval. Image representations are learned in an unsupervised fashion, and each image is modeled as the mixture of topics/object parts depicted in the image. This allows us to put images into subspaces for higher-level reasoning which in turn can be used to find similar images. Different similarity measures based on the described image representation are studied. The presented approach is evaluated on a real world image database consisting of more than 246,000 images and compared to image models based on probabilistic Latent Semantic Analysis (pLSA). Results show the suitability of the approach for large-scale databases. Finally we incorporate active learning with user relevance feedback in our framework, which further boosts the retrieval performance.

Automatic medical image annotation and retrieval

With the development of the era of internet, an increasing number of images flow into people’s daily life, and it’s really a challenge to quickly search interesting images in such a huge image database. The most advanced method is using a deep neural network to get hash code of images to achieve fast image retrieval at present. However, people usually use the single pooling method to screen the image pixels when designing the neural network, and some effective information of the image will be gradually lost due to the single pooling method with the number of network deepening. Aiming at this problem, this paper proposes a convolutional neural network combining multiple pooling methods to preserve the effective information of the image as much as possible. To verify the effectiveness of the proposed method, many experiments are carried out on the CIFRA-10, NUS-WIDE and MNIST datasets. The experimental results show that the proposed method is better than most existing hash-based image retrieval methods.

Most existing object-based image retrieval systems are based on single object matching, with its main limitation being that one individual image region (object) can hardly represent the user’s retrieval target, especially when more than one object of interest is involved in the retrieval. Integrated Region Matching (IRM) has been used to improve the retrieval accuracy by evaluating the overall similarity between images and incorporating the properties of all the regions in the images. However, IRM does not take the user’s preferred regions into account and has undesirable time complexity. In this article, we present a Feedback-based Image Clustering and Retrieval Framework (FIRM) using a novel image clustering algorithm and integrating it with Integrated Region Matching (IRM) and Relevance Feedback (RF). The performance of the system is evaluated on a large image database, demonstrating the effectiveness of our framework in catching users’ retrieval interests in object-based image retrieval.

Medical image indexing and storage is gaining increased importance. Indexing and retrieval of these images efficiently is becomes an essential task. Indexing of medical images using text or numbers is a cumbersome task, difficult to memorize and time consuming. Indexing and Retrieval of images can be done through query by text and query by image which is also known as content based image retrieval (CBIR). Different medical images possess different texture and shape features. This paper presents a novel and hybrid approach of managing a huge medical image database and retrieving a medical image from the database by fusion of shape and texture features. Image is retrieved by comparing the features of query image and images of database. We attach some document information with each medical image such as patient identity, diseases, age, and case history. It can be used for further diagnostic and analysis purpose. Further the proposed algorithm also updates the database automatically if new query is found. MATLAB ® 7.01 and its image processing toolbox have been used to implement the algorithm.