Optimal query-based relevance feedback in medical image retrieval using score fusion-based classification.

Relevance feedback (RF) approaches are use to improve the performance of content-based image retrieval (CBIR) systems. In this paper, a RF approach based on modification of similarity measure using particle swarm optimization (PSO) in a medical X-ray image retrieval system is proposed. In this algorithm, using PSO, the significance of each feature in the similarity measure is modified to image retrieval. This modification causes that good features have major effect in relevant image retrieval. The defined fitness function in PSO uses relevant and irrelevant retrieved images with different strategies, simultaneously. The relevant and irrelevant images are used to exhort and penalize similarity measure, respectively. To evaluate, the proposed RF is integrated to a CBIR system based on semantic classification. In this system, using merging scheme in a hierarchical structure, the overlapped classes are merged together and determined search space for each query image. The proposed RF evaluated on a database consisting of 10000 medical X-ray images of 57 classes. The proposed algorithm provides the improvement, effectiveness more than the literature.

As the use of digital media increases, effective retrieval and management techniques become more important. In order to extract useful information from this huge amount of data, many content based image retrieval (CBIR) systems have been developed. A typical CBIR system captures image features such as color, texture, or shape of objects in the query image and tries to retrieve images from the database with similar features. The main advantage of CBIR systems with relevance feedback is that these systems take into account the gap between the high-level concepts and low-level features and subjectivity of human perception of visual content. In this paper, a CBIR system based on color histogram has been presented and a new approach for image storage and retrieval called association-based image retrieval (ABIR) has been studied. The results of both systems have been shown.

With the rapid growing volume of images in medical databases, development of efficient image retrieval systems to retrieve relevant or similar images to a query image has become an active research area. Despite many efforts to improve the performance of techniques for accurate image retrieval, its success in biomedicine thus far has been quite limited. This article presents an adaptive content-based image retrieval (CBIR) system for improving the performance of image retrieval in mammographic databases. In this work, the authors propose a new relevance feedback approach based on incremental learning with support vector machine (SVM) regression. Also, the authors present a new local perturbation method to further improve the performance of the proposed relevance feedback system. The approaches enable efficient online learning by adapting the current trained model to changes prompted by the user's relevance feedback, avoiding the burden of retraining the CBIR system. To demonstrate the proposed image retrieval system, the authors used two mammogram data sets: A set of 76 mammograms scored based on geometrical similarity and a larger set of 200 mammograms scored by expert radiologists based on pathological findings. The experimental results show that the proposed relevance feedback strategy improves the retrieval precision for both data sets while achieving high efficiency compared to offline SVM. For the data set of 200 mammograms, the authors obtained an average precision of 0.48 and an area under the precision-recall curve of 0.79. In addition, using the same database, the authors achieved a high pathology matching rate greater than 80% between the query and the top retrieved images after relevance feedback. Using mammographic databases, the results demonstrate that the proposed approach is more accurate than the model without using relevance feedback not only in image retrieval but also in pathology matching while maintaining its effectiveness for online relevance feedback applications.

Content-based image retrieval (CBIR) has suffered from the lack of linkage between low-level features and high-level semantics. Although relevance feedback (RF) CBIR provides a promising solution involving human interaction, certain query images poorly represented by low-level features still have unsatisfactory retrieval results. An innovative method has been proposed to increase the percentage of relevance of target image database by using graph cuts theory with the maximum-flow/minimum-cut algorithm and relevance feedback. As a result, the database is reformed by keeping relevant images while discarding irrelevant images. The relevance is increased and thus during following RF-CBIR process, previously poorly represented relevant images have higher probability to appear for selection. Better performance and retrieval results can thus be achieved.

Bag of Features (BoF) has been successfully exploited in content-based image retrieval for several years. Due to its performance and popularity, several extensions have been proposed that involve feature description, dictionary building, feature encoding and post-query process, etc. This paper proposes a dynamically weighting scheme for BoF-based image retrieval based on feedback. It involves two contributions: (i) analyzing the statistical distribution characteristic of similar BoF representations and (ii) computing weights dynamically based on the feedback obtained from different initial query results. We quantitatively evaluate the proposed method on two different databases. Experiments confirm that the proposed weighting scheme has better performance than the baseline of BoF-based image retrieval systems. Meanwhile, the results demonstrate the effectiveness of the weighting scheme in terms of the precision of top-N returned images.

In this paper, we present a novel approach for medical content based image retrieval (searching for images which are pathologically similar to a given example) and demonstrate its performance on a dataset containing maxillofacial lesions. Recently, distributed databases at hospitals and CT scanning centers are related through Picture Archiving and Communication Systems (PACS). Hence, a content based image retrieval system is helpful for radiologists in medical diagnosis. In our proposed framework, a feature vector is extracted for each image and SIFT sparse codes are employed in this step. After feature extraction based on sparse coding and maximum pooling, we have utilized different similarity measures such as Euclidean norm, Manhattan distance and SVM classifier to choose most relevant images to the query image. We have evaluated our proposed framework on a dataset containing 2023 images belonging to 5 different categories: cleft, impaction, fracture, maxillary sinus cyst and healthy. Classification rate of 97.3% and precision versus recall curve indicate the effectiveness of sparse coding in content-based medical image retrieval.

Iterative brain tumor retrieval for MR images based on user’s intention model

Introduction: Content-based image retrieval (CBIR) retrieves the images from the vast image repositories. Research in the CBIR domain gets attention due to the massive number of images generated by the mobiles and various image-capturing machines. Objectives: Any image retrieval system’s primary goal is to reduce the semantic gap between low-level features and high-level perception. Methods: The CBIR techniques are classified into multiple categories based on the feature extraction and retrieval mechanism. These categories are feature-based, machine-learning-based, and deep-learning-based methods. The pioneer techniques for each category are explained in detail in this chapter. Results: The comparative analysis has been done to highlight the advantages of techniques over others. Conclusion: The various application area of the CBIR has been described. The most evolving application is the content-based medical image retrieval (CBMIR) system. The applicability of the CBMIR system in different medical science domains is explained in detail.

This paper presents an image representation and matching framework for image categorization in medical image archives. Categorization enables one to determine automatically, based on the image content, the examined body region and imaging modality. It is a basic step in content-based image retrieval (CBIR) systems, the goal of which is to augment text-based search with visual information analysis. CBIR systems are currently being integrated with picture archiving and communication systems for increasing the overall search capabilities and tools available to radiologists. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (GMM) along with information-theoretic image matching via the Kullback-Leibler (KL) measure. The GMM-KL framework is used for matching and categorizing X-ray images by body regions. A multidimensional feature space is used to represent the image input, including intensity, texture, and spatial information. Unsupervised clustering via the GMM is used to extract coherent regions in feature space that are then used in the matching process. A dominant characteristic of the radiological images is their poor contrast and large intensity variations. This presents a challenge to matching among the images, and is handled via an illumination-invariant representation. The GMM-KL framework is evaluated for image categorization and image retrieval on a dataset of 1500 radiological images. A classification rate of 97.5% was achieved. The classification results compare favorably with reported global and local representation schemes. Precision versus recall curves indicate a strong retrieval result as compared with other state-of-the-art retrieval techniques. Finally, category models are learned and results are presented for comparing images to learned category models.

The traditional privacy-preserving image retrieval schemes not only bring large computational and communication overhead but also cannot well protect the image and query privacy in multi-user scenarios. To solve the above problems, we first propose a basic privacy-preserving content-based image retrieval (CBIR) framework which significantly reduces storage and communication overhead compared with the previous works. Furthermore, we design a new efficient key conversion protocol to support unshared key multi-owner multi-user image retrieval without losing search precision. Moreover, our framework supports unbounded attributes and can trace malicious users according to leaked secret keys, which significantly improve the usability of multi-source data sharing. Strict security analysis shows that the user privacy and outsourced data security can be guaranteed during the image retrieval process, and the performance analysis using real-world dataset shows that the proposed image retrieval framework is efficient and feasible for practical applications.

Relevance feedback (Y. Rui et al., 1998) has been a powerful tool for interactive content based image retrieval (CBIR). During the retrieval process, the user selects the most relevant images and provides a weight of preference for each relevant image. The user's high level query and perception subjectivity can be captured to some extent by dynamically updated low-level feature weights based on the user's feedback. However, in MARS (Y. Rui et al., 1997), only the positive feedbacks, i.e., relevant images are considered. A novel approach is proposed by providing both positive and negative feedbacks for support vector machine (SVM) learning. The SVM learning results are used to update the weights of preference for relevant images. Priorities are given to the positive feedbacks that have larger distances to the hyperplane determined by the support vectors. This approach releases the user from manually providing preference weight for each positive example, i.e., relevant image as before. Experimental results show that the proposed approach has reasonable improvement over relevance feedback with possible examples only.

An accurate diagnosis is significant for the treatment of any disease in its early stage. Content-Based Medical Image Retrieval (CBMIR) is used to find similar medical images in a huge database to help radiologists in diagnosis. The main difficulty in CBMIR is semantic gaps between the lower-level visual details, captured by computer-aided tools and higher-level semantic details captured by humans. Many existing methods such as Manhattan Distance, Triplet Deep Hashing, and Transfer Learning techniques for CBMIR were developed but showed lower efficiency and the computational cost was high. To solve such issues, a new feature extraction approach is proposed using Histogram of Gradient (HoG) with Local Ternary Pattern (LTP) to automatically retrieve medical images from the Contrast-Enhanced Magnetic Resonance Imaging (CE-MRI) database. Adam optimization algorithm is utilized to select features and the Euclidean measure calculates the similarity for query images. From the experimental analysis, it is clearly showing that the proposed HoG-LTP method achieves higher accuracy of 98.8%, a sensitivity of 98.5%, and a specificity of 99.416%, which is better when compared to the existing Random Forest (RF) method which displayed an accuracy, sensitivity, and specificity of 81.1%, 81.7% and 90.5% respectively.

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.

This paper presents a novel feature extraction framework for content-based image retrieval (CBIR). Discrete wavelet transform (DWT) based Local tetra pattern (LTrP) is used to obtain the feature map from an input image. Decomposition of DWT up to single level and the features obtained from it would make the CBIR system very sensitive to noise. Therefore, decomposition up to three scales is used to remove noise. On each of the sub band LTrP is applied and 130 features are extracted. Further, Artificial Neural Network (ANN) is employed for index matching and image retrieval task which gives the classification accuracy of 97.9 % for Corel 1K database. We have compared our proposed feature extraction scheme with LTrP and other existing local descriptor. Results show that combination of DWT and Local Tetra Patterns (DWT + LTrP) extracts more robust features than LTrP alone. Also, the effect of different wavelet filters on the accuracy of the system has been analyzed. Proposed framework has been tested on Corel-1000 and Corel-10000 databases. We have used average retrieval rate, precision, and recall for performance measure of the CBIR system. Proposed method outperforms the retrieval rate from 75.9% using LTrP to 97.9% using proposed method on Corel 1K database. Improvement in performance measures as compared to the other existing methods evidence that, the proposed spatio-frequency local descriptor is more robust for image retrieval task.

In the current decade, we are witnessing a great interest in Content Based Image Retrieval (CBIR) together with a wealth of promising technologies, paved for a large number of new mechanisms and systems. In terms of mechanisms, a strong trend towards the employment of diverse Relevance Feedback (RF) approaches in CBIR systems to capture image(s) of interest has emerged. However, the need to select a particular technique in a given application domain depends on the nature of images in the collection at hand. So our paper mainly reviews and compares different approaches of CBIR using RF. Its ultimate goal is to present information about image database aspects and image features setting so as to support the selection of the appropriate CBIR with RF Techniques.