- Book Chapter
54
- 10.1016/b978-012373583-6.50008-6
4 - Content-Based Medical Image Retrieval
- Jan 01, 2008
- Biomedical Information Technology
- Tom Weidong Cai + 2 more +2
4 - Content-Based Medical Image Retrieval
Content-based medical image retrieval with opponent class adaptive margin loss
4 - Content-Based Medical Image Retrieval
4 - Content-Based Medical Image Retrieval
Improving relevance feedback for content based medical image retrieval
Content-Based Image Retrieval (CBIR) refers to techniques that retrieve images based on their content, as opposed to based on metadata. A CBIR system performs indexing and retrieval tasks using features like color, texture and shape computed from images as opposed to using the whole images. In the medical field, content based image retrieval is used to aid radiologist to retrieve of images with similar contents. CBIR methods are usually developed for specific features of images, so that those methods are not readily applicable across different kinds of medical images. Content-Based Medical Image Retrieval (CBMIR) refers to techniques that retrieve images from medical image databases. A CBMIR system using the medical image features like Haralick features, Zernike moments, histogram intensity features and run-length features. In this study, CBMIR system With improved feature selection method is developed using a hybrid approach of branch and bound and artificial bee colony using the breast cancer, Brain tumor and thyroid images and classification is performed using Fuzzy based Relevance Vector Machine (FRVM) to form groups of relevant image features The Euclidean distance measurement is used to assess the similarity between query images and database images. A Relevance feedback method using diverse density algorithm is used to improve the performance of content-based medical image Retrieval. An improved feature selection method is used to reduces the existing system dimensionality curse problem and improve the performance of the system.
Read moreContent-based Retrieval of Medical Images: Landmarking, Indexing, and Relevance Feedback
Content-based image retrieval (CBIR) is the process of retrieval of images from a database that are similar to a query image, using measures derived from the images themselves, rather than relying on accompanying text or annotation. To achieve CBIR, the contents of the images need to be characterized by quantitative features; the features of the query image are compared with the features of each image in the database and images having high similarity with respect to the query image are retrieved and displayed. CBIR of medical images is a useful tool and could provide radiologists with assistance in the form of a display of relevant past cases. One of the challenging aspects of CBIR is to extract features from the images to represent their visual, diagnostic, or application-specific information content. In this book, methods are presented for preprocessing, segmentation, landmarking, feature extraction, and indexing of mammograms for CBIR. The preprocessing steps include anisotropic di fusion and the Wiener filter to remove noise and perform image enhancement. Techniques are described for segmentation of the breast and fibroglandular disk, including maximum entropy, a moment-preserving method, and Otsu's method. Image processing techniques are described for automatic detection of the nipple and the edge of the pectoral muscle via analysis in the Radon domain. By using the nipple and the pectoral muscle as landmarks, mammograms are divided into their internal, external, upper, and lower parts for further analysis. Methods are presented for feature extraction using texture analysis, shape analysis, granulometric analysis, moments, and statistical measures. The CBIR system presented provides options for retrieval using the Kohonen self-organizing map and the k-nearest-neighbor method. Methods are described for inclusion of expert knowledge to reduce the semantic gap in CBIR, including the query point movement method for relevance feedback (RFb). Analysis of performanc is described in terms of precision, recall, and relevance-weighted precision of retrieval. Results of application to a clinical database of mammograms are presented, including the input of expert radiologists into the CBIR and RFb processes. Models are presented for integration of CBIR and computer-aided diagnosis (CAD) with a picture archival and communication system (PACS) for efficient workflow in a hospital. Table of Contents: Introduction to Content-based Image Retrieval / Mammography and CAD of Breast Cancer / Segmentation and Landmarking of Mammograms / Feature Extraction and Indexing of Mammograms / Content-based Retrieval of Mammograms / Integration of CBIR and CAD into Radiological Workflow
Read moreImage retrieval: Benchmarking visual information indexing and retrieval systems
Image retrieval: Benchmarking visual information indexing and retrieval systems
Content Based Medical Image Retrieval for Accurate Disease Diagnosis
Introduction: Content Based Image Retrieval (CBIR) system is an innovative technology to retrieve images from various media types. One of the CBIR applications is Content Based Medical Image Retrieval (CBMIR). The image retrieval system retrieves the most similar images from the historical cases, and such systems can only support the physician's decision to diagnose a disease. To extract the useful features from the query image for linking similar types of images is the major challenge in the CBIR domain. The Convolution Neural Network (CNN) can overcome the drawbacks of traditional algorithms, dependent on the low-level feature extraction technique. Objective: The objective of the study is to develop a CNN model with a minimum number of convolution layers and to get the maximum possible accuracy for the CBMIR system. The minimum number of convolution layers reduces the number of mathematical operations and the time for the model's training. It also reduces the number of training parameters, like weights and bias. Thus, it reduces the memory requirement for the model storage. This work mainly focused on developing an optimized CNN model for the CBMIR system. Such systems can only support the physicians' decision to diagnose a disease from the images and retrieve the relevant cases to help the doctor decide the precise treatment. Methods: The deep learning-based model is proposed in this paper. The experiment is done with several convolution layers and various optimizers to get the maximum accuracy with a minimum number of convolution layers. Thus, the ten-layer CNN model is developed from scratch and used to derive the training and testing images' features and classify the test image. Once the image class is identified, the most relevant images are determined based on the Euclidean distance between the query features and database features of the identified class. Based on this distance, the most relevant images are displayed from the respective class of images. The general dataset CIFAR10, which has 60,000 images of 10 different classes, and the medical dataset IRMA, which has 2508 images of 9 various classes, have been used to analyze the proposed method. The proposed model is also applied for the medical x-ray image dataset of chest disease and compared with the other pre-trained models. Results: The accuracy and the average precision rate are the measurement parameters utilized to compare the proposed model with different machine learning techniques. The accuracy of the proposed model for the CIFAR10 dataset is 93.9%, which is better than the state-of-the-art methods. After the success for the general dataset, the model is also tested for the medical dataset. For the x-ray images of the IRMA dataset, it is 86.53%, which is better than the different pre-trained model results. The model is also tested for the other x-ray dataset, which is utilized to identify chest-related disease. The average precision rate for such a dataset is 97.25%. Also, the proposed model fulfills the major challenge of the semantic gap. The semantic gap of the proposed model for the chest disease dataset is 2.75%, and for the IRMA dataset, it is 13.47%. Also, only ten convolution layers are utilized in the proposed model, which is very small in number compared to the other pre-trained models. Conclusion: The proposed technique shows remarkable improvement in performance metrics over CNN-based state-of-the-art methods. It also offers a significant improvement in performance metrics over different pre-trained models for the two different medical x-ray image datasets.
Read moreContent Based Medical Image Retrieval for Accurate Disease Diagnosis
Introduction: Content Based Image Retrieval (CBIR) system is an innovative technology to retrieve images from various media types. One of the CBIR applications is Content Based Medical Image Retrieval (CBMIR). The image retrieval system retrieves the most similar images from the historical cases, and such systems can only support the physician's decision to diagnose a disease. To extract the useful features from the query image for linking similar types of images is the major challenge in the CBIR domain. The Convolution Neural Network (CNN) can overcome the drawbacks of traditional algorithms, dependent on the low-level feature extraction technique. Objective: The objective of the study is to develop a CNN model with a minimum number of convolution layers and to get the maximum possible accuracy for the CBMIR system. The minimum number of convolution layers reduces the number of mathematical operations and the time for the model's training. It also reduces the number of training parameters, like weights and bias. Thus, it reduces the memory requirement for the model storage. This work mainly focused on developing an optimized CNN model for the CBMIR system. Such systems can only support the physicians' decision to diagnose a disease from the images and retrieve the relevant cases to help the doctor decide the precise treatment. Methods: The deep learning-based model is proposed in this paper. The experiment is done with several convolution layers and various optimizers to get the maximum accuracy with a minimum number of convolution layers. Thus, the ten-layer CNN model is developed from scratch and used to derive the training and testing images' features and classify the test image. Once the image class is identified, the most relevant images are determined based on the Euclidean distance between the query features and database features of the identified class. Based on this distance, the most relevant images are displayed from the respective class of images. The general dataset CIFAR10, which has 60,000 images of 10 different classes, and the medical dataset IRMA, which has 2508 images of 9 various classes, have been used to analyze the proposed method. The proposed model is also applied for the medical x-ray image dataset of chest disease and compared with the other pre-trained models. Results: The accuracy and the average precision rate are the measurement parameters utilized to compare the proposed model with different machine learning techniques. The accuracy of the proposed model for the CIFAR10 dataset is 93.9%, which is better than the state-of-the-art methods. After the success for the general dataset, the model is also tested for the medical dataset. For the x-ray images of the IRMA dataset, it is 86.53%, which is better than the different pre-trained model results. The model is also tested for the other x-ray dataset, which is utilized to identify chest-related disease. The average precision rate for such a dataset is 97.25%. Also, the proposed model fulfills the major challenge of the semantic gap. The semantic gap of the proposed model for the chest disease dataset is 2.75%, and for the IRMA dataset, it is 13.47%. Also, only ten convolution layers are utilized in the proposed model, which is very small in number compared to the other pre-trained models. Conclusion: The proposed technique shows remarkable improvement in performance metrics over CNN-based state-of-the-art methods. It also offers a significant improvement in performance metrics over different pre-trained models for the two different medical x-ray image datasets.
Read moreContent-based image retrieval in medical domains: An overview of methods, issues and systems
Content-based image retrieval (CBIR) is one of the most interesting research areas of computer vision in recent years. CBIR uses feature comparison to search, browse, and retrieve images from large databases. It is considered as the main research topic in medical fields because of the daily increase in the number of recorded medical images. CBIR can help physicians diagnose various types of diseases as well as medical references, teaching purposes , training and research. This paper introduces the main concepts, stages, and methods of CBIR systems; explores the main types of features that can be extracted from images; and identifies the techniques to evaluate the systems. It also reviews and evaluates the advantages and disadvantages of several existing commercial and academic content-based medical image retrieval systems. Finally, the paper introduces suggestions for the future of medical domains to solve the discussed disadvantages. It proposes the technique of automatic medical image annotation/classification to enhance the accuracy of CBIR systems.
Read moreContent Based Medical Image Retrieval Based on Salient Regions Combined with Deep Learning
In traditional text based medical image retrieval system, it is hard to find visually similar images in large medical image database. Content-based image retrieval is developed to retrieve similar images and it is based on visual attributes describing the content of an image. Developing a method for content based medical image retrieval is a challenging task. This paper proposed a new method for content-based medical image retrieval based on salient regions and deep learning. The proposed method includes two stages: an offline task to extract local object features and an online task for content-based image retrieval in database. In first stage, we extract local object features of medical image depending on shape, texture and intensity, and features extracted by deep learning applied in saliency of decomposition. Secondly, we make online task for content-based image retrieval in database. The user gives query image as an input and the system will retrieve n top most similar images by similarity comparison with bag of code words feature values obtained in the first stage. Evaluation of the proposed method is based on Precision and Recall values. Our dataset includes 5 groups of medical images with their quality varying from low to high. With the best medical image quality group, the accuray can be 91.61% for Precision and 89.61% for Recall. Comparing the average values with others methods, the results of the proposed method are more than 2 to 5% better.
Read moreContent-based medical image retrieval method using multiple pre-trained convolutional neural networks feature extraction models
Content-based medical image retrieval (CBMIR), a specialized area within content-based image retrieval (CBIR), involves two main stages: feature extraction and retrieval ranking. The feature extraction stage is particularly crucial for developing an effective retrieval system with high performance. Lately, pre-trained deep convolutional neural networks (CNNs) have become the preferred tools for feature extraction due to their excellent performance and versatility, which includes the ability to be re-trained and adapt through transfer learning. Various pre-trained deep CNN models are employed as feature extraction tools in content-based medical image retrieval systems. Researchers have effectively used many such models either individually or in combined forms by merging feature vectors from several models. In this study, a method using multiple pre-trained deep CNNs for CBMIR is introduced, utilizing two popular models, ResNet-18 and GoogleNet, for extracting features. This method combines the feature vectors from both models in a way that selects the best model for each image based on the highest classification probability during training. The method's effectiveness is assessed using two well-known medical image datasets, Kvasir and PH2. The evaluation results show that the proposed method achieved average precision scores of 94.13% for Kvasir and 55.67% for PH2 at the top 10 cut-offs, surpassing some leading methods in this research area.
Read moreA Panoramic View of Content-based Medical Image Retrieval system
Content-Based Image Retrieval (CBIR) system for medical applications is one of the trending research fields in computer vision and Digital image processing over the last 20 years. Content-Based Image Retrieval is a technique to perform retrieval of the images from a large database. The CBIR technique is used in different domains such as medical images, weather forecasting and remote sensing. But, the application of CBIR system in the medical field is of utmost importance as it can help in the diagnosis and treatment of the diseases by studying the medical images. It allows indexing of images, retrieval of images and understanding of large collections of image data. This paper presents a review of the already existing Content-based Medical Image Retrieval (CBMIR) systems and techniques used mainly for the feature selection and classification used in these systems.
Read moreQuery Expansion Based on Top-Ranked Images for Content-Based Medical Image Retrieval
Recently the collection of varied digital image databases has increased. Many users have found that searching and retrieving required images from large collections are very difficult tasks, and successful and effective retrieval methods were developed to provide an effective and efficient search and retrieval process. In most content-based image retrieval (CBIR) methods, various visual features have been considered indirect to retrieve the images from databases. Content-based medical image retrieval (CBMIR), like any CBIR method, is a technique for retrieving medical images on the basis of automatically derived image features, such as colour and texture. Although a number of methods and approaches have been suggested, retrieval performance remains one of the most challenging problems in current CBMIR studies, this due to the well-known `semantic gap' issue that exists between machine-captured low-level image features and human-perceived high-level semantic concepts. To bridge this gap, much research has been proposed. This study proposes two expansion methods to increase and enhance precision of the retrieval model; both of our proposed methods depend on top-ranked images. Our first expansion method is to reformulate the new expand query image based on mean values for features of top-ranked images, while the second method is based on selecting the most important features only. The expansion process was performed on eighteen colour features and twelve texture features extracted from two common medical images datasets, Kvasir and PH2. Experimental results show that our proposed methods performed better and have a precision of 95.8% and 80.7% for Kvasir and PH2 data set, respectively.
Read moreThe MedGIFT Project: Global Perspectives of a Medical Doctor
Medical image retrieval covers an important and wide application area in medical imaging, with one major goal being in providing assistance in medical diagnosis. With the availability of image data in the electronic patient records and thus to all medical doctors, the need to search and retrieve images efficiently and accurately from large image databases such as picture archiving and communication systems (PACS) has drastically increased. To address this demand, content-based image retrieval (CBIR) has been proposed for a long time with many systems being built to meet several application demands. However, in most systems there is still a gap between the end user’s expectations and the system’s retrieval capabilities. Relatively little attention has so far been paid to the process of evaluating the effectiveness of CBIR methods and its real impact among the health care professionals through optimized user interaction. This paper addresses the actual problem of content-based medical image retrieval, which aims more particularly at highlighting the perspectives of a medical doctor on content-based image retrieval, its pos
Read moreDual-Force ISOMAP: A New Relevance Feedback Method for Medical Image Retrieval
With great potential for assisting radiological image interpretation and decision making, content-based image retrieval in the medical domain has become a hot topic in recent years. Many methods to enhance the performance of content-based medical image retrieval have been proposed, among which the relevance feedback (RF) scheme is one of the most promising. Given user feedback information, RF algorithms interactively learn a user’s preferences to bridge the “semantic gap” between low-level computerized visual features and high-level human semantic perception and thus improve retrieval performance. However, most existing RF algorithms perform in the original high-dimensional feature space and ignore the manifold structure of the low-level visual features of images. In this paper, we propose a new method, termed dual-force ISOMAP (DFISOMAP), for content-based medical image retrieval. Under the assumption that medical images lie on a low-dimensional manifold embedded in a high-dimensional ambient space, DFISOMAP operates in the following three stages. First, the geometric structure of positive examples in the learned low-dimensional embedding is preserved according to the isometric feature mapping (ISOMAP) criterion. To precisely model the geometric structure, a reconstruction error constraint is also added. Second, the average distance between positive and negative examples is maximized to separate them; this margin maximization acts as a force that pushes negative examples far away from positive examples. Finally, the similarity propagation technique is utilized to provide negative examples with another force that will pull them back into the negative sample set. We evaluate the proposed method on a subset of the IRMA medical image dataset with a RF-based medical image retrieval framework. Experimental results show that DFISOMAP outperforms popular approaches for content-based medical image retrieval in terms of accuracy and stability.
Read moreContent based image retrieval for maxillofacial lesions
In this paper, we present a novel approach for medical content based image retrieval (searching for images which are pathologically similar to a given example) and demonstrate its performance on a dataset containing maxillofacial lesions. Recently, distributed databases at hospitals and CT scanning centers are related through Picture Archiving and Communication Systems (PACS). Hence, a content based image retrieval system is helpful for radiologists in medical diagnosis. In our proposed framework, a feature vector is extracted for each image and SIFT sparse codes are employed in this step. After feature extraction based on sparse coding and maximum pooling, we have utilized different similarity measures such as Euclidean norm, Manhattan distance and SVM classifier to choose most relevant images to the query image. We have evaluated our proposed framework on a dataset containing 2023 images belonging to 5 different categories: cleft, impaction, fracture, maxillary sinus cyst and healthy. Classification rate of 97.3% and precision versus recall curve indicate the effectiveness of sparse coding in content-based medical image retrieval.
Read moreA content based image retrieval using K-means algorithm
The study of Medical image retrieval, which is concerned with efficiently and effectively accessing desired medical images from varied and large image collections, has become more important, challenging and interesting. Until now extracting the lesions by automatic segmentation is considered the bottleneck in content-based medical image retrieval. Above that many approaches which are based on one-to-one blocks of an image are still sensitive to rotation, shifting and scaling. To address the last problems, we propose a new approach based on Hungarian algorithm which compares one block from the query image to all blocks from each image in the dataset and returns the closet matching. This comparison is based on features vector of gray-level intensity for each block. We used K-mean clustering algorithm to separate each window into K clusters. We will enforce the histogram of each cluster into Gaussian distribution, and then based on this histogram, the mean, variance and skewness are computed. The proposed content-based medical image retrieval approach gives acceptable results.
Read more