MF‐Re‐Rank: A modality feature‐based Re‐Ranking model for medical image retrieval

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.

In this paper, we present an empirical study for monolingual medical image retrieval. In particular, we present a series of experiments in ImageCLEFmed 2009 task. There are three main goals. First, we evaluate traditional well-known weighting models in the text retrieval domain, such as BM25, TFIDF and Language Model (LM), for context-based image retrieval. Second, we evaluate statistical-based feedback models and ontology-based feedback models. Third, we investigate how content-based image retrieval can be integrated with these two basic technologies in traditional text retrieval domain. The experimental results have shown that: 1) traditional weighting models work well in context-based medical image retrieval task especially when the parameters are tuned properly; 2) statistical-based feedback models can further improve the retrieval performance when a small number of documents are used for feedback; however, the medical image retrieval can not benefit from ontology-based query expansion method used in this paper; 3) the retrieval performance can be slightly boosted via an integrated retrieval approach.

Different types of search engines are used to search image and text contents. Two types of image search methods are available in the Internet. They are query keyword based model and content based image retrieval models. Text query strings are used in the textual image retrieval model. Content based image retrieval (CBIR) model uses the visual information of the images. Image search methods use the text annotation and image visual features. Google image search and Bing image search engines are used to fetch images from the web. Image query string is used to search image on Internet. One click query image selection method is used to submit user intention for image retrieval. Content based image re-ranking is performed with visual and textual similarity metrics. Adaptive Weight Schema is used for the similarity analysis. Feature weight learning algorithm is applied to estimate feature weights for the images and its category. Query is expanded with keyword and visual information. Rank boost framework algorithm is enhanced to rank images with photographic quality. Content similarity and visual quality factors are used for the re-ranking process. In this paper, we propose an image indexing and retrieval using speech annotations based on a predefined structured syntax. To improve the retrieval effectively, N-best lists for index generation is used .so, a query expansion technique is explored to enhance the query terms. All this process is automatic, without extra effort from the user. This is critically important in web-based image search engine for any commercial, where the user interface has to be extremely simple.

The deep hash method has been proved to be the most efficient and approximate nearest neighbor search technique for large-scale image retrieval. However, the existing deep hashing methods have few cases in medical image retrieval, so we propose a deep hash network model for medical image retrieval. Firstly, in order to enhance the feature extraction and learning ability in the network model, we add SENet visual attention mechanism into each residual module of the backbone network ResNet50 for feature extraction. Then the extracted high-dimensional semantic features are added to the Cauchy hash module to generate a compact, centralized hash code, and then complete medical image retrieval. So as to enhance experimental comparison, the proposed method was validated experimentally on NUS-WIDE and MURA data sets. A large number of experiments verify that the proposed method is reliable and can be used for medical image retrieval. Compared with other existing hash algorithms, the retrieval performance of the proposed method has certain improved. The framework of the proposed method is shown in Figure 1.

Medical image indexing and storage is gaining increased importance. Indexing and retrieval of these images efficiently is becomes an essential task. Indexing of medical images using text or numbers is a cumbersome task, difficult to memorize and time consuming. Indexing and Retrieval of images can be done through query by text and query by image which is also known as content based image retrieval (CBIR). Different medical images possess different texture and shape features. This paper presents a novel and hybrid approach of managing a huge medical image database and retrieving a medical image from the database by fusion of shape and texture features. Image is retrieved by comparing the features of query image and images of database. We attach some document information with each medical image such as patient identity, diseases, age, and case history. It can be used for further diagnostic and analysis purpose. Further the proposed algorithm also updates the database automatically if new query is found. MATLAB ® 7.01 and its image processing toolbox have been used to implement the algorithm.

Effective World Wide Web image retrieval systems are required to locate relevant images as more and more images are used in HTML documents. There are two general approaches to image retrieval: one is based on text descriptions and the other on image contents (low-level features, such as colour and texture). They have their respective strengths and weaknesses. In this paper, we describe how to improve text-based and content-based image retrieval techniques and to integrate these two types of techniques in one system to take advantage of their complementing strengths. Our experimental results show that the integrated approach has higher retrieval performance than either the text-based or the content-based techniques used by themselves.

In this paper, we propose a new color edge model and color edge histogram descriptor for contents-based image retrieval. The edge descriptor proposed by MPEG-7 standard is a representative approach for the contents-based image retrieval using the edge histogram that is derived from a gray-level image. This paper introduces a novel method that extracts color edge information from spectral color images rather than monochrome images and a new color edge histogram descriptor for the contents-based image retrieval. The proposed color edge model is obtained in two phases. In the first phase, we characterize the R,G,B channel components as a linear map (the differential) and impose a statistical interpretation on this map by considering the behavior of the map as it applied to unit normed vectors in the second phase. As a result, the constructed edge model will be expressed in a statistical fashion and will provide a mechanism to determine the possibility of the edge existence. The color edge histogram based on the direction of the color edge model is subsequently applied to the contents-based e-catalogue image retrieval. For the evaluation, the results of image retrieval using the proposed method are compared with those of image retrieval using the edge descriptor by MPEG-7 and other approaches. The experimental result supports the efficiency of the proposed method.KeywordsColor ImageImage RetrievalQuery ImageColor HistogramColor EdgeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper presents an image representation and matching framework for image categorization in medical image archives. Categorization enables one to determine automatically, based on the image content, the examined body region and imaging modality. It is a basic step in content-based image retrieval (CBIR) systems, the goal of which is to augment text-based search with visual information analysis. CBIR systems are currently being integrated with picture archiving and communication systems for increasing the overall search capabilities and tools available to radiologists. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (GMM) along with information-theoretic image matching via the Kullback-Leibler (KL) measure. The GMM-KL framework is used for matching and categorizing X-ray images by body regions. A multidimensional feature space is used to represent the image input, including intensity, texture, and spatial information. Unsupervised clustering via the GMM is used to extract coherent regions in feature space that are then used in the matching process. A dominant characteristic of the radiological images is their poor contrast and large intensity variations. This presents a challenge to matching among the images, and is handled via an illumination-invariant representation. The GMM-KL framework is evaluated for image categorization and image retrieval on a dataset of 1500 radiological images. A classification rate of 97.5% was achieved. The classification results compare favorably with reported global and local representation schemes. Precision versus recall curves indicate a strong retrieval result as compared with other state-of-the-art retrieval techniques. Finally, category models are learned and results are presented for comparing images to learned category models.

Automatic medical image annotation and retrieval

In the evolving field of healthcare, centralized cloud-based medical image retrieval faces challenges related to security, availability, and adversarial threats. Existing deep learning-based solutions improve retrieval but remain vulnerable to adversarial attacks and quantum threats, necessitating a shift to more secure distributed cloud solutions. This article proposes SFMedIR, a secure and fault tolerant medical image retrieval framework that contains an adversarial attack-resistant federated learning for hashcode generation, utilizing a ConvNeXt-based model to improve accuracy and generalizability. The framework integrates quantum-chaos-based encryption for security, dynamic threshold-based shadow storage for fault tolerance, and a distributed cloud architecture to mitigate single points of failure. Unlike conventional methods, this approach significantly improves security and availability in cloud-based medical image retrieval systems, providing a resilient and efficient solution for healthcare applications. The framework is validated on Brain MRI and Kidney CT datasets, achieving a 60-70% improvement in retrieval accuracy for adversarial queries and an overall 90% retrieval accuracy, outperforming existing models by 5-10%. The results demonstrate superior performance in terms of both security and retrieval efficiency, making this framework a valuable contribution to the future of secure medical image management.

Cluster sampling algorithm is a scheme for sequential data assimilation developed to handle general non-Gaussian and nonlinear settings. The cluster sampling algorithm can be used to solve a wide spectrum of problems that requires data inversion such as image retrieval, tomography, weather prediction amongst others. This paper develops parallel cluster sampling algorithms, and show that a multi-chain version is embarrassingly parallel, and can be used efficiently for medical image retrieval amongst other applications. Moreover, it presents a detailed complexity analysis of the proposed parallel cluster samplings scheme and discuss their limitations. Numerical experiments are carried out using a synthetic one dimensional example, and a medical image retrieval problem. The experimental results show the accuracy of the cluster sampling algorithm to retrieve the original image from noisy measurements, and uncertain priors. Specifically, the proposed parallel algorithm increases the acceptance rate of the sampler from 45% to 81% with Gaussian proposal kernel, and achieves an improvement of 29% over the optimally-tuned Tikhonov-based solution for image retrieval. The parallel nature of the proposed algorithm makes the it a strong candidate for practical and large scale applications.

The purpose of dissertation is to utilize connotative messages for enhancing image retrieval and browsing. By adopting semiotics as a theoretical tool, this study explores problems of image retrieval and proposes an image retrieval model. The semiotics approach conceptually demonstrates that: 1) a fundamental reason for the dissonance between retrieved images and user needs is representation of connotative messages, and 2) the image retrieval model which makes use of denotative index terms is able to facilitate users to browse connotatively related images effectively even when the users' needs are potentially expressed in the form of denotative query. Two experiments are performed for verifying the semiotic-based image retrieval model and evaluating the effectiveness of the model. As data sources, 5,199 records are collected from Artefacts Canada: Humanities by Canadian Heritage Information Network, and the candidate terms of connotation and denotation are extracted from Art & Architecture Thesaurus. The first experiment, by applying term association measures, verifies that the connotative messages of an image can be derived from denotative messages of the image. The second experiment reveals that the association thesaurus which is constructed based on the associations between connotation and denotation facilitates assigning connotative terms to image documents. In addition, the result of relevant judgments presents that the association thesaurus improves the relative recall of retrieved image documents as well as the relative recall of browsing sets. This study concludes that the association thesaurus indicating associations between connotation and denotation is able to improve the accessibility of the connotative messages. The results of the study are hoped to contribute to the conceptual knowledge of image retrieval by providing understandings of connotative messages within an image and to the practical design of image retrieval system by proposing an association thesaurus which can supplement the limitations of the current content-based image retrieval systems (CBIR).

In recent years, secure image archival and retrieval model in cloud computing (CC) received significant attention to ensure data confidentiality and secured data transmission among cloud server and clients cloud storage and users. Traditional secure image retrieval (IR) techniques are not applicable to adapt with a large-scale IR in cloud environment. In order to overcome these issues, this article introduces an effective secure IR scheme using Inception with ResNet v2 (SIRS-IR) and Multiple Share Creation (MSC) scheme. The proposed method uses Inception with ResNet v2 model based feature extraction process. At the same time, the MSC process is utilized for multiple share generation and then encryption of shares takes place using double chaotic logistic map (DLCM) technique. Besides, the encrypted shares and the respective feature vectors are saved in the cloud server with the corresponding image identification number. During the IR process for the applied QI, the SIRS-IR model extracts the feature vectors and performs similarity measurement to retrieve the related images from the database interms of encrypted shares. Finally, the share decryption process is carried out for the reconstruction of original images with no loss of quality. Extensive experimentations were performed to verify the retrieval performance and image quality of the reconstructed images using Corel10K dataset. The obtained results stated that the presented SIRS-IR model is found to be superior to other methods in a considerable way.

Due to the huge growth of the World Wide Web, medical images are now available in large numbers in online repositories, and there exists the need to retrieval the images through automatically extracting visual information of the medical images, which is commonly known as content-based image retrieval (CBIR). Since each feature extracted from images just characterizes certain aspect of image content, multiple features are necessarily employed to improve the retrieval performance. Meanwhile, experiments demonstrate that a special feature is not equally important for different image queries. Most of existed feature fusion methods for image retrieval only utilize query independent feature fusion or rely on explicit user weighting. In this paper, we present a novel query dependent feature fusion method for medical image retrieval based on one class support vector machine. Having considered that a special feature is not equally important for different image queries, the proposed query dependent feature fusion method can learn different feature fusion models for different image queries only based on multiply image samples provided by the user, and the learned feature fusion models can reflect the different importances of a special feature for different image queries. The experimental results on the IRMA medical image collection demonstrate that the proposed method can improve the retrieval performance effectively and can outperform existed feature fusion methods for image retrieval.

This chapter provides an introduction to information retrieval and image retrieval. Types of image retrieval techniques, i.e., text-based image retrieval and content-based image retrieval techniques are introduced. A brief introduction to visual features like color, texture, and shape is provided. Similarity measures used in content-based image retrieval and performance evaluation of content-based image retrieval techniques are also given. Importance of user interaction in retrieval systems is also discussed.