Secure and fault tolerant cloud based framework for medical image storage and retrieval in a distributed environment

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.

One of the main challenges in medical image retrieval is the increasing volume of image data, which render it difficult for domain experts to find relevant information from large data sets. Effective and efficient medical image retrieval systems are required to better manage medical image information. Text‐based image retrieval (TBIR) was very successful in retrieving images with textual descriptions. Several TBIR approaches rely on models based on bag‐of‐words approaches, in which the image retrieval problem turns into one of standard text‐based information retrieval; where the meanings and values of specific medical entities in the text and metadata are ignored in the image representation and retrieval process. However, we believe that TBIR should extract specific medical entities and terms and then exploit these elements to achieve better image retrieval results. Therefore, we propose a novel reranking method based on medical‐image‐dependent features. These features are manually selected by a medical expert from imaging modalities and medical terminology. First, we represent queries and images using only medical‐image‐dependent features such as image modality and image scale. Second, we exploit the defined features in a new reranking method for medical image retrieval. Our motivation is the large influence of image modality in medical image retrieval and its impact on image‐relevance scores. To evaluate our approach, we performed a series of experiments on the medical ImageCLEF data sets from 2009 to 2013. The BM25 model, a language model, and an image‐relevance feedback model are used as baselines to evaluate our approach. The experimental results show that compared to the BM25 model, the proposed model significantly enhances image retrieval performance. We also compared our approach with other state‐of‐the‐art approaches and show that our approach performs comparably to those of the top three runs in the official ImageCLEF competition.

Automatic medical image annotation and retrieval

This paper presents an effective and efficient framework for Crowd-assisted Mobile Similarity Retrieval of the large-scale medical images in the resource-constraint mobile telemedicine systems (MTS), called the CMSR. The CMSR processing works as follows: when a user submits a retrieval medical image IR, a buffer checking processing is first invoked to check if the full (or partial) retrieval results have been cached in the buffer previously. After that, a parallel image data filtering and refinement processing is conducted at a master node level. Finally, the candidate images are concurrently validated by a mCrowd system to derive an answer set that is transmitted to the retrieval node. To better facilitate the effective and efficient CMSR processing, three enabling techniques, i.e., category-based image data interleaving placement scheme, hindex-support image filtering algorithm and a kNN-based buffering scheme are developed. To improve the retrieval throughput, finally, we propose an extension of the CMSR method called mCMSR to optimize the multiple CMSRs. The experimental results show that the performances of the CMSR and the mCMSR methods are: 1) effective in improving the retrieval accuracy; 2) efficient in minimizing the response time by decreasing the network transmission cost while increasing the parallelism of I/O and CPU.

Medical image indexing and storage is gaining increased importance. Indexing and retrieval of these images efficiently is becomes an essential task. Indexing of medical images using text or numbers is a cumbersome task, difficult to memorize and time consuming. Indexing and Retrieval of images can be done through query by text and query by image which is also known as content based image retrieval (CBIR). Different medical images possess different texture and shape features. This paper presents a novel and hybrid approach of managing a huge medical image database and retrieving a medical image from the database by fusion of shape and texture features. Image is retrieved by comparing the features of query image and images of database. We attach some document information with each medical image such as patient identity, diseases, age, and case history. It can be used for further diagnostic and analysis purpose. Further the proposed algorithm also updates the database automatically if new query is found. MATLAB ® 7.01 and its image processing toolbox have been used to implement the algorithm.

The security and privacy of medical images are crucial due to their sensitive nature and the potential for severe consequences from unauthorized modifications, including data breaches and inaccurate diagnoses. This paper introduces a method for lossless medical image retrieval from encrypted images stored on third-party clouds. The proposed approach employs a symmetric integrity-centric image encryption scheme, leveraging multiple chaotic maps and cryptographic hash techniques, to ensure lossless image reconstruction. Medical images are first encrypted by the image owners and converted into hashcodes encapsulating essential features using a deep hashing technique with the ConvNeXt network as the backbone in parallel. To ensure index privacy, these hashcodes are encrypted in a searchable manner. The encrypted medical images, along with a secure index, are subsequently uploaded to cloud storage. Authorized medical image users can request similar medical images for diagnostic purposes by submitting a query image, from which a search trapdoor is generated and sent to the cloud. The retrieval process involves a secure similar image search over the encrypted indexes, followed by decryption along with integrity verification of the retrieved images. The proposed method has been rigorously tested on three standard medical datasets, demonstrating an improvement of 5-20% in retrieval accuracy compared to standard baselines. Formal security analysis and experimental results indicate that the proposed scheme offers enhanced security and retrieval accuracy, making it an effective solution for the encrypted storage and secure retrieval of medical image data.

Editorial Board

Due to the huge growth of the World Wide Web, medical images are now available in large numbers in online repositories, and there exists the need to retrieval the images through automatically extracting visual information of the medical images, which is commonly known as content-based image retrieval (CBIR). Since each feature extracted from images just characterizes certain aspect of image content, multiple features are necessarily employed to improve the retrieval performance. Meanwhile, experiments demonstrate that a special feature is not equally important for different image queries. Most of existed feature fusion methods for image retrieval only utilize query independent feature fusion or rely on explicit user weighting. In this paper, we present a novel query dependent feature fusion method for medical image retrieval based on one class support vector machine. Having considered that a special feature is not equally important for different image queries, the proposed query dependent feature fusion method can learn different feature fusion models for different image queries only based on multiply image samples provided by the user, and the learned feature fusion models can reflect the different importances of a special feature for different image queries. The experimental results on the IRMA medical image collection demonstrate that the proposed method can improve the retrieval performance effectively and can outperform existed feature fusion methods for image retrieval.

BackgroundPrevious research studies have demonstrated that medical content image retrieval can play an important role by assisting dermatologists in skin lesion diagnosis. However, current state-of-the-art approaches have not been adopted in routine consultation, partly due to the lack of interpretability limiting trust by clinical users.ObjectiveThis study developed a new image retrieval architecture for polarized or dermoscopic imaging guided by interpretable saliency maps. This approach provides better feature extraction, leading to better quantitative retrieval performance as well as providing interpretability for an eventual real-world implementation.MethodsContent-based image retrieval (CBIR) algorithms rely on the comparison of image features embedded by convolutional neural network (CNN) against a labeled data set. Saliency maps are computer vision–interpretable methods that highlight the most relevant regions for the prediction made by a neural network. By introducing a fine-tuning stage that includes saliency maps to guide feature extraction, the accuracy of image retrieval is optimized. We refer to this approach as saliency-enhanced CBIR (SE-CBIR). A reader study was designed at the University Hospital Zurich Dermatology Clinic to evaluate SE-CBIR’s retrieval accuracy as well as the impact of the participant’s confidence on the diagnosis.ResultsSE-CBIR improved the retrieval accuracy by 7% (77% vs 84%) when doing single-lesion retrieval against traditional CBIR. The reader study showed an overall increase in classification accuracy of 22% (62% vs 84%) when the participant is provided with SE-CBIR retrieved images. In addition, the overall confidence in the lesion’s diagnosis increased by 24%. Finally, the use of SE-CBIR as a support tool helped the participants reduce the number of nonmelanoma lesions previously diagnosed as melanoma (overdiagnosis) by 53%.ConclusionsSE-CBIR presents better retrieval accuracy compared to traditional CBIR CNN-based approaches. Furthermore, we have shown how these support tools can help dermatologists and residents improve diagnosis accuracy and confidence. Additionally, by introducing interpretable methods, we should expect increased acceptance and use of these tools in routine consultation.

In this paper, we present an empirical study for monolingual medical image retrieval. In particular, we present a series of experiments in ImageCLEFmed 2009 task. There are three main goals. First, we evaluate traditional well-known weighting models in the text retrieval domain, such as BM25, TFIDF and Language Model (LM), for context-based image retrieval. Second, we evaluate statistical-based feedback models and ontology-based feedback models. Third, we investigate how content-based image retrieval can be integrated with these two basic technologies in traditional text retrieval domain. The experimental results have shown that: 1) traditional weighting models work well in context-based medical image retrieval task especially when the parameters are tuned properly; 2) statistical-based feedback models can further improve the retrieval performance when a small number of documents are used for feedback; however, the medical image retrieval can not benefit from ontology-based query expansion method used in this paper; 3) the retrieval performance can be slightly boosted via an integrated retrieval approach.

The explosive growth of medical images brings about massive amounts of high-dimensional images. The medical image retrieval technique is effective in selecting some similar medical images for medical analysis. A vast majority of existing medical image retrieval methods aim to extract a single feature to represent a medical image. Although these methods have improved the performance, they ignore that different characteristics express different information and they are an indispensable part of content information. Moreover, conventional features consume much storage, which is not conducive to data storage. Therefore, to reduce the loss of information and save storage space, we propose a novel jointly subspace hashing (JSSH) method for medical image retrieval. We solve the high-dimensionality of medical by the simplest image segmentation technology, and then study different subspace projection matrices. Finally, we integrate these matrices into the hash learning model, and build an objective function and learn a series of discriminative binary codes. By conducting comprehensive experiments on three medical image benchmark datasets, we demonstrate the effectiveness of our proposed JSSH.

Medical records generated in hospitals are treasures for academic research and future references. Medical Image Retrieval (MIR) Systems contribute significantly to locating the relevant records required for a particular diagnosis, analysis, and treatment. An efficient classifier and effective indexing technique are required for the storage and retrieval of medical images. In this paper, a retrieval framework is formulated by adopting a modified Local Binary Pattern feature (AvN-LBP) for indexing and an optimized Fuzzy Art Map (FAM) for classifying and searching medical images. The proposed indexing method extracts LBP considering information from neighborhood pixels and is robust to background noise. The FAM network is optimized using the Differential Evaluation (DE) algorithm (DEFAMNet) with a modified mutation operation to minimize the size of the network without compromising the classification accuracy. The performance of the proposed DEFAMNet is compared with that of other classifiers and descriptors; the classification accuracy of the proposed AvN-LBP operator with DEFAMNet is higher. The experimental results on three benchmark medical image datasets provide evidence that the proposed framework classifies the medical images faster and more efficiently with lesser computational cost.

Digital imaging has revolutionized the field of medical imaging and has led to the development of sophisticated computer hardware technologies and specialized software that empower physicians to better distinguish abnormalities, characterize findings, supervise interventions and predict prognosis. In fact, CAD is one of the major research subjects in medical imaging and diagnostic radiology. These benefits have motivated researchers to develop dedicated systems to specific medical domains from clinical decision making to medical education and research. The medical imaging field has generated additional interest in methods and tools for the management and analysis of these images. It is important to extend such applications by supporting the retrieval of medical images by content.KeywordsComputer Aided DiagnosisHealth Database ManagementMedical Image RetrievalSemanticRelevance Feedback

The deep hash method has been proved to be the most efficient and approximate nearest neighbor search technique for large-scale image retrieval. However, the existing deep hashing methods have few cases in medical image retrieval, so we propose a deep hash network model for medical image retrieval. Firstly, in order to enhance the feature extraction and learning ability in the network model, we add SENet visual attention mechanism into each residual module of the backbone network ResNet50 for feature extraction. Then the extracted high-dimensional semantic features are added to the Cauchy hash module to generate a compact, centralized hash code, and then complete medical image retrieval. So as to enhance experimental comparison, the proposed method was validated experimentally on NUS-WIDE and MURA data sets. A large number of experiments verify that the proposed method is reliable and can be used for medical image retrieval. Compared with other existing hash algorithms, the retrieval performance of the proposed method has certain improved. The framework of the proposed method is shown in Figure 1.

Interpreting medical images is certainly a complex task which requires extensive knowledge [1]. According to Computer Aided Diagnosis (CAD) serves as a second opinion that will help radiologists in diagnosis and on the other hand Content-based Image Retrieval uses visual content to help users browse, search and retrieve similar medical images from a database based on the user’s interest [2-4]. The competency of the CBMIR system depends on feature extraction methods [5]. The textural features are very important to determine the content of a medical image. Textural features provide scenic depth, the spatial distribution of tonal variation, and surface orientation [6]. Therefore, this study seeks to compare and evaluate some of the hand-crafted texture feature extraction techniques in CBMIR. This is to help those concerned in enhancing CBIR systems to make informed decisions concerning the selection of the best textural feature extraction techniques. Since there is no clear indication of which of the various texture feature extraction techniques is best suited for a given performance metric when considering which of the techniques to choose for a particular study in CBMIR systems. The objective of this work, therefore, is to comparatively evaluate the performance of the following texture feature extraction techniques; Local Binary Pattern (LBP), Gabor Filter, Gray-Level Co-occurrence Matrix (GLCM), Haralick Descriptor, Features from Accelerated Segment Test (FAST) and Features from Accelerated Segment Test and Binary Robust Independent Elementary Features (FAST &BRIEF) using the metrics; precision, recall, f1-score, mean squared error (MSE), accuracy and time. These techniques are coupled with specific similarity measure to obtain results. The results showed that LBP, Haralick Descriptor, FAST, and GLCM had the best results in terms of (Precision and Accuracy), Time, F1-Score, and Recall respectively. LBP had 82.05% and 88.23% scores for precision and accuracy respectively. The following scores represent the performance of the Haralick descriptor, FAST, and GLCM models respectively; 0.88s, 38.7%, and 44.82%. These test scores are obtained from datasets ranging from 1k-10.5k. Aside from LBP outperforming the other 5 models mentioned, it still outperformed the following proposed models ' [7]’, ‘Tamura texture feature and wavelet transform combined with Hausdorff distance - [8]’, ‘ [9]’ in terms of (precision, accuracy, and recall) and (precision and recall) respectively and probably f1-score (since f1-score is the weighted average of precision and recall). It is believed that an ensemble of LBP, Haralick descriptors, and Support Vector Machine (SVM) can represent a robust system for both medical image retrieval and classification.