Challenges of Deep Learning in Medical Image Analysis—Improving Explainability and Trust

The rapid development of medical practices and imaging technology tools creates substantial growth in the amount of medical image data each year in our present era. This research aims to develop a hybrid approach that integrates Machine Learning (ML) and Deep Learning (DL) techniques to enhance the accuracy and reliability of medical image classification for diagnostic purposes. Medical imaging data complexity and growing volume serve as the research motivation, which leads to an investigation of standalone ML or DL limitations and their combination into a single framework. The medical image processing starts with normalization, then noise reduction, and continues to grayscale conversion before performing histogram equalization. This research uses VGG16 and ResNet50 alongside MobileNet and InceptionV3 for feature extraction, then applies ten different ML algorithms, including SVM and MLP, and Ran-dom Forest, for classification. Five public medical image datasets from Kaggle are used: COVID-19 chest X-rays, melanoma skin lesions, pneumonia chest X-rays, acute stroke facial images, and various eye diseases. Hybrid models display superior performance compared to stand-alone ML or DL models based on accuracy, precision, recall, and F1-score evaluation measures. Multiple datasets demonstrate that the MobileNet+MLP combination de-livers the most accurate results, which demonstrates its reliable and efficient performance. The developed AI diagnostic tool presents a scalable system alongside accuracy and interpretability to enhance clinical decision outcomes.

Deep Learning has emerged as a strategic area in medical image diagnosis, with numerous studies exploring automation in this field. However, many existing approaches attempt to fully replace medical professionals by relying exclusively on model predictions. This thesis presents an alternative strategy, proposing Deep Learning architectures designed to support, rather than replace, physicians in diagnostic tasks. Two distinct methodologies are explored to achieve this goal: Content-Based Image Retrieval and eXplainable Artificial Intelligence. Content-Based Image Retrieval system enables physicians to compare complex cases with similar ones from a curated database, enhancing decision-making through feature-based similarities. The eXplainable Artificial Intelligence approach provides visual explanations through heatmaps, indicating the regions of interest detected by the model. The thesis introduces two novel architecturesMulti-Output Classification Autoencoder (MOC-AE) and Multi-Output Classification Variational Autoencoder (MOC-VAE). These models employ multi-objective training to generate rich image descriptors, capturing both pathological and anatomical information. These descriptors are then used to identify visually similar cases in a low-dimensional space. To address the challenge of quantifying visual similarity in Content-Based Image Retrieval, the Sliced Wasserstein distance is proposed, offering a robust metric for comparative diagnosis. Furthermore, the thesis demonstrates the utility of these architectures in generating precise, non-linear explanations for diagnoses, leveraging the factorization process inherent to the models. The simplicity and enhanced performance of the proposed approaches, compared to prior methods, offer a foundation for future research and real-world applications in medical diagnostics using Deep Learning. RESUMEN El Deep Learning ha surgido como un área estratégica en el diagnóstico de imágenes médicas, con numerosos estudios que exploran la automatización en este campo. Sin embargo, muchos enfoques actuales buscan reemplazar por completo a los profesionales médicos basándose únicamente en las predicciones de los modelos. Esta tesis presenta una estrategia alternativa, proponiendo arquitecturas de Deep Learning diseñadas para apoyar, en lugar de reemplazar, a los médicos en tareas diagnósticas. Se exploran dos metodologías distintas para alcanzar este objetivo: Sistemas de Recuperación de Imágenes Basada en Contenido e Inteligencia Artificial Explicable. Los Sistemas de Recuperación de Imágenes Basada en Contenido permiten a los médicos comparar casos complejos con otros similares de una base de datos curada, mejorando la toma de decisiones mediante similitudes basadas en características. El enfoque de la Inteligencia Artificial Explicable proporciona explicaciones visuales a través de mapas de calor, que indican las regiones de interés detectadas por el modelo. La tesis introduce dos arquitecturas novedosas: el Autoencoder de Clasificación Multi-Salida (MOC-AE) y el Autoencoder Variacional de Clasificación Multi-Salida (MOC-VAE). Estos modelos emplean un entrenamiento multiobjetivo para generar descriptores de imágenes enriquecidos que capturan tanto información patológica como anatómica. Estos descriptores se utilizan para identificar casos visualmente similares en un espacio de baja dimensionalidad. Para abordar el desafío de cuantificar la similitud visual en Sistemas de Recuperación de Imágenes Basada en Contenido, se propone la distancia de Wasserstein por secciones, ofreciendo una métrica robusta para el diagnóstico comparativo. Además, la tesis demuestra la utilidad de estas arquitecturas para generar explicaciones precisas y no lineales de los diagnósticos, aprovechando el proceso de factorización inherente a los modelos. La simplicidad y el rendimiento mejorado de los enfoques propuestos, en comparación con métodos anteriores, ofrecen una base sólida para futuras investigaciones y aplicaciones reales en diagnósticos médicos utilizando Deep Learning.

We propose data augmentation using fractal images to train deep learning models for medical image diagnosis. Deep learning models for image classification typically demand large datasets, which can be challenging in the context of medical image diagnosis. Current approaches often involve pre-training of model parameters using natural image databases such as ImageNet and fine-tuning of the parameters with specific medical image data. However, natural and medical images have distinct characteristics, which questions the suitability of pre-training using natural image data. Moreover, the scalability of natural image databases is limited; thus, acquiring sufficient data for large-scale deep learning models is difficult. In contrast, Kataoka et al. introduced a mathematical model for generating image data and demonstrated its effectiveness when used in pre-training for natural image classification. In this study, we employed a pre-trained model utilizing fractals among mathematical models and experimentally classified CT images of COVID-19 pneumonia. The experimental results demonstrated that this fractal-based pre-training model achieved accuracy comparable to conventional natural image-based approach. Fractal images are easily generated compared to natural images. Furthermore, generating appropriate data for specific applications may be possible by adjusting the parameters. This flexibility in generating data allows customization and optimization of the model for different scenarios or specific requirements. We believe that this approach holds promise in medical image diagnosis, where the number of samples is often limited.

The medical diagnosis of diseases involves the detection and classification of patients from medical image data. It has an important Artificial Intelligence (AI) application in Healthcare and Neuroscience fields. In addition, Machine Learning (ML) and Deep Learning (DL) techniques provide important tasks on Neurodegenerative Diseases (ND) diagnosis from Brain MRI (Magnetic Resonance Imaging) data for image processing with good classification results. This chapter aims at comprehensive and comparative study on the performance of medical image classification with those techniques using Brain MRI datasets. We introduce a brief overview of the medical diagnosis of ND with AI applications in the neuroscience domain. In the literature part, we present the understanding of the medical image classification with Machine and Deep Learning techniques. Therefore, we describe the concept of MRI data and the Brain MRI datasets that are used in this work. In the comparison part, we focus on related works of the Artificial Neural Network (ANN) and the Deep Convolutional Neural Network (DCNN) models and we compare these models in the literature. In the methodology part, we combine the ANN and DCNN in order to propose a new Deep model named ANN-DCNN model when it is more accurate and effective to classify from those Brain images. To improve the performance of those Neural Networks in medical diagnosis applications, we discuss the recent insights and future directions.

Recently, deep learning has been exploited in the field of medical image analysis. However, the training of deep learning models with medical images is time-consuming since most medical image data are three-dimensional volumes or high-resolution two-dimensional images. Moreover, the optimization of numerous hyperparameters strongly affects the performance of deep learning. If a framework for training deep learning with hyperparameter optimization on a supercomputer system can be realized, it is expected to accelerate the training of deep learning with medical images. In this study, we described our novel environment for training deep learning with medical images on the supercomputer system in our institute (Reedbush-H supercomputer system) based on asynchronous parallel Bayesian optimization. We trained two types of automated lesion detection application in a constructed environment. The constructed environment enabled us to train deep learning with hyperparameter tuning in a short time.

Deep learning has emerged as a powerful artificial intelligence tool to interpret medical images for a growing variety of applications. However, the paucity of medical imaging data with high-quality annotations that is necessary for training such methods ultimately limits their performance. Medical data is challenging to acquire due to privacy issues, shortage of experts available for annotation, limited representation of rare conditions and cost. This problem has previously been addressed by using synthetically generated data. However, networks trained on synthetic data often fail to generalize to real data. Cinematic rendering simulates the propagation and interaction of light passing through tissue models reconstructed from CT data, enabling the generation of photorealistic images. In this paper, we present one of the first applications of cinematic rendering in deep learning, in which we propose to fine-tune synthetic data-driven networks using cinematically rendered CT data for the task of monocular depth estimation in endoscopy. Our experiments demonstrate that: (a) convolutional neural networks (CNNs) trained on synthetic data and fine-tuned on photorealistic cinematically rendered data adapt better to real medical images and demonstrate more robust performance when compared to networks with no fine-tuning, (b) these fine-tuned networks require less training data to converge to an optimal solution, and (c) fine-tuning with data from a variety of photorealistic rendering conditions of the same scene prevents the network from learning patient-specific information and aids in generalizability of the model. Our empirical evaluation demonstrates that networks fine-tuned with cinematically rendered data predict depth with 56.87% less error for rendered endoscopy images and 27.49% less error for real porcine colon endoscopy images.

The development of computer hardware allows rapid accumulation of medical imaging data. Deep learning has shown great potential in medical imaging data analysis and establish a new area of machine learning. The commonly used deep learning models were firstly introduced in the paper, and then, summarized with the application of deep learning in the detection, classification, diagnosis, segmentation, identification of medical imaging. The application of deep learning in oral and maxillofacial radiology and other discipline of stomatology was proposed. At the end, the paper discussed the problems of deep learning in medical imaging research.

Medical data is increasing rapidly. The rapid fusion of medical record text data, medical laboratory data and image data in many regional hospitals have brought light to the screening, diagnosis and treatment of diseases. Medical text data records the patient's detailed condition and treatment process, with abundant information. As a branch of artificial intelligence, deep learning is gradually penetrating the medical field. The application of deep learning in the medical field shows great application prospects. Therefore, mining the knowledge contained in medical texts can provide better medical services for the majority of patients, which has good theoretical and practical significance. In this paper, the content of medical texts was analyzed, and the test questions and standard answers of the USMLE Step 2 Clinical Skills Exam were used as data sets to explore the core content of the medical chief texts. After comparing and analyzing different methods, such as BERT and DeBERTa, an intelligent follow-up system based on DeBERTa was finally proposed, hoping to help clinicians read medical data more effectively and weaken the problem of poor diagnosis results caused by personal academic ability differences.

Deep neural networks represent the state of the art for computer-aided medical imaging assessment, e.g. lesion detection, organ segmentation and disease classification. While for large datasets their superior performance is a clear argument, medical imaging data is often small and highly heterogeneous. In combination with the typical parameter amount in deep neural networks, this often leads to overfitting and results in a low level of generalization performance. We propose a straight-forward combination of random forests and deep neural networks for superior performance on medical imaging datasets with only small data, and provide an extensive evaluation of survival prediction for metastatic colorectal cancer patients using computed tomography imaging data, with our proposed method clearly outperforming other approaches.

With the rise of deep learning, medical image analysis has undergone fundamental change, presenting new possibilities to physicians in terms of achieving higher accuracy more efficiently, and greater automation in disease diagnosis and prognosis. In this paper, we discuss recent progress in utilizing deep learning techniques such as convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), and transformer-based model for medical image analysis. Image classification, segmentation, and anomaly detection in modalities such MRI, CT, Xray, and ultrasound have been strongly improved by these architectures. Yet, such progress brings challenges, namely scarcity of sufficient data, interpretability of models, computational intensity, and robust generalization to diverse clinical datasets. At the same time regulatory concerns and ethical questions on the use of AI diagnostics are still constraining large scale adoption. This paper provides a comprehensive review of state-of-the art deep learning models in medical imaging, reviews the current limitations, and envisions future research directions in federated learning, explainable AI, and quantum enhanced deep learning. Such insights are intended to guide researchers and clinicians in the optimization of AI driven medical image analysis for optimized patient care and clinical decision making.

The approach that is most helpful to medical professionals in the process of diagnosing disorders in patients is medical imaging diagnosis, which employs a variety of imaging test modalities. In fact, the purpose of deep learning is to imitate human cognitive processes. A paradigm change is being brought about in the area of medical imaging as a result of the growing availability of data pertaining to medical imaging and the development of methodologies that revolve on deep learning. For a variety of tasks in medical imaging, such as image capture, image retrieval, segmentation, and illness prediction, deep learning algorithms have effectively emerged as the method of choice. In this study, we provide a summary that examines the application of deep learning to medical imaging from a variety of perspectives.

Medical image analysis using deep neural networks (DNN) has demonstrated state-of-the-art performance in image classification and segmentation tasks, aiding disease diagnosis. The accuracy of the DNN is largely governed by the quality and quantity of the data used to train the model. However, for the medical images, the critical security and privacy concerns regarding sharing of local medical data across medical establishments precludes exploiting the full DNN potential for clinical diagnosis. The federated learning (FL) approach enables the use of local model's parameters to train a global model, while ensuring data privacy and security. In this paper, we review the federated learning applications in medical image analysis with DNNs, highlight the security concerns, cover some efforts to improve FL model performance, and describe the challenges and future research directions.

Deep learning is a fast flattering highest development in the machine learning technique leading to augment performance in diverse medical applications. The tributary artificial intelligence is rapidly becoming metamorphic in the domain of Healthcare Technology rendering facility to analyze the medical data with high rapidity and exactness. Due to the overwhelming development of artificial intelligence (AI), the contemporary encroachments of deep learning employing advanced and futuristic deep learning methods has become a vivacious research domain for processing medical data in the health care industries. This paper presents a complete analysis about the deep structured learning based methods used in medical image analysis and also elucidate about the advancement of convolutional neural network which is a major class of deep neural networks used in medical image processing. It also gives insight about the fundamentals of deep learning methods and reviews in image interpretation. It also explains about the implementation of an integrated framework for medical image synthesis combining Speckle GAN and Di-Conv-AE-Net.

Non-negative subspace feature representation for few-shot learning in medical imaging

We review literature in top journals and conferences on the usage of deep learning for medical image analysis in modern healthcare. As a result it is shown that deep learning offers unique capabilities and breakthroughs in identifying, classifying and segmenting different kinds of medical images, especially related to cancer in the breast, lung, and brain.