Implementation and evaluation of segmentation algorithms according to multimodal imaging in personalized medicine

Masked Autoencoders for Unsupervised Anomaly Detection in Medical Images

Deep learning (DL) has been applied in proofs of concept across biomedical imaging, including across modalities and medical specialties. Labeled data are critical to training and testing DL models, but human expert labelers are limited. In addition, DL traditionally requires copious training data, which is computationally expensive to process and iterate over. Consequently, it is useful to prioritize using those images that are most likely to improve a model's performance, a practice known as instance selection. The challenge is determining how best to prioritize. It is natural to prefer straightforward, robust, quantitative metrics as the basis for prioritization for instance selection. However, in current practice, such metrics are not tailored to, and almost never used for, image datasets. To address this problem, we introduce ENRICH-Eliminate Noise and Redundancy for Imaging Challenges-a customizable method that prioritizes images based on how much diversity each image adds to the training set. First, we show that medical datasets are special in that in general each image adds less diversity than in nonmedical datasets. Next, we demonstrate that ENRICH achieves nearly maximal performance on classification and segmentation tasks on several medical image datasets using only a fraction of the available images and without up-front data labeling. ENRICH outperforms random image selection, the negative control. Finally, we show that ENRICH can also be used to identify errors and outliers in imaging datasets. ENRICH is a simple, computationally efficient method for prioritizing images for expert labeling and use in DL.

Anomaly detection (AD) is a challenging problem in computer vision. Particularly in the field of medical imaging, AD poses even more challenges due to a number of reasons, including insufficient availability of ground truth (annotated) data. In recent years, AD models based on generative adversarial networks (GANs) have made significant progress. However, their effectiveness in biomedical imaging remains underexplored. In this paper, we present an overview of using GANs for AD, as well as an investigation of state-of-the-art GAN-based AD methods for biomedical imaging and the challenges encountered in detail. We have also specifically investigated the advantages and limitations of AD methods on medical image datasets, conducting experiments using 3 AD methods on 7 medical imaging datasets from different modalities and organs/tissues. Given the highly different findings achieved across these experiments, we further analyzed the results from both data-centric and model-centric points of view. The results showed that none of the methods had reliable performance for detecting abnormalities in medical images. Factors such as the number of training samples, the subtlety of the anomaly, and the dispersion of the anomaly in the images are among the phenomena that highly impact the performance of the AD models. The obtained results were highly variable (AUC: 0.475-0.991; Sensitivity: 0.17-0.98; Specificity: 0.14-0.97). In addition, we provide recommendations for deployment of AD models in medical imaging and foresee important research directions.

Graph neural networks (GNNs) have achieved extraordinary enhancements in various areas including the fields medical imaging and network neuroscience where they displayed a high accuracy in diagnosing challenging neurological disorders such as autism. In the face of medical data scarcity and high-privacy, training such data-hungry models remains challenging. Federated learning brings an efficient solution to this issue by allowing to train models on multiple datasets, collected independently by different hospitals, in fully data-preserving manner. Although both state-of-the-art GNNs and federated learning techniques focus on boosting classification accuracy, they overlook a critical unsolved problem: investigating the reproducibility of the most discriminative biomarkers (i.e., features) selected by the GNN models within a federated learning paradigm. Quantifying the reproducibility of a predictive medical model against perturbations of training and testing data distributions presents one of the biggest hurdles to overcome in developing translational clinical applications. To the best of our knowledge, this presents the first work investigating the reproducibility of federated GNN models with application to classifying medical imaging and brain connectivity datasets. We evaluated our framework using various GNN models trained on medical imaging and connectomic datasets. More importantly, we showed that federated learning boosts both the accuracy and reproducibility of GNN models in such medical learning tasks. Our source code is available at https://github.com/basiralab/reproducibleFedGNN.KeywordsGraph neural networksFederated LearningReproducibilityBrain connectivity graphsPredictive medicine

Semisupervised strategy has been utilized to alleviate issues from segmentation applications due to challenges in collecting abundant annotated segmentation masks, which is an essential prerequisite for training high-performance 3D convolutional neural networks (CNNs) . Existing semisupervised segmentation methods are mainly concerned with how to generate the pseudo labels with regularization but not evaluate the quality of the pseudo labels explicitly. To alleviate this problem, we offer a simple yet effective reciprocal learning strategy for semisupervised volumetric medical image segmentation, which generates more reliable pseudo labels for the unannotateddata. Our proposed reciprocal learning is achieved through a pair of networks, one as a teacher network and the other as a student network. The student network learns from pseudo labels generated by the teacher network. In addition, the teacher network autonomously optimizes its parameters based on the reciprocal feedback signals from the student's performance on the annotated images. The efficacy of the proposed method is evaluated on three medical image data sets, including 82 pancreas computed tomography (CT) scans (training/testing: 62/20), 100 left atrium gadolinium-enhanced magnetic resonance (MR) scans (training/testing: 80/20), and 200 breast cancer MR scans (training/testing: 68/132). The comparison methods include mean teacher (MT) model, uncertainty-aware MT (UA-MT) model, shape-aware adversarial network (SASSNet), and transformation-consistent self-ensembling model (TCSM). The evaluation metrics are Dice similarity coefficient (Dice), Jaccard index (Jaccard), 95% Hausdorff distance (95HD), and average surface distance (ASD). The Wilcoxon signed-rank test is used to conduct the statisticalanalyses. By utilizing 20% labeled data and 80% unlabeled data for training, our proposed method achieves an average Dice of 84.77%/90.46%/78.53%, Jaccard of 73.71%/82.67%/69.00%, ASD of 1.58/1.90/0.57, and 95HD of 6.24/5.97/4.34 on pancreas/left atrium/breast data sets, respectively. These results outperform several cutting-edge semisupervised approaches, showing the feasibility of our method for the challenging semisupervised segmentationapplications. The proposed reciprocal learning strategy is a general semisupervised solution and has the potential to be applied for other 3D segmentationtasks.

The federated learning (FL) paradigm is well-suited for the field of medical image analysis, as it can effectively cope with machine learning on isolated multi-center data while protecting the privacy of participating parties. However, current research on optimization algorithms in FL often focuses on limited datasets and scenarios, primarily centered around natural images, with insufficient comparative experiments in medical contexts. In this work, we conduct a comprehensive evaluation of several state-of-the-art FL algorithms in the context of medical imaging. We conduct a fair comparison of classification models trained using various FL algorithms across multiple medical imaging datasets. Additionally, we evaluate system performance metrics, such as communication cost and computational efficiency, while considering different FL architectures. Our findings show that medical imaging datasets pose substantial challenges for current FL optimization algorithms. No single algorithm consistently delivers optimal performance across all medical FL scenarios, and many optimization algorithms may under-perform when applied to these datasets. Our experiments provide a benchmark and guidance for future research and application of FL in medical imaging contexts. Furthermore, we propose an efficient and robust method that combines generative techniques using denoising diffusion probabilistic models with label smoothing to augment datasets, widely enhancing the performance of FL on classification tasks across various medical imaging datasets.

Research in disease diagnosis is a challenging task due to inconsistent, class imbalance, conflicting, and the high dimensionality of medical data sets. The excellent features of each data set play an important role in improving performance of classifiers that may follow either iterative or non-iterative approaches. In the present study, a comparative study is carried out to show the performance of iterative and non-iterative classifiers in combination with genetic algorithm (GA)-based feature selection approach over some widely used medical data sets. The experiment assists to identify the clinical data sets for which feature reduction is necessary for improving performance of classifiers. For iterative approaches, two popular classifiers, namely C4.5 and RIPPER, are chosen, whereas k-NN and naïve Bayes are taken as non-iterative learners. Fourteen real-world medical domain data sets are selected from the University of California, Irvine (UCI Repository) for conducting experiment over the learners.

Multimodal medical image fusion plays an important role in clinical applications. However, multimodal medical image fusion methods ignore the feature dependence among modals, and the feature fusion ability with different granularity is not strong. A Long-Range Correlation-Guided Dual-Encoder Fusion Network for Medical Images is proposed in this paper. The main innovations of this paper are as follows: Firstly, A Cross-dimension Multi-scale Feature Extraction Module (CMFEM) is designed in the encoder, by extracting multi-scale features and aggregating coarse-to-fine features, the model realizes fine-grained feature enhancement in different modalities. Secondly, a Long-range Correlation Fusion Module (LCFM) is designed, by calculating the long-range correlation coefficient between local features and global features, the same granularity features are fused by the long-range correlation fusion module. long-range dependencies between modalities are captured by the model, and different granularity features are aggregated. Finally, this paper is validated on clinical multimodal lung medical image dataset and brain medical data dataset. On the lung medical image dataset, IE, AG, {textbf {Q}}^{{textbf {AB/F}}}, and EI metrics are improved by 4.53%, 4.10%, 6.19%, and 6.62% respectively. On the brain medical image dataset, SF, VIF, and {textbf {Q}}^{{textbf {AB/F}}} metrics are improved by 3.88%, 15.71%, and 7.99% respectively. This model realizes better fusion performance, which plays an important role in the fusion of multimodal medical images.

Deep learning methods, especially convolutional neural networks, have become more and more popular in medical image classifications. However, training a deep neural network from scratch can be a luxury for many medical image datasets as the process requires a large and well-balanced sample to output satisfactory results. Unlike natural image datasets, medical images are expensive to collect owing to labor and equipment costs. Besides, the class labels in medical image datasets are usually severely imbalanced subject to the availability of patients. Further, aggregating medical images from multiple sources can be challenging due to policy restrictions, privacy concerns, communication costs, and data heterogeneity caused by equipment differences and labeling discrepancies. In this paper, we propose to address these issues with the help of transfer learning and artificial samples created by generative models. Instead of requesting medical images from source data, our method only needs a parsimonious supplement of model parameters pre-trained on the source data. The proposed method preserves the data privacy in the source data and significantly reduces the communication cost. Our study shows transfer learning together with artificial samples can improve the pneumonia classification accuracy on a small but heavily imbalanced chest X-ray image dataset by \(11.53\%\) which performs even better than directly augmenting that source data into the training process.KeywordsDeep learningGenerative modelsMedical image classificationPrivacy preservationTransfer learning

Hyperspectral imaging is a rising imaging modality in the field of medical applications, and the combination of both spectral and spatial information provides wealth information for cell classification. In this paper, deep convolutional neural network (CNN) is employed to achieve blood cell discrimination in medical hyperspectral images (MHSI). As a deep learning architecture, CNNs are expected to get more discriminative and semantic features, which effect classification accuracy to a certain extent. Experimental results based on two real medical hyperspectral image data sets demonstrate that cell classification using CNNs is effective. In addition, compared to traditional support vector machine (SVM), the proposed method, which jointly exploits spatial and spectral features, can achieve better classification performance, showcasing the CNN-based methods' tremendous potential for accurate medical hyperspectral data classification.

Automatic segmentation of salient objects in real-world images has gained increasing interests owing to its popularity in diverse real-world applications, such as autonomous driving, medical diagnosis, aviation security, and underwater surveillance. In this research, we propose Firefly Algorithm (FA)-enhanced evolving ensemble deep networks for semantic segmentation and visual saliency prediction. An improved FA model is proposed to optimize network hyper-parameters. Specifically, it employs mutation operators and a neighbouring search strategy with granular search steps to establish search intensification. It also emphasizes search diversification by adopting multiple dynamic hybrid leaders and diverse adaptive sine and cosine search trajectories in full and randomly selected sub-dimensions to overcome stagnation. Because of its competent segmentation performance, DeepLabV3+ is fine-tuned using transfer learning with FA-based hyper-parameter identification. We optimize the learning rate, momentum and weight decay of the transfer learning network. A number of optimized DeepLabV3+ networks with distinguishing learning configurations are yielded. An ensemble model is subsequently constructed by incorporating three optimized base networks to further strengthen segmentation performance. Evaluated using diverse challenging semantic segmentation and saliency prediction tasks using underwater and medical image data sets, our evolving ensemble deep network illustrates significant superiority over other state-of-the-art deep networks and existing studies. The proposed FA model also outperforms other search methods in solving diverse mathematical landscapes with statistical significance.

Cartilage defects heal imperfectly and osteoarthritic changes develop frequently as a result. Although the existence of specific behaviours of chondrocytes derived from various depth-related zones in vitro has been known for over 20 years, only a relatively small body of in vitro studies has been performed with zonal chondrocytes and current clinical treatment strategies do not reflect these native depth-dependent (zonal) differences. This is surprising since mimicking the zonal organization of articular cartilage in neo-tissue by the use of zonal chondrocyte subpopulations could enhance the functionality of the graft. Although some research groups including our own have made considerable progress in tailoring culture conditions using specific growth factors and biomechanical loading protocols, we conclude that an optimal regime has not yet been determined. Other unmet challenges include the lack of specific zonal cell sorting protocols and limited amounts of cells harvested per zone. As a result, the engineering of functional tissue has not yet been realized and no long-term in vivo studies using zonal chondrocytes have been described. This paper critically reviews the research performed to date and outlines our view of the potential future significance of zonal chondrocyte populations in regenerative approaches for the treatment of cartilage defects. Secondly, we briefly discuss the capabilities of additive manufacturing technologies that can not only create patient-specific grafts directly from medical imaging data sets but could also more accurately reproduce the complex 3D zonal extracellular matrix architecture using techniques such as hydrogel-based cell printing.

Abstract: A brain tumour is described by the presence of abnormal cells in the brain's tissues. Brain tumours can be benign (not cancerous) or malignant (cancerous). The malignant brain tumour is one of the leading and common cancers in the world. There are two types of tumours, primary tumours that develop in the brain and secondary tumours that start in another region of the body and then spread to the brain. The precise identification of the size and location of a brain tumour is crucial in the diagnosis of a brain tumour and is often diagnosed with magnetic resonance imaging (MRI). This book chapter discusses the major types of brain tumours and the advancements in computeraided approaches for detecting brain tumours. The manuscript gives an overview of various recent machine learning and medical image processing approaches developed recently for the identification and classification of brain tumours. Several medical image dataset available for the research works in this domain is also briefed in this article. The major research challenges which can be addressed by the researchers in the domain of brain tumour detection are also discussed in this article. Method: This manuscript mainly discusses various medical image processing methods to detect brain tumors. Results: The manuscript discusses the efficiency of the existing schemes and the key areas where further improvement is required Conclusion: This manuscript gives an overview of various categories of brain tumors, existing methods to detect brain tumors, the critical challenges in this domain and the medical image dataset available for the study.

Deep neural networks represent the state of the art for computer-aided medical imaging assessment, e.g. lesion detection, organ segmentation and disease classification. While for large datasets their superior performance is a clear argument, medical imaging data is often small and highly heterogeneous. In combination with the typical parameter amount in deep neural networks, this often leads to overfitting and results in a low level of generalization performance. We propose a straight-forward combination of random forests and deep neural networks for superior performance on medical imaging datasets with only small data, and provide an extensive evaluation of survival prediction for metastatic colorectal cancer patients using computed tomography imaging data, with our proposed method clearly outperforming other approaches.

Vision foundation models have shown great potential in improving generalizability and data efficiency, especially for medical image segmentation since medical image datasets are relatively small due to high annotation costs and privacy concerns. However, current research on foundation models predominantly relies on transformers. The high quadratic complexity and large parameter counts make these models computationally expensive, limiting their potential for clinical applications. In this work, we introduce Swin-UMamba†, a novel Mamba-based model for medical image segmentation that seamlessly leverages the power of the vision foundation model, which is also computationally efficient with the linear complexity of Mamba. Moreover, we investigated and verified the impact of the vision foundation model on medical image segmentation, in which a self-supervised model adaptation scheme was designed to bridge the gap between natural and medical data. Notably, Swin-UMamba† outperforms 7 state-of-the-art methods, including CNN-based, transformer-based, and Mamba-based approaches across AbdomenMRI, Encoscopy, and Microscopy datasets. The code and models are publicly available at: https://github.com/JiarunLiu/Swin-UMamba.