EMED-UNet: An Efficient Multi-Encoder-Decoder Based UNet for Medical Image Segmentation

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for various imaging related tasks such as artificial image generation to support AI training. GANs are especially useful for medical imaging-related tasks where training datasets are usually limited in size and heavily imbalanced against the diseased class. We present a systematic review, following the PRISMA guidelines, of recent GAN architectures used for medical image analysis to help the readers in making an informed decision before employing GANs in developing medical image classification and segmentation models. We have extracted 54 papers that highlight the capabilities and application of GANs in medical imaging from January 2015 to August 2020 and inclusion criteria for meta-analysis. Our results show four main architectures of GAN that are used for segmentation or classification in medical imaging. We provide a comprehensive overview of recent trends in the application of GANs in clinical diagnosis through medical image segmentation and classification and ultimately share experiences for task-based GAN implementations.

BackgroundMagnetic Resonance Imaging (MRI) is a cornerstone in diagnosing brain tumors. However, the complex nature of these tumors makes accurate segmentation in MRI images a demanding task.ObjectiveAccurate brain tumor segmentation remains a critical challenge in medical image analysis, with early detection crucial for improving patient outcomes.MethodsTo develop and evaluate a novel UNet-based architecture for improved brain tumor segmentation in MRI images. This paper presents a novel UNet-based architecture for improved brain tumor segmentation. The UNet model architecture incorporates Leaky ReLU activation, batch normalization, and regularization to enhance training and performance. The model consists of varying numbers of layers and kernel sizes to capture different levels of detail. To address the issue of class imbalance in medical image segmentation, we employ focused loss and generalized Dice (GDL) loss functions.ResultsThe proposed model was evaluated on the BraTS'2020 dataset, achieving an accuracy of 99.64% and Dice coefficients of 0.8984, 0.8431, and 0.8824 for necrotic core, edema, and enhancing tumor regions, respectively.ConclusionThese findings demonstrate the efficacy of our approach in accurately predicting tumors, which has the potential to enhance diagnostic systems and improve patient outcomes.

Convolutional neural networks (CNNs), as a typical deep learning technique, have been widely used in image segmentation, but they often require a large amount of annotated data. However, the number of available pixel-wise labeled medical images is extremely small, and this prevents the application of CNNs in many medical image segmentation tasks. We proposed a self-paced learning with diversity SPLD) framework to boost the performances of medical image segmentation models with a limited amount of annotated data. Self-paced learning (SPL) is a learning regime that selects training samples in order from the easiest to the most difficult for model training. In addition, we took the diversity of the data into consideration. The proposed self-paced learning with diversity by query-by-committee (SPLD-QBC) algorithm dynamically and diversely selects the appropriate training data to boost the performance of an image segmentation model. SPLD-QBC incorporates the query-by-committee (QBC) technique for data selection and affinity propagation for optimizing the data diversity. By dynamically selecting the optimal sequence of training samples from different probability distributions, the segmentation models achieved improved performances. To verify the effectiveness of the proposed SPLD-QBC framework, we conducted experiments on three medical image segmentation tasks with five different datasets. The experimental results indicated that the proposed SPLD-QBC algorithm significantly improved upon the segmentation performances of the baseline models and resulted in a higher Dice score, surface distance and mean intersection over union (mIoU). The proposed SPLD significantly boosts the segmentation performances of models and is easily embedded into CNN-based image segmentation models.

Medical image segmentation of tissue abnormalities, key organs, or blood vascular system is of great significance for any computerized diagnostic system. However, automatic segmentation in medical image analysis is a challenging task since it requires sophisticated knowledge of the target organ anatomy. This paper develops an end-to-end deep learning segmentation method called Contextual Multi-Scale Multi-Level Network (CMM-Net). The main idea is to fuse the global contextual features of multiple spatial scales at every contracting convolutional network level in the U-Net. Also, we re-exploit the dilated convolution module that enables an expansion of the receptive field with different rates depending on the size of feature maps throughout the networks. In addition, an augmented testing scheme referred to as Inversion Recovery (IR) which uses logical “OR” and “AND” operators is developed. The proposed segmentation network is evaluated on three medical imaging datasets, namely ISIC 2017 for skin lesions segmentation from dermoscopy images, DRIVE for retinal blood vessels segmentation from fundus images, and BraTS 2018 for brain gliomas segmentation from MR scans. The experimental results showed superior state-of-the-art performance with overall dice similarity coefficients of 85.78%, 80.27%, and 88.96% on the segmentation of skin lesions, retinal blood vessels, and brain tumors, respectively. The proposed CMM-Net is inherently general and could be efficiently applied as a robust tool for various medical image segmentations.

Ultrasound imaging can distinctly display the morphology and structure of internal organs within the human body, enabling the examination of organs like the breast, liver, and thyroid. It can identify the locations of tumors, nodules, and other lesions, thereby serving as an efficacious tool for treatment detection and rehabilitation evaluation. Typically, the attending physician is required to manually demarcate the boundaries of lesion locations, such as tumors, in ultrasound images. Nevertheless, several issues exist. The high noise level in ultrasound images, the degradation of image quality due to the impact of surrounding tissues, and the influence of the operator’s experience and proficiency on the determination of lesion locations can all contribute to a reduction in the accuracy of delineating the boundaries of lesion sites. In the wake of the advancement of deep learning, its application in medical image segmentation is becoming increasingly prevalent. For instance, while the U-Net model has demonstrated a favorable performance in medical image segmentation, the convolution layers of the traditional U-Net model are relatively simplistic, leading to suboptimal extraction of global information. Moreover, due to the significant noise present in ultrasound images, the model is prone to interference. In this research, we propose an Attention Residual Network model (ARU-Net). By incorporating residual connections within the encoder section, the learning capacity of the model is enhanced. Additionally, a spatial hybrid convolution module is integrated to augment the model’s ability to extract global information and deepen the vertical architecture of the network. During the feature fusion stage of the skip connections, a channel attention mechanism and a multi-convolutional self-attention mechanism are respectively introduced to suppress noisy points within the fused feature maps, enabling the model to acquire more information regarding the target region. Finally, the predictive efficacy of the model was evaluated using publicly accessible breast ultrasound and thyroid ultrasound data. The ARU-Net achieved mean Intersection over Union (mIoU) values of 82.59% and 84.88%, accuracy values of 97.53% and 96.09%, and F1-score values of 90.06% and 89.7% for breast and thyroid ultrasound, respectively.

The U-Net model for lung area segmentation in medical images is considered in this work. A preprocessed and enhanced dataset was used to train the U-Net model across 70 epochs using an encoder-decoder architecture with skip connections. To estimate the accuracy of the model, important performance indicators such as the Dice coefficient, Intersection over Union (IOU), and training/validation loss were observed. The coefficient for the dice went up from 0.5 to 0.9 in the results, while the IOU value calmed at 0.9, representing the model’s efficiency in proper segmentation. Strong generalization to previously unseen data and minimal overfitting were shown by the loss metrics’ in accordance reduction. In agreement to the study, U-Net has the potential to be used in real-world medical applications. Further study is recommended to enhance performance through the application of transfer learning and consideration devices.

Deep learning has achieved great success in the field of computer vision, and the precision in image classification and image detection has surpassed humans. Therefore, this paper combines deep learning and medical image segmentation, focusing on how to improve the accuracy and speed of segmentation algorithm of medical exercise rehabilitation image. Aiming at the shortcomings of traditional medical image recognition methods, a medical exercise rehabilitation image segmentation algorithm based on hierarchical features of convolutional neural networks is proposed, this paper calls it as hierarchical features of convolutional neural networks (HFCNN). The algorithm firstly samples the convolution output of multiple layers in the convolutional neural network to a unified scale and combines them to construct a hierarchical feature. This hierarchical feature combines the structural information of objects contained in the shallow layer of the network with the semantic information of objects contained in the deep layers of the network, so it has a strong ability to express. Secondly, the image can be segmented into multiple super pixels by the super pixel segmentation algorithm. The classifier is trained using the hierarchical features of the super pixel, and then the classification result is mapped back to the pixel. Finally, a fully connected conditional random field algorithm including one-potential potential energy and paired potential energy is constructed. The corresponding energy function is used to smooth the classification result of the pixel, and the regional consistency and continuity of the pixel mark are improved. Compared with many classical convolutional neural network algorithms, this algorithm not only accelerates the network convergence speed, shortens the training time, but also significantly improves the accuracy of segmentation algorithm of medical exercise rehabilitation image, showing good practical value.

Segment anything model (SAM) has recently demonstrated tremendous potential in the artificial intelligence-based medical image segmentation. However, its outstanding performance is always accompanied by surging computation costs due to the extremely long patch sequences and their quadratic complexity in vision transformer, which hamper the development of real-time engineeringapplications. This work aims to accelerate SAM for medical image segmentation with better segmentation quality and less memoryusage. Instead of upscaling training images to larger resolutions to match the pre-trained SAM's input size, which may introduce image distortions, we propose a convolutional neural network (CNN)-assisted tuning strategy. This approach enables SAM to process smaller inputs directly, significantly reducing the number of patches and memory consumption. Moreover, based on the observation that the less-informative patches in medical images tend to be well-classified earlier than others, revealing the massive computation redundancy in the SAM, a token pausing strategy is presented to identify these easy-to-predicted patches in the feed-forward process and then pause their computation. Finally, we combine the CNN-assisted tuning strategy with the token pausing strategy, meets various run-time requirements by simply adjusting the pause parameters, and avoids repeatedtraining. Extensive experiments on two medical image segmentation benchmarks, namely Synapse and ACDC, demonstrate that our model can run 12 faster than existing SAM-based solutions, and meanwhile achieves a better segmentationperformance. This study identifies two practical limitations when applying SAM to medical imaging: resizing low-resolution inputs to match the pre-trained scale and uniformly processing all patches, which lead to redundant computation. Therefore, we present an efficient strategy that integrates adapter-based tuning with token pausing to enhance throughput while preserving segmentation performance. Relevant codes and models of this paper will be available at https://github.com/wuwen1994/FasterSAM.

Semantic image segmentation has a wide range of applications. When it comes to medical image segmentation, its accuracy is even more important than those of other areas because the performance gives useful information directly applicable to disease diagnosis, surgical planning, and history monitoring. The state-of-the-art models in medical image segmentation are variants of encoder-decoder architecture, which is called U-Net. To effectively reflect the spatial features in feature maps in encoder-decoder architecture, we propose a spatially adaptive weighting scheme for medical image segmentation. Specifically, the spatial feature is estimated from the feature maps, and the learned weighting parameters are obtained from the computed map, since segmentation results are predicted from the feature map through a convolutional layer. Especially in the proposed networks, the convolutional block for extracting the feature map is replaced with the widely used convolutional frameworks: VGG, ResNet, and Bottleneck Resent structures. In addition, a bilinear up-sampling method replaces the up-convolutional layer to increase the resolution of the feature map. For the performance evaluation of the proposed architecture, we used three data sets covering different medical imaging modalities. Experimental results show that the network with the proposed self-spatial adaptive weighting block based on the ResNet framework gave the highest IoU and DICE scores in the three tasks compared to other methods. In particular, the segmentation network combining the proposed self-spatially adaptive block and ResNet framework recorded the highest 3.01% and 2.89% improvements in IoU and DICE scores, respectively, in the Nerve data set. Therefore, we believe that the proposed scheme can be a useful tool for image segmentation tasks based on the encoder-decoder architecture.

Medical image segmentation plays a crucial role in intelligent medical image processing systems, serving as the foundation for effective medical image analysis, particularly in assisting diagnosis and surgical planning. Over the past few years, UNet has achieved tremendous success in the field of image segmentation, with several UNet-based extension models widely applied in medical image segmentation tasks. However, the application of these models is limited to scenarios where large medical equipment can be deployed, such as hospitals. The significant computational costs associated with these segmentation models pose significant challenges when deploying them on portable devices with limited hardware resources. This hinders the realization of rapid and efficient image segmentation in Homelab. In this paper, we present a lightweight model, RGShuffleNet, specifically designed for resource-constrained mobile devices for medical image segmentation. To reduce parameters and computational complexity, we first propose Reshaped Group Convolution, a novel convolutional method for effectively restructuring dimensions of different feature groups. Modifying the feature structure enhances correlations between different groups. Additionally, we introduce the MSC-Shuffle block to facilitate information flow between different feature groups. Unlike traditional Shuffle operations that focus solely on channel correlation, the MSC-Shuffle block proposed in this paper enables information exchange between different groups in both channel and spatial dimensions, thereby achieving superior segmentation performance. Experimental evaluations on two cardiac ultrasound image datasets and one chest CT image dataset demonstrate that RGShuffleNet achieves performance superior to various other state-of-the-art methods while maintaining lower complexity. Finally, RGShuffleNet is deployed on portable devices. The source code of the project is available at https://github.com/Zemin-Cai/RGShuffleNet.

MA-SAM: Modality-agnostic SAM adaptation for 3D medical image segmentation

DSEU-net: A novel deep supervision SEU-net for medical ultrasound image segmentation

Brain tumor segmentation from MRIs has always been a challenging task for radiologists, therefore, an automatic and generalized system to address this task is needed. Among all other deep learning techniques used in medical imaging, U-Net-based variants are the most used models found in the literature to segment medical images with respect to different modalities. Therefore, the goal of this paper is to examine the numerous advancements and innovations in the U-Net architecture, as well as recent trends, with the aim of highlighting the ongoing potential of U-Net being used to better the performance of brain tumor segmentation. Furthermore, we provide a quantitative comparison of different U-Net architectures to highlight the performance and the evolution of this network from an optimization perspective. In addition to that, we have experimented with four U-Net architectures (3D U-Net, Attention U-Net, R2 Attention U-Net, and modified 3D U-Net) on the BraTS 2020 dataset for brain tumor segmentation to provide a better overview of this architecture’s performance in terms of Dice score and Hausdorff distance 95%. Finally, we analyze the limitations and challenges of medical image analysis to provide a critical discussion about the importance of developing new architectures in terms of optimization.

Vision foundation models have shown great potential in improving generalizability and data efficiency, especially for medical image segmentation since medical image datasets are relatively small due to high annotation costs and privacy concerns. However, current research on foundation models predominantly relies on transformers. The high quadratic complexity and large parameter counts make these models computationally expensive, limiting their potential for clinical applications. In this work, we introduce Swin-UMamba†, a novel Mamba-based model for medical image segmentation that seamlessly leverages the power of the vision foundation model, which is also computationally efficient with the linear complexity of Mamba. Moreover, we investigated and verified the impact of the vision foundation model on medical image segmentation, in which a self-supervised model adaptation scheme was designed to bridge the gap between natural and medical data. Notably, Swin-UMamba† outperforms 7 state-of-the-art methods, including CNN-based, transformer-based, and Mamba-based approaches across AbdomenMRI, Encoscopy, and Microscopy datasets. The code and models are publicly available at: https://github.com/JiarunLiu/Swin-UMamba.

Segmentation of breast ultrasound images is a crucial and challenging task in computer-aided diagnosis systems. Accurately segmenting masses in benign and malignant cases and identifying regions with no mass is a primary objective in breast ultrasound image segmentation. Deep learning (DL) has emerged as a powerful tool in medical image segmentation, revolutionizing how medical professionals analyze and interpret complex imaging data. The UNet architecture is a highly regarded and widely used DL model in medical image segmentation. Its distinctive architectural design and exceptional performance have made it popular among researchers. With the increase in data and model complexity, optimization and fine-tuning models play a vital and more challenging role than before. This paper presents a comparative study evaluating the effect of image preprocessing and different optimization techniques and the importance of fine-tuning different UNet segmentation models for breast ultrasound images. Optimization and fine-tuning techniques have been applied to enhance the performance of UNet, Sharp UNet, and Attention UNet. Building upon this progress, we designed a novel approach by combining Sharp UNet and Attention UNet, known as Sharp Attention UNet. Our analysis yielded the following quantitative evaluation metrics for the Sharp Attention UNet: the Dice coefficient, specificity, sensitivity, and F1 score values obtained were 0.93, 0.99, 0.94, and 0.94, respectively. In addition, McNemar’s statistical test was applied to assess significant differences between the approaches. Across a number of measures, our proposed model outperformed all other models, resulting in improved breast lesion segmentation.