Accelerating vision foundation model for efficient medical image segmentation.

Deep learning has achieved great success in the field of computer vision, and the precision in image classification and image detection has surpassed humans. Therefore, this paper combines deep learning and medical image segmentation, focusing on how to improve the accuracy and speed of segmentation algorithm of medical exercise rehabilitation image. Aiming at the shortcomings of traditional medical image recognition methods, a medical exercise rehabilitation image segmentation algorithm based on hierarchical features of convolutional neural networks is proposed, this paper calls it as hierarchical features of convolutional neural networks (HFCNN). The algorithm firstly samples the convolution output of multiple layers in the convolutional neural network to a unified scale and combines them to construct a hierarchical feature. This hierarchical feature combines the structural information of objects contained in the shallow layer of the network with the semantic information of objects contained in the deep layers of the network, so it has a strong ability to express. Secondly, the image can be segmented into multiple super pixels by the super pixel segmentation algorithm. The classifier is trained using the hierarchical features of the super pixel, and then the classification result is mapped back to the pixel. Finally, a fully connected conditional random field algorithm including one-potential potential energy and paired potential energy is constructed. The corresponding energy function is used to smooth the classification result of the pixel, and the regional consistency and continuity of the pixel mark are improved. Compared with many classical convolutional neural network algorithms, this algorithm not only accelerates the network convergence speed, shortens the training time, but also significantly improves the accuracy of segmentation algorithm of medical exercise rehabilitation image, showing good practical value.

Objective. Although convolutional neural networks (CNN) and Transformers have performed well in many medical image segmentation tasks, they rely on large amounts of labeled data for training. The annotation of medical image data is expensive and time-consuming, so it is common to use semi-supervised learning methods that use a small amount of labeled data and a large amount of unlabeled data to improve the performance of medical imaging segmentation. Approach. This work aims to enhance the segmentation performance of medical images using a triple-teacher cross-learning semi-supervised medical image segmentation with shape perception and multi-scale consistency regularization. To effectively leverage the information from unlabeled data, we design a multi-scale semi-supervised method for three-teacher cross-learning based on shape perception, called Semi-TMS. The three teacher models engage in cross-learning with each other, where Teacher A and Teacher C utilize a CNN architecture, while Teacher B employs a transformer model. The cross-learning module consisting of Teacher A and Teacher C captures local and global information, generates pseudo-labels, and performs cross-learning using prediction results. Multi-scale consistency regularization is applied separately to the CNN and Transformer to improve accuracy. Furthermore, the low uncertainty output probabilities from Teacher A or Teacher C are utilized as input to Teacher B, enhancing the utilization of prior knowledge and overall segmentation robustness. Main results. Experimental evaluations on two public datasets demonstrate that the proposed method outperforms some existing semi-segmentation models, implicitly capturing shape information and effectively improving the utilization and accuracy of unlabeled data through multi-scale consistency. Significance. With the widespread utilization of medical imaging in clinical diagnosis, our method is expected to be a potential auxiliary tool, assisting clinicians and medical researchers in their diagnoses.

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for various imaging related tasks such as artificial image generation to support AI training. GANs are especially useful for medical imaging-related tasks where training datasets are usually limited in size and heavily imbalanced against the diseased class. We present a systematic review, following the PRISMA guidelines, of recent GAN architectures used for medical image analysis to help the readers in making an informed decision before employing GANs in developing medical image classification and segmentation models. We have extracted 54 papers that highlight the capabilities and application of GANs in medical imaging from January 2015 to August 2020 and inclusion criteria for meta-analysis. Our results show four main architectures of GAN that are used for segmentation or classification in medical imaging. We provide a comprehensive overview of recent trends in the application of GANs in clinical diagnosis through medical image segmentation and classification and ultimately share experiences for task-based GAN implementations.

Medical image segmentation is a critical task in computer-assisted diagnosis and therapy planning. Accurate and efficient fully automated segmentation methods are essential for accelerating decision-making processes. Compared with natural image segmentation methods, current deep learning–based segmentation models for medical imaging face challenges related to data scarcity, inconsistent annotations, and high computational cost. Consequently, this thesis aims to develop efficiency and generalised medical image segmentation models, which are fundamental to the clinical adoption and industrial deployment of this technology in healthcare settings. This thesis systematically addresses these challenges through a series of progressively advanced deep learning models. The foundational work, CTranS, forms a baseline medical image segmentation model that combines a convolutional neural network (CNN) based encoder with a Transformer-based decoder. This initial contribution leverages the strengths of CNNs for local feature extraction and Transformers for global contextual understanding, achieving state-of-the-art (SOTA) performance. Subsequently, to enhance the generalisability, MO-CTranS is introduced, which is capable of learning from a mixture of multiple partially labelled datasets. MO-CTranS employs a CNN encoder and Transformer decoder with task-specific tokens to handle label inconsistencies and class imbalances, tackling the challenges of training a single unified model from fragmented and heterogeneously annotated medical data. This innovation moves beyond single-task, fully-annotated model training towards a more scalable and clinically realistic approach. Despite the superior performance of CTranS and MO-CTranS, the computational efficiency of the Transformer blocks hinders their clinical application. Thus, CRFTrans is proposed to enhance the model efficiency. As a novel recursive Vision Transformer layer designed to replace the commonly used cascaded Transformer layers, this approach represents a fundamental shift from simply stacking layers to a recursive refinement process, demonstrating that comparable performance can be achieved with significantly reduced computational complexity, thus facilitating deployment in resource-constrained clinical environments. However, these methods still depend on extensive manual annotation, which remains challenging and time-consuming in clinical practice. Thus, to further enhance both efficiency and generality, a novel semi-supervised medical image segmentation framework, referred to as SSL-MedSAM2, has been developed. SSL-MedSAM2 integrates a training-free few-shot segmentation branch for pseudo labelling and a fully supervised learning branch for label refinement. This design allows the model to leverage a small fraction of labelled images together with a large unlabelled set, dramatically decreasing the annotation effort. Together, these contributions result in deep learning models that are significantly more data-efficient and broadly applicable than conventional approaches. The overall impact of this work is to facilitate the reliable and scalable deployment of segmentation models in real-world clinical settings by bridging the gap among accuracy, efficiency, and generalisability.

Convolutional neural networks (CNNs), as a typical deep learning technique, have been widely used in image segmentation, but they often require a large amount of annotated data. However, the number of available pixel-wise labeled medical images is extremely small, and this prevents the application of CNNs in many medical image segmentation tasks. We proposed a self-paced learning with diversity SPLD) framework to boost the performances of medical image segmentation models with a limited amount of annotated data. Self-paced learning (SPL) is a learning regime that selects training samples in order from the easiest to the most difficult for model training. In addition, we took the diversity of the data into consideration. The proposed self-paced learning with diversity by query-by-committee (SPLD-QBC) algorithm dynamically and diversely selects the appropriate training data to boost the performance of an image segmentation model. SPLD-QBC incorporates the query-by-committee (QBC) technique for data selection and affinity propagation for optimizing the data diversity. By dynamically selecting the optimal sequence of training samples from different probability distributions, the segmentation models achieved improved performances. To verify the effectiveness of the proposed SPLD-QBC framework, we conducted experiments on three medical image segmentation tasks with five different datasets. The experimental results indicated that the proposed SPLD-QBC algorithm significantly improved upon the segmentation performances of the baseline models and resulted in a higher Dice score, surface distance and mean intersection over union (mIoU). The proposed SPLD significantly boosts the segmentation performances of models and is easily embedded into CNN-based image segmentation models.

Medical image segmentation is an essential process in disease analysis and diagnosis, in which medical images are divided into specific areas of anatomic or pathological structures. However, this process has challenges, such as the quality, variability, and complexity of medical images. Thus, we propose a medical image segmentation method using the Convolutional Neural Network (CNN) and the Histogram Equalization (HE) image preprocessing. We evaluated medical image segmentation using 2 secondary datasets, namely Lung CT-Scan and Chest X-Rays, with 267 and 3616 images, respectively. The HE method uses an averaging of the optimal cumulative distribution function (CDF) values with a range of 0 to 39. The experimental results show that adding the HE method can increase the accuracy of CNN on the Lung CT-Scan and Chest X-rays data sets, respectively, by 0.73 percentage points from 91.28% to 92.08 % and 1.23 percentage points from 95.56% to 96.85% for the training process. The results of the testing process (validation) have an increase in accuracy of 2.82 percentage points from 89.77% to 92.59% and 0.91 percentage points from 96.06% to 96.97%. Implementing the HE method increases accuracy in CNN medical image segmentation based on the value of dice similarity coefficient (DSC) and structural similarity index measurement (SSIM) close to 0.95.

Many current and state-of-the-art deep learning models for accurate image segmentation are based on the U-Net architecture, a convolutional neural network designed for biomedical applications. Despite its widespread adoption in the medical imaging community, U-Net has two major limitations. First, due to its deep structure and the large number of filters used, the number of parameters and Floating-Point Operations Per Second (FLOPS) are high. This results in high computational complexity and demands a large memory size, making real-time implementation and deployment of U-Net models challenging. Second, the base U-Net model only uses a single type of kernel (i.e., 3×3) throughout the network for all convolution operations. Feature extraction using a single spatial extent throughout the network is suitable if the size, location, and shape of the salient regions remain static in all the images of a dataset, which is not necessarily the case in medical imaging. To address these two limitations, we propose an Efficient Multi-Encoder-Decoder based UNet (EMED-UNet), a novel architecture for efficient medical image segmentation. We evaluated our network on four medical imaging datasets: Montgomery County, Shenzhen CXR, COVID-19 CT LS, and BraTS (Brain Tumor Segmentation) dataset. EMED-UNet outperforms the U-Net and its variants in terms of accuracy, with around 77% reduction in parameters, a 60% reduction in FLOPS, and a 79.2% reduction in memory usage (all as compared to U-Net). The results demonstrate that EMED-UNet is a lightweight and accurate model for image segmentation that substantially improves upon the U-Net base model and is more feasible to deploy, given its decreased computational cost.

Cold SegDiffusion: A novel diffusion model for medical image segmentation

DSEU-net: A novel deep supervision SEU-net for medical ultrasound image segmentation

Medical image segmentation is a critical application of computer vision in the analysis of medical images. Its primary objective is to isolate regions of interest in medical images from the background, thereby assisting clinicians in accurately identifying lesions, their sizes, locations, and their relationships with surrounding tissues. However, compared to natural images, medical images present unique challenges, such as low resolution, poor contrast, inconsistency, and scattered target regions. Furthermore, the accuracy and stability of segmentation results are subject to more stringent requirements. In recent years, with the widespread application of Convolutional Neural Networks (CNNs) in computer vision, deep learning-based methods for medical image segmentation have become a focal point of research. This paper categorizes, reviews, and summarizes the current representative methods and research status in the field of medical image segmentation. A comparative analysis of relevant experiments is presented, along with an introduction to commonly used public datasets, performance evaluation metrics, and loss functions in medical image segmentation. Finally, potential future research directions and development trends in this field are predicted and analyzed.

Bridging 2D and 3D segmentation networks for computation-efficient volumetric medical image segmentation: An empirical study of 2.5D solutions.

Explainable multi-module semantic guided attention based network for medical image segmentation

In medical image segmentation, U-Net and its variants are widely utilized. However, CNN-based models are limited in long-range dependency modeling, while Transformer-based models are constrained by quadratic computational complexity. Recently, state space models (SSMs) such as Mamba have emerged as a promising solution, as they excel at modeling long-range interactions while maintaining linear computational complexity. This paper proposes the CIHM, a hybrid Mamba model that provides contextual insights. Its key component is the context-insight Mamba-CNN layer (CiMC layer). The CiMC layer adopts a hybrid Mamba-CNN approach to process features across different scales and levels in parallel, thereby enhancing the semantic segmentation of medical images. Specifically, the SSM branch of this layer captures long-range dependencies with comprehensive contextual semantic understanding, while the CNN block achieves superior local feature extraction using T-shaped convolution that focuses on patch centers. Moreover, to further aggregate multi-scale contextual information of medical images, we design the multiscale refining detail bridge (MRDB) module. This module significantly improves feature representation through the dense multiplication and concatenation of features for fusion. Comprehensive experiments on the ISIC2017, ISIC2018, Synapse, and DSB18 datasets demonstrate that CIHM exhibits strong competitive performance and efficiency in medical image segmentation tasks. Notably, CIHM achieves better segmentation performance than the well-known U-Net and the modern hybrid network U-Mamba, while reducing the parameter count by 40 times and 345 times, respectively. This demonstrates the potential of our approach to advance model lightweighting.

CASF-Net: Cross-attention and cross-scale fusion network for medical image segmentation

Segmentation on medical image requires good quality due to affect the interpretation and diagnosis of medical experts. On medical image segmentation, there is merging phase to increase the quality of the segmentation result. However, stopping criteria on merging phase was determined manually by medical experts. It implied the subjectivity of segmentation result. To increase the objectivity of segmentation result, a method to automate merging phase on medical image segmentation is required. Therefore, we propose a novel method on medical image segmentation which combine two-stage SOM and T-cluster method. Experiments were performed on dental panoramic as medical image sample and evaluated by using segmentation quality formula. Experiments show that the proposed method can perform segmentation on dental panoramic image automatically and objectively with the best average of segmentation quality value is 4,40.
 Index Terms—dental panoramic image, image segmentation, medical image, Self-Organizing Map, T-cluster