CIHM: context-insight hybrid Mamba for efficient medical image segmentation

Medical image segmentation is a fundamental task in medical image analysis, playing a crucial role in disease diagnosis, treatment planning, and clinical decision-making. Accurate segmentation of anatomical structures, such as blood vessels, organs, and lesions, is essential for reliable medical interpretations. However, existing segmentation models often face challenges in effectively capturing both local and global features within medical images. Traditional convolutional neural networks (CNNs) excel at extracting fine-grained local features but struggle to model long-range dependencies due to their limited receptive fields. Conversely, transformer-based models can capture global contextual relationships but often fail to preserve detailed local structures, leading to suboptimal segmentation performance. The purpose of this study is to develop SEFormer, a novel hybrid network that integrates SENet, ResNet, and Transformer to enhance medical image segmentation. SEFormer aims to effectively capture both local and global features, addressing the limitations of traditional CNNs and transformer-based models.SENet is incorporated to recalibrate feature maps by adaptively emphasizing informative features, thereby compensating for the global representation limitations of CNNs. Meanwhile, the ResNet backbone ensures deep feature extraction while maintaining computational efficiency. Additionally, the Transformer module is introduced to capture long-range dependencies and enhance contextual awareness, complementing the local feature extraction capabilities of CNNs. This hybrid approach allows SEFormer to balance fine-grained spatial details with broader contextual information, leading to more precise segmentation results.Furthermore, to mitigate information loss during feature extraction and ensure a more complete and hierarchical representation of image information, we introduce a feature pyramid structure inspired by multi-scale image pyramid models. By progressively increasing the receptive field across different scales and employing skip connections, SEFormer effectively fuses multi-scale local and global features at each stage of the pyramid. This hierarchical fusion mechanism ensures a richer and more robust feature representation, which is particularly beneficial for segmenting complex medical structures such as blood vessels. We evaluate SEFormer on the CHASE_DB1 dataset, a widely used benchmark for retinal vessel segmentation. Experimental results demonstrate that SEFormer outperforms existing state-of-the-art segmentation methods, achieving a 3.25% improvement in segmentation accuracy. Additionally, we conduct ablation studies to verify the contributions of different components within SEFormer. The results show that incorporating SENet enhances feature recalibration and channel attention, while the Transformer module significantly improves global context awareness. The feature pyramid structure further contributes to performance gains by ensuring a multi-scale representation of vascular structures. Compared to conventional CNN-based methods, our approach achieves better segmentation accuracy with improved robustness to variations in vessel thickness, noise, and image contrast. Furthermore, SEFormer maintains computational efficiency, making it suitable for real-world clinical applications where both accuracy and processing speed are critical. SEFormer provides an efficient and accurate approach for medical image segmentation by leveraging the complementary strengths of CNNs, SENet, and Transformers while integrating a feature pyramid structure to maximize feature representation. Future research directions include extending SEFormer to other medical imaging modalities, such as CT and MRI scans, as well as exploring further optimization techniques to reduce computational costs while maintaining high segmentation performance. Additionally, integrating SEFormer with active learning frameworks could further improve segmentation performance by leveraging expert-annotated data more efficiently.

ABSTRACTSemantic segmentation is a key task in computer vision, with medical image segmentation as a prominent downstream application that has seen significant advancements in recent years. However, the challenge of requiring extensive annotations in medical image segmentation remains exceedingly difficult. In addressing this issue, semi‐supervised semantic segmentation has emerged as a new approach to mitigate annotation burdens. Nonetheless, existing methods in semi‐supervised medical image segmentation still face challenges in fully exploiting unlabeled data and efficiently integrating labeled and unlabeled data. Therefore, this paper proposes a novel network model—feature similarity multilevel information fusion network (FSMIFNet). First, the feature similarity module is introduced to harness deep feature similarity among unlabeled images, predicting true label constraints and guiding segmentation features with deep feature relationships. This approach fully exploits deep feature information from unlabeled data. Second, the multilevel information fusion framework integrates labeled and unlabeled data to enhance segmentation quality in unlabeled images, ensuring consistency between original and feature maps for comprehensive optimization of detail and global information. In the ACDC dataset, our method achieves an mDice of 0.684 with 5% labeled data, 0.873 with 10%, 0.884 with 20%, and 0.897 with 50%. Experimental results demonstrate the effectiveness of FSMIFNet in semi‐supervised semantic segmentation of medical images, outperforming existing methods on public benchmark datasets. The code and models are available at https://github.com/liujiayin12/FSMIFNet.git.

Semi-supervised medical image segmentation via feature similarity and reliable-region enhancement

Medical Image Semantic Segmentation (MISS), the process of assigning a semantic label to each pixel in an image, is a foundational task in computational medicine, critical for quantitative diagnostics and treatment planning. However, developing robust MISS models faces two intertwined challenges. First, there is an architectural dilemma: Convolutional Neural Networks (CNNs), like U-Net, excel at learning local features but are limited by their receptive fields, failing to capture global context essential for segmenting organs with large deformations. Conversely, Vision Transformers (ViTs) effectively model long-range dependencies but lack the inductive biases of CNNs, leading to poor generalization on the small datasets typical in medicine without extensive pre-training. Second, the prohibitive cost and expertise required for creating pixel-level annotations create a severe data scarcity bottleneck. While Semi-Supervised Learning (SSL) aims to mitigate this by leveraging unlabeled data, existing methods often fail to learn high-level semantic relations and are susceptible to confirmation bias from noisy pseudo-labels, class imbalance, and suboptimal contrastive sample selection. This thesis presents a comprehensive investigation to systematically address these challenges, delivering a cohesive suite of novel deep learning frameworks. The contributions are four-fold: First, to resolve the architectural trade-off, this work introduces CS-Unet, a pure Transformer network built upon a U-Net-like architecture. Its core innovation is the Convolutional Swin Transformer (CST) block, which integrates convolutions directly within the Multi-Head Self-Attention and Feed-Forward Network modules. This design imbues the Transformer with inherent localized spatial context and strong inductive biases, enabling it to efficiently learn both local and global features. Without pre-training, CS-Unet outperforms existing Transformer and CNN-based models on multi-organ and cardiac datasets, achieving state-of-the-art performance with fewer parameters. Second, to address data scarcity, a novel Multi-Scale Cross Supervised Contrastive Learning (MCSC) framework for SSL is developed. MCSC jointly trains CNN and Transformer models, using a cross-teaching paradigm where each network provides pseudo-labels for the other. Crucially, it moves beyond simple output consistency by applying a contrastive loss to feature maps at multiple scales, enforcing hierarchical semantic consistency. To handle the class imbalance endemic to medical imaging, a class-prevalence-aware loss is used to ensure features for infrequent classes are learned robustly. Third, to fortify SSL against noisy pseudo-labels, a certainty-guided contrastive learning strategy is proposed. This approach mitigates the impact of inaccurate pseudo-labels by using a certainty metric to guide the selection of samples for contrastive learning. The framework’s computational efficiency is enhanced through novel sampling strategies that select a few representative samples for contrasting, and a negative memory bank is used to increase sample diversity and eliminate dependence on batch size. Fourth, this thesis introduces a new paradigm for SSL by leveraging external anatomical priors through the Contrastive Cross-Teaching with Registration (CCT-R) framework. CCT-R is the first method to integrate spatial registration transforms into the learning process. It features two novel modules: a Registration Supervision Loss (RSL), which uses transforms between labeled and unlabeled volumes to generate an additional, highly reliable source of pseudo-labels, and Registration-Enhanced Positive Sampling (REPS), which uses registration to identify anatomically-corresponding positive pairs across volumes for contrastive learning. Overall, these contributions provide a powerful toolkit that significantly alleviates the annotation bottleneck in medical AI. The proposed methods demonstrate state-of-the-art performance on challenging segmentation benchmarks, delivering a pathway to develop accurate, data-efficient models for real-world clinical applications and opening new avenues for research into fusing geometric priors with semantic segmentation.

Semantic segmentation of medical images provides an important cornerstone for subsequent tasks of image analysis and understanding. With rapid advancements in deep learning methods, conventional U-Net segmentation networks have been applied in many fields. Based on exploratory experiments, features at multiple scales have been found to be of great importance for the segmentation of medical images. In this paper, we propose a scale-attention deep learning network (SA-Net), which extracts features of different scales in a residual module and uses an attention module to enforce the scale-attention capability. SA-Net can better learn the multi-scale features and achieve more accurate segmentation for different medical image. In addition, this work validates the proposed method across multiple datasets. The experiment results show SA-Net achieves excellent performances in the applications of vessel detection in retinal images, lung segmentation, artery/vein(A/V) classification in retinal images and blastocyst segmentation. To facilitate SA-Net utilization by the scientific community, the code implementation will be made publicly available.

Medical image segmentation is a fundamental task in medical image analysis and has been widely applied in multiple medical fields. The latest transformer-based deep learning segmentation model, Segment Anything Model (SAM), has demonstrated outstanding performance in natural image segmentation tasks through large-scale pre-training, achieving zero-shot image semantic understanding and pixel-level segmentation. However, medical images present challenges such as style variability, ill-defined object boundaries, and feature ambiguities, limiting the direct applicability of the SAM to medical image segmentation tasks. To enhance the robustness of the SAM in the domain of medical segmentation, we propose the SAM-RCCF framework. This approach aims to enhance the generalizability and precision of segmentation performance across diverse intracranial tumor types, including gliomas, metastatic tumors, and meningiomas. The study collected 484 axial T1-weighted contrast-enhanced (T1CE) magnetic resonance imaging (MRI) data of brain tumor patients, including 164 cases of glioma, 158 cases of metastatic tumors, and 162 cases of meningioma. All imaging data were randomly divided into training and testing sets. We employed the proposed SAM-RCCF model to perform segmentation experiments on these data, and five-fold cross-validation was adopted to evaluate the model's performance. This framework integrates the RefineNet module and the conditional control field with a conditional controller and Mask generator, enabling precise feature recognition and tailored segmentation for medical images, optimizing segmentation accuracy RESULTS: In the glioma segmentation experiment, the SAM-RCCF model achieved outstanding performance with an IOU of 0.90, DSC of 0.912, and HD of 13.13. For the meningioma segmentation task, it obtained an IOU of 0.9214, DSC of 0.93, and HD of 11.41, significantly outperforming other classic segmentation models. The segmentation experiment results demonstrate that in the segmentation tasks of glioma, metastatic tumors, and meningioma MRI images, the SAM-RCCF algorithm significantly outperformed the original SAM in terms of DSC, HD, and IOU segmentation performance metrics. The experimental results verify the effectiveness of the SAM-RCCF framework in segmenting complex and variable brain tumor images, enhancing segmentation accuracy and robustness.

Medical image segmentation plays a crucial role in intelligent medical image processing systems, serving as the foundation for effective medical image analysis, particularly in assisting diagnosis and surgical planning. Over the past few years, UNet has achieved tremendous success in the field of image segmentation, with several UNet-based extension models widely applied in medical image segmentation tasks. However, the application of these models is limited to scenarios where large medical equipment can be deployed, such as hospitals. The significant computational costs associated with these segmentation models pose significant challenges when deploying them on portable devices with limited hardware resources. This hinders the realization of rapid and efficient image segmentation in Homelab. In this paper, we present a lightweight model, RGShuffleNet, specifically designed for resource-constrained mobile devices for medical image segmentation. To reduce parameters and computational complexity, we first propose Reshaped Group Convolution, a novel convolutional method for effectively restructuring dimensions of different feature groups. Modifying the feature structure enhances correlations between different groups. Additionally, we introduce the MSC-Shuffle block to facilitate information flow between different feature groups. Unlike traditional Shuffle operations that focus solely on channel correlation, the MSC-Shuffle block proposed in this paper enables information exchange between different groups in both channel and spatial dimensions, thereby achieving superior segmentation performance. Experimental evaluations on two cardiac ultrasound image datasets and one chest CT image dataset demonstrate that RGShuffleNet achieves performance superior to various other state-of-the-art methods while maintaining lower complexity. Finally, RGShuffleNet is deployed on portable devices. The source code of the project is available at https://github.com/Zemin-Cai/RGShuffleNet.

In recent years, deep learning has achieved good results in the semantic segmentation of medical images. A typical architecture for segmentation networks is an encoder-decoder structure. However, the design of the segmentation networks is fragmented and lacks a mathematical explanation. Consequently, segmentation networks are inefficient and less generalizable across different organs. To solve these problems, we reconstructed the segmentation network based on mathematical methods. We introduced the dynamical systems view into semantic segmentation and proposed a novel segmentation network based on Runge-Kutta methods, referred to hereafter as the Runge-Kutta segmentation network (RKSeg). RKSegs were evaluated on ten organ image datasets from the Medical Segmentation Decathlon. The experimental results show that RKSegs far outperform other segmentation networks. RKSegs use few parameters and short inference time, yet they can achieve competitive or even better segmentation results compared to other models. RKSegs pioneer a new architectural design pattern for segmentation networks.

Objective. Although convolutional neural networks (CNN) and Transformers have performed well in many medical image segmentation tasks, they rely on large amounts of labeled data for training. The annotation of medical image data is expensive and time-consuming, so it is common to use semi-supervised learning methods that use a small amount of labeled data and a large amount of unlabeled data to improve the performance of medical imaging segmentation. Approach. This work aims to enhance the segmentation performance of medical images using a triple-teacher cross-learning semi-supervised medical image segmentation with shape perception and multi-scale consistency regularization. To effectively leverage the information from unlabeled data, we design a multi-scale semi-supervised method for three-teacher cross-learning based on shape perception, called Semi-TMS. The three teacher models engage in cross-learning with each other, where Teacher A and Teacher C utilize a CNN architecture, while Teacher B employs a transformer model. The cross-learning module consisting of Teacher A and Teacher C captures local and global information, generates pseudo-labels, and performs cross-learning using prediction results. Multi-scale consistency regularization is applied separately to the CNN and Transformer to improve accuracy. Furthermore, the low uncertainty output probabilities from Teacher A or Teacher C are utilized as input to Teacher B, enhancing the utilization of prior knowledge and overall segmentation robustness. Main results. Experimental evaluations on two public datasets demonstrate that the proposed method outperforms some existing semi-segmentation models, implicitly capturing shape information and effectively improving the utilization and accuracy of unlabeled data through multi-scale consistency. Significance. With the widespread utilization of medical imaging in clinical diagnosis, our method is expected to be a potential auxiliary tool, assisting clinicians and medical researchers in their diagnoses.

Deep learning has achieved great success in the field of computer vision, and the precision in image classification and image detection has surpassed humans. Therefore, this paper combines deep learning and medical image segmentation, focusing on how to improve the accuracy and speed of segmentation algorithm of medical exercise rehabilitation image. Aiming at the shortcomings of traditional medical image recognition methods, a medical exercise rehabilitation image segmentation algorithm based on hierarchical features of convolutional neural networks is proposed, this paper calls it as hierarchical features of convolutional neural networks (HFCNN). The algorithm firstly samples the convolution output of multiple layers in the convolutional neural network to a unified scale and combines them to construct a hierarchical feature. This hierarchical feature combines the structural information of objects contained in the shallow layer of the network with the semantic information of objects contained in the deep layers of the network, so it has a strong ability to express. Secondly, the image can be segmented into multiple super pixels by the super pixel segmentation algorithm. The classifier is trained using the hierarchical features of the super pixel, and then the classification result is mapped back to the pixel. Finally, a fully connected conditional random field algorithm including one-potential potential energy and paired potential energy is constructed. The corresponding energy function is used to smooth the classification result of the pixel, and the regional consistency and continuity of the pixel mark are improved. Compared with many classical convolutional neural network algorithms, this algorithm not only accelerates the network convergence speed, shortens the training time, but also significantly improves the accuracy of segmentation algorithm of medical exercise rehabilitation image, showing good practical value.

DSEU-net: A novel deep supervision SEU-net for medical ultrasound image segmentation

Ultrasound imaging can distinctly display the morphology and structure of internal organs within the human body, enabling the examination of organs like the breast, liver, and thyroid. It can identify the locations of tumors, nodules, and other lesions, thereby serving as an efficacious tool for treatment detection and rehabilitation evaluation. Typically, the attending physician is required to manually demarcate the boundaries of lesion locations, such as tumors, in ultrasound images. Nevertheless, several issues exist. The high noise level in ultrasound images, the degradation of image quality due to the impact of surrounding tissues, and the influence of the operator’s experience and proficiency on the determination of lesion locations can all contribute to a reduction in the accuracy of delineating the boundaries of lesion sites. In the wake of the advancement of deep learning, its application in medical image segmentation is becoming increasingly prevalent. For instance, while the U-Net model has demonstrated a favorable performance in medical image segmentation, the convolution layers of the traditional U-Net model are relatively simplistic, leading to suboptimal extraction of global information. Moreover, due to the significant noise present in ultrasound images, the model is prone to interference. In this research, we propose an Attention Residual Network model (ARU-Net). By incorporating residual connections within the encoder section, the learning capacity of the model is enhanced. Additionally, a spatial hybrid convolution module is integrated to augment the model’s ability to extract global information and deepen the vertical architecture of the network. During the feature fusion stage of the skip connections, a channel attention mechanism and a multi-convolutional self-attention mechanism are respectively introduced to suppress noisy points within the fused feature maps, enabling the model to acquire more information regarding the target region. Finally, the predictive efficacy of the model was evaluated using publicly accessible breast ultrasound and thyroid ultrasound data. The ARU-Net achieved mean Intersection over Union (mIoU) values of 82.59% and 84.88%, accuracy values of 97.53% and 96.09%, and F1-score values of 90.06% and 89.7% for breast and thyroid ultrasound, respectively.

Ubiquitous accumulation of large volumes of data, and increased availability of annotated medical data in particular, has made it possible to show the many and varied benefits of deep learning to the semantic segmentation of medical images. Nevertheless, data access and annotation come at a high cost in clinician time. The power of Vision Transformer (ViT) is well-documented for generic computer vision tasks involving millions of images of every day objects, of which only relatively few have been annotated. Its translation to relatively more modest (i.e. thousands of images of) medical data is not immediately straightforward. This paper presents practical avenues for training a Computationally-Efficient Semi-Supervised Vision Transformer (CESS-ViT) for medical image segmentation task.We propose a self-attention-based image segmentation network which requires only limited computational resources. Additionally, we develop a dual pseudo-label supervision scheme for use with semi-supervision in a simple pure ViT.Our method has been evaluated on a publicly available cardiac MRI dataset with direct comparison against other semi-supervised methods. Our results illustrate the proposed ViT-based semi-supervised method outperforms the existing methods in the semantic segmentation of cardiac ventricles.

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for various imaging related tasks such as artificial image generation to support AI training. GANs are especially useful for medical imaging-related tasks where training datasets are usually limited in size and heavily imbalanced against the diseased class. We present a systematic review, following the PRISMA guidelines, of recent GAN architectures used for medical image analysis to help the readers in making an informed decision before employing GANs in developing medical image classification and segmentation models. We have extracted 54 papers that highlight the capabilities and application of GANs in medical imaging from January 2015 to August 2020 and inclusion criteria for meta-analysis. Our results show four main architectures of GAN that are used for segmentation or classification in medical imaging. We provide a comprehensive overview of recent trends in the application of GANs in clinical diagnosis through medical image segmentation and classification and ultimately share experiences for task-based GAN implementations.

Cold SegDiffusion: A novel diffusion model for medical image segmentation