Attention based multi-scale parallel network for polyp segmentation

Automatic polyp segmentation from colonoscopy images is an essential step in computer aided diagnosis for colorectal cancer. Most of polyp segmentation methods reported in recent years are based on fully supervised deep learning. However, annotation for polyp images by physicians during the diagnosis is time-consuming and costly. In this paper, we present a novel semi-supervised polyp segmentation via collaborative and adversarial learning of focused and dispersive representations learning model, where focused and dispersive extraction module are used to deal with the diversity of location and shape of polyps. In addition, confidence maps produced by a discriminator in an adversarial training framework shows the effectiveness of leveraging unlabeled data and improving the performance of segmentation network. Consistent regularization is further employed to optimize the segmentation networks to strengthen the representation of the outputs of focused and dispersive extraction module. We also propose an auxiliary adversarial learning method to better leverage unlabeled examples to further improve semantic segmentation accuracy. We conduct extensive experiments on two famous polyp datasets: Kvasir-SEG and CVC-Clinic DB. Experimental results demonstrate the effectiveness of the proposed model, consistently outperforming state-of-the-art semi-supervised segmentation models based on adversarial training and even some advanced fully supervised models.

Unveiling camouflaged and partially occluded colorectal polyps: Introducing CPSNet for accurate colon polyp segmentation

Prevention of colorectal cancer (CRC) by inspecting and removing colorectal polyps has become a global health priority because CRC is one of the most frequent cancers in the world. Although recent U-Net-based convolutional neural networks (CNNs) with deep feature representation and skip connections have shown to segment polyps effectively, U-Net-based approaches still have limitations in modeling explicit global contexts, due to the intrinsic nature locality of convolutional operations. To overcome these problems, this study proposes a novel deep learning model, SwinE-Net, for polyp segmentation that effectively combines a CNN-based EfficientNet and Vision Transformer (ViT)-based Swin Ttransformer. The main challenge is to conduct accurate and robust medical segmentation in maintaining global semantics without sacrificing low-level features of CNNs through Swin Transformer. First, the multidilation convolutional block generates refined feature maps to enhance feature discriminability for multilevel feature maps extracted from CNN and ViT. Then, the multifeature aggregation block creates intermediate side outputs from the refined polyp features for efficient training. Finally, the attentive deconvolutional network-based decoder upsamples the refined and combined feature maps to accurately segment colorectal polyps. We compared the proposed approach with previous state-of-the-art methods by evaluating various metrics using five public datasets (Kvasir, ClinicDB, ColonDB, ETIS, and EndoScene). The comparative evaluation, in particular, proved that the proposed approach showed much better performance in the unseen dataset, which shows the generalization and scalability in conducting polyp segmentation. Furthermore, an ablation study was performed to prove the novelty and advantage of the proposed network. The proposed approach outperformed previous studies.

Colorectal cancer (CRC) caused by polyps has a high mortality rate worldwide. Accurately segmenting the polyp from colonoscopy images is important for the clinical treatment of CRC. The traditional method of polyp segmentation involves the physician manually marking the location of the polyp, resulting in unreliable segmentation results. The complex structure of polyps, the low contrast with mucosal tissue, and the fact that polyp boundary is usually hidden in the background make the task of polyp segmentation extremely challenging. To address these issues, we propose a boundary-aware polyp segmentation network. Specifically, we first propose an attention-aware location module to accurately identify the primary location of polyp. In order to improve the missing polyp portion in the initial region prediction and to mine the polyp boundary hidden in the background, we propose a residual pyramid convolution. Further, we propose a boundary-guided refinement module for more accurate segmentation in order to use the boundary information provided from residual pyramid convolution for constrained polyp region prediction. Extensive experiments show that our proposed network has advantages over existing state-of-the-art methods on five challenging polyp datasets.KeywordsPolyp segmentationContextual attentionBoundary guidanceResidual pyramid convolution

BA-Net: Brightness prior guided attention network for colonic polyp segmentation

Polyp segmentation is essential in reducing the incidence of colorectal cancer by enabling the timely detection and removal of polyps, which are the precursors to most colorectal cancers. Accurate segmentation provides crucial information that aids in the early diagnosis and treatment of the disease. However, this task is challenging due to the considerable variation in polyp texture, color, and size. Additionally, some polyps closely resemble the surrounding healthy tissue in color, making it difficult to distinguish the boundaries between the polyp and its surrounding area. To address these challenges, we propose a deep learning model based on an encoder-decoder architecture. In the encoder section, we utilize a Pyramid Vision Transformer (PVT) to extract robust global feature representations across multiple stages. We introduce a multi-scale fusion module that combines features from different scales, allowing the model to capture diverse information. A custom convolution refinement attention module is employed to further refine the features, and it is integrated with the output of an efficient simple residual path, which preserves essential polyp features. Experiments are conducted on five publicly available polyp datasets. CVC-Clinic and Kvasir-SEG datasets are used for training, while CVC-ColonDB, Etis LaribDB, and CVC-300 datasets are used to evaluate generalization. The proposed approach achieves state-of-the-art performance, with a mean Dice coefficient of 93.9% and a mean Intersection over Union of 89% on CVC-Clinic, and 91.0% and 85.9% on Kvasir-SEG, respectively. Furthermore, our approach demonstrates the ability to adapt to new, unseen polyp features, achieving optimal results on the CVC-ColonDB, Etis LaribDB, and CVC-300 datasets. This adaptability makes it suitable for real clinical scenarios.

Accurate segmentation of colorectal polyps is essential for the early detection and prevention of colorectal cancer, as it directly influences the accuracy of risk assessment, treatment planning, and long-term patient outcomes. Federated learning (FL) provides a privacy-preserving approach to polyp segmentation. However, most existing methods rely on aggregation mechanisms that average the weights across clients, which fail to account for the non-independent and identically distributed (non-IID) data from different clients or the dynamic nature of clinical applications. This paper presents an Adaptive Weight Aggregation Federated Learning Framework (FedDW) for polyp segmentation tasks. Specifically, we first propose a medical value evaluation method based on fuzzy membership mapping, which achieves fair quantification of the contribution differences of different medical institutions through normalization of multidimensional indicators and the application of membership function mapping. Subsequently, we design a clinically-driven dual-path weight fusion mechanism based on real-time contribution membership degrees, which switches between client collaboration and lesion optimization strategies according to clinical needs, ensuring better adaptability to varying clinical scenarios. Finally, we develop a dynamic federated collaborative optimization strategy for global model aggregation, which fairly quantifies the clinical value of local model updates to dynamically generate server-side aggregation weights. It strengthens updates from high-quality clients while suppressing interference from low-quality clients, achieving coordination between the personalization of local models and the enhancement of global model performance. Experimental results demonstrate that the proposed framework significantly outperforms comparative methods in both segmentation accuracy and institutional consistency on polyp datasets, achieving a 2.19% higher Intersection over Union (IoU), a 2.04 mm lower Average Symmetric Surface Distance (ASSD), and a one-point reduction in the standard deviation of IoU compared to baseline approaches.

Objective. Cardiovascular disease (CVD) is a group of diseases affecting cardiac and blood vessels, and short-axis cardiac magnetic resonance (CMR) images are considered the gold standard for the diagnosis and assessment of CVD. In CMR images, accurate segmentation of cardiac structures (e.g. left ventricle) assists in the parametric quantification of cardiac function. However, the dynamic beating of the heart renders the location of the heart with respect to other tissues difficult to resolve, and the myocardium and its surrounding tissues are similar in grayscale. This makes it challenging to accurately segment the cardiac images. Our goal is to develop a more accurate CMR image segmentation approach. Approach. In this study, we propose a regional perception and multi-scale feature fusion network (RMFNet) for CMR image segmentation. We design two regional perception modules, a window selection transformer (WST) module and a grid extraction transformer (GET) module. The WST module introduces a window selection block to adaptively select the window of interest to perceive information, and a windowed transformer block to enhance global information extraction within each feature window. The WST module enhances the network performance by improving the window of interest. The GET module grids the feature maps to decrease the redundant information in the feature maps and enhances the extraction of latent feature information of the network. The RMFNet further introduces a novel multi-scale feature extraction module to improve the ability to retain detailed information. Main results. The RMFNet is validated with experiments on three cardiac data sets. The results show that the RMFNet outperforms other advanced methods in overall performance. The RMFNet is further validated for generalizability on a multi-organ data set. The results also show that the RMFNet surpasses other comparison methods. Significance. Accurate medical image segmentation can reduce the stress of radiologists and play an important role in image-guided clinical procedures.

Automated polyp segmentation from colonoscopy images is crucial for colorectal cancer diagnosis. The accuracy of such segmentation, however, is challenged by two main factors. First, the variability in polyps' size, shape, and color, coupled with the scarcity of well-annotated data due to the need for specialized manual annotation, hampers the efficacy of existing deep learning methods. Second, concealed polyps often blend with adjacent intestinal tissues, leading to poor contrast that challenges segmentation models. Recently, diffusion models have been explored and adapted for polyp segmentation tasks. However, the significant domain gap between RGB-colonoscopy images and grayscale segmentation masks, along with the low efficiency of the diffusion generation process, hinders the practical implementation of these models. To mitigate these challenges, we introduce the Highlighted Diffusion Model Plus (HDM+), a two-stage polyp segmentation framework. This framework incorporates the Highlighted Diffusion Model (HDM) to provide explicit semantic guidance, thereby enhancing segmentation accuracy. In the initial stage, the HDM is trained using highlighted ground-truth data, which emphasizes polyp regions while suppressing the background in the images. This approach reduces the domain gap by focusing on the image itself rather than on the segmentation mask. In the subsequent second stage, we employ the highlighted features from the trained HDM's U-Net model as plug-in priors for polyp segmentation, rather than generating highlighted images, thereby increasing efficiency. Extensive experiments conducted on six polyp segmentation benchmarks demonstrate the effectiveness of our approach.

Automatic and precise polyp segmentation is crucial for the early diagnosis of colorectal cancer. Existing polyp segmentation methods are mostly based on convolutional neural networks (CNNs), which usually utilize the global features to enhance local features through well-designed modules, thereby dealing with the diversity of polyps. Although CNN-based methods achieve impressive results, they are powerless to model explicit long-range relations, which limits their performance. Different from CNN, Transformer has a strong capability of modeling long-range relations owing to self-attention. However, self-attention always spreads attention to unexpected regions and the Transformer’s ability of local feature extraction is insufficient, resulting in inaccurate localization and fuzzy boundary. To address these issues, we propose PPFormer for accurate polyp segmentation. Specifically, we first adopt a shallow CNN encoder and a deep Transformer encoder to extract rich features. In the decoder, we present the PP-guided self-attention that uses prediction maps to guide self-attention to focus on the hard regions so as to enhance the model’s perception of polyp boundary. Meanwhile, the Local-to-Global mechanism is designed to encourage the Transformer to capture more information in the local-window for better polyp localization. Extensive experiments on five challenging datasets show that PPFormer outperforms other advanced methods and achieves state-of-the-art results with six metrics, i.e. mean Dice and mean IoU.KeywordsColorectal cancerPolyp segmentationTransformerLocal-to-Global mechanismPP-guided self-attention

Polyp segmentation is crucial for early colorectal cancer detection, but accurately delineating polyps is challenging due to their variations in size, shape, and texture and low contrast with surrounding tissues. Existing methods often rely solely on spatial-domain processing, which struggles to separate high-frequency features (edges, textures) from low-frequency ones (global structures), leading to suboptimal segmentation performance. We propose the Dynamic Frequency-Decoupled Refinement Network (DFDRNet), a novel segmentation framework that integrates frequency-domain and spatial-domain processing. DFDRNet introduces the Frequency Adaptive Decoupling (FAD) module, which dynamically separates high- and low-frequency components, and the Frequency Adaptive Refinement (FAR) module, which refines these components before fusing them with spatial features to enhance segmentation accuracy. Embedded within a U-shaped encoder-decoder framework, DFDRNet achieves state-of-the-art performance across three benchmark datasets, demonstrating superior robustness and efficiency. Our extensive evaluations and ablation studies confirm the effectiveness of DFDRNet in balancing segmentation accuracy with computational efficiency.

Accurate polyp segmentation from colonoscopy is essential for the early detection of colorectal cancer. However, the variety of polyps manifested in images and the blurry boundary between a polyp and its surrounding mucosa make segmentation challenging. Hence, it is crucial to accurately identify regional boundaries of polyps in colonoscopy. In this paper, we propose Gated Semantic Boundary Network (Gate-SBNet), which is a novel twostream CNN architecture based on an encoder-decoder framework to segment polyps in colonoscopy images. One stream of Gate-SBNet uses a pre-trained ConvNeXt-B model from the image classification task as the semantic encoder to obtain multi-level semantic features from colonoscopy images. Another branch uses the Semantic Boundary Learning Module (SBLM) to learn boundary features based on multi-level semantic features, which process information in parallel with the semantic encoder. By introducing the Gate Convolution Layer (GCLs) into the SBLM module, the semantic information is converted more accurately to boundary information. Therefore, only boundary-related information will be processed by the SBLM. Then, we merge semantic and boundary features as input to an Unet model to obtain the final segmentation result. Our proposed approach was evaluated on five benchmark datasets: Kvasir, CVC-ClinicDB, CVC-ColonDB, CVC-300, and ETIS-LaribPolypDB. Experiments have demonstrated that it is an efficient architecture and capable of making accurate predictions about object boundaries and significantly improving the performance of finding thin and small objects.

Polyp segmentation is crucial in the diagnosis of colorectal cancer. The introduction of the Segment Any-thing Model (SAM) provides powerful pretraining capabilities for polyp segmentation, but it faces two main challenges when applied to endoscopic images: first, its Transformer- based architecture tends to overlook local details, leading to feature bias; second, its performance on out-of-distribution (OOD) data is suboptimal, affecting prediction accuracy and confidence estimation. To address these issues, we propose an enhanced approach based on SAM that incorporates a Local Feature Enhancement Module (LFEM) and a Channel Attention Enhancement Module (CAEM). LFEM improves the capture of high-frequency information by enhancing local detail features, playing a crucial role in extracting polyp edges and textures. CAEM introduces a channel attention mechanism that dynamically adjusts the weight of feature channels, thereby enhancing the model's sensitivity and generalization ability. Additionally, we draw on the design principles of the Cross-Branch Feature Enhancement module and the Uncertainty- Guided Prediction Regularization (UPR) module to further improve SAM's performance in multi-scale feature extraction and on out-of-distribution data. Experimental results demon-strate that the inclusion of LFEM and CAEM significantly improves the model's segmentation accuracy on multiple public polyp datasets, particularly excelling in complex background scenarios.

BCL-Former: Localized Transformer Fusion with Balanced Constraint for polyp image segmentation

Colorectal cancer remains a leading cause of cancer-related deaths worldwide, with early detection and removal of polyps being critical in preventing disease progression. Automated polyp segmentation, particularly in colonoscopy images, is a challenging task due to the variability in polyp appearance and the low contrast between polyps and surrounding tissues. In this work, we propose an edge-enhanced network (EENet) designed to address these challenges by integrating two novel modules: the covariance edge-enhanced attention (CEEA) and cross-scale edge enhancement (CSEE) modules. The CEEA module leverages covariance-based attention to enhance boundary detection, while the CSEE module bridges multi-scale features to preserve fine-grained edge details. To further improve the accuracy of polyp segmentation, we introduce a hybrid loss function that combines cross-entropy loss with edge-aware loss. Extensive experiments show that the EENet achieves a Dice score of 0.9208 and an IoU of 0.8664 on the Kvasir-SEG dataset, surpassing state-of-the-art models such as Polyp-PVT and PraNet. Furthermore, it records a Dice score of 0.9316 and an IoU of 0.8817 on the CVC-ClinicDB dataset, demonstrating its strong potential for clinical application in polyp segmentation. Ablation studies further validate the contribution of the CEEA and CSEE modules.