BEGA-UNet: Boundary-Explicit Guided Attention U-Net with Multi-Scale Feature Aggregation for Colonoscopic Polyp Segmentation

Automated polyp segmentation from colonoscopy images is crucial for colorectal cancer diagnosis. The accuracy of such segmentation, however, is challenged by two main factors. First, the variability in polyps' size, shape, and color, coupled with the scarcity of well-annotated data due to the need for specialized manual annotation, hampers the efficacy of existing deep learning methods. Second, concealed polyps often blend with adjacent intestinal tissues, leading to poor contrast that challenges segmentation models. Recently, diffusion models have been explored and adapted for polyp segmentation tasks. However, the significant domain gap between RGB-colonoscopy images and grayscale segmentation masks, along with the low efficiency of the diffusion generation process, hinders the practical implementation of these models. To mitigate these challenges, we introduce the Highlighted Diffusion Model Plus (HDM+), a two-stage polyp segmentation framework. This framework incorporates the Highlighted Diffusion Model (HDM) to provide explicit semantic guidance, thereby enhancing segmentation accuracy. In the initial stage, the HDM is trained using highlighted ground-truth data, which emphasizes polyp regions while suppressing the background in the images. This approach reduces the domain gap by focusing on the image itself rather than on the segmentation mask. In the subsequent second stage, we employ the highlighted features from the trained HDM's U-Net model as plug-in priors for polyp segmentation, rather than generating highlighted images, thereby increasing efficiency. Extensive experiments conducted on six polyp segmentation benchmarks demonstrate the effectiveness of our approach.

Prevention of colorectal cancer (CRC) by inspecting and removing colorectal polyps has become a global health priority because CRC is one of the most frequent cancers in the world. Although recent U-Net-based convolutional neural networks (CNNs) with deep feature representation and skip connections have shown to segment polyps effectively, U-Net-based approaches still have limitations in modeling explicit global contexts, due to the intrinsic nature locality of convolutional operations. To overcome these problems, this study proposes a novel deep learning model, SwinE-Net, for polyp segmentation that effectively combines a CNN-based EfficientNet and Vision Transformer (ViT)-based Swin Ttransformer. The main challenge is to conduct accurate and robust medical segmentation in maintaining global semantics without sacrificing low-level features of CNNs through Swin Transformer. First, the multidilation convolutional block generates refined feature maps to enhance feature discriminability for multilevel feature maps extracted from CNN and ViT. Then, the multifeature aggregation block creates intermediate side outputs from the refined polyp features for efficient training. Finally, the attentive deconvolutional network-based decoder upsamples the refined and combined feature maps to accurately segment colorectal polyps. We compared the proposed approach with previous state-of-the-art methods by evaluating various metrics using five public datasets (Kvasir, ClinicDB, ColonDB, ETIS, and EndoScene). The comparative evaluation, in particular, proved that the proposed approach showed much better performance in the unseen dataset, which shows the generalization and scalability in conducting polyp segmentation. Furthermore, an ablation study was performed to prove the novelty and advantage of the proposed network. The proposed approach outperformed previous studies.

A zero-shot industrial process fault diagnosis method based on domain-shift constraints

Colorectal cancer remains a leading cause of cancer-related deaths worldwide, with early detection and removal of polyps being critical in preventing disease progression. Automated polyp segmentation, particularly in colonoscopy images, is a challenging task due to the variability in polyp appearance and the low contrast between polyps and surrounding tissues. In this work, we propose an edge-enhanced network (EENet) designed to address these challenges by integrating two novel modules: the covariance edge-enhanced attention (CEEA) and cross-scale edge enhancement (CSEE) modules. The CEEA module leverages covariance-based attention to enhance boundary detection, while the CSEE module bridges multi-scale features to preserve fine-grained edge details. To further improve the accuracy of polyp segmentation, we introduce a hybrid loss function that combines cross-entropy loss with edge-aware loss. Extensive experiments show that the EENet achieves a Dice score of 0.9208 and an IoU of 0.8664 on the Kvasir-SEG dataset, surpassing state-of-the-art models such as Polyp-PVT and PraNet. Furthermore, it records a Dice score of 0.9316 and an IoU of 0.8817 on the CVC-ClinicDB dataset, demonstrating its strong potential for clinical application in polyp segmentation. Ablation studies further validate the contribution of the CEEA and CSEE modules.

Early diagnosis is essential for the successful treatment of bowel cancers including colorectal cancer (CRC), and capsule endoscopic imaging with robotic actuation can be a valuable diagnostic tool when combined with automated image analysis. We present a deep learning rooted detection and segmentation framework for recognizing lesions in colonoscopy and capsule endoscopy images. We restructure established convolution architectures, such as VGG and ResNets, by converting them into fully-connected convolution networks (FCNs), fine-tune them and study their capabilities for polyp segmentation and detection. We additionally use shape-from-shading (SfS) to recover depth and provide a richer representation of the tissue’s structure in colonoscopy images. Depth is incorporated into our network models as an additional input channel to the RGB information and we demonstrate that the resulting network yields improved performance. Our networks are tested on publicly available datasets and the most accurate segmentation model achieved a mean segmentation interception over union (IU) of 47.78% and 56.95% on the ETIS-Larib and CVC-Colon datasets, respectively. For polyp detection, the top performing models we propose surpass the current state-of-the-art with detection recalls superior to 90% for all datasets tested. To our knowledge, we present the first work to use FCNs for polyp segmentation in addition to proposing a novel combination of SfS and RGB that boosts performance.

Single image deraining is a fundamental pre-processing step in many computer vision applications for improving the visual effect and system performance of the downstream high-level tasks in adverse weather conditions. This study proposes a novel multi-scale context aggregation network, to effectively solve the single image deraining problem. Specifically, we exploit a lightweight residual structure subnet as the baseline architecture to extract fine and detailed texture context at the original scale and further incorporate a multi-scale progressive aggregation module (MPAM) to learn the complementary high-level context for enhancing the modeling capability of the overall deraining network. The MPAM, designed as a plug-and-play module to be utilized in the arbitrary network, is composed of multi-scale convolution blocks to learn a wide variety of contexts in multiple receptive fields, and then carries out progressive context aggregation between adjacent scales with residual connections, which is expected to concurrently disentangle the multi-scale structures of scene contents and multiple rain layers in the rainy images, and models more representative contexts for reconstructing the clean image. To reduce the learnable parameters in the MPAM, we further explore a context hallucinate block for replacing the multi-scale convolution block, and propose a lightweight MPAM. Moreover, for being specially adaptive to deal with the input rainy images with a lot of unwanted components (rain layers), we delve into the artifact-attenuating pooling and activation functions via taking into consideration of the surrounding spatial context instead of pixel-wise operation and propose the spatial context-aware pooling (SCAP) and activation (SCAA) for incorporating with our deraining network to boost performance. Extensive experiments on the benchmark datasets demonstrate that our proposed method performs favorably against state-of-the-art deraining approaches.

Accurate polyp segmentation from colonoscopy is essential for the early detection of colorectal cancer. However, the variety of polyps manifested in images and the blurry boundary between a polyp and its surrounding mucosa make segmentation challenging. Hence, it is crucial to accurately identify regional boundaries of polyps in colonoscopy. In this paper, we propose Gated Semantic Boundary Network (Gate-SBNet), which is a novel twostream CNN architecture based on an encoder-decoder framework to segment polyps in colonoscopy images. One stream of Gate-SBNet uses a pre-trained ConvNeXt-B model from the image classification task as the semantic encoder to obtain multi-level semantic features from colonoscopy images. Another branch uses the Semantic Boundary Learning Module (SBLM) to learn boundary features based on multi-level semantic features, which process information in parallel with the semantic encoder. By introducing the Gate Convolution Layer (GCLs) into the SBLM module, the semantic information is converted more accurately to boundary information. Therefore, only boundary-related information will be processed by the SBLM. Then, we merge semantic and boundary features as input to an Unet model to obtain the final segmentation result. Our proposed approach was evaluated on five benchmark datasets: Kvasir, CVC-ClinicDB, CVC-ColonDB, CVC-300, and ETIS-LaribPolypDB. Experiments have demonstrated that it is an efficient architecture and capable of making accurate predictions about object boundaries and significantly improving the performance of finding thin and small objects.

BA-Net: Brightness prior guided attention network for colonic polyp segmentation

Colorectal cancer (CRC) caused by polyps has a high mortality rate worldwide. Accurately segmenting the polyp from colonoscopy images is important for the clinical treatment of CRC. The traditional method of polyp segmentation involves the physician manually marking the location of the polyp, resulting in unreliable segmentation results. The complex structure of polyps, the low contrast with mucosal tissue, and the fact that polyp boundary is usually hidden in the background make the task of polyp segmentation extremely challenging. To address these issues, we propose a boundary-aware polyp segmentation network. Specifically, we first propose an attention-aware location module to accurately identify the primary location of polyp. In order to improve the missing polyp portion in the initial region prediction and to mine the polyp boundary hidden in the background, we propose a residual pyramid convolution. Further, we propose a boundary-guided refinement module for more accurate segmentation in order to use the boundary information provided from residual pyramid convolution for constrained polyp region prediction. Extensive experiments show that our proposed network has advantages over existing state-of-the-art methods on five challenging polyp datasets.KeywordsPolyp segmentationContextual attentionBoundary guidanceResidual pyramid convolution

Automatic polyp segmentation from colonoscopy images is an essential step in computer aided diagnosis for colorectal cancer. Most of polyp segmentation methods reported in recent years are based on fully supervised deep learning. However, annotation for polyp images by physicians during the diagnosis is time-consuming and costly. In this paper, we present a novel semi-supervised polyp segmentation via collaborative and adversarial learning of focused and dispersive representations learning model, where focused and dispersive extraction module are used to deal with the diversity of location and shape of polyps. In addition, confidence maps produced by a discriminator in an adversarial training framework shows the effectiveness of leveraging unlabeled data and improving the performance of segmentation network. Consistent regularization is further employed to optimize the segmentation networks to strengthen the representation of the outputs of focused and dispersive extraction module. We also propose an auxiliary adversarial learning method to better leverage unlabeled examples to further improve semantic segmentation accuracy. We conduct extensive experiments on two famous polyp datasets: Kvasir-SEG and CVC-Clinic DB. Experimental results demonstrate the effectiveness of the proposed model, consistently outperforming state-of-the-art semi-supervised segmentation models based on adversarial training and even some advanced fully supervised models.

Colon cancer is thought about as the third most regularly identified cancer after Brest and lung cancer. Most colon cancers are adenocarcinomas developing from adenomatous polyps, grow on the intima of the colon. The standard procedure for polyp detection is colonoscopy, where the success of the standard colonoscopy depends on the colonoscopist experience and other environmental factors. Nonetheless, throughout colonoscopy procedures, a considerable number (8-37%) of polyps are missed due to human mistakes, and these missed polyps are the prospective reason for colorectal cancer cells. In the last few years, many research groups developed deep learning-based computer-aided (CAD) systems that recommended many techniques for automated polyp detection, localization, and segmentation. Still, accurate polyp detection, segmentation is required to minimize polyp miss out rates. This paper suggested a Super-Resolution Generative Adversarial Network (SRGAN) assisted Encoder-Decoder network for fully automated colon polyp segmentation from colonoscopic images. The proposed deep learning model incorporates the SRGAN in the up-sampling process to achieve more accurate polyp segmentation. We examined our model on the publicly available benchmark datasets CVC-ColonDB and Warwick- QU. The model accomplished a dice score of 0.948 on the CVC-ColonDB dataset, surpassed the recently advanced state-of-the-art (SOTA) techniques. When it is evaluated on the Warwick-QU dataset, it attains a Dice Score of 0.936 on part A and 0.895 on Part B. Our model showed more accurate results for sessile and smaller-sized polyps.

Medical image segmentation is important for computer-aided diagnosis. Good segmentation demands the model to see the big picture and fine details simultaneously, i.e., to learn image features that incorporate large context while keep high spatial resolutions. To approach this goal, the most widely used methods -- U-Net and variants, extract and fuse multi-scale features. However, the fused features still have small "effective receptive fields" with a focus on local image cues, limiting their performance. In this work, we propose Segtran, an alternative segmentation framework based on transformers, which have unlimited "effective receptive fields" even at high feature resolutions. The core of Segtran is a novel Squeeze-and-Expansion transformer: a squeezed attention block regularizes the self attention of transformers, and an expansion block learns diversified representations. Additionally, we propose a new positional encoding scheme for transformers, imposing a continuity inductive bias for images. Experiments were performed on 2D and 3D medical image segmentation tasks: optic disc/cup segmentation in fundus images (REFUGE'20 challenge), polyp segmentation in colonoscopy images, and brain tumor segmentation in MRI scans (BraTS'19 challenge). Compared with representative existing methods, Segtran consistently achieved the highest segmentation accuracy, and exhibited good cross-domain generalization capabilities.

Automatic and accurate segmentation of the ventricles and myocardium from multi-sequence cardiac MRI (CMR) is crucial for the diagnosis and treatment management for patients suffering from myocardial infarction (MI). However, due to the existence of domain shift among different modalities of datasets, the performance of deep neural networks drops significantly when the training and testing datasets are distinct. In this paper, we propose an unsupervised domain alignment method to explicitly alleviate the domain shifts among different modalities of CMR sequences, \emph{e.g.,} bSSFP, LGE, and T2-weighted. Our segmentation network is attention U-Net with pyramid pooling module, where multi-level feature space and output space adversarial learning are proposed to transfer discriminative domain knowledge across different datasets. Moreover, we further introduce a group-wise feature recalibration module to enforce the fine-grained semantic-level feature alignment that matching features from different networks but with the same class label. We evaluate our method on the multi-sequence cardiac MR Segmentation Challenge 2019 datasets, which contain three different modalities of MRI sequences. Extensive experimental results show that the proposed methods can obtain significant segmentation improvements compared with the baseline models.

Polyp segmentation with distraction separation

Polyp segmentation is essential in reducing the incidence of colorectal cancer by enabling the timely detection and removal of polyps, which are the precursors to most colorectal cancers. Accurate segmentation provides crucial information that aids in the early diagnosis and treatment of the disease. However, this task is challenging due to the considerable variation in polyp texture, color, and size. Additionally, some polyps closely resemble the surrounding healthy tissue in color, making it difficult to distinguish the boundaries between the polyp and its surrounding area. To address these challenges, we propose a deep learning model based on an encoder-decoder architecture. In the encoder section, we utilize a Pyramid Vision Transformer (PVT) to extract robust global feature representations across multiple stages. We introduce a multi-scale fusion module that combines features from different scales, allowing the model to capture diverse information. A custom convolution refinement attention module is employed to further refine the features, and it is integrated with the output of an efficient simple residual path, which preserves essential polyp features. Experiments are conducted on five publicly available polyp datasets. CVC-Clinic and Kvasir-SEG datasets are used for training, while CVC-ColonDB, Etis LaribDB, and CVC-300 datasets are used to evaluate generalization. The proposed approach achieves state-of-the-art performance, with a mean Dice coefficient of 93.9% and a mean Intersection over Union of 89% on CVC-Clinic, and 91.0% and 85.9% on Kvasir-SEG, respectively. Furthermore, our approach demonstrates the ability to adapt to new, unseen polyp features, achieving optimal results on the CVC-ColonDB, Etis LaribDB, and CVC-300 datasets. This adaptability makes it suitable for real clinical scenarios.