Channel-Gated Transformers With Affinity CAM for Weakly Supervised Multi-Class Brain Tumor Segmentation.

Existing weakly supervised semantic segmentation (WSSS) methods based on image‐level labels always rely on class activation maps (CAMs), which measure the relationships between features and classifiers. However, CAMs only focus on the most discriminative regions of images, resulting in their poor coverage performance. We attribute this to the deficiency in the recognition ability of a single classifier and the negative impacts caused by magnitudes during the CAMs normalisation process. To address the aforementioned issues, we propose to construct selective multiple classifiers (SMC). During the training process, we extract multiple prototypes for each class and store them in the corresponding memory bank. These prototypes are divided into foreground and background prototypes, with the former used to identify foreground objects and the latter aimed at preventing the false activation of background pixels. As for the inference stage, multiple prototypes are adaptively selected from the memory bank for each image as SMC. Subsequently, CAMs are generated by measuring the angle between SMC and features. We enhance the recognition ability of classifiers by adaptively constructing multiple classifiers for each image, while only relying on angle measurement to generate CAMs can alleviate the suppression phenomenon caused by magnitudes. Furthermore, SMC can be integrated into other WSSS approaches to help generate better CAMs. Extensive experiments conducted on standard WSSS benchmarks such as PASCAL VOC 2012 and MS COCO 2014 demonstrate the superiority of our proposed method.

Mutated Aquila Optimizer for assisting brain tumor segmentation

Segmentation of brain tumor from magnetic resonance imaging is a challenging and time-consuming task due to the unpredictable appearance of tumor tissue in practical applications. In this paper we propose a novel level-set-based model for tumor segmentation from multi-modality magnetic resonance imaging. We formulate the tumor segmentation on pixel-level with three classes: tumor, edema and healthy tissue. First, we detect abnormal regions by using a region-based active contour model on T2-weighted images with fluid-attenuated inversion recovery modality, and a variational level set formulation is applied locally to approximate the image intensities on two sides of the contour. In the second stage, we distinguish the edema and tumor tissues in the abnormal regions based on the contrast enhancement T1 modality. Compared with traditional one-modality methods, our model can better represent the specific tissue of tumor in a simple way. The validation experiments on synthetic and clinical brain magnetic resonance images demonstrate the effectiveness and simplicity of the proposed method for brain tumor segmentation.

Deformation-aware and reconstruction-driven multimodal representation learning for brain tumor segmentation with missing modalities

Grid conditional random fields (CRFs) are widely applied in both natural and medical image segmentation tasks. However, they only consider the label coherence in neighborhood pixels or regions, which limits their ability to model long-range connections within the image and generally results in excessive smoothing of tumor boundaries. In this paper, we present a novel method for brain tumor segmentation in MR images based on fully-connected CRF (FC-CRF) model that establishes pairwise potentials on all pairs of pixels in the images. We employ a hierarchical approach to differentiate different structures of tumor and further formulate a FC-CRF model with learned data-driven prior knowledge of tumor core. The methods were evaluated on the testing and leaderboard set of Brain Tumor Image Segmentation Benchmark (BRATS) 2013 challenge. The precision of segmented tumor boundaries is improved significantly and the results are competitive compared to the start-of-the-arts.

CLCU-Net: Cross-level connected U-shaped network with selective feature aggregation attention module for brain tumor segmentation

Weakly supervised semantic segmentation (WSSS) generally utilizes the Class Activation Map (CAM) to synthesize pseudo-labels. However, the current methods of obtaining CAM focus on salient features of a specific layer, resulting in highlighting the most discriminative regions and further leading to rough segmentation results for WSSS. In this paper, we rethink the potential of the ordinary classifier and find that if features of all the layers are applied, the classifier will obtain CAM with complete discriminative regions. Inspired by this, we propose Fully-CAM for WSSS, which can fully exploit the potential of the ordinary classifier and yield more accurate segmentation results. Precisely, Fully-CAM firstly weights feature with their corresponding gradients to yield CAMs of each layer, then fusing these layers’ CAMs could generate an ultimate CAM with complete discriminative regions. Furthermore, Fully-CAM is encapsulated into a plug-in, which can be mounted on any trained ordinary classifier with convolution layer, and it exceeds its previous performance without extra training.

DCAM: Disturbed class activation maps for weakly supervised semantic segmentation

A brain tumor is one of the conditions in which a patient's brain develops abnormal cells. They are called tumors and there are several varieties of them. In this paper, brain tumor segmentation and detection methods are being developed that use images from an MRI series as input images to identify the type of tumor. The use of a convolutional neural network (CNN) model to recognize brain tumors from X-ray images pictures is described in this paper. This investigation was carried out with the help of Google Collab. The MRI images of brain tumors are categorized into glioma, meningioma, and pituitary, and no tumor is one such difficult problem. In this work, the datasets comprise 388 images for testing purposes. With an accuracy of 70%, the projected arrangement achieves a commendable performance. The successful conclusion demonstrates the suggested algorithm’s ability to classify brain tumors.

Detection and segmentation of a brain tumor such as glioblastoma multiforme (GBM) in magnetic resonance (MR) images are often challenging due to its intrinsically heterogeneous signal characteristics. A robust segmentation method for brain tumor MRI scans was developed and tested. Simple thresholds and statistical methods are unable to adequately segment the various elements of the GBM, such as local contrast enhancement, necrosis, and edema. Most voxel-based methods cannot achieve satisfactory results in larger data sets, and the methods based on generative or discriminative models have intrinsic limitations during application, such as small sample set learning and transfer. A new method was developed to overcome these challenges. Multimodal MR images are segmented into superpixels using algorithms to alleviate the sampling issue and to improve the sample representativeness. Next, features were extracted from the superpixels using multi-level Gabor wavelet filters. Based on the features, a support vector machine (SVM) model and an affinity metric model for tumors were trained to overcome the limitations of previous generative models. Based on the output of the SVM and spatial affinity models, conditional random fields theory was applied to segment the tumor in a maximum a posteriori fashion given the smoothness prior defined by our affinity model. Finally, labeling noise was removed using "structural knowledge" such as the symmetrical and continuous characteristics of the tumor in spatial domain. The system was evaluated with 20GBM cases and the BraTS challenge data set. Dice coefficients were computed, and the results were highly consistent with those reported by Zikic et al. (MICCAI 2012, Lecture notes in computer science. vol 7512, pp 369-376, 2012). A brain tumor segmentation method using model-aware affinity demonstrates comparable performance with other state-of-the art algorithms.

In weakly supervised semantic segmentation (WSSS) tasks on remote sensing images, it is a common practice to train a classification network from scratch using a large batch of images with a limited number of classes. Subsequently, class activation maps are extracted from the model based on predefined class indices, and these maps are then optimized to obtain pseudolabels. To make this strategy effective when introducing a new class, a substantial amount of data needs to be provided to the model. In this article, we present an innovative framework, RS-TextWS-Seg, designed to efficiently generate high-quality segmentation results for a wide range of remote sensing objects using concise descriptions. Our proposed framework comprises three sequential stages: initially, we undertake parameter fine-tuning of the contrastive language-image pretraining (CLIP) model to swiftly strengthen its capacity for zero-shot detection of a limited number of remote sensing features. Subsequently, we introduce a text-driven background suppression mechanism aimed at deriving class activation maps from the refined CLIP model based on textual cues, while concurrently mitigating background noises. Finally, we use the segment anything model (SAM) to refine the edges of the extracted class activation map. We widely researched the leading-edge methodologies in WSSS and conducted a range of comparative experiments and ablation studies to prove the efficacy of our proposed framework. The research findings underscore that RS-TextWS-Seg outperforms other state-of-the-art methods on renowned datasets such as DLRSD and Potsdam, as well as on bespoke datasets specifically curated for overground petroleum pipelines and oil well fields.

The costly process of obtaining semantic segmentation labels has driven research towards weakly supervised semantic segmentation (WSSS) methods, using only image-level, point, or box labels. Such annotations introduce limitations and challenges that results in overly-tuned methods specialized in specific domains or scene types. The over reliance of image-level based methods on generation of high quality class activation maps (CAMs) results in limited applicable dataset complexity range, mostly focusing on object centric scenes. Additionally, the lack of dense annotations requires methods to increase network complexity to obtain additional semantic information, often done through multiple stages of training and refinement. Here, we present a single-stage approach generalizable to a wide range of dataset complexities, that is trainable from scratch, without any dependency on pre-trained backbones, classification, or separate refinement tasks. We utilize point annotations to generate reliable, on-the-fly pseudo-masks through refined and spatially filtered features. We are to demonstrate SOTA performance on benchmark datasets (PascalVOC 2012), as well as significantly outperform other SOTA WSSS methods on recent real-world datasets (CRAID, CityPersons, IAD, ADE20K, CityScapes) with up to 28.1% and 22.6% performance boosts compared to our single-stage and multi-stage baselines respectively.

Weakly supervised semantic segmentation (WSSS) models relying on class activation maps (CAMs) have achieved desirable performance comparing to the non-CAMs-based counterparts. However, to guarantee WSSS task feasible, we need to generate pseudo labels by expanding the seeds from CAMs which is complex and time-consuming, thus hindering the design of efficient end-to-end (single-stage) WSSS approaches. To tackle the above dilemma, we resort to the off-the-shelf and readily accessible saliency maps for directly obtaining pseudo labels given the image-level class labels. Nevertheless, the salient regions may contain noisy labels and cannot seamlessly fit the target objects, and saliency maps can only be approximated as pseudo labels for simple images containing single-class objects. As such, the achieved segmentation model with these simple images cannot generalize well to the complex images containing multi-class objects. To this end, we propose an end-to-end multi-granularity denoising and bidirectional alignment (MDBA) model, to alleviate the noisy label and multi-class generalization issues. Specifically, we propose the online noise filtering and progressive noise detection modules to tackle image-level and pixel-level noise, respectively. Moreover, a bidirectional alignment mechanism is proposed to reduce the data distribution gap at both input and output space with simple-to-complex image synthesis and complex-to-simple adversarial learning. MDBA can reach the mIoU of 69.5% and 70.2% on validation and test sets for the PASCAL VOC 2012 dataset. The source codes and models have been made available at https://github.com/NUST-Machine-Intelligence-Laboratory/MDBA.

Accurate and automated brain tumor segmentation using multi modal MR images is essential for the evaluation of the disease progression in order to improve disease diagnosis and treatment planning. We present a new fully automated method for high graded brain tumor segmentation combining sparse autoencoder and multimodal Fuzzy C-means clustering. The approach utilizes multimodal MRI contrast: T1, T2, FLAIR and T1c (contrast-enhanced) for 15 high graded glioma (HGG) subjects. The objective of the proposed study is to segment tumor tissues from HGG including edema and tumor core within edema. The segmentation was performed on the training data of the multimodal brain tumor image segmentation benchmark 2015. Sparse autoencoder, which is an unsupervised learning algorithm, was used to automatically learn features from unlabeled dataset of tumor in order to segment edema. Followed by edema segmentation, tumor core was segmented from edema using multimodal FCM clustering. Evaluating the performance of the segmentation results with the ground truth yields high dice score (DS) of 0.9866±0.01 and 0.9843±0.01 for edema and tumor core respectively and high Jaccard similarity (JS) of 0.9738±0.02 and 0.9692±0.02 for edema and tumor core respectively; showed high accuracy in segmenting the complex tumor structures from multi-contrast MR scans of HGG patients. We also compared our methodology in terms of segmentation efficiency with some recent techniques reported in proceedings of MICCAI-BRATS challenge 2015.

Glioma is one of the most common and aggressive brain tumors. Segmentation and subsequent quantitative analysis of brain tumor MRI are routine and crucial for treatment. Due to the time-consuming and tedious manual segmentation, automatic segmentation methods are required for accurate and timely treatment. Recently, segmentation methods based on deep learning are popular because of their self-learning and generalization ability. Therefore, we propose a novel automatic 3D CNN-based method for brain tumor segmentation. In order to better capture the contextual information, we design the network architecture based on u-net and replace the simple skip connection with encoder adaptation blocks. To further improve the performance and reduce computational burden at the same time, we also use dense connected fusion blocks in decoder. We train our model with generalised dice loss function to alleviate the problem of class imbalance. The proposed model is evaluated on the BRATS 2015 testing dataset and obtains dice scores of 0.84, 0.72 and 0.62 for whole tumor, tumor core and enhancing tumor, respectively. Our model is accurate and efficient, achieving results that comparable to the reported state-of-the-art results.