Automatic Brain Tumor Segmentation Using Cascaded Anisotropic Convolutional Neural Networks

We utilize 3-D fully convolutional neural networks (CNN) to segment gliomas and its constituents from multimodal Magnetic Resonance Images (MRI). The architecture uses dense connectivity patterns to reduce the number of weights and residual connection and is initialized with weights obtained from training this model with BraTS 2018 dataset. Hard mining is done during training to train for the difficult cases of segmentation tasks by increasing the dice similarity coefficient (DSC) threshold to choose the hard cases as epoch increases. On the BraTS2020 validation data (n = 125), this architecture achieved a tumor core, whole tumor, and active tumor dice of 0.744, 0.876, 0.714, respectively. On the test dataset, we get an increment in DSC of tumor core and active tumor by approximately 7%. In terms of DSC, our network performances on the BraTS 2020 test data are 0.775, 0.815, and 0.85 for enhancing tumor, tumor core, and whole tumor, respectively. Overall survival of a subject is determined using conventional machine learning from rediomics features obtained using generated segmentation mask. Our approach has achieved 0.448 and 0.452 as the accuracy on the validation and test dataset.KeywordsGliomasMRI3D CNNSegmentationHard miningOverall survival

Three-dimensional (3D) multimodal magnetic resonance (MR) image registration aims to align similar things in different MR images spatially. Such a technology is useful in auxiliary disease diagnosis and surgical treatment. However, inconsistent intensity correspondence and large initial displacement contribute to the difficulty in registering multimodal MR volumes. A coarse-to-fine method is proposed in this study for pairwise 3D MR image rigid registration. Firstly, the proposed method extracts image feature points to form unregistered point sets and performs coarse registration based on point set registration to reduce the initial displacements of offset images effectively. Then, this method calculates a grey histogram based on voxels in the adaptive region of interest and further improves registration accuracy by maximizing mutual information of coarse-registered images. Some representative registration methods are compared on the basis of three MR image datasets to evaluate the performance of the proposed method. Experimental results show that the proposed method improved more in registration success rate and accuracy compared with conventional registration methods, especially when initial displacements are large.

Accurate and automated brain tumor segmentation using multi modal MR images is essential for the evaluation of the disease progression in order to improve disease diagnosis and treatment planning. We present a new fully automated method for high graded brain tumor segmentation combining sparse autoencoder and multimodal Fuzzy C-means clustering. The approach utilizes multimodal MRI contrast: T1, T2, FLAIR and T1c (contrast-enhanced) for 15 high graded glioma (HGG) subjects. The objective of the proposed study is to segment tumor tissues from HGG including edema and tumor core within edema. The segmentation was performed on the training data of the multimodal brain tumor image segmentation benchmark 2015. Sparse autoencoder, which is an unsupervised learning algorithm, was used to automatically learn features from unlabeled dataset of tumor in order to segment edema. Followed by edema segmentation, tumor core was segmented from edema using multimodal FCM clustering. Evaluating the performance of the segmentation results with the ground truth yields high dice score (DS) of 0.9866±0.01 and 0.9843±0.01 for edema and tumor core respectively and high Jaccard similarity (JS) of 0.9738±0.02 and 0.9692±0.02 for edema and tumor core respectively; showed high accuracy in segmenting the complex tumor structures from multi-contrast MR scans of HGG patients. We also compared our methodology in terms of segmentation efficiency with some recent techniques reported in proceedings of MICCAI-BRATS challenge 2015.

Nowadays, there are various kinds of methods in medical image segmentation tasks, in which Cascaded FCN is an effective one. The idea of this method is to convert multiple classification tasks into a sequence of two categorization tasks, according to a series of sub-hierarchy regions of multi-modal Magnetic Resonance Images. We propose a model based on this idea, by combining the mainstream deep learning models for two dimensional images and modifying the 2D model to adapt to 3D medical image data set. Our model uses the Inception model, 3D Squeeze and Excitation structures, and dilated convolution filters, which are well known in 2D image segmentation tasks. When segmenting the whole tumor, we set the bounding box of the result, which is used to segment tumor core, and the bounding box of tumor core segmentation result will be used to segment enhancing tumor. We not only use the final output of the model, but also combine the results of intermediate output. In MICCAI BraTs 2018 gliomas segmentation task, we achieve a competitive performance without data augmentation.

Gliomas are the most prevalent and aggressive primary brain tumors characterized by rapid progressions and infiltration. Timely and precise diagnosis is crucial for effective oncology treatment. Magnetic Resonance Imaging (MRI) facilitates noninvasive assessment of brain lesions. Manual brain tumor evaluation from MRI scans is labor-intensive, relies heavily on clinician experience, and is prone to errors. Consequently, automated diagnosis of brain tumors is essential for optimal clinical management and glioma surgical interventions. This study introduces an Integrated Deep Convolutional Neural Network (IDCNN)-based framework for segmenting and grading glioma tumors from multimodal MRI scans. The framework integrates two state-of-the-art CNN architectures. Based on 3D U-Net architecture, the first CNN performs tumor segmentation from multimodal MRI volumes. The segmentation network utilizes the encoder for feature extraction and dimensionality reduction, while the decoder reconstructs the output to the original input size. A thorough performance evaluation of CNN architectures based on pre-trained (ResNet50, EfficientNetB0, DensNet121, EfficientNetB2, and ResNet152V2) was conducted on a three-class dataset of glioma, meningioma, and pituitary tumors to determine the best model for tumor grading. EfficientNetB2 surpassed other models across all evaluation metrics, achieving 99.19% test accuracy, 99.17% precision, 98.94% sensitivity, 99.57% specificity, and 99.06% F1-score. The optimal EfficientNetB2 model was subsequently utilized to grade tumors identified by the segmentation network. The proposed framework exhibited exceptional performance in both segmentation and grading tasks, attaining dice coefficient scores (DSC) of 86.13, 86.75, and 92.41 in enhancing tumor, tumor core, and whole tumor, respectively, alongside 98.49% classification accuracy for high- and low-grade glioma. Experimental findings validate the superior capabilities of the proposed framework compared to the existing methods. These results highlight the potential of the proposed model to aid radiologists in achieving accurate and reliable diagnoses, improving patient outcomes, and supporting clinical decision-making.

Gliomas are the most prevalent and aggressive primary brain tumors characterized by rapid progressions and infiltration. Timely and precise diagnosis is crucial for effective oncology treatment. Magnetic Resonance Imaging (MRI) facilitates noninvasive assessment of brain lesions. Manual brain tumor evaluation from MRI scans is labor-intensive, relies heavily on clinician experience, and is prone to errors. Consequently, automated diagnosis of brain tumors is essential for optimal clinical management and glioma surgical interventions. This study introduces an Integrated Deep Convolutional Neural Network (IDCNN)-based framework for segmenting and grading glioma tumors from multimodal MRI scans. The framework integrates two state-of-the-art CNN architectures. Based on 3D U-Net architecture, the first CNN performs tumor segmentation from multimodal MRI volumes. The segmentation network utilizes the encoder for feature extraction and dimensionality reduction, while the decoder reconstructs the output to the original input size. A thorough performance evaluation of CNN architectures based on pre-trained (ResNet50, EfficientNetB0, DensNet121, EfficientNetB2, and ResNet152V2) was conducted on a three-class dataset of glioma, meningioma, and pituitary tumors to determine the best model for tumor grading. EfficientNetB2 surpassed other models across all evaluation metrics, achieving 99.19% test accuracy, 99.17% precision, 98.94% sensitivity, 99.57% specificity, and 99.06% F1-score. The optimal EfficientNetB2 model was subsequently utilized to grade tumors identified by the segmentation network. The proposed framework exhibited exceptional performance in both segmentation and grading tasks, attaining dice coefficient scores (DSC) of 86.13, 86.75, and 92.41 in enhancing tumor, tumor core, and whole tumor, respectively, alongside 98.49% classification accuracy for high- and low-grade glioma. Experimental findings validate the superior capabilities of the proposed framework compared to the existing methods. These results highlight the potential of the proposed model to aid radiologists in achieving accurate and reliable diagnoses, improving patient outcomes, and supporting clinical decision-making.

We developed a fully automated method for brain tumor segmentation using deep learning; 285 brain tumor cases with multiparametric magnetic resonance images from the BraTS2018 data set were used. We designed 3 separate 3D-Dense-UNets to simplify the complex multiclass segmentation problem into individual binary-segmentation problems for each subcomponent. We implemented a 3-fold cross-validation to generalize the network's performance. The mean cross-validation Dice-scores for whole tumor (WT), tumor core (TC), and enhancing tumor (ET) segmentations were 0.92, 0.84, and 0.80, respectively. We then retrained the individual binary-segmentation networks using 265 of the 285 cases, with 20 cases held-out for testing. We also tested the network on 46 cases from the BraTS2017 validation data set, 66 cases from the BraTS2018 validation data set, and 52 cases from an independent clinical data set. The average Dice-scores for WT, TC, and ET were 0.90, 0.84, and 0.80, respectively, on the 20 held-out testing cases. The average Dice-scores for WT, TC, and ET on the BraTS2017 validation data set, the BraTS2018 validation data set, and the clinical data set were as follows: 0.90, 0.80, and 0.78; 0.90, 0.82, and 0.80; and 0.85, 0.80, and 0.77, respectively. A fully automated deep learning method was developed to segment brain tumors into their subcomponents, which achieved high prediction accuracy on the BraTS data set and on the independent clinical data set. This method is promising for implementation into a clinical workflow.

Computer Aided Diagnostic (CAD) systems are already of proven value in healthcare, especially for surgical planning, nevertheless much remains to be done. Gliomas are the most common brain tumours (70%) in adults, with a survival time of just 2-3 months if detected at WHO grades III or higher. Such tumours are extremely variable, necessitating multi-modal Magnetic Resonance Images (MRI). The use of Gadolinium-based contrast agents is only relevant at later stages of the disease where it highlights the enhancing rim of the tumour. Currently, there is no single accepted method that can be used as a reference. There are three main challenges with such images: to decide whether there is tumour present and is so localize it; to construct a mask that separates healthy and diseased tissue; and to differentiate between the tumour core and the surrounding oedema. This paper presents two contributions. First, we develop tumour seed selection based on multiscale multi-modal texture feature vectors. Second, we develop a method based on a local phase congruency based feature map to drive level-set segmentation. The segmentations achieved with our method are more accurate than previously presented methods, particularly for challenging low grade tumours.

Multi-modality magnetic resonance (MR) images provide complementary information for disease diagnoses. However, modality missing is quite usual in real-life clinical practice. Current methods usually employ convolution-based generative adversarial network (GAN) or its variants to synthesize the missing modality. With the development of vision transformer, we explore its application in the MRI modality synthesis task in this work. We propose a novel supervised deep learning method for synthesizing a missing modality, making use of a transformer-based encoder. Specifically, a model is trained for translating 2D MR images from T1-weighted to T2-weighted based on conditional GAN (cGAN). We replace the encoder with transformer and input adjacent slices to enrich spatial prior knowledge. Experimental results on a private dataset and a public dataset demonstrate that our proposed model outperforms state-of-the-art supervised methods for MR image synthesis, both quantitatively and qualitatively. Clinical relevance- This work proposes a method to synthesize T2-weighted images from T1-weighted ones to address the modality missing issue in MRI.

This article explores the application of self-supervised contrastive learning in the medical domain, focusing on classification of multi-modality Magnetic Resonance (MR) images. To address the challenges of limited and hard-to-annotate medical data, we introduce multi-modality data augmentation (MDA) and cross-modality group convolution (CGC). In the pre-training phase, we leverage Simple Siamese networks to maximize the similarity between two augmented MR images from a patient, without a handcrafted pretext task. Our approach also combines 3D and 2D group convolution with a channel shuffle operation to efficiently incorporate different modalities of image features. Evaluation on liver MR images from a well-known hospital in Taiwan demonstrates a significant improvement over previous methods. This work contributes to advancing multi-modality contrastive learning, particularly in the context of medical imaging, offering enhanced tools for analyzing complex image data.

Gliomas are the most frequent primary brain tumors in adults. Improved quantification of the various aspects of a glioma requires accurate segmentation of the tumor in magnetic resonance images (MRI). Since the manual segmentation is time-consuming and subject to human error and irreproducibility, automatic segmentation has received a lot of attention in recent years. This paper presents a fully automated segmentation method which is capable of automatic segmentation of brain tumor from multi-modal MRI scans. The proposed method is comprised of a deeply-supervised neural network based on Holistically-Nested Edge Detection (HED) network. The HED method, which is originally developed for the binary classification task of image edge detection, is extended for multiple-class segmentation. The classes of interest include the whole tumor, tumor core, and enhancing tumor. The dataset provided by 2017 Multimodal Brain Tumor Image Segmentation Benchmark (BraTS) challenge is used in this work for training the neural network and performance evaluations. Experiments on BraTS 2017 challenge datasets demonstrate that the method performs well compared to the existing works. The assessments revealed the Dice scores of 0.86, 0.60, and 0.69 for whole tumor, tumor core, and enhancing tumor classes, respectively.

Reliable segmentation of brain tumors from magnetic resonance imaging (MRI) is essential for clinical decision-making, radiotherapy planning, and disease progression analysis. Among the different tumor sub-regions, Whole Tumor (WT), which comprises enhancing tumor, tumor core, and peritumoral edema, offers clinically meaningful information while being comparatively robust to annotation variability. This work proposes an attention-enhanced U-Net architecture for binary WT segmentation using multimodal MRI data. The network integrates four MRI modalities—T1, T1ce, T2, and FLAIR—and employs attention gates within skip connections to selectively highlight tumor-relevant features and suppress background responses. Experimental evaluation on the BraTS dataset demonstrates improved Dice similarity, IoU, and sensitivity compared to baseline models.

The accuracy of three-dimensional (3D) brain tumor image segmentation is of great significance to brain tumor diagnosis. To enhance the accuracy of segmentation, this study proposes an algorithm integrating a cascaded anisotropic fully convolutional neural network (FCNN) and the hybrid level set method. The algorithm first performs bias field correction and gray value normalization on T1, T1C, T2, and fluid-attenuated inversion recovery magnetic resonance imaging (MRI) images for preprocessing. It then uses a cascading mechanism to perform preliminary segmentation of whole tumors, tumor cores, and enhancing tumors by an anisotropic FCNN based on the relationships among the locations of the three types of tumor structures. This simplifies multiclass brain tumor image segmentation problems into three binary classification problems. At the same time, the anisotropic FCNN adopts dense connections and multiscale feature merging to further enhance performance. Model training is respectively conducted on the axial, coronal, and sagittal planes, and the segmentation results from the three different orthogonal views are combined. Finally, the hybrid level set method is adopted to refine the brain tumor boundaries in the preliminary segmentation results, thereby completing fine segmentation. The results indicate that the proposed algorithm can achieve 3D MRI brain tumor image segmentation of high accuracy and stability. Comparison of the whole-tumor, tumor-core, and enhancing-tumor segmentation results with the gold standards produced Dice similarity coefficients (Dice) of 0.9113, 0.8581, and 0.7976, respectively.

Brain tumor significantly impacts the quality of life and changes everything for a patient and their loved ones. Diagnosing a brain tumor usually begins with magnetic resonance imaging (MRI). The manual brain tumor diagnosis from the MRO images always requires an expert radiologist. However, this process is time-consuming and costly. Therefore, a computerized technique is required for brain tumor detection in MRI images. Using the MRI, a novel mechanism of the three-dimensional (3D) Kronecker convolution feature pyramid (KCFP) is used to segment brain tumors, resolving the pixel loss and weak processing of multi-scale lesions. A single dilation rate was replaced with the 3D Kronecker convolution, while local feature learning was performed using the 3D Feature Selection (3DFSC). A 3D KCFP was added at the end of 3DFSC to resolve weak processing of multi-scale lesions, yielding efficient segmentation of brain tumors of different sizes. A 3D connected component analysis with a global threshold was used as a post-processing technique. The standard Multimodal Brain Tumor Segmentation 2020 dataset was used for model validation. Our 3D KCFP model performed exceptionally well compared to other benchmark schemes with a dice similarity coefficient of 0.90, 0.80, and 0.84 for the whole tumor, enhancing tumor, and tumor core, respectively. Overall, the proposed model was efficient in brain tumor segmentation, which may facilitate medical practitioners for an appropriate diagnosis for future treatment planning.

Brain tumor segmentation in Magnetic Resonance Imaging (MRI) scans provides vital information to radiologists in the diagnosis and staging of disease. However, these MRI scans are often corrupted with noise during its acquisition. Traditional approaches to this problem employ denoising which leads, in general, to edge smoothing and development of artifacts in MRI slices, thereby affecting tumor segmentation performance. In this paper, we employed graph signal processing (GSP) theory to first segment tumor core in each MRI slice using graph Laplacian followed by edge-aware denoising which is performed in synergy. The paper aims to present a novel technique to tackle the two problems of segmentation and denoising both under the GSP framework. The experimental results demonstrated on simulated and clinical brain MRI datasets, show highly competitive performance both in terms of tumor core segmentation under Dice and Sensitivity measures, and in terms of edge-aware denoising under PSNR and SSIM measures.