Brain Tumour Segmentation Method Based on Supervoxels and Sparse Dictionaries

597 Background: Automated breast tumor identification and segmentation in magnetic resonance imaging (MRI) is a difficult and crucial area of study in breast cancer research. Artificial intelligence (AI) models are increasingly being developed for automated localization of lesions in imaging studies to facilitate quantitative assessment of features for improved diagnostic, prognostic and predictive performance. Such models have had success in detecting breast cancers in mammography, ultrasound and CT, but few have achieved three-dimensional (3D) volumetric tumor segmentation from breast MRI. The purpose of this study was to apply two state-of-the-art AI – specifically, deep learning (DL) - algorithms to 3D MRI breast cancer data and identify the higher performing algorithm for precise segmentation of breast tumors. Methods: We evaluated pre-treatment, T1 post-gadolinium contrast enhanced breast MRI from 222 patients with known breast cancers (n = 262). Images were split into training (n = 142), validation (n = 36), and hold-out test (n = 44) datasets. Two DL algorithms, U-Net and VAE-UNet, were trained to classify tumors on the training dataset across 1000 epochs. The output for each is a precise localization and segmentation of each tumor at the pixel level from every MRI image. We evaluated the performance of each algorithm using 5-fold cross-validation and testing on the validation and test sets. We calculated a dice accuracy score for each model as the performance comparison metric. Results: The highest dice accuracy score achieved on the validation dataset by generic U-Net was 83.38%, with an average across 1000 epochs of 62.41%. The highest dice accuracy achieved on the validation dataset by VAE-UNet was 82.62%, with an average across epochs of 61.28%. On our test dataset, the highest dice accuracy score achieved by U-Net was 93.09%, with an average across epochs of 66.31%, and the highest accuracy score for VAE-UNet was 90.98%, with average across epochs of 50.47%. Although U-Net appeared to perform slightly better than VAE-Unet for most cases, there were distinct cases where VAE-UNet outperformed U-Net (dice score up to 59% better than U-Net). Subsequent analysis indicated that VAE-UNet preferentially outperforms U-Net for tumors with low sphericity (p = 0.001). Conclusions: Our results suggest that U-Net is well suited for segmenting breast tumors from breast MRI in most cases, but that VAE-UNet outperforms U-Net when the tumor shapes are less spherical. Our findings could inform the choice of DL algorithms in research and clinical endeavors that rely on accurate breast cancer tumor segmentation. In particular, these two tools could be configured to facilitate tumor assessment from breast MRI in the clinical setting for: breast cancer screening in high-risk patient populations, pre-surgical planning, and monitoring of treatment response.

We developed a fully automated method for brain tumor segmentation using deep learning; 285 brain tumor cases with multiparametric magnetic resonance images from the BraTS2018 data set were used. We designed 3 separate 3D-Dense-UNets to simplify the complex multiclass segmentation problem into individual binary-segmentation problems for each subcomponent. We implemented a 3-fold cross-validation to generalize the network's performance. The mean cross-validation Dice-scores for whole tumor (WT), tumor core (TC), and enhancing tumor (ET) segmentations were 0.92, 0.84, and 0.80, respectively. We then retrained the individual binary-segmentation networks using 265 of the 285 cases, with 20 cases held-out for testing. We also tested the network on 46 cases from the BraTS2017 validation data set, 66 cases from the BraTS2018 validation data set, and 52 cases from an independent clinical data set. The average Dice-scores for WT, TC, and ET were 0.90, 0.84, and 0.80, respectively, on the 20 held-out testing cases. The average Dice-scores for WT, TC, and ET on the BraTS2017 validation data set, the BraTS2018 validation data set, and the clinical data set were as follows: 0.90, 0.80, and 0.78; 0.90, 0.82, and 0.80; and 0.85, 0.80, and 0.77, respectively. A fully automated deep learning method was developed to segment brain tumors into their subcomponents, which achieved high prediction accuracy on the BraTS data set and on the independent clinical data set. This method is promising for implementation into a clinical workflow.

Accurate and automated brain tumor segmentation using multi modal MR images is essential for the evaluation of the disease progression in order to improve disease diagnosis and treatment planning. We present a new fully automated method for high graded brain tumor segmentation combining sparse autoencoder and multimodal Fuzzy C-means clustering. The approach utilizes multimodal MRI contrast: T1, T2, FLAIR and T1c (contrast-enhanced) for 15 high graded glioma (HGG) subjects. The objective of the proposed study is to segment tumor tissues from HGG including edema and tumor core within edema. The segmentation was performed on the training data of the multimodal brain tumor image segmentation benchmark 2015. Sparse autoencoder, which is an unsupervised learning algorithm, was used to automatically learn features from unlabeled dataset of tumor in order to segment edema. Followed by edema segmentation, tumor core was segmented from edema using multimodal FCM clustering. Evaluating the performance of the segmentation results with the ground truth yields high dice score (DS) of 0.9866±0.01 and 0.9843±0.01 for edema and tumor core respectively and high Jaccard similarity (JS) of 0.9738±0.02 and 0.9692±0.02 for edema and tumor core respectively; showed high accuracy in segmenting the complex tumor structures from multi-contrast MR scans of HGG patients. We also compared our methodology in terms of segmentation efficiency with some recent techniques reported in proceedings of MICCAI-BRATS challenge 2015.

Brain tumour segmentation from MRI using superpixels based spectral clustering

Glioma is one of the most common and aggressive brain tumors. Segmentation and subsequent quantitative analysis of brain tumor MRI are routine and crucial for treatment. Due to the time-consuming and tedious manual segmentation, automatic segmentation methods are required for accurate and timely treatment. Recently, segmentation methods based on deep learning are popular because of their self-learning and generalization ability. Therefore, we propose a novel automatic 3D CNN-based method for brain tumor segmentation. In order to better capture the contextual information, we design the network architecture based on u-net and replace the simple skip connection with encoder adaptation blocks. To further improve the performance and reduce computational burden at the same time, we also use dense connected fusion blocks in decoder. We train our model with generalised dice loss function to alleviate the problem of class imbalance. The proposed model is evaluated on the BRATS 2015 testing dataset and obtains dice scores of 0.84, 0.72 and 0.62 for whole tumor, tumor core and enhancing tumor, respectively. Our model is accurate and efficient, achieving results that comparable to the reported state-of-the-art results.

Automated segmentation of brain tumors in multi-channel Magnetic Resonance Image (MRI) is a challenging task. Heterogeneous appearance of brain tumors in MRI poses critical challenges in diagnosis, prognosis and survival prediction. In this paper, we present a novel approach for glioma tumor segmentation and survival prediction with Deep Learning Radiomics Algorithm for Gliomas (DRAG) Model using 3D patch based U-Net model in Brain Tumor Segmentation (BraTS) challenge 2018. Radiomics feature extraction and classification was done on segmented tumor for overall survival (OS) prediction task. Preliminary results of DRAG model on BraTS 2018 validation dataset demonstrated that the proposed method achieved a good performance with Dice scores as 0.88, 0.83 and 0.75 for whole tumor, tumor core and enhancing tumor, respectively. For survival prediction, 57.1% accuracy was achieved on the validation dataset. The proposed DRAG model was one of the top performing models and accomplished third place for OS prediction task in BraTS 2018 challenge.

BACKGROUNDPediatric diffuse midline gliomas (DMG) are aggressive central nervous system tumors with a median overall survival (OS) of less than one year from diagnosis. There is no curative therapy for DMG, and radiation therapy (RT) is the standard treatment. Magnetic resonance imaging (MRI) is the common noninvasive tool for DMG diagnosis and monitoring of tumor response to therapy. We developed an automatic pipeline to segment subregions of DMG and select MRI features that predict patient OS. METHODSWe acquired diagnostic and post-RT multisequence MRI (T1, T1ce, T2, T2 FLAIR) and manual segmentations of pediatric DMG patients from two centers: 53 from Children’s National Hospital (internal) and 16 from Children’s Hospital of Philadelphia (external). We pretrained a deep learning model on a public adult brain tumor dataset, and finetuned the model to segment tumor core (TC) and whole tumor (WT) of DMG. We used PyRadiomics and sequential feature selection to select discriminative features based on the segmented volumes. Two machine learning models were trained on our internal cohort to predict patient 1-year survival from diagnosis. One model used only diagnostic tumor features and the other used both diagnostic and post-RT features. RESULTSFor segmentation, Dice score (mean [median]±SD) was 0.91 (0.94)±0.12 and 0.74 (0.83)±0.32 for TC, and 0.88 (0.91)±0.07 and 0.86 (0.89)±0.06 for WT for internal and external cohorts, respectively. For OS prediction, accuracy was 77% and 81% at time of diagnosis, and 85% and 78% post-RT for internal and external cohorts, respectively. Heterogeneous WT intensity in baseline T2 FLAIR and smaller post-RT TC/WT volume ratio indicate longer OS. CONCLUSIONSOur method accurately predict OS for DMG patients and can be extended to other rare pediatric brain tumors. With automated and standardized analysis, early prognostication of OS can guide patient risk stratification and clinical decisions.

The proliferation of unmanned aerial vehicles (UAVs) has driven the emergence of urban air mobility (UAM) as a solution to alleviate the increasing demand for UAVs applications in various domains. The concept of UAM envisions a future where UAVs play a pivotal role in short-distance transportation, delivery services, surveillance, emergency rescue, and more. However, the rapid growth of UAVs poses significant challenges arising from the need to ensure the safe coexistence of UAVs with other manned/unmanned aerial vehicles. To ensure the safe integration of UAVs into low-altitude urban airspace and support high-density on-demand mobility, this research focuses on handling the safety-centric challenges related to UAV collision risk assessment, obstacle avoidance, and conflict resolution in complex urban environments.
\nOne of the main challenges is the intrusion of UAVs into airport runways. Airports serve as critical hubs for civil aircraft takeoff and landing operations. However, the increasing presence of UAVs has raised concerns about intruding into airports, leading to flight delays and temporary runway closures. To quantify the potential interference of UAVs on civil aircraft, a collision-course based trajectory planning model is proposed to evaluate the collision risk between manned civil aircraft and intruding UAVs. In this risk assessment model, the trajectory of the intruding UAV is formulated using a collision-course concept, taking into account the limited trajectory information available about intruding UAVs. The planned trajectory of the UAVs is then used to predict its relative position concerning civil aircraft. To account for the possible disturbances of the relative distance, a stochastic position prediction model based on Brownian motion is developed. The collision risk is assessed based on this stochastic kinematic model, providing a quantitative measure of the potential collisions. The proposed model is validated through extensive simulations involving different UAV initial positions, position updates, and different sizes of collision zone tests, as well as comparative tests with the Monte-Carlo algorithm, demonstrating its robustness in assessing the collision risk of the airport restricted area. To facilitate the application, the model is finally used to develop a platform to generate runway heatmaps, accounting for airport runway risk distributions. 
\nAnother safety-centric challenge is ensuring autonomous obstacle avoidance and conflict resolution of UAVs. To enable UAV autonomous safe operations in uncontrolled urban low-altitude airspace, a comprehensive collision-free path planning framework is developed to address the avoidance of both static obstacles and dynamic threats. Firstly, to handle static obstacles, a novel algorithm of 3D voxel jump point research is introduced to generate a global reference path for UAV navigation to avoid static obstacles, particularly for a free flight at different altitudes. Additionally, to optimize the reference path for specific mission requirements, techniques such as trajectory de-diagonalization, reconstruction, and smoothness are proposed to reduce the risk of static collisions, shorten the fly range, and enable UAVs to negotiate turns more efficiently. Secondly, real-time conflict resolution actions based on the Markov decision process are developed to avoid local dynamic threats encountered along the optimized desired path. This allows the host UAV to dynamically adjust its course to a resolution trajectory while maintaining its planned speed without hovering to avoid the potential collision with the approaching intruders. By avoiding near mid-air collisions (NMAC) and preventing the UAV from hovering, the proposed method ensures the timely completion of UAV missions. Simulation results demonstrate that the developed method successfully achieves UAV autonomous global path planning and local real-time conflict resolution in complex urban low-altitude environments, thereby avoiding both static obstacles and dynamic threats. 
\nMoreover, to bridge the research gap of UAV hard conflict resolution in mitigating NMAC risks arising from local dynamic threats, a novel situational NMAC risk assessment approach is developed by integrating the conditional random field (CRF) algorithm with safety metrics and conflict resolution performance to facilitate UAV tiered aerial passage operations. To improve UAV operating environments in urban airspace, a concept of tiered aerial passages is first designed to provide a practical urban airspace planning scheme that addresses UAV heterogeneity and optimizes the utilization of urban airspace. For UAV tiered aerial passage operations, a situational NMAC risk assessment module coupled with a conflict resolution decision-making module is formulated, enabling the host UAV to evaluate and respond to real-time conflicts, ensuring avoidance of potential collisions. To assess NMAC risks, a general risk assessment model is proposed based on the CRF algorithm, considering UAV safety metrics and resolution performance. Then the overtaking, approaching head-on, and converging UAV encounters are analyzed, and situational risk potential scores are created using the Gaussian kernel function in the CRF algorithm, considering advisory, caution, and warning risk alert levels. To resolve conflict, acceptable, tolerable, and unacceptable scenarios are proposed to enhance the flexibility and personalization of the resolution decision-making. The proposed scheme is validated by simulating geometric encounter situations and comparative tests with the previous hard resolution technique. The results indicate that the proposed method outperforms UAV NMAC risk assessment and conflict resolution. Even in the most dangerous unacceptable resolution decision-making scenarios across three conflict encounters, the proposed algorithm achieves NMAC rates of 0.036, 0.070, and 0.043, which are lower than the corresponding NMAC rates associated with the hard resolution approach. Together with the acceptable and tolerable scenarios, the proposed negotiation resolution effectively tailors different risk requirements to accommodate diverse UAV missions. 
\nWith a primary focus on ensuring the safety of the UAM architecture, this research endeavors to assess UAV NMAC risks and efficiently manage potential conflicts. The proposed work are thoroughly validated to demonstrate their effectiveness in real-world scenarios. As a result, this study significantly contributes to the establishment of a robust UAM architecture, facilitating the realization of safe and reliable UAM.

There is an emerging need for medical imaging data to provide patients with timely diagnosis. Magnetic resonance imaging (MRI) images based on brain tumor segmentation approaches possess greater importance in planning treatment. Though, mechanizing the process with different imaging conditions and accuracy is a major challenge due to variations in tumor structures. Hence, an efficient optimization-driven classifier, called BirCat optimization-based deep belief network (BirCat-based DBN)isdeveloped to detect brain tumors. The introduced BirCat is devised by incorporating birdswarm algorithm (BSA) into cat swarm optimization (CSO) algorithm and is employed in tuning the DBN classifier. Here, the first step is pre-processing, where noises, as well as artifacts in input image, are eliminated by means of ROI extraction and filtering method. Then, for segmentation, region growing algorithm is used in which the distance is calculated by the modified Bhattacharya measure. Afterward, each segment is adapted for mining the segment-based features and pixel-based features used for classification. Then, the feature vector is formed and given to the DBN classifier, which is tuned with the help of the introduced BirCat for brain tumor detection. The introduced technique effectively determines the regions with the tumor in the input MRI image. Finally, the change detection is evaluated by analyzing the post-operative MRI image and the segmented image by means of pixel mapping strategy with respect to SURF features. The pixel mapping is utilized to evaluate the percentage change in tumor pixels. The proposed BirCat surpassed other prevailing approaches by producing maximal values of specificity, accuracy, sensitivity, F1-score, and Dice score at 0.92, 0.927, 0.938, 0.909, and 0.937, correspondingly, for dataset 2.

Grid conditional random fields (CRFs) are widely applied in both natural and medical image segmentation tasks. However, they only consider the label coherence in neighborhood pixels or regions, which limits their ability to model long-range connections within the image and generally results in excessive smoothing of tumor boundaries. In this paper, we present a novel method for brain tumor segmentation in MR images based on fully-connected CRF (FC-CRF) model that establishes pairwise potentials on all pairs of pixels in the images. We employ a hierarchical approach to differentiate different structures of tumor and further formulate a FC-CRF model with learned data-driven prior knowledge of tumor core. The methods were evaluated on the testing and leaderboard set of Brain Tumor Image Segmentation Benchmark (BRATS) 2013 challenge. The precision of segmented tumor boundaries is improved significantly and the results are competitive compared to the start-of-the-arts.

This paper presents a comparative study of various classification methods in the application of automatic brain tumor segmentation. The data used in the study are 3D MRI volumes from MICCAI2016 brain tumor segmentation (BRATS) benchmark. 30 volumes are chosen randomly as a training set and 57 volumes are randomly chosen as a test set. The volumes are preprocessed and a feature vector is retrieved from each volume's four modalities (T1, T1 contrast-enhanced, T2 and Fluid-attenuated inversion recovery). The popular Dice score is used as an accuracy measure to record each classifier recognition results. All classifiers are implemented in the popular machine learning suit of algorithms, WEKA.

Gliomas are the most frequent primary brain tumors in adults. Improved quantification of the various aspects of a glioma requires accurate segmentation of the tumor in magnetic resonance images (MRI). Since the manual segmentation is time-consuming and subject to human error and irreproducibility, automatic segmentation has received a lot of attention in recent years. This paper presents a fully automated segmentation method which is capable of automatic segmentation of brain tumor from multi-modal MRI scans. The proposed method is comprised of a deeply-supervised neural network based on Holistically-Nested Edge Detection (HED) network. The HED method, which is originally developed for the binary classification task of image edge detection, is extended for multiple-class segmentation. The classes of interest include the whole tumor, tumor core, and enhancing tumor. The dataset provided by 2017 Multimodal Brain Tumor Image Segmentation Benchmark (BraTS) challenge is used in this work for training the neural network and performance evaluations. Experiments on BraTS 2017 challenge datasets demonstrate that the method performs well compared to the existing works. The assessments revealed the Dice scores of 0.86, 0.60, and 0.69 for whole tumor, tumor core, and enhancing tumor classes, respectively.

A cascade of fully convolutional neural networks is proposed to segment multi-modal Magnetic Resonance (MR) images with brain tumor into background and three hierarchical regions: whole tumor, tumor core and enhancing tumor core. The cascade is designed to decompose the multi-class segmentation problem into a sequence of three binary segmentation problems according to the subregion hierarchy. The whole tumor is segmented in the first step and the bounding box of the result is used for the tumor core segmentation in the second step. The enhancing tumor core is then segmented based on the bounding box of the tumor core segmentation result. Our networks consist of multiple layers of anisotropic and dilated convolution filters, and they are combined with multi-view fusion to reduce false positives. Residual connections and multi-scale predictions are employed in these networks to boost the segmentation performance. Experiments with BraTS 2017 validation set show that the proposed method achieved average Dice scores of 0.7859, 0.9050, 0.8378 for enhancing tumor core, whole tumor and tumor core, respectively. The corresponding values for BraTS 2017 testing set were 0.7831, 0.8739, and 0.7748, respectively.

The segmentation of brain tumor plays an important role in diagnosis, treatment planning, and surgical simulation. The precise segmentation of brain tumor can help clinicians obtain its location, size, and shape information. We propose a fully automatic brain tumor segmentation method based on kernel sparse coding. It is validated with 3D multiple-modality magnetic resonance imaging (MRI). In this method, MRI images are pre-processed first to reduce the noise, and then kernel dictionary learning is used to extract the nonlinear features to construct five adaptive dictionaries for healthy tissues, necrosis, edema, non-enhancing tumor, and enhancing tumor tissues. Sparse coding is performed on the feature vectors extracted from the original MRI images, which are a patch of m×m×m around the voxel. A kernel-clustering algorithm based on dictionary learning is developed to code the voxels. In the end, morphological filtering is used to fill in the area among multiple connected components to improve the segmentation quality. To assess the segmentation performance, the segmentation results are uploaded to the online evaluation system where the evaluation metrics dice score, positive predictive value (PPV), sensitivity, and kappa are used. The results demonstrate that the proposed method has good performance on the complete tumor region (dice: 0.83; PPV: 0.84; sensitivity: 0.82), while slightly worse performance on the tumor core (dice: 0.69; PPV: 0.76; sensitivity: 0.80) and enhancing tumor (dice: 0.58; PPV: 0.60; sensitivity: 0.65). It is competitive to the other groups in the brain tumor segmentation challenge. Therefore, it is a potential method in differentiation of healthy and pathological tissues.

Intracranial tumors are commonly known as brain tumors, which can be life-threatening in severe cases. Magnetic resonance imaging (MRI) is widely used in diagnosing brain tumors because of its harmless to the human body and high image resolution. Due to the heterogeneity of brain tumor height, MRI imaging is exceptionally irregular. How to accurately and quickly segment brain tumor MRI images is still one of the hottest topics in the medical image analysis community. However, according to the brain tumor segmentation algorithms, we could find now, most segmentation algorithms still stay in two-dimensional (2D) image segmentation, which could not obtain the spatial dependence between features effectively. In this study, we propose a brain tumor automatic segmentation method called scSE-NL V-Net. We try to use three-dimensional (3D) data as the model input and process the data by 3D convolution to get some relevance between dimensions. Meanwhile, we adopt non-local block as the self-attention block, which can reduce inherent image noise interference and make up for the lack of spatial dependence due to convolution. To improve the accuracy of convolutional neural network (CNN) image recognition, we add the “Spatial and Channel Squeeze-and-Excitation” Network (scSE-Net) to V-Net. The dataset used in this paper is from the brain tumor segmentation challenge 2020 database. In the test of the official BraTS2020 verification set, the Dice similarity coefficient is 0.65, 0.82, and 0.76 for the enhanced tumor (ET), whole tumor (WT), and tumor core (TC), respectively. Thereby, our model can make an auxiliary effect on the diagnosis of brain tumors established.