Reducing annotation burden in MR: A novel MR-contrast guided contrastive learning approach for image segmentation.

A Validation Framework for Brain Tumor Segmentation

Malignant brain tumor is a serious disease to human, causing cancer even death. Surgery is the main method for treatment of brain tumor. It’s a key issue to detect the existence and position of brain tumor, for doctors to analyze the brain image and make a plan for treatment. This task, brain tumor segmentation, is mainly performed on MRI (Magnetic Resonance Image). In recent years, machine learning and deep learning received extensive attention and developed rapidly. This passage introduces a new deep learning model, TransNUNet, to perform tumor segmentation task on brain MRI dataset. This model introduces attention mechanism, Cbam, to TransUNet, and refines loss function. Research shows that TransNUNet has a higher Dice score than other two image segmentation nets, U-Net and TransUNet, which proves its potential on this task.

Introduction: Magnetic resonance imaging (MRI) is the most common tool to examine glioblastoma. Preoperative MRI can be used to initial diagnosis, and surgery planning, while follow-up MRIs can be used to evaluate treatment responses, identify recurrency, and detect side effects. The follow-up MRIs are usually taken after the first-line therapy, such as maximal safe resection. In the past, radiologists manually segment the tumor regions from normal brain tissue on follow-up MRIs, which is time-consuming, error-prone, and challenging. Several deep-learning (DL) models have been developed utilizing preoperative images, but their performance has yet to be evaluated on follow-up MRIs. In this research, we built the largest follow-up MRI cohort (311 patients) to assess these DL models and their generalizability and performance on independent preoperative and follow-up images. We also made the first follow-up-based deep learning models for this specific task. Methods: All evaluation deep learning models (10 models) were trained by the Brain Tumor Segmentation challenge 2020 (BraTS’20) and evaluated by fifty pairs of preoperative and follow-up scans from our institution. The segmentation form our institution is evaluated by board certified radiologist. MRIs in the BraTS’20 dataset were all preoperative scans. After the evaluation, we randomly assigned 264 patients’ scans from our institution to the training dataset and 47 patients’ scans to the testing dataset. We compared three types of models in our follow-up deep learning model, including 1) UNet-3D, 2) UNet-3D+transfer-learning, and 3) UNet-3D+transfer-learning+baysian-learning. The benchmark for all models was the Dice similarity coefficient (DSC). DSC can measure the spatial overlap between model prediction and ground truth. The value of DSC is between zero to one. Zero means no overlap and one indicates complete overlap. Results: Our study demonstrates that the BraTS'20 trained models' performance decreased by 13.05% in independent preoperative MRI scans and 19.04% in follow-up MRI scans. The most significant mismatch regions were FLAIR hyperintense regions (3.68% drop in independent preoperative scans and 10.61% drop in independent follow-up scans) and Non-enhancing core (5.20% drop in independent preoperative scans and 11.99% drop in independent follow-up scans). Our best model can achieve the best DSC among three tumor regions compared to all evaluation models (FLAIR hyperintense regions: DSC 0.77 V.S. DSC 0.58; Enhancing tumor region: DSC 0.87 V.S. DSC 0.68; Non-enhancing tumor region: DSC 0.92 V.S. DSC 0.55). Conclusion: Maximal safe resection induced brain structure change, decreasing the performance of the preoperative-based DL model. Implementing a follow-up MRI-based segmentation model is essential to make accurate and generalizable results to address structural changes after maximal safe resection. Our follow-up DL model demonstrates the DSC score can be recovered. We commit to further developing the tool to assist radiologists in handling follow-up MRIs for glioblastoma patients. Citation Format: Kang Lin Hsieh, Tanjida Kabir, Luis Nunez-Rubiano, Yu-Chun Hsu, Yu Cai, Juan Rodriguez Quintero, Octavio Arevalo, Kangyi Zhao, Jackie Zhang, Jiguang Zhu Zhu, Roy Riascos, Xioaqian Jiang, Shayan Shams. A confident and operator-independent deep segmentation model to measure residual tumor volume in the follow-up MRIs for glioblastoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 2 (Clinical Trials and Late-Breaking Research); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(8_Suppl):Abstract nr LB067.

Segmentation and classification of brain tumor are time-consuming and challenging chore in clinical image processing. Magnetic Resonance Imaging (MRI) offers more information related to human soft tissues that assists in diagnosing brain tumor. Precise segmentation of the MRI images is vital to diagnose brain tumor by means of computer-aided medical tools. Afterwards suitable segmentation of MRI brain tumor images, tumor classification is performed that is a hard chore owing to complications. Therefore, Gannet Aquila Optimization Algorithm_deep maxout network (GAOA_DMN) and GAOA_K-Net+speech enhancement generative adversarial network (GAOA_K-Net+Segan) is presented for classification and segmentation of brain tumor utilizing MRI images. Here, pre-processing phase performs noise removal from input image utilizing the Laplacian filter and also the region of interest (ROI) extraction is also carried out. Then, segmentation of brain tumor is conducted by K-Net+Segan, which is combined by Motyka similarity. However, K-Net+Segan for segmentation is trained by GAOA that is an amalgamation of Gannet Optimization Algorithm (GOA) and Aquila Optimizer (AO). From segmented image, features are extracted for performing classification phase. At last, brain tumor classification is conducted by DMN, which is tuned by GAOA and thus, output is obtained. Furthermore, GAOA_K-Net+Segan obtained better outcomes in terms of segmentation accuracy whereas devised GAOA_DMN achieved maximum accuracy, true negative rate (TNR) and true positive rate (TPR) of 92.7%, 94.5% and 91.5%.

Brain tumors are one of the most common neurological complications that affect millions each year. Current studies have focused only on either the segmentation of brain tumors or healthy anatomical regions. Unfortunately, there is a lack of research on whole brain segmentation of patients with existing tumors. The segmentation of these regions is cruical for understanding patient pathologies. Due to the distortions in the brain caused by tumors, whole-brain segmentation models trained on healthy brains will not perform accurately on diseased ones. To address this issue, this paper proposes a novel unified 3D nnU-Net model for the segmentation of both brain tumor and whole-brain structures for the first time. Training a model on datasets that include both healthy and cancerous brain regions could enhance the generalizability of machine learning (ML) models for performing both whole-brain segmentation and tumor segmentation tasks. Currently, there are no datasets that are publicly available for this purpose. This paper proposes to address this challenge by augmenting data from OASIS-1 and BraTS2023 to train the baseline model. The model successfully achieves a Dice Similarity Coefficient (DSC) of 0.814 for whole brain structures and 0.874 for tumor segmentation, demonstrating its effectiveness across diverse regions.

This study aims to develop an end-to-end deep learning (DL) model to predict neoadjuvant chemotherapy (NACT) response in osteosarcoma (OS) patients using routine magnetic resonance imaging (MRI). We retrospectively analyzed data from 112 patients with histologically confirmed OS who underwent NACT prior to surgery. Multi-sequence MRI data (including T2-weighted and contrast-enhanced T1-weighted images) and physician annotations were utilized to construct an end-to-end DL model. The model integrates ResUNet for automatic tumor segmentation and 3D-ResNet-18 for predicting NACT efficacy. Model performance was assessed using area under the curve (AUC) and accuracy (ACC). Among the 112 patients, 51 exhibited a good NACT response, while 61 showed a poor response. No statistically significant differences were found in age, sex, alkaline phosphatase levels, tumor size, or location between these groups (P > 0.05). The ResUNet model achieved robust performance, with an average Dice coefficient of 0.579 and average Intersection over Union (IoU) of 0.463. The T2-weighted 3D-ResNet-18 classification model demonstrated superior performance in the test set with an AUC of 0.902 (95% CI: 0.766-1), ACC of 0.783, sensitivity of 0.909, specificity of 0.667, and F1 score of 0.800. Our proposed end-to-end DL model can effectively predict NACT response in OS patients using routine MRI, offering a potential tool for clinical decision-making.

In this work, we propose a fully automatic brain tumor and edema segmentation technique in brain magnetic resonance (MR) images. Different brain tissues are characterized using the novel texture features such as piece-wise triangular prism surface area (PTPSA), multi-fractional Brownian motion (mBm) and Gabor-like textons, along with regular intensity and intensity difference features. Classical Random Forest (RF) classifier is used to formulate the segmentation task as classification of these features in multi-modal MRIs. The segmentation performance is compared with other state-of-art works using a publicly available dataset known as Brain Tumor Segmentation (BRATS) 2012 [1]. Quantitative evaluation is done using the online evaluation tool from Kitware/MIDAS website [2]. The results show that our segmentation performance is more consistent and, on the average, outperforms other state-of-the art works in both training and challenge cases in the BRATS competition.

In this paper, we look at how to design and build a system to find tumors using 2 Convolutional Neural Network (CNN) models. With the help of digital image processing and deep Learning, we can make a system that automatically diagnoses and finds different diseases and abnormalities. The tumor detection system may include image enhancement, segmentation, data enhancement, feature extraction, and classification. These options are set up so that the CNN model can give the best results. During the training phase, the learning rate is used to change the weights and bias. The learning rate also changes the weights. One Epoch is when all of the training images are shown to the model. As the training data may be very large, the data in each epoch are split into batches. Every epoch has a training session and a test session. After each epoch, the weights are changed based on how fast the CNN is learning. This is done with the help of optimization algorithms. The suggested technique uses the anticipated mean intersection over union value to identify failure instances in addition to forecasting the mean intersection over union. This paper talks about how to separate brain tumors from magnetic resonance images of patients taken from "Brain web." Using basic ideas of digital image processing, magnetic resonance images are used to extract and find tumors using a hybrid method. In this paper, the proposed algorithm is applied with the help of MATLAB. In medical image processing, brain tumor segmentation is an important task. The goal of this paper is to look at different ways to divide brain tumors using magnetic resonance imaging. Recently, automatic segmentation using deep learning methods has become popular because these methods get the best results and are better at solving this problem than others. Deep learning methods can also be used to process and evaluate large amounts of magnetic resonance imaging image data quickly and objectively. A classification method based on a convolution neural network is also added to the proposed scheme to make it more accurate and cut down on the amount of time it takes to do the calculations. Also, the results of the classification are given as images of a tumor or a healthy brain. The training is 98.5% correct. In the same way, both the validation accuracy and validation loss are high.

Segmentation of brain tumor from magnetic resonance imaging is a challenging and time-consuming task due to the unpredictable appearance of tumor tissue in practical applications. In this paper we propose a novel level-set-based model for tumor segmentation from multi-modality magnetic resonance imaging. We formulate the tumor segmentation on pixel-level with three classes: tumor, edema and healthy tissue. First, we detect abnormal regions by using a region-based active contour model on T2-weighted images with fluid-attenuated inversion recovery modality, and a variational level set formulation is applied locally to approximate the image intensities on two sides of the contour. In the second stage, we distinguish the edema and tumor tissues in the abnormal regions based on the contrast enhancement T1 modality. Compared with traditional one-modality methods, our model can better represent the specific tissue of tumor in a simple way. The validation experiments on synthetic and clinical brain magnetic resonance images demonstrate the effectiveness and simplicity of the proposed method for brain tumor segmentation.

Brain tumors pose a significant clinical challenge due to their aggressive nature and the complexity of accurate diagnosis. Traditional deep learning models, primarily based on Convolutional Neural Networks (CNNs), have shown promise in tumor segmentation and classification from Magnetic Resonance Imaging (MRI) scans. However, these approaches often struggle with capturing global spatial dependencies, generalizing from limited annotated data, and fine-tuning performance across varying imaging conditions. This paper presents an advanced hybrid framework that combines self-supervised learning, Vision Transformer (ViT), and Quantum Particle Swarm Optimization (QPSO) to enhance segmentation and classification of brain tumors. The methodology begins with preprocessing involving skull stripping, contrast enhancement, and denoising using a Cycle-Consistent Generative Adversarial Network (CycleGAN). Feature representation is learned through a Simple Framework for Contrastive Learning of Visual Representations (SimCLR), followed by segmentation using a DeepLabv3+ architecture with a ViT encoder. Tumor classification is handled by a lightweight ViT model, whose hyperparameters are optimized using QPSO. Evaluation on the Brain Tumor Segmentation (BraTS) 2021 dataset demonstrates a Dice Similarity Coefficient (DSC) of 95.4% and classification accuracy of 96.7%, confirming the robustness and superiority of the proposed approach over conventional methods.

Brain tumor segmentation in magnetic resonance imaging (MRI) is considered a complex procedure because of the variability of tumor shapes and the complexity of determining the tumor location, size, and texture. Manual tumor segmentation is a time-consuming task highly prone to human error. Hence, this study proposes an automated method that can identify tumor slices and segment the tumor across all image slices in volumetric MRI brain scans. First, a set of algorithms in the pre-processing stage is used to clean and standardize the collected data. A modified gray-level co-occurrence matrix and Analysis of Variance (ANOVA) are employed for feature extraction and feature selection, respectively. A multi-layer perceptron neural network is adopted as a classifier, and a bounding 3D-box-based genetic algorithm is used to identify the location of pathological tissues in the MRI slices. Finally, the 3D active contour without edge is applied to segment the brain tumors in volumetric MRI scans. The experimental dataset consists of 165 patient images collected from the MRI Unit of Al-Kadhimiya Teaching Hospital in Iraq. Results of the tumor segmentation achieved an accuracy of 89% ± 4.7% compared with manual processes.

Dual examiner consistency learning with dynamic receptive fields and class-balance refinement for Barely-supervised brain tumor segmentation

Background: Magnetic resonance (MR) image guided radiation therapy system, enables real time MR guided radiotherapy (RT) without additional radiation exposure to patients during treatment. However, MR image lacks electron density information required for dose calculation. Image fusion algorithm with deformable registration between MR and computed tomography (CT) was developed to solve this issue. However, delivered dose may be different due to volumetric changes during image registration process. In this respect, synthetic CT generated from the MR image would provide more accurate information required for the real time RT. Materials and Methods: We analyzed 1,209 MR images from 16 patients who underwent MR guided RT. Structures were divided into five tissue types, air, lung, fat, soft tissue and bone, according to the Hounsfield unit of deformed CT. Using the deep learning model (U-NET model), synthetic CT images were generated from the MR images acquired during RT. This synthetic CT images were compared to deformed CT generated using the deformable registration. Pixelto- pixel match was conducted to compare the synthetic and deformed CT images. Results and Discussion: In two test image sets, average pixel match rate per section was more than 70% (67.9 to 80.3% and 60.1 to 79%; synthetic CT pixel/deformed planning CT pixel) and the average pixel match rate in the entire patient image set was 69.8%. Conclusion: The synthetic CT generated from the MR images were comparable to deformed CT, suggesting possible use for real time RT. Deep learning model may further improve match rate of synthetic CT with larger MR imaging data.

BackgroundCurrently, there is no fully automated tool available for evaluating the degree of cervical spinal stenosis. The aim of this study was to develop and validate the use of artificial intelligence (AI) algorithms for the assessment of cervical spinal stenosis.MethodsIn this retrospective multi-center study, cervical spine magnetic resonance imaging (MRI) scans obtained from July 2020 to June 2023 were included. Studies of patients with spinal instrumentation or studies with suboptimal image quality were excluded. Sagittal T2-weighted images were used. The training data from the Fourth People’s Hospital of Shanghai (Hos. 1) and Shanghai Changzheng Hospital (Hos. 2) were annotated by two musculoskeletal (MSK) radiologists following Kang’s system as the standard reference. First, a convolutional neural network (CNN) was trained to detect the region of interest (ROI), with a second Transformer for classification. The performance of the deep learning (DL) model was assessed on an internal test set from Hos. 2 and an external test set from Shanghai Changhai Hospital (Hos. 3), and compared among six readers. Metrics such as detection precision, interrater agreement, sensitivity (SEN), and specificity (SPE) were calculated.ResultsOverall, 795 patients were analyzed (mean age ± standard deviation, 55±14 years; 346 female), with 589 in the training (75%) and validation (25%) sets, 206 in the internal test set, and 95 in the external test set. Four tasks with different clinical application scenarios were trained, and their accuracy (ACC) ranged from 0.8993 to 0.9532. When using a Kang system score of ≥2 as a threshold for diagnosing central cervical canal stenosis in the internal test set, both the algorithm and six readers achieved similar areas under the receiver operating characteristic curve (AUCs) of 0.936 [95% confidence interval (CI): 0.916–0.955], with a SEN of 90.3% and SPE of 93.8%; the AUC of the DL model was 0.931 (95% CI: 0.917–0.946), with a SEN in the external test set of 100%, and a SPE of 86.3%. Correlation analysis comparing the DL method, the six readers, and MRI reports between the reference standard showed a moderate correlation, with R values ranging from 0.589 to 0.668. The DL model produced approximately the same upgrades (9.2%) and downgrades (5.1%) as the six readers.ConclusionsThe DL model could fully automatically and reliably assess cervical canal stenosis using MRI scans.

Brain tumor segmentation and classification (BTSC) from magnetic resonance imaging (MRI) is an important procedure in medical predictions, improving the precision and speed of treatment plans. However, the automatic detection of tumors is challenging because of similar intensity patterns, indistinct boundaries, and varying tumor shapes. This paper suggests a novel deep transfer learning (DTL) methodology to accurately segment and detect brain tumors (BTs) from MRI. Initially, the denoising is carried out using improved anisotropic diffusion filtering (IADF), and then dataset diversity is enhanced by applying data augmentation. Afterwards, a Swin Transformer U-shaped network with dense connections (STUNetDS) is utilized for segmenting the tumor, and finally, Kullback–Leibler loss with Optimized InceptionResNetv2 (KLOINRNetv2) is used for the classification of the tumor. The system uses the Multi-Strategy Improved Kookaburra Optimization (MSIKBO) algorithm for tuning the network parameters. The model achieves 98.67% accuracy for detecting and classifying BTs from the Kaggle BraTS2019 dataset, higher compared to the baseline techniques. The reliable and scalable solutions are offered by the proposed model, making it more suitable for automated tumor assessment in clinical applications.