Interlayer information fusion-based and dual-attention improved U-Net for ABVS image sequence intelligent tumor segmentation

Convolutional neural networks have achieved excellent results in automatic medical image segmentation. In this study, we proposed a novel three-dimensional (3D) multipath DenseNet for generating the accurate glioblastoma (GBM) tumor contour from four multimodal pre-operative MR images. We hypothesized that the multipath architecture could achieve more accurate segmentation than a singlepath architecture. Two hundred and fifty-eight GBM patients were included in this study. Each patient had four MR images (T1-weighted, contrast-enhanced T1-weighted, T2-weighted, and FLAIR) and the manually segmented tumor contour. We built a 3D multipath DenseNet that could be trained to achieve an end-to-end mapping from four MR images to the corresponding GBM tumor contour. A 3D singlepath DenseNet was also built for comparison. Both DenseNets were based on the encoder-decoder architecture. All four images were concatenated and fed into a single encoder path in the singlepath DenseNet, while each input image had its own encoder path in the multipath DenseNet. The patient cohort was randomly split into a training set of 180 patients, a validation set of 39 patients, and a testing set of 39 patients. Model performance was evaluated using the Dice similarity coefficient (DSC), average surface distance (ASD), and 95% Hausdorff distance (HD95% ). Wilcoxon signed-rank tests were conducted to assess statistical significances. The singlepath DenseNet achieved the DSC of 0.911±0.060, ASD of 1.3±0.7mm, and HD95% of 5.2±7.1mm, while the multipath DenseNet achieved the DSC of 0.922±0.041, ASD of 1.1±0.5mm, and HD95% of 3.9±3.3mm. The P-values of all Wilcoxon signed-rank tests were less than 0.05. Both DenseNets generated GBM tumor contours in good agreement with the manually segmented contours from multimodal MR images. The multipath DenseNet achieved more accurate tumor segmentation than the singlepath DenseNet. Here presented the 3D multipath DenseNet that demonstrated an improved accuracy over comparable algorithms in the clinical task of GBM tumor segmentation.

Abstract: Accurate tumor segmentation from medical images is critical for diagnosis, treatment planning, and prognosis assessment in oncology. With the advent of deep learning, U-Net and its variants have demonstrated remarkable performance in medical image segmentation tasks. This study investigates the comparative effectiveness of U-Net variants—Vanilla U-Net, Attention U-Net, and Residual U-Net—on publicly available datasets such as BraTS and ISIC. By applying various preprocessing techniques and using Dice Similarity Coefficient (DSC) and Intersection over Union (IoU) for evaluation, we identify significant improvements in segmentation accuracy. Our findings show that the Attention U-Net outperforms others, achieving an average DSC of 0.89 and IoU of 0.83, suggesting its suitability for clinical deployment. Keywords: Medical imaging, tumor segmentation, U-Net, deep learning, Attention U-Net, Residual U-Net, BraTS, ISIC, DSC, IoU

Brain tumor segmentation is the most important way to separate tumor areas from healthy brain tissue in medical imaging. This is necessary for making an accurate diagnosis and planning treatment. But building strong deep learning models is often hard because there isn't much labeled medical data available, and strict privacy rules stop data from being shared in one place. Federated Learning (FL) helps keep patient data private by keeping it local, but its performance often drops when data from different hospitals have big differences in quality, imaging protocols, and distribution. Our research seeks to create a privacy-preserving federated learning framework that adeptly manages significant data heterogeneity while ensuring high segmentation accuracy across various institutions. We propose a new two-stage FL framework that allows multiple institutions to work together while keeping their privacy and effectively dealing with complicated non-IID data distributions. To start, we use a Federated Masked Autoencoder (MAE) for self-supervised pre-training. This lets the model learn strong anatomical features from unlabeled MRI scans. Second, the model is carefully fine-tuned using an Attention ResUNet3D architecture to get very accurate tumor segmentation. We use the SCAFFOLD optimization algorithm to keep training stable across all clients, even when the scanner varies from site to site, thereby directly addressing client drift. We also use strategic foreground-biased sampling and Test-Time Augmentation (TTA) techniques to greatly improve segmentation accuracy in difficult, uneven tumor sub-regions. We ran extensive experiments on the BraTS 2024 dataset in simulated federated settings with 10, 50, and 100 different clients. The Dice coefficients we got were 0.826, 0.824, and 0.818, which demonstrate strong performance. In the end, these strong results show that the suggested method works well on a larger scale and can be used in a clinical setting.

Achieving accurate bladder wall and tumor segmentation from MRI is critical for diagnosing and treating bladder cancer. However, automated segmentation remains challenging due to factors such as comparable density distributions, intricate tumor morphologies, and unclear boundaries. Considering the attributes of bladder MRI images, we propose an efficient multiscale hierarchical hybrid attention with a transformer (MH2AFormer) for bladder cancer and wall segmentation. Specifically, a multiscale hybrid attention and transformer (MHAT) module in the encoder is designed to adaptively extract and aggregate multiscale hybrid feature representations from the input image. In the decoder stage, we devise a multiscale hybrid attention (MHA) module to generate high-quality segmentation results from multiscale hybrid features. Combining these modules enhances the feature representation and guides the model to focus on tumor and wall regions, which helps to solve bladder image segmentation challenges. Moreover, MHAT utilizes the Fast Fourier Transformer with a large kernel (e.g., 224 × 224) to model global feature relationships while reducing computational complexity in the encoding stage. The model performance was evaluated on two datasets. As a result, the model achieves relatively best results regarding the intersection over union (IoU) and dice similarity coefficient (DSC) on both datasets (Dataset A: IoU = 80.26%, DSC = 88.20%; Dataset B: IoU = 89.74%, DSC = 94.48%). These advantageous outcomes substantiate the practical utility of our approach, highlighting its potential to alleviate the workload of radiologists when applied in clinical settings.

Objective: Segmentation of liver and liver tumors from 3D medical images is a challenging and computationally expensive task. Organs that are in close proximity may have similar shape, texture, and intensity, which makes it difficult for accurate segmentation. Accurate segmentation of liver tumors is important for diagnosis and treatment planning of liver cancer. Methods: A hybrid model with a U-Net based structure and the Whale Optimization Algorithm (WOA) was proposed. WOA was used to optimize the hyperparameters of the conventional LiTS-Res-UNet to obtain the best segmentation performance of the deep learning model. Results: The LiTS-Res-Unet + WOA hybrid model achieved a performance of 99.54% for accuracy, with a Dice coefficient of 92.38% and a Jaccard index of 86.73% on the benchmark dataset, outperforming state-of-the-art methods. Conclusions: The WOA-based adaptive search space was able to obtain an optimal set of hyperparameters for deep learning model convergence while increasing the accuracy of the model in the proposed hybrid model. The robust performance and clinical applicability of the model in liver tumor segmentation were demonstrated.

BackgroundEsophageal squamous cell carcinoma (ESCC) is a highly prevalent and aggressive malignancy associated with poor prognosis. Accurate tumor segmentation is critical for the detection and characterization of ESCC. However, manual segmentation remains labor-intensive and time-consuming. Deep learning presents a promising approach for automating this process.Materials and methodsThis retrospective study assessed the performance of three deep learning models—UNet, nnUNet-2D/3D, and UMamba—for ESCC segmentation using contrast-enhanced free-breathing 3DGRE MRI with 1 mm isotropic resolution. The dataset comprised 192 patients, divided into 171 for training and 21 for validation. Manual annotations excluded calcifications and hemorrhage, with maximum tumor cross-sections utilized for 2D model training and full volumetric tumor layers employed for 3D architectures. Segmentation performance was evaluated using the Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and mean surface distance (MSD).ResultUMamba achieved the highest DSC of 0.764, significantly outperforming nnUNet-3D (0.738) and nnUNet-2D (0.702). Additionally, UMamba demonstrated superior boundary delineation, as evidenced by the lowest HD of 5.048 mm and MSD of 1.088 mm. These results highlight the importance of 3D contextual analysis over traditional 2D approaches, as illustrated by the 5.1% improvement in DSC observed with nnUNet-3D compared to nnUNet-2D. Furthermore, the computational efficiency of each model was evaluated, revealing that 3D models exhibited longer inference times than their 2D counterparts.ConclusionUMamba demonstrates superior performance for automated ESCC segmentation on high-resolution MRI, providing a robust tool that has the potential to reduce manual segmentation time and inter-observer variability in clinical practice.

Three-dimensional (3D) liver tumor segmentation from Computed Tomography (CT) images is a prerequisite for computer-aided diagnosis, treatment planning, and monitoring of liver cancer. Despite many years of research, 3D liver tumor segmentation remains a challenging task. In this paper, we propose an effective and efficient method for tumor segmentation in liver CT images using encoder-decoder based octave convolution networks. Compared with other convolution networks utilizing standard convolution for feature extraction, the proposed method utilizes octave convolutions for learning multiple-spatial-frequency features, thus can better capture tumors with varying sizes and shapes. The proposed network takes advantage of a fully convolutional architecture which performs efficient end-to-end learning and inference. More importantly, we introduce a deep supervision mechanism during the learning process to combat potential optimization difficulties, and thus the model can acquire a much faster convergence rate and more powerful discrimination capability. Finally, we integrate octave convolutions into the encoder-decoder architecture of UNet, which can generate high resolution tumor segmentation in one single forward feeding without post-processing steps. Both architectures are trained on a subset of the LiTS (Liver Tumor Segmentation) Challenge. The proposed approach is shown to significantly outperform other networks in terms of various accuracy measures and processing speed.

Accurate kidney and tumor segmentation of computed tomography (CT) scans is vital for diagnosis and treatment, but manual methods are time-consuming and inconsistent, highlighting the value of AI automation. This study develops a fully automated AI model using vision transformers (ViTs) and convolutional neural networks (CNNs) to detect and segment kidneys and kidney tumors in Contrast-Enhanced (CECT) scans, with a focus on improving sensitivity for small, indistinct tumors. The segmentation framework employs a ViT-based model for the kidney organ, followed by a 3D UNet model with enhanced connections and attention mechanisms for tumor detection and segmentation. Two CECT datasets were used: a public dataset (KiTS23: 489 scans) and a private institutional dataset (Private: 592 scans). The AI model was trained on 389 public scans, with validation performed on the remaining 100 scans and external validation performed on all 592 private scans. Tumors were categorized by TNM staging as small (≤4 cm) (KiTS23: 54%, Private: 41%), medium (>4 cm to ≤7 cm) (KiTS23: 24%, Private: 35%), and large (>7 cm) (KiTS23: 22%, Private: 24%) for detailed evaluation. Kidney and kidney tumor segmentations were evaluated against manual annotations as the reference standard. The model achieved a Dice score of 0.97 ± 0.02 for kidney organ segmentation. For tumor detection and segmentation on the KiTS23 dataset, the sensitivities and average false-positive rates per patient were as follows: 0.90 and 0.23 for small tumors, 1.0 and 0.08 for medium tumors, and 0.96 and 0.04 for large tumors. The corresponding Dice scores were 0.84 ± 0.11, 0.89 ± 0.07, and 0.91 ± 0.06, respectively. External validation on the private data confirmed the model's effectiveness, achieving the following sensitivities and average false-positive rates per patient: 0.89 and 0.15 for small tumors, 0.99 and 0.03 for medium tumors, and 1.0 and 0.01 for large tumors. The corresponding Dice scores were 0.84 ± 0.08, 0.89 ± 0.08, and 0.92 ± 0.06. The proposed model demonstrates consistent and robust performance in segmenting kidneys and kidney tumors of various sizes, with effective generalization to unseen data. This underscores the model's significant potential for clinical integration, offering enhanced diagnostic precision and reliability in radiological assessments.

Segmentation of liver tumors from Computed Tomography (CT) and tumor burden analysis play an important role in the choice of therapeutic strategies for liver diseases and treatment monitoring. In this paper, a new segmentation method for liver tumors from contrast-enhanced CT imaging is proposed. As manual segmentation of tumors for liver treatment planning is both labor intensive and time-consuming, a highly accurate automatic tumor segmentation is desired. The proposed framework is fully automatic requiring no user interaction. The proposed segmentation evaluated on real-world clinical data from patients is based on a hybrid method integrating cuckoo optimization and fuzzy c-means algorithm with random walkers algorithm. The accuracy of the proposed method was validated using a clinical liver dataset containing one of the highest numbers of tumors utilized for liver tumor segmentation containing 127 tumors in total with further validation of the results by a consultant radiologist. The proposed method was able to achieve one of the highest accuracies reported in the literature for liver tumor segmentation compared to other segmentation methods with a mean overlap error of 22.78 % and dice similarity coefficient of 0.75 in 3Dircadb dataset and a mean overlap error of 15.61 % and dice similarity coefficient of 0.81 in MIDAS dataset. The proposed method was able to outperform most other tumor segmentation methods reported in the literature while representing an overlap error improvement of 6 % compared to one of the best performing automatic methods in the literature. The proposed framework was able to provide consistently accurate results considering the number of tumors and the variations in tumor contrast enhancements and tumor appearances while the tumor burden was estimated with a mean error of 0.84 % in 3Dircadb dataset.

Renal cancer is one of ten most common cancers in human beings. The laparoscopic partial nephrectomy (LPN) becomes the main therapeutic approach in treating renal cancer. Accurate kidney and tumor segmentation in CT images is a prerequisite step in the surgery planning. However, automatic and accurate kidney and renal tumor segmentation in CT images remains a challenge. In this paper, we propose a new method to perform a precise segmentation of kidney and renal tumor in CT angiography images. This method relies on a three-dimensional (3D) fully convolutional network (FCN) which combines a pyramid pooling module (PPM). The proposed network is implemented as an end-to-end learning system directly on 3D volumetric images. It can make use of the 3D spatial contextual information to improve the segmentation of the kidney as well as the tumor lesion. The experiments conducted on 140 patients show that these target structures can be segmented with a high accuracy. The resulting average dice coefficients obtained for kidney and renal tumor are equal to 0.931 and 0.802 respectively. These values are higher than those obtained from the other two neural networks.

Application of the bicharacteristic attention residual pyramid for the treatment of brain tumors

Automated segmentation and source prediction of bone tumors using ConvNeXtv2 Fusion based Mask R-CNN to identify lung cancer metastasis

Magnetic resonance imaging (MRI) scans are known to suffer from a variety of acquisition artifacts as well as equipment-based variations that impact image appearance and segmentation performance. It is still unclear whether a direct relationship exists between magnetic resonance (MR) image quality metrics (IQMs) (e.g., signal-to-noise, contrast-to-noise) and segmentation accuracy. Deep learning (DL) approaches have shown significant promise for automated segmentation of brain tumors on MRI but depend on the quality of input training images. We sought to evaluate the relationship between IQMs of input training images and DL-based brain tumor segmentation accuracy toward developing more generalizable models for multi-institutional data. We trained a 3D DenseNet model on the BraTS 2020 cohorts for segmentation of tumor subregions enhancing tumor (ET), peritumoral edematous, and necrotic and non-ET on MRI; with performance quantified via a 5-fold cross-validated Dice coefficient. MRI scans were evaluated through the open-source quality control tool MRQy, to yield 13 IQMs per scan. The Pearson correlation coefficient was computed between whole tumor (WT) dice values and IQM measures in the training cohorts to identify quality measures most correlated with segmentation performance. Each selected IQM was used to group MRI scans as "better" quality (BQ) or "worse" quality (WQ), via relative thresholding. Segmentation performance was re-evaluated for the DenseNet model when (i) training on BQ MRI images with validation on WQ images, as well as (ii) training on WQ images, and validation on BQ images. Trends were further validated on independent test sets derived from the BraTS 2021 training cohorts. For this study, multimodal MRI scans from the BraTS 2020 training cohorts were used to train the segmentation model and validated on independent test sets derived from the BraTS 2021 cohort. Among the selected IQMs, models trained on BQ images based on inhomogeneity measurements (coefficient of variance, coefficient of joint variation, coefficient of variation of the foreground patch) and the models trained on WQ images based on noise measurement peak signal-to-noise ratio (SNR) yielded significantly improved tumor segmentation accuracy compared to their inverse models. Our results suggest that a significant correlation may exist between specific MR IQMs and DenseNet-based brain tumor segmentation performance. The selection of MRI scans for model training based on IQMs may yield more accurate and generalizable models in unseen validation.

Gliomas are the most common primary brain malignancies. Accurate and robust tumor segmentation is important for diagnosis, treatment planning and risk factor identification. In this paper, we present a deep learning-based framework for brain tumor segmentation, using multimodal MRI scans. A 3D U-net based deep learning model has been trained for the brain tumor segmentation task. Our architecture is essentially a deeply-supervised encoder-decoder network, where the encoder and decoder sub-networks are connected through a series of nested, dense skip paths. We train our model on the BraTS 2019 dataset. Dice coefficients for enhancing tumor, tumor core, and the whole tumor are 0.757, 0.817 and 0.891 respectively on the validation dataset.

Accurate tumor segmentation in PET is crucial in many oncology applications. We developed an adaptive region-growing (ARG) algorithm with a maximum curvature strategy (ARG_MC) for tumor segmentation in PET. The ARG_MC repeatedly applied a confidence connected region-growing algorithm with increasing relaxing factor f. The optimal relaxing factor (ORF) was then determined at the transition point on the f-volume curve, where the volume just grew from the tumor into the surrounding normal tissues. The ARG_MC along with five widely used algorithms were tested on a phantom with 6 spheres at different signal to background ratios and on two clinic datasets including 20 patients with esophageal cancer and 11 patients with non-Hodgkin lymphoma (NHL). The ARG_MC did not require any phantom calibration or any a priori knowledge of the tumor or PET scanner. The identified ORF varied with tumor types (mean ORF = 9.61, 3.78 and 2.55 respectively for the phantom, esophageal cancer, and NHL datasets), and varied from one tumor to another. For the phantom, the ARG_MC ranked the second in segmentation accuracy with an average Dice similarity index (DSI) of 0.86, only slightly worse than Daisne’s adaptive thresholding method (DSI = 0.87), which required phantom calibration. For both the esophageal cancer dataset and the NHL dataset, the ARG_MC had the highest accuracy with an average DSI of 0.87 and 0.84, respectively. The ARG_MC was robust to parameter settings and region of interest selection, and it did not depend on scanners, imaging protocols, or tumor types. Furthermore, the ARG_MC made no assumption about the tumor size or tumor uptake distribution, making it suitable for segmenting tumors with heterogeneous FDG uptake. In conclusion, the ARG_MC was accurate, robust and easy to use, it provides a highly potential tool for PET tumor segmentation in clinic.