Automatic tumor segmentation in 3D automated breast ultrasound using convolutional neural network

Due to the low contrast, blurry boundaries, and large amount of shadows in breast ultrasound (BUS) images, automatic tumor segmentation remains a challenging task. Deep learning provides a solution to this problem, since it can effectively extract representative features from lesions and the background in BUS images. A novel automatic tumor segmentation method is proposed by combining a dilated fully convolutional network (DFCN) with a phase-based active contour (PBAC) model. The DFCN is an improved fully convolutional neural network with dilated convolution in deeper layers, fewer parameters, and batch normalization techniques; and has a large receptive field that can separate tumors from background. The predictions made by the DFCN are relatively rough due to blurry boundaries and variations in tumor sizes; thus, the PBAC model, which adds both region-based and phase-based energy functions, is applied to further improve segmentation results. The DFCN model is trained and tested in dataset 1 which contains 570BUS images from 89 patients. In dataset 2, a 10-fold support vector machine (SVM) classifier is employed to verify the diagnostic ability using 460 features extracted from the segmentation results of the proposed method. Advantages of the present method were compared with three state-of-the-art networks; the FCN-8s, U-net, and dilated residual network (DRN). Experimental results from 170BUS images show that the proposed method had a Dice Similarity coefficient of 88.97±10.01%, a Hausdorff distance (HD) of 35.54±29.70 pixels, and a mean absolute deviation (MAD) of 7.67±6.67 pixels, which showed the best segmentation performance. In dataset 2, the area under curve (AUC) of the 10-fold SVM classifier was 0.795 which is similar to the classification using the manual segmentation results. The proposed automatic method may be sufficiently accurate, robust, and efficient for medical ultrasound applications.

Manual segmentation in lumbar paraspinal muscle studies is time-consuming, subject to variability, and limits comprehensive evaluation. To solve the problems, we aim to develop a deep learning algorithm for the automatic segmentation and multi-parameter quantification of eight paraspinal muscles in lumbar CT. We collected CT images covering the range from the superior endplate of the L1 vertebra to the inferior endplate of the S1 vertebra. The dataset was partitioned into training, validation, and test sets in a ratio of 7:1:2. Six convolutional neural networks were employed to automatically segment the bilateral psoas major, quadratus lumborum, erector spinae, and multifidus across all slices. Model performance was evaluated using the Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and mean Intersection over Union (mIoU). Muscle parameters including cross-sectional area, muscle volume, fat infiltration, CT density, and paraspinal muscle index were calculated. The Intraclass Correlation Coefficient [ICC(3,1)] was used to assess agreement between model's segmentations and manual ground truth. 100 lumbar CT scans (age, 55.02 ± 16.2; 62 female) were used for model development. The mean DSC across all six models ranged from 0.754 to 0.903. TransUNet demonstrated the best overall (DSC, 0.903; 95% CI: 0.902, 0.913) (HD, 9.934mm; 95% CI: 9.815, 11.902) (mIoU, 0.841; 95% CI: 0.839, 0.850). For individual muscles segmented by TransUNet, the highest DSC was 0.936 (95% CI: 0.934-0.938) for the right multifidus. The mean ICC for all parameters was 0.931. The highest ICC value was 0.999 (p < 0.001). The deep learning tool enables automatic and accurate segmentation of eight lumbar paraspinal muscles. Muscle parameter measurements derived from model's segmentations and manual ground truth exhibit high agreement. Accurate segmentation and quantification of paraspinal muscle parameters facilitates large-scale epidemiological studies of paraspinal muscles and spine-related diseases, overcoming the limitations in manual methods.

PurposeThe aim of this study was to develop and test a post-processing technique for detection and classification of lesions according to the BI-RADS atlas in automated breast ultrasound (ABUS) based on deep convolutional neural networks (dCNNs).Methods and materialsIn this retrospective study, 645 ABUS datasets from 113 patients were included; 55 patients had lesions classified as high malignancy probability. Lesions were categorized in BI-RADS 2 (no suspicion of malignancy), BI-RADS 3 (probability of malignancy < 3%), and BI-RADS 4/5 (probability of malignancy > 3%). A deep convolutional neural network was trained after data augmentation with images of lesions and normal breast tissue, and a sliding-window approach for lesion detection was implemented. The algorithm was applied to a test dataset containing 128 images and performance was compared with readings of 2 experienced radiologists.ResultsResults of calculations performed on single images showed accuracy of 79.7% and AUC of 0.91 [95% CI: 0.85–0.96] in categorization according to BI-RADS. Moderate agreement between dCNN and ground truth has been achieved (κ: 0.57 [95% CI: 0.50–0.64]) what is comparable with human readers. Analysis of whole dataset improved categorization accuracy to 90.9% and AUC of 0.91 [95% CI: 0.77–1.00], while achieving almost perfect agreement with ground truth (κ: 0.82 [95% CI: 0.69–0.95]), performing on par with human readers. Furthermore, the object localization technique allowed the detection of lesion position slice-wise.ConclusionsOur results show that a dCNN can be trained to detect and distinguish lesions in ABUS according to the BI-RADS classification with similar accuracy as experienced radiologists.Key Points• A deep convolutional neural network (dCNN) was trained for classification of ABUS lesions according to the BI-RADS atlas.• A sliding-window approach allows accurate automatic detection and classification of lesions in ABUS examinations.

Automatic superpixel-based segmentation method for breast ultrasound images

Automatic Breast Ultrasound (ABUS) image segmentation is of great significance for breast cancer diagnosis and treatment. However, similar to most medical datasets, ABUS image datasets are often small-scale and seriously imbalanced, which makes ABUS image segmentation become a challenge. To solve this problem, we propose the Residual Swin Transformer Unet with Consistency Regularization (RSTUnet-CR) which can make full use of non-lesion and unlabeled images for high-precision tumor segmentation on ABUS images. We design a consistency-regularization decoder to reconstruct the input image, which can learn well from non-lesion and unlabeled data. The reconstruction task makes the model more suitable for the imbalanced medical image datasets. In addition, observing that the ABUS images have global semantic correlation, we establish long-distance dependence of images by the residual Swin Transformer block to improve segmentation performance. We evaluate our method on the ABUS dataset collected from 256 subjects and demonstrate the superiority of the proposed method over other state-of-the-art methods in this imbalanced dataset.

In this study, we present deep learning‐based approaches to automatic segmentation and applicator reconstruction with high accuracy and efficiency in the planning computed tomography (CT) for cervical cancer brachytherapy (BT). A novel three‐dimensional (3D) convolutional neural network (CNN) architecture was proposed and referred to as DSD‐UNET. The dataset of 91 patients received CT‐based BT of cervical cancer was used to train and test DSD‐UNET model for auto‐segmentation of high‐risk clinical target volume (HR‐CTV) and organs at risk (OARs). Automatic applicator reconstruction was achieved with DSD‐UNET‐based segmentation of applicator components followed by 3D skeletonization and polynomial curve fitting. Digitization of the channel paths for tandem and ovoid applicator in the planning CT was evaluated utilizing the data from 32 patients. Dice similarity coefficient (DSC), Jaccard Index (JI), and Hausdorff distance (HD) were used to quantitatively evaluate the accuracy. The segmentation performance of DSD‐UNET was compared with that of 3D U‐Net. Results showed that DSD‐UNET method outperformed 3D U‐Net on segmentations of all the structures. The mean DSC values of DSD‐UNET method were 86.9%, 82.9%, and 82.1% for bladder, HR‐CTV, and rectum, respectively. For the performance of automatic applicator reconstruction, outstanding segmentation accuracy was first achieved for the intrauterine and ovoid tubes (average DSC value of 92.1%, average HD value of 2.3 mm). Finally, HDs between the channel paths determined automatically and manually were 0.88 ± 0.12 mm, 0.95 ± 0.16 mm, and 0.96 ± 0.15 mm for the intrauterine, left ovoid, and right ovoid tubes, respectively. The proposed DSD‐UNET method outperformed the 3D U‐Net and could segment HR‐CTV, bladder, and rectum with relatively good accuracy. Accurate digitization of the channel paths could be achieved with the DSD‐UNET‐based method. The proposed approaches could be useful to improve the efficiency and consistency of treatment planning for cervical cancer BT.

IntroductionNarrow Band Imaging (NBI) is an endoscopic visualization technique useful for upper aero-digestive tract (UADT) cancer detection and margins evaluation. However, NBI analysis is strongly operator-dependent and requires high expertise, thus limiting its wider implementation. Recently, artificial intelligence (AI) has demonstrated potential for applications in UADT videoendoscopy. Among AI methods, deep learning algorithms, and especially convolutional neural networks (CNNs), are particularly suitable for delineating cancers on videoendoscopy. This study is aimed to develop a CNN for automatic semantic segmentation of UADT cancer on endoscopic images.Materials and MethodsA dataset of white light and NBI videoframes of laryngeal squamous cell carcinoma (LSCC) was collected and manually annotated. A novel DL segmentation model (SegMENT) was designed. SegMENT relies on DeepLabV3+ CNN architecture, modified using Xception as a backbone and incorporating ensemble features from other CNNs. The performance of SegMENT was compared to state-of-the-art CNNs (UNet, ResUNet, and DeepLabv3). SegMENT was then validated on two external datasets of NBI images of oropharyngeal (OPSCC) and oral cavity SCC (OSCC) obtained from a previously published study. The impact of in-domain transfer learning through an ensemble technique was evaluated on the external datasets.Results219 LSCC patients were retrospectively included in the study. A total of 683 videoframes composed the LSCC dataset, while the external validation cohorts of OPSCC and OCSCC contained 116 and 102 images. On the LSCC dataset, SegMENT outperformed the other DL models, obtaining the following median values: 0.68 intersection over union (IoU), 0.81 dice similarity coefficient (DSC), 0.95 recall, 0.78 precision, 0.97 accuracy. For the OCSCC and OPSCC datasets, results were superior compared to previously published data: the median performance metrics were, respectively, improved as follows: DSC=10.3% and 11.9%, recall=15.0% and 5.1%, precision=17.0% and 14.7%, accuracy=4.1% and 10.3%.ConclusionSegMENT achieved promising performances, showing that automatic tumor segmentation in endoscopic images is feasible even within the highly heterogeneous and complex UADT environment. SegMENT outperformed the previously published results on the external validation cohorts. The model demonstrated potential for improved detection of early tumors, more precise biopsies, and better selection of resection margins.

Simple SummaryUterine cervical cancer (CC) is a leading cause of cancer-related deaths in women worldwide. Pelvic magnetic resonance imaging (MRI) allows the assessment of local tumor extent and guides the choice of primary treatment. MRI tumor segmentation enables whole-volume radiomic tumor profiling, which is potentially useful for prognostication and individualization of therapy in CC. Manual tumor segmentation is, however, labor intensive and thus not part of routine clinical workflow. In the current work, we trained a deep learning (DL) algorithm to automatically segment the primary tumor in CC patients. Although the achieved segmentation performance of the trained DL algorithm is slightly lower than that for human experts, it is still relatively good. This study suggests that automated MRI primary tumor segmentations by DL algorithms without any human interaction is possible in patients with CC.Uterine cervical cancer (CC) is the most common gynecologic malignancy worldwide. Whole-volume radiomic profiling from pelvic MRI may yield prognostic markers for tailoring treatment in CC. However, radiomic profiling relies on manual tumor segmentation which is unfeasible in the clinic. We present a fully automatic method for the 3D segmentation of primary CC lesions using state-of-the-art deep learning (DL) techniques. In 131 CC patients, the primary tumor was manually segmented on T2-weighted MRI by two radiologists (R1, R2). Patients were separated into a train/validation (n = 105) and a test- (n = 26) cohort. The segmentation performance of the DL algorithm compared with R1/R2 was assessed with Dice coefficients (DSCs) and Hausdorff distances (HDs) in the test cohort. The trained DL network retrieved whole-volume tumor segmentations yielding median DSCs of 0.60 and 0.58 for DL compared with R1 (DL-R1) and R2 (DL-R2), respectively, whereas DSC for R1-R2 was 0.78. Agreement for primary tumor volumes was excellent between raters (R1-R2: intraclass correlation coefficient (ICC) = 0.93), but lower for the DL algorithm and the raters (DL-R1: ICC = 0.43; DL-R2: ICC = 0.44). The developed DL algorithm enables the automated estimation of tumor size and primary CC tumor segmentation. However, segmentation agreement between raters is better than that between DL algorithm and raters.

Comparison of mammography, digital breast tomosynthesis, automated breast ultrasound, magnetic resonance imaging in evaluation of residual tumor after neoadjuvant chemotherapy

Epicardial adipose tissue (EAT) is a visceral fat deposit, that’s known for its association with factors, such as obesity, diabetes mellitus, age, and hypertension. Segmentation of the EAT in a fast and reproducible way is important for the interpretation of its role as an independent risk marker intricate. However, EAT has a variable distribution, and various diseases may affect the volume of the EAT, which can increase the complexity of the already time-consuming manual segmentation work. We propose a 3D deep attention U-Net method to automatically segment the EAT from coronary computed tomography angiography (CCTA). Five-fold cross-validation and hold-out experiments were used to evaluate the proposed method through a retrospective investigation of 200 patients. The automatically segmented EAT volume was compared with physician-approved clinical contours. Quantitative metrics used were the Dice similarity coefficient (DSC), sensitivity, specificity, Jaccard index (JAC), Hausdorff distance (HD), mean surface distance (MSD), residual mean square distance (RMSD), and the center of mass distance (CMD). For cross-validation, the median DSC, sensitivity, and specificity were 92.7%, 91.1%, and 95.1%, respectively, with JAC, HD, CMD, MSD, and RMSD are 82.9% ± 8.8%, 3.77 ± 1.86 mm, 1.98 ± 1.50 mm, 0.37 ± 0.24 mm, and 0.65 ± 0.37 mm, respectively. For the hold-out test, the accuracy of the proposed method remained high. We developed a novel deep learning-based approach for the automated segmentation of the EAT on CCTA images. We demonstrated the high accuracy of the proposed learning-based segmentation method through comparison with ground truth contour of 200 clinical patient cases using 8 quantitative metrics, Pearson correlation, and Bland-Altman analysis. Our automatic EAT segmentation results show the potential of the proposed method to be used in computer-aided diagnosis of coronary artery diseases (CADs) in clinical settings.

Delineating the clinical target volume (CTV) and organs at risk (OARs) is crucial in rectal cancer radiotherapy. However, the accuracy of manual delineation (MD) is variable and the process is time consuming. Automatic delineation (AD) may be a solution to produce quicker and more accurate contours. In the present study, a convolutional neural network (CNN)-based AD tool was clinically evaluated to analyze its accuracy and efficiency in rectal cancer. CT images were collected from 148 supine patients in whom tumor stage and type of surgery were not differentiated. The rectal cancer contours consisted of CTV and OARs, where the OARs included the bladder, left and right femoral head, left and right kidney, spinal cord and bowel bag. The MD contours reviewed and modified together by a senior radiation oncologist committee were set as the reference values. The Dice similarity coefficient (DSC), Jaccard coefficient (JAC) and Hausdorff distance (HD) were used to evaluate the AD accuracy. The correlation between CT slice number and AD accuracy was analyzed, and the AD accuracy for different contour numbers was compared. The time recorded in the present study included the MD time, AD time for different CT slice and contour numbers and the editing time for AD contours. The Pearson correlation coefficient, paired-sample t-test and unpaired-sample t-test were used for statistical analyses. The results of the present study indicated that the DSC, JAC and HD for CTV using AD were 0.80±0.06, 0.67±0.08 and 6.96±2.45 mm, respectively. Among the OARs, the highest DSC and JAC using AD were found for the right and left kidney, with 0.91±0.06 and 0.93±0.04, and 0.84±0.09 and 0.88±0.07, respectively, and HD was lowest for the spinal cord with 2.26±0.82 mm. The lowest accuracy was found for the bowel bag. The more CT slice numbers, the higher the accuracy of the spinal cord analysis. However, the contour number had no effect on AD accuracy. To obtain qualified contours, the AD time plus editing time was 662.97±195.57 sec, while the MD time was 3294.29±824.70 sec. In conclusion, the results of the present study indicate that AD can significantly improve efficiency and a higher number of CT slices and contours can reduce AD efficiency. The AD tool provides acceptable CTV and OARs for rectal cancer and improves efficiency for delineation.

There has been a significant increase from 2010 to 2016 in the number of people suffering from spine problems. The automatic image segmentation of the spine obtained from a computed tomography (CT) image is important for diagnosing spine conditions and for performing surgery with computer-assisted surgery systems. The spine has a complex anatomy that consists of 33 vertebrae, 23 intervertebral disks, the spinal cord, and connecting ribs. As a result, the spinal surgeon is faced with the challenge of needing a robust algorithm to segment and create a model of the spine. In this study, we developed a fully automatic segmentation method to segment the spine from CT images, and we compared our segmentation results with reference segmentations obtained by well-known methods. We use a hybrid method. This method combines the convolutional neural network (CNN) and fully convolutional network (FCN), and utilizes class redundancy as a soft constraint to greatly improve the segmentation results. The proposed method was found to significantly enhance the accuracy of the segmentation results and the system processing time. Our comparison was based on 12 measurements: the Dice coefficient (94%), Jaccard index (93%), volumetric similarity (96%), sensitivity (97%), specificity (99%), precision (over segmentation 8.3 and under segmentation 2.6), accuracy (99%), Matthews correlation coefficient (0.93), mean surface distance (0.16 mm), Hausdorff distance (7.4 mm), and global consistency error (0.02). We experimented with CT images from 32 patients, and the experimental results demonstrated the efficiency of the proposed method. Highlights A method to enhance the accuracy of spine segmentation from CT data was proposed. The proposed method uses Convolutional Neural Network via redundant generation of class labels. Experiments show the segmentation accuracy has been enhanced.

Accurate prostate volume estimation is crucial for effective prostate disease management. Ultrasound (US) imaging, particularly transrectal ultrasound, offers a cost-effective and rapid assessment. However, US images often suffer from artifacts and poor contrast, making prostate volume estimation challenging. This review explores recent advancements in deep learning (DL) techniques for automatic prostate segmentation in US images as a primary step toward prostate volume estimation. We examine various DL architectures, including traditional U-Net modifications and innovative designs incorporating residual connections, multi-directional image data, and attention mechanisms. Additionally, we discuss pre-processing methods to enhance image quality, the integration of shape information, and strategies to improve the consistency and robustness of DL models. The effectiveness of these techniques is evaluated through metrics such as the Dice Similarity Coefficient, Jaccard Index, and Hausdorff Distance. The review highlights the potential of DL in improving prostate volume estimation accuracy and reducing clinical workload while also identifying areas for future research to enhance model performance and generalizability.

Intraoperative ultrasound (iUS) is increasingly used in pediatric brain tumor surgery. However, accurate tumor tissue identification and localization within the patient with iUS remains challenging. Automatic tumor segmentation may overcome these difficulties by providing quicker, more consistent, and reliable image analysis. This study evaluated the feasibility of automatic 3-dimensional segmentation of pediatric brain tumors using navigated iUS. A retrospective analysis was conducted on 109 navigated iUS acquisitions from 60 pediatric patients who underwent neurosurgical tumor resection. 3-dimensional nnU-Net was trained on 79 preresectional iUS acquisitions with corresponding tumor segmentations. Evaluation was performed on 30 iUS acquisitions against segmentations validated by 2 neurosurgeons. Performance benchmarks were set at a minimum Dice Similarity Coefficient (DSC) of 0.85 and a maximum mean surface distance and 95th percentile Hausdorff distance of 5 mm. The trained algorithm predicted tumor segmentations for all acquisitions in the test set, which met the predefined benchmark values with a median DSC of 0.89 (IQR: 0.13), a median mean surface distance of 1.1 mm (IQR: 1.0 mm), and a median 95th percentile Hausdorff distance of 5.6 mm (IQR: 3.5 mm). The lowest quartile of DSC values consisted of 5 patients, including craniopharyngiomas (n = 3) and high-grade gliomas (n = 2). Oversegmentation of the lateral ventricle as tumor was observed in 2 patients. In 1 high-grade glioma case, edema was misclassified as tumor because of unclear tumor boundaries. This study demonstrates that automatic segmentation of pediatric brain tumors using navigated iUS is feasible and achieved high performance in most cases, with few outliers that showed moderate performance in tumor prediction. Future efforts will focus on real-time segmentation of residual tumor tissue during tumor resection. Ultimately, real-time visualization of automatically segmented tumors in 3 dimensions could improve intraoperative navigation and support neurosurgical decision making.

To develop and evaluate a comprehensive AI-driven pipeline for automated segmentation and multi-class classification of ovarian tumors in ultrasound images using advanced transformer-based models and radiomic analysis. The study was a multi-center, retrospective study that included ultrasound data of 1,907 patients in 4 clinical centers, with 1,364 patients being used as a model training and validation and 543 as an external test. Five transformer-based segmentation models were used to outline tumour regions namely SegFormer, Swin-UNet, DPT, nnU-Net and HRFormer. Dice Similarity Coefficient (DSC), Hausdorff Distance (HD) and Relative Absolute Volume Difference (RAVD) were used to measure segmentation accuracy. Radiomic features (n = 215) were taken out through SERA platform, following the IBSI instructions. Mutual Information (MI), LASSO and Recursive Feature Elimination (RFE) were used as feature selection methods. Five models were considered to classify them: TabTransformer, TabNet, MLP, XGBoost, and KAN. Three clinically significant classification tasks, including tumour type, histological subtype, and FIGO staging, were covered. The models were checked by five-fold cross-validation and externally tested. SHAP analysis delivered interpretability. SegFormer was found to be the most accurate in terms of segmentation (mean DSC > 91%), which is higher than other U-Net-based models in earlier research. To classify, the TabTransformer selected by MI had the highest test accuracies in all tasks: tumour type (92.7 per cent.), histological subtype (92.1 per cent.), and FIGO stage (92.0 per cent). The accuracy of the external tests was over 90 per cent in all the tasks. SHAP analysis demonstrated the important radiomic features that make classification decisions, which is clinically transparent. There was no difference in performance between centers and imaging conditions. The suggested pipeline proves to be very accurate, general and interpretable in automated ovarian cancer diagnostics based on ultrasound imaging. It provides a powerful framework of clinical implementation and future application with multimodal data to support decisions better.