Automatic Cardiac Structure Contouring for Small Datasets with Cascaded Deep Learning Models

Implementation of automation routines leveraging deep learning (DL) methods has been a growing topic of interest. The focus of this work is on the Gamma Knife (GK) workflow, specifically the approximation of dose distributions from GK plan parameters and the prediction of dose distributions using DL. These works contribute towards the larger goal of treatment plan prediction and are seen as intermediate steps towards that end. The approximation of dose distributions was motivated by the closed nature of the GK system, which causes complications with the evaluation of the predictions made by the DL models. The approximation utilizes a superposition method, accumulating dose from a basis set of dose distributions which correspond to specific sector activations of the GK system. The approximation method was compared to the original dose distributions calculated by the GK treatment planning system (TPS), Leksell GammaPlan, for 30 patients using the similarity metrics of Dice Similarity Coefficient (DSC) and Hausdorff Distance (HD) as well as the plan quality metrics of coverage, selectivity, and gradient index. In the range of clinical planning significance, the metrics reported strong agreement to the original calculation (DSC > 0.85, HD < 2.5mm, coverage difference = 0.014 ± 0.035, selectivity difference = 0.008 ± 0.037, and gradient index difference = 0.118 ± 0.192). The second focus of this work was to improve the performance of DL models used to predict dose distributions from target contour information. This was accomplished by applying transforms to the binary target contour volumes, resulting in alternate model input channels that encode the distance of each voxel to the boundary of the nearest contour, both within and outside of the contours. Training and evaluation were performed with a standard UNet and HD-UNet architecture for large and small targets separately. DSC and Predictions resulting from these encodings resulted in significantly better predictions (p < 0.05) than their binary target counterparts at all (excluding 90% isodose for the HD-Unet) threshold levels for small targets. The large target trainings were significantly (p < 0.05) improved in the mid-to-high dose ranges.

Introduction: Magnetic resonance imaging (MRI) is the most common tool to examine glioblastoma. Preoperative MRI can be used to initial diagnosis, and surgery planning, while follow-up MRIs can be used to evaluate treatment responses, identify recurrency, and detect side effects. The follow-up MRIs are usually taken after the first-line therapy, such as maximal safe resection. In the past, radiologists manually segment the tumor regions from normal brain tissue on follow-up MRIs, which is time-consuming, error-prone, and challenging. Several deep-learning (DL) models have been developed utilizing preoperative images, but their performance has yet to be evaluated on follow-up MRIs. In this research, we built the largest follow-up MRI cohort (311 patients) to assess these DL models and their generalizability and performance on independent preoperative and follow-up images. We also made the first follow-up-based deep learning models for this specific task. Methods: All evaluation deep learning models (10 models) were trained by the Brain Tumor Segmentation challenge 2020 (BraTS’20) and evaluated by fifty pairs of preoperative and follow-up scans from our institution. The segmentation form our institution is evaluated by board certified radiologist. MRIs in the BraTS’20 dataset were all preoperative scans. After the evaluation, we randomly assigned 264 patients’ scans from our institution to the training dataset and 47 patients’ scans to the testing dataset. We compared three types of models in our follow-up deep learning model, including 1) UNet-3D, 2) UNet-3D+transfer-learning, and 3) UNet-3D+transfer-learning+baysian-learning. The benchmark for all models was the Dice similarity coefficient (DSC). DSC can measure the spatial overlap between model prediction and ground truth. The value of DSC is between zero to one. Zero means no overlap and one indicates complete overlap. Results: Our study demonstrates that the BraTS'20 trained models' performance decreased by 13.05% in independent preoperative MRI scans and 19.04% in follow-up MRI scans. The most significant mismatch regions were FLAIR hyperintense regions (3.68% drop in independent preoperative scans and 10.61% drop in independent follow-up scans) and Non-enhancing core (5.20% drop in independent preoperative scans and 11.99% drop in independent follow-up scans). Our best model can achieve the best DSC among three tumor regions compared to all evaluation models (FLAIR hyperintense regions: DSC 0.77 V.S. DSC 0.58; Enhancing tumor region: DSC 0.87 V.S. DSC 0.68; Non-enhancing tumor region: DSC 0.92 V.S. DSC 0.55). Conclusion: Maximal safe resection induced brain structure change, decreasing the performance of the preoperative-based DL model. Implementing a follow-up MRI-based segmentation model is essential to make accurate and generalizable results to address structural changes after maximal safe resection. Our follow-up DL model demonstrates the DSC score can be recovered. We commit to further developing the tool to assist radiologists in handling follow-up MRIs for glioblastoma patients. Citation Format: Kang Lin Hsieh, Tanjida Kabir, Luis Nunez-Rubiano, Yu-Chun Hsu, Yu Cai, Juan Rodriguez Quintero, Octavio Arevalo, Kangyi Zhao, Jackie Zhang, Jiguang Zhu Zhu, Roy Riascos, Xioaqian Jiang, Shayan Shams. A confident and operator-independent deep segmentation model to measure residual tumor volume in the follow-up MRIs for glioblastoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 2 (Clinical Trials and Late-Breaking Research); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(8_Suppl):Abstract nr LB067.

AIUM Practice Parameter for the Performance of Fetal Echocardiography.

BackgroundNasopharyngeal carcinoma (NPC), an endemic malignancy in Southeast Asia and southern China, necessitates precise delineation of the gross tumor volume (GTV) for radiotherapy to optimize patient prognosis. Positron emission tomography (PET) imaging offers valuable metabolic insights to guide radiotherapy planning but is hampered by manual GTV segmentation, which is labor-intensive and prone to inter- and intra-observer variability. Conventional segmentation methods are inherently limited, and comparative evaluations of deep learning (DL) frameworks for PET-based primary NPC lesion segmentation remain scarce. Thus, this study aimed to optimize the delineation of primary NPC radiotherapy target volumes by leveraging PET-image-based DL segmentation technology, with the goal of enhancing both the precision and efficiency of lesion delineation.MethodsResearchers retrospectively collected PET imaging data from 212 NPC patients at the Affiliated Hospital of Guangdong Medical University. The patients were randomly divided into training (170 patients) and testing (42 patients) datasets. A radiation oncologist and a nuclear medicine physician collaboratively delineated the cancer lesion boundaries through consensus. After data preprocessing, three DL models (Res-Unet, Nn-Unet, and Nn-Former) were used to automatically segment the lesions. Training was based on the training dataset. During evaluation, the models’ segmentation performance was comprehensively assessed using Dice similarity coefficient (DSC) and Hausdorff distance (HD). A visual analysis of the results was also conducted to intuitively understand the models’ segmentation capabilities.ResultsIn the training set, Nn-Unet achieved the highest DSC of 0.869, whereas in the testing set, its DSC was 0.833. The DSC for Nn-Former hovered around 0.8 in both the training and testing sets. In contrast, Res-Unet demonstrated the lowest DSC values among the three models, particularly in the testing set (DSC =0.794). Statistical analysis revealed that the DSC of Nn-Unet was significantly higher than that of both Res-Unet and Nn-Former in the training set (P<0.01 for both comparisons). Regarding the 95% HD and average surface distance (ASD) values, Nn-Unet outperformed the other two models in both the training and testing sets. However, Res-Unet exhibited a much higher HD value in the testing set compared to the training set. The Loss curves for all models gradually decreased as training progressed, indicating that the models were learning relevant features of NPC lesions. The final Loss value for Res-Unet was approximately 0.7, whereas Nn-Unet and Nn-Former had final Loss values below 0.8.ConclusionsThe PET-image-based automatic segmentation model for the primary tumor of NPC established in this study demonstrates the clinical potential of Nn-Unet in primary lesion of NPC segmentation tasks.

Noninvasive assessment of diastolic dysfunction relies on multiple echocardiographic indicators, including measurements from both standard B-mode images and Doppler, obtained at various cardiac locations such as the mitral annulus, tricuspid annulus, left ventricle, and left atrium. The diagnostic process is complex and subject to interobserver variability, making accurate and rapid evaluation challenging. Automated semantic segmentation of key cardiac structures, such as the left atrium, left ventricle, and mitral valve annulus, offers a potential solution by capturing temporal changes throughout the cardiac cycle. This study aims to improve the accuracy of segmenting the left atrium, left ventricle, and mitral valve annulus in echocardiographic images and to leverage the resulting temporal segmentation features for more reliable identification of diastolic dysfunction. This study presents Diff-TransUNet, a novel segmentation model incorporating a noise-robust Differential Transformer module. Evaluations on private (1137 training images, 135 validation images, and 88 test images), CAMUS (1400 training images, 200 validation images, and 200 test images), and EchoNet-Dynamic (5000 training images, 2546 validation images, and 2528 test images) datasets demonstrate improved performance over state-of-the-art methods, assessed by Dice coefficient (Dice), Intersection-over-Union (IoU), and 95th percentile Hausdorff Distance (HD95) metrics. Statistical analysis was performed to compare Diff-TransUNet with baseline methods across evaluation metrics. To control for errors arising from multiple comparisons, p-values were adjusted using the Benjamini-Hochberg false discovery rate (FDR) correction. Statistical significance was assessed at a 95% confidence level. In addition to p-values, Cohen's d effect size was computed to quantify the practical significance of performance differences. The proposed Diff-TransUNet achieved a Dice of 87.49%, IoU of 79.07%, and HD95 of 1.48 on the private dataset. Compared with state-of-the-art models, Dice improved by 1.35%-4.30% (p<0.05, Cohen's d=0.32-0.90), IoU by 1.97%-5.67% (p<0.05, Cohen's d=0.37-1.03), and HD95 by 0.16-0.83 (p<0.05, Cohen's d=0.21-0.90). On the CAMUS dataset, the model achieved a Dice of 88.74%, IoU of 80.58%, and HD95 of 2.83, showing improvements of 1.07%-4.96% (p<0.05, Cohen's d=0.18-0.63) in Dice, 1.55%-6.89% (p<0.05, Cohen's d=0.19-0.71) in IoU, and 0.41-2.85 (p<0.05, Cohen's d=0.12-0.46) in HD95 compared to advanced models. On the EchoNet-Dynamic dataset, the model obtained a Dice of 92.25%, IoU of 85.87%, and HD95 of 1.65, outperforming other methods by 0.42%-2.00% (p<0.05, Cohen's d=0.10-0.40) in Dice, 0.69%-3.21% (p<0.05, Cohen's d=0.10-0.43) in IoU, and 0.21-1.12 (p<0.05, Cohen's d=0.09-0.34) in HD95. Furthermore, by extracting volumetric segmentation features, the proposed method achieved an accuracy of 88.95% (95 % CI 87.15% to 90.08%) in identifying diastolic dysfunction. The proposed Diff-TransUNet model achieves significant improvements in ultrasound segmentation. Features extracted from the left ventricle, left atrium, and mitral annulus segmented by Diff-TransUNet can be effectively used for the identification of diastolic dysfunction.

Segmentation of cardiac structures is essential for cardiac function evaluation using cine magnetic resonance imaging (MRI). Deep learning can be used to segment cardiac structures in cine cardiac MRI with high accuracy, but this approach requires fully annotated datasets for training, which are difficult to obtain. Semi-supervised segmentation methods provide a way to alleviate the burden of manual labeling by using labeled and unlabeled data for training. However, these methods generally provide suboptimal segmentationaccuracies. To develop a semi-supervised method that utilizes relatively small training datasets and under-annotations for improved cine cardiac MRIsegmentation. The proposed approach consists of deformable registration, fully and weakly supervised segmentation, and a temporal attention perceiver (TAP). The registration module was trained to warp labeled frames to generate pseudo labels for unlabeled frames. The warped labeled images were used to train the fully supervised segmentation network. The unlabeled images and the pseudo label were used to train the weakly supervised segmentation model, and the segmentation prediction was compared with the input pseudo label as an auxiliary loss to the registration module. The TAP module was employed to generate optimized features for the warped labeled and the original unlabeled images both paired with the original labeled image. Consistency between the resulting features was enforced to refine cross-instance feature alignment to facilitate the registration. One hundred, twenty, and ten subjects from the Automatic Cardiac Diagnosis Challenge (ACDC) and seventy-five, thirty, and fifteen cases from the Multi-Vendor & Multi-Disease (M&Ms) Cardiac Image Segmentation Challenge were used for training, each with random end-systolic (ES)/end-diastolic (ED) frames labeled. The optimized models were used to segment the remaining 50 ACDC and 50 M&Ms subjects. The proposed approach was compared with several commonly used semi-supervised segmentation methods in terms of Dice-similarity-coefficients (DSC), average-symmetric-surface-distance (ASSD), and Hausdorff-distance (HD) for left (LV) and right (RV) ventricular cavity and myocardium (Myo). A Unet trained on the same subjects each with both frames labeled was used as an upper bound (Unet_UB). Using 100 ACDC training subjects, our approach yielded DSC = 0.910 0.063, ASSD = 1.37 0.63 mm, and HD = 6.38 2.99 mm for RV, DSC = 0.894 0.024, ASSD = 1.20 1.12 mm, and HD = 4.67 3.22 mm for Myo, and DSC = 0.934 0.056, ASSD = 1.25 1.63 mm, and HD = 3.97 5.76 mm for LV. A bidirectional copy-paste (BCP) method performed the best among the comparative methods and generated DSC = 0.902 0.060, ASSD = 1.45 0.60 mm, and HD = 7.50 3.20 mm for RV, DSC = 0.885 0.030, ASSD = 1.28 0.80 mm, and HD = 5.80 2.80 mm for Myo, and DSC = 0.920 0.068, ASSD = 1.15 0.40 mm, and HD = 4.20 3.30 mm for LV. For Unet_UB, these were 0.905 0.068, 1.48 0.61 mm, and 6.35 2.85 mm for RV, 0.895 0.030, 1.05 0.45 mm, and 4.40 3.09 mm for Myo, and 0.941 0.044, 1.02 0.34 mm, and 3.17 1.63 mm for LV. Similar trends were observed when using 75 M&Ms training subjects. For all the experiments, our approach outperformed BCP in general and yielded segmentation accuracies comparable to Unet_UB. The proposed approach outperformed several commonly used semi-supervised segmentation methods and yielded segmentation accuracies on par with fully supervised Unet using various relatively small datasets and under annotations fortraining.

MP43-02 LESSONS LEARNED IN APPLYING DEEP LEARNING TO FACILITATE PROSTATE MR-US FUSION BIOPSY WORKFLOW

This study evaluated the performance of a multi-stage Segmentation Residual Network (SegResNet)-based deep learning (DL) model for the automatic segmentation of cone-beam computed tomography (CBCT) images of patients with stage III and IV periodontitis. Seventy pre-processed CBCT scans from patients undergoing periodontal rehabilitation were used for training and validation. The model was tested on 10 CBCT scans independent from the training dataset by comparing results with semi-automatic (SA) segmentations. Segmentation accuracy was assessed using the Dice similarity coefficient (DSC), Intersection over Union (IoU), and Hausdorff distance 95th percentile (HD95). Linear periodontal measurements were performed on four tooth surfaces to assess the validity of the DL segmentation in the periodontal region. The DL model achieved a mean DSC of 0.9650 ± 0.0097, with an IoU of 0.9340 ± 0.0180 and HD95 of 0.4820 mm ± 0.1269 mm, showing strong agreement with SA segmentation. Linear measurements revealed high statistical correlations between the mesial, distal, and lingual surfaces, with intraclass correlation coefficients (ICC) of 0.9442 (p < 0.0001), 0.9232 (p < 0.0001), and 0.9598(p < 0.0001), respectively, while buccal measurements revealed lower consistency, with an ICC of 0.7481 (p < 0.0001). The DL method reduced the segmentation time by 47 times compared to the SA method. Acquired 3D models may enable precise treatment planning in cases where conventional diagnostic modalities are insufficient. However, the robustness of the model must be increased to improve its general reliability and consistency at the buccal aspect of the periodontal region. This study presents a DL model for the CBCT-based segmentation of periodontal defects, demonstrating high accuracy and a 47-fold time reduction compared to SA methods, thus improving the feasibility of 3D diagnostics for advanced periodontitis.

Aim To develop an optimal method for automated segmentation of atherosclerotic plaque structural components in optical coherence tomography (OCT) images using an ensemble of deep learning neural network models based on a comparison of nine artificial neural network architectures.Material and methods This study utilized a multidisciplinary OCT dataset obtained from examinations of 103 patients. Each OCT session was annotated as a set of two-dimensional binary masks corresponding to the pixel boundaries of four key morphological features of plaques: vascular lumen, fibrous cap, lipid core, and microvessels. Nine deep learning models, including U-Net, DeepLabV3, and others, were used to segment the anatomical structures. Model hyperparameters were tuned using Bayesian optimization, and the performance was assessed using the Dice Similarity Coefficient (DSC) metric.Results The models demonstrated high segmentation accuracy for the vascular lumen (DSC: 0.987) and moderate accuracy for the fibrous cap (DSC: 0.736) and lipid core (DSC: 0.751). Microvessel segmentation proved more challenging, with a final DSC accuracy of 61%. A weighted ensemble of models, accounting for the uneven representation of different morphological structures and model confidence, achieved an average DSC of 88.2%, which significantly improved the overall accuracy compared to individual models. This accuracy value exceeds those of all individual models, including the highest DSC values of 0.784 (for microvessels) and 0.751 (for lipid core), indicating a consistent improvement in the segmentation accuracy by integrating the models.Conclusion The study confirms the effectiveness of the proposed strategy for segmentation of atherosclerotic plaques in OCT images. This strategy is based on using specialized models for various morphological features and a weighted ensemble adapted to the uneven representation of different morphological structures and morphological complexity. This approach provided a high segmentation accuracy despite the pronounced inequality in the representation of classes. The study results can contribute to the development of decision support methods in cardiology aimed at improving diagnostic accuracy and treatment of cardiovascular diseases.

Automated extraction of left atrial volumes from two-dimensional computer tomography images using a deep learning technique

PurposeTo conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation.MethodsThis retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC).ResultsThirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05).ConclusionCross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Sentiment analysis is a subfield of Natural Language Processing (NLP) which tries to process a text to extract opinions or attitudes towards topics or entities. Recently, the use of deep learning methods for sentiment analysis has received noticeable attention from researchers. Generally, different deep learning methods have shown superb performance in sentiment analysis problem. However, deep learning models are different in nature and have different strengths and limitations. For example, convolutional neural networks are useful for extracting local structures from data, while recurrent models are able to learn order dependence in sequential data. In order to combine the advantages of different deep models, in this paper we have proposed a novel approach for aspect-based sentiment analysis which utilizes deep ensemble learning. In the proposed method, we first build four deep learning models, namely CNN, LSTM, BiLSTM and GRU. Then the outputs of these models are combined using stacking ensemble approach where we have used logistic regression as meta-learner. The results of applying the proposed method on the real datasets show that our method has increased the accuracy of aspect-based prediction by 5% to 20% compared to the basic deep learning methods.

Factors predicting MRI glioma segmentation accuracy in deep learning models: a systematic review and meta-analysis.

Subclinical atrial fibrillation (SCAF), also known as atrial high-rate episodes (AHREs), refers to asymptomatic heart rate elevations associated with increased risks of atrial fibrillation and cardiovascular events. Although deep learning (DL) models leveraging echocardiographic images from ultrasound are widely used for cardiac function analysis, their application to AHRE prediction remains unexplored. This study introduces a novel DL-based framework for automatic AHRE detection using echocardiograms. The approach encompasses left atrium (LA) segmentation, LA strain feature extraction, and AHRE classification. Data from 117 patients with cardiac implantable electronic devices undergoing echocardiography were analyzed, with 80% allocated to the development set and 20% to the test set. LA segmentation accuracy was quantified using the Dice coefficient, yielding scores of 0.923 for the LA cavity and 0.741 for the LA wall. For AHRE classification, metrics such as area under the curve (AUC), accuracy, sensitivity, and specificity were employed. A transformer-based model integrating patient characteristics demonstrated robust performance, achieving mean AUC of 0.815, accuracy of 0.809, sensitivity of 0.800, and specificity of 0.783 for a 24-h AHRE duration threshold. This framework represents a reliable tool for AHRE assessment and holds significant potential for early SCAF detection, enhancing clinical decision-making and patient outcomes.