Knee orientation detection in MR scout scans using 3D U-net

BackgroundNasopharyngeal carcinoma (NPC), an endemic malignancy in Southeast Asia and southern China, necessitates precise delineation of the gross tumor volume (GTV) for radiotherapy to optimize patient prognosis. Positron emission tomography (PET) imaging offers valuable metabolic insights to guide radiotherapy planning but is hampered by manual GTV segmentation, which is labor-intensive and prone to inter- and intra-observer variability. Conventional segmentation methods are inherently limited, and comparative evaluations of deep learning (DL) frameworks for PET-based primary NPC lesion segmentation remain scarce. Thus, this study aimed to optimize the delineation of primary NPC radiotherapy target volumes by leveraging PET-image-based DL segmentation technology, with the goal of enhancing both the precision and efficiency of lesion delineation.MethodsResearchers retrospectively collected PET imaging data from 212 NPC patients at the Affiliated Hospital of Guangdong Medical University. The patients were randomly divided into training (170 patients) and testing (42 patients) datasets. A radiation oncologist and a nuclear medicine physician collaboratively delineated the cancer lesion boundaries through consensus. After data preprocessing, three DL models (Res-Unet, Nn-Unet, and Nn-Former) were used to automatically segment the lesions. Training was based on the training dataset. During evaluation, the models’ segmentation performance was comprehensively assessed using Dice similarity coefficient (DSC) and Hausdorff distance (HD). A visual analysis of the results was also conducted to intuitively understand the models’ segmentation capabilities.ResultsIn the training set, Nn-Unet achieved the highest DSC of 0.869, whereas in the testing set, its DSC was 0.833. The DSC for Nn-Former hovered around 0.8 in both the training and testing sets. In contrast, Res-Unet demonstrated the lowest DSC values among the three models, particularly in the testing set (DSC =0.794). Statistical analysis revealed that the DSC of Nn-Unet was significantly higher than that of both Res-Unet and Nn-Former in the training set (P<0.01 for both comparisons). Regarding the 95% HD and average surface distance (ASD) values, Nn-Unet outperformed the other two models in both the training and testing sets. However, Res-Unet exhibited a much higher HD value in the testing set compared to the training set. The Loss curves for all models gradually decreased as training progressed, indicating that the models were learning relevant features of NPC lesions. The final Loss value for Res-Unet was approximately 0.7, whereas Nn-Unet and Nn-Former had final Loss values below 0.8.ConclusionsThe PET-image-based automatic segmentation model for the primary tumor of NPC established in this study demonstrates the clinical potential of Nn-Unet in primary lesion of NPC segmentation tasks.

Introduction: Magnetic resonance imaging (MRI) is the most common tool to examine glioblastoma. Preoperative MRI can be used to initial diagnosis, and surgery planning, while follow-up MRIs can be used to evaluate treatment responses, identify recurrency, and detect side effects. The follow-up MRIs are usually taken after the first-line therapy, such as maximal safe resection. In the past, radiologists manually segment the tumor regions from normal brain tissue on follow-up MRIs, which is time-consuming, error-prone, and challenging. Several deep-learning (DL) models have been developed utilizing preoperative images, but their performance has yet to be evaluated on follow-up MRIs. In this research, we built the largest follow-up MRI cohort (311 patients) to assess these DL models and their generalizability and performance on independent preoperative and follow-up images. We also made the first follow-up-based deep learning models for this specific task. Methods: All evaluation deep learning models (10 models) were trained by the Brain Tumor Segmentation challenge 2020 (BraTS’20) and evaluated by fifty pairs of preoperative and follow-up scans from our institution. The segmentation form our institution is evaluated by board certified radiologist. MRIs in the BraTS’20 dataset were all preoperative scans. After the evaluation, we randomly assigned 264 patients’ scans from our institution to the training dataset and 47 patients’ scans to the testing dataset. We compared three types of models in our follow-up deep learning model, including 1) UNet-3D, 2) UNet-3D+transfer-learning, and 3) UNet-3D+transfer-learning+baysian-learning. The benchmark for all models was the Dice similarity coefficient (DSC). DSC can measure the spatial overlap between model prediction and ground truth. The value of DSC is between zero to one. Zero means no overlap and one indicates complete overlap. Results: Our study demonstrates that the BraTS'20 trained models' performance decreased by 13.05% in independent preoperative MRI scans and 19.04% in follow-up MRI scans. The most significant mismatch regions were FLAIR hyperintense regions (3.68% drop in independent preoperative scans and 10.61% drop in independent follow-up scans) and Non-enhancing core (5.20% drop in independent preoperative scans and 11.99% drop in independent follow-up scans). Our best model can achieve the best DSC among three tumor regions compared to all evaluation models (FLAIR hyperintense regions: DSC 0.77 V.S. DSC 0.58; Enhancing tumor region: DSC 0.87 V.S. DSC 0.68; Non-enhancing tumor region: DSC 0.92 V.S. DSC 0.55). Conclusion: Maximal safe resection induced brain structure change, decreasing the performance of the preoperative-based DL model. Implementing a follow-up MRI-based segmentation model is essential to make accurate and generalizable results to address structural changes after maximal safe resection. Our follow-up DL model demonstrates the DSC score can be recovered. We commit to further developing the tool to assist radiologists in handling follow-up MRIs for glioblastoma patients. Citation Format: Kang Lin Hsieh, Tanjida Kabir, Luis Nunez-Rubiano, Yu-Chun Hsu, Yu Cai, Juan Rodriguez Quintero, Octavio Arevalo, Kangyi Zhao, Jackie Zhang, Jiguang Zhu Zhu, Roy Riascos, Xioaqian Jiang, Shayan Shams. A confident and operator-independent deep segmentation model to measure residual tumor volume in the follow-up MRIs for glioblastoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 2 (Clinical Trials and Late-Breaking Research); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(8_Suppl):Abstract nr LB067.

For pancreatic cancer patients, image guided radiation therapy and real-time tumor tracking (RTTT) techniques can deliver radiation to the target accurately. Currently, for the radiation therapy machine with kV X-ray imaging systems, internal markers must be implemented to facilitate tumor tracking. The purpose of this study was to develop a markerless deep learning-based pancreatic tumor positioning procedure for real-time tumor tracking with a kV X-ray imaging system. Fourteen pancreatic cancer patients treated with intensity-modulated radiation therapy from six fixed gantry angles with a gimbal-head radiotherapy system were included in this study. For a gimbal-head radiotherapy system, the three-dimensional (3D) intrafraction target position can be determined using an orthogonal kV X-ray imaging system. All patients underwent four-dimensional computed tomography (4DCT) simulations for treatment planning, which were divided into 10 respiratory phases. After a patient's 4DCT was acquired, for each X-ray tube angle, 10 digitally reconstructed radiograph (DRR) images were obtained. Then, a data augmentation procedure was conducted. The data augmentation procedure first rotated the CT volume around the superior-inferior and anterior-posterior directions from -3° to 3° in 1.5° intervals. Then, the Super-SloMo model was adapted to interpolate 10 frames between respiratory phases. In total, the data augmentation procedure expanded the data scale 250-fold. In this study, for each patient, 12 datasets containing the DRR images from each specific X-ray tube angle based on the radiation therapy plan were obtained. The augmented dataset was randomly divided into training and testing datasets. The training dataset contained 2000 DRR images with clinical target volume (CTV) contours labeled for fine-tuning the pre-trained target contour prediction model. After the fine-tuning, the patient and X-ray tube angle-specific CTV contour prediction model was acquired. The testing dataset contained the remaining 500 images to evaluate the performance of the CTV contour prediction model. The dice similarity coefficient (DSC) between the area enclosed by the CTV contour and predicted contour was calculated to evaluate the model's contour prediction performance. The 3D position of the CTV was calculated based on the centroid of the contour in the orthogonal DRR images, and the 3D error of the prediction position was calculated to evaluate the CTV positioning performance. For each patient, the DSC results from 12 X-ray tube angles and 3D error from 6 gantry angles were calculated, representing the novelty of this study. The mean and standard deviation (SD) of all patients' DSCs were 0.98 and 0.015, respectively. The mean and SD of the 3D error were 0.29mm and 0.14mm, respectively. The global maximum 3D error was 1.66mm, and the global minimum DSC was 0.81. The mean calculation time for CTV contour prediction was 55ms per image. This fulfills the requirement of RTTT. Regarding the positioning accuracy and calculation efficiency, the presented procedure can provide a solution for markerless real-time tumor tracking for pancreatic cancer patients.

Purpose: Rigid image registration (RIR) accuracy is crucial for image guided radiotherapy (IGRT). However, existing clinical image registration assessment methods cannot separate and quantify RIR error sources. Herein, we develop an extension of the ‘full circle method’ for RIR consistency. Paired registration circuits are used to isolate sources of RIR error caused by reference dataset substitution, from those inherent to the underlying RIR. This approach was demonstrated in the context of MRI-only IGRT, assessing substitution of MRI-derived synthetic-CT (sCT) for conventional CT, in a cohort of rectal cancer patients. Materials and methods: Planning CT, MRI-derived sCT, and two CBCTs from seven rectal cancer patients were retrospectively registered with global and soft tissue clipbox based RIR. Paired registration circuits were constructed using two moving (cone beam CT) images and two reference images (CT and sCT), per patient. Differences between inconsistencies in registration circuits containing CT and sCT were used to determine changes in registration accuracy due to substitution of sCT for CT. Results: sCT was found to be equivalent to CT under global RIR, with median differences of 0.05 mm and 0.01°. Soft tissue clipbox based RIR with sCT exhibited gross misregistration (>5 mm or 3°) for 3 patients. Registration consistency was degraded compared to CT across the cohort, with median differences of 0.54 mm and 0.15°. Conclusion: A paired registration circuit methodology for assessing RIR accuracy without ground truth information was developed and demonstrated for MRI-only IGRT in rectal cancer. This highlighted a reduction in clipbox based RIR consistency when sCT was substituted for conventional CT. The developed method enabled separation of degraded registration accuracy, from other error sources within the overall registration inconsistency. This novel methodology is applicable to any RIR scenario and enables analysis of the change in RIR performance on modification of image data or process.

Background[18F]fluoro-2-deoxy-D-glucose ([18F]FDG) positron emission tomography (PET) is a valuable tool for monitoring response to therapy in oncology. In longitudinal studies, however, patients are not scanned in exactly the same position. Rigid and non-rigid image registration can be applied in order to reuse baseline volumes of interest (VOI) on consecutive studies of the same patient. The purpose of this study was to investigate the impact of various image registration strategies on standardized uptake value (SUV) and metabolic volume test-retest variability (TRT).MethodsTest-retest whole-body [18F]FDG PET/CT scans were collected retrospectively for 11 subjects with advanced gastrointestinal malignancies (colorectal carcinoma). Rigid and non-rigid image registration techniques with various degrees of locality were applied to PET, CT, and non-attenuation corrected PET (NAC) data. VOI were drawn independently on both test and retest scans. VOI drawn on test scans were projected onto retest scans and the overlap between projected VOI and manually drawn retest VOI was quantified using the Dice similarity coefficient (DSC). In addition, absolute (unsigned) differences in TRT of SUVmax, SUVmean, metabolic volume and total lesion glycolysis (TLG) were calculated in on one hand the test VOI and on the other hand the retest VOI and projected VOI. Reference values were obtained by delineating VOIs on both scans separately.ResultsNon-rigid PET registration showed the best performance (median DSC: 0.82, other methods: 0.71-0.81). Compared with the reference, none of the registration types showed significant absolute differences in TRT of SUVmax, SUVmean and TLG (p > 0.05). Only for absolute TRT of metabolic volume, significant lower values (p < 0.05) were observed for all registration strategies when compared to delineating VOIs separately, except for non-rigid PET registrations (p = 0.1). Non-rigid PET registration provided good volume TRT (7.7%) that was smaller than the reference (16%).ConclusionIn particular, non-rigid PET image registration showed good performance similar to delineating VOI on both scans separately, and with smaller TRT in metabolic volume estimates.

HomeRadiology: Imaging CancerVol. 2, No. 2 Previous Research HighlightsFree AccessA New Approach for Automated Image Segmentation of Organs at Risk in Cervical CancerMeredith Jones, Lacey R. McNallyMeredith Jones, Lacey R. McNallyMeredith JonesLacey R. McNallyPublished Online:Mar 27 2020https://doi.org/10.1148/rycan.2020204010MoreSectionsPDF ToolsImage ViewerAdd to favoritesCiteTrack CitationsPermissionsReprints ShareShare onFacebookTwitterLinked InEmail Take-Away Points■ Major Focus: Development of an automated multiclass image segmentation model to identify organs at risk (OARs) on CT scans in patients with locally advanced cervical cancer, prior to radiation therapy.■ Key Results: The convolutional neural network model outperformed model-based, intensity-based, and atlas-based segmentation algorithms, with comparable results to other deep learning segmentation models while also decreasing the time needed for image segmentation by several thousand-fold.■ Impact: Reduction of intra- and interuser variability imposed by manual OAR delineation can lead to more accurate and efficient radiation therapy planning.The recent article by Liu et al developed an automated multiclass image segmentation model to identify organs at risk (OARs) on CT scans. To plan effective and safe radiation therapy in the context of cervical cancer, a CT scan is often acquired prior to the first treatment. Currently, a radiation oncologist will segment the CT images by hand, which is a time-consuming process, subject to intra- and interobserver variability, to identify OARs. To overcome these potential limitations and biases, automated image segmentation techniques can be utilized to improve accuracy and speed of annotating each image. A major problem in creation of deep learning image segmentation models for medical images is the lack of available datasets for model training and validation. Liu et al developed a two-dimensional image segmentation model based off the established convolutional neural network architecture, U-Net. U-Net was utilized because it is feasible to train from scratch with a small training set.This study looked at the CT scans of 105 patients with locally advanced cervical cancer. OAR segmentation for cervical cancer is a particularly challenging problem due to variability in organ size and frequent unclear boundaries. Even an experienced radiologist will have trouble. Seven organs were identified as OARs for cervical cancer radiation therapy: bladder, bone marrow, femoral head left, femoral head right, rectum, small intestine, and spinal cord.The model uses context aggregation blocks where U-Net has convolution layers. These context aggregation blocks use dilation convolution layers with different dilated rates and normal convolution layers with different kernel sizes to yield a low false-positive rate and accurate feature extraction in a large receptive field. A squeeze-extract block was used to ensure proper consideration of organs that are small. A class-weighted cross-entropy loss function was also used to overcome the imbalance in the datasets created by the differences in size and shape of each OAR.A ground truth was available based on the radiation oncologist’s annotations, allowing a Dice similarity coefficient (DSC) and 95th Hausdorff distance (HD) to be calculated and used as an evaluation metric. The DSC and the HD metrics indicated a comparable performance to other deep learning image segmentation models in most of the organs analyzed; inferior results were seen in the rectum and superior results in the intestines. All results were analyzed by an oncologist who determined that greater than 90% of results produced by this model were highly acceptable to be used during radiation therapy planning. To make the model more robust and generalizable, it is necessary to acquire a larger dataset with scans from different sources. Translation of this model to three dimensional may ultimately improve clinical translation.Highlighted ArticleLiu Z, Liu X, Xiao B, et al. Segmentation of organs-at-risk in cervical cancer CT images with a convolutional neural network. Physica Medica 2020;69:184-191. doi: https://doi.org/10.1016/j.ejmp.2019.12.008Highlighted ArticleLiu Z, Liu X, Xiao B, et al. Segmentation of organs-at-risk in cervical cancer CT images with a convolutional neural network. Physica Medica 2020;69:184-191. doi: https://doi.org/10.1016/j.ejmp.2019.12.008 Crossref, Medline, Google ScholarArticle HistoryPublished online: Mar 27 2020 FiguresReferencesRelatedDetailsCited ByJournal of Biomimetics, Biomaterials and Biomedical Engineering, Vol. 49Mathematical Problems in Engineering, Vol. 2020Recommended Articles nnU-Net: Further Automating Biomedical Image AutosegmentationRadiology: Imaging Cancer2021Volume: 3Issue: 1Deep Learning: A Primer for RadiologistsRadioGraphics2017Volume: 37Issue: 7pp. 2113-2131Generative Adversarial Networks: A Primer for RadiologistsRadioGraphics2021Volume: 41Issue: 3pp. 840-857Current Applications and Future Impact of Machine Learning in RadiologyRadiology2018Volume: 288Issue: 2pp. 318-328Deep Generative Adversarial Networks: Applications in Musculoskeletal ImagingRadiology: Artificial Intelligence2021Volume: 3Issue: 3See More RSNA Education Exhibits A Multidisciplinary Approach for Program Development with Artificial Intelligence in Pancreatic Cancer: How We Fit InDigital Posters2019Abnormal Changes After Radiation Therapy: What Radiologists Should KnowDigital Posters2018Artificial Intelligence in Diagnostic Imaging: Current Applications and Future PerspectiveDigital Posters2019 RSNA Case Collection Acute Radiation EnteropathyRSNA Case Collection2021Small Bowel LymphomaRSNA Case Collection2021Hirschsprung DiseaseRSNA Case Collection2021 Vol. 2, No. 2 Metrics Downloaded 310 times Altmetric Score

In robotics-assisted surgery, accurate segmentation of surgical instruments from background pixels is crucial for the success of various downstream tasks. Currently, popular segmentation models such as the U-Net exhibit excellent performance under major surgical conditions. However, when benchmarking the segmentation models under unforeseen surgical complications such as overbleeding, concerns toward their robustness remain. In fact, due to the lack of training datasets for the surgical segmentation models under complications, these models often exhibit degraded performances and thus result in failures for conducting accurate segmentations at the pixel level. In this paper, we leverage the Generative Adversarial Network (GAN), specifically through generating realistic images that simulate the condition of overbleeding to enlarge the training datasets a process known as data augmentation. With more diverse training data from different scenarios, segmentation models can be trained better and reduce the risk of errors. In addition, a dataset consisting of 17 mock endoscopic video sequences with binary masks indicating ground truths was used for training/testing. With the dataset, a U-Net architecture is used to examine performance through metrics such as the Normalized Surface Distance (NSD) and Dice Similarity Coefficient (DSC). As a result, integrations with GAN-based data augmentation allow segmentation models performances to improve 9.07% for NSD and 6.63% for DSC. From the results, our GAN-based augmentation is proven to be effective, providing a novel direction for improving models robustness during the training procedure, paving the way for future exploration and clinical translation of the cutting-edge deep learning models into real robotics surgery.

Recent studies on Parkinson's Disease (PD) discovered potential markers in brain MR images for its diagnosis and staging.One of them is the neuromelanin, which can be found in regions such as the substantia nigra and the locus coeruleus, both located in the brainstem.The analysis of these regions is normally conducted manually by a specialized professional, which can be costly and timeconsuming.Fortunately, there have been advances in automatic segmentation approaches in several areas.In medical imaging, U-Nets are showing state-of-the-art performance, but we did not find many works using this architecture to extract brainstem structures.A likely cause for this is the scarcity of large databases with well annotated segmentation of these areas.Thus, this paper explores the use of a dataset of our research group composed of T1-weighted MR images to train U-Net models with different types of slices to automatically demarcate the brainstem (as a first stage).The analysis starts with a scripted segmentation of the target region using the Freesurfer package.Thereby, we trained 4 models and evaluated them using Dice Similarity Coefficient (DSC) and Intersection over Union (IoU).Three models were trained on single anatomical planes, and one on multi-plane slices.Also, we applied grid search, varying optimizers, numbers of filters in the first layer, and learning rates.The best model trained with the larger subset with the best performance was the axial (DSC: 91.76% and IoU: 86.95%), followed by sagittal (DSC: 90.61% and IoU: 84.11%), coronal (DSC: 89.38% and IoU: 68.76%), and all slices (DSC: 16.36% and IoU: 7.58%).The next steps of our research are to differentiate the midbrain out of the brainstem, considering manual segmentation as ground truth, and test some approaches on determining the threshold and evaluating separately the 2 hemispheres of the midbrain.

Landmark-guided deformable image registration for accurate cumulative dose evaluation in lung reirradiation.

A Comparative Analysis of Individual Readers and an Autonomous Deep-Learning-Based Algorithm for Assessing Total Metabolic Tumor Volume from Positron Emission Tomography/Computed Tomography Scans

BackgroundPrecise volume quantification of intracerebral hemorrhage (ICH), intraventricular hemorrhage (IVH), and perihematomal edema (PHE) is a critical parameter for guiding therapy decisions, monitoring therapeutic effects over time, and predicting patient outcomes.PurposeTo evaluate a nnU-Net-based deep learning model for automated, multilesion segmentation on non-contrast CT.Materials and MethodsRetrospective data from acute spontaneous ICH patients admitted to 4 stroke centers (2015-2022) and controls (2022-2023) were analyzed. Manual segmentations served as ground truth with repeated segmentations as reference standard. nnU-Net was trained (n = 775) using 5-fold cross-validation and tested on a holdout set (n = 189). Lesion detection, segmentation, and volumetric accuracy were evaluated using the Dice similarity coefficient (DSC) and Pearson correlation coefficients (r), with subanalyses for anatomical location and impact of other hemorrhage types (subarachnoid, subdural, or epidural hematoma). The model was validated on internal (n = 121) and external (n = 169) datasets. Processing time was compared to manual segmentation.ResultsTest set sensitivity was 99% for ICH and PHE and 97% for IVH. Segmentation achieved a DSC of 0.91 (ICH), 0.71 (PHE), and 0.76 (IVH), with r = 0.99 (ICH, IVH) and r = 0.92 (PHE). DSC for lobar and deep hemorrhages were 0.90 and 0.92, respectively, and 0.70 in the brainstem, with other hemorrhage types showing no significant impact on segmentation accuracy, P > .05. For internal validation, DSC was 0.88 (ICH), 0.66 (PHE), and 0.80 (IVH), with r of 0.98, 0.88, and 0.98, respectively. External validation yielded DSC values of 0.85 (ICH), 0.61 (PHE), and 0.80 (IVH), with r values of 0.97, 0.85, and 0.96. Mean processing time was 18.2 s (±5 SD), compared to 18.01 min (±20.47 SD) for manual segmentations.ConclusionnnU-Net enables reliable, time-efficient segmentation of ICH, PHE, and IVH, validated across multicenter, multivendor datasets of spontaneous ICH, showing potential to enhance clinical workflows.

Task automation is essential for efficient and consistent image segmentation in radiation oncology. We report on a deep learning architecture, comprising a U-Net and a variational autoencoder (VAE) for automatic contouring of the prostate gland incorporating interobserver variation for radiotherapy treatment planning. The U-Net/VAE generates an ensemble set of segmentations for each image CT slice. A novel outlier mitigation (OM) technique was implemented to enhance the model segmentation accuracy. The primary source dataset (source_prim) consisted of 19 200 CT slices (from 300 patient planning CT image datasets) with manually contoured prostate glands. A smaller secondary source dataset (source_sec) comprised 640 CT slices (from 10 patient CT datasets), where prostate glands were segmented by 5 independent physicians on each dataset to account for interobserver variability. Data augmentation via random rotation (<5 degrees), cropping, and horizontal flipping was applied to each dataset to increase sample size by a factor of 100. A probabilistic hierarchical U-Net with VAE was implemented and pretrained using the augmented source_prim dataset for 30 epochs. Model parameters of the U-Net/VAE were fine-tuned using the augmented source_sec dataset for 100 epochs. After the first round of training, outlier contours in the training dataset were automatically detected and replaced by the most accurate contours (based on Dice similarity coefficient, DSC) generated by the model. The U-Net/OM-VAE was retrained using the revised training dataset. Metrics for comparison included DSC, Hausdorff distance (HD, mm), normalized cross-correlation (NCC) coefficient, and center-of-mass (COM) distance (mm). Results for U-Net/OM-VAE with outliers replaced in the training dataset versus U-Net/VAE without OM were as follows: DSC=0.82±0.01 versus 0.80±0.02 (p=0.019), HD=9.18±1.22 versus 10.18±1.35mm (p=0.043), NCC=0.59±0.07 versus 0.62±0.06, and COM=3.36±0.81 versus 4.77±0.96mm over the average of 15 contours. For the average of 15 highest accuracy contours, values were as follows: DSC=0.90±0.02 versus 0.85±0.02, HD=5.47±0.02 versus 7.54±1.36mm, and COM=1.03±0.58 versus 1.46±0.68mm (p<0.03 for all metrics). Results for the U-Net/OM-VAE with outliers removed were as follows: DSC=0.78±0.01, HD=10.65±1.95mm, NCC=0.46±0.10, COM=4.17±0.79mm for the average of 15 contours, and DSC=0.88±0.02, HD=7.00±1.17mm, COM=1.58±0.63mm for the average of 15 highest accuracy contours. All metrics for U-Net/VAE trained on the source_prim and source_sec datasets via pretraining, followed by fine-tuning, show statistically significant improvement over that trained on the source_sec dataset only. Finally, all metrics for U-Net/VAE with or without OM showed statistically significant improvement over those for the standard U-Net. A VAE combined with a hierarchical U-Net and an OM strategy (U-Net/OM-VAE) demonstrates promise toward capturing interobserver variability and produces accurate prostate auto-contours for radiotherapy planning. The availability of multiple contours for each CT slice enables clinicians to determine trade-offs in selecting the "best fitting" contour on each CT slice. Mitigation of outlier contours in the training dataset improves prediction accuracy, but one must be wary of reduction in variability in the training dataset.

Pedicle screw fixation is a common yet technically demanding procedure. Due to the proximity of the inserted implant to the spinal column, a malplaced screw can cause neurological injury and subsequent postoperative complications. A common surgical routine starts with preoperative volumetric image acquisition (e.g. computed tomography) based on which the surgeons can highlight the planned trajectory. This process is generally done manually , which is error prone and time consuming.The primary purpose of this paper is to develop an automatic pedicle region localization based on preoperative CTs. This system can automatically annotate the CT scans to identify the regions corresponding to the pedicles and thus provide important information about the anatomical placement of the CT scan that can be useful for intraoperative implant position assessment (e.g. to initialize the 2D-3D registration). On the other hand, the pedicle localization can be exploited for preoperative planning.We designed and evaluated a fully convolutional neural network for the task of pedicle localization. A large training, validation and testing datasets (5000, 1000, 1000 images separately) were created using a custom data augmentation process that could generate unique vertebral morphologies for each image. After evaluation on the validation and test data, the Dice similarity coefficients between the pedicle regions detected by the trained network and the ground truth was 0.85 and 0.83 respectively.The proposed deep-learning-based algorithm was capable of automatically localizing the regions corresponding to the pedicles based on the preoperative CT scans. Therefore, a reliable initial guess for the 2D-3D registration process needed for intraoperative implant position assessment can be achieved. This system also has potential use in automating the preoperative planning.

Breast cancer, the most common cancer type among women worldwide, requires early detection and accurate diagnosis for improved treatment outcomes. Segmenting fat and fibroglandular tissue (FGT) in magnetic resonance imaging (MRI) is essential for creating volumetric models, enhancing surgical workflow, and improving clinical outcomes. Manual segmentation is time-consuming and subjective, prompting the development of automated deep-learning algorithms to perform this task. However, configuring these algorithms for 3D medical images is challenging due to variations in image features and preprocessing distortions. Automated machine learning (AutoML) frameworks automate model selection, hyperparameter tuning, and architecture optimization, offering a promising solution by reducing reliance on manual intervention and expert knowledge. We compare nnU-Net and Auto3Dseg, two AutoML frameworks, in segmenting fat and FGT on T1-weighted MRI images from the Duke breast MRI dataset (100 patients). We used threefold cross-validation, employing the Dice similarity coefficient (DSC) and Hausdorff distance (HD) metrics for evaluation. The -test and Tukey honestly significant difference analysis were used to assess statistical differences across methods. nnU-Net achieved DSC scores of (fat) and (FGT), whereas Auto3DSeg achieved (fat) and (FGT). Significant differences in fat HD ( , ) originated from the full resolution and the 3D cascade U-Net. No evidence of significant differences was found in FGT HD or DSC metrics. Ensemble approaches of Auto3Dseg and nnU-Net demonstrated comparable performance in segmenting fat and FGT on breast MRI. The significant differences in fat HD underscore the importance of boundary-focused metrics in evaluating segmentation methods.

Cardiac visceral fat was segmented from low-dose non-contrast chest CT images using a fully automated method. Cardiac visceral fat is defined as the fatty tissues surrounding the heart region, enclosed by the lungs and posterior to the sternum. It is measured by constraining the heart region with an Anatomy Label Map that contains robust segmentations of the lungs and other major organs and estimating the fatty tissue within this region. The algorithm was evaluated on 124 low-dose and 223 standard-dose non-contrast chest CT scans from two public datasets. Based on visual inspection, 343 cases had good cardiac visceral fat segmentation. For quantitative evaluation, manual markings of cardiac visceral fat regions were made in 3 image slices for 45 low-dose scans and the Dice similarity coefficient (DSC) was computed. The automated algorithm achieved an average DSC of 0.93. Cardiac visceral fat volume (CVFV), heart region volume (HRV) and their ratio were computed for each case. The correlation between cardiac visceral fat measurement and coronary artery and aortic calcification was also evaluated. Results indicated the automated algorithm for measuring cardiac visceral fat volume may be an alternative method to the traditional manual assessment of thoracic region fat content in the assessment of cardiovascular disease risk.