Leveraging SAM for automatic prostate segmentation on micro-ultrasound images

Multiparametric ultrasound (mpUS) is being investigated as an alternative to magnetic resonance imaging (MRI) for detection of prostate cancer (PC). Automated prostate segmentation facilitates workflows, and zonal segmentation can aid in PC diagnosis, accounting for differences in imaging characteristics and tumor incidence. Our aim was to develop a deep learning algorithm that can automatically segment the prostate and its zones on three-dimensional (3D) contrast-enhanced ultrasound (CEUS) and conventional brightness-mode (B-mode) images (NCT04605276). A total of 259 3D mpUS images were collected from men with suspicion for PC in a prospective multicenter trial to develop a computer-aided diagnosis system for PC. Manual segmentation was performed using a custom tool, and an algorithm was developed using a convolutional neural network based on the U-Net architecture. Cross-validation of the automated segmentation algorithm revealed Dice similarity coefficients (DSCs) of 0.91 (95% confidence interval [CI] 0.90-0.91) for CEUS and 0.94 (95% CI 0.93-0.94) for B-mode ultrasound for 3D prostate segmentation. Zonal segmentation was less accurate, with DSCs of 0.83 (95% CI 0.82-0.84) for CEUS and 0.86 (95% CI 0.85-0.87) for B-mode ultrasound. There was high agreement for prostate volume between automatic segmentation on CEUS and physician-estimated volumes on MRI (R2=0.96). Qualitative assessment of prostate segmentation using a scale from 1 to 5 revealed a median grade of 5 (interquartile range [IQR] 4-5) for manual segmentation and 4 (IQR 4-5) for automated segmentation (p=0.10). Our deep learning algorithm demonstrated strong performance for automatic prostate and zonal segmentation from 3D CEUS and B-mode ultrasound images. We developed a computer tool to automatically identify the prostate in three-dimensional ultrasound images. The results show high accuracy and closely match manual assessments by urologists. This tool has potential for use in a computer-aided diagnostic system for prostate cancer.

Cross-modal Transfer Learning Based on an Improved CycleGAN Model for Accurate Kidney Segmentation in Ultrasound Images

Attention-guided multi-scale learning network for automatic prostate and tumor segmentation on MRI

Accurate prostate volume estimation is crucial for effective prostate disease management. Ultrasound (US) imaging, particularly transrectal ultrasound, offers a cost-effective and rapid assessment. However, US images often suffer from artifacts and poor contrast, making prostate volume estimation challenging. This review explores recent advancements in deep learning (DL) techniques for automatic prostate segmentation in US images as a primary step toward prostate volume estimation. We examine various DL architectures, including traditional U-Net modifications and innovative designs incorporating residual connections, multi-directional image data, and attention mechanisms. Additionally, we discuss pre-processing methods to enhance image quality, the integration of shape information, and strategies to improve the consistency and robustness of DL models. The effectiveness of these techniques is evaluated through metrics such as the Dice Similarity Coefficient, Jaccard Index, and Hausdorff Distance. The review highlights the potential of DL in improving prostate volume estimation accuracy and reducing clinical workload while also identifying areas for future research to enhance model performance and generalizability.

Automatic prostate segmentation from MR images is an important task in various clinical applications such as prostate cancer staging and MR-guided radiotherapy planning. However, the large appearance and shape variations of the prostate in MR images make the segmentation problem difficult to solve. Traditional Active Shape/Appearance Model (ASM/AAM) has limited accuracy on this problem, since its basic assumption, i.e., both shape and appearance of the targeted organ follow Gaussian distributions, is invalid in prostate MR images. To this end, the authors propose a sparse dictionary learning method to model the image appearance in a nonparametric fashion and further integrate the appearance model into a deformable segmentation framework for prostate MR segmentation. To drive the deformable model for prostate segmentation, the authors propose nonparametric appearance and shape models. The nonparametric appearance model is based on a novel dictionary learning method, namely distributed discriminative dictionary (DDD) learning, which is able to capture fine distinctions in image appearance. To increase the differential power of traditional dictionary-based classification methods, the authors' DDD learning approach takes three strategies. First, two dictionaries for prostate and nonprostate tissues are built, respectively, using the discriminative features obtained from minimum redundancy maximum relevance feature selection. Second, linear discriminant analysis is employed as a linear classifier to boost the optimal separation between prostate and nonprostate tissues, based on the representation residuals from sparse representation. Third, to enhance the robustness of the authors' classification method, multiple local dictionaries are learned for local regions along the prostate boundary (each with small appearance variations), instead of learning one global classifier for the entire prostate. These discriminative dictionaries are located on different patches of the prostate surface and trained to adaptively capture the appearance in different prostate zones, thus achieving better local tissue differentiation. For each local region, multiple classifiers are trained based on the randomly selected samples and finally assembled by a specific fusion method. In addition to this nonparametric appearance model, a prostate shape model is learned from the shape statistics using a novel approach, sparse shape composition, which can model nonGaussian distributions of shape variation and regularize the 3D mesh deformation by constraining it within the observed shape subspace. The proposed method has been evaluated on two datasets consisting of T2-weighted MR prostate images. For the first (internal) dataset, the classification effectiveness of the authors' improved dictionary learning has been validated by comparing it with three other variants of traditional dictionary learning methods. The experimental results show that the authors' method yields a Dice Ratio of 89.1% compared to the manual segmentation, which is more accurate than the three state-of-the-art MR prostate segmentation methods under comparison. For the second dataset, the MICCAI 2012 challenge dataset, the authors' proposed method yields a Dice Ratio of 87.4%, which also achieves better segmentation accuracy than other methods under comparison. A new magnetic resonance image prostate segmentation method is proposed based on the combination of deformable model and dictionary learning methods, which achieves more accurate segmentation performance on prostate T2 MR images.

Automatic and reliable prostate segmentation is an essential prerequisite for assisting the diagnosis and treatment, such as guiding biopsy procedure and radiation therapy. Notwithstanding, automatic segmentation is challenging due to the lack of clear prostate boundaries owing to similar appearance of prostate and surrounding tissues and the wide variation in size and shape among different patients. Therefore, this paper proposes an automatic method for prostate segmentation on 3D MRI scans using a content-sensitive superpixels technique, deep convolutional neural networks with residual connections (ResCNN), and the particle swarm optimization (PSO) algorithm. The proposed method has been evaluated on the Prostate 3T and PROMISE12 databases, presenting a Dice similarity coefficient of 86.68 %, Jaccard index of 76.58%, relative volume difference of 3.92%, volumetric similarity of 96.69 %, sensitivity of 88.36 %, specificity of 93.43%, accuracy of 91.97% and an area under ROC curve of 90.90 %. Experimental results demonstrate the high performance-potential of the proposed method comparable to those previously published.

PurposeTo overcome the severe intensity inhomogeneity and blurry boundaries in HIFU (High Intensity Focused Ultrasound) ultrasound images, an accurate and efficient multi-scale and shape constrained localized region-based active contour model (MSLCV), was developed to accurately and efficiently segment the target region in HIFU ultrasound images of uterine fibroids.MethodsWe incorporated a new shape constraint into the localized region-based active contour, which constrained the active contour to obtain the desired, accurate segmentation, avoiding boundary leakage and excessive contraction. Localized region-based active contour modeling is suitable for ultrasound images, but it still cannot acquire satisfactory segmentation for HIFU ultrasound images of uterine fibroids. We improved the localized region-based active contour model by incorporating a shape constraint into region-based level set framework to increase segmentation accuracy. Some improvement measures were proposed to overcome the sensitivity of initialization, and a multi-scale segmentation method was proposed to improve segmentation efficiency. We also designed an adaptive localizing radius size selection function to acquire better segmentation results.ResultsExperimental results demonstrated that the MSLCV model was significantly more accurate and efficient than conventional methods. The MSLCV model has been quantitatively validated via experiments, obtaining an average of 0.94 for the DSC (Dice similarity coefficient) and 25.16 for the MSSD (mean sum of square distance). Moreover, by using the multi-scale segmentation method, the MSLCV model’s average segmentation time was decreased to approximately 1/8 that of the localized region-based active contour model (the LCV model).ConclusionsAn accurate and efficient multi-scale and shape constrained localized region-based active contour model was designed for the semi-automatic segmentation of uterine fibroid ultrasound (UFUS) images in HIFU therapy. Compared with other methods, it provided more accurate and more efficient segmentation results that are very close to those obtained from manual segmentation by a specialist.

BackgroundAge estimation is vital in forensic science, with maxillary sinus development serving as a reliable indicator. This study developed an automatic segmentation model for maxillary sinus identification and parameter measurement, combined with regression and machine learning models for age estimation.MethodsCone Beam Computed Tomography (CBCT) images from 292 Han individuals (ranging from 5 to 53 years) were used to train and validate the segmentation model. Measurements included sinus dimensions (length, width, height), inter-sinus distance, and volume. Age estimation models using multiple linear regression and random forest algorithms were built based on these variables.ResultsThe automatic segmentation model achieved high accuracy, which yielded a Dice similarity coefficient (DSC) of 0.873, an Intersection over Union (IoU) of 0.7753, a Hausdorff Distance 95% (HD95) of 9.8337, and an Average Surface Distance (ASD) of 2.4507. The regression model performed best, with mean absolute errors (MAE) of 1.45 years (under 18) and 3.51 years (aged 18 and above), providing relatively precise age predictions.ConclusionThe maxillary sinus-based model is a promising tool for age estimation, particularly in adults, and could be enhanced by incorporating additional variables like dental dimensions.

Heterogeneous intensity distribution inside the prostate gland, significant variations in prostate shape, size, inter dataset contrast variations, and imaging artifacts like shadow regions and speckle in Trans Rectal Ultrasound (TRUS) images challenge computer aided automatic or semi-automatic segmentation of the prostate. In this paper, we propose a supervised learning schema based on random forest for automatic initialization and propagation of statistical shape and appearance model. Parametric representation of the statistical model of shape and appearance is derived from principal component analysis (PCA) of the probability distribution inside the prostate and PCA of the contour landmarks obtained from the training images. Unlike traditional statistical models of shape and intensity priors, the appearance model in this paper is derived from the posterior probabilities obtained from random forest classification. This probabilistic information is then used for the initialization and propagation of the statistical model. The proposed method achieves mean Dice Similarity Coefficient (DSC) value of 0.96±0.01, with a mean segmentation time of 0.67±0.02 seconds when validated with 24 images from 6 datasets with considerable shape, size, and intensity variations, in a leave-one-patient-out validation framework. The model achieves statistically significant t-test p-value<0.0001 in mean DSC and mean mean absolute distance (MAD) values compared to traditional statistical models of shape and intensity priors.

Precise and rapid ultrasound-based breast cancer diagnosis is essential for effective treatment. However, existing ultrasound image segmentation methods often fail to capture both global contextual features and fine-grained boundary details. This study proposes a dual-branch network architecture that combines the Swin Transformer and Segment Anything Model (SAM) to enhance breast ultrasound image (BUSI) segmentation accuracy and reliability. Our network integrates the global attention mechanism of the Swin Transformer with fine-grained boundary detection from SAM through a multi-stage feature fusion module. We evaluated our method against state-of-the-art methods on two datasets: the Breast Ultrasound Images dataset from Wuhan University (BUSI-WHU), which contains 927 images (560 benign and 367 malignant) with ground truth masks annotated by radiologists, and the public BUSI dataset. Performance was evaluated using mean Intersection-over-Union (mIoU), 95th percentile Hausdorff Distance (HD95) and Dice Similarity coefficients, with statistical significance assessed using two-tailed independent t-tests with Holm-Bonferroni correction ( ). On our proposed dataset, the network achieved a mIoU of 90.82% and a HD95 of 23.50 pixels, demonstrating significant improvements over current state-of-the-art methods with effect sizes for mIoU ranging from 0.38 to 0.61 (p 0.05). On the BUSI dataset, the network achieved a mIoU of 82.83% and a HD95 of 71.13 pixels, demonstrating comparable improvements with effect sizes for mIoU ranging from 0.45 to 0.58 (p 0.05). Our dual-branch network leverages the complementary strengths of Swin Transformer and SAM through a fusion mechanism, demonstrating superior breast ultrasound segmentation performance. Our code is publicly available at https://github.com/Skylanding/DSATNet.

MP43-02 LESSONS LEARNED IN APPLYING DEEP LEARNING TO FACILITATE PROSTATE MR-US FUSION BIOPSY WORKFLOW

Automatic superpixel-based segmentation method for breast ultrasound images

Early detection and treatment are crucial for improving the prognosis of breast cancer. While ultrasound imaging is widely utilized for screening, its interpretation is subjective and heavily dependent on clinician expertise. Although supervised models have shown great promise in medical image segmentation, they encounter challenges such as the need for large labeled datasets, extended training times, and limited generalization to external datasets. We propose TFSeg, a Training-Free automatic image segmentation framework based on Segment Anything Model 2 (SAM2). TFSeg eliminates the need for re-training new models and tuning hyper-parameters by generating image sequences through the retrieval of the top-k most similar images from the training set, using their mask prompts to guide SAM2 in achieving accurate and efficient segmentation. Our approach, validated on a breast ultrasound image dataset, achieved a Dice score of 82.66%, surpassing the performance of most supervised and training-free segmentation models. Notably, it achieved the highest precision (87.93%) among all the compared methods. By combining image retrieval with sequence generation, TFSeg leverages SAM2 to provide an efficient and effective solution for breast cancer ultrasound image segmentation and demonstrates potential for broader applications in medical image analysis. The Training-Free automatic image Segmentation framework addresses challenges associated with supervised models, making it a promising method for improving breast cancer diagnosis.

Objective. Automated biopsy needle segmentation in 3D ultrasound images can be used for biopsy navigation, but it is quite challenging due to the low ultrasound image resolution and interference similar to the needle appearance. For 3D medical image segmentation, such deep learning networks as convolutional neural network and transformer have been investigated. However, these segmentation methods require numerous labeled data for training, have difficulty in meeting the real-time segmentation requirement and involve high memory consumption. Approach. In this paper, we have proposed the temporal information-based semi-supervised training framework for fast and accurate needle segmentation. Firstly, a novel circle transformer module based on the static and dynamic features has been designed after the encoders for extracting and fusing the temporal information. Then, the consistency constraints of the outputs before and after combining temporal information are proposed to provide the semi-supervision for the unlabeled volume. Finally, the model is trained using the loss function which combines the cross-entropy and Dice similarity coefficient (DSC) based segmentation loss with mean square error based consistency loss. The trained model with the single ultrasound volume input is applied to realize the needle segmentation in ultrasound volume. Main results. Experimental results on three needle ultrasound datasets acquired during the beagle biopsy show that our approach is superior to the most competitive mainstream temporal segmentation model and semi-supervised method by providing higher DSC (77.1% versus 76.5%), smaller needle tip position (1.28 mm versus 1.87 mm) and length (1.78 mm versus 2.19 mm) errors on the kidney dataset as well as DSC (78.5% versus 76.9%), needle tip position (0.86 mm versus 1.12 mm) and length (1.01 mm versus 1.26 mm) errors on the prostate dataset. Significance. The proposed method can significantly enhance needle segmentation accuracy by training with sequential images at no additional cost. This enhancement may further improve the effectiveness of biopsy navigation systems.

Accurate automatic segmentation of the prostate in magnetic resonance images (MRI) is a challenging task due to the high variability of prostate anatomic structure. Artifacts such as noise and similar signal intensity of tissues around the prostate boundary inhibit traditional segmentation methods from achieving high accuracy. We investigate both patch-based and holistic (image-to-image) deep-learning methods for segmentation of the prostate. First, we introduce a patch-based convolutional network that aims to refine the prostate contour which provides an initialization. Second, we propose a method for end-to-end prostate segmentation by integrating holistically nested edge detection with fully convolutional networks. Holistically nested networks (HNN) automatically learn a hierarchical representation that can improve prostate boundary detection. Quantitative evaluation is performed on the MRI scans of 250 patients in fivefold cross-validation. The proposed enhanced HNN model achieves a mean ± standard deviation. A Dice similarity coefficient (DSC) of [Formula: see text] and a mean Jaccard similarity coefficient (IoU) of [Formula: see text] are used to calculate without trimming any end slices. The proposed holistic model significantly ([Formula: see text]) outperforms a patch-based AlexNet model by 9% in DSC and 13% in IoU. Overall, the method achieves state-of-the-art performance as compared with other MRI prostate segmentation methods in the literature.