A Regional-Attentive Multi-Task Learning Framework for Breast Ultrasound Image Segmentation and Classification

The segmentation and classification of breast ultrasound (BUS) images are crucial for the early diagnosis of breast cancer and remain a key focus in BUS image processing. Numerous machine learning and deep learning algorithms have shown their effectiveness in the segmentation and diagnosis of BUS images. In this work, we propose a multi-task learning network with an object contextual attention module (MTL-OCA) for the segmentation and classification of BUS images. The proposed method utilizes the object contextual attention module to capture pixel-region relationships, enhancing the quality of segmentation masks. For classification, the model leverages high-level features extracted from unenhanced segmentation masks to improve accuracy. Cross-validation on a public BUS dataset demonstrates that MTL-OCA outperforms several current state-of-the-art methods, achieving superior results in both classification and segmentation tasks.

Automatic superpixel-based segmentation method for breast ultrasound images

Nowadays, inspired by the great success of Transformers in Natural Language Processing, many applications of Vision Transformers (ViTs) have been investigated in the field of medical image analysis including breast ultrasound (BUS) image segmentation and classification. In this paper, we propose an efficient multi-task framework to segment and classify tumors in BUS images using hybrid convolutional neural networks (CNNs)-ViTs architecture and Multi-Perceptron (MLP)-Mixer. The proposed method uses a two-encoder architecture with EfficientNetV2 backbone and an adapted ViT encoder to extract tumor regions in BUS images. The self-attention (SA) mechanism in the Transformer encoder allows capturing a wide range of high-level and complex features while the EfficientNetV2 encoder preserves local information in image. To fusion the extracted features, a Channel Attention Fusion (CAF) module is introduced. The CAF module selectively emphasizes important features from both encoders, improving the integration of high-level and local information. The resulting feature maps are reconstructed to obtain the segmentation maps using a decoder. Then, our method classifies the segmented tumor regions into benign and malignant using a simple and efficient classifier based on MLP-Mixer, that is applied for the first time, to the best of our knowledge, for the task of lesion classification in BUS images. Experimental results illustrate the outperformance of our framework compared to recent works for the task of segmentation by producing 83.42% in terms of Dice coefficient as well as for the classification with 86% in terms of accuracy.

Computer Aided Segmentation of Breast Ultrasound Images Using Scale Invariant Feature Transform (SIFT) and Bag Of Features

Breast ultrasound (BUS) is an important breast imaging tool. Automatic BUS image segmentation can measure the breast tumor size objectively and reduce doctors' workload. In this article, we proposed a deep supervised transformer U-shaped full-resolution residual network (DSTransUFRRN) to segment BUS images. In the proposed method, a full-resolution residual stream and a deep supervision mechanism were introduced into TransU-Net. The residual stream can keep full resolution features from different levels and enhance features fusion. Then, the deep supervision can suppress gradient dispersion. Moreover, the transformer module can suppress irrelevant features and improve feature extraction process. Two datasets (dataset A and B) were used for training and evaluation. The dataset A included 980 BUS image samples and the dataset B had 163 BUS image samples. Cross-validation was conducted. For the dataset A, the proposed DSTransUFRRN achieved significantly higher Dice (91.04±0.86%) than all compared methods (p<0.05). For the dataset B, the Dice was lower than that for the dataset A due to the small number of samples, but the Dice of DSTransUFRRN (88.15%±2.11%) was significantly higher than that of other compared methods (p<0.05). In this study, we proposed DSTransUFRRN for BUS image segmentation. The proposed methods achieved significantly higher accuracy than the compared previous methods.

EH-former: Regional easy-hard-aware transformer for breast lesion segmentation in ultrasound images

Breast cancer is one of the deadliest cancer types affecting women worldwide. As with all types of cancer, early detection of breast cancer is of vital importance. Early diagnosis plays an important role in reducing deaths and fighting cancer. Ultrasound (US) imaging is a painless and common technique used in the early detection of breast cancer. In this article, deep learning-based approaches for the classification of breast US images have been extensively reviewed. Classification performance of breast US images of architectures such as AlexNet, VGG, ResNet, GoogleNet and EfficientNet, which are among the most basic CNN architectures, has been compared. Then, transformer models, which are one of the most popular deep learning architectures these days and show similar performance to the performance of CNN' architectures in medical images, are examined. BUSI, the only publicly available dataset, was used in experimental studies. Experimental studies have shown that the transformer and CNN models successfully classify US images of the breast. It has been observed that vision transformer model outperforms other models with 88.6% accuracy, 90.1% precison, 87.4% recall and 88.7% F1-score. This study shows that deep learning architectures are successful in classification of US images and can be used in the clinic experiments in the near future.

BackgroundClassification of breast ultrasound (BUS) images is an important step in the computer-aided diagnosis (CAD) system for breast cancer. In this paper, a novel phase-based texture descriptor is proposed for efficient and robust classifiers to discriminate benign and malignant tumors in BUS images.MethodThe proposed descriptor, namely the phased congruency-based binary pattern (PCBP) is an oriented local texture descriptor that combines the phase congruency (PC) approach with the local binary pattern (LBP). The support vector machine (SVM) is further applied for the tumor classification. To verify the efficiency of the proposed PCBP texture descriptor, we compare the PCBP with other three state-of-art texture descriptors, and experiments are carried out on a BUS image database including 138 cases. The receiver operating characteristic (ROC) analysis is firstly performed and seven criteria are utilized to evaluate the classification performance using different texture descriptors. Then, in order to verify the robustness of the PCBP against illumination variations, we train the SVM classifier on texture features obtained from the original BUS images, and use this classifier to deal with the texture features extracted from BUS images with different illumination conditions (i.e., contrast-improved, gamma-corrected and histogram-equalized). The area under ROC curve (AUC) index is used as the figure of merit to evaluate the classification performances.Results and conclusionsThe proposed PCBP texture descriptor achieves the highest values (i.e. 0.894) and the least variations in respect of the AUC index, regardless of the gray-scale variations. It’s revealed in the experimental results that classifications of BUS images with the proposed PCBP texture descriptor are efficient and robust, which may be potentially useful for breast ultrasound CADs.

Ultrasound imaging is a crucial modality for breast cancer diagnosis, with lesion segmentation being an essential component of computer-aided diagnosis systems. However, the high cost and effort associated with pixel-level annotation significantly hinder the advancement of breast ultrasound (BUS) image segmentation algorithms. In this paper, we propose a multi-level semi-supervised generative adversarial network (MLSS-GAN) that incorporates a generator and a novel multi-level classifier acting as the discriminator. While some previous approaches have incorporated multi-level adversarial learning, our discriminator employs an encoder for feature extraction and two decoders designed for distinct image-level and pixel-level adversarial tasks, enabling more comprehensive segmentation supervision. Specifically, one decoder performs image-level classification, while the other conducts pixel-level classification. This multi-level adversarial strategy compels the generator to produce images that closely resemble real BUS images at different feature representations, thereby enriching the classification feature space. To assess the effectiveness of the proposed method, comprehensive comparisons with both fully supervised and semi-supervised approaches are conducted on the BUSI dataset. Experimental results demonstrated the superior performance of our method, especially under conditions with only a limited number of annotated BUS images.

Due to the low contrast, blurry boundaries, and large amount of shadows in breast ultrasound (BUS) images, automatic tumor segmentation remains a challenging task. Deep learning provides a solution to this problem, since it can effectively extract representative features from lesions and the background in BUS images. A novel automatic tumor segmentation method is proposed by combining a dilated fully convolutional network (DFCN) with a phase-based active contour (PBAC) model. The DFCN is an improved fully convolutional neural network with dilated convolution in deeper layers, fewer parameters, and batch normalization techniques; and has a large receptive field that can separate tumors from background. The predictions made by the DFCN are relatively rough due to blurry boundaries and variations in tumor sizes; thus, the PBAC model, which adds both region-based and phase-based energy functions, is applied to further improve segmentation results. The DFCN model is trained and tested in dataset 1 which contains 570BUS images from 89 patients. In dataset 2, a 10-fold support vector machine (SVM) classifier is employed to verify the diagnostic ability using 460 features extracted from the segmentation results of the proposed method. Advantages of the present method were compared with three state-of-the-art networks; the FCN-8s, U-net, and dilated residual network (DRN). Experimental results from 170BUS images show that the proposed method had a Dice Similarity coefficient of 88.97±10.01%, a Hausdorff distance (HD) of 35.54±29.70 pixels, and a mean absolute deviation (MAD) of 7.67±6.67 pixels, which showed the best segmentation performance. In dataset 2, the area under curve (AUC) of the 10-fold SVM classifier was 0.795 which is similar to the classification using the manual segmentation results. The proposed automatic method may be sufficiently accurate, robust, and efficient for medical ultrasound applications.

Segmentation of breast ultrasound images is a crucial and challenging task in computer-aided diagnosis systems. Accurately segmenting masses in benign and malignant cases and identifying regions with no mass is a primary objective in breast ultrasound image segmentation. Deep learning (DL) has emerged as a powerful tool in medical image segmentation, revolutionizing how medical professionals analyze and interpret complex imaging data. The UNet architecture is a highly regarded and widely used DL model in medical image segmentation. Its distinctive architectural design and exceptional performance have made it popular among researchers. With the increase in data and model complexity, optimization and fine-tuning models play a vital and more challenging role than before. This paper presents a comparative study evaluating the effect of image preprocessing and different optimization techniques and the importance of fine-tuning different UNet segmentation models for breast ultrasound images. Optimization and fine-tuning techniques have been applied to enhance the performance of UNet, Sharp UNet, and Attention UNet. Building upon this progress, we designed a novel approach by combining Sharp UNet and Attention UNet, known as Sharp Attention UNet. Our analysis yielded the following quantitative evaluation metrics for the Sharp Attention UNet: the Dice coefficient, specificity, sensitivity, and F1 score values obtained were 0.93, 0.99, 0.94, and 0.94, respectively. In addition, McNemar’s statistical test was applied to assess significant differences between the approaches. Across a number of measures, our proposed model outperformed all other models, resulting in improved breast lesion segmentation.

Precise and rapid ultrasound-based breast cancer diagnosis is essential for effective treatment. However, existing ultrasound image segmentation methods often fail to capture both global contextual features and fine-grained boundary details. This study proposes a dual-branch network architecture that combines the Swin Transformer and Segment Anything Model (SAM) to enhance breast ultrasound image (BUSI) segmentation accuracy and reliability. Our network integrates the global attention mechanism of the Swin Transformer with fine-grained boundary detection from SAM through a multi-stage feature fusion module. We evaluated our method against state-of-the-art methods on two datasets: the Breast Ultrasound Images dataset from Wuhan University (BUSI-WHU), which contains 927 images (560 benign and 367 malignant) with ground truth masks annotated by radiologists, and the public BUSI dataset. Performance was evaluated using mean Intersection-over-Union (mIoU), 95th percentile Hausdorff Distance (HD95) and Dice Similarity coefficients, with statistical significance assessed using two-tailed independent t-tests with Holm-Bonferroni correction ( ). On our proposed dataset, the network achieved a mIoU of 90.82% and a HD95 of 23.50 pixels, demonstrating significant improvements over current state-of-the-art methods with effect sizes for mIoU ranging from 0.38 to 0.61 (p 0.05). On the BUSI dataset, the network achieved a mIoU of 82.83% and a HD95 of 71.13 pixels, demonstrating comparable improvements with effect sizes for mIoU ranging from 0.45 to 0.58 (p 0.05). Our dual-branch network leverages the complementary strengths of Swin Transformer and SAM through a fusion mechanism, demonstrating superior breast ultrasound segmentation performance. Our code is publicly available at https://github.com/Skylanding/DSATNet.

Ultrasound imaging is an effective approach for diagnosing breast cancer, but it is highly operator-dependent. Recent advances in computer-aided diagnosis have suggested that it can assist physicians in diagnosis. Definition of the region of interest before computer analysis is still needed. Since manual outlining of the tumor contour is tedious and time-consuming for a physician, developing an automatic segmentation method is important for clinical application. The present paper represents a novel method to segment breast ultrasound images. It utilizes a combination of region-based active contour and neutrosophic theory to overcome the natural properties of ultrasound images including speckle noise and tissue-related textures. First, due to inherent speckle noise and low contrast of these images, we have utilized a non-local means filter and fuzzy logic method for denoising and image enhancement, respectively. This paper presents an improved weighted region-scalable active contour to segment breast ultrasound images using a new feature derived from neutrosophic theory. This method has been applied to 36 breast ultrasound images. It generates true-positive and false-positive results, and similarity of 95%, 6%, and 90%, respectively. The purposed method indicates clear advantages over other conventional methods of active contour segmentation, i.e., region-scalable fitting energy and weighted region-scalable fitting energy.

The accurate segmentation of breast ultrasound images is an important precondition for the lesion determination. The existing segmentation approaches embrace massive parameters, sluggish inference speed, and huge memory consumption. To tackle this problem, we propose T 2KD Attention U-Net (dual-Teacher Knowledge Distillation Attention U-Net), a lightweight semantic segmentation method combined double-path joint distillation in breast ultrasound images. Primarily, we designed two teacher models to learn the fine-grained features from each class of images according to different feature representation and semantic information of benign and malignant breast lesions. Then we leveraged the joint distillation to train a lightweight student model. Finally, we constructed a novel weight balance loss to focus on the semantic feature of small objection, solving the unbalance problem of tumor and background. Specifically, the extensive experiments conducted on Dataset BUSI and Dataset B demonstrated that the T 2KD Attention U-Net outperformed various knowledge distillation counterparts. Concretely, the accuracy, recall, precision, Dice, and mIoU of proposed method were 95.26%, 86.23%, 85.09%, 83.59%and 77.78% on Dataset BUSI, respectively. And these performance indexes were 97.95%, 92.80%, 88.33%, 88.40% and 82.42% on Dataset B, respectively. Compared with other models, the performance of this model was significantly improved. Meanwhile, compared with the teacher model, the number, size, and complexity of student model were significantly reduced (2.2×10 6 vs. 106.1×10 6, 8.4 MB vs. 414 MB, 16.59 GFLOPs vs. 205.98 GFLOPs, respectively). Indeedy, the proposed model guarantees the performances while greatly decreasing the amount of computation, which provides a new method for the deployment of clinical medical scenarios.

As a significant global health threat, breast cancer needs accurate ultrasound images segmentation for early intelligent diagnosis. Medical image segmentation has made extensive use of the well-known CNNs, transformers, and their variations. However, because of the intrinsic restrictions resulting from the intense complexity of transformers and breast ultrasound pictures, high performance and fast speed cannot be accomplished simultaneously. The research proposes a unique BS-Mamba architecture for rapid and reliable tumor segmentation of breast ultrasound images. It incorporates three new modules to extract multi-level feature for improving the breast image segmentation performance and speed. On the one hand, the BreastSegMamba block effectively captures both global and local high-resolution information, and processes complex features fast. On the other hand, the hybrid attention block is designed to strengthen the extraction of positional and channel features for accurate segmentation. Additionally, the data-sensitive fuzzy logic function focuses on the edges of the ultrasound images to enhance features in critical areas for higher segmentation performance. Through comprehensive experiments and ablation studies on the BUSI and Dataset B breast ultrasound datasets, BS-Mamba significantly outperforms the current state-of-the-art methods in terms of segmentation accuracy. With a Dice coefficient of 92.13% and an IoU of 86.42% on the BUSI dataset and a Dice coefficient of 92.62% and an IoU of 86.67% on Dataset B, respectively, BS-Mamba demonstrated exceptional segmentation performance. Furthermore, in comparison to other established models, BS-Mamba achieves a training time acceleration of up to 2.88×, further substantiating its computational efficiency.