Source-Resilient Joint Learning Framework for Preserving Stable Generalization on Diverse Ultrasonic Source Scenarios.

Medical Ultrasound (US), despite its wide use, is characterized by artefacts and operator dependency. Those attributes hinder the gathering and utilization of US datasets for the training of deep neural networks used for computer-assisted intervention systems. Data augmentation is commonly used to enhance model generalization and performance. However, common data augmentation techniques, such as affine transformations do not align with the physics of US and, when used carelessly can lead to unrealistic US images. To this end, we propose a set of physics-inspired transformations, including deformation, reverb and signal-to-noise ratio, that we apply on US B-mode images for data augmentation. We evaluate our method on a new spine US dataset for the tasks of bone segmentation and classification.

Ultrasonography plays an essential role in breast cancer diagnosis. Current deep learning based studies train the models on either images or videos in a centralized learning manner, lacking consideration of joint benefits between two different modality models or the privacy issue of data centralization. In this study, we propose the first decentralized learning solution for joint learning with breast ultrasound video and image, called FedBCD. To enable the model to learn from images and videos simultaneously and seamlessly in client-level local training, we propose a Joint Ultrasound Video and Image Learning (JUVIL) model to bridge the dimension gap between video and image data by incorporating temporal and spatial adapters. The parameter-efficient design of JUVIL with trainable adapters and frozen backbone further reduces the computational cost and communication burden of federated learning, finally improving the overall efficiency. Moreover, considering conventional model-wise aggregation may lead to unstable federated training due to different modalities, data capacities in different clients, and different functionalities across layers. We further propose a Fisher information matrix (FIM) guided Layer-wise Aggregation method named FILA. By measuring layer-wise sensitivity with FIM, FILA assigns higher contributions to the clients with lower sensitivity, improving personalized performance during federated training. Extensive experiments on three image clients and one video client demonstrate the benefits of joint learning architecture, especially for the ones with small-scale data. FedBCD significantly outperforms nine federated learning methods on both video-based and image-based diagnoses, demonstrating the superiority and potential for clinical practice. Code is released at https://github.com/tianpeng-deng/FedBCD.

Accurate segmentation of gastric cavities from ultrasound images remains a challenging task due to the presence of ultrasound shadow and varying anatomical structures. To address these challenges, we collected a Gastric Ultrasound Image (GUSI) dataset using transabdominal techniques, after administering an echoic cellulose-based gastric ultrasound contrast agent (TUS-OCCA), and annotated the gastric cavity regions. We propose a model called Shadow Adaptive Tracing U-net (SATU-net) for gastric cavity segmentation on the GUSI dataset. SATU-net is specifically designed for gastric cavity segmentation in ultrasound images. The method introduces an Adaptive Shadow Tracing Module (ASTM), Shadow Separation Module (SSM), and an affine transformation mechanism to mitigate the impact of ultrasound shadow. The affine transformation aligns ultrasound image regions to reduce geometric distortion, while the ASTM dynamically tracks and compensates for ultrasound shadow, and the SSM extracts the shadow separation image. Extensive experiments on the gastric ultrasound dataset demonstrate that SATU-net achieves superior segmentation performance compared to several state-of-the-art deep learning methods, with an IoU improvement of 2.26% over the second-best competitor. Further robustness analysis and limited external validation provide preliminary evidence that SATU-net generalizes across diverse clinical scenarios. Our method provides a robust solution for ultrasound image segmentation and can be extended to other medical imaging tasks. Additionally, the ASTM module can be flexibly applied to existing network frameworks.

Purpose: Herein we describe the development of a dynamic fusion technology for ultrasound (US) and other static 3D data sets. We tested whether the high US frame rate is sufficient to accurately track objects subject to respiratory motion. In addition, we used the spatial registration functions of the system to test whether aggressive compounding of US images would be efficacious in terms of artifact reduction and boundary detection. Materials/Methods: The system's dynamic spatial accuracy was tested using a phantom translated at a velocity of either 4 or 16 mm/s. To test inaccuracies due to video processing we offset the arm and video frame data in single frame increments. Static 3D US data sets were acquired incorporating an acoustic obstruction. Data sets were reconstructed using images from multiple angles of interrogation. Similar experiments were performed to assess the system's ability to reduce reverberation artifacts and tested on a human subject. Results: By correcting for video processing latency, the system was able to track object motion to 0.2 and 0.5 mm for speeds of 4 and 16 mm/s, respectively. Aggressive image compounding was able to reveal objects that were otherwise obstructed from a single view. Reverberation artifacts were reduced, in addition to enhancing certain object boundaries . These results were present although less obvious in early human testing. Conclusions: Our imaging system is spatially accurate over a range that permits interrogation of an adult abdomen from a wide range of angles. The system's ability to track respiratory motion has been demonstrated. Aggressive compounding demonstrated that obstructed test objects could be visualized by non‐intelligently summing images from varying angles. Soft tissue boundaries and other US artifacts may also benefit from application of the technology.Supported by NIH R01EB002899.

Cross-modal Transfer Learning Based on an Improved CycleGAN Model for Accurate Kidney Segmentation in Ultrasound Images

We propose a novel BIRADS-SSDL network that integrates clinically-approved breast lesion characteristics (BIRADS features) into task-oriented semi-supervised deep learning (SSDL) for accurate diagnosis of ultrasound (US) images with a small training dataset. Breast US images are converted to BIRADS-oriented feature maps (BFMs) using a distance-transformation coupled with a Gaussian filter. Then, the converted BFMs are used as the input of an SSDL network, which performs unsupervised stacked convolutional auto-encoder (SCAE) image reconstruction guided by lesion classification. This integrated multi-task learning allows SCAE to extract image features with the constraints from the lesion classification task, while the lesion classification is achieved by utilizing the SCAE encoder features with a convolutional network. We trained the BIRADS-SSDL network with an alternative learning strategy by balancing the reconstruction error and classification label prediction error. To demonstrate the effectiveness of our approach, we evaluated it using two breast US image datasets. We compared the performance of the BIRADS-SSDL network with conventional SCAE and SSDL methods that use the original images as inputs, as well as with an SCAE that use BFMs as inputs. The experimental results on two breast US datasets show that BIRADS-SSDL ranked the best among the four networks, with a classification accuracy of around 94.23 ± 3.33% and 84.38 ± 3.11% on two datasets. In the case of experiments across two datasets collected from two different institutions/and US devices, the developed BIRADS-SSDL is generalizable across the different US devices and institutions without overfitting to a single dataset and achieved satisfactory results. Furthermore, we investigate the performance of the proposed method by varying the model training strategies, lesion boundary accuracy, and Gaussian filter parameters. The experimental results showed that a pre-training strategy can help to speed up model convergence during training but with no improvement of the classification accuracy on the testing dataset. The classification accuracy decreases as the segmentation accuracy decreases. The proposed BIRADS-SSDL achieves the best results among the compared methods in each case and has the capacity to deal with multiple different datasets under one model. Compared with state-of-the-art methods, BIRADS-SSDL could be promising for effective breast US computer-aided diagnosis using small datasets.

Deep Neural Networks (DNNs) suffer from the performance degradation when image appearance shift occurs, especially in ultrasound (US) image segmentation. In this paper, we propose a novel and intuitive framework to remove the appearance shift, and hence improve the generalization ability of DNNs. Our work has three highlights. First, we follow the spirit of universal style transfer to remove appearance shifts, which was not explored before for US images. Without sacrificing image structure details, it enables the arbitrary style-content transfer. Second, accelerated with Adaptive Instance Normalization block, our framework achieved real-time speed required in the clinical US scanning. Third, an efficient and effective style image selection strategy is proposed to ensure the target-style US image and testing content US image properly match each other. Experiments on two large US datasets demonstrate that our methods are superior to state-of-the-art methods on making DNNs robust against various appearance shifts.

Breast cancer diagnosis using three-dimensional ultrasound and pixel relation analysis.

Previous research in active learning has primarily focused on selecting examples from closed-set data, which consists solely of unlabeled examples from the target classes. However, this approach overlooks the more prevalent scenario of open-set data in real-world applications. Open-set data encompasses examples from both target classes and non-target classes. To fill this gap, we propose a novel framework called JOSAL, which enhances the accuracy of the classifier by precisely selecting the target class examples from open-set data. The JOSAL framework introduces the concept of joint learning, where the Sampler and Classifier components perform sampling and classification tasks, respectively, by sharing example features extracted from a pre-trained Encoder. To maximize the classification accuracy of the Classifier, the framework adopts a novel joint learning strategy. This strategy initially prioritizes optimizing the Sampler and gradually shifts the optimization attention to the Classifier. The experimental results demonstrate that, compared to baselines, our approach exhibits stronger sampling precision and achieves higher classification accuracy. To the best of our knowledge, this is the first work to address the open-set active learning problem using the joint learning paradigm.

DSEU-net: A novel deep supervision SEU-net for medical ultrasound image segmentation

Due to the rapid increase in wireless communication traffic in recent years, load balancing is becoming increasingly important for ensuring the quality of service. However, variations in traffic patterns near different serving base stations make this task challenging. On one hand, crafting a single control policy that performs well across all base station sectors is often difficult. On the other hand, maintaining separate controllers for every sector introduces overhead, and leads to redundancy if some of the sectors experience similar traffic patterns. In this paper, we propose to construct a concise set of controllers that cover a wide range of traffic scenarios, allowing the operator to select a suitable controller for each sector based on local traffic conditions. To construct these controllers, we present a method that clusters similar scenarios and learns a general control policy for each cluster. We use deep reinforcement learning (RL) to first train separate control policies on diverse traffic scenarios, and then incrementally merge together similar RL policies via knowledge distillation. Experimental results show that our concise policy set reduces redundancy with very minor performance degradation compared to policies trained separately on each traffic scenario. Our method also outperforms handcrafted control parameters, joint learning on all tasks, and two popular clustering methods.

4D ultrasound is one of the most advancing visualization techniques of the ultrasound dataset. it is subdivided into main five visualization modes volume render, maximum, minimum, mean, and the surface render. The surface render visualization mode is the most realistic mode among the five modes of visualizing the ultrasound 3D dataset. We enhance the rendered image quality of the surface render mode by enhancing the pre-processing of the 2D ultrasound image. Surface render is subdivided into four major stages, 2D image pre-processing, volume render with surface detection, surface shading and finally post-processing. Surface render mode gives a low rendered image quality because of the speckle noise nature of the ultrasound, which is the reason for the coherent nature of the ultrasound imaging. And so, pre-processing of the 2D image is needed to remove the speckle noise keeping and enhancing the surface edges. Our processing meets the Three major challenges which must be concerned when designing the pre-processing of the 2D ultrasound image. Firstly, robustly smooth the speckle noise. Secondly, preserves and enhances the organs edges. and finally, the time constrains must be met that is because of this processing is part of many intensive processing stages. in addition, the processing implemented on the current GPUs platform, which shows a very high processing power with significant low coast value. A recent review of the literature, on this topic found that, a vast publication in the area of the 4D rendering pipeline [1] and 2D image enhancement each separately, despite this interest, no one to the best of our knowledge has studied the suitable 2D image pre-processing to get the best 4D data visualization [2], considering the nature of the ultrasound image [3] and the time constrains of the ultrasound physics. Our processing is a merge between local statistics and non-linear filters. It guarantees the required suitable quality of the 2D ultrasound image and performance constrains.

3D ultrasound (US) to computed tomography (CT) registration is a topic of significant interest because it can potentially improve many minimally invasive procedures such as laparoscopic partial nephrectomy. Partial nephrectomy patients often receive preoperative CT angiography, which helps define the important structures of the kidney such as the vasculature. Intraoperatively, dynamic real-time imaging information can be captured using ultrasound and compared with the preoperative data. Providing accurate registration between the two modalities would enhance navigation and guidance for the surgeon. However, one of the major problems of developing and evaluating registration techniques is obtaining sufficiently accurate and realistic phantom data especially for soft tissue. We present a detailed procedure for constructing tissue phantoms using porcine kidneys, which incorporates contrast agent into the tissue such that the kidneys appear representative of in vivo human CT angiography. These phantoms are also imaged with US and resemble US images from human patients. We then perform registration on corresponding CT and US datasets using a simulation-based algorithm. The method simulates an US image from the CT, generating an intermediate modality that resembles ultrasound. This simulated US is then registered to the original US dataset. Embedded fiducial markers provide a gold standard for registration. Being able to test our registration method on realistic datasets facilitates the development of novel CT to US registration techniques such that we can generate an effective method for human studies.

Polycystic Ovary Syndrome (PCOS) is a common endocrine disorder in women that is frequently diagnosed by ultrasound imaging by examining ovarian abnormality. Ultrasound images, though, are normally contaminated with speckle noise that degrades image quality and poses difficulties for diagnosis. Conventional denoising methods like mean and median filtering are unable to eliminate noise properly while maintaining fine details. To solve this problem, this paper introduces a new Attention-based Autoencoder (AAE) for denoising PCOS ultrasound images. The model uses an attention mechanism to selectively amplify significant image areas and suppress noise, enhancing image quality for diagnosis. The introduced method was tested on a publicly available ultrasound dataset with synthetic speckle noise at various levels (variance 0.5, 0.02, and 0.001). Experimental results prove that the suggested method performs better than conventional denoising methods, with peak signal-to-noise ratio (PSNR) values of 31.33, 34.25, and 36.23, respectively. The structural similarity index measure (SSIM) also reveals notable improvements and corresponding scores of 85.21, 92.33, and 99.25. Beyond technical performance, this work supports the development of scalable, AI-driven PCOS diagnostic tools within a technopreneurship incubator model. These results indicate that AAEs can improve ultrasound image quality, facilitating more accurate PCOS diagnosis.

The joint learning of multi-resolution feature for multi-class retinal vessel segmentation