Data Augmentation and Transfer Learning to Improve Generalizability of an Automated Prostate Segmentation Model.

Combinations of data augmentation methods and deep learning architectures for automatic pancreas segmentation on CT images are proposed and evaluated. Images from a public CT dataset of pancreas segmentation were used to evaluate the models. Baseline U-net and deep U-net were chosen for the deep learning models of pancreas segmentation. Methods of data augmentation included conventional methods, mixup, and random image cropping and patching (RICAP). Ten combinations of the deep learning models and the data augmentation methods were evaluated. Four-fold cross validation was performed to train and evaluate these models with data augmentation methods. The dice similarity coefficient (DSC) was calculated between automatic segmentation results and manually annotated labels and these were visually assessed by two radiologists. The performance of the deep U-net was better than that of the baseline U-net with mean DSC of 0.703–0.789 and 0.686–0.748, respectively. In both baseline U-net and deep U-net, the methods with data augmentation performed better than methods with no data augmentation, and mixup and RICAP were more useful than the conventional method. The best mean DSC was obtained using a combination of deep U-net, mixup, and RICAP, and the two radiologists scored the results from this model as good or perfect in 76 and 74 of the 82 cases.

Purpose To investigate the accuracy and robustness of prostate segmentation using deep learning across various training data sizes, MRI vendors, prostate zones, and testing methods relative to fellowship-trained diagnostic radiologists. Materials and Methods In this systematic review, Embase, PubMed, Scopus, and Web of Science databases were queried for English-language articles using keywords and related terms for prostate MRI segmentation and deep learning algorithms dated to July 31, 2022. A total of 691 articles from the search query were collected and subsequently filtered to 48 on the basis of predefined inclusion and exclusion criteria. Multiple characteristics were extracted from selected studies, such as deep learning algorithm performance, MRI vendor, and training dataset features. The primary outcome was comparison of mean Dice similarity coefficient (DSC) for prostate segmentation for deep learning algorithms versus diagnostic radiologists. Results Forty-eight studies were included. Most published deep learning algorithms for whole prostate gland segmentation (39 of 42 [93%]) had a DSC at or above expert level (DSC ≥ 0.86). The mean DSC was 0.79 ± 0.06 (SD) for peripheral zone, 0.87 ± 0.05 for transition zone, and 0.90 ± 0.04 for whole prostate gland segmentation. For selected studies that used one major MRI vendor, the mean DSCs of each were as follows: General Electric (three of 48 studies), 0.92 ± 0.03; Philips (four of 48 studies), 0.92 ± 0.02; and Siemens (six of 48 studies), 0.91 ± 0.03. Conclusion Deep learning algorithms for prostate MRI segmentation demonstrated accuracy similar to that of expert radiologists despite varying parameters; therefore, future research should shift toward evaluating segmentation robustness and patient outcomes across diverse clinical settings. Keywords: MRI, Genital/Reproductive, Prostate Segmentation, Deep Learning Systematic review registration link: osf.io/nxaev © RSNA, 2024.

Few-shot learning aims to obtain a better deep learning model through training with a smaller amount of data. In recent years, with the development and application of deep learning, deep learning has faced the problem of few-shot in many scenarios. Therefore, the study of few-shot learning will promote the application and deployment of deep learning in the industrial field. First of all, this survey specifically introduces two few-shot learning implementation methods: based on data augmentation and based on transfer learning. Then, this survey lists actual cases to illustrate the application of few-shot learning in industry. One case is about Few-shot parts surface defect detection based on metric learning. Another case is about poultry egg detection in few-shot based on meta learning. Finally, this survey summarizes the research achievements and research value of few-shot learning. Survey summarizes the problems and challenges faced by few-shot learning and the future development direction of few-shot learning.KeywordsFew-shot learningData augmentationTransfer learningMetric learningMeta learning

The application of deep learning for small object detection in Unmanned Aerial Vehicle (UAV) remote sensing imagery has garnered increasing attention across various domains. However, current detection methods for small objects face significant challenges, including background noise and poor data quality, which adversely affect their performance. This paper proposes an enhanced YOLOv8 model through Deep Convolutional Adversarial Network (DCGAN)-based data augmentation and transfer learning strategies to address these limitations. The data augmentation strategy employs DCGAN to emphasize the representation of small objects while mitigating the impact of background noise and data quality issues. Subsequently, transfer learning techniques with various fine-tuning strategies are applied to improve the performance of the existing YOLOv8 model in the context of UAV small object Detection. Experimental results on VisdroneDET-2019, CARPK, and HIT-UAV datasets demonstrate that the proposed YDGAN-TL model achieves superior performance, as indicated by metrics such as mean Average Precision (mAP) for small objects and a reduction in model parameters. Notably, the effectiveness of the YDGAN-TL model is further corroborated by enhanced performance metrics, including precision and recall, with mean values reaching 0.95 and 0.97, respectively. These findings suggest that the enhanced model offers promising results across multiple performance parameters, indicating its potential as a robust solution for UAV small object detection.

Jump Markov linear systems are linear systems whose parameters evolve with time according to a finite-state Markov chain. Given a set of observations, our aim is to estimate the states of the finite-state Markov chain and the continuous (in space) states of the linear system. The computational cost in computing conditional mean or maximum a posteriori (MAP) state estimates of the Markov chain or the state of the jump Markov linear system grows exponentially in the number of observations. We present three globally convergent algorithms based on stochastic sampling methods for state estimation of jump Markov linear systems. The cost per iteration is linear in the data length. The first proposed algorithm is a data augmentation (DA) scheme that yields conditional mean state estimates. The second proposed scheme is a stochastic annealing (SA) version of DA that computes the joint MAP sequence estimate of the finite and continuous states. Finally, a Metropolis-Hastings DA scheme based on SA is designed to yield the MAP estimate of the finite-state Markov chain. Convergence results of the three above-mentioned stochastic algorithms are obtained. Computer simulations are carried out to evaluate the performances of the proposed algorithms. The problem of estimating a sparse signal developing from a neutron sensor based on a set of noisy data from a neutron sensor and the problem of narrow-band interference suppression in spread spectrum code-division multiple-access (CDMA) systems are considered.

The problem of application of deep neural networks (DNNs) for extremely small dataset is considered to find the effect of dataset size and quality on performance metrics. Classification task is considered for the set of problems where images of Lyme disease symptoms were collected and structured in the open-access dataset with the different subsets of the whole dataset: clean images only (C), dirty images only (D), and clean and dirty images in the equal proportions (C_D) (https://www.kaggle.com/yoctoman/lyme-clean-and-dirty) . Two different data preprocessing techniques were applied: normalization without data augmentation (NODA) and normalization with data augmentation (DA). The two versions of the modern NASNet DNN architectures of different sizes were used, namely NASNetLarge and NASNetMobile, with the weights obtained after training on ImageNet dataset. These DNNs were trained in two experiments, namely with frozen layers, aka transfer learning (TL), and with unfrozen layers when all weights were updated (NOTL). After fivefold cross-validated training the best results for mean validation metrics were demonstrated by the models trained in C_NOTL_NODA, C_NOTL_DA, and C_TL_NODA experiments for NASNetLarge, and in C_TL_NODA by NASNetMobile. But the testing on the new unknown subset of Lyme disease images with the equal proportion of clean and dirty images gave quite different results. The models trained in the before-mentioned experiments demonstrated poor performance in comparison with the models trained in C_NOTL_DA, D_NOTL_DA experiments, and, especially in D_C_NOTL_DA ones. The successful training strategy should include tuning ImageNet-trained weights without their freezing (NOTL) and intensive data augmentation (DA), but it would not be enough if some diverse relevant background will not be added. That is why the additional methods of “context-relevant data augmentation” (CRDA) mimicked by inclusion of D_C data configuration should be considered. These findings can be used for the more effective selection of training experiments of DNNs in the healthcare context in the view of the small number of available medical data in practice.KeywordsNeural networksDeep learningSmall dataset problemData augmentationTransfer learningHealth careLyme disease

Enhanced Indonesian Ethnic Speaker Recognition using Data Augmentation Deep Neural Network

MP43-02 LESSONS LEARNED IN APPLYING DEEP LEARNING TO FACILITATE PROSTATE MR-US FUSION BIOPSY WORKFLOW

Breast density and background parenchymal enhancement (BPE) are suggested to be related to the risk of breast cancer. The first step to quantitative analysis of breast density and BPE is segmenting the breast from body. Nowadays, convolutional neural networks (CNNs) are widely used in image segmentation and work well in semantic segmentation, however, CNNs have been rarely used in breast region segmentation. In this paper, the CNN was employed to segment the breast region in transverse fat-suppressed breast dynamic contrast enhanced magnetic resonance imaging (DCE-MRI). Image normalization was initially performed. Subsequently, the dataset was divided into three sets randomly: train set validation set and test set. The 2-D U-Net was trained by train set and the optimum model was chosen by validation set. Finally, segmentation results of test set obtained by U-Net were adjusted in the postprocessing. In this step, two largest volumes were computed to determine whether the smaller volume is the scar after mastectomy. With the limitation of small dataset, 5-fold cross-validation and data augmentation were used in this study. Final results on the test set were evaluated by volume-based and boundary-based metrics with manual segmentation results. By using this method, the mean dice similarity coefficient (DSC), dice difference coefficient (DDC), and root-mean-square distance reached 97.44%, 5.11%, and 1.25 pixels, respectively.

Improved classification of oral cancer through a personalized transfer learning CNN architecture

Otosclerosis is a common disease of the middle ear leading to stapedial fixation. Its rapid and non-invasive diagnosis could be achieved through wideband tympanometry (WBT), but the interpretation of the raw data provided by this tool is complex and time-consuming. Convolutional neural networks (CNN) could potentially be applied to this situation to help the clinicians categorize WBT data. A dataset containing 135 samples from 80 patients with otosclerosis and 55 controls was obtained. We designed a lightweight CNN to categorize samples into the otosclerosis and control. Receiver operating characteristic (ROC) analysis showed an area under the curve (AUC) of 0.95 ±0.011, and the F1-score was 0.89 ±0.031 ( r=10). The performance was further improved by data augmentation schemes and transfer learning strategies (AUC: 0.97 ±0.010, F1-score: 0.94 ±0.016, , ANOVA). Finally, the most relevant diagnostic features employed by the CNN were assessed via the activation pattern heatmaps. These results are crucial for the visual interpretation of WBT graphic outputs which clinicians use in routine, and for a better understanding of the WBT signal in relation to the ossicular mechanics.

In robotics-assisted surgery, accurate segmentation of surgical instruments from background pixels is crucial for the success of various downstream tasks. Currently, popular segmentation models such as the U-Net exhibit excellent performance under major surgical conditions. However, when benchmarking the segmentation models under unforeseen surgical complications such as overbleeding, concerns toward their robustness remain. In fact, due to the lack of training datasets for the surgical segmentation models under complications, these models often exhibit degraded performances and thus result in failures for conducting accurate segmentations at the pixel level. In this paper, we leverage the Generative Adversarial Network (GAN), specifically through generating realistic images that simulate the condition of overbleeding to enlarge the training datasets a process known as data augmentation. With more diverse training data from different scenarios, segmentation models can be trained better and reduce the risk of errors. In addition, a dataset consisting of 17 mock endoscopic video sequences with binary masks indicating ground truths was used for training/testing. With the dataset, a U-Net architecture is used to examine performance through metrics such as the Normalized Surface Distance (NSD) and Dice Similarity Coefficient (DSC). As a result, integrations with GAN-based data augmentation allow segmentation models performances to improve 9.07% for NSD and 6.63% for DSC. From the results, our GAN-based augmentation is proven to be effective, providing a novel direction for improving models robustness during the training procedure, paving the way for future exploration and clinical translation of the cutting-edge deep learning models into real robotics surgery.

The accurate reconstruction of craniofacial defects requires the precise segmentation and mirroring of healthy anatomy. Conventional workflows rely on manual interaction, making them time-consuming and subject to operator variability. This study developed and validated a fully automated digital pipeline that integrates deep learning–based segmentation with algorithmic mirroring for craniofacial reconstruction. A total of 388 cranial CT scans were used to train a three-dimensional nnU-Net model for skull and mandible segmentation. A Principal Component Analysis–Iterative Closest Point (PCA–ICP) algorithm was then applied to compute the sagittal symmetry plane and perform mirroring. Automated results were compared with expert-generated segmentations and manually defined symmetry planes using Dice Similarity Coefficient (DSC), Mean Surface Distance (MSD), Hausdorff Distance (HD), and angular deviation. The nnU-Net achieved high segmentation accuracy for both the mandible (mean DSC 0.956) and the skull (mean DSC 0.965). Mirroring results showed minimal angular deviation from expert reference planes (mandible: 1.32° ± 0.71° in defect cases, 1.58° ± 1.12° in intact cases; skull: 1.75° ± 0.84° in defect cases, 1.15° ± 0.81° in intact cases). The presence of defects did not significantly affect accuracy. This automated workflow demonstrated robust performance and clinical applicability, offering standardized, reproducible, and time-efficient planning for craniofacial reconstruction.

Semi-supervised medical image segmentation methods based on mean teacher (MT) framework provide a promising means for addressing the dense prediction problems with limited annotated images and numerous unlabeled images. However, the confirmation bias caused by the distribution difference between labeled and unlabeled data and the parameters-coupling problem of MT prevent the model from further improving the segmentationperformance. To reduce confirmation bias and alleviate the parameter coupling problem in MT framework, a novel data augmentation strategy and a cross exponential moving averaging (crossEMA) architecture are proposed in thiswork. Specifically, a dual swap mixing data augmentation method was first proposed, which exchanges the patches between labeled and unlabeled images twice to decrease the confirmation bias caused by distribution divergency. Subsequently, a novel architecture for both student and teacher networks was designed with structurally identical dual decoders, one of which adopted a dropout operation. Labeled, unlabeled, and mixed images are fed into this MT architecture. For unlabeled data, the pseudo-labels generated by the dual decoders of the teacher network were used to supervise the predictions of the corresponding decoders of the student network. For mixed data, the real labels of the labeled data are mixed with the pseudo-labels of the unlabeled data predicted by the teacher network to form the supervisory information, which is used to constrain the prediction consistency for mixed data between student and teacher networks. To overcome the parameter coupling problem between the student and teacher networks, the encoder parameters of the teacher network were updated using an exponential moving average (EMA) strategy, while its dual decoder parameters were updated using a cross EMA strategy, which means the perturbed decoder parameters of the student network were updated with the non-perturbed decoder parameters of the student network and viceversa. By comparing with several state-of-the-art (SOTA) semi-supervised segmentation methods on four publicly available datasets, we validated that the proposed method outperforms existing models. The Dice similarity coefficient (DSC) and volume similarity (VS) were improved by at least 2.33% and 1.86%, respectively, compared to the corresponding sub-optimal methods. Through multiple ablation experiments, we verified that the proposed dual swap strategy can reduce the distributional differences between unlabeled data and labeled+mixed data. In addition, the cross EMA strategy can avoid early convergence of the student and teachernetworks. The proposed strategies can alleviate the confirmation bias caused by the distribution discrepancy between labeled and unlabeled data in semi-supervised learning, as well as the issue of parameter coupling between the student and teacher networks in the MT architecture, providing therefore a promising approach to semi-supervised medical image segmentation.

Much research on object detection focuses on building better model architectures and detection algorithms. Changing the model architecture, however, comes at the cost of adding more complexity to inference, making models slower. Data augmentation, on the other hand, doesn’t add any inference complexity, but is insufficiently studied in object detection for two reasons. First it is more difficult to design plausible augmentation strategies for object detection than for classification, because one must handle the complexity of bounding boxes if geometric transformations are applied. Secondly, data augmentation attracts less research attention perhaps because it is believed to add less value and to transfer poorly compared to advances in network architectures.This paper serves two main purposes. First, we propose to use AutoAugment [3] to design better data augmentation strategies for object detection because it can address the difficulty of designing them. Second, we use the method to assess the value of data augmentation in object detection and compare it against the value of architectures. Our investigation into data augmentation for object detection identifies two surprising results. First, by changing the data augmentation strategy to our method, AutoAugment for detection, we can improve RetinaNet with a ResNet-50 backbone from 36.7 to 39.0 mAP on COCO, a difference of +2.3 mAP. This gain exceeds the gain achieved by switching the backbone from ResNet-50 to ResNet-101 (+2.1 mAP), which incurs additional training and inference costs. The second surprising finding is that our strategies found on the COCO dataset transfer well to the PASCAL dataset to improve accuracy by +2.7 mAP. These results together with our systematic studies of data augmentation call into question previous assumptions about the role and transferability of architectures versus data augmentation. In particular, changing the augmentation may lead to performance gains that are equally transferable as changing the underlying architecture.