GAN Inversion for Data Augmentation to Improve Colonoscopy Lesion Classification.

PurposeTo develop generative adversarial networks (GANs) that synthesize realistic anterior segment optical coherence tomography (AS-OCT) images and evaluate deep learning (DL) models that are trained on real and synthetic datasets for detecting angle closure.MethodsThe GAN architecture was adopted and trained on the dataset with AS-OCT images collected from the Joint Shantou International Eye Center of Shantou University and the Chinese University of Hong Kong, synthesizing open- and closed-angle AS-OCT images. A visual Turing test with two glaucoma specialists was performed to assess the image quality of real and synthetic images. DL models, trained on either real or synthetic datasets, were developed. Using the clinicians’ grading of the AS-OCT images as the reference standard, we compared the diagnostic performance of open-angle vs. closed-angle detection of DL models and the AS-OCT parameter, defined as a trabecular-iris space area 750 µm anterior to the scleral spur (TISA750), in a small independent validation dataset.ResultsThe GAN training included 28,643 AS-OCT anterior chamber angle (ACA) images. The real and synthetic datasets for DL model training have an equal distribution of open- and closed-angle images (all with 10,000 images each). The independent validation dataset included 238 open-angle and 243 closed-angle AS-OCT ACA images. The image quality of real versus synthetic AS-OCT images was similar, as assessed by the two glaucoma specialists, except for the scleral spur visibility. For the independent validation dataset, both DL models achieved higher areas under the curve compared with TISA750. Two DL models had areas under the curve of 0.97 (95% confidence interval, 0.96–0.99) and 0.94 (95% confidence interval, 0.92–0.96).ConclusionsThe GAN synthetic AS-OCT images appeared to be of good quality, according to the glaucoma specialists. The DL models, trained on all-synthetic AS-OCT images, can achieve high diagnostic performance.Translational RelevanceThe GANs can generate realistic AS-OCT images, which can also be used to train DL models.

Deep learning methods have made remarkable strides in surface defect detection. But, they heavily rely on large amount of training data, which can be a costly endeavor, especially for specific applications like steel strip surface defect detection, where acquiring and labeling large-scale data is impractical due to the rarity of certain defective categories in production environment. Hence, realistic defect image synthesis can greatly alleviate this issue. However, training image generation networks also demand substantial data, making image data augmentation merely an auxiliary effort. In this work, we propose a Generative Adversarial Network (GAN)-based image synthesis framework. We selectively extract the defect edges of the original image as well as the background texture information, and use them as network input through the spatially-adaptive (de)normalization (SPADE) module. This enriches the input information, thus significantly reducing the amount of training data for GAN network in image generation, and enhancing the background details as well as the defect boundaries in the generated images. Additionally, we introduce a novel generator loss term that balances the similarity and perceptual fidelity between synthetic and real images by constraining high-level features at different feature levels. This provides more valuable information for data augmentation in training object detection models using synthetic images. Our experimental results demonstrate the sophistication of the proposed image synthesis method and its effectiveness in data augmentation for steel strip surface defect detection tasks.

Ultra-high field 7T MRI can provide excellent tissue contrast and anatomical details, but is often cost prohibitive, and is not widely accessible in clinical practice. To generate synthetic 7T images from widely acquired 3T images with deep learning and to evaluate the feasibility of this approach for brain imaging. Prospective. 33 healthy volunteers and 89 patients with brain diseases, divided into training, and evaluation datasets in the ratio 4:1. T1-weighted nonenhanced or contrast-enhanced magnetization-prepared rapid acquisition gradient-echo sequence at both 3T and 7T. A generative adversarial network (SynGAN) was developed to produce synthetic 7T images from 3T images as input. SynGAN training and evaluation were performed separately for nonenhanced and contrast-enhanced paired acquisitions. Qualitative image quality of acquired 3T and 7T images and of synthesized 7T images was evaluated by three radiologists in terms of overall image quality, artifacts, sharpness, contrast, and visualization of vessel using 5-point Likert scales. Wilcoxon signed rank tests to compare synthetic 7T images with acquired 7T and 3T images and intraclass correlation coefficients to evaluate interobserver variability. P < 0.05 was considered significant. Of the 122 paired 3T and 7T MRI scans, 66 were acquired without contrast agent and 56 with contrast agent. The average time to generate synthetic images was ~11.4 msec per slice (2.95 sec per participant). The synthetic 7T images achieved significantly improved tissue contrast and sharpness in comparison to 3T images in both nonenhanced and contrast-enhanced subgroups. Meanwhile, there was no significant difference between acquired 7T and synthetic 7T images in terms of all the evaluation criteria for both nonenhanced and contrast-enhanced subgroups (P ≥ 0.180). The deep learning model has potential to generate synthetic 7T images with similar image quality to acquired 7T images. 2 TECHNICAL EFFICACY: Stage 1.

Large annotated datasets are required for training deep learning models, but in medical imaging data sharing is often complicated due to ethics, anonymization and data protection legislation. Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images, and can potentially facilitate data sharing. However, in order to share synthetic medical images it must first be demonstrated that they can be used for training different networks with acceptable performance. Here, we therefore comprehensively evaluate four GANs (progressive GAN, StyleGAN 1–3) and a diffusion model for the task of brain tumor segmentation (using two segmentation networks, U-Net and a Swin transformer). Our results show that segmentation networks trained on synthetic images reach Dice scores that are 80%–90% of Dice scores when training with real images, but that memorization of the training images can be a problem for diffusion models if the original dataset is too small. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required. The trained generative models and the generated synthetic images are shared on AIDA data hub.

Deep learning architectures can automatically learn complex features and patterns associated with glaucomatous optic neuropathy (GON). However, developing robust algorithms requires a large number of data sets. We sought to train an adversarial model for generating high-quality optic disc images from a large, diverse data set and then assessed the performance of models on generated synthetic images for detecting GON. A total of 17,060 (6874 glaucomatous and 10,186 healthy) fundus images were used to train deep convolutional generative adversarial networks (DCGANs) for synthesizing disc images for both classes. We then trained two models to detect GON, one solely on these synthetic images and another on a mixed data set (synthetic and real clinical images). Both the models were externally validated on a data set not used for training. The multiple classification metrics were evaluated with 95% confidence intervals. Models' decision-making processes were assessed using gradient-weighted class activation mapping (Grad-CAM) techniques. Following receiver operating characteristic curve analysis, an optimal cup-to-disc ratio threshold for detecting GON from the training data was found to be 0.619. DCGANs generated high-quality synthetic disc images for healthy and glaucomatous eyes. When trained on a mixed data set, the model's area under the receiver operating characteristic curve attained 99.85% on internal validation and 86.45% on external validation. Grad-CAM saliency maps were primarily centered on the optic nerve head, indicating a more precise and clinically relevant attention area of the fundus image. Although our model performed well on synthetic data, training on a mixed data set demonstrated better performance and generalization. Integrating synthetic and real clinical images can optimize the performance of a deep learning model in glaucoma detection. Optimizing deep learning models for glaucoma detection through integrating DCGAN-generated synthetic and real-world clinical data can be improved and generalized in clinical practice.

This study aims to explore the feasibility of employing generative adversarial networks (GAN) to generate synthetic contrast-enhanced ultrasound (CEUS) from grayscale ultrasound images of patients with thyroid nodules while dispensing with the need for ultrasound contrast agents injection. Patients who underwent preoperative thyroid CEUS examinations between January 2020 and July 2022 were collected retrospectively. The cycle-GAN framework integrated paired and unpaired learning modules was employed to develop the non-invasive image generation process. The syntheticCEUS imageswas generated in three phases: pre-arterial, plateau, and venous. The evaluation included quantitative similarity metrics, classification performance, and qualitative assessment by radiologists. CEUS videos of 360 thyroid nodules from 314 patients (45 years ± 12 [SD]; 272 women) in the internal dataset and 202 thyroid nodules from 183 patients (46 years ± 13 [SD]; 148 women) in the external dataset were included. In the external testing dataset, quantitative analysis revealed a significant degree of similarity between real and synthetic CEUS images (structure similarity index, 0.89 ± 0.04; peak signal-to-noise ratio, 28.17 ± 2.42). Radiologists deemed 126 of 132 [95%] synthetic CEUS images diagnostically useful. The accuracy of radiologists in distinguishing between real and synthetic images was 55.6% (95% CI: 0.49, 0.63), with an AUC of 61.0% (95% CI: 0.65, 0.68). No statistically significant difference (p > 0.05) was observed when radiologists assessed peak intensity and enhancement patterns using real CEUS and synthetic CEUS. Both quantitative analysis and radiologist evaluations exhibited that synthetic CEUS images generated by generative adversarial networks were similar to real CEUS images. QuestionIt is feasible to generate synthetic thyroid contrast-enhanced ultrasound images using generative adversarial networks without ultrasound contrast agents injection. FindingsCompared to real contrast-enhanced ultrasound images, synthetic contrast-enhanced ultrasound images exhibited high similarity and image quality. Clinical relevanceThis non-invasive and intelligent transformation may reduce the requirement for ultrasound contrast agents in certain cases, particularly in scenarios where ultrasound contrast agents administration is contraindicated, such as in patients with allergies, poor tolerance, or limited access to resources.

Deep learning has become the main way of the object detection task for autonomous vehicles. Meanwhile, this method typically requires vast amounts of training data to reach their full potential. However, collecting the data from real world and labeling manually is an expensive, time-consuming and error-prone process. Synthetic image has the potential to replace real image for training neural networks, because image creation and labeling annotations are free in this way. For the network trained by synthetic images, the reality gap between real and synthetic images is the main obstacle to use it in the real world. And most previous works are only devoted to generate synthetic images with a good performance on model training, but lack of analysis of the domain gap that affects the performance. This work designs a method of generating the real and synthetic images to analyze and reduce the reality gap between synthetic and real images for car detection. Firstly, this work put one single car with no-background in a random background image to generate real single car and synthetic single car images. In order to further reduce the domain gap in content level, this method keeps the car distribution in synthetic images is similar with the distribution of car in real world. For the purpose of reducing the domain gap in appearance level, the parameters of camera model are same as the camera parameters of image collecting cars and the image is rendered by using the PBRT(Physically Based Ray Tracing) when we generated the synthetic images. Secondly, by training the neural network of instance segmentation with different datasets, the across validation result proves that the reality gap between synthetic and real images is no more than the domain gap between real images. Thirdly, the training results of datasets with different samples diversity show that the diversity of the samples yields better generalization between different datasets for car detection which can effectively reduce the domain gap.

AI-generated content (AIGC) in the context of dermoscopic image analysis describes the application of artificial intelligence (AI) approaches to produce synthetic images for training and enriching machine learning (ML) methods. In dermatology, where dermoscopic images are essential in skin lesion detection, AIGC helps overcome the challenges of inadequate datasets and class imbalances. By employing advanced generative complex systems like Generative Adversarial Networks (GANs), AIGC will simulate different dermoscopic conditions by generating realistic-view synthetic images. These AI-generated images help to enhance the training dataset, contributing to the ML models with many widespread and varied sets of instances. This aids in increasing the ability of the model to precisely predict and generalize while handling new, unnoticed dermoscopic images. Deep learning (DL) methods, specifically convolutional neural networks (CNNs), can provide a massive breakthrough in an extensive range of computer vision (CV) tasks of complex systems, predominantly by exploiting large-scale annotated datasets. This paper develops a GAN-based Synthetic Medical Image Augmentation for the imbalanced Dermoscopic Image Analysis (GANSMIA-CDIA) method. The GANSMIA-CDIA technique’s primary target is exploiting the GAN model for synthetic image generation to handle class imbalance data problems. In addition, the GANSMIA-CDIA technique diagnoses the melanoma using the optimal DL model. The GANSMIA-CDIA technique applies a contrast enhancement process for the noise eradication process. The GANSMIA-CDIA technique follows a feature fusion method comprising different DL methods such as MobileNetV2, AlexNet, ResNet50, and Inceptionv3 to learn feature patterns in the preprocessed images. Meanwhile, the Social Network Search (SNS) technique is utilized for the hyperparameter tuning process. Also, the bidirectional long short-term memory (BiLSTM) technique is implemented to detect and classify melanoma. A series of simulations were performed on the standard dermoscopic image dataset to evaluate the performance of the GANSMIA-CDIA technique. The experimental values indicate the excellence of the GANSMIA-CDIA technique over existing techniques.

Synthetic Bone Marrow Smears Are a Privacy-Preserving Substitute for Developing Accurate Leukemia Classification Models in Hematological Microscopy

Deep learning (DL) techniques, which do not need complex pre-processing and feature analysis, are used in many areas of medicine and achieve promising results. On the other hand, in medical studies, a limited dataset decreases the abstraction ability of the DL model. In this context, we aimed to produce synthetic brain images including three tumor types (glioma, meningioma, and pituitary), unlike traditional data augmentation methods, and classify them with DL. This study proposes a tumor classification model consisting of a Dense Convolutional Network (DenseNet121)-based DL model to prevent forgetting problems in deep networks and delay information flow between layers. By comparing models trained on two different datasets, we demonstrated the effect of synthetic images generated by Cycle Generative Adversarial Network (CycleGAN) on the generalization of DL. One model is trained only on the original dataset, while the other is trained on the combined dataset of synthetic and original images. Synthetic data generated by CycleGAN improved the best accuracy values for glioma, meningioma, and pituitary tumor classes from 0.9633, 0.9569, and 0.9904 to 0.9968, 0.9920, and 0.9952, respectively. The developed model using synthetic data obtained a higher accuracy value than the related studies in the literature. Additionally, except for pixel-level and affine transform data augmentation, synthetic data has been generated in the figshare brain dataset for the first time.

Area under the curve may hide poor generalisation to external datasets

This work aims to investigate the clinical feasibility of deep learning-based synthetic CT images for cervix cancer, comparing them to MR for calculating attenuation (MRCAT). Patient cohort with 50 pairs of T2-weighted MR and CT images from cervical cancer patients was split into 40 for training and 10 for testing phases. We conducted deformable image registration and Nyul intensity normalization for MR images to maximize the similarity between MR and CT images as a preprocessing step. The processed images were plugged into a deep learning model, generative adversarial network. To prove clinical feasibility, we assessed the accuracy of synthetic CT images in image similarity using structural similarity (SSIM) and mean-absolute-error (MAE) and dosimetry similarity using gamma passing rate (GPR). Dose calculation was performed on the true and synthetic CT images with a commercial Monte Carlo algorithm. Synthetic CT images generated by deep learning outperformed MRCAT images in image similarity by 1.5% in SSIM, and 18.5 HU in MAE. In dosimetry, the DL-based synthetic CT images achieved 98.71% and 96.39% in the GPR at 1% and 1 mm criterion with 10% and 60% cut-off values of the prescription dose, which were 0.9% and 5.1% greater GPRs over MRCAT images.

BackgroundMalignant melanoma (MM) is the most aggressive skin cancer, requiring early diagnosis for better outcomes. While deep learning models have shown promise in dermatological image analysis, their performance is constrained by limited training data. Generative Adversarial Networks (GANs) offer a solution by generating synthetic images for data augmentation. However, assessing their clinical reliability remains difficult as automated metrics may not fully capture visual realism or clinical usability.The ObjectiveThis study presents a comprehensive framework for creating high-quality synthetic dermatoscopic pictures of MM lesions, as well as a holistic validation methodology that uses quantitative metrics and qualitative dermatologist assessment to provide a full clinical evaluation of the generated images. Three GAN architectures (DCGAN, StyleGAN2, and StyleGAN3-t) are explored. Lesions on the face, palms, and soles are excluded due to their unique dermoscopic patterns.Materials and MethodsA dataset of 1,774 dermatoscopic body MM images was used to train the models, assessed with Fréchet Inception Distance (FID), Kernel Inception Distance (KID), precision, and recall. Afterwards, a panel of 17 dermatologists with different levels of expertise assessed image quality using a 7-point Likert scale, with accuracy, sensitivity, specificity, and inter-rater agreement analysed.ResultsStyleGAN2 achieved the best image fidelity (FID: 18.89, KID: 0.0025), while StyleGAN3-t demonstrated stable but slower convergence. Both StyleGAN models outperformed DCGAN in diversity and fidelity. The validation study showed that StyleGAN2-generated images were often indistinguishable from real ones, reflected in low specificity and accuracy values among evaluators.ConclusionsThe study highlights the effectiveness of GANs in generating high-quality synthetic images, proposing a validation framework that integrates expert assessments with state-of-the-art quantitative metrics. This approach advances standardisation in GAN evaluation, ensuring synthetic images are clinically relevant for dermatological AI applications.

In part due to its ability to mimic any data distribution, Generative Adversarial Network (GAN) algorithms have been successfully applied to many applications, such as data augmentation, text‐to‐image translation, image‐to‐image translation, and image inpainting. Learning from data without crafting loss functions for each application provides broader applicability of the GAN algorithm. Medical image synthesis is also another field that the GAN algorithm has great potential to assist clinician training. This paper proposes a synthetic wound image generation model based on GAN architecture to increase the quality of clinical training. The proposed model is trained on chronic wound datasets with various sizes taken from real hospital environments. Hyperparameters such as epoch count and dataset size for training tasks are studied to find optimum training conditions as well. The performance of the developed model was evaluated through a mean squared error (MSE) metric to determine the similarity between generated and actual wounds. Visual inspection is performed to examine generated wound images. The results show that the proposed synthetic wound image generation (WG2AN) model has great potential to be used in medical training and performs well in producing synthetic wound images with a 1000‐image training dataset and 200 epochs of training.

A deep learning model for generating fundus autofluorescence images from color fundus photography