Generative Adversarial Networks for face generation: A survey

In this paper, a generative adversarial network based on SMPL model is used to reconstruct three-dimensional (3D) human mesh from a single image. The generative adversarial network consists of two parts (generative network and adversarial network). The generative network is made up of a deep residual network and a multilayer perception network. The goal is to generate suitable SMPL model parameters and camera parameters which fit the 2D and 3D position annotation of human skeleton joints. The adversarial network judges whether the shape and pose parameters generated by the generative network match the real human body through a discriminator. The purpose is to guide the generative network output parameters that conform to the laws of human kinematics. A geometric prior of the 3D joint angle limitation of the human body is introduced into the network to further constrain the generated human pose. Experiments verify the effectiveness of the algorithm.

Generative Adversarial Network is the topic of interest in today’s research in the field of image processing and computer vision. A basic GAN model was introduced by Ian Goodfellow et al. in 2014. After that advancement in the field of research in GAN models has been application specific. In computer vision and image to image translation GANs are playing very effective role either in the case of face detection and recognition or in image resolution enhancement and image augmentation. This paper represents a concise overview of various GAN models along with their features and applications. Pix2Pix and conditional GAN models work upon paired datasets while other models like cycle GAN, discover GAN, dual GAN, info GAN, deep convolutional GAN etc. work upon unpaired datasets. Various image datasets which are commonly used for training of generator and discriminator networks are also discussed in this paper. Since partial mode collapse is a common problem to occur during training process for all models, therefore various normalization techniques are also preferred during the training of generator and discriminator networks. As the advancements in GAN models are increasing at a very fast rate, soon these models will also be preferred in commercial applications.

Image dehazing has always been a challenging topic in image processing. The development of deep learning methods, especially the generative adversarial networks (GAN), provides a new way for image dehazing. In recent years, many deep learning methods based on GAN have been applied to image dehazing. However, GAN has two problems in image dehazing. Firstly, For haze image, haze not only reduces the quality of the image but also blurs the details of the image. For GAN network, it is difficult for the generator to restore the details of the whole image while removing the haze. Secondly, GAN model is defined as a minimax problem, which weakens the loss function. It is difficult to distinguish whether GAN is making progress in the training process. Therefore, we propose a guided generative adversarial dehazing network (GGADN). Different from other generation adversarial networks, GGADN adds a guided module on the generator. The guided module verifies the network of each layer of the generator. At the same time, the details of the map generated by each layer are strengthened. Network training is based on the pre-trained VGG feature model and L1-regularized gradient prior which is developed by new loss function parameters. From the dehazing results of synthetic images and real images, the proposed method is better than the state-of-the-art dehazing methods.

Generative Adversarial Networks (GANs) can accurately model complex multi-dimensional data and generate realistic samples. However, due to their implicit estimation of data distributions, their evaluation is a challenging task. The majority of research efforts associated with tackling this issue were validated by qualitative visual evaluation. Such approaches do not generalize well beyond the image domain. Since many of those evaluation metrics are proposed and bound to the vision domain, they are difficult to apply to other domains. Quantitative measures are necessary to better guide the training and comparison of different GANs models. In this work, we leverage Siamese neural networks to propose a domain-agnostic evaluation metric: (1) with a qualitative evaluation that is consistent with human evaluation, (2) that is robust relative to common GAN issues such as mode dropping and invention, and (3) does not require any pretrained classifier. The empirical results in this paper demonstrate the superiority of this method compared to the popular Inception Score and are competitive with the FID score.

Generative artificial intelligence, especially with regard to the generative adversarial network (GAN), is an important research area in radiology as evidenced by a number of literature reviews on the role of GAN in radiology published in the last few years. However, no review article about GAN in pediatric radiology has been published yet. The purpose of this paper is to systematically review applications of GAN in pediatric radiology, their performances, and methods for their performance evaluation. Electronic databases were used for a literature search on 6 April 2023. Thirty-seven papers met the selection criteria and were included. This review reveals that the GAN can be applied to magnetic resonance imaging, X-ray, computed tomography, ultrasound and positron emission tomography for image translation, segmentation, reconstruction, quality assessment, synthesis and data augmentation, and disease diagnosis. About 80% of the included studies compared their GAN model performances with those of other approaches and indicated that their GAN models outperformed the others by 0.1–158.6%. However, these study findings should be used with caution because of a number of methodological weaknesses. For future GAN studies, more robust methods will be essential for addressing these issues. Otherwise, this would affect the clinical adoption of the GAN-based applications in pediatric radiology and the potential advantages of GAN could not be realized widely.

This study presents a new approach based on artificial neural networks for generating one biometric feature (faces) from another (only fingerprints). An automatic and intelligent system was designed and developed to analyze the relationships among fingerprints and faces and also to model and to improve the existence of the relationships. The new proposed system is the first study that generates all parts of the face including eyebrows, eyes, nose, mouth, ears and face border from only fingerprints. It is also unique and different from similar studies recently presented in the literature with some superior features. The parameter settings of the system were achieved with the help of Taguchi experimental design technique. The performance and accuracy of the system have been evaluated with 10-fold cross validation technique using qualitative evaluation metrics in addition to the expanded quantitative evaluation metrics. Consequently, the results were presented on the basis of the combination of these objective and subjective metrics for illustrating the qualitative properties of the proposed methods as well as a quantitative evaluation of their performances. Experimental results have shown that one biometric feature can be determined from another. These results have once more indicated that there is a strong relationship between fingerprints and faces.

The gas chromatography analysis method for chemical substances enables accurate analysis to precisely distinguish the components of a mixture. This paper presents a technique for augmenting time-series data of chemicals measured by gas chromatography instruments with artificial intelligence techniques such as generative adversarial networks (GAN). We propose a novel GAN algorithm called GCGAN for gas chromatography data, a unified model of autoencoder (AE) and GAN for effective time-series data learning with an attention mechanism. The proposed GCGAN utilizes AE to learn a limited number of data more effectively. We also build a layer of high-performance generative adversarial neural networks based on the analysis of the features of data measured by gas chromatography instruments. Then, based on the proposed learning, we synthesize the features embedded in the gas chromatography data into a feature distribution that extracts the temporal variability. GCGAN synthesizes the features embedded in the gas chromatography data into a feature distribution that extracts the temporal variability of the data over time. We have fully implemented the proposed GCGAN and experimentally verified that the data augmented by the GCGAN have the characteristic properties of the original gas chromatography data. The augmented data demonstrate high quality with the Pearson correlation coefficient, Spearman correlation coefficient, and cosine similarity all exceeding 0.9, significantly enhancing the performance of AI classification models by 40%. This research can be effectively applied to various small dataset domains other than gas chromatography data, where data samples are limited and difficult to obtain.

<p>High data collection costs and complicated data access regulations increase the demand for synthetic data. Generative Adversarial Networks (GANs) are a novel generative framework with great potential for high quality synthetic data generation. GANs formulate the true distribution of data implicitly, and the success of GANs are often measured based on the similarity of generated data to this true distribution. GANs were originally designed to work with continuous data. However, many important real-world datasets such as medical images involve discontinuous distributions. GAN training for discontinuous distributions is relatively more challenging, as the training procedure often suffers from instability and mode collapse issues. This dissertation focuses on designing novel GAN architectures to generate representative synthetic image data, and proposes new structures to alleviate GANs' mode collapse issue. As part of this thesis, novel applications of image data generation with GANs have been also investigated for important problems arising in the telecommunication industry and medical domain. Specifically, we first explore various GAN structures to generate engineered electromagnetic surfaces. We consider the continuous approximation of the data and explore the capabilities of feed-forward and convolutional GANs for synthetic data generation. Next, we introduce a novel GAN architecture to address the problem of mode collapse in GAN training. The proposed structure incorporates a third network that penalizes the generator for generating low diversity samples. Lastly, we study the challenging problem of object generation in 3D space using GANs, and we propose extensions to existing 3D GAN structures to generate connected 3D volumes. Additionally, we explore a more challenging version of this 3D volume generation problem by generating connected volumes packed with spheres. This research has applications in radiosurgery treatment planning, and the proposed 3D GAN structure can help generate rare, unseen 3D tumor volumes and information on how to treat these tumors. Accordingly, our analysis contributes to overcoming data scarcity issues (e.g., due to privacy considerations) for an important practical problem in the medical domain.</p>

This paper introduces a novel automotive radar interference mitigation approach using a generative adversarial network (GAN). Instead of tackling the mutual interference in the time domain, a generative adversarial network is trained and used to recover the complex signal in the frequency domain, namely on the complex range profile obtained after the fast Fourier transform of fast-time samples (RFFT spectrum). It is shown that by employing the gated convolution and an attention mechanism, the generator network has the ability to learn the amplitude and phase information for missing data from the remaining signal. Experimental results show that the proposed method can provide a remarkable improvement in signal-to-interference-plus-noise ratio (SINR) and preserves its robustness in severely disturbed scenarios that are much more complex than the training dataset.

Recovering images from highly undersampled measurements has a wide range of applications across imaging sciences. State-of-the-art analytics however are not aware of the image perceptual quality, and demand iterative algorithms that incur significant computational overhead. To sidestep these hurdles, this paper brings forth a novel compressive imaging framework using deep neural networks that approximates a low-dimensional manifold of images using generative adversarial networks. To ensure the images are consistent with the measurements a recurrent GAN (RGAN) architecture is deployed that consists of multiple alternative blocks of generator networks and affine projection, which is then followed by a discriminator network to score the perceptual quality of the generated images. A deep residual network with skip connections is used for the generator, while the discriminator is a multilayer perceptron. Experiments performed with real-world contrast enhanced MRI data corroborate the diagnostic quality of the retrieved images relative to state-of-the-art CS schemes. In addition, it achieves about two-orders of magnitude faster reconstruction.

Due to the light absorption and scattering induced by the water medium, underwater images usually suffer from some degradation problems, such as low contrast, color distortion, and blurring details, which aggravate the difficulty of downstream underwater understanding tasks. Therefore, how to obtain clear and visually pleasant images has become a common concern of people, and the task of underwater image enhancement (UIE) has also emerged as the times require. Among existing UIE methods, Generative Adversarial Networks (GANs) based methods perform well in visual aesthetics, while the physical model-based methods have better scene adaptability. Inheriting the advantages of the above two types of models, we propose a physical model-guided GAN model for UIE in this paper, referred to as PUGAN. The entire network is under the GAN architecture. On the one hand, we design a Parameters Estimation subnetwork (Par-subnet) to learn the parameters for physical model inversion, and use the generated color enhancement image as auxiliary information for the Two-Stream Interaction Enhancement sub-network (TSIE-subnet). Meanwhile, we design a Degradation Quantization (DQ) module in TSIE-subnet to quantize scene degradation, thereby achieving reinforcing enhancement of key regions. On the other hand, we design the Dual-Discriminators for the style-content adversarial constraint, promoting the authenticity and visual aesthetics of the results. Extensive experiments on three benchmark datasets demonstrate that our PUGAN outperforms state-of-the-art methods in both qualitative and quantitative metrics. The code and results can be found from the link of https://rmcong.github.io/proj PUGAN.html.

Generative adversarial networks (GANs) with gaming abilities have been widely applied in image generation. However, gamistic generators and discriminators may reduce the robustness of the obtained GANs in image generation under varying scenes. Enhancing the relation of hierarchical information in a generation network and enlarging differences of different network architectures can facilitate more structural information to improve the generation effect for image generation. In this paper, we propose an enhanced GAN via improving a generator for image generation (EIGGAN). EIGGAN applies a spatial attention to a generator to extract salient information to enhance the truthfulness of the generated images. Taking into relation the context account, parallel residual operations are fused into a generation network to extract more structural information from the different layers. Finally, a mixed loss function in a GAN is exploited to make a tradeoff between speed and accuracy to generate more realistic images. Experimental results show that the proposed method is superior to popular methods, i.e., Wasserstein GAN with gradient penalty (WGAN-GP) in terms of many indexes, i.e., Frechet Inception Distance, Learned Perceptual Image Patch Similarity, Multi-Scale Structural Similarity Index Measure, Kernel Inception Distance, Number of Statistically-Different Bins, Inception Score and some visual images for image generation.

Brief hospital course (BHC) summaries are clinical documents that summarize a patient's hospital stay. While large language models (LLMs) depict remarkable capabilities in automating real-world tasks, their capabilities for healthcare applications such as synthesizing BHCs from clinical notes have not been shown. We introduce a novel preprocessed dataset, the MIMIC-IV-BHC, encapsulating clinical note and BHC pairs to adapt LLMs for BHC synthesis. Furthermore, we introduce a benchmark of the summarization performance of 2 general-purpose LLMs and 3 healthcare-adapted LLMs. Using clinical notes as input, we apply prompting-based (using in-context learning) and fine-tuning-based adaptation strategies to 3 open-source LLMs (Clinical-T5-Large, Llama2-13B, and FLAN-UL2) and 2 proprietary LLMs (Generative Pre-trained Transformer [GPT]-3.5 and GPT-4). We evaluate these LLMs across multiple context-length inputs using natural language similarity metrics. We further conduct a clinical study with 5 clinicians, comparing clinician-written and LLM-generated BHCs across 30 samples, focusing on their potential to enhance clinical decision-making through improved summary quality. We compare reader preferences for the original and LLM-generated summary using Wilcoxon signed-rank tests. We further request optional qualitative feedback from clinicians to gain deeper insights into their preferences, and we present the frequency of common themes arising from these comments. The Llama2-13B fine-tuned LLM outperforms other domain-adapted models given quantitative evaluation metrics of Bilingual Evaluation Understudy (BLEU) and Bidirectional Encoder Representations from Transformers (BERT)-Score. GPT-4 with in-context learning shows more robustness to increasing context lengths of clinical note inputs than fine-tuned Llama2-13B. Despite comparable quantitative metrics, the reader study depicts a significant preference for summaries generated by GPT-4 with in-context learning compared to both Llama2-13B fine-tuned summaries and the original summaries (P<.001), highlighting the need for qualitative clinical evaluation. We release a foundational clinically relevant dataset, the MIMIC-IV-BHC, and present an open-source benchmark of LLM performance in BHC synthesis from clinical notes. We observe high-quality summarization performance for both in-context proprietary and fine-tuned open-source LLMs using both quantitative metrics and a qualitative clinical reader study. Our research effectively integrates elements from the data assimilation pipeline: our methods use (1) clinical data sources to integrate, (2) data translation, and (3) knowledge creation, while our evaluation strategy paves the way for (4) deployment.

Single image super-resolution (SISR) is of great importance as a low-level computer vision task. The fast development of Generative Adversarial Network (GAN) based deep learning architectures realises an efficient and effective SISR to boost the spatial resolution of natural images captured by digital cameras. However, the SISR for medical images is still a very challenging problem. This is due to (1) compared to natural images, in general, medical images have lower signal to noise ratios, (2) GAN based models pre-trained on natural images may synthesise unrealistic patterns in medical images which could affect the clinical interpretation and diagnosis, and (3) the vanilla GAN architecture may suffer from unstable training and collapse mode that can also affect the SISR results. In this paper, we propose a novel lesion focused SR (LFSR) method, which incorporates GAN to achieve perceptually realistic SISR results for brain tumour MRI images. More importantly, we test and make comparison using recently developed GAN variations, e.g., Wasserstein GAN (WGAN) and WGAN with Gradient Penalty (WGAN-GP), and propose a novel multi-scale GAN (MS-GAN), to achieve a more stabilised and efficient training and improved perceptual quality of the super-resolved results. Based on both quantitative evaluations and our designed mean opinion score, the proposed LFSR coupled with MS-GAN has performed better in terms of both perceptual quality and efficiency.

BackgroundMalignant melanoma (MM) is the most aggressive skin cancer, requiring early diagnosis for better outcomes. While deep learning models have shown promise in dermatological image analysis, their performance is constrained by limited training data. Generative Adversarial Networks (GANs) offer a solution by generating synthetic images for data augmentation. However, assessing their clinical reliability remains difficult as automated metrics may not fully capture visual realism or clinical usability.The ObjectiveThis study presents a comprehensive framework for creating high-quality synthetic dermatoscopic pictures of MM lesions, as well as a holistic validation methodology that uses quantitative metrics and qualitative dermatologist assessment to provide a full clinical evaluation of the generated images. Three GAN architectures (DCGAN, StyleGAN2, and StyleGAN3-t) are explored. Lesions on the face, palms, and soles are excluded due to their unique dermoscopic patterns.Materials and MethodsA dataset of 1,774 dermatoscopic body MM images was used to train the models, assessed with Fréchet Inception Distance (FID), Kernel Inception Distance (KID), precision, and recall. Afterwards, a panel of 17 dermatologists with different levels of expertise assessed image quality using a 7-point Likert scale, with accuracy, sensitivity, specificity, and inter-rater agreement analysed.ResultsStyleGAN2 achieved the best image fidelity (FID: 18.89, KID: 0.0025), while StyleGAN3-t demonstrated stable but slower convergence. Both StyleGAN models outperformed DCGAN in diversity and fidelity. The validation study showed that StyleGAN2-generated images were often indistinguishable from real ones, reflected in low specificity and accuracy values among evaluators.ConclusionsThe study highlights the effectiveness of GANs in generating high-quality synthetic images, proposing a validation framework that integrates expert assessments with state-of-the-art quantitative metrics. This approach advances standardisation in GAN evaluation, ensuring synthetic images are clinically relevant for dermatological AI applications.