DivGAN: A diversity enforcing generative adversarial network for mode collapse reduction

HSGAN: Reducing mode collapse in GANs by the latent code distance of homogeneous samples

<p>High data collection costs and complicated data access regulations increase the demand for synthetic data. Generative Adversarial Networks (GANs) are a novel generative framework with great potential for high quality synthetic data generation. GANs formulate the true distribution of data implicitly, and the success of GANs are often measured based on the similarity of generated data to this true distribution. GANs were originally designed to work with continuous data. However, many important real-world datasets such as medical images involve discontinuous distributions. GAN training for discontinuous distributions is relatively more challenging, as the training procedure often suffers from instability and mode collapse issues. This dissertation focuses on designing novel GAN architectures to generate representative synthetic image data, and proposes new structures to alleviate GANs' mode collapse issue. As part of this thesis, novel applications of image data generation with GANs have been also investigated for important problems arising in the telecommunication industry and medical domain. Specifically, we first explore various GAN structures to generate engineered electromagnetic surfaces. We consider the continuous approximation of the data and explore the capabilities of feed-forward and convolutional GANs for synthetic data generation. Next, we introduce a novel GAN architecture to address the problem of mode collapse in GAN training. The proposed structure incorporates a third network that penalizes the generator for generating low diversity samples. Lastly, we study the challenging problem of object generation in 3D space using GANs, and we propose extensions to existing 3D GAN structures to generate connected 3D volumes. Additionally, we explore a more challenging version of this 3D volume generation problem by generating connected volumes packed with spheres. This research has applications in radiosurgery treatment planning, and the proposed 3D GAN structure can help generate rare, unseen 3D tumor volumes and information on how to treat these tumors. Accordingly, our analysis contributes to overcoming data scarcity issues (e.g., due to privacy considerations) for an important practical problem in the medical domain.</p>

In the field of deep learning, the generative model did not attract much attention until GANs (generative adversarial networks) appeared. In 2014, Google’s Ian Goodfellow proposed a generative model called GANs. GANs use different structures and objective functions from the existing generative model. For example, GANs use two neural networks: a generator that creates a realistic image, and a discriminator that distinguishes whether the input is real or synthetic. If there are no problems in the training process, GANs can generate images that are difficult even for experts to distinguish in terms of authenticity. Currently, GANs are the most researched subject in the field of computer vision, which deals with the technology of image style translation, synthesis, and generation, and various models have been unveiled. The issues raised are also improving one by one. In image synthesis, BEGAN (Boundary Equilibrium Generative Adversarial Network), which outperforms the previously announced GANs, learns the latent space of the image, while balancing the generator and discriminator. Nonetheless, BEGAN also has a mode collapse wherein the generator generates only a few images or a single one. Although BEGAN-CS (Boundary Equilibrium Generative Adversarial Network with Constrained Space), which was improved in terms of loss function, was introduced, it did not solve the mode collapse. The discriminator structure of BEGAN-CS is AE (AutoEncoder), which cannot create a particularly useful or structured latent space. Compression performance is not good either. In this paper, this characteristic of AE is considered to be related to the occurrence of mode collapse. Thus, we used VAE (Variational AutoEncoder), which added statistical techniques to AE. As a result of the experiment, the proposed model did not cause mode collapse but converged to a better state than BEGAN-CS.

A Generative Adversarial Network (GAN) is an artifi- cial intelligence model developed specifically to pro- duce synthetic data that resembles real data by training a generative model and a discriminative model simulta- neously using adversarial training. A GAN can be ex- tensively used for generating replicated data, however, it suffers from several issues, one of which is mode col- lapse. Mode collapse takes place when the generator is unable to capture the complete range of diversity in the target data distribution, resulting in the production of limited and repeating variations of samples. Multiple metrics exist to quantify mode collapse in GANs, al- though no individual metric is capable of consistently providing accurate results. This research focuses on the critical need for accurate mode collapse detection tech- niques in GANs, to strengthen the credit card fraud detection systems. In this work, we utilize a GAN to generate numerical data instead of image data. Our ap- proach utilizes a wide range of measures, such as Gener- ator and Discriminator Loss, Wasserstein Distance, pre- cision, recall, and visualization tools, to provide a com- prehensive framework for detecting mode collapse. In addition, we introduce an alert mechanism that identi- fies possible mode collapse at an early stage, allowing for earlier intervention and modifications to the training process. We have further proposed suggestions regard- ing monitoring and analyzing generator and discrimina- tor loss values to identify potential instances of mode collapse to help the developer optimize GAN training and improve the quality of synthetic data.

Generative adversarial network (GAN) is an implicit generative model known for its ability to generate sharp images. However, it is poor at generating diverse data, which refers to the mode collapse problem. It turns out that GAN is prone to emphasizing the quality of samples but ignoring their diversity. When mode collapse happens, the support of the generated data distribution is not aligned with that of the real data distribution. We thus propose Support Regularized-GAN (SR-GAN) to address such a mode collapse issue by matching their support. Our experiments on synthetic and real-world datasets show that our regularization can mitigate the mode collapse and also improve the data quality.

Single image super-resolution (SISR) is of great importance as a low-level computer vision task. The fast development of Generative Adversarial Network (GAN) based deep learning architectures realises an efficient and effective SISR to boost the spatial resolution of natural images captured by digital cameras. However, the SISR for medical images is still a very challenging problem. This is due to (1) compared to natural images, in general, medical images have lower signal to noise ratios, (2) GAN based models pre-trained on natural images may synthesise unrealistic patterns in medical images which could affect the clinical interpretation and diagnosis, and (3) the vanilla GAN architecture may suffer from unstable training and collapse mode that can also affect the SISR results. In this paper, we propose a novel lesion focused SR (LFSR) method, which incorporates GAN to achieve perceptually realistic SISR results for brain tumour MRI images. More importantly, we test and make comparison using recently developed GAN variations, e.g., Wasserstein GAN (WGAN) and WGAN with Gradient Penalty (WGAN-GP), and propose a novel multi-scale GAN (MS-GAN), to achieve a more stabilised and efficient training and improved perceptual quality of the super-resolved results. Based on both quantitative evaluations and our designed mean opinion score, the proposed LFSR coupled with MS-GAN has performed better in terms of both perceptual quality and efficiency.

Synthesizing high-resolution realistic images from text description using one iteration Generative Adversarial Network (GAN) is difficult without using any additional techniques because mostly the blurry artifacts and mode collapse problems are occurring. To reduce these problems, this paper proposes an Iterative Generative Adversarial Network (iGAN) which takes three iterations to synthesize high-resolution realistic image from their text description. In the \(1^{st}\) iteration, GAN synthesizes a low-resolution \(64 \times 64\) pixels basic shape and basic color image from the text description with less mode collapse and blurry artifacts problems. In the \(2^{nd}\) iteration, GAN takes the result of the \(1^{st}\) iteration and text description again and synthesizes a better resolution \(128 \times 128\) pixels better shape and well color image with very less mode collapse and blurry artifacts problems. In the last iteration, GAN takes the result of the \(2^{nd}\) iteration and text description as well and synthesizes a high-resolution \(256 \times 256\) well shape and clear image with almost no mode collapse and blurry artifacts problems. Our proposed iGAN shows a significant performance on CUB birds and Oxford-102 flowers datasets. Moreover, iGAN improves the inception score and human rank as compare to the other state-of-the-art methods.

We propose MAD-GAN, an intuitive generalization to the Generative Adversarial Networks (GANs) and its conditional variants to address the well known problem of mode collapse. First, MAD-GAN is a multi-agent GAN architecture incorporating multiple generators and one discriminator. Second, to enforce that different generators capture diverse high probability modes, the discriminator of MAD-GAN is designed such that along with finding the real and fake samples, it is also required to identify the generator that generated the given fake sample. Intuitively, to succeed in this task, the discriminator must learn to push different generators towards different identifiable modes. We perform extensive experiments on synthetic and real datasets and compare MAD-GAN with different variants of GAN. We show high quality diverse sample generations for challenging tasks such as image-to-image translation and face generation. In addition, we also show that MAD-GAN is able to disentangle different modalities when trained using highly challenging diverse-class dataset (e.g. dataset with images of forests, icebergs, and bedrooms). In the end, we show its efficacy on the unsupervised feature representation task.

Generative Adversarial Networks(GANs) are powerful generative models on numerous tasks and datasets but are also known for their training instability and mode collapse. The latter is because the optimal transportation map is discontinuous, but DNNs can only approximate continuous ones. One way to solve the problem is to introduce multiple discriminators or generators. However, their impacts are limited because the cost function of each component is the same. That is, they are homogeneous. In contrast, multiple discriminators with different cost functions can yield various gradients for the generator, which indicates we can use them to search for more transportation maps in the latent space. Inspired by this, we have proposed a framework to combat the mode collapse problem, containing multiple discriminators with different cost functions, named CES-GAN. Unfortunately, it may also lead to the generator being hard to train because the performance between discriminators is unbalanced, according to the Cannikin Law. Thus, a gradient selecting mechanism is also proposed to pick up proper gradients. We provide mathematical statements to prove our assumptions and conduct extensive experiments to verify the performance. The results show that CES-GAN is lightweight and more effective for fighting against the mode collapse problem than similar works.

Image restoration is a critical task in computer vision that involves recovering an original image from corrupted or damaged versions of it. Traditional image restoration relies on interpolation and completion techniques, such as Navier–Stokes equations and the fast multipole boundary element method. However, these methods often fail to capture the advanced semantic information of an image, leading to inaccurate restoration results. In recent years, generative adversarial networks (GANs) have emerged as a practical approach in computer vision for image restoration tasks, as they can address image restoration issues related to damaged or missing image information. GANs are a deep‐learning network structure with a generator and discriminator. In image restoration, the generator is employed to restore damaged images, and the discriminator is used to assess the authenticity of the repair results. Competition between the generator and discriminator improves the quality of the repair results. This study proposes a novel generative image restoration method that utilizes contextual and perceptual semantic information mechanisms to strengthen GANs. Our approach demonstrates the effectiveness of GANs in restoring images through learning how to fuse the missing or damaged parts of an image with the undamaged parts surrounding them, resulting in visually appealing restoration results.

Unified gradient- and intensity-discriminator generative adversarial network for image fusion

The generation of high-quality, long-sequenced time-series data is essential due to its wide range of applications. In the past, standalone Recurrent and Convolutional Neural Network-based Generative Adversarial Networks (GAN) were used to synthesize time-series data. However, they are inadequate for generating long sequences of time-series data due to limitations in the architecture, such as difficulties in capturing long-range dependencies, limited temporal coherence, and scalability challenges. Furthermore, GANs are well known for their training instability and mode collapse problem. To address this, we propose TransFusion , a diffusion, and transformers-based generative model to generate high-quality long-sequence time-series data. We extended the sequence length to 384, surpassing the previous limit, and successfully generated high-quality synthetic data. Also, we introduce two evaluation metrics to evaluate the quality of the synthetic data as well as its predictive characteristics. TransFusion is evaluated using a diverse set of visual and empirical metrics, consistently outperforming the previous state-of-the-art by a significant margin.

Stock market forecasting using DRAGAN and feature matching

To address the problem of poor applicability of the existing generative adversarial network (GAN) evaluation system, this paper introduces the remote sensing image information quantity calculation system into the generative adversarial network evaluation indicator system, proposes an end-to-end GAN evaluation method based on the single-pixel image information quantity indicator, and uses EDS, ENDS, and InfoDS to evaluate the stability of the generator training process, mode collapse, diversity generation, and training failure problem is effectively monitored, which independent of loss function, learning rate.

Generative Adversarial Networks (GANs) are a major advancement in generative modeling, surpassing traditional machine learning models in tasks such as image generation, super-resolution, and image-to-text translation. A GAN consists of two neural networks: a Generator (G), which creates data from noise or a latent vector, and a Discriminator (D), which determines whether the data is real or generated. These networks train competitively, improving each other iteratively to produce increasingly realistic outputs. However, GANs face challenges like mode collapse, unstable training, and convergence issues, leading to the adoption of strategies such as instance normalization and enhanced loss functions. Future research can focus on improving stability, developing novel loss functions, and applying GANs in unsupervised learning. Performance metrics like Inception Score, Fréchet Inception Distance (FID), and Structural Similarity Index (SSIM) are essential for evaluating and comparing GAN architectures. Additionally, ethical concerns, including the misuse of GANs for deepfakes and synthetic data, underscore the importance of transparency, accountability, and ethical standards in research and deployment. This review provides an accessible introduction to GANs for novice researchers, along with a detailed analysis of their limitations, applications, and future prospects, offering valuable insights and direction for advancing this field.