Posit Arithmetic for the Training and Deployment of Generative Adversarial Networks

Distributed coevolutionary Generative Adversarial Network (GAN) training has empirically shown success in overcoming GAN training pathologies. This is mainly due to diversity maintenance in the populations of generators and discriminators during the training process. The method studied here coevolves sub-populations on each cell of a spatial grid organized into overlapping Moore neighborhoods. We investigate the impact on performance of two algorithm components that influence the diversity during coevolution: the performance-based selection/replacement inside each sub-population and the communication through migration of solutions (networks) among overlapping neighborhoods. In experiments on MNIST dataset, we find that the combination of these two components provides the best generative models. In addition, migrating solutions without applying selection in the sub-populations achieves competitive results, while selection without communication between cells reduces performance.

<p>High data collection costs and complicated data access regulations increase the demand for synthetic data. Generative Adversarial Networks (GANs) are a novel generative framework with great potential for high quality synthetic data generation. GANs formulate the true distribution of data implicitly, and the success of GANs are often measured based on the similarity of generated data to this true distribution. GANs were originally designed to work with continuous data. However, many important real-world datasets such as medical images involve discontinuous distributions. GAN training for discontinuous distributions is relatively more challenging, as the training procedure often suffers from instability and mode collapse issues. This dissertation focuses on designing novel GAN architectures to generate representative synthetic image data, and proposes new structures to alleviate GANs' mode collapse issue. As part of this thesis, novel applications of image data generation with GANs have been also investigated for important problems arising in the telecommunication industry and medical domain. Specifically, we first explore various GAN structures to generate engineered electromagnetic surfaces. We consider the continuous approximation of the data and explore the capabilities of feed-forward and convolutional GANs for synthetic data generation. Next, we introduce a novel GAN architecture to address the problem of mode collapse in GAN training. The proposed structure incorporates a third network that penalizes the generator for generating low diversity samples. Lastly, we study the challenging problem of object generation in 3D space using GANs, and we propose extensions to existing 3D GAN structures to generate connected 3D volumes. Additionally, we explore a more challenging version of this 3D volume generation problem by generating connected volumes packed with spheres. This research has applications in radiosurgery treatment planning, and the proposed 3D GAN structure can help generate rare, unseen 3D tumor volumes and information on how to treat these tumors. Accordingly, our analysis contributes to overcoming data scarcity issues (e.g., due to privacy considerations) for an important practical problem in the medical domain.</p>

PSGAN: Revisit the binary discriminator and an alternative for face frontalization

As a new unsupervised learning algorithm framework, generative adversarial networks(GAN) have been favored by more and more researchers, and it has become a research hotspot now. GAN is inspired by the two-person zero-sum game theory in game theory. Its unique adversarial training idea can generate high-quality samples and has more powerful feature learning and feature expression capabilities than traditional machine learning algorithms. At present, GAN has achieved remarkable success in the field of computer vision, especially in the field of sample generation. Every year, a large number of GAN-related research papers are produced, reflecting the fiery degree of research on GAN model. Aiming at the hot model of GAN, first introduce the research status of GAN; then introduce the theory and framework of GAN, which analyzes in detail why the gradient disappears and the mode collapses during the training of GAN; then discussed some typical GAN improvement models, and summarized their theoretical improvements, advantages, limitations, application scenarios and implementation costs; Finally, the application results of GAN in data generation, image super-resolution, and image style conversion are shown, and the current challenges and future research directions of GAN are discussed.

Generative Adversarial Networks (GANs) currently achieve the state-of-the-art sound synthesis quality for pitched musical instruments using a 2-channel spectrogram representation consisting of log magnitude and instantaneous frequency (the "IFSpectrogram"). Many other synthesis systems use representations derived from the magnitude spectra, and then depend on a backend component to invert the output magnitude spectrograms that generally result in audible artefacts associated with the inversion process. However, for signals that have closely-spaced frequency components such as non-pitched and other noisy sounds, training the GAN on the 2-channel IFSpectrogram representation offers no advantage over the magnitude spectra based representations. In this paper, we propose that training GANs on single-channel magnitude spectra, and using the Phase Gradient Heap Integration (PGHI) inversion algorithm is a better comprehensive approach for audio synthesis modeling of diverse signals that include pitched, non-pitched, and dynamically complex sounds. We show that this method produces higher-quality output for wideband and noisy sounds, such as pops and chirps, compared to using the IFSpectrogram. Furthermore, the sound quality for pitched sounds is comparable to using the IFSpectrogram, even while using a simpler representation with half the memory requirements.

We present a mathematically well-founded approach for the synthetic modeling of turbulent flows using generative adversarial networks (GAN). Based on the analysis of chaotic, deterministic systems in terms of ergodicity, we outline a mathematical proof that GAN can actually learn to sample state snapshots from the invariant measure of the chaotic system. Based on this analysis, we study a hierarchy of chaotic systems starting with the Lorenz attractor and then carry on to the modeling of turbulent flows with GAN. As training data, we use fields of velocity fluctuations obtained from large-eddy simulations (LES). Two architectures are investigated in detail: we use a deep, convolutional GAN (DCGAN) to synthesize the turbulent flow around a cylinder. We furthermore simulate the flow around a low-pressure turbine stator using the pix2pixHD architecture for a conditional DCGAN being conditioned on the position of a rotating wake in front of the stator. The settings of adversarial training and the effects of using specific GAN architectures are explained. We thereby show that GAN are efficient in simulating turbulence in technically challenging flow problems on the basis of a moderate amount of training data. GAN training and inference times significantly fall short when compared with classical numerical methods, in particular, LES, while still providing turbulent flows in high resolution. We furthermore analyze the statistical properties of the synthesized and LES flow fields, which agree excellently. We also show the ability of the conditional GAN to generalize over changes of geometry by generating turbulent flow fields for positions of the wake that are not included in the training data.

Computed tomography (CT) technology has been rapidly developed and widely used in the medical field, at the same time, it has also attracted people’s attention to its radiation dose. Although reducing the dose of X-rays can greatly reduce a patient’s chance of developing cancer, the lower the dose, the worse the quality of the generated image, generating speckle noise and streaking artifacts that can seriously affect the diagnostic results. Therefore, low-dose CT images need to be optimized and enhanced to facilitate expert diagnosis. Based on WGAN-VGG, this paper proposes an improved generative adversarial network for CT image denoising. The algorithm improves the loss functions of the generator and discriminator, and introduces noise loss, which constitutes a composite perceptual loss function with perceptual loss. This method not only prevents over fitting of results and low PSNR and SSIM, but also can better measure the internal similarity between images, so that the visual texture details of the image can achieve a more detailed restoration effect, which is helpful to reduce noise and see clearer details. Experiments have shown that the algorithm accelerates the convergence of loss function, effectively improves the problems of difficult training of generative adversarial network and slow convergence of loss function, and can better remove noise and restore more image details at the same time.

Network architecture search achieves state-of-the-art results in various tasks such as classification and semantic segmentation. Recently, a reinforcement learning-based approach has been proposed for generative adversarial networks (GANs) search. In this work, we propose an alternative strategy for GAN search by using a proxy task instead of common GAN training. Our method is called differentiable efficient generator search, which focuses on efficiently finding the generator in the GAN. Our search algorithm is inspired by the differential architecture search strategy and the global latent optimization procedure. This leads to both an efficient and stable GAN search. After the generator architecture is found, it can be plugged into any existing framework for GAN training. For consistency-term GAN, which we use in this work, the new model outperforms the original inception score results by 0.25 for CIFAR-10.

We study two important concepts in adversarial deep learning---adversarial training and generative adversarial network (GAN). Adversarial training is the technique used to improve the robustness of discriminator by combining adversarial attacker and discriminator in the training phase. GAN is commonly used for image generation by jointly optimizing discriminator and generator. We show these two concepts are indeed closely related and can be used to strengthen each other---adding a generator to the adversarial training procedure can improve the robustness of discriminators, and adding an adversarial attack to GAN training can improve the convergence speed and lead to better generators. Combining these two insights, we develop a framework called Rob-GAN to jointly optimize generator and discriminator in the presence of adversarial attacks---the generator generates fake images to fool discriminator; the adversarial attacker perturbs real images to fool the discriminator, and the discriminator wants to minimize loss under fake and adversarial images. Through this end-to-end training procedure, we are able to simultaneously improve the convergence speed of GAN training, the quality of synthetic images, and the robustness of discriminator under strong adversarial attacks. Experimental results demonstrate that the obtained classifier is more robust than the state-of-the-art adversarial training approach, and the generator outperforms SN-GAN on ImageNet-143.

Training Generative Adversarial Networks (GAN) to generate high-quality images typically requires large datasets. Network pruning during training has recently emerged as a significant advancement for data-efficient GAN. However, simple and straightforward pruning can lead to the risk of losing key information, resulting in suboptimal results due to GAN’s competitive dynamics between generator (G) and discriminator (D). Addressing this, we present RG-GAN, a novel approach that marks the first incorporation of dynamic weight regeneration and pruning in GAN training to improve the quality of the generated samples, even with limited data. Specifically, RG-GAN initiates layer-wise dynamic pruning by removing less important weights to the quality of the generated images. While pruning enhances efficiency, excessive sparsity within layers can pose a risk of model collapse. To mitigate this issue, RG-GAN applies a dynamic regeneration method to reintroduce specific weights when they become important, ensuring a balance between sparsity and image quality. Though effective, the sparse network achieved through this process might eliminate some weights important to the combined G and D performance, a crucial aspect for achieving stable and effective GAN training. RG-GAN addresses this loss of weights by integrating learned sparse network weights back into the dense network at the previous stage during a follow-up regeneration step. Our results consistently demonstrate RG-GAN’s robust performance across a variety of scenarios, including different GAN architectures, datasets, and degrees of data scarcity, reinforcing its value as a generic training methodology. Results also show that data augmentation exhibits improved performance in conjunction with RG-GAN. Furthermore, RG-GAN can achieve fewer parameters without compromising, and even enhancing, the quality of the generated samples. Code can be found at this link: https://github.com/IntellicentAI-Lab/RG-GAN

ABSTRACTData mining and natural language processing (NLP) are fundamental fields that interact in many ways. Text mining shares many topics, such as sentiment analysis and content understanding. Combining these two fields enables more efficient mining of text data and the extraction of valuable information. In particular, the GAN (Generative Adversarial Network) architecture has achieved success in image generation and has started to be used on text data. However, training GANs is fraught with difficulties due to the complexity of text data. Linguistic studies show important differences between languages. Language is characterized by fluidity, ambiguity, and context‐sensitive interpretations, and text‐generating GAN models can struggle to deal with these complexities. The interaction between data quality, language structure, and complex interpretation can lead to inconsistency and ambiguity in the text production of GAN models. These problems are particularly pronounced when complexities such as semantic subtleties, idiomatic expressions, and context‐dependent usages come into play. Text generation is an area of GAN models used in NLP to generate language and enrich text‐based applications. Work in this area can contribute to analyzing, classifying, and processing text data. Many methods and techniques have been proposed to improve the performance of text GANs. However, some problems may be encountered in the optimization of these methods. Therefore, it is essential to use optimized methods. In conclusion, GANs can be an important tool to improve text generation in NLP. Still, they require continuous research and innovation to deal with factors such as language complexity and data quality.

Deep learning algorithms are incredibly powerful and have achieved impressive results in various classification tasks. However, one of their limitations is their dependence on large amounts of training data. When there is limited training data available, the standard approach is to increase the dataset size by using data augmentation and training a neural network on the expanded dataset. This method can be effective, but it requires significant computational resources and may not always be feasible. In our work, we proposed a new approach to address the problem of limited training data. Instead of relying solely on increasing the dataset size, we train Generative Adversarial Networks (GANs) to generate distributions of individual categories. The classification of the unknown element is then performed using distributions generated by trained GAN networks. We proposed four methods that compare the unknown element with the elements generated by trained GAN networks and establish estimates of the conditional probabilities of the unknown element belonging to individual categories. These conditional probabilities are then used for the classification of the unknown element into individual categories. This approach enables us to make informed decisions and achieve accurate classification results even when dealing with limited training data.

In practical application, the research of synthetic aperture radar (SAR) target recognition has fallen into a bottle-neck due to the lack of samples. Data argumentation methods based on generative adversarial networks (GAN) have received widespread attention in solving this type of few-shot sample problem. However, the generated images suffer from various shortcomings, such as lack of diversity, low signal-to-noise ratio, blur, etc. In this article, the VAE-WGANGP is proposed, which combines GAN and variational autoencoder (VAE) to alleviate these shortcomings. The innovations of this paper are as follows: firstly, the generator of GAN is replaced with VAE, which constructs an asymmetric network ensuring the stability of GAN training; secondly, the asymmetric loss function is composed of four parts, including reconstruction loss, divergence loss, adversarial loss, and gradient penalty. In this way, the problem of gradient explosion or gradient disappearance is alleviated. The experimental results with the MSTAR dataset show that the images generated by the proposed model outperform the advanced technology with many similar deep features and achieve significant improvement in the target recognition accuracy rate.

Generative Adversarial Networks (GANs) have proven to be a preferred method of synthesizing fake images of ob-jects, such as faces, animals, and automobiles. It is not surprising these models can also generate ISO-compliant, yet synthetic iris images, which can be used to augment training data for iris matchers and liveness detectors. In this work, we trained one of the most recent GAN mod-els (StyleGAN3 [15]) to generate fake iris images with two primary goals: (i) to understand the GAN's ability to produce "never-before-seen" irises, and (ii) to investigate the phenomenon of identity leakage as a function of the GAN's training time. Previous work has shown that personal biometric data can inadvertently flow from training data into synthetic samples, raising a privacy concern for subjects who accidentally appear in the training dataset. This paper presents analysis for three different iris matchers at varying points in the GAN training process to diagnose where and when authentic training samples are in jeopardy of leaking through the generative process. Our results show that while most synthetic samples do not show signs of identity leak-age, a handful of generated samples match authentic (training) samples nearly perfectly, with consensus across all matchers. In order to prioritize privacy, security, and trust in the machine learning model development process, the re-search community must strike a delicate balance between the benefits of using synthetic data and the corresponding threats against privacy from potential identity leakage.

Generative Adversarial Networks (GANs) were first used to generate images that were similar to images in the data the model was trained on. The GANs training is based on a zero-sum game where the constituent models are adversaries. The mathematical interpretation of GAN training is the mapping of an unknown distribution to the dataset distribution. Future works in the field led to the generation of music, texts, and types of data and GANs still are being explored in scientific, entertainment, fashion, advertising, videogames, and other miscellaneous applications. This review focuses on the versatility of GANs. First, the GAN model is explored with its mathematical intuition. Then come the popular variants of GANs and their applications. Finally, the most recent applications of GANs in different fields are discussed, and the review ends with a discussion of future possible applications of GANs.