Survey on Generative Adversarial Behavior in Artificial Neural Tasks

In recent years, deep neural networks have demonstrated high efficiency in solving the problems associated with image processing and analysis. The latest studies in generative adversarial neural networks open up broad prospects for solving problems in image style transfer, cross-domain adaptation, and the generation of new data for a target probability distribution. The ability of generative networks to implement complex and weakly formalized transformations allows the information systems based on generative adversarial technologies to solve problems in machine learning in a manner similar to human creative thinking. This paper presents the results of an experimental study on the design, training and deployment of an intelligent control system for a CNC laser engraver based on the generative adversarial network. In order to train deep models, large amounts of data are required. Therefore, one of the main challenges in this experimental study is a lack of labelled training dataset. The proposed control system for the laser engraver based on the deep generative model allows decreasing the time of the manufacturing process.

This work presents Unified Contrastive Arbitrary Style Transfer (UCAST), a novel style representation learning and transfer framework, that can fit in most existing arbitrary image style transfer models, such as CNN-based, ViT-based, and flow-based methods. As the key component in image style transfer tasks, a suitable style representation is essential to achieve satisfactory results. Existing approaches based on deep neural networks typically use second-order statistics to generate the output. However, these hand-crafted features computed from a single image cannot leverage style information sufficiently, which leads to artifacts such as local distortions and style inconsistency. To address these issues, we learn style representation directly from a large number of images based on contrastive learning by considering the relationships between specific styles and the holistic style distribution. Specifically, we present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components: a parallel contrastive learning scheme for style representation and transfer, a domain enhancement (DE) module for effective learning of style distribution, and a generative network for style transfer. Qualitative and quantitative evaluations show the results of our approach are superior to those obtained via state-of-the-art methods. The code is available at https://github.com/zyxElsa/CAST_pytorch .

In this work, we focus on building style transfer, which transforms ruin or old buildings to modern architecture. Inspired by Gaty's and Goodfellow's style transfer and generative adversarial network (GAN), we use CycleGAN to conquer this type of problem. As we know, image style transfer usually generated unexpected artifacts. To avoid the artifacts and generate better images, we add so called “perception loss” into the network, which is the feature loss extracted by VGG pre-trained model. In the part of “cycle” structure, we adjust cycle loss by changing the ratio of weighting parameters. Finally, we collect images of both ruin (or old) and modern architecture from websites and use unsupervised learning to train the model. The experimental results show our proposed method indeed realize the modern architecture style transfer for ruin or old buildings.

To provide semantic image style transfer results which are consistent with human perception, transferring styles of semantic regions of the style image to their corresponding semantic regions of the content image is necessary. However, when the object categories between the content and style images are not the same, it is difficult to match semantic regions between two images for semantic image style transfer. To solve the semantic matching problem and guide the semantic image style transfer based on matched regions, we propose a novel semantic context-aware image style transfer method by performing semantic context matching followed by a hierarchical local-to-global network architecture. The semantic context matching aims to obtain the corresponding regions between the content and style images by using context correlations of different object categories. Based on the matching results, we retrieve semantic context pairs where each pair is composed of two semantically matched regions from the content and style images. To achieve semantic context-aware style transfer, a hierarchical local-to-global network architecture, which contains two sub-networks including the local context network and the global context network, is proposed. The former focuses on style transfer for each semantic context pair from the style image to the content image, and generates a local style transfer image storing the detailed style feature representations for corresponding semantic regions. The latter aims to derive the stylized image by considering the content, the style, and the intermediate local style transfer images, so that inconsistency between different corresponding semantic regions can be addressed and solved. The experimental results show that the stylized results using our method are more consistent with human perception compared with the state-of-the-art methods.

Synthesizing high-resolution realistic images from text description using one iteration Generative Adversarial Network (GAN) is difficult without using any additional techniques because mostly the blurry artifacts and mode collapse problems are occurring. To reduce these problems, this paper proposes an Iterative Generative Adversarial Network (iGAN) which takes three iterations to synthesize high-resolution realistic image from their text description. In the \(1^{st}\) iteration, GAN synthesizes a low-resolution \(64 \times 64\) pixels basic shape and basic color image from the text description with less mode collapse and blurry artifacts problems. In the \(2^{nd}\) iteration, GAN takes the result of the \(1^{st}\) iteration and text description again and synthesizes a better resolution \(128 \times 128\) pixels better shape and well color image with very less mode collapse and blurry artifacts problems. In the last iteration, GAN takes the result of the \(2^{nd}\) iteration and text description as well and synthesizes a high-resolution \(256 \times 256\) well shape and clear image with almost no mode collapse and blurry artifacts problems. Our proposed iGAN shows a significant performance on CUB birds and Oxford-102 flowers datasets. Moreover, iGAN improves the inception score and human rank as compare to the other state-of-the-art methods.

Image style transfer refers to the transformation of the style of image, so that the image details are retained to the maximum extent while the style is transferred. Aiming at the problem of low clarity of style transfer images generated by CycleGAN network, this paper improves the CycleGAN network. In this paper, the network model of auto-encoder and variational auto-encoder is added to the structure. The encoding part of the auto-encoder is used to extract image content features, and the variational auto-encoder is used to extract style features. At the same time, the generating network of the model in this paper uses first to adjust the image size and then perform the convolution operation to replace the traditional deconvolution operation. The discriminating network uses a multi-scale discriminator to force the samples generated by the generating network to be more realistic and approximate the target image, so as to improve the effect of image style transfer.

Style transfer is a useful image synthesis technique that can re-render given image into another artistic style while preserving its content information. Generative Adversarial Network (GAN) is a widely adopted framework toward this task for its better representation ability on local style patterns than the traditional Gram-matrix based methods. However, most previous methods rely on sufficient amount of pre-collected style images to train the model. In this paper, a novel Patch Permutation GAN (P <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> -GAN) network that can efficiently learn the stroke style from a single style image is proposed. We use patch permutation to generate multiple training samples from the given style image. A patch discriminator that can simultaneously process patch-wise images and natural images seamlessly is designed. We also propose a local texture descriptor based criterion to quantitatively evaluate the style transfer quality. Experimental results showed that our method can produce finer quality re-renderings from single style image with improved computational efficiency compared with many state-of-the-arts methods.

A Generative Adversarial Network (GAN) is an artifi- cial intelligence model developed specifically to pro- duce synthetic data that resembles real data by training a generative model and a discriminative model simulta- neously using adversarial training. A GAN can be ex- tensively used for generating replicated data, however, it suffers from several issues, one of which is mode col- lapse. Mode collapse takes place when the generator is unable to capture the complete range of diversity in the target data distribution, resulting in the production of limited and repeating variations of samples. Multiple metrics exist to quantify mode collapse in GANs, al- though no individual metric is capable of consistently providing accurate results. This research focuses on the critical need for accurate mode collapse detection tech- niques in GANs, to strengthen the credit card fraud detection systems. In this work, we utilize a GAN to generate numerical data instead of image data. Our ap- proach utilizes a wide range of measures, such as Gener- ator and Discriminator Loss, Wasserstein Distance, pre- cision, recall, and visualization tools, to provide a com- prehensive framework for detecting mode collapse. In addition, we introduce an alert mechanism that identi- fies possible mode collapse at an early stage, allowing for earlier intervention and modifications to the training process. We have further proposed suggestions regard- ing monitoring and analyzing generator and discrimina- tor loss values to identify potential instances of mode collapse to help the developer optimize GAN training and improve the quality of synthetic data.

In the field of deep learning, the generative model did not attract much attention until GANs (generative adversarial networks) appeared. In 2014, Google’s Ian Goodfellow proposed a generative model called GANs. GANs use different structures and objective functions from the existing generative model. For example, GANs use two neural networks: a generator that creates a realistic image, and a discriminator that distinguishes whether the input is real or synthetic. If there are no problems in the training process, GANs can generate images that are difficult even for experts to distinguish in terms of authenticity. Currently, GANs are the most researched subject in the field of computer vision, which deals with the technology of image style translation, synthesis, and generation, and various models have been unveiled. The issues raised are also improving one by one. In image synthesis, BEGAN (Boundary Equilibrium Generative Adversarial Network), which outperforms the previously announced GANs, learns the latent space of the image, while balancing the generator and discriminator. Nonetheless, BEGAN also has a mode collapse wherein the generator generates only a few images or a single one. Although BEGAN-CS (Boundary Equilibrium Generative Adversarial Network with Constrained Space), which was improved in terms of loss function, was introduced, it did not solve the mode collapse. The discriminator structure of BEGAN-CS is AE (AutoEncoder), which cannot create a particularly useful or structured latent space. Compression performance is not good either. In this paper, this characteristic of AE is considered to be related to the occurrence of mode collapse. Thus, we used VAE (Variational AutoEncoder), which added statistical techniques to AE. As a result of the experiment, the proposed model did not cause mode collapse but converged to a better state than BEGAN-CS.

Image style transfer has become a key technique in modern photo-editing applications. Although significant progress has been made to blend content from one image with style from another image, the synthesized image may have a hallucinatory effect when the texture from the style image is rich when processing high-resolution image style transfer tasks. In this paper, we propose a novel attention mechanism, named compositional attention, to design a compositional transformer-based autoencoder (CTA) to solve this above-mentioned issue. With the support from this module, our model is capable of generating high-quality images when transferring from texture-riched style images to content images with semantics. Additionally, we embed region-based consistency terms in our loss function for ensuring internal structure semantic preservation in our synthesized image. Moreover, information theory-based CTA is discussed and Kullback–Leibler divergence loss is introduced to preserve more brightness information for photo-realistic style transfer. Extensive experimental results based on three benchmark datasets, namely Churches, Flickr Landscapes, and Flickr Faces HQ, confirmed excellent performance when compared to several state-of-the-art methods. Based on a user study assessment, the majority number of users, ranging from 61% to 66%, gave high scores on the transfer effects of our method compared to 9% users who supported the second best method. Further, for the questions of realism and style transfer quality, we achieved the best score, i.e., an average of 4.5 out of 5 compared to other style transfer methods.

Style transfer means that the characteristic information of the style image is transferred to the content image under a given content and style picture. Meanwhile, the transferred image is faithful to the content image. Currently, the transferred image has many problems such as artifacts and distortion of the spatial structure. To solve these problems, a dynamic residual-multi-level and multi-pooling network combined with our improved style transfer algorithm is proposed in this paper to achieve better effect of arbitrary image style transfer. To elaborate more specifically, First of all, We add a dynamic-residual network layer to the network to speed up the training speed of the network model and improve the robustness of the network. Secondly, we use a multi-level and multi-pooling layer to extract more specific spatial structure, edge texture and other information in the image, and the network layer also has an additional denoising effect. Third, the improved style transfer algorithm aligns the mean and variance of the content and style images from a statistical point of view, and realizes adaptive arbitrary style transfer of the original input image features. Finally, during model training, the convergence speed of the proposed approach is faster than other current advanced methods, and the transferred renderings are better than other network models in quantitative comparisons such as SSIM and Gram. It is worth mentioning that our model has the fastest real-time transfer speed and can realize any image style transfer.

In many fields, image style transfer has been a popular topic. It also has a long history; the earliest examples date back to the previous century. A number of style transfer techniques are currently flourishing, from manual modeling to the use of neural networks. Image style transfer methods is improving and taking less time. This paper is separated into two main categories to analyze based on picture iteration and model iteration - and summarizes the various techniques of image style transfer based on deep learning in accordance with the timeline. This paper introduces maximum mean difference, Markov random field, and deep image analogy in picture iteration-based image style transfer. In terms of model iteration-based image style migration, this paper introduces generative models and image reconstruction decoders. Finally, the paper presents some recommendations and outlook on the future.

&lt;p&gt;High data collection costs and complicated data access regulations increase the demand for synthetic data. Generative Adversarial Networks (GANs) are a novel generative framework with great potential for high quality synthetic data generation. GANs formulate the true distribution of data implicitly, and the success of GANs are often measured based on the similarity of generated data to this true distribution. GANs were originally designed to work with continuous data. However, many important real-world datasets such as medical images involve discontinuous distributions. GAN training for discontinuous distributions is relatively more challenging, as the training procedure often suffers from instability and mode collapse issues. This dissertation focuses on designing novel GAN architectures to generate representative synthetic image data, and proposes new structures to alleviate GANs' mode collapse issue. As part of this thesis, novel applications of image data generation with GANs have been also investigated for important problems arising in the telecommunication industry and medical domain. Specifically, we first explore various GAN structures to generate engineered electromagnetic surfaces. We consider the continuous approximation of the data and explore the capabilities of feed-forward and convolutional GANs for synthetic data generation. Next, we introduce a novel GAN architecture to address the problem of mode collapse in GAN training. The proposed structure incorporates a third network that penalizes the generator for generating low diversity samples. Lastly, we study the challenging problem of object generation in 3D space using GANs, and we propose extensions to existing 3D GAN structures to generate connected 3D volumes. Additionally, we explore a more challenging version of this 3D volume generation problem by generating connected volumes packed with spheres. This research has applications in radiosurgery treatment planning, and the proposed 3D GAN structure can help generate rare, unseen 3D tumor volumes and information on how to treat these tumors. Accordingly, our analysis contributes to overcoming data scarcity issues (e.g., due to privacy considerations) for an important practical problem in the medical domain.&lt;/p&gt;

The rapid development of artificial intelligence technology and deep learning technology has also promoted the popularization and application of image style transfer in all walks of life. In the traditional ceramic industry, the application of image style transfer technology is still rare. This article is based on the style transfer training of ceramic blue and white images and uses three methods of rapid style transfer, direct style transfer and image pre-processing style transfer to conduct experiments. According to the results of image quality evaluation, it is concluded that the method suitable for the style transfer of ceramic blue and white images is the image pre-processing style transfer method.

With the development of the convolutional neural network, image style transfer has drawn increasing attention. However, most existing approaches adopt a global feature transformation to transfer style patterns into content images (e.g., AdaIN and WCT). Such a design usually destroys the spatial information of the input images and fails to transfer fine-grained style patterns into style transfer results. To solve this problem, we propose a novel STyle TRansformer (STTR) network which breaks both content and style images into visual tokens to achieve a fine-grained style transformation. Specifically, two attention mechanisms are adopted in our STTR. We first propose to use self-attention to encode content and style tokens such that similar tokens can be grouped and learned together. We then adopt cross-attention between content and style tokens that encourages fine-grained style transformations. To compare STTR with existing approaches, we conduct user studies on Amazon Mechanical Turk (AMT), which are carried out with 50 human subjects with 1,000 votes in total. Extensive evaluations demonstrate the effectiveness and efficiency of the proposed STTR in generating visually pleasing style transfer results (Code is available at https://github.com/researchmm/STTR ).